Training 17: Reading the verification report and deviations

Before relying on a calculation type, find out what it was checked against. Everything below is in the app, under Settings > Verification, and on the public site.

Settings, Verification: every example and value with the app's result

The numbers, read carefully #

This release: 9,654 values compared in 2,841 examples, 2,143 of them published and 698 hand calculations from the cited equations: 9,254 within tolerance, 400 recorded deviations.

Those are counts of tests, not a grade for the program. Weigh them this way:

A published example is someone else's worked calculation. Agreement is real evidence, for the case that example covers. Most are single cases in one set of conditions, many written to an older edition.
A hand calculation is ours, from the cited equations. It catches arithmetic and programming errors. It does not catch a misreading of the code, because the same person read it twice.
A table cell read back (a load table, a shape table) checks that one value.
The percentage on the page mostly reflects how many easy comparisons exist for a type. Read the examples, not the percentage.

The statement in each chapter #

Every calculation chapter opens its Verification examples section with a status line. For the steel beam:

Verification status: 168 values compared in 37 examples (37 published, 0 hand calculations from the cited equations): 164 within tolerance, 4 recorded deviations. Code basis: partial. Only some provisions were read in the target edition; the rest rests on older editions, examples or summaries.

Code basis says how the module was written: read (the provisions it uses were read in the target edition), partial, older edition, from knowledge (no document on hand) or not determined (no record). ILX Studio plans to verify every module against the purchased references once the module list is complete.

Types with little behind them #

No published example, hand calculations only (51): AAC masonry wall, Atmospheric ice loads, Beam grid (grillage), Bond beam chord and collector, Built-up cold-formed steel member, Cold-formed steel bypass and infill framing, Cold-formed steel ceiling framing, Cold-formed steel clip angle, Cold-formed steel floor vibration, Cold-formed steel girt, Cold-formed steel jamb studs, Cold-formed steel parapet, Cold-formed steel shear wall chord studs, Cold-formed steel sloped and mansard wall, Cold-formed steel soffit framing, Cold-formed steel special bolted moment frame, Cold-formed steel storage rack, Cold-formed steel track, Cold-formed steel truss bracing, Concrete stair, Corbelled masonry, Curved and tapered glulam, Dapped-end beam, Fall-protection anchorage, Fluid and hydrostatic pressure, Freestanding masonry wall, Gabion retaining wall, Gravity retaining wall, Helical pile, Joist and waffle slab, Laterally loaded pile (p-y), Ledge and inverted-tee beam, Liquefaction triggering, Liquid-containing tank wall, Masonry arch, Masonry bearing, Masonry chimney, Masonry parapet, Micropile, Modal response spectrum, Movement joints, Notched and tapered member ends, Pile capacity from CPT, Prestressed concrete beam, Roof geometry, Roof ponding, Segmental retaining wall, Seismic member checks, Special moment frame members, Wood shrinkage and moisture, Wood stair.

Thin, 2 published examples or fewer (36): Basement (restrained) retaining wall, Beam with varying stiffness, Braced excavation, Built-up and spaced columns, Cable and catenary, Cold-formed steel track anchorage, Combined footing, Concrete deflection, Continuous concrete beam, Corbel, Coupling beam, Glulam and SCL beam, Ground improvement, Guard or barrier post, Gust-effect factor, Impact and machinery loads, Influence lines, Ledger to masonry wall, Load chasedown, Masonry beam and deep beam, Masonry partition wall, Mat foundation, Monorail beam, One-way slab, Plate girder, Post-tensioned slab strip, Prestressed masonry wall, Seismic site parameters and spectrum, Sheathing-braced cold-formed steel stud, Slope stability, Soldier pile wall, Steel side and knife plate connections, Strap footing, Stress transformation, Unreinforced masonry wall, Wood floor vibration.

Being on this list does not make a type wrong. It means less stands behind it, so check its results more closely.

Reading a deviation #

A deviation is a published value the program does not reproduce within tolerance, kept on the record with the reason. They are not all alike. On the Verification page, select the status Documented deviation and read each one. Ask of each:

Who differs, and which way? Real examples from this release's record:

Steel beam, F.8B. The example takes a simpler, conservative section; the program follows the specification's wording and reads a little higher: 46.3 against 45.4 kip-ft. A difference of method, stated.
Steel column, E.14A. The example rounds Lc/r up to 127 before finding the stress. Unrounded it is 126.6, so the program reads 52.8 against 52.3 kips. Harmless.
Strap footing. The one free example is a software output with unstated soil weights. None of its four pressures matches within tolerance: differences of up to 6 % that nobody can trace. That is a weak check, and the chapter says so.
Sheet pile wall, Fig. 4-43. The program's cantilever moment is 8 % below the printed value. It is on the unsafe side. It comes from equilibrium of the stated pressures, and no line-by-line example is on hand. If this case governs you, check it yourself: see the chapter.

The last one is the kind to look for: a program value on the unsafe side of a published one.

What verification does not cover #

Inputs outside the examples, combinations of options no example uses, and whatever a type's Limitations leave out. Passing every example does not make a calculation right for your structure. See Limits and your responsibility.

The whole method, with the tolerances, is in Verification.