# Lab metrics: what each figure in a lab result means Every result from the model lab's insurance lanes carries a `metrics` block (schema `lab-metrics/1`) beside the fields older clients read. This page says what each figure is, in the terms an underwriter or a catastrophe modeler uses, how the lab computes it, and what it cannot tell you. The lab is at `https://lab.oncraton.com`; how to submit a run is in [the agent walk](agents.md). Every lab model is an open model, run as its authors published it on fixed inputs. Craton has not validated it. The figures come from these model versions, and a document from an older version has no `metrics` block: | Lane | Model | Version | `metrics.basis` | |---|---|---|---| | Historical tropical cyclone | `climada-tropical-cyclone` | `climada-6.1.0-r3` | `historical_replay` | | Synthetic tropical cyclone | `climada-tropical-cyclone-storm` | `climada-6.1.0-storm-v4-r2` | `event_set` | | Demonstration windstorm (PiWind) | `oasis-piwind` | `oasislmf-2.5.7-r3` | `event_set` | | Earthquake scenario | `openquake-scenario` | `openquake-3.26.2-r2` | `scenario` | | Historical earthquake | `openquake-shakemap` | `openquake-3.26.2-shakemap-r1` | `scenario` | | River flood | `river-flood` | `jrc-glofas-2.1.2-r1` | `hazard_map` | | Coastal flood | `coastal-flood` | `aqueduct-floods-2-r1` | `hazard_map` | | Wildfire | `wildfire` | `usfs-fsim-2023-r1` | `hazard_map` | | Europe and Turkey earthquake | `openquake-esrm` | `openquake-3.26.2-esrm20-r1` | `event_set` | | A provider's model on the exchange | `exchange:` | the listing's own version | `event_set` | History documents from `climada-6.1.0-r2` also carry the block; they spread one reported storm size along each track and used a West Pacific wind-pressure fit everywhere (both described under the history lane below). Every figure is **ground-up**: the loss to the buildings before any deductible, limit or reinsurance. Values are the replacement values you send with each place. The one exception is PiWind with policy terms, which adds insured and net-of-reinsurance blocks beside the ground-up one ([below](#insured-and-net-perspectives)). ## Four kinds of answer **An event set** (`event_set`, the STORM lane) is the probabilistic view: 10,000 sample years of synthetic tropical cyclones per basin (STORM v4, Bloemendaal et al. 2020), statistically resampled from the storms observed from 1980 to 2017. Each sample year is one plausible year of present climate. Averages and return periods come from counting across those years. It is not a forecast of any particular year. The European earthquake lane is an event set too: 10,000 sample years of earthquakes drawn from the ESHM20 source model over 20 sampled branches of its logic tree. Its figures read the same way; what differs is on [its own page](lab-esrm.md): the 1 in 475 it adds, the blocks that each hold whole branches, and the warnings about how much of its average rests on rare years. **A historical replay** (`historical_replay`, the history lane) runs the storms that actually happened, from the IBTrACS record, over the seasons you choose (at most 30). It answers "what would these seasons have cost this schedule". Thirty seasons is too short a record for return periods, so the lane publishes none; see the standard error below for how loosely it pins the average. **A scenario** (`scenario`, the quake lane) is one caller-defined earthquake. It answers "what would this event do", averaged over 100 simulated shaking patterns. A single event has no frequency, so there is no annual figure and no return period. **A hazard map** (`hazard_map`, the river flood, coastal flood and wildfire lanes) reads published return-period maps at each place instead of simulating years; [Hazard-map lanes](#hazard-map-lanes) below has how. ## Average annual loss `aal` is the expected loss per year: every modeled loss over the whole sample, divided by the number of years (STORM) or seasons (history). It is the pure loss cost, before expenses, capital or margin. Zero years count: most years bring no loss to most places, and they are part of the average. - `aal_share` is the AAL as a fraction of the value the model covers: every place on the tropical-cyclone lanes, the places inside the grid on PiWind (`modeled_value`). - `aal_per_mille` is the AAL per 1,000 of value, the usual way to state a loss cost. For one point in Miami valued at $10,000,000, the STORM lane's AAL under the Emanuel curve is $24,405: a share of 0.24% and 2.44 per mille. ## How much the average can be trusted - `sd` is the standard deviation of the annual loss over every year, zero years included. It measures how much one year differs from another. - `cv` is `sd / aal`, the coefficient of variation. In the lab's own 10,000-year results it runs from about 3 (Manila) to about 7 (Miami): the average is small next to the years that matter. - `aal_standard_error` is `sd / sqrt(years)`: how far the average itself could sit from the true long-run average because the sample is finite. This is the number that separates the lanes. For Miami, Houston and New Orleans at $3.33 million each, 30 seasons of history give an AAL of $31,115 with a standard error of $20,583, about 66% of the average (run `lab-climada-tc:99b716bdf06230bf`). The same places over 10,000 STORM years give an AAL of $13,672 with a standard error of $682, about 5% (stored run `lab-climada-storm:df5dab0ff0f6c19a`). That is why Craton's own pages show the STORM figure: one season, 2005, makes up two thirds of the history lane's total. ## Exceedance probability: OEP and AEP A return period of 1 in N years is a loss exceeded, on average, once every N years. The lab gives two, from the same sample years, in `ep`: - **OEP**, the occurrence exceedance probability, looks at the **largest single event in each year**. The 1-in-100 OEP is the loss that the worst event of a year exceeds once in 100 years on average. It is the view for per-occurrence protection. - **AEP**, the aggregate exceedance probability, looks at the **sum of all events in each year**. The 1-in-100 AEP is the annual total exceeded once in 100 years. It is the view for annual aggregate protection and for budgeting. AEP is never below OEP at the same return period. The lab reads both straight off the sample: sort the per-year values from largest to smallest, and the 1-in-N value is the one at rank years / N. For 10,000 years, the 1 in 100 is the 100th largest year. **The ten-times rule.** A return period is published only when the run holds at least ten times as many sample years: 1 in 100 needs 1,000 years, 1 in 1,000 needs all 10,000. A 1-in-1,000 figure read from 1,000 years is simply the single largest storm in the sample, which is noise, not a tail. Withheld return periods are listed in `ep_withheld` and named in a warning. The history lane publishes none. For the three places above, STORM over 10,000 years gives a 1-in-100 OEP of $356,824 and AEP of $369,776, and a 1-in-250 OEP of $566,518 and AEP of $580,271. ## Tail value at risk `tvar` is the average of the worst years beyond a return period, on the AEP basis: the 1-in-100 TVaR is the mean of the worst 100 years in 10,000. It answers "when it is bad, how bad on average", and it is always at least the AEP at the same return period. For the three places: $568,640 at 1 in 100 and $721,076 at 1 in 250. ## How much the sample itself moves the answer `sampling` splits the 10,000 years into ten blocks of 1,000 and reports the lowest and highest block value of the AAL, the 1-in-100 and 1-in-250 OEP, and the 1-in-100 and 1-in-250 AEP. A block of 1,000 years is what the lab used before 23 September 2026. For the three places, the block AAL runs from $10,665 to $18,396 and the block 1-in-100 OEP from $273,239 to $539,893: any one 1,000-year slice can land anywhere in that range. The 1-in-250 block figures are shown as spread only; each block holds fewer than ten times 250 years. ## Places, events and the loss tables - `per_point` gives each place's own AAL, its share of the total (`aal_share_of_total`), the damage curve applied there, its country and its calibration region. - `events` lists up to 50 of the largest events, largest first, with the loss at each place and the peak wind there (for schedules of up to 200 places). History events carry the storm's IBTrACS name, season, the date of its closest approach to the place that lost most, and which pressure source drove its winds. STORM events carry an id such as `NA-4-15-2` (basin, file, year in the file, storm number) and a sample year from 0 to 9,999. - `event_loss_table` is every event with a loss, as `[id, year, loss]`. - `year_loss_table` is every year (or season) with a loss, as `[year, loss]`. Years not listed had no loss. These two tables are the usual input to a reinsurance or capital model. The history lane also groups losses by season in `seasons`, largest first, with the storms in each, and counts the seasons with no loss in `zero_seasons`. For the three places over 1995 to 2024, 2005 alone (Katrina and Wilma) is $621,935 of the $933,450 total, and 21 of the 30 seasons had no loss. ## Damage curves A damage curve turns wind speed at a place into a mean damage ratio: the fraction of the building's value lost. Every curve here has the same shape (Emanuel 2011): no damage below 25.7 m/s, half the value lost at a wind speed called `v_half`, rising steeply between. What differs is `v_half`, and that is the largest single uncertainty in these results outside the US. - `emanuel_2011`: `v_half` 74.7 m/s, calibrated to US losses. - `eberenz_2021_tdr`, `eberenz_2021_rmsf`, `eberenz_2021_edr`: Eberenz et al. (2021) recalibrated `v_half` for ten world regions against reported disaster damages, three ways (total damage ratio, root mean squared fraction, median event damage ratio). The lab uses the calibration files that ship with the pinned CLIMADA. - `auto`, the default: Emanuel for places in the United States, the Eberenz RMSF curve for its region everywhere else. The US curve is never applied outside the US unless you ask for it by name, and then a warning says so. A place's country and region come from the Natural Earth country grid that ships with CLIMADA; a coastal place that falls in a sea cell takes the nearest land within 50 km. Puerto Rico and other territories take their own region's curve, not the US one. `damage_curve_sensitivity` reruns the same hazard through each of the five choices, so you can see how much the curve alone moves the answer. For Manila, $10,000,000, over 10,000 STORM years the AAL is $116,933 under the Philippines RMSF curve, $189,735 under Emanuel and $6,263 under the Philippines total damage ratio curve: a thirty-fold spread from the curve alone. ## The earthquake scenario - `loss_mean` is the mean loss over the 100 shaking patterns, and `loss_ratio.mean` is that loss over the value of the places within reach (`modeled_value`). `loss_ratio.p10`, `p50` and `p90` show the spread across the 100 patterns: the same earthquake can do very different damage. - `pga_g` is peak ground acceleration in g, across patterns and places: the `geometric_mean` (the engine's own average, and what the older `mean_pga_g` field always was), the `arithmetic_mean`, which is higher, and percentiles. - `vulnerability` names the curve that turned shaking into loss, with its source. The default is the HAZUS 5.1 curve for a mid-rise concrete frame office (C1M) designed to moderate code; high-code and pre-code versions are available, and so are two curves from the engine's own Nepal demonstration, labeled as such. The HAZUS curves are built from the manual's equivalent peak ground acceleration fragility tables and its office repair cost ratios; the script that builds them is in the repository beside the runner. - `vulnerability_sensitivity` runs the same 100 shaking patterns through every curve. For a magnitude 7.0 rupture under central Tokyo and one $10,000,000 building there, the mean loss ratio is 29.6% for moderate-code concrete, 16.3% for high code and 58.8% for pre-code. The Nepal demonstration concrete curve gives 59.3%, which is why it is no longer the default. ## A historical earthquake The `openquake-shakemap` lane runs a real earthquake instead of a rupture you describe: the USGS ShakeMap of one of ten historical earthquakes, the shaking built from that earthquake's own recordings, with 100 patterns spread around it by the ShakeMap's uncertainty. Its `metrics` block has the scenario shape above, plus `event` (which earthquake), and each place adds the spread of its own loss ratio (`loss_ratio_p10`, `loss_ratio_p50`, `loss_ratio_p90`) and the ShakeMap's own shaking there (`pga_shakemap_g`, `mmi_shakemap`, `shaking`). For Northridge over Sherman Oaks and Santa Monica at $5,000,000 each, the mean loss ratio is 34.9% under the default curve. The list of earthquakes, and how a place outside a ShakeMap is treated, are in [historical earthquakes](lab-historical-earthquakes.md). ## Assumptions and warnings `assumptions` lists, in plain sentences, what every figure leaves out: ground-up only, wind only (no storm surge, no rainfall flooding), no demand surge, no secondary uncertainty around the damage curve, one curve per place, no terrain roughness or gusts, buildings only (no contents, no business interruption); on the history lane, how each fix's radius of maximum wind was set; for PiWind, wind and storm surge on a demonstration grid, one building type, and sampled damage; for earthquakes, shaking only (no fire, liquefaction, landslide or tsunami), a synthesized rupture and one ground condition everywhere. `warnings` lists what an insurance reader must see next to the figure, and each one carries an `id`: - `short_record`: the history lane's standard error, as a share of the AAL. - `return_periods_withheld`: return periods held back by the ten-times rule. - `single_event_dominates`: one event makes up at least half of a place's AAL. In the three-place history, Wilma is 94% of Miami's average. - `curve_outside_calibration` and `curve_rest_of_world`: a curve applied outside the region it was calibrated for. - `pressure_estimated`: history storms whose central pressure had to be estimated from wind at some fixes (below). - `radius_estimated`: history storms whose radius of maximum wind had to be estimated at most of their fixes (below). - `demonstration_model`: PiWind is a demonstration model. - `named_storms_only`, `zero_loss`, `demo_curve`, `unmodeled_points`. - `event_note`: a historical earthquake's own caveat, such as a mainshock shown without its sequence, or a tsunami that is not modeled. - `sampling_noise` and `rare_years_dominate` (the European earthquake lane): how far the average and the 1-in-250 move with the sample, and when most of the average comes from years rarer than 1 in 250. ## Hazard-map lanes The river flood, coastal flood and wildfire lanes (`basis` `hazard_map`) read published maps at each place instead of counting across simulated years, so a few fields differ; [Hazard-map lanes](lab-hazard-maps.md) has the method and worked numbers. - Flood `aal` integrates each place's loss over annual exceedance probability across the published return periods; `aal_method` says how the tails are treated and how much of the AAL the rarest tail is. - Flood `ep` rows are `{return_period, loss}`, with `ep_basis` `sum_of_single_site_losses`: each place's own 1-in-N loss, added up. They are not OEP or AEP, and there is no `sd`, `tvar`, `sampling` or event table. - Flood `per_point` rows add `depths_m` and `losses` (one value per return period in `return_periods`), `continent`, `damage_curve` and, when a place is not modeled, its `reason`. - Wildfire has **no `aal` and no `ep`**. Its one loss figure is `aal_ceiling`, burn probability times value, with `aal_ceiling_nearby` beside it; each place carries its `burn_probability`, `flame_length` (six conditional probabilities) and `nearby_burn_probability_max`. Their warnings add `undefended`, `map_resolution`, `baseline_2010`, `snapped_to_land`, `global_curve`, `sum_of_places`, `rare_tail`, `depth_not_monotone` (flood) and `ceiling_not_estimate`, `landscape_2020`, `non_burnable_cell` (wildfire). ## History lane pressure and storm size The history lane's wind model (Holland 2008, as CLIMADA computes it) is driven, fix by fix along each track, by the storm's central pressure and its radius of maximum wind: how far from the center the strongest wind blows, which sets how wide the damaging winds reach. **Central pressure.** Where the US agencies' pressure is missing, the lane takes the WMO agency's, then the other regional agencies' in the record. Where none reports one, it estimates the pressure from the reported wind with the Knaff and Zehr (2007) wind-pressure relationship in the form Courtney and Knaff (2009) set out for warning centers: it weighs latitude, storm size and forward speed, has its own equation below 18 degrees, and was fitted to aircraft data rather than to one basin. No fix that falls back this far reports a radius of gales, so size enters at its climatological value. Up to `climada-6.1.0-r2` the last resort was Atkinson and Holliday (1977), a West Pacific fit, in every basin; before 23 September 2026 the gap took the environmental pressure, which left typhoons before about 2001 with almost no wind. `pressure_sources` records which source drove each storm that caused a loss, and the `pressure_estimated` warning counts the storms with any estimated fix. **Radius of maximum wind.** Each fix takes the radius the record reports there, or within an hour of it (the lane's time step: the record often puts a landfall report a few minutes off the hour). Where it reports none, the fix takes CLIMADA's own estimate from its central pressure (`estimate_rmw`, a fit to the IBTrACS record of 1980 to 2019): about 15 nautical miles for a deep storm, widening to about 50 as the pressure rises toward 1,010 hPa. Up to `climada-6.1.0-r2` one reported radius was carried along the whole storm, or 40 nautical miles when none was reported. The US record reports few radii before 2004 and the other basins fewer, so older storms are where this moves results. Each `pressure_sources` row also gives the storm's `radius_fixes`, and the `radius_estimated` warning counts storms whose radius is estimated at most of their fixes. What the two changes do, each schedule replayed by `climada-6.1.0-r2` and `climada-6.1.0-r3` (AAL in dollars; three places share $10,000,000, a single place holds it): | Schedule | Curve | r2 | r3 | What moved | |---|---|---|---|---| | Miami, Houston, New Orleans, 1995 to 2024 | auto | 31,162 | 31,115 | Opal (1995) no longer reaches New Orleans; Frances (2004) falls and Irene (1999) rises at Miami | | Manila, 1995 to 2024 | auto | 107,425 | 102,950 | Nanmadol (2004) no longer reaches Manila; Babs (1998) falls, Angela (1995) rises | | Manila, 1995 to 2024 | emanuel_2011 | 175,898 | 167,595 | the same storms | | The worked example in the agent walk | emanuel_2011 | 379,136 | 374,012 | Ivan (2004), whose radius is reported at 2 of 179 fixes | For Manila two of the eleven storms that caused a loss (Angela and Babs) have fixes with no reported pressure, and swapping the wind-pressure relationship leaves the AAL unchanged; all of the change comes from the radius. The storms with station records in the 2026-09-23 credibility review (Katrina, Wilma, Irma, Ike, Ida, Isaac, Xangsane and Rammasun) report their radius at the fixes that set their peak wind, so their site winds do not change, and neither does Georges (1998) at New Orleans: its 80 nautical miles is reported on its Mississippi landfall fix, the hour of its peak. Isaac's modeled wind at New Orleans stays above the storm's own best-track maximum at that hour, and the modeled winds still run above the station readings at inland places: that is the missing terrain roughness (`no_terrain`), not the storm size. Older records move more. Three places per basin, 1980 to 2009, `auto`: | Basin and places | r2 | Radius only | r3 | Storms with a loss using the relationship (r3) | |---|---|---|---|---| | North Atlantic: Miami, Houston, New Orleans | 47,859 | 51,040 | 51,040 | 0 of 10 | | Eastern Pacific: Acapulco, Cabo San Lucas, Honolulu | 59,735 | 22,688 | 18,680 | 4 of 13 | | West Pacific: Manila, Hong Kong, Tokyo | 60,009 | 29,852 | 29,852 | 21 of 49 | | North Indian: Mumbai, Kolkata, Chittagong | 113,239 | 76,991 | 71,300 | 7 of 9 | | South Indian: Port Louis, Saint-Denis, Beira | 567,383 | 314,094 | 311,800 | 11 of 20 | | South Pacific: Suva, Noumea, Cairns | 663,166 | 274,577 | 274,577 | 14 of 21 | The 40 nautical mile default made storms with no reported radius far wider than CLIMADA's estimate for a deep storm, so places near but not under the track lost too much: Odile (1984) at Acapulco falls from 43.8 to 31.5 m/s. Andrew (1992) goes the other way. It reports 10 nautical miles at its Florida landfall and nothing on the fixes either side, where CLIMADA estimates about 15, so its peak wind at Miami rises from 54.4 to 56.8 m/s and its Miami loss from $560,802 to $684,192. ## The PiWind lane PiWind is the Oasis Loss Modelling Framework's reference windstorm model, which describes itself as a toy. Its hazard grid is 200 area-peril cells (the same 100 squares for wind and for storm surge) covering lat 52.734 to 52.795, lon -0.918 to -0.857 in England, about 4 km by 7 km, with 1,447 synthetic events over 1,000 sample years. From `oasislmf-2.5.7-r2` its results carry the same `metrics` block, read from Oasis's own ground-up output tables: - `aal` and `sd` come from the average loss table: the mean and the spread over every sample of every year. `aal_standard_error` divides by the square root of the 1,000 years, because samples redraw the damage, not the storms. - `ep` and `tvar` come from the exceedance-probability table on Oasis's full-uncertainty basis (`ep_basis`), where every sample of every year counts, so the spread of damage around each curve reaches the tail. The ten-times rule applies to the 1,000 years: 1 in 10 to 1 in 100 are published, 1 in 200 and beyond are withheld. - `year_loss_table` and `event_loss_table` come from the period loss table (means over samples); `events` lists up to 50 of the largest event occurrences with each place's loss, from a second summary by place. - `per_point` comes from the same summary by place. Places outside the grid are listed with `modeled: false` and a loss of zero. - `samples` repeats the spec's sample count. A schedule with no place inside the grid is answered, not failed: every place comes back unmodeled, with no loss figure and no `metrics` block, because nothing was measured. Before `oasislmf-2.5.7-r2` the Oasis kernel stopped with an error on such a schedule; the lab now checks each place against the grid first, with the same geometry and rule as PiWind's own lookup. ## Insured and net perspectives From `oasislmf-2.5.7-r3` a PiWind spec may carry policy terms and reinsurance (`terms`; the fields are on [the PiWind page](lab-piwind.md)). Oasis's own financial module then computes two more views of the same simulated years, and each comes back as a full `lab-metrics/1` block with every field above: - `result.metrics` stays the **ground-up** block, unchanged, so every reader of it keeps working. It carries no `perspective` field: an absent `perspective` means ground-up, on every lane. - `result.perspectives.il` is the **insured** block (`perspective: "il"`): the loss after each place's deductible and limit and the account's deductible, limit and share. - `result.perspectives.ri` is the **net of reinsurance** block (`perspective: "ri"`), present when the spec gives reinsurance: the insured loss after the quota share and the catastrophe layers. - `result.terms_summary` says in one sentence (`text`) and in parts (`split`, each with its `aal` and its `share` of the ground-up average) where the ground-up average annual loss goes: kept by the policyholder, written by other insurers when the account share is below 100%, ceded to reinsurers, kept by the insurer. The insured and net blocks read Oasis's insured and net output tables exactly the way the ground-up block reads the ground-up ones, so AAL, OEP, AEP and TVaR mean the same things there: an insured 1-in-100 AEP is the annual insured total exceeded once in 100 years. Their `per_point` and per-event figures share account and reinsurance amounts back to the places in proportion to each place's loss at the step before, as Oasis does, so the places add up to the total. Their first assumptions state the terms as applied, in plain words. ## Models from the exchange A model listed on the exchange (`"model": "exchange:"`, as `GET /models` lists it) is its provider's own catastrophe model, run on the provider's own Oasis platform. The lab writes your places as the OED location file the PiWind lane writes, with every place covered for all perils so the provider's model decides which of its perils apply, sends the places inside the area the provider lists for the model to that platform, and reads Oasis's own ground-up tables back. The `metrics` block is the PiWind lane's, read the same way from the same tables (`basis` `event_set`, `ep_basis` `full_uncertainty`): PiWind run on a provider's platform gives the PiWind lane's figures exactly. What differs: - The spec takes the places alone. Every figure is ground-up, and the provider sets the number of samples. - `years` is read from the provider's own outputs, so the ten-times rule applies to the provider's event set as to every lane's. - Places outside the provider's listed area are never sent and come back unmodeled, and so does a place the provider's own lookup does not take; `result.locations` counts the places submitted, sent and modeled. - The assumptions say that your places and their values were sent to the provider, that the hazard, damage curves and event set are the provider's and Craton has not checked them, and what each place was sent as. The warning `provider_copy_kept` says when the copy of your places on the provider's platform could not be removed after the run. - The document names the provider (`provider.name`), the listing and its version (`title`, `model_version`), and carries a `notice` that running the model sends your places to the provider. A run id for an exchange model is a content address over the spec, the listing and its version, and the provider's model on its platform, so a new listing version is a new run. A failed exchange run is run again when the same spec is sent again: the usual cause is the provider's platform being down or busy. ## Replaying a result Run ids are content addresses over the spec, the model version and the pinned inputs. A new model version gives the same spec a new id; every document stored under an older version still answers its own id, unchanged. The STORM lane computes wind fields in parallel, and the number of processes never changes a result.