This page as Markdown, byte for byte: /accumulation.md

Accumulation across a set of objects#

POST /accumulation takes a set of your own canonical objects and answers the three questions a book raises once it is more than one file: how much sits behind each peril, how much sits in each region you declared, and where the value is concentrated. It answers synchronously — the numbers come from the exposures already stored on those objects, so there is nothing to compute against and nothing to wait for.

It is one of the four routes Analyse answers on, beside the point hazard lookup, the event record and the analyse job over a single object.

Read the quickstart first for the two values you export (CEDE_BASE_URL, CEDE_API_KEY) and how to get a key from POST /signup.

What it is for, and what it is not#

Every object you ingest is a file's worth of exposure. An accumulation is the view across them: the number you take into a capacity conversation, and the number that tells you a single half-degree square of Tokyo is carrying two thirds of the book.

What it is not:

not a price. No expected loss, no loading, no premium. That is POST /objects/{id}/price, one object at a time.
not a view of risk. It says what your objects hold, never whether the accumulation should be carried or on what terms. Every response carries that sentence in its disclosure field.
not a modelled loss. Nothing here applies a hazard, a vulnerability curve or a correlation. Two sites in one cell may be damaged by quite different events; the cell says they are near each other and nothing more.
not your schedule handed back. See the next section — that is a rule, not an omission.

Portfolio level, by construction#

Craton's aggregate views are portfolio level and never individual-level records. That rule is why this response is shaped the way it is: everything in it is a sum, a count or a square cell of the globe. No location's reference, name, address, exact coordinates, occupancy or own insured value appears anywhere in the document, and a unit test plants all of those in the input and fails if any of them can be found in the output.

If you want the schedule, read the object: GET /objects/{id} is yours and gives you every row of it. This route exists to say what the set holds.

The request#

{ "object_ids": ["<id>", "<id>", "<id>"] }

That is the whole body — the set is the request, which is why this is a POST rather than a GET: a set of two hundred ids does not fit in a query string. It writes nothing. Reading any of those objects afterwards gives you exactly what it gave you before.

Rule	What happens
Up to 200 objects per call	More is a `422` telling you to split the set. The sums are per currency, so two rollups add.
Only your own objects	An id your account does not hold is a `404` naming it, whether it never existed or belongs to somebody else.
No duplicates	The same id twice would count its exposure twice, so it is a `422`.
Nothing else in the body	An unread member is a `422` rather than a silent ignore.

Not metered. Craton bills per object and per model run; an accumulation mints no object and runs no model, and every object in the set was already billed when it was made. Usage, and cancelling is where the units are.

Walk it#

The block below builds a small book from two sources — a Japanese quake schedule read out of a CSV, and a Thai flood exposure sent as JSON — rolls them up, and then checks two things a reader would otherwise have to take on trust: that the currencies were kept apart, and that another account's object is refused.

set -euo pipefail

: "${CEDE_BASE_URL:?export CEDE_BASE_URL first — see the quickstart}"

# Read one field out of a JSON document on stdin: `field a.b.0.c`.
field() {
  python3 -c 'import json, sys
document = json.load(sys.stdin)
for step in sys.argv[1].split("."):
    document = document[int(step)] if step.isdigit() else document[step]
print(document)' "$1"
}

# 1 — An account of this block's own.
CEDE_API_KEY=$(curl -sS --fail-with-body \
  -H "Content-Type: application/json" \
  -d '{"label": "accumulation walkthrough"}' \
  "$CEDE_BASE_URL/signup" | field api_key.secret)
export CEDE_API_KEY
auth="Authorization: Bearer $CEDE_API_KEY"

# 2 — A schedule, through Ingest. Three buildings around Tokyo Bay, two of
#     them inside the same half-degree square.
#     The currency is on the value column's own header, which is where Ingest
#     reads one from; a bare `Currency` column is recorded in
#     `source_fidelity` as unmapped and the amounts stay untagged.
cat > kanto.csv <<'CSV'
Location ID,Address,Latitude,Longitude,Occupancy,Building Value (JPY)
LOC-0001,"2-16-1 Konan, Minato-ku, Tokyo",35.6284,139.7387,Warehouse,8400000000
LOC-0002,"1-1 Soga, Chuo-ku, Chiba",35.5687,140.1247,Light Industrial,5100000000
LOC-0003,"3-1 Nishi-Shinjuku, Tokyo",35.6900,139.6917,Office,2200000000
CSV

ingest_job=$(curl -sS --fail-with-body -H "$auth" \
  -F "[email protected];type=text/csv" "$CEDE_BASE_URL/ingest" | field id)

state=unknown
for _ in $(seq 1 150); do
  curl -sS --fail-with-body -H "$auth" "$CEDE_BASE_URL/jobs/$ingest_job" > job.json
  state=$(field status < job.json)
  case "$state" in succeeded|failed) break ;; esac
  sleep 0.2
done
if [ "$state" != succeeded ]; then
  printf 'ingest did not succeed: %s\n' "$(cat job.json)" >&2
  exit 1
fi
kanto_id=$(field result.objects.0.id < job.json)

# 3 — The file said nothing about the peril or the territory, because a
#     location schedule cannot. PATCH writes what the file could not say, and
#     that is what the rollup groups by: nothing is inferred from a coordinate.
curl -sS --fail-with-body -X PATCH -H "$auth" \
  -H "Content-Type: application/json" \
  -d '{"peril": {"code": "earthquake", "region": {"countries": ["JP"]}}}' \
  "$CEDE_BASE_URL/objects/$kanto_id" > /dev/null

# 4 — A second object, in another currency and another territory, sent whole.
cat > bangkok.json <<'JSON'
{
  "schema_version": "0.1.0",
  "status": "draft",
  "exposure": {
    "kind": "index",
    "currency": "THB",
    "values": { "business_interruption": { "amount": 250000000, "currency": "THB" } },
    "source_fidelity": { "unmapped_columns": [], "guessed_units": [], "ambiguous_rows": [] }
  },
  "peril": { "code": "flood", "region": { "countries": ["TH"] } }
}
JSON

bangkok_id=$(curl -sS --fail-with-body -H "$auth" \
  -H "Content-Type: application/json" \
  -d @bangkok.json "$CEDE_BASE_URL/objects" | field id)

# 5 — The rollup.
curl -sS --fail-with-body -H "$auth" \
  -H "Content-Type: application/json" \
  -d "{\"object_ids\": [\"$kanto_id\", \"$bangkok_id\"]}" \
  "$CEDE_BASE_URL/accumulation" > rollup.json

python3 -c 'import json
rollup = json.load(open("rollup.json"))
print("objects  :", rollup["set"]["object_count"])
print("totals   :", ", ".join(
    "%s %s" % (row["amount"], row.get("currency", "(untagged)"))
    for row in rollup["totals"]["exposure"]))
print("by peril :")
for row in rollup["by_peril"]:
    print("   %-10s %s" % (
        row.get("peril", "(undeclared)"),
        ", ".join("%s %s" % (part["amount"], part.get("currency", "(untagged)"))
                  for part in row["exposure"])))
print("by region:")
for row in rollup["by_region"]:
    print("   %-10s %s" % (
        "+".join(row.get("territories", ["(undeclared)"])),
        ", ".join("%s %s" % (part["amount"], part.get("currency", "(untagged)"))
                  for part in row["exposure"])))
concentration = rollup["concentrations"]
print("cells    : %s degree squares, %s location(s) placed, %s unplaced" % (
    concentration["cell_size_degrees"], concentration["placed_locations"],
    concentration["unplaced_locations"]))
for group in concentration["groups"]:
    label = group.get("currency", "(untagged)")
    for cell in group["cells"]:
        print("   %s %sN..%sN %sE..%sE  %s  %s location(s), %.1f%% of the group" % (
            label, cell["min_latitude"], cell["max_latitude"],
            cell["min_longitude"], cell["max_longitude"], cell["amount"],
            cell["locations"], 100 * cell["share_of_group"]))
print("vintages :")
for entry in rollup["data_vintages"]:
    if entry["pinned"]:
        source = entry["source"]
        print("   %-10s %s %s %s…" % (entry["peril"], source["id"],
                                      source["vintage"], source["sha256"][:12]))
    else:
        print("   %-10s nothing pinned speaks to it" % entry["peril"])
print("disclosure:", rollup["disclosure"])'

# 6 — Two checks a reader should not have to take on trust.
python3 -c 'import json, sys
rollup = json.load(open("rollup.json"))
totals = {row.get("currency"): row["amount"] for row in rollup["totals"]["exposure"]}
# (a) currencies were kept apart, and the JPY sum is the schedule, exactly.
ok = totals.get("JPY") == 8400000000 + 5100000000 + 2200000000
ok = ok and totals.get("THB") == 250000000
print("currencies kept apart, JPY sum exact:", ok)
# (b) no location survived into the rollup, at any depth.
planted = ["LOC-0001", "Minato-ku", "139.7387", "Warehouse"]
body = json.dumps(rollup)
leaked = [needle for needle in planted if needle in body]
print("nothing individual-level in the response:", not leaked, leaked or "")
sys.exit(0 if ok and not leaked else 1)'

# 7 — Somebody else's object is not in your set. A second account, a second
#     object, and the id refused by name.
STRANGER_KEY=$(curl -sS --fail-with-body \
  -H "Content-Type: application/json" \
  -d '{"label": "another account entirely"}' \
  "$CEDE_BASE_URL/signup" | field api_key.secret)
stranger_id=$(curl -sS --fail-with-body \
  -H "Authorization: Bearer $STRANGER_KEY" \
  -H "Content-Type: application/json" \
  -d @bangkok.json "$CEDE_BASE_URL/objects" | field id)

status=$(curl -sS -o refusal.json -w '%{http_code}' -H "$auth" \
  -H "Content-Type: application/json" \
  -d "{\"object_ids\": [\"$kanto_id\", \"$stranger_id\"]}" \
  "$CEDE_BASE_URL/accumulation")
printf 'a stranger'\''s object in the set: %s %s\n' \
  "$status" "$(field error.code < refusal.json)"
test "$status" = 404

The last two blocks exit non-zero if the sums were ever wrong or if a schedule row ever appeared in a rollup. They cannot, and that is the point.

Reading the answer#

set — the ids you sent, echoed in your order, so a rollup is always attributable to the exact objects behind it.
totals, by_peril, by_region — objects counted, locations counted, and the sums. Per currency, never converted: a set holding two currencies has two sums and no single total, because Craton has no exchange rate it could defend. The rows of each table add up to totals, including one row for the objects that declared no peril or no territory, so nothing goes missing into a gap.
concentrations — where the value sits, as ranked half-degree cells within each currency. Each cell carries its box, the amount, how many locations and objects are behind it, and its share of that currency's placed exposure. The top ten cells per currency are published: a ranked list is a summary.
coverage — what the sums did and did not see. Which objects were counted from their locations and which from their exposure-level totals, how many carried no values at all, and how many locations had no coordinates and so fell in no cell.
assumptions — the method, the currency basis, and limitations: what this rollup cannot tell you, in the response itself rather than on this page.
data_vintages — one entry per peril in the set, naming the pinned snapshot this build would measure that peril against, with its vintage and the SHA-256 of its exact bytes. The sums read no feed; these are the vintages a hazard question about the same set would be answered from, and a peril nothing is pinned for says so rather than being left out.
disclosure — the fixed sentence stating that this is a rollup of stored exposure and not a view of risk.

The full response shape is published as [accumulation.v0.schema.json](/accumulation.v0.schema.json); the map from every route to the schema governing its responses is [api-contract.v0.json](/api-contract.v0.json).

Where the numbers come from#

Ingest writes a schedule's totals into exposure.values and each location's values into exposure.locations[].values. Adding both would double every ingested object, so the rule is: an object whose locations carry values is summed from its locations; an object whose locations carry none is summed from its exposure-level totals. Never both. coverage says which basis each object was counted on.

Regions are the territories the objects themselves declare — peril.region.countries, else jurisdiction.territories. Nothing is derived from a coordinate: turning a coordinate into a country is geocoding, which is enrichment, and Ingest does not enrich. An object declaring several territories is counted once under the set it declares rather than split between them, because nothing in the object says how a split would go.

When it refuses#

Every refusal is in the standard error envelope (error reference).

An id your account does not hold — including one that belongs to somebody else, which is answered the same way because "it exists but is not yours" is itself a disclosure:

{
  "schema_version": "0.1.0",
  "error": {
    "status": 404,
    "code": "not_found",
    "message": "this account holds no object with this id: 4f2c… . An object another account owns is not found rather than refused, because 'it exists but is not yours' is itself a disclosure"
  }
}

An empty set:

{
  "schema_version": "0.1.0",
  "error": {
    "status": 422,
    "code": "invalid_request",
    "message": "object_ids is empty, and an accumulation is computed over at least one object. Name the objects the rollup is over: {\"object_ids\": [\"<id>\", \"<id>\"]}"
  }
}

Every id that was not found is named in one refusal rather than the first one only: fixing a set of forty should take one round trip.