AGENTS.md — Active Fire Detections collection

Guidance for AI agents and automated clients working with this catalog.

One rule survives every edit to this file. Every claim here is either quoted from a source or measured from the data. If you cannot point at where a fact came from, it does not belong in this file. An agent acting on an invented join key or an invented column name produces a confident wrong answer, and nothing downstream catches it.

Each factual claim below is tagged with how it was established: [attested] the publisher states it, with a link; [researched] found outside the source's own metadata, with a link; [derived] computed from the published data, with the query that computed it.

What this collection holds

Global satellite active fire and thermal anomaly detections. Public root: https://data.source.coop/portolan-mirrors/firms-catalog/catalog.json

This is a mirror. NASA LANCE FIRMS is the producer and the authoritative source. See NASA FIRMS.

Sensors carried [attested, data_availability API]:

GOES and Landsat are published by FIRMS but are not carried here. GOES is geostationary at a 10-minute cadence and would dominate the archive by volume. Landsat NRT covers only the US and Canada.

Layout and how to exploit it

Three different tables share the year= directory, so glob the filename, not *.parquet:

year=*/*.parquet reads all three at once and is always wrong: the aggregates have no acq_date, and live.parquet repeats rows that detections.parquet already holds. partition:glob in the collection names year=*/detections.parquet for this reason.

One GeoParquet 2.0 file per year, hive-partitioned on year only, written with zstd level 22 and about 100,000 rows per row group [derived]. Rows inside each file are ordered along a Hilbert curve.

What that ordering buys, measured on the 2010 file [derived]:

• year= in the path prunes whole files without reading them.
• A bounding-box filter prunes row groups, because the Hilbert sort keeps each one spatially tight: row groups 1 to 4 cover 0.21%, 0.16%, 0.06% and 0.13% of the globe. Row group 0 is the exception at 12.6%, being the sparse start of the curve.
• A date filter does not prune row groups. The sort is purely spatial, so every row group spans the whole year: 0 of 48 row groups in the 2010 file sit inside a single month. Filtering by date is correct but reads the file.

Filter on year explicitly when you can. A predicate on acq_date alone does not prune the partition, because the reader cannot map a date onto the path key.

If you want a count or a sum over dates rather than the rows themselves, read the aggregates instead — they are built for exactly that and are a thousandth of the size.

Columns

sensor, quality and type are columns rather than path keys. Each is constant or near-constant within a file, so row-group statistics prune them without deepening the directory tree.

latitude and longitude are not carried. geometry holds the same information. Recover them with ST_X(geometry) and ST_Y(geometry).

Satellite code mapping

FIRMS publishes short codes. This catalog expands them attested for the code meanings, derived for the expansion]:

The mapping lives in SAT_SQL in tools/firms_fetch.py.

Transforms applied to the upstream data

Nothing is filtered, interpolated, or reclassified. Every row FIRMS publishes is carried. These are the complete changes made between the upstream response and the published file, in the order they happen. The code is tools/firms_fetch.py and tools/firms_build.py.

Near-real-time and science-quality rows are never mixed within a sensor and date: FIRMS publishes non-overlapping date ranges for its _NRT and _SP sources, so the two cannot double-count. The quality column records which a row came from.

Derived aggregates

The A5 hexagon aggregates published alongside the detections are built by tools/firms_aggregate.py. gpio pivots exactly one categorical column per run, so day, sensor and day/night are aggregated separately and joined on a5_cell. That join is safe because every run uses the same resolution and therefore the same cells. The cells hold no cross-tabs: there is a count_modis and a count_20260903, but no count_modis_20260903. A query that needs one sensor on one day must go to the detections, not the cells.

What is in a cell

The all-time files carry totals and the monthly buckets only [derived]: the per-sensor and day/night splits are not aggregated across years. Ask a year file for those.

When to use them

The aggregates answer any question that is a sum over whole cells and whole days. They cannot answer anything needing a single detection, a sub-cell location, an FRP distribution, or two dimensions at once.

Measured on 2025 — detections.parquet is 1,032 MB, aggregate-r5.parquet is 2.0 MB, aggregate-r8.parquet is 15.9 MB [derived]:

Local reads of a warm file, so the times understate the difference: over HTTP the aggregate is 2 MB against a gigabyte, and that ratio is the point rather than the milliseconds.

Pick r5 for continental questions and r8 for regional ones: r5 is roughly 7,600 cells globally and r8 roughly 150,000 [derived]. r10 exists only for all time, where there is no raw-point layer to fall back on.

The all-time files lag the year files. They totalled 614,362,367 detections when last built, against 631,316,915 across the year files [derived] — the all-time archive is rebuilt less often than the years it summarises. Use the year files when the answer has to be current.

Quirks that produce silently wrong answers

confidence is not comparable across instruments. MODIS publishes an integer 0-100. VIIRS publishes l, n, or h [attested]. Both live in one VARCHAR column. VIIRS values are normalised to the documented letters, because the bulk feeds spell them out as low/nominal/high; see transform 4. confidence_pct holds the numeric value for MODIS rows only. This catalog deliberately publishes no crosswalk between the two, because no authoritative threshold table was found. Do not invent one. WHERE confidence > 80 silently excludes every VIIRS row.

type is NULL for every NRT row. The FIRMS FAQ states NRT data "does not attribute the static sources/inferred hotspot 'type'" [attested, FAQ], and the NRT CSV has 14 columns against the science-quality 15 [derived]. WHERE type = 0 therefore drops all recent data. Write WHERE type = 0 OR type IS NULL.

NRT and science-quality rows coexist and differ in accuracy. Filter on quality. NASA states NRT MODIS/Aqua locations can be off by "several kilometers" after spacecraft manoeuvres [attested, FAQ].

A detection is not a fire. It is a pixel flagged as a thermal anomaly. Gas flares, volcanoes, and industrial heat sources are included, distinguished by type in science-quality data only.

Coordinates are pixel centres, not fire locations [attested]. MODIS pixels are 1 km; do not treat a point as a precise ignition site.

Worked queries

Every query below was run before it was written down. They were verified against a locally built year file with the identical layout while the catalog was being prepared; the s3:// paths resolve once the data is published.

Count everything, one glob across the whole record:

One year, one month. year prunes the file; the month does not prune row groups, because the sort is spatial, so this reads 2020 and filters:

If the per-sensor split is all you need, aggregate-r5.parquet answers it from 2 MB rather than 800, and agrees exactly — see the aggregates section.

High-power night-time detections in a bounding box. This one returns the January 2020 Australian fires:

Join keys

The catalog holds one collection, so nothing joins to anything else here. There is no stable per-detection identifier: FIRMS does not publish one, and a detection is uniquely identified only by the tuple (sensor, acq_datetime, geometry).

Structure

Assets and structural links resolve relative to the object that carries them.