Soft Commodity Infrastructure

Name

llms.txt

Size

7.53 KB

Content Type

text/plain; charset=utf-8

Last Modified

20 Aug 2026

Source URL

https://data.source.coop/tristangruppwri/soft-commodity-infrastructure/llms.txt

Cloud URI

s3://us-west-2.opendata.source.coop/tristangruppwri/soft-commodity-infrastructure/llms.txt

# Soft Commodity Infrastructure (Brazil)

> The geolocated midstream of Brazilian agriculture: the silos, slaughterhouses, dairy plants
> and sugar/ethanol mills that farms sell into, plus the cooperatives that farms belong to.
> Built entirely from government registries. Published by the World Resources Institute under
> CC-BY-4.0.

This is a Portolan catalog (a STAC profile) of cloud-native GeoParquet. Query it directly over
the network with DuckDB — no download and no credentials required.

It exists to answer one question: for a given rural property, **who do you contact** about
deforestation on that land — the cooperative that counts the farmer as a member, or the buyer
that receives their grain, cattle or cane.

Root: https://data.source.coop/tristangruppwri/soft-commodity-infrastructure/catalog.json
Agent guide: https://data.source.coop/tristangruppwri/soft-commodity-infrastructure/AGENTS.md

## Collections

One collection, `facilities`, holding a unified layer plus the components it is built from.

- [facilities](facilities/collection.json): 83,811 rows across seven files, describing 40,669
  distinct entities. Geometry is WGS84 (EPSG:4326).

## Read the geometry column directly

The geometry is WKB inside the file, but DuckDB's spatial extension recognises the GeoParquet
`geo` metadata and hands it back already typed:

```sql
INSTALL spatial; LOAD spatial; INSTALL httpfs; LOAD httpfs;
SELECT typeof(geometry)
FROM read_parquet('https://data.source.coop/tristangruppwri/soft-commodity-infrastructure/facilities/BR_facilities.parquet')
LIMIT 1;
-- GEOMETRY('OGC:CRS84')
```

So **do not wrap it in `ST_GeomFromWKB`**. That raises
`No function matches the given name and argument types 'st_geomfromwkb(GEOMETRY('OGC:CRS84'))'`.
Pass the column straight into `ST_Within`, `ST_Distance_Sphere` and the rest. An earlier draft
of this file, and the pipeline README, said the opposite. They were wrong.

## Files

Start with `BR_facilities.parquet`. The rest are the component layers behind it.

- `BR_facilities.parquet` — the unified layer, every entity as one row across all tiers
  (40,669 rows: 32,309 points, 8,360 polygons). Columns: `entity_id, entity_kind, tier,
  weight, basis, geom_method, source, geometry`.
- `BR_intake_points.parquet` — grain silos from CONAB (18,802): cooperative, trader and
  official, with registered coordinates and static capacity in tonnes.
- `BR_sisbi_points.parquet` — state and municipal (SISBI) slaughterhouses and dairy plants
  (11,582), the frontier long tail, via Trase, with `inspection_level` (SIE / SIM / CONSORCIO).
- `BR_muni.parquet` — IBGE municipality boundaries (5,570), the geography everything keys to.
- `BR_membership_muni.parquet` — cooperative-member farm counts per município (5,257), joined
  to municipality polygons.
- `BR_buyer_points.parquet` — federal (SIF) slaughterhouses and dairy plants (1,514), geocoded,
  tagged by commodity (beef / poultry / pork / dairy).
- `BR_mill_points.parquet` — sugar and ethanol mills from ANP (417), town-centroid, with
  production capacity in m³/day.

The seven row counts sum to 83,811, but `BR_facilities` is a union of the other six plus the
modelled catchments. **Do not add them together and call the result a facility count.** The
number of distinct entities is 40,669.

```sql
INSTALL spatial; LOAD spatial; INSTALL httpfs; LOAD httpfs;
SELECT tier, count(*)
FROM read_parquet('https://data.source.coop/tristangruppwri/soft-commodity-infrastructure/facilities/BR_facilities.parquet')
GROUP BY tier;
```

## Tiers

`tier` in `BR_facilities` answers two different questions, and they should not be collapsed.
Counts are measured over all 40,669 rows.

**Who has the relationship?**

- `membership_muni` (5,257) — cooperative-member density per município. Matched by município
  code, not by distance. `weight` is the member count.

**Where would the product be delivered?**

- `intake_point` (18,796) — grain silos. `weight` is storage capacity in tonnes.
- `slaughter_point` (13,096) — animal-product buyers (SIF federal + SISBI state/municipal).
  These are **buyers, not cooperatives** — `entity_kind` says so, and a match here implies no
  membership relationship.
- `mill_point` (417) — sugar and ethanol mills. `weight` is capacity in m³/day.
- `gravity_catchment` (3,103) — modelled silo draw-areas. The **only** `modelled` tier.

`weight` therefore carries a different unit in every tier. Never aggregate it without a `tier`
filter.

## Trust labels on every row

Not every point is equally precise, so each carries two labels.

- `basis` — `observed` (37,566 rows) or `modelled` (3,103 rows, `gravity_catchment` only).
- `geom_method` — five values occur, not three: `registered` (18,796, the source supplied a GPS
  coordinate; CONAB silos), `geocoded` (12,007, an address turned into a point, and it can land
  on the wrong lot), `municipality` (5,257, the row is the municipality polygon itself),
  `derived` (3,103, a modelled catchment area) and `municipality_centroid` (1,506, all that was
  available was the town).

Filter on these before drawing conclusions about distance. A `municipality_centroid` point is
not evidence about which side of a municipal line a facility sits on.

## Joins, and where they go wrong

- `cod_ibge` (7-digit IBGE code) is the only clean join key. Unique in `BR_muni` (5,570) and
  `BR_membership_muni` (5,257). It repeats in `BR_intake_points`, and one of its 1,940 distinct
  values there matches no municipality, so an inner join silently drops those silos.
- `BR_facilities` carries **no `cod_ibge` and no municipality name**. Join it to municipalities
  with a spatial predicate, not by code.
- `entity_id` is **not** a primary key: 40,669 rows, 35,478 distinct values, and the namespace
  changes with `tier` (CNPJ strings, CONAB `cda` codes, SIF numbers, Trase `facility_id`s and
  IBGE codes all share the column). Filter on `tier` first.
- `municipio_nome` is free text and is written `SINOP-MT` in `BR_intake_points` but plain in
  `BR_muni`. Never join on it.
- 313 of the 5,570 municipalities have no cooperative-membership row.
- Three municipality polygons fail an OGC validity test. Wrap in `ST_MakeValid` if a predicate
  errors.

## Coverage limits

Read these before treating an absence as a finding.

- **Brazil only.**
- **Rice, cotton and coffee are not covered** — no authoritative open geolocated source exists.
  Neither is oil palm. For an EU Deforestation Regulation workflow, this means coffee and oil
  palm have **no delivery tier in this product**. That is a gap in the data, not evidence that
  a property has no buyer.
- Cooperative membership is a per-município count, not a named cooperative.
- CONAB regenerates its warehouse file daily and keeps no history; snapshot dates live in the
  source repository.
- The files are GeoParquet 1.0.0 with one row group each and no bbox covering column, so a
  bounding-box filter reads the whole file. The largest is 4.9 MB, so this costs little.

## Provenance

Official publication, not a mirror. Pipeline, tests and full methodology:
https://github.com/wri/rural-land (`soft_commodity_infrastructure/METHODOLOGY.md`).

Sources: CONAB (grain warehouses), MAPA/SIF (federal meat and dairy inspection), Trase/SISBI
(state and municipal inspection), ANP (ethanol producers), IBGE (Censo Agropecuário 2017 and
municipal boundaries).

Attribution: `Soft Commodity Infrastructure (Brazil), WRI. Sources: CONAB, MAPA/SIF, ANP, IBGE,
Trase (SISBI) — CC-BY 4.0.`