README
Major TOM ELLIOT-Pretrain
ELLIOT-Pretrain is a Major TOM expansion focused on fast pre-training of multi-modal AI foundation models on Earth Observation data.
- Built on the original MajorTOM global 10 km grid
- 10.56 km tiles with harmonised 10 m resolution across all sensors for fast access
- Co-registered optical, thermal, radar, elevation and land cover in every tile, with a cloud mask for every optical acquisition, the weather at each acquisition, and the tile's OpenStreetMap features
- Three sub-datasets covering complementary temporal regimes
Tile locations were selected using hierarchical spherical k-means clustering over AlphaEarth Foundation embeddings to maximise global environmental diversity. The dataset follows the TACO v3 specification.
Sub-datasets
All three parts share the same layers.
Each tile also carries its WorldCover histogram, DEM statistics, cloud summaries and its climate, soil, socioeconomic and administrative context.
Quick Start
Pick a tile index from Part A (monotemporal) and visualize its layers.
Notebooks
example.ipynb(open in Colab): all three parts with general-purpose libraries (pandas,rasterio,geopandas): metadata queries with filtering, every layer, and a streaming PyTorch DataLoader with parallel fetching.example_taco.ipynb(open in Colab): the same data through the TACO reader, which decodes every layer of a tile in one call from the dataset's own contract (bands, units, scaling, frame times).
For vision-language tasks built from these layers (captions, grounding, grids, phenology and change questions, with scorers and a PyTorch loader), see majortom-elliot-tasks.
Dataset Structure
A row of a per-file table belongs to the tile whose index starts its internal:relative_path (400/s2/img_03.tif is tile 400). The tile centroid in sample.parquet is ml:geometry, WKB in EPSG:4326.
facts/<part>/facts-NNNNNN.parquet holds, for every tile, the measured fact sheet, the series facts of a monthly or burst tile, and NDVI and land-cover grids, as written by majortom-elliot-tasks, so its tasks and captions can be built without reading a pixel.
Changes in this version
- Corrected metadata. In
burst, the per-tile columns and the per-acquisition rows were attached to the wrong tiles; monotemporal Sentinel-2 timestamps were partly in microseconds and its Sentinel-2 geotransforms were wrong; several unit descriptions were wrong. All are corrected, each correction is listed in everyCOLLECTION.jsonunderml:metadata_revision, and the pixels are unchanged. - New layers. Sentinel-1, cloud and shadow masks, ERA5, OpenStreetMap and per-tile statistics, added by the Image and Signal Processing Group (ISP), Universitat de València.
- TACO 0.14. Every part is written by the TACO 0.14 writer, with an
ml:contractdeclaring each layer's bands, units and scaling, and validates with it. Metadata files follow the TACO v3 level names (sample.parquet,children__*.parquet, formerlycollection.parquetandsample__*.parquet). wc/removed.monotemporal/DATA/{tile_id}/wc/data.tifwas a byte-identical copy oflc/data.tifon every tile.
Citation
License
CC-BY-SA-4.0, except the OpenStreetMap files (osm.parquet, admin.parquet) and facts/, which holds OpenStreetMap names and geometries: those are under the Open Database License (ODbL-1.0), © OpenStreetMap contributors.
Required credits: contains modified Copernicus Sentinel data; contains modified Copernicus Climate Change Service information (ERA5); Sentinel-1 RTC by Catalyst, as published on Microsoft Planetary Computer (CC-BY-4.0); ESA WorldCover (CC-BY-4.0); Copernicus DEM; Landsat courtesy of the U.S. Geological Survey; cloud masks computed with OmniCloudMask; © OpenStreetMap contributors.
Acknowledgements
ELLIOT-Pretrain has been made possible thanks to Asterisk Labs, the ELLIOT project (European Commission, Horizon Europe, Grant 101214398), and the Image and Signal Processing Group (ISP) at Universitat de Valencia.
