Smithsonian Open Access Archive

This is a mirror of public domain data archived from the Smithsonian Institution's Open Access S3 bucket. The source data can also be searched via the Smithsonian's Collections Search Center by limiting results to CC0 media. We look forward to enhancing the usability and discoverability of this data in the coming months.

This repository is maintained by the Library Innovation Lab at Harvard Law School Library as part of our Public Data Project.

Contents

• Navigation
  • 3D models
  • Images
  • Metadata
• Search
  • Files catalog
  • Metadata catalog
  • Linkages table
• Downloading data
• Update frequency
• Smithsonian unit codes

Navigation

At present, this repository mirrors the directory structure used by the Smithsonian:

Each root-level directory contains a different type of data: 3d contains 3D models, media contains images, and metadata contains metadata for all objects. For more information on working with a given type of data, please read the corresponding section below.

3D models

3D models are organized solely by identifier. Unlike images and metadata, they are not grouped by Smithsonian unit code. Each subdirectory under 3d matches an object identifier, and may contain a number of objects including 3D geometry files (GLB, GLTF, OBJ) and other material:

Also included in each 3d subdirectory is scene.svx.json, an SVX file comprising Smithsonian Voyager scene data as well as general object metadata. This metadata typically includes the object's name, description, and accession number, as well as associated links and identifiers.

Images

Images are organized by Smithsonian unit code. Each subdirectory under media is named for a Smithsonian unit and contains JPEG and TIFF images for that unit:

There is typically, though not always, a high-resolution TIFF for every JPEG and vice versa. Each image file is referenced in an associated metadata record.

Metadata

Metadata is organized by Smithsonian unit code and collected in large text files containing newline-delimited JSON records:

Also included in each subdirectory is index.txt, an index file listing all the metadata files for that directory.

More than 17 million metadata records, constituting over 47 GB, are included. As a consequence, the metadata files are very large and querying them is memory- and time-intensive. If you want to search or analyze the archive, consider using our search catalogs as a starting point.

Search

As a convenience, we've generated lightweight Parquet-based search catalogs for this collection under the search directory:

These catalogs can be readily searched or analyzed using tools like DuckDB. Further information on the catalog files is included below.

Files catalog

The files catalog lists all assets (that is, 3D model and image files) in the collection. files.parquet includes the following fields:

• url: Web URL for the file
• size: Size of the file in bytes
• last_modified: Timestamp for the file's last modified time in UTC

As an example, here's a DuckDB SQL query for computing aggregate image counts and sizes by directory from files.parquet:

Results:

Metadata catalog

metadata.parquet lists all metadata records that are associated with one or more files in the collection. This catalog is generated from the newline-delimited JSON files described above. Note that the metadata files also include many records not associated with an asset (that is, an image or 3D model file); such records are excluded from the catalog. metadata.parquet includes the following fields:

• unit_code: Smithsonian unit code representing the contributing unit or project
• record_id: Unique identifier for the record in its home system
• title: Primary or descriptive title for the record
• guid: DOI or ARK number associated with the record
• metadata_url: Web URL for the newline-delimited JSON file from which the record was sourced

As an example, here's a DuckDB SQL query for aggregating record counts by unit code from metadata.parquet:

Results:

Linkages table

linkages.parquet is an association table representing the many-to-many relation between files and metadata records in this collection. Use this table when you want to join asset files with their associated metadata. linkages.parquet includes the following foreign key fields:

• url: Web URL for the file
• record_id: Unique identifier for the record in its home system

As an example, let's say we want to find public domain images of Sojourner Truth from the National Portrait Gallery. We'll query for matching metadata records and join them to the files catalog via the linkages table. Here's a DuckDB SQL query we might try:

Results:

Downloading data

To download a data object by name, access its source URL from the Source Cooperative user interface:

To download large numbers of files, we recommend using tools such as the AWS CLI or Rclone to access the S3 endpoint directly:

Update frequency

The files in this repository were first collected beginning in August 2025. The repository is updated weekly to mirror additions to the Smithsonian Institution's Open Access S3 bucket.

Smithsonian unit codes

Here is a list of Smithsonian Institution unit codes used to organize parts of this collection: