Benchmark data and provenance

Data classes

reference_data

Digitized or author-provided literature data. Each dataset records its citation, units, extraction method, and any known uncertainty.

baselines

Internal directory name for compact reference results generated by a specific Otter or documented precursor revision and accepted after review.

cache

Full electronic solver states used to avoid recomputation during development. Caches are local artifacts and are not committed.

Repository policy

Version-controlled benchmark data may contain CSV, JSON, and compressed NumPy arrays that can be loaded with allow_pickle=False. Python pickles, absolute local paths, generated figures, SCF scratch data, and parameter-scan caches are excluded.

Every accepted reference result has a JSON manifest containing:

  • a schema version and benchmark identifier;

  • thermodynamic conditions and units;

  • all relevant model choices;

  • producer project and Git commit;

  • array-file checksum;

  • literature citation and provenance;

  • expected comparison metrics and tolerances.

Execution tiers

Fast tests

Validate schemas, invariants, and compact cached curves.

Documentation gallery

Replot accepted arrays. It must remain suitable for a local or CI build.

Slow regeneration

Explicitly rerun the electronic and ionic calculations. These scripts are opt-in and are not invoked by Sphinx.

The accepted reference result never replaces convergence testing when a new physical regime or numerical method is introduced.