Methods & Methodology

All datasets in this catalogue originate from official Australian government sources:

• Australian Bureau of Statistics (ABS) — national statistical agency; all ABS data released under CC BY 4.0
• South Australian Government agencies — SA Health, SA Police, DEW, DIT, CBS, EPA, PIRSA; mix of CC BY 4.0 and Crown Copyright with reuse permissions
• Federal agencies — AIHW, Productivity Commission, Attorney-General's Department, NIAA; predominantly CC BY 4.0

Every dataset includes a README with: source URL, retrieval date, licence verbatim from the source, field definitions, temporal coverage, and known limitations. No unofficial or unattributed data is included.

View detailed provenance graphs →

Datasets are manually curated (July 2024) using a four-criteria gate documented in DR-001:

1. Official source — published by a government statistical agency or department
2. Aggregate or de-identified — no individual records; see DR-003
3. Open licence — CC BY, CC0, or Crown Copyright with explicit reuse permissions; see DR-002
4. Complete metadata — source, retrieval date, field definitions, licence

This process curated 95 datasets from an initial scan of 120+ candidates. The 25 excluded datasets failed licence compatibility (8), metadata completeness (9), or were microdata-only (8).

The catalogue tracks three quality dimensions for each dataset:

• Completeness — proportion of expected metadata fields present (name, domains, licence, temporal coverage, file count, README path)
• Timeliness — days since last data update; datasets with data from the current or previous year are considered current
• Documentation — presence of field definitions, provenance, and known limitations in the README

Aggregate metrics and per-dataset quality scores are available on the Data Quality Dashboard.

All quantitative results shown on this site include uncertainty quantification:

• Proportions — Wilson score intervals (95% confidence), more accurate than Wald intervals for small samples or extreme proportions
• Means and custom statistics — bootstrap percentile intervals (10,000 resamples, seeded for reproducibility)
• Comparative analyses — effect sizes (Cohen's d) reported alongside p-values

Sample sizes (n) are stated for every estimate. Seeds are fixed and displayed for reproducibility. Statistical helper functions are unit-tested against reference values from scipy (Python) and R.

View demo analyses with full uncertainty →

• Snapshot at a point in time — datasets were curated July 2024. Government agencies publish updates continuously; this catalogue is not automatically synced.
• No data validation — metadata is transcribed from source READMEs. The actual data files are not checked for consistency, completeness, or accuracy.
• No inferential statistics on original data — the demo analyses are illustrative showcases of method, not research claims. They use synthetic or aggregate data only.
• Licence interpretation — licence compatibility decisions (DR-002) reflect the maintainer's interpretation of open licence terms. Users should verify licence compliance for their use case.

• Automated sync — run a weekly scraper to check source URLs for updates and flag stale datasets
• Data validation — parse CSV/Excel files to verify field counts, data types, and value ranges match README documentation
• Cross-dataset join keys — document common identifiers (LGA codes, SA2 codes, year) to enable linked analyses
• Contribution workflow — accept dataset proposals via GitHub issues with a structured template (see contribution guide)
• Real consistency checks — validate that LGA-level totals reconcile with state totals across paired datasets (e.g., population estimates vs. health service usage)