Skip to content

Development & Contributing

This page explains how to work on sdmxflow locally: dev environment, tests/lint, docs preview, and how to add provider support in a disciplined way.

Local dev setup

This project uses uv for development.

uv sync --group dev

Run tests

uv run pytest

Lint and format

uv run ruff check .
uv run ruff format .

Docs preview (Zensical)

Preview locally:

uv run zensical serve

Build the static site:

uv run zensical build --clean

Repository structure (high level)

  • sdmxflow/dataset.py: user-facing SdmxDataset entrypoint and refresh workflow
  • sdmxflow/download/: provider download implementations
  • sdmxflow/query/: upstream metadata queries (e.g., “last updated”)
  • sdmxflow/metadata/: metadata schema + read/write helpers
  • sdmxflow/extract/: structure/codelist extraction
  • tests/: unit tests (provider branches, error classification, contracts)

How to add a provider (implementation outline)

Provider support should preserve the stable artifact contract:

  • dataset.csv with leading last_updated
  • metadata.json with append-only version history
  • codelists/ extraction and mapping

Acceptance criteria for a new provider:

  1. Change detection

  2. Implement an upstream “last updated” resolver for the provider.

  3. Add unit tests for changed vs unchanged behavior.

  4. Download and materialization

  5. Implement a downloader that writes a provider slice to a CSV without the internal last_updated column.

  6. Ensure append_version_slice() can tag it and append deterministically.

  7. Structures and codelists

  8. Ensure structures can be downloaded and codelists can be mapped from dataset columns.

  9. Error classification

  10. Timeouts, unreachable errors, and interruptions should raise typed sdmxflow errors.

  11. Docs update

  12. Add the provider to Provider Support and describe any provider-specific caveats.

PR expectations

  • Keep changes focused and additive.
  • Update docs when behavior changes.
  • Add tests for new branches and error cases.

See the repository contributing guide for details:

Release notes (PyPI)

Publishing to PyPI is automated via GitHub Actions on version tags matching v* (e.g. v0.1.1).

The publish workflow validates that the tag version matches pyproject.toml before building and publishing.

Prerequisite: configure PyPI “Trusted Publishing” (OIDC) for the GitHub repository so the publish step can upload without storing an API token in GitHub secrets.