Testing

uv run pytest review_sync_service/tests/ -q

The suite has six files, each covering a different layer of the service.

File	Purpose
test_app.py	HTTP contract tests through TestClient: binding round-trip (PUT/GET /v1/bindings), POST /v1/bindings/validate semantic-error reporting, POST /v1/sync-jobs wiring (with the executor monkeypatched), and the full /v1/langfuse-profiles CRUD round-trip including 404s for unknown profiles
test_api_versions.py	Every /v1 operation has a deprecated unversioned twin; retirement headers appear only on a scheduled mount; the request log carries api_version and user_agent
test_openapi_snapshot.py	The committed openapi/v1.json matches the app, holds only /v1 paths, and has no deprecated operations
test_executor.py	Sync-job execution end to end: policy-skip-then-continue, unchanged-then-continue, failed-publish-stops-and-defers, checkpoint-based re-run skipping (including a reviewed item re-added with a new identity), and the full queue-cleanup lifecycle — delete on first pass, retry a failed delete on a later job, and confirm dry runs never delete or persist cleanup state
test_mapping.py	Projection field resolution — fallback-chain selectors, nested-output lookups, reviewer-comment passthrough — and validate_review_sync_binding()'s schema-shape checks
test_policy.py	evaluate_review_policy() against both bare numeric scores and normalized {value, comment, score_id, data_type} score objects

Shared fixtures live in conftest.py.

Shared test fixtures

Fixture	Purpose
build_binding_payload()	Returns one complete, valid binding payload (including langfuse_profile_id) that tests can mutate freely — each call returns a fresh deep copy
configure_service_state(tmp_path)	Redirects every REVIEW_SYNC_* state and artifact path into a temporary directory via monkeypatch.setenv, and pre-populates one Langfuse profile (test-profile) so executor and endpoint tests can resolve credentials without a real Langfuse project

What each executor test proves

test_executor.py is the largest file because the execution flow has the most service-specific behavior to protect:

• A job with a mix of policy-failing and policy-passing records skips the failing ones (policy_skipped) and still promotes the passing ones in the same run.
• A record that already matches the golden dataset is skipped_unchanged and does not block later records.
• A failed_publish outcome stops the job and defers every later pending record — those records are not marked handled, so a later job retries them.
• Re-running a job after a successful one skips every previously handled (queue_item_id, record_id, reviewed_at) version; a queue item that reappears with a new reviewed_at is treated as a new version, not a duplicate.
• Queue cleanup deletes eligible items when enabled, retries a previously failed delete on a later job (merged with that job's own new candidates), and — critically — a dry run neither deletes anything nor writes any cleanup or checkpoint state.