Review sync glossary
This glossary defines terms that are local to review_sync_app/ — the shared
review-sync service and its supporting docs. For terms used across more than
one package in the workspace (for example shared service, review sync,
annotation queue, golden dataset, reviewer comments, or human review scores),
see the repository glossary instead of looking for
a second definition here.
The shared review-sync data contracts (QueueToDatasetBinding,
ReviewPolicy, Projection, ValueSelector, JobRequest/JobResult,
CheckpointState, and so on) are owned by sdk/semantic_evaluation and
defined in
06-review-sync.md and
its glossary. The entries
below name the same concepts the way this service's docs and code talk about
them, and link to the SDK's field-level reference rather than repeating it.
Review Binding
A review binding connects one Langfuse annotation queue to one target golden
dataset and defines how reviewed data should be mapped into the target
schema. It is implemented as the QueueToDatasetBinding contract in
sdk/semantic_evaluation — see
06-review-sync.md — Declarative bindings
for the full field reference.
The binding is configuration data. It does not run by itself; a sync job reads it and executes against it. See 01-bindings.md.
Review Policy
A review policy describes which normalized score names must exist, what
score range counts as valid, and when corrected output is expected before a
reviewed record can be promoted. It is implemented as the ReviewPolicy
contract in sdk/semantic_evaluation — see
06-review-sync.md — ReviewPolicy
for the field reference.
review_sync_service/policy.py enforces this policy during sync-job
execution. Reviews that fail the policy are reported as policy_skipped and
are not promoted. See 01-bindings.md — Review policy.
Sync Job
A sync job is one immediate execution of the review-sync service for one
stored binding. The job reads reviewed items, attempts publication, and
stores a machine-readable result (JobResult). See
02-sync-execution.md.
Langfuse Connection Profile
A named set of Langfuse project credentials registered with the review-sync
service via POST /v1/langfuse-profiles. Each profile stores a profile_id,
base_url, public_key, and secret_key. The service resolves profiles at
sync-job execution time so pod code never carries credentials directly.
Unlike the binding contract, this model is owned entirely by
review_sync_service/langfuse_profiles.py — there is no SDK-level
equivalent. public_key and secret_key each accept a literal string
(local development) or an Azure Key Vault secret URI of the form
https://{vault}.vault.azure.net/secrets/{name}. See
03-langfuse-profiles-and-cleanup.md.
Binding Store
The file-backed registry (review_sync_service/binding_store.py) that
persists review bindings, keyed by binding_id, for local development and
service execution.
Checkpoint State
The service-owned record of which reviewed queue-item versions have already
been handled for a given binding, plus which completed queue items are still
pending deletion retry. Persisted per binding by
review_sync_service/checkpoint_store.py. On each non-dry-run sync job that
handles at least one record or changes pending cleanup state, the executor
advances the checkpoint so the next run skips already-handled items,
regardless of whether they are still present in the queue. The contract
shape (CheckpointState, HandledVersion, CheckpointAdvance) is documented
in
06-review-sync.md — Checkpointing.
See 02-sync-execution.md — Checkpointing.
Job Record
The persisted result of one sync job — a JobResult written to
REVIEW_SYNC_JOBS_DIR/{job_id}.json, including item counts, publication
outcomes, and failure details when a job does not finish cleanly. Returned to
callers via GET /v1/sync-jobs/{job_id}.
Cleanup Audit Record
A JSON file written under REVIEW_SYNC_CLEANUP_AUDITS_DIR/{binding_id}/{job_id}.json
for every non-dry-run sync job on a cleanup-enabled binding that had
cleanup candidates or attempted deletes. It captures the current-job and
retry candidate lists, the outcome of each delete attempt, and the resulting
pending-retry set — independent of the persist_artifacts job artifacts. See
03-langfuse-profiles-and-cleanup.md — Queue cleanup.
API Major
The major version of a shared service's HTTP API, carried in the URL path
(/v1). A breaking change adds the next major beside the current one, and each
semantic_evaluation major calls exactly one major per service. See
Service API versions.