Bindings

Definition: review binding A review binding connects one Langfuse annotation queue to one target golden dataset and defines how reviewed data is mapped into the target schema. A binding is data only — it does not run by itself. Implemented as the QueueToDatasetBinding contract in sdk/semantic_evaluation. See review binding.

The field-level contract for a binding (QueueToDatasetBinding, ReviewPolicy, Projection, ProjectionField, ValueSelector, MetadataMapping, QueueCleanupConfig) is owned by sdk/semantic_evaluation and documented in full, with a worked Python example, in 06-review-sync.md — Declarative bindings. This page covers what is specific to how review_sync_app stores, validates, and serves bindings over HTTP.

Binding fields at a glance

Field	Required	Meaning
binding_id	yes	Stable unique identifier; also the /v1/bindings/{binding_id} path segment
queue_id	yes	Langfuse annotation queue ID to read from
langfuse_profile_id	yes	ID of a Langfuse connection profile registered with this service
dataset_scope	yes	Evaluation scope name written into promoted record metadata
product_id, dataset_version, created_by	yes	Stamped onto every promoted record's metadata
review_policy	yes	See Review policy below
dataset_target	yes	Target Langfuse dataset name, description, metadata, and JSON Schemas
input_projection, expected_output_projection	yes	See Field projections below
description	no	Human-readable note
queue_cleanup	no	See 03-langfuse-profiles-and-cleanup.md
metadata_mapping	no	Additional per-record metadata: record_id, workflow_version, selected_agent, interaction_type, tags

A binding must set langfuse_profile_id to a profile ID already registered through POST /v1/langfuse-profiles (see 03-langfuse-profiles-and-cleanup.md) — the sync job resolves that profile's credentials at execution time, not at registration time, so registering a binding for a profile that does not yet exist succeeds; the sync job then fails with a clear error until the profile is registered.

Review policy and field projections

Review policyField projections

The ReviewPolicy contract shape (required_score_names, normalized_score_min, normalized_score_max, low_score_correction_threshold) is documented in 06-review-sync.md — ReviewPolicy. review_sync_service/policy.py enforces it during a sync job, checking each required score in order and returning the first reason a record is not promotable yet:

Reason	When it applies
missing_required_score	A required score name is absent from review_scores
non_numeric_score	The required score's value is not numeric (bare number or a {value: ...} object with a numeric value)
score_out_of_range	The numeric value falls outside [normalized_score_min, normalized_score_max]
missing_required_correction	The value is at or below low_score_correction_threshold and the reviewed record carries no corrected_output

A record that fails any of these checks is reported as policy_skipped and is not promoted; see 02-sync-execution.md — Promotion outcomes for how that fits into the rest of a sync job.

input_projection and expected_output_projection each describe one side of the resulting dataset record as an ordered list of ProjectionField entries; each field resolves through an ordered fallback chain of ValueSelector entries (dotted queue_item_path lookup, or a literal constant), with an optional coerce to string, boolean, integer, or number. The full selector and coercion contract is documented in 06-review-sync.md — Projections, fields, and selectors.

review_sync_service/mapping.py resolves each selector against a lookup context built from the reviewed record plus a selected_output key (the corrected output when present, otherwise the original output). Path lookup (_lookup_path) walks dict keys, list indices, and object attributes in the same dotted path, so a selector such as selected_output.routing_decision.handoff_to also falls back cleanly to selected_output.handoff_to when the queue payload nests the field differently — this is what the fallback-chain example in the SDK doc is for.

Binding validation

POST /v1/bindings/validate and the semantic checks inside PUT /v1/bindings/{binding_id} are service-owned logic, not part of the shared contract. validate_review_sync_binding() checks each projection against its target dataset_target schema and returns one error string per problem found:

Check	Error condition
Duplicate fields	The same dataset_field appears more than once in one projection
Undeclared field	A projection field is not listed in the schema's properties
Missing required field	A schema required field has no matching projection field

These are pure Pydantic-level and schema-shape checks — they do not read Langfuse or require the binding's queue or Langfuse profile to exist. This lets POST /v1/bindings/validate catch mapping mistakes before a binding is ever stored, and PUT /v1/bindings/{binding_id} rejects a binding outright (HTTP 400) if it fails any of the same checks.

Binding storage

The binding store (review_sync_service/binding_store.py) persists all bindings as one JSON document, keyed by binding_id. PUT /v1/bindings/{binding_id} creates the binding if it is new or fully replaces it if the ID already exists — there is no partial update. GET /v1/bindings returns every stored binding sorted by binding_id.

Example

Full binding — reference-pod-routing

{
  "binding_id": "reference-pod-routing",
  "description": "Promote reviewed routing decisions into the routing golden dataset.",
  "queue_id": "queue-123",
  "langfuse_profile_id": "reference-pod",
  "dataset_scope": "routing",
  "product_id": "offline_eval",
  "dataset_version": "1.0",
  "created_by": "review-sync-service",
  "review_policy": {
    "required_score_names": ["Answer Relevance", "Answer Correctness"],
    "normalized_score_min": 1.0,
    "normalized_score_max": 5.0,
    "low_score_correction_threshold": 3.0
  },
  "dataset_target": {
    "name": "offline_eval_routing_golden_v1.0",
    "description": "Routing golden dataset populated from reviewed queue items.",
    "metadata": {"dataset_kind": "golden", "scope": "routing"},
    "input_schema": {
      "type": "object",
      "properties": {"query": {"type": "string"}},
      "required": ["query"]
    },
    "expected_output_schema": {
      "type": "object",
      "properties": {
        "handoff_to": {"type": "string"},
        "tool_called": {"type": "boolean"},
        "reason": {"type": "string"}
      },
      "required": ["handoff_to", "tool_called", "reason"]
    }
  },
  "input_projection": {
    "fields": [
      {
        "dataset_field": "query",
        "selectors": [
          {"queue_item_path": "original_input.query", "coerce": "string"},
          {"queue_item_path": "original_input.args.0", "coerce": "string"}
        ]
      }
    ]
  },
  "expected_output_projection": {
    "fields": [
      {
        "dataset_field": "handoff_to",
        "selectors": [
          {"queue_item_path": "selected_output.routing_decision.handoff_to", "coerce": "string"},
          {"queue_item_path": "selected_output.handoff_to", "coerce": "string"}
        ]
      },
      {
        "dataset_field": "tool_called",
        "selectors": [
          {"queue_item_path": "selected_output.routing_decision.tool_called", "coerce": "boolean"},
          {"queue_item_path": "selected_output.tool_called", "coerce": "boolean"}
        ]
      },
      {
        "dataset_field": "reason",
        "selectors": [
          {"queue_item_path": "selected_output.routing_decision.reason", "coerce": "string"},
          {"queue_item_path": "selected_output.reason", "coerce": "string"}
        ]
      }
    ]
  },
  "queue_cleanup": {"enabled": true},
  "metadata_mapping": {"tags": ["offline-eval", "golden", "routing"]}
}

Note that langfuse_profile_id is required — a binding cannot be registered without one, and the profile it names must be registered (via POST /v1/langfuse-profiles) before the first non-dry-run sync job for that binding.

Register and validate it with:

curl -s -X POST http://127.0.0.1:8002/v1/bindings/validate \
  -H "Content-Type: application/json" \
  -d @binding.json

curl -s -X PUT http://127.0.0.1:8002/v1/bindings/reference-pod-routing \
  -H "Content-Type: application/json" \
  -d @binding.json

See 04-api-reference.md for the full route list and 02-sync-execution.md for what happens when a sync job runs against a stored binding.