Documentation
OpenAPIAgent index

Automatic learning API

Automatic learning policies connect one agent or endpoint to evidence capture, 100-request review windows, a versioned Failure Atlas, candidate skills, and release gates. Atlas findings can also propose a better rubric and grader. Proposed changes are not automatically trusted: grader calibration and skill validation use separately reserved evidence.

Learning contract 0.1.14 and observer package 0.1.6 were verified available in production on 18 September 2026 UTC. Read x-dharma-learning-contract-version in the published OpenAPI before integrating. OpenAPI contains the exact request, response, pagination and error schemas. A contract version is not a policy revision, skill version or benchmark result. Availability does not prove a successful mutation or lifecycle transition; those require operation-specific receipts. Controlled-proof execution is not a completed customer qualification; do not infer release readiness from these schemas.

Availability and acceptance

An organization must be enabled for automatic learning and have an owned endpoint, active evaluation contract, evidence-consent policy and billing bindings. An administrator configures these; obtaining a token does not provision them. Check the organization's actual policy and release receipts rather than inferring readiness from this page.

The HTTP and MCP reference-submission interfaces are deployed. Deployment does not establish independently correct labels, a calibrated grader, a passing domain-specific skill, or completed canary and customer-runtime adoption. Image calibration requires retained managed evidence or a sealed external-native capture, plus a separately reserved reference cohort. Verify evidence availability and independent labels for your endpoint before enabling a workflow that depends on calibration.

Gemini 3.8 controlled proof status

Contract 0.1.13 can expose an administrator-created, isolated proof that binds requested and executed model identity to gemini-3.8-flash, records every retry and charge, selects a skill on 20 development cases, calibrates a grader on 100 independent references, and keeps the 100-case final test sealed until an immutable freeze exists. The final proof requires at least 96 complete, schema-valid, evidence-grounded treatment results plus the ordinary non-regression gates. Missing outputs and unresolved all-null grades are failures. This controlled lifecycle does not enroll or overwrite a customer production endpoint.

Install the server SDK

The learning observer 0.1.6 is distributed as a versioned, compiled Node package from Dharma, not through the public npm registry. It preserves the existing observation, capture and signed-pin operations and records HTTP contract 0.1.14 in its source manifest. It does not perform administrator-only policy registration or budget reconciliation, and does not dispatch free preflight automatically. Verify the published archive and integrity manifest before use. Node.js 20 or later is required; CommonJS, named ESM imports and TypeScript declarations are included. There are no runtime dependencies, install scripts, background workers or provider calls. Do not substitute an unrelated CLI or similarly named registry package.

npm install --save-exact --ignore-scripts https://www.dharma-ai.io/sdk/learning-observer/0.1.6/dharma-ai-labs-learning-observer-0.1.6.tgz

Review the release integrity manifest and exact archive. Record the approved SHA-512 integrity in your package lock and use npm ci for deployments. The manifest also records SHA-256, compiler version, an exact file allowlist and per-source hashes. Earlier archives remain immutable. Same-host hashes verify consistency, not independent signing or npm-registry provenance. Signed skills use separate operator-pinned public keys. Installing the observer does not enroll an endpoint, authorize spending or prove runtime adoption.

import {createDharmaLearningObserver} from '@dharma-ai-labs/learning-observer';

const observer = createDharmaLearningObserver({
  baseUrl: 'https://www.dharma-ai.io',
  organizationId, policyId, endpointId,
  token: async () => secretStore.read('dharma-learning-token'),
  trustedKeys, // deployment configuration, never keys from a skill response
});

The identifiers, secret store and trusted keys are your server configuration, not browser input. The package README describes every export. Use pinSkill before your provider call and the appropriate capture method after finalization, as described below. The SDK neither runs Google nor starts an outbox worker for you.

Choose the traffic path

  • Dharma-managed Gemini or BYOK: enroll the owned agent or endpoint once. Finalized eligible requests enter the shared asynchronous capture path. Keep the original generation body, schema, model and decoding. Read the x-dharma-evaluation-status header; enrolled means captured, not evaluated or improved. Other states include not_enrolled, paused, excluded, evidence_unavailable and status_unavailable.
  • Your service calls Google directly: Dharma cannot see that traffic without an observer. Keep generation and its provider credentials in your service. Persist authorized finalized evidence in a durable outbox and upload it through the observations interface. A failed evaluation upload must not replace or delay your application's generation response.
  • A bounded study or Evaluation Runs campaign: keep it separate from recurring production learning. A diagnostic endpoint, imported report, manual Atlas analysis or sidecar event does not enroll your production endpoint in this API. See Evaluation Runs for that separate workflow.

Authentication and operations

The prefix below is /api/v1/orgs/{orgId}/agent-fabric/learning-policies. Use only organization-owned IDs. Every mutation POST or PATCH needs an Idempotency-Key of 16–200 characters. Free-preflight POST is read-only and does not require one. Preserve the key and exact mutation on retries; a reused key with different content conflicts.

  • GET (empty suffix): evals:read. Policy catalog and cursor.
  • POST (empty suffix): organization administrator, evals:run. Policy receipt, HTTP 201.
  • GET /{id}: evals:read. Configuration, progress, costs and blockers.
  • PATCH /{id}: organization administrator, evals:run. Configuration mutation receipt.
  • GET /{id}/windows: evals:read. Frozen window membership and status.
  • GET /{id}/windows/{windowId}: evals:read. Paginated retained evidence and receipts.
  • POST /{id}/preflight: evals:read. Free complete native quote, HTTP 200; no generation, enrollment, reservation or accepted grade.
  • GET /{id}/budget-reconciliations: evals:read. Private controlled-proof budget snapshot, HTTP 200.
  • POST /{id}/budget-reconciliations: organization administrator, evals:run. Idempotent supplemental pre-review budget receipt, HTTP 200; no cap reset or evidence repartition.
  • POST /{id}/calibration-references: organization administrator, evals:run. Signed reference admission, 201; identical replay, 200.
  • POST /{id}/calibration-captures: developer bearer token, learning:calibration-ingest. Prospective native reference capture, HTTP 202; no labels or paid review.
  • POST /{id}/observations: learning:ingest. Evidence intake, HTTP 202.
  • POST /{id}/observations/native: developer bearer token, learning:ingest. Chunk or seal receipt, HTTP 202.
  • GET /{id}/skill?requestId=...: evals:read. Read-only preview; pinned:false.
  • POST /{id}/skill: learning:ingest and evals:read. Durable request assignment; pinned:true.

Read and ingest tokens belong in your server secret store. Native uploads cannot be authorized with a browser cookie. A developer token with evals:run is not organization-admin authority. Use the existing signed-in admin surface or normal Clerk-authenticated MCP connection for configuration; never extract browser cookies into scripts. All resources are tenant-scoped, and denied requests must return no protected content.

Free preflight uses the complete native body, including supplied images, system instructions, response schema and decoding. Independent evidence-consent, MIME, image-byte and 4 MiB packet limits still apply. The response distinguishes the original wire hash from the countTokens projection and lists excluded generation-only metadata. Token estimates are not billed execution receipts: executedModel:null and financialAdmission:false remain explicit. The server-owned input-capacity quote cannot be replaced by a smaller estimated charge. Oversized requests fail rather than lose evidence.

Budget continuation is an optional administrator operation for a paused, quiet controlled proof before any review window. It preserves the original plan, membership, model, retries, final-test denominator and shared lifetime ceiling. Uncertain charges, stale snapshots or started reviews prevent acceptance. It is not a manual Atlas step for enrolled production traffic. Ordinary learning still triggers asynchronously after each100 eligible finalized requests.

The MCP tools fabric_list_learning_policies, fabric_get_learning_policy and fabric_list_learning_windows provide reads. fabric_create_learning_policy, fabric_update_learning_policy and fabric_submit_calibration_references are confirmed admin mutations. Discover their current schemas through tools/list; they call the same authorization and validation handlers as HTTP.

Enroll and configure

Create with { "config": ... }; update with { "expectedRevision": 1, "config": ... }. PATCH takes the complete configuration, not a partial merge. Build it from the published schema and the organization's actual resource IDs:

  • schema: dharma.learning-policy/v1; a descriptive name.
  • Exactly one of managedAgentId and endpointId is non-null.
  • Owned evaluationContractId, evidencePolicyId, generationBindingId and evaluationBindingId.
  • batchSize:100, promotionMode:"automatic_gated".
  • Required dailyCapMicrousd and monthlyCapMicrousd; daily cannot exceed monthly. One USD is 1,000,000 micro-USD. Set only explicitly authorized recurring caps; a previous experiment budget is not recurring authorization.
  • state:"paused" while configuring; switch to state:"enabled" only after checking consent, bindings, evidence and caps.
  • rubricLearning:"calibrated_automatic" and calibrationSetId for independent grader calibration. A null calibration set cannot produce a calibrated-grader pass.
  • A versioned validationEvidenceContract fixing representative request shape, model, permitted image MIME types and image-count limits before comparing scores. A null generation binding cannot support replay.

Configuration changes are revision-checked. Substantive changes create a new version; they do not rewrite old reviews or frozen candidates. State-only pause/resume preserves partial batches. An immutable studyScope cannot be removed to turn a bounded diagnostic into a production rollout.

Observe external requests

For metadata/text observation, submit dharma.learning-observation/v1 with the stable logicalRequestId, completion timestamp, status, model, original instructions, request, response, decoding, ordered attachment fingerprints, nullable provider receipt ID and active skill hash. Status is completed, failed or cancelled. Do not invent successful output for failed requests.

JSON intake is limited to 1 MiB and nesting depth 40. The managed: logical-ID namespace is reserved. Strip credentials before creating evidence; rejection of credential-shaped keys is not a complete secret scanner. Attachment hashes alone do not make an image request replayable.

For full external Gemini replay, use native capture instead:

  1. Allocate and persist one logical request ID. Obtain observer.pinSkill(requestId) before generation; do not retroactively pin an old result. GET skill preview is not a durable assignment.
  2. Persist the exact native request. Preserve images, response schema, decoding and unrelated instruction text. Add the signed skill only in the owned instruction span when the pin supplies one; preserve the original instructions separately.
  3. Run the existing Google call with those settings. Retain actual start/completion timestamps, final status, HTTP status and provider body, including operational failures.
  4. Store a dharma.external-native-capture/v1 and its pin in the authorized durable outbox. Call observer.captureNative(capture, pin) after finalization. Do not also call observe() for that logical request.
  5. Acknowledge delivery only after the returned receipt has sealed:true and a memberId. Retry unchanged packets and deterministic keys after transport interruption; do not regenerate the answer to retry upload.

The capture includes organization/policy/revision/endpoint IDs, request and pin IDs, model, timestamps, status, original instructions, the owned instruction span, native request and provider result. Instruction-span offsets are UTF-8 bytes, not JavaScript character positions. The server checks composition rather than accepting an uploader's claimed skill-adoption flag.

The serialized capture limit is 8 MiB, split into at most 32 chunks of 256 KiB. Each HTTP packet is at most 400,000 bytes including base64 overhead. Upload every kind:"chunk" packet, then a kind:"seal" packet with the same scope and hash but no chunk payload. The source modules provide prepareNativeCapture() and receipt verification; the OpenAPI supplies the wire schema. Partial uploads are not complete trajectories.

Native receipts retain provenance:"customer_attested" and providerExecutionVerified:false. This is authorized customer evidence, not independent proof that Google executed the request. Current consent, source hashes and retention are checked again before use; withdrawing evidence can stop pending work. Never replace a missing image with another photograph or reconstruct it from an answer.

Follow each 100-request window

Learning counts finalized logical production requests, not retry deliveries. Failed requests with usable receipts remain in the operational denominator. Synthetic fixtures and designated validation traffic do not fill the production learning batch. Separate real requests with identical content remain observable, but their content group must not cross learning, calibration and held-out boundaries.

A partial batch waits. A logical 100-member review covers every member, using recorded subcalls when necessary; it must not silently sample or truncate. Preflight failures, missing evidence, provider failures and rejected evaluator output remain distinct from semantic findings. Check the queue receipt rather than assuming an exact completion time from the hundredth request.

Use GET policy for progress and costs; GET windows for frozen membership; GET window detail with section=results, atlas, changes, candidates or receipts for retained content. Follow the section's returned cursor until exhausted. A page of 20 rows is not the whole window. Review counts are covered requests in accepted checkpoints, not provider attempts, checkpoint counts or passing outcomes. Missing/withdrawn content is an explicit limitation, not an empty successful Atlas.

The Atlas groups evidence-linked failures. An accepted checkpoint can propose additions, merges, corrections and retirement of families, plus a complete candidate skill or no change. Uploaded customer text is evidence, never privileged instructions. Store checkpoint lineage and hashes; do not silently revise prior results.

Improve the rubric without moving the goalposts

Atlas findings may propose a rubric or grader revision as well as a skill. A candidate cannot validate its own grading standard. Calibration compares proposed grading against a separate, signed reference set; acceptance applies to future eligible windows, not retrospectively to old scores or an in-flight candidate. Skill validation keeps its frozen release contract.

POST /{id}/calibration-references accepts expectedRevision, the exact UTF-8 signedManifest string and signatureBase64. Sign the exact bytes with an independently admitted Ed25519 authority. The request does not enroll a public key or allow its sender to invent a trusted signer.

The OpenAPI CalibrationReferenceManifest binds the organization, endpoint, policy, evaluation-contract ID/hash, consent, authority, issue/expiry times and exactly 100 distinct original requests. Each member binds original trajectory/revision and source, observation and request hashes. Labels cover source-support, history-semantics and evidence-coverage, each with at least 30 passes, 30 failures and one severe failure. Labels must be independently adjudicated or deterministic reference results, not the candidate grader's answers renamed as ground truth.

Reserve the reference cohort separately before learning or held-out assignment. The server rejects overlap and never moves already-used requests to calibration. Existing signed sets with invalid overlap are withheld before exposing labels. Reference evidence must already be retained and consent-bound; uploading a manifest does not upload missing images. External-native image calibration requires a sealed original capture: the reader verifies its original member revision, request-bound pin and bytes against the current consent and evaluation contract. Metadata-only uploads cannot supply missing images.

For a prospectively selected external reference request, obtain the original pin before generation, then use observer.captureCalibrationNative(capture, pin) instead of ordinary captureNative or observe. The uploader requires the separate opt-in learning:calibration-ingest scope; pin creation still needs learning:ingest and evals:read. Separate pinning and upload workers can use separate scoped tokens. Uploads use the same bounded native chunks and final seal. This server-to-server operation is available through HTTP and the source SDK, not a binary-upload MCP tool.

The seal derives content identity from verified bytes and atomically reserves the calibration lane. It returns evidenceRole:"calibration" and a reference containing trajectoryId, trajectoryRevision, sourceHash, observationHash and requestHash; chunks have reference:null. Preserve that receipt for the later independently signed manifest. Calibration requests do not fill learning windows, held-out skill validation, canary/control or monitoring counts. A content group already used in learning or validation returns HTTP 409, even with a new request ID. Never upload a reference through ordinary learning intake first. Sealing creates no labels, trusted signer, reference set or paid model call.

A successful receipt is {setId, manifestHash, replayed}. It proves admission, not correctness of labels, paid calibration or activation. Bind the set through a separate complete policy PATCH under calibrationSetId and the approved caps. Repeated invalid submissions must not be “fixed” by renaming cases, weakening labels or reusing held-out evidence.

Validate, release and verify adoption

The intended gated sequence is frozen candidate validation, signed 10% canary, verified activation and continuing monitoring. Failed candidates leave the previous skill active. Publication alone is not runtime adoption: an external service must verify the signed payload, organization/endpoint scope, hash, expiry and trusted key, pin the revision per request, and retain its actual use in the source evidence.

The initial release requirements are schema validity at least 95% without deterioration, operational failures no more than 5% without deterioration, mean improvement at least 0.05, no negative dimension delta, net wins at least 10 and zero severe regressions. Comparison requires 100 complete representative held-out requests with identical settings and blinded grading, with at least 30 comparable scores per dimension. Missing evidence waits; it is not replaced with unrelated requests or synthetic fixtures.

Canary acceptance requires at least 100 canary and 100 contemporaneous control requests, passing safety/reliability checks and zero detected severe grounding regressions. A severe regression or failed safety check triggers rollback; insufficient evidence after 24 hours reverts the canary to pending validation. Promotions are serialized per endpoint. These are release conditions, not a claim that your organization has passed them. Inspect the actual validation, signing, assignment, monitoring and rollback receipts.

Keep the last verified revision available. If skill delivery, signature verification or adoption proof fails, retain the known-good serving behavior and record the failure. Never label a downloaded or published skill “active” without runtime evidence.

Billing, recovery and acceptance checklist

Generation, evaluation, validation replay and storage are separate cost categories. Direct Google generation stays on your provider account; Dharma bills configured evaluation services separately. Every paid dispatch requires budget reservation. Settled charges plus outstanding and uncertain exposure count against caps. Budget exhaustion pauses learning or promotion, not ordinary serving. Do not retry an uncertain paid response as a fresh operation.

Successive controlled-study versions can share one administrator-bound lifetime budget group. Creating a new version does not reset this ceiling. The existing generation and evaluation ledgers atomically account for all attached versions; verified settlements can release surplus, while uncertain charges and conservative outside-ledger carryover remain reserved. An unexpected overage is retained and stops further study dispatches across the group. Ordinary customer serving keeps its existing billing policy.

The policy read's controlledProof.sharedBudget is null for an ungrouped proof. Otherwise it reports groupId, capMicrousd, conservativeExposureMicrousd, carryoverMicrousd, remainingMicrousd, reservationOverrun and studyVersions. Exposure is not a fully settled invoice. Group configuration and membership are immutable, tenant-bound operator controls, not authority to raise a customer's spending cap or promote a skill. A study must be paused before attaching it; no provider call is required for attachment.

For Gemini-compatible ingress, a failure after managed dispatch retains x-dharma-run-id and a truthful x-dharma-evaluation-status, with x-dharma-billing-status: requires_reconciliation. It does not invent a successful Gemini answer or a zero-charge receipt. Authentication and pre-dispatch failures expose no run receipt. Reconcile the retained run and provider attempts before considering regeneration: replaying the same idempotency key can return the same terminal failure, not a fresh provider attempt.

Native generation permits at most two retries after explicit retryable HTTP responses, using identical serialized evidence, model and decoding. It respects numeric and HTTP-date Retry-After values; if the requested wait cannot fit the original request deadline, it stops rather than retrying early. Transport timeouts remain uncertain and are not automatically regenerated. First-attempt and final outcomes must stay separate in benchmark denominators. Check the deployed runtime source receipt as well as the web/API commit before relying on this behavior.

  • 401/403: resolve identity or scope, without copying session cookies or widening an ingest token to admin.
  • 409: inspect the safe reason and correlation ID for revision, consent, pin, authority, conflict or paused-state conditions; blindly allocating a new key is not recovery.
  • 413: preserve the evidence and resolve the actual size limit; do not silently trim images or split one request into invented cases.
  • 422 on reference submission: fix invalid signature/scope, independent class coverage, unavailable original evidence or cohort overlap.
  • 429: respect allowance/rate-limit conditions with bounded retries and unchanged identities.
  • 503 or uncertain transport: preserve the original operation and receipts, reconcile before retry, and never log tokens or private evidence bodies.

Before client acceptance, verify ordinary member access, real request enrollment, full 100-member coverage, immutable held-out/calibration boundaries, calibrated grader behavior, passing candidate validation, canary/control evidence, signed runtime adoption, rollback, costs and visible results. Report incomplete phases separately. Software tests and deployed APIs do not substitute for those production receipts.