Teaching an AI appraisal system to learn from expert corrections.

THE STUDY

Start with the work the app actually does.

The platform supplied 100 labeled appraisal requests with 377 photographs. Each item had an expert price and reasoning, plus labels for era, authenticity, and category. A separate set of 20 appraisals was reserved for blind comparison; the platform kept those expert answers.

We preserved image order and the platform’s original v190 instructions. Baseline and later skill-assisted appraisals used Gemini 3.8 Flash at temperature zero with medium thinking. The study ran through direct Google Vertex as a BYOK experiment. It did not exercise automatic enrollment or skill delivery in Dharma’s managed runtime.

Those boundaries matter. The supplied labels give this learning cycle a meaningful target. They do not turn a repeated comparison on the same 100 items into an independent test.

FROM ERROR TO PROCEDURE

What did the expert know that the model missed?

Dharma reviewed the 100 baseline appraisals in ten batches, comparing each answer with the expert’s value and reasoning. The Failure Atlas organized recurring problems: unsupported premiums for famous graphics, heavy discounts for missing tags, weak distinctions between original and later variants, and prices pushed down to a scrap floor for wearable garments.

Those findings informed a procedural skill. It asks the model to corroborate era with construction and tag evidence, distinguish the exact issue from a later printing, check how damage and sizing affect demand, and keep modeled price history distinct from documented sales. The skill does not contain an answer table, record identifiers, or memorized prices.

We generated three evaluated candidates and compared them on the labeled items. The third version gave the strongest eligible local score and produced a usable price on all 100 requests. A fourth version tightened discount rules but scored lower, so it was not selected. Every choice in this stage used the learning labels; that is why the result below needs a separate test.

ILLUSTRATIVE CASES

Small changes in reasoning can move a price a long way.

These four garments illustrate what the combined skill changed. They are selected cases from the learning set, including a regression. The expert prices are the platform’s labels, not independently verified sale prices. No single example proves which instruction caused a price change.

READING THE RESULT

The gain has two parts.

The provisional operational price score rose from 70.18% to 75.79% across all 100 learning items, a gain of 5.61 percentage points. An invalid or absent price counts as zero. The original arm produced usable prices for 94 items; the learned arm did so for all 100.

Recovering those six missing prices contributed 3.65 points of the total increase. On the 94 items where both arms returned a usable price, the average paired gain was 2.08 points. Looking at all 100 pairs, 48 improved, 22 tied, and 30 got worse.

Other diagnostics moved too: category matches rose from 79 to 91, era matches from 81 to 82, and authenticity matches from 93 to 99. The authenticity labels were overwhelmingly one class, so the last number does not establish reliable reproduction detection. The score reflects expert estimates, not independently observed sales.

THE INDEPENDENT CHECK

The 20-item test is frozen and awaiting scores.

Before the blind requests ran, we fixed the selected skill, model settings, output schema, and analysis procedure. Each blind item then received two fresh answers: original v190 and original v190 with the learned skill. The original produced 19 schema-valid answers; the learned version produced 20. The platform holds the expert prices, so we cannot yet calculate the blind pricing difference or claim its target of at least two percentage points.

That customer score is the decision point. If the learned answer improves the customer’s mean pricing score without unacceptable regressions, the next step is a staged release with signed revision pins, expert feedback, a small canary against contemporaneous control, and rollback rules. No customer production skill or monitored canary has been changed by this study.

FOR APPRAISAL BUILDERS

Make corrections usable beyond one item.

An appraisal company already owns the most valuable part of this process: cases where a human expert can say what the model got wrong and why. The practical starting package is representative inputs, complete images or other evidence, the current output contract, expert answers, and the exact scoring rule used to judge business value.

Dharma can organize recurring errors into an Atlas and propose a procedural skill for the same model. Separate cases then test whether the skill improves new appraisals, preserves useful output, and avoids costly mistakes. Once connected to a production endpoint, the same approach can review incoming trajectories in batches and propose later revisions; expert labels remain necessary for claims about pricing accuracy.

The method could apply to watches, collectibles, equipment, property, or any product that turns evidence into an estimate. This study tested garments. Each other category needs its own expert examples, evidence rules, and scoring standard.