Measure first; change weights last

The vendor proposal leads with custom model. Seven figures. Nine months. Slides full of GPUs. Your CMIO asks about clinical validation before Legal finishes reading the SOW. Meanwhile you cannot answer a simpler question: what is our baseline quality on the stack we have today? That gap is why Training strategy is last resort training: measure first, change weights only when cheaper layers fail.

Training is more than fine-tune

Executives hear Training and think weights. In the Eight Words, Training is the whole improvement loop: evaluation, monitoring, feedback, and sometimes fine-tune or distillation. Buying fine-tune without eval is fine-tune theater: high ACV, thin accountability.

The investment sequence

Fund in dependency order:

1. Gateway: logs, routing, spend visibility 2. System Message: executable policy 3. Knowledge: governed corpora with citations 4. Model tiers: portfolio routing 5. Orchestration, Tools, Interface: how work runs 6. Training: measure, then improve weights only when the list above is exhausted

Skipping to step six is how organizations buy the wrong science project.

When fine-tune is actually on the table

Narrow cases: stable task, proprietary labeled data, retrieval and prompting exhausted, change-control accepted.

Examples: coding assistance on note templates with clinician-labeled edits; classification on an internal taxonomy with thousands of verified labels.

"We have a lot of PDFs" is a data hoarding story. Unique data with unique governance is different.

Baseline before weights

Baseline before weights is the rule your delegate enforces: golden questions, weekly sampling, rubric owned by quality and clinical informatics. The eval loop your quality team runs belongs after that baseline exists—Strategy freezes Training CAPEX until the score exists.

Counterargument: "We have unique data." Do you have maintained labels, owners, and validation protocol?

Healthcare adds cost: model version change triggers clinical governance. Treat weight updates like device software: intentional, reviewed, reversible.

The expensive mistake is asking whether to buy a custom model before anyone scores today's stack. Measure the system, then move the weights.