From Placement to Foundation: Designing AI for Production

Viable placement is not enough. Many AI deployments that should work, do not. The problem is never the AI model - it is the design work that precedes building.

Placement evaluation identifies where AI belongs. It does not determine what happens there.

What follows is a design framework that determines whether implementation has a foundation to build on.

The Problem

Many AI initiatives start with "Use AI to process insurance claims", "Use AI to replace agents in customer support", etc.

These are directions. They are not problems.

Teams that skip clear problem definition build in the wrong direction. Vague scope means shifting requirements, discarded features, ungrounded decisions. Production reveals the gap - teams cannot agree if it works, leadership questions ROI, funding gets pulled because no one can articulate what problem was solved.

A clear problem definition is complete, measurable, and bounded. It answers: what exactly needs solving, how much of it exists, what changes if solved, what limits the solution, what success means.

This ensures AI capabilities align with business objectives - serving defined outcomes, not showcasing impressive features for their own sake.

Here is what that looks like in practice:

Specific work: Insurance claims team receives 1,200 property damage claims monthly. Each requires initial review - reading submissions, checking policy coverage, categorizing damage, routing to appropriate adjusters. Takes ~15 minutes per claim.

Business problem: Processors spend 300 hours monthly ($12,000 in labor) on routine initial review that follows predictable patterns in 80% of cases, creating 2-3 day backlog before claims reach adjusters.

Impact: Extended claim resolution time. Customer frustration requiring status updates. Processor capacity consumed by categorization instead of complex edge cases requiring judgment.

Constraints: Regulatory compliance requires manual review for fraud indicators, high-value claims over $50k, and specific policy exclusions. Misrouting accuracy threshold - routing errors delay resolution further. Implementation must show ROI within 6 months.

Success criteria: Automate 80% of straightforward claims, reduce initial review from 2-3 days to same-day, maintain 90% routing accuracy, recover 200 processor-hours monthly.

Clear problem definition changes everything downstream. Teams can evaluate ROI before investing. Engineering knows what data matters, which models fit capability and cost requirements, what infrastructure handles the volume and latency constraints. Product can scope realistically. Leadership can recognize success or failure definitively. Every decision - from model selection to evaluation frameworks - has a clear anchor point. Without it, every decision is negotiable.

Most AI failures are not model failures - they are undefined-problem failures. When teams cannot clearly articulate what is being solved, how much of it exists, and what success looks like, performance becomes subjective. The AI is blamed for outcomes that were never structurally anchored.

The framework that creates this structure:

1. Identify the specific workflow or task. Not "customer support" but "password reset requests" or "initial claims review" or "subscription upgrade processing".
2. Quantify current state. Volume handled, time per task, labor cost, delays created. Numbers matter - they establish the baseline.
3. Specify business impact. Financial cost, operational bottleneck, customer friction. Connect the task to outcomes leadership cares about.
4. Document constraints. Regulatory requirements, accuracy thresholds, budget limits, timeline expectations. These bound what is viable.
5. Set measurable success criteria. What changes, by how much, when measured how. "Reduce initial review time from 2-3 days to same-day" not "improve efficiency".

This foundation enables solution design. AI's role, scope boundaries, architectural choices, evaluation frameworks - all build from these criteria.

The AI's Role

Teams often treat role design as an implementation artifact. It is not. It is a product decision.

The boundary between automation and augmentation determines accountability, risk exposure, operational burden, and long-term sustainability. If that boundary is unclear, engineering cannot compensate for it.

AI's role bridges problem and implementation - defining how AI participates in solving the problem, where boundaries exist, and what structure makes that participation viable.

Participation modes

Map the work to two categories based on structural characteristics.

Full automation: Routine, predictable patterns where AI's capabilities match task requirements. Low-stakes outcomes where errors are containable. Clear success criteria that AI can optimize toward. No consequence-grounded judgment required.

Augmentation: AI provides analysis, recommendations, synthesis. Humans own the decision. Use when judgment matters but AI can surface insights humans cannot generate at scale. Task requires weighing tradeoffs AI cannot fully model. Accountability must stay human.

For claims processing:

AI automates routine categorization and routing for 80% of straightforward claims - vehicle accidents, water damage, roof damage, etc. Predictable patterns, containable errors, clear success criteria.

AI augments complex multi-damage claims - multiple vehicles in accident, fire + water damage, etc. Provides category suggestions and flags ambiguity, but humans decide given judgment requirements.

High-value claims over $50k, fraud indicators, precedent-setting decisions start outside AI's scope.

HITL structure

Define where humans stay in the loop and why. Not just "AI escalates uncertainty" but explicit design of escalation triggers, handoff mechanics, and feedback loops.