Start with the use case
Define the user, workflow, decision, or business problem first. The model and architecture should follow the problem rather than become the starting point.
AI & Data Perspective
Designing AI systems forproduction, not just prototypes.
A practical perspective on what changes when an AI experiment becomes a real software system that people and organizations need to rely on.
Key Points
Build on trusted data
AI quality depends heavily on the information available to the system, how that information is prepared, and whether its provenance and access are understood.
Evaluate continuously
Teams need repeatable ways to assess quality, failure modes, retrieval, latency, cost, and other behavior as models, prompts, data, and applications change.
Design for operation
Security, observability, deployment, incident handling, ownership, and lifecycle management should be considered before an AI capability becomes critical.
01 · Use Case
The model should not define the problem.
Define the user
Understand who interacts with the system and what knowledge, permissions, expectations, and responsibilities they bring.
Define the task
Identify the specific workflow or decision the AI capability is expected to support rather than beginning with a general-purpose assistant.
Define acceptable failure
A brainstorming tool and a system supporting a critical operational workflow require very different levels of control and validation.
Define success
Agree on observable measures of usefulness before optimizing prompts, models, or infrastructure.
02 · Data
AI quality starts before the model.
Source quality
Understand which sources are authoritative, current, complete, and appropriate for the intended use case.
Retrieval quality
Chunking, indexing, metadata, embeddings, ranking, filtering, and permissions all influence what context reaches the model.
Access controls
The system should respect the same information boundaries that apply to the users and systems around it.
Data lifecycle
Plan how new, changed, or removed information propagates through the AI system over time.
03 · Evaluation
If quality cannot be measured, it cannot be managed.
Task quality
Measure whether the system performs the actual task well enough for the intended user and operating context.
Retrieval evaluation
Check whether the right supporting information is being found before evaluating what the model does with it.
Failure analysis
Capture recurring failure modes and use them to improve prompts, data, retrieval, guardrails, or product design.
Regression testing
Re-run representative evaluations when models, prompts, retrieval logic, data, or application behavior changes.
04 · Architecture
Keep AI behind clear system boundaries.
Explicit AI service boundaries
Centralize model interaction, prompt logic, retrieval, policy, and related behavior behind maintainable application interfaces.
Model flexibility
Avoid architecture that assumes one model provider will always be the correct choice for every workload.
Fallback behavior
Define how the application behaves when models fail, time out, exceed cost limits, or produce output that cannot be trusted.
Deterministic controls
Use conventional software for permissions, validation, transactions, workflow state, and other behavior that should remain deterministic.
05 · Operations
Production AI needs observability beyond infrastructure health.
Technical observability
Monitor latency, availability, errors, dependencies, resource use, queues, and application health.
AI observability
Track model usage, retrieval behavior, evaluation outcomes, prompt versions, failure patterns, and other use-case-specific signals.
Cost visibility
Measure the cost of model calls, embeddings, storage, retrieval, infrastructure, and supporting services relative to the value of the workflow.
Operational ownership
Define who investigates quality issues, changes prompts or models, maintains evaluations, and approves production changes.
06 · Security
AI introduces new risks without replacing existing ones.
Least privilege
Give AI-enabled services and tools only the permissions required for their intended workflows.
Untrusted inputs
Treat user prompts, retrieved content, uploaded documents, and external information as potentially untrusted.
Sensitive information
Understand what data is sent to models, logs, vector stores, external services, and downstream tools.
Controlled actions
Use explicit authorization and deterministic validation when AI systems can trigger external actions or modify important data.
Practical Takeaway
The prototype is the beginning, not the architecture.
Moving AI into production requires engineering discipline across the whole system. The strongest implementations treat AI as a capability embedded within a dependable software, data, security, and operational foundation.
Begin with a specific workflow or business problem, not a preferred model or AI technique.
Invest in data quality, retrieval, access controls, and source understanding before assuming the model is the primary quality problem.
Build evaluation into development so changes to prompts, models, data, and retrieval can be assessed systematically.
Keep probabilistic AI behavior behind clear application boundaries and retain deterministic controls where certainty is required.
Monitor AI quality, cost, latency, retrieval, and failure behavior alongside traditional application and infrastructure health.
Design for ownership: production AI needs people, processes, documentation, and operational responsibility after launch.
Related
Continue exploring the topic.
AI & Data
Moving an AI use case beyond the prototype?
We can help evaluate the use case, data foundations, architecture, evaluation strategy, security, and operational capabilities needed to turn experimentation into a dependable production system.