Designing an AI Project from Use Case to Architecture

When I start an AI project, I want to answer three questions: what work are we improving, where is the uncertainty, and what would count as a useful result? Those answers give me a much better starting point than a model name or an architecture diagram.

A request to "add AI" might mean finding information, interpreting a document, predicting an outcome, generating a draft, or carrying out a sequence of actions. Each calls for different data, different components, and different evidence that the system works.

This issue walks through the design process I would use across those projects: gather the operating context, translate requirements into capabilities, choose the system boundaries, and prove the smallest useful slice. The repository contains reusable design documents and several short requirement examples.

Start With the Work and Its Current Baseline

The first conversation should be with the person who does the work and the person accountable for its outcome. Ask them to walk through a real case. What starts the process? What information do they use? Where do they make a judgment? What happens after their decision?

I want to see actual inputs, useful outputs, and examples of failure. "Answer questions about company policy" becomes more specific when we know who is asking, which sources they can access, how often those sources change, and whether an unsupported answer is worse than asking for help.

Record the existing baseline too: manual work, a search page, a SQL query, a rules engine, or an existing predictive model. That gives the project something to improve on. If ordinary software already meets the requirement, it can remain the solution for that part of the system.

What I Gather Before Choosing Components

A useful discovery pack should let another engineer understand the operating conditions without sitting through every meeting. I would gather:

• The workflow: users, triggers, current steps, decisions, handoffs, and the owner of the final outcome.
• Representative cases: real inputs, expected outputs, ambiguous requests, common errors, and expensive mistakes.
• The data: sources, formats, owners, access rights, labels, quality, update frequency, and what is available at decision time.
• The constraints: latency, volume, concurrency, cost per task, deployment environment, residency, and retention.
• The authority: what the system can read, suggest, change, or execute, and where a person must remain involved.
• The success criteria: the baseline, task-quality measures, acceptable failure behavior, and who signs off on the first release.

Unknowns belong in the documents with an owner and a way to resolve them. A missing labelled dataset can become a discovery task. Unclear access rights can block a proposed data source. Neither should disappear behind a box labelled "AI service."

Map Requirements to Areas of AI Engineering

This is where the architecture starts to take shape. For each requirement, identify the capability that is missing, then choose candidate techniques and components. The table below is a starting map; a project may combine several rows.

Some choices need another question. Exact account balances usually call for an authorized query against the system of record. Finding relevant passages may call for lexical search, semantic search, or a combination. Needing knowledge does not automatically mean needing embeddings, and needing several steps does not automatically mean needing an agent.

For agentic work, Anthropic's workflow and agent distinction is useful: predefined paths and model-directed next steps have different control needs. Start with the simpler candidate and test whether it meets the task.

Define What Each Component Owns

Once I know which capabilities the project needs, I decide what each part of the application owns. Choosing retrieval, for example, still leaves us with decisions about who selects the sources, applies access rules, builds the model context, and handles missing evidence.

Start by giving each responsibility a clear boundary. The contract describes the agreement between that component and the rest of the application:

• Input: what data and caller context it accepts.
• Output: what result the next step can expect.
• Failure: what it returns when it cannot complete the work.
• Authority: what state or external systems it is allowed to change.

Take the requirement to answer questions from internal documents. Retrieval should return passages the caller is permitted to read, together with their source IDs. Context assembly should preserve those references when preparing the model input. The answering component should return a cited answer or an explicit insufficient-evidence result. Each handoff now has a defined purpose and expected behavior.

These responsibilities can begin as modules in one application. What matters at this stage is knowing where a decision belongs and who owns changes to it. That gives us a concrete flow to draw, rather than a collection of technology names.

Those agreements also tell us what to check next. We can test access filtering and source references with fixed inputs, then evaluate whether the complete flow produces useful, supported answers. That leads directly into the quality and operational requirements of the design.

Design Quality and Operations Alongside the Flow

Before building the first slice, decide what evidence would justify keeping the design. For a classifier, that may involve precision and recall on important classes. For retrieval, it includes whether the right evidence is found and used. For generated work, it includes usefulness and unsupported claims. For a predictor, it includes performance against a sensible baseline on data unavailable during training.

Keep deterministic contract tests separate from model-quality evaluations. Protect evaluation examples from tuning, and preserve time order when the task requires it. The scikit-learn guidance on leakage explains why preprocessing and training must not learn from test data.

Then ask what happens under load or failure. Can a request wait in a queue? What is the total deadline? Which operations are safe to retry? What may be logged? Who can stop the feature or roll back a model, prompt, dataset, or index? Quality, latency, spend, permissions, and recovery are architecture inputs. Microsoft's AI workload design principles place these concerns within the wider system design.

System Structure Starts With a Candidate Design

At this point, we have enough to draw a candidate flow. Include the data dependencies, component responsibilities, failure exits, state changes, and action authority. The process is iterative:

This is the design process, rather than a runtime stack every project must deploy. A document intake service may use a fixed extraction pipeline, as in Issue #49. The research task in Issue #50 needed an adaptive investigation loop. A historical prediction task would add dataset, training, validation, and serving concerns. The requirements explain those differences.

Let Responsibilities Shape the Repository

I would organize the project after identifying those responsibilities. For a small .NET application, this is a proposed starting shape:

.
+-- AiProject.slnx
+-- src/
|   +-- AiProject/
|       +-- Program.cs
|       +-- appsettings.json
|       +-- Contracts/
|       +-- Application/
|       +-- Capabilities/
|       +-- Policies/
|       +-- Infrastructure/
+-- tests/
|   +-- AiProject.Tests/
+-- evals/
+-- docs/
  +-- design/

Application/ coordinates the use case. Capabilities/ contains the inference, retrieval, extraction, or tool boundaries the project actually needs. Policies/ owns application decisions and authority. Infrastructure/ owns external services and storage. Tests check software behavior; evals measure task quality on representative cases.

This is a layout proposal, not a supplied application skeleton. A Python project centred on training or data processing will have a different package structure. The same ownership decisions still apply. Split deployable services when scaling, access isolation, ownership, or release cadence requires it.

What You Are Building

The repository makes this process reusable. Start with the workflow, requirements reference, and conditional component catalogue, then fill out six project documents:

.
+-- START_HERE.md
+-- REQUIREMENTS_MAP.md
+-- COMPONENT_CATALOG.md
+-- templates/
|   +-- PROJECT_BRIEF.md
|   +-- REQUIREMENTS_MAP.md
|   +-- ARCHITECTURE.md
|   +-- CONTRACTS.md
|   +-- EVALUATION_PLAN.md
|   +-- FIRST_SLICE.md
+-- examples/
  +-- requirement-cases.md

The examples compare knowledge lookup, document extraction, research, and scoring requirements. They are design exercises with references to existing companion implementations. The documents are the deliverable: a reviewable set of decisions that a team can use before implementation.

Prove the Smallest Useful Slice

The first slice should complete one real task through the chosen boundaries, including a success case, an insufficient-evidence or invalid-input case, and a dependency failure. It should produce something you can inspect and compare with the baseline.

For a search assistant, that might be one authorized question through retrieval to a supported answer or abstention. For an extractor, one file through evidence checks to a saved decision. For a predictor, one repeatable dataset split through a baseline and held-out evaluation before exposing a prediction endpoint.

A useful first slice resolves the biggest uncertainty in the design. If the data cannot support the task, more orchestration will not fix it. If the model adds little over the baseline, simplify. If quality is promising but latency or review effort is unacceptable, those measurements tell us what to change next.

Why This Architecture Process Works

Every component can be traced to a requirement, and every important requirement has evidence attached to it. We can explain why retrieval belongs in one project, why another needs a feature pipeline, and why a third benefits from an agent loop.

The architecture remains open to revision. Record the chosen option, the reason, the cost accepted, and what would make you reconsider it. That is more useful than treating the first diagram as a permanent commitment.

Potential Enhancements

As the first slice produces evidence, add the next boundary it justifies: stronger retrieval, improved training data, a review workflow, durable execution, concurrency control, or release automation. Update the requirement, contract, and evaluation case together.

I would also record a reason for leaving a component out. "No persistent user context is required" is a useful memory decision. "The task follows a known sequence" is a useful orchestration decision. That keeps the project small while preserving the reasoning for the next engineer.

Final Notes

This is how I would approach the architecture of an AI project: understand the work, inspect the data, identify the missing capabilities, and connect each design decision to evidence.

You should finish with a clear explanation of the first version: who it helps, what it needs, what the AI is responsible for, what the application controls, and how we will know whether it is useful.

Explore the repository at the GitHub repository.

See you in the next issue.

Stay curious.