In this guide
  1. Who this is for
  2. Why AI products fail
  3. What is Build vs. Cut?
  4. The Architecture Ladder
  5. What a V1 looks like
  6. AI and automatic actions
  7. Proving it works
  8. Arabic AI products
  9. FAQ

Most founders come to an AI project with the same first question: what should we build — a chatbot, an agent, something with RAG? It feels like the right question. It isn't. It's a technology question asked before the product question, and it's how teams end up six months into an impressive system nobody trusts, nobody can debug, and nobody needed.

The better first question is: what is the simplest AI product that can pass a real test?

That question is the heart of Build vs. Cut — Sprint966's method for scoping AI products. Sprint966 is an AI Product Studio for Saudi startup founders. We help founders scope, design, build, launch, and improve focused AI products without overbuilding, hiring a full tech team, or wasting months on the wrong product. Our promise: build the right AI product first. Most of the risk in an AI product is decided before the first line of code — in what you choose to build, and just as importantly, in what you choose to cut.

This guide is the full method: why AI products go wrong at the scoping stage, the eight rungs of the Sprint966 Architecture Ladder, what a real V1 looks like, how to handle risky actions, and how to prove — with tests, not vibes — that a product works.

Not sure where your idea stands?
Check Your Product Scope → — a fixed-scope way to find out before you commit to a build.

Who this guide is for

This guide is for Saudi startup founders who are deciding whether to build an AI product — or who have already started and want to know whether they're building the right thing. You don't need a technical background to use it. You do need to be willing to define the job before choosing the technology.

It's also useful if you're evaluating agencies or AI studios and want to understand what questions a credible partner should be asking before they recommend a stack.

Why do most AI products fail before the model matters?

In Sprint966's experience, many AI product failures start as scoping failures, not model failures. The model was never the problem. The problem was that a stochastic AI component was placed where plain business logic would have worked, or an autonomous system was built for a process nobody had mapped, or a product shipped with no test that could tell anyone whether it was getting better or worse. That is our view from practice — and it is consistent with how the most careful builders and standards bodies approach the problem.

NIST — the US standards body — frames generative AI risk as something managed across the whole AI lifecycle: design, development, deployment, operation, and beyond. Risk is not a property of a model you pick once; it's a property of the system you run continuously.1

Anthropic's engineering guidance points the same direction: find the simplest solution possible, and add complexity only when the simpler design demonstrably falls short. Many applications are well served by a single optimized model call with retrieval and good examples — no agent required.5

Founder takeaway: don't start by picking a model. Start by defining the job, the risk, and the test. The technology choice falls out of those three — almost never the other way around.

What is Build vs. Cut?

Build vs. Cut — in one sentence

Build the simplest AI architecture that can pass a real-world test, and cut every layer of complexity, autonomy, and automation that has not yet earned its place through measured results.

Build vs. Cut treats AI product decisions as evidence decisions, not ambition decisions. Before anything is built, the task, the success criteria, and the acceptable error are defined and frozen into a test set drawn from real work. The product starts on the lowest rung of the architecture ladder that could plausibly pass — rules before prompts, prompts before retrieval, workflows before agents — and only climbs a rung when testing shows the simpler design genuinely failing. Anything risky or irreversible stays behind human approval, and every AI step is instrumented from day one so failures can be replayed, priced, and fixed. The result is a focused product that works, instead of an impressive system that can't be trusted, debugged, or afforded.

The five steps

  1. 1
    Scope Check.

    Define the job in one sentence: who uses it, what "correct" means, and what a wrong answer or wrong action would cost. Decide the acceptable error before choosing any technology. Sort the product into what must stay deterministic — pricing, permissions, eligibility — and what genuinely benefits from AI: understanding, drafting, extracting.

  2. 2
    Freeze the evidence.

    Collect real examples of the task from support logs, documents, or by running the work manually and logging everything. This becomes the golden dataset: the exam every version of the product must pass. If the process isn't understood well enough to build this set, run it manually first. That's not a delay — it's a build step.

  3. 3
    Build the lowest rung that can pass.

    Start at the simplest rung of the Architecture Ladder that could plausibly pass the exam. Add retrieval only if the exam proves the failure is missing knowledge. Choose a workflow with bounded AI steps over an agent whenever the process can be drawn in advance.

  4. 4
    Gate the risk.

    The AI proposes; deterministic checks and humans approve. Every irreversible action — money, records, outbound communication — sits behind an approval gate, and tools get the narrowest permissions that work.3

  5. 5
    Instrument, measure, and only then escalate.

    Ship V1 with tracing — inputs, evidence, model version, cost, latency, outcome — and the golden-dataset score as a regression gate. Climb to the next rung, or widen autonomy, only when measurement shows the current design failing on the metric that matters.

Steps 1 and 2 are what we've productized as the AI Product Scope Check — a fixed-scope engagement that ends with your golden dataset, your starting rung, your risk posture, and your pass bar in writing. Steps 3 to 5 are delivery.

A Saudi lens on the method

Build vs. Cut is a Sprint966 product-scoping method, not a SDAIA requirement. Its practical value is that it puts the job, the risk, the human-approval points, the evidence bar, and the monitoring plan in writing before the product expands. That discipline supports the priorities reflected in SDAIA's AI Ethics Principles: accountability and human oversight, reliability and safety, testing before deployment, continuous monitoring afterward, and stronger controls where an outcome is difficult to reverse.17 SDAIA does not mandate this framework or a particular product architecture.

What is the Sprint966 Architecture Ladder?

The Sprint966 Architecture Ladder places every AI product idea at the right level of complexity. Eight rungs, ordered from simplest to most complex.

The ladder rule

Every rung adds capability — and also adds cost, failure modes, and evaluation burden. You climb only when the rung below fails a real test. Never because a higher rung sounds more impressive.

# Name Fits when Cut when Biggest risk
1 Smart rules Logic is stable and expressible as conditions Exceptions overwhelm the rules Stale rules, edge cases
2 Prompt-based app Any language task — always prototype here first Exact determinism is required Silent output drift
3 AI Q&A Most questions map to a bounded knowledge set Real need is process execution or large document reasoning Expectation mismatch
4 RAG / document AI Knowledge is private, changing, large, or needs citations Small stable corpus; task is action not answering Retrieval failure
5 AI workflow automation Process is known and multi-step; one or two steps need AI No real process — just one language task Missing logs and approval gates
6 AI agents Steps can't be predicted; tools required; success verifiable You can flowchart it Compounding errors; security surface
7 Multi-agent systems Single-agent traces prove a decomposition bottleneck V1, or chosen because it sounds impressive Coordination failures; cost blowup
8 Fine-tuning Measured gap that prompts and retrieval can't close Missing knowledge, unclear scope Maintenance burden masking a scope problem

01 Smart rules

What it is
Plain software logic — if this, then that. No AI in the decision.
Build it when
The logic is stable and you could write it on a whiteboard — eligibility, pricing, routing, validations.
Cut it when
Inputs are messy human language and exceptions overwhelm the rules.
Biggest risk
Edge cases and stale rules — but never a fabricated answer.
Minimum viable version
One workflow, fixed rules, a manual queue for exceptions.

If part of your product is an if/then, keep it as code. Deterministic logic can be tested with ordinary software methods, and replacing it with a model call trades that certainty for nothing.

02 Prompt-based app

What it is
One AI call with careful instructions — rewrite, summarize, extract, classify, draft.
Build it when
Always first, for any language task. It's the cheapest way to learn whether AI can already do the job. Anthropic's guidance makes the same point: many applications are adequately served by a single well-optimized call.5
Cut it when
You need exact determinism, or answers that are provably grounded in your own documents.
Biggest risk
Silent drift — outputs change when the model or prompt changes, with no code change anywhere.
Minimum viable version
One prompt, one output parser, one test set, one human fallback.

03 AI Q&A

What it is
A system that answers questions from a bounded knowledge set — an FAQ with language understanding.
Build it when
Most user questions map to a few hundred known answers — support deflection, policy lookup.
Cut it when
The real need is executing a process, or reasoning over large, changing document sets.
Biggest risk
Expectation mismatch. A chat box implies general intelligence the system doesn't have.
Minimum viable version
FAQ matching, templated answers, and a designed "I don't know — here's a human" path.

Abstention is a feature, not a failure. A bounded product that knows its limits earns more trust than a broad one that guesses.

04 RAG / document AI

What it is
The AI looks up your documents before answering and shows its sources. RAG — retrieval-augmented generation — can improve grounding and reduce fabrication risk when retrieval quality is strong.12
Build it when
Your knowledge is private, changing, too big for one prompt, or answers must cite sources.
Cut it when
The corpus is small and stable — long context may be enough. Note that long context does not universally replace retrieval; the right choice depends on the model, the corpus, and the task, and models can under-use information buried in the middle of very long inputs.13 6
Biggest risk
Retrieval failure disguised as "the AI is dumb." Retrieval quality is typically the bottleneck — wrong chunks, stale indexes, missing keyword search — which is why retrieval and answer quality should be measured separately.9
Minimum viable version
A small curated corpus, simple chunking, top-k retrieval, cited answers, and an abstention path. When you upgrade: combine keyword and semantic search before anything exotic — the two fail in different ways — and reranking can improve results further, though published percentage gains are benchmark-specific and worth re-testing on your own corpus.6

05 AI workflow automation

What it is
Your business process runs as code, step by step, with AI doing bounded jobs inside it — classify this, extract that, draft this — and humans approving what matters.
Build it when
The process is known and multi-step, and one or two steps need language understanding. For most operations-shaped products, this is the rung we recommend founders start from — and it's what we typically build.
Cut it when
There is no real process — just one language task. That's Rung 2.
Biggest risk
Skipping approval gates and logging, then discovering you can't reconstruct why a bad outcome happened.
Minimum viable version
One deterministic flow, one AI step, human approval on the consequential action, and a log of every step.

Not sure which rung your idea sits on?
Check Your Product Scope → — you'll know in days, not months.

06 AI agents

What it is
An AI that decides its own steps — which tools to use, in what order, and when it's done. Anthropic draws the line by control structure: workflows orchestrate models and tools through predefined code paths, while agents dynamically direct their own process and tool use.5
Build it when
The steps genuinely can't be predicted in advance, external tools are required, success can be verified from real system state, and the business can tolerate the trade-offs — agentic systems often trade latency and cost for flexibility, and they're harder to debug.5
Cut it when
You can flowchart the process. If you can draw the steps, it's a workflow, not an agent.
Biggest risk
Compounding multi-step errors — and a security surface where a malicious document the AI reads can become a harmful action, not just a wrong sentence.3
Minimum viable version
One agent, two or three narrowly scoped tools, strict step limits, a sandbox, and heavy logging.

07 Multi-agent systems

What it is
Several specialized AIs coordinating on one job.
Build it when
Almost never in V1. Current evidence suggests multi-agent systems can help on some decomposable tasks, but gains are task-specific and can reverse on sequential or coordination-heavy work — and every added agent adds cost, latency, and places for failures to hide.
Cut it when
It's being chosen because it sounds advanced. That is the most common reason it gets chosen, and it's not a good one.
Biggest risk
Coordination failures you can't debug, and token costs that quietly break your unit economics.
Minimum viable version
Two roles — say, a planner and a reviewer — benchmarked against the single-agent baseline on the same test set. If it doesn't beat the baseline, it doesn't ship.

08 Fine-tuning / custom model

What it is
Training the model itself on your examples so a specific behavior or format becomes built-in.
Build it when
You have a repeated, measured gap that prompting and retrieval can't close — exact output formats, domain-specific classification, instruction-following fixes, or cutting cost by distilling to a smaller model. Fine-tuning usually comes after evals and a prompting/retrieval baseline: OpenAI's guidance says to set up evaluations first, and reports common gains in the 50–100 high-quality example range.7
Cut it when
The real problem is missing knowledge, unclear instructions, or unclear scope. Fine-tuning is usually better suited to shaping behavior, format, and instruction-following than to adding knowledge — and it cannot repair a scoping mistake.
Biggest risk
Buying a permanent maintenance burden — datasets, versions, retraining — to solve a problem a week of prompting would have fixed.
Minimum viable version
One labeled dataset, a baseline comparison, one narrow task.

What does an AI product V1 actually look like?

Here's the method applied to a problem shape we see often: a delivery operations team drowning in exceptions.

The problem

A last-mile logistics startup handles failed deliveries, wrong addresses, and damaged goods across WhatsApp threads and spreadsheets. Every exception is handled from scratch, by whoever sees it first. Customers wait; nothing is consistent; nobody can say how many exceptions happened last month or how they were resolved.

The founder's opening ask: "We want an AI agent that handles delivery problems."

The Scope Check answer: this is Rung 5 — AI workflow automation. The process is knowable; it just hasn't been mapped. The steps can be drawn in advance, so it's a workflow, not an agent.

What we'd build first

  • One deterministic exception flow in code, covering the top exception types.
  • An AI step that classifies the exception and extracts key fields.
  • An AI step that drafts the customer message.
  • A human who approves the consequential action before it happens.
  • A log of every step: what came in, what the AI decided, what it cost, what a person approved.

What we'd cut from V1

  • Autonomous compensation or refund decisions.
  • A multi-agent "dispatcher."
  • Coverage of every rare exception type — the long tail goes to the manual queue.
  • Integrations not needed to run the test.

Risk posture: human-approved automation

The AI does the reading, sorting, and drafting. A person presses the button.

This is what Build vs. Cut does: it turns "build an AI agent" into a focused V1 that can be tested — and produces a written list of everything deliberately left out, with the evidence bar each cut item must clear before it earns its way in.

Should AI take actions automatically?

Sprint966's default risk posture

Human-approved automation: the AI does the work; a person presses the button.

The AI can draft, classify, extract, route, and recommend — that's most of the value. But humans approve anything irreversible or high-risk. This isn't caution for its own sake. OWASP, the organization behind widely used application-security guidance, recommends the principle of least privilege for AI systems and explicitly advises requiring human approval for high-risk actions — because prompt injection, where malicious text tricks a model into ignoring its instructions, is a built-in application-layer risk that current defenses mitigate but do not fully solve.3 4

One honest caveat: adding a human is not automatically a fix. Research on human-AI decision-making shows reviewers can over-trust AI outputs and approve wrong answers — and whether review actually works depends on how the review step is designed.14 15 16 A real approval step needs checklists, sampled audits, and clear criteria — not a glance and a click.

What stays human-approved in every Sprint966 build

  • Moving money in any form — refunds, payouts, pricing changes.
  • Legal, contractual, or regulatory communication.
  • Permanent changes to records.
  • Personnel and HR decisions.
  • Any new action type, until its approval history proves it reliable.

Autonomy is then earned action by action: when the approval log shows a person almost never rejects a specific action, that specific action — and only that one — becomes a candidate for automation.

How do you know an AI product works?

You know because it passes a test that existed before the product did.

The golden dataset

A golden dataset is your product's driving test. Before building, we collect real examples of the job — real questions, real documents, real exceptions — with a written correct result for each. In practice, 50–100 well-chosen examples is a useful starting range for a first product, though the right size depends on the task. Every version of the product takes the same exam. Score goes up, the change ships; score goes down, it doesn't. OpenAI's guidance reflects the same discipline: set up evaluations first, and measure against them before reaching for heavier tools.8

Before launch

Measure against the golden dataset — several runs, not one, because AI systems vary between runs. Measure the parts separately, because a single "accuracy" number hides where the failure lives. For RAG products, retrieval quality is measured separately from answer quality; retrieval is often the bottleneck.9 For workflows and agents, score both the steps and the final outcome.

After launch

The measurement continues: sampled real traffic against the same rubric, plus cost per task, latency, failure and override rates, and behavior across model or prompt changes. Every AI step is traced — inputs, outputs, evidence, model version, cost, outcome — so any incident can be replayed and fixed. This kind of trace-level observability is standard implementation guidance across major AI platforms.10 11 NIST likewise treats ongoing, post-deployment monitoring as part of managing generative AI risk.1

The Sprint966 starting bar

There is no industry pass mark — NIST is explicit that it does not prescribe universal risk tolerances.2 Every Sprint966 build ships with a product-specific pass bar, set in writing before launch, based on two things: what a wrong output costs, and how reversible it is. The bar is yours, it's explicit, and it's on paper before we build.

What is different about building Arabic AI products?

Many common AI testing examples, benchmarks, and implementation guides are easier to find in English than in the exact Arabic, bilingual, and dialect context your product may face. When your product lives in Arabic, and especially in real dialect rather than textbook Arabic, you inherit fewer ready-made tests for your exact context.

Sprint966's answer is practice, not claims: we build a client-specific Arabic golden dataset for every engagement. Real customer questions in the dialect your customers actually write. Real bilingual documents — because Saudi business runs in Arabic and English at once, often in the same PDF. Real edge cases: mixed-script product codes, right-to-left formatting quirks, the way a frustrated customer actually phrases a complaint on WhatsApp.

We don't claim proprietary Arabic benchmarks, and we won't tell you Arabic AI is generally good or bad — sweeping claims like that aren't testable and don't help you ship. What we will do is make sure your product is tested against your Arabic, before launch and after, to the same written standard as everything else we build.

For Saudi founders, this is the practical difference between a demo that impressed the room in English and a product that holds up in front of your actual customers.

Frequently asked questions

What is the Build vs. Cut method?
Build vs. Cut is Sprint966's method for scoping AI products: build the simplest AI architecture that can pass a real-world test, and cut every layer of complexity, autonomy, and automation that hasn't yet earned its place through measured results. It runs in five steps — Scope Check, freeze the evidence, build the lowest rung that can pass, gate the risk, and instrument, measure, and only then escalate. The method exists because, in our experience, most AI product risk is decided at scoping time, before any code is written.
What is the Sprint966 Architecture Ladder?
The Sprint966 Architecture Ladder is an eight-rung framework for placing an AI product idea at the right level of complexity: smart rules, prompt-based app, AI Q&A, RAG / document AI, AI workflow automation, AI agents, multi-agent systems, and fine-tuning. The ladder rule: every rung adds capability and also adds cost, failure modes, and evaluation burden — so you climb only when the rung below fails a real test.
Do I need RAG for my AI product?
Only if external knowledge is your real bottleneck. RAG fits when your knowledge is private, changing, too large for one prompt, or when answers must cite sources. It can improve grounding and reduce fabrication risk when retrieval quality is strong.12 For a small, stable corpus, long context may be enough — though long context does not universally replace retrieval, and the right choice depends on the model, corpus, and task.13
Should my AI product use agents?
Usually not in V1. An agent decides its own steps and tools; a workflow runs your process in code with AI doing bounded jobs inside it.5 Agents earn their place when steps genuinely can't be predicted in advance, tools are required, and success can be verified — and they cost more in latency, spend, and debugging. If you can flowchart the process, it's a workflow, not an agent.
Can AI products guarantee zero hallucinations?
No — and be wary of anyone who promises that. Confidently wrong output, which NIST calls confabulation, is a known risk of generative AI that can be reduced and bounded, not eliminated.1 The practical question isn't whether it can happen but what happens when it does: well-scoped products ground answers in sources, say "I don't know" when evidence is missing, and route risky cases to a human.
What is a golden dataset?
A golden dataset is your AI product's driving test: a frozen set of real task examples — real questions, real documents, real exceptions — each with a known correct result. Every version of the product takes the same exam, so you can tell objectively whether a change made things better or worse. It's built before the product, from real work, and it's the first deliverable of a Sprint966 Scope Check.
What accuracy should an AI product reach before launch?
There is no universal standard. NIST explicitly does not prescribe risk tolerances — acceptable risk depends on context and use case.2 The credible approach is a product-specific pass bar set in writing before launch. Sprint966 calls this the starting bar: what does a wrong output cost, and how reversible is it? A human-reviewed draft can tolerate a different bar than an automated action.
Should AI take actions automatically?
Sprint966's default is human-approved automation: the AI does the work; a person presses the button. The AI drafts, classifies, extracts, and recommends — a human approves anything irreversible, like moving money, changing records, or sending legal communication. OWASP's security guidance explicitly recommends least privilege and human approval for high-risk actions.3 Autonomy is then earned action by action, as the approval log proves reliability.
What is the AI Product Scope Check?
The AI Product Scope Check is Sprint966's fixed-scope starting engagement — Steps 1 and 2 of Build vs. Cut, productized. You bring the idea; we define the job, the risk, and the test. You leave with your golden dataset, your starting rung on the Architecture Ladder, your risk posture, and your written starting bar — before committing to a build. Check Your Product Scope →