How to Ship Your First Production AI Feature in 30 Days
Last updated September 2026
To ship an AI feature to production in 30 days, freeze the evaluation set before you write a prompt, put the cost ceiling, kill switch, and rollback into the first slice, and cut scope instead of cutting gates. Teams miss the date on integration and stability, not on model quality.
Why do most AI features never ship to production?
Not because the model was not good enough. The evidence points the other way.
MIT’s NANDA initiative studied more than 300 publicly disclosed AI initiatives, interviewed 52 organizations, and surveyed 153 senior leaders in the first half of 2025. Its report, The GenAI Divide: State of AI in Business 2025, found that 95 percent of organizations were getting zero measurable return on 30 to 40 billion dollars of enterprise investment. The funnel it describes for task-specific tools is the part engineering leaders should read twice: roughly 60 percent of candidate use cases get investigated, 20 percent reach a pilot, and 5 percent reach production. The report also found that work done with an outside partner was about twice as likely to reach full deployment as work built entirely in house, 66 percent against 33 percent.
Gartner reached the same conclusion from a different direction. Its June 2025 forecast, over 40 percent of agentic AI projects will be canceled by the end of 2027, names three causes: escalating costs, unclear business value, and inadequate risk controls. A January 2025 poll of 3,412 webinar attendees found only 19 percent had made a significant investment. Gartner also estimated that of the thousands of vendors claiming agentic AI, roughly 130 were real.
Read the three cancellation reasons again. None of them is a capability problem. Cost, value, and risk controls are all decided in the first week, by the team, before any model is involved. That is the gap in most of the advice published on this topic: it lists the capabilities a production AI system needs and leaves out the order you build them in and the moments you are allowed to stop.
What has to be true before you ship an AI feature to production?
Eight things. Every one of them is cheap on day 3 and expensive on day 25. We treat these as gates rather than a checklist, which means the feature does not advance until the gate is passed, and scope gets cut instead.
- A written definition of failure. One sentence describing what a wrong answer looks like to the user, and who sees it first. If nobody can write that sentence, the feature is not ready to be built.
- A frozen evaluation set. 100 to 300 examples drawn from real traffic, labeled by someone who understands the domain, with at least 20 percent adversarial and edge cases. Frozen means you do not add to it after you start tuning.
- A baseline that is not AI. Keyword search, a rules table, or the current manual process, scored on the same set. Sometimes the baseline wins, and finding that out in week one is a success, not a failure.
- A pass bar set before you build. A number on the evaluation set that the feature must clear to ship, agreed while nobody is emotionally invested in the outcome.
- A latency budget. A stated p95 target in milliseconds and a defined behaviour when it is exceeded: stream, serve from cache, or degrade to the baseline.
- A cost ceiling enforced in code. Dollars per thousand requests, plus per-user and per-tenant rate limits, a maximum output token setting, and a hard monthly cap that fails closed.
- A kill switch you have tested. A flag that disables the feature without a deploy, exercised once in staging and once in production before the ramp starts.
- Logging with retention and redaction decided. Inputs, outputs, model version, token counts, and latency, with the retention period and the redaction rules written down before the first request is logged.
Gates 5 through 8 are the ones teams defer, and they are the ones that turn a launch into an incident. The 2025 DORA report is blunt about the mechanism: in a survey of nearly 5,000 technology professionals, AI adoption showed a positive relationship with throughput and product performance and a negative relationship with delivery stability. The same report found 90 percent of respondents using AI at work and 30 percent reporting little or no trust in the code it produces. Speed without control systems is the failure pattern, and it is visible in the data.
What does the 30-day schedule look like, week by week?
This is the sequence we run. The dates matter less than the order, and the order is deliberate: the measurement exists before the thing being measured, and the controls exist before the traffic.
| Days | What gets built | Gate to pass before continuing |
|---|---|---|
| 1 to 3 | Problem statement, failure definition, blast radius, owner named | One sentence describing a wrong answer and who sees it |
| 4 to 7 | Evaluation set built from real traffic and frozen, non-AI baseline scored | 100 or more labeled examples, baseline number recorded, pass bar agreed |
| 8 to 14 | Thin vertical slice behind a flag: real data in, real answer out, no UI polish | End to end in staging with logging, token metering, and the flag working |
| 15 to 18 | Guardrails: input validation, output handling, rate limits, cost caps, timeouts | A red team pass on prompt injection and unsafe output handling |
| 19 to 22 | Internal use by 10 to 30 real users with a one-click report button | Evaluation score at or above the pass bar, reported failures triaged |
| 23 to 26 | Ramp to 5 percent of traffic, on-call rotation live, dashboards published | p95 latency, cost per thousand requests, and error rate inside budget |
| 27 to 30 | Ramp, hold, or roll back. Written go or no-go against the day 3 definition | A decision recorded by a named person, not a vote |
Two things about this schedule surprise people. The first is that roughly a third of it is spent before any prompt is written. The second is that the slice built in days 8 to 14 is deliberately ugly. Polishing an interface around an answer you have not yet proven is the most common way a 30-day plan becomes a 90-day plan.
Why write the evals before the prompt?
Because a prompt without an evaluation set is a demo, and a demo is exactly the artifact that impresses a stakeholder and then stalls for six months. The evaluation set is what converts an opinion into a number, and the number is what lets you cut scope without arguing.
Three rules make the set worth having. Draw the examples from real traffic, not from imagination, because imagined inputs are always politer than real ones. Label them before you look at any model output, so the labels are not contaminated by what the model happened to produce. And freeze the set once tuning starts, keeping a second unlabeled sample in reserve for a final check, because a set you keep editing will always show improvement.
Set the pass bar against the baseline, not against perfection. The question is never whether the model is good. It is whether the model is enough better than a rules table to justify the operating cost, the latency, and the new failure modes you are about to own. Plenty of features fail that test honestly, and killing one on day 7 costs a week instead of a quarter.
Which failure modes should be designed for rather than discovered?
The OWASP Foundation publishes the Top 10 for LLM Applications, updated for 2025. It is the closest thing this field has to a shared vocabulary for what goes wrong, and six of the ten entries map directly onto decisions made during a 30-day build.
| OWASP risk | What it looks like in your product | Control to build in days 15 to 18 |
|---|---|---|
| LLM01 Prompt Injection | Text in a document, email, or web page instructs the model to ignore your rules | Treat all retrieved content as untrusted, separate instructions from data, test with a known injection corpus |
| LLM02 Sensitive Information Disclosure | Customer records or secrets appear in an answer to the wrong user | Filter by permission before retrieval, never after, and redact at the logging boundary |
| LLM05 Improper Output Handling | Model output is rendered as HTML or executed as SQL or a shell command | Validate and escape output as you would any user input, never eval it |
| LLM06 Excessive Agency | The feature can send, delete, refund, or post without a human in the loop | Read-only by default, explicit confirmation for writes, scoped credentials per action |
| LLM08 Vector and Embedding Weaknesses | Retrieval returns documents the user is not entitled to see | Tenant and permission filters enforced inside the vector query, tested per tenant |
| LLM10 Unbounded Consumption | One loop, one retry storm, or one abusive user produces a five figure bill | Max tokens, per-user and per-tenant quotas, circuit breakers, a hard monthly cap |
LLM10 deserves its own paragraph because it is the one that reaches the finance team. Unbounded consumption is not usually an attack. It is a retry policy that backs off badly, an agent loop with no iteration limit, or a batch job that runs against the full table instead of a page. Meter tokens per request from the first slice, alert at 50 percent of the monthly budget, and make the cap fail closed rather than fail expensive.
How do you keep the running cost predictable?
Model prices move constantly, so any specific per-token figure in an article ages badly. The discipline that does not age is measuring cost in the unit your business already uses.
Pick a denominator on day 8: cost per thousand requests, per resolved ticket, per active user, or per document processed. Instrument it in the first slice alongside latency, so the number exists before anyone asks for it. Then apply the four levers in order of how much they cost you to pull: cap output length, cache aggressively on repeated inputs, route the easy cases to a smaller model and the hard ones to a larger one, and only then consider fine tuning. The ordering matters because the first two are configuration and the last one is a project.
One more figure worth tracking from the start: how often the feature falls back to the baseline. A fallback rate that climbs is the earliest signal that something upstream changed, usually before your error rate moves at all.
When should you kill the feature instead of shipping it?
Deciding this in advance is what separates a 30-day plan from a 30-day hope. Four conditions end the project, and agreeing on them on day 2 means nobody has to be the person who loses the argument on day 26.
- Nobody can write the failure sentence. If the team cannot describe a wrong answer, it cannot detect one in production either.
- The non-AI baseline is within a few points of the model. Ship the baseline. It is cheaper, faster, and explainable, and you can revisit the model when the gap is real.
- The evaluation score clears the bar only after the set has been edited. That is not a passing score. It is a moved goalpost.
- The cost per unit of value is higher than the value. Compute it with the denominator you chose on day 8, not with a hand wave about future prices.
A killed feature is a good outcome when it is killed in week one or two. Gartner’s three cancellation causes all describe projects that were killed late, after the cost had been spent. The plan above exists to move that decision to the front.
What does day 30 actually look like?
Not a launch event. A ramp, with a rollback path that somebody has used.
Turn the feature on for a small share of traffic, watch the four numbers you have been instrumenting since day 8 (evaluation score on live samples, p95 latency, cost per unit, fallback rate), and raise the share only when all four hold for a full business cycle. Keep the flag in place for at least a month after full ramp. Write down the model version, the prompt version, and the retrieval index version in the same place you record deploys, because the first production surprise is usually a change in one of the three that nobody logged.
Then plan the second sprint before you celebrate the first. AI features age differently from ordinary code: the inputs drift, the vendor updates the model, and the failure modes that appear in month two were not present in month one. Budget for that, or you will be reading our piece on why AI-generated code compounds technical debt from the wrong side of the problem.
When is outside help worth it?
Three situations. The team has never run a model in production and the first attempt is also a commitment to a board or a customer. The feature touches regulated data, where the cost of a wrong answer is legal rather than embarrassing. Or the calendar is already fixed and the team is fully committed to the existing roadmap.
The MIT finding about partnered work reaching deployment at twice the rate of internal builds is less about talent than about sequence. An outside team that has done this before starts with the evaluation set and the cost ceiling because it has been burned by skipping them.
The TopDevz AI Integration Sprint is 24,500 dollars and runs exactly this schedule, with the final milestone waived if delivery misses day 30. Where the constraint is ongoing capacity rather than a single feature, Delivery Pods start at 15,000 dollars per month on 30 days’ notice. Either way, insist on the same gates from any vendor you hire: a frozen evaluation set, a published pass bar, and a cost ceiling in code.
The short version
Thirty days is enough to ship an AI feature to production, and it is not enough to ship one and then work out how to operate it. So build the operating apparatus first. Days 1 to 7 buy you the ability to tell whether the thing works. Days 8 to 18 buy you the ability to run it without a surprise bill or an unsafe action. Days 19 to 30 are the ramp.
If the schedule slips, cut scope. Cut the number of cases handled, cut the interface, cut the second language, cut the agent down to a suggestion. Do not cut the evaluation set, the cost ceiling, the kill switch, or the logging, because those four are what make day 31 survivable.
TopDevz is an onshore engineering firm with offices in Sacramento, California and Toronto, Ontario, staffed from a vetted pool of 15,643 developers averaging more than ten years of experience, with 96 percent team retention. Read how the team is built, or Book a Code Review and a senior engineer will walk your planned AI feature against these gates and tell you where the 30 days would break.