Senior AI engineering for startups
One AI feature, in production, in two weeks.
Most AI features in startups break in the same three ways: they only work on the examples the founder typed, nobody can tell whether a prompt change made them better or worse, and they hang when the model is slow or down. The two-week starter is built around those three. A senior engineer works in your repo: an eval set from your real data on day 2, the smallest model that passes it, a fallback for every model call, and the feature behind a flag on day 9. If the eval set says it is not ready, it stays behind the flag and you get the numbers.
Email io@deemwar.comEmail io@deemwar.com. Work starts once the scope is agreed in writing.
Transcript
When a startup adds an AI feature, it usually breaks in one of three ways. It only works on the examples the founder typed. Nobody can tell whether a prompt change made it better or worse. And it hangs when the model is slow, or down. The two-week starter is a plan against those three. One senior engineer works in your repo. Coding agents do routine parts, and the engineer reviews every change. On day two, we write an eval set from your real data. It becomes the test the feature must pass. Then we pick the smallest thing that passes it: a plain rule, a small model, or a large one, with the cost of each written down. Every model call gets a timeout and a fallback, and each time a fallback fires, it is logged. On day nine, it goes live behind a flag, through your own review and CI. On day ten, we hand it over. Each day, a short written note in Slack or Teams. Once a week, a live demo. The code and the eval set are yours from the first commit. If the eval set says it is not ready, it stays behind the flag, and you get the numbers. To start, email io@deemwar.com: what the product does, who uses it, and the AI feature you want.
The two-week starter, day by day
One AI feature, from your real code to production behind a flag. Ten working days.
- Day 1
Kickoff
A 60-minute call: the feature, the users, what "working" means. We get access to your repo and pick where the code will live.
- Day 2
Read the code, write the eval set
We read the parts of your product the feature touches and write 30 to 100 real examples with the answer you expect. This becomes the test the feature must pass.
- Day 3
Pick the smallest thing that works
For each decision in the feature we try a plain rule, a small model, and a large model against the eval set, and write down the cost and latency of each.
- Days 4–5
First working version
The feature running end to end in a branch of your repo, against the eval set. First demo on day 5.
- Days 6–7
Failure paths
What happens when the model times out, returns nonsense or is down: a fallback for each, logged, and tested.
- Day 8
Measure and cut cost
Accuracy on the eval set, cost per request, latency. Anything slow or expensive gets a cheaper path.
- Day 9
Production, behind a flag
Merged through your normal review and CI, deployed behind a feature flag to a small group of users or internal staff.
- Day 10
Handover
A demo of the live feature, the numbers, a written note on what we would do next, and a walkthrough for whoever owns the code.
What you have after day 10
- One AI feature merged through your review and live behind a flag.
- The eval set: real examples with expected answers, running in CI.
- A fallback for every model call, with a log of each time it fired.
- Accuracy, cost per request and latency for each option tried: a rule, a small model, a large model.
- A two-page handover: how it works, what to watch, what to do next.
The traps startups hit when they add AI
What goes wrong, and what we do instead.
Demo-ware
The feature works in the demo, on the examples the founder typed. Real users type something else and it falls apart.
Real examples first, from your data or your users, and the demo runs on those.
No eval set
Nobody can say whether a prompt change made it better or worse, so every change is a guess and regressions ship.
An eval set on day 2, run on every change in CI.
A large model where a rule or a small model wins
Every decision, even yes/no and pick-one, goes to the largest model because it is already wired up. It is slow, costs more at every step, and is harder to test.
Rules where rules work. For classification-style decisions, a small open-source decision model such as openjevx, which returns a calibrated probability per option in one fast call. The large model only where it earns its cost.
Lock-in to one vendor
Prompts, tools and data formats tied to one provider’s API, so a price change or an outage is your emergency.
A thin layer between your code and the model, so changing providers is a config change, tested against the same eval set.
No fallback
When the model is slow, down or wrong, the feature hangs or shows nonsense to the user.
A timeout, a fallback answer or a plain non-AI path for every model call, and a log of how often it was used.
Nobody on the team can maintain it
An outside team ships something clever and leaves; nobody inside understands it.
Code in your repo, through your review, with a walkthrough for the person who will own it.
How we work
In your repo
Code goes into your repository through your pull requests and your CI. Nothing runs on our servers.
Async by default
A written note in your Slack or Teams at the end of each day: what changed, what the eval set says, what is blocked. One live demo a week.
A senior engineer plus agents
One senior engineer owns your product. Coding agents do routine parts; the engineer reviews every agent change before it is pushed.
You own everything
Code, eval sets, prompts, configuration and cloud accounts are yours from the first commit.
Already using coding agents across the team? See the AI-native assessment and one-day session.
Habits from our own systems
A 201 that delivered nothing
An API returned 201 Created four times while the post went live without its image. Checks now read the published output, not the API response.
Small checks on a small model
Simple yes/no checks in our agent loops run on openjevx, a small open-source decision model, so they are cheap enough to run on every step.
Agent work checked against its task
Our engineering uses coding agents, and every 30 minutes a separate process compares what each agent produced with what it was asked to do.
Questions
Who does it fit?
A startup with a product in production, users, and one AI feature it can describe in a paragraph; no senior AI engineer on the team yet, or the engineers are busy on the core product; and the wish to have the feature in its own codebase, under its own review.
Who is it not for?
Ideas with no product or users yet, research with no production target, or code that has to live outside your repo.
Starter or pod?
The starter is two weeks at a fixed price, scope agreed in writing before day 1, following the ten-day plan above. The pod is the same senior engineer on one product, monthly, with daily notes and a weekly demo; cancel any month. The starter continues as the pod only if you choose.
What does it cost?
Rate on request: both are quoted in writing in the first reply, before anything starts.
What if the feature is not ready after two weeks?
It stays behind the flag. You keep the code, the eval set and the measurements, and a written note on what is missing. No commitment to continue.
Do you train anything on our data?
No. Your data stays in your systems and your accounts.
Which models?
Whatever passes the eval set at the lowest cost: a plain rule, a small open model you run yourself, or a hosted large model. You see the numbers for each.
NDA?
Yes.
Accelerator discount?
Yes. Name the accelerator in your first email.
Next step
Email io@deemwar.com with what the product does, who uses it, and the AI feature you want. The reply has a proposed two-week scope, a written price and a start date.
io@deemwar.comOr send it from here: