Build vs Buy AI: A 6-Month Custom Tool Migration Post-Mortem

TL;DR

Custom AI tools can cut SaaS costs significantly, but the maintenance burden is real and often invisible at build time. After 6 months, our total cost of ownership exceeded the SaaS we replaced. The decision comes down to one question: do you have someone who owns the tool full-time?

The Build Looked Perfect on Paper

Six months ago, we documented what looked like a clean win: a custom Claude-powered support triage tool replacing a $490/month Intercom plan. API costs were running $28/month. The build took about 40 hours. At a $100/hour contractor rate, that is $4,000 in build cost amortizing across a $462/month saving. Breakeven in 9 months. Clean math.

What the spreadsheet did not show: the 6 hours spent in month 2 when Anthropic updated a model deprecation notice and our prompt outputs shifted. The 4 hours in month 3 debugging a Supabase edge function that broke after a dependency update nobody scheduled. The full Saturday in month 5 when the tool went silent because an API key rotated and there was no alert.

By month 6, we had logged 47 hours of unplanned maintenance time. At $100/hour, that is $4,700. Added to the build cost, total spend was $8,700 across 6 months against $2,940 we would have paid Intercom. The custom tool was now $5,760 more expensive, not cheaper.

This is the core tension in any build vs buy AI decision: the initial comparison is almost always too optimistic, because it captures build cost and API cost while leaving out the long tail of ownership.

Why Maintenance Cost Surprises Every Team

The 47 hours of unplanned maintenance did not arrive as one obvious failure. They came as small interruptions spread across six months, each individually defensible, collectively devastating to the ROI case.

Month 2 brought the Anthropic model deprecation. The Claude model version we had hardcoded was flagged for deprecation, and the replacement model produced slightly different output formatting. Two hours diagnosing, two hours rewriting prompt logic, two hours testing edge cases. Six hours total, none of it budgeted.

Month 3 brought the Supabase edge function failure. A dependency used in our serverless function received a breaking update. The tool silently failed for roughly 14 hours before anyone noticed. Four hours of debugging, one hour writing a monitoring alert that should have existed from day one.

Month 5 brought the API key rotation incident. Our cloud environment rotated secrets on a schedule we had not mapped to our tool’s configuration. The tool stopped processing tickets entirely during a Friday evening. One developer spent most of a Saturday tracing the failure, restoring service, and documenting the fix.

None of these incidents involved bad architecture. The tool was built competently. What they reveal is that any production system requires ongoing stewardship, and that stewardship has a real dollar cost whether you account for it or not.

For teams evaluating custom AI vs SaaS, this is the line item most spreadsheets omit entirely: unplanned incident response.

The On-Call Cost Nobody Budgets For

The maintenance hours are the obvious line item. What is harder to quantify is the on-call cost: the mental overhead of owning a production tool that real workflows depend on.

Someone has to monitor it. Someone has to know how it works when the person who built it is on vacation. Someone has to triage it at 9pm when a client emails saying their support queue is broken.

For SMBs without a dedicated DevOps or internal tooling function, that someone is usually the founder or the ops lead. That is not $100/hour work. It is $300/hour opportunity cost work being spent on infrastructure instead of customers.

SaaS companies price in that reliability. You are paying for their on-call team, their uptime SLA, their support desk. When you go custom, you absorb all of that. There is no separate line item for it on your invoice because you are paying with your own time instead of cash, which makes it easy to treat as free. It is not free.

This dynamic is well documented in the context of internal tooling. Retool’s State of Internal Tools research found that internal tools are abandoned or rewritten within 12 to 24 months at a high rate, primarily due to changing requirements and maintenance burden. The tools do not fail because they were built wrong. They fail because the cost of keeping them current eventually outweighs the benefit.

Technical Architecture Decisions That Change the Math

Not all custom AI builds carry the same maintenance burden. The architecture choices made at the start of a project have a significant effect on how expensive the tool becomes to own over time.

These are the decisions that tend to create the most downstream cost.

Model version pinning vs. version tracking. Hardcoding a specific model version (for example, claude-3-opus-20240229) gives you output consistency but requires a manual migration when that version is deprecated. Building a configuration layer that allows version changes without code deploys adds a few hours upfront and saves multiple hours across every future deprecation cycle.

Synchronous vs. asynchronous processing. Tools that process requests synchronously in real time create immediate failure visibility but also create user-facing downtime when something breaks. Async queue-based architectures allow failures to accumulate and retry without user impact, which changes an incident from “the tool is down” to “processing is delayed.” For internal tools, async is almost always the right choice.

Secret management. Storing API keys in environment variables is standard, but failing to connect secret rotation schedules to application configuration is the single most common cause of silent outages. Using a secrets manager with rotation notifications (AWS Secrets Manager, HashiCorp Vault, or even a simple cron alert) eliminates an entire category of incidents.

Observability from day one. Every custom AI tool should have three things before it touches production: a health check endpoint, an alert for zero-output conditions (the tool ran but produced nothing), and a log of input/output pairs for the last 30 days. These add roughly 4 to 8 hours to the build and remove the majority of diagnostic time from every future incident.

Teams that implement these four patterns consistently report lower maintenance overhead. Teams that skip them in the interest of shipping faster pay for it in the kind of incidents described above.

Where Custom AI Still Wins

This is not an argument against building. It is an argument for building the right things.

Custom AI tools perform well when the use case is narrow, the inputs are stable, and human oversight is not time-critical. A nightly batch process that classifies incoming vendor invoices: build it. A quote generator that pulls from your own product catalog and outputs a PDF: build it. An internal knowledge base assistant trained on your SOPs: build it.

The pattern that works: the tool runs on a schedule or on demand, a failure does not break a customer-facing workflow, and the only person who needs to interact with it is you.

The pattern that breaks: real-time customer interactions, multi-user interfaces, anything that touches billing or auth, and anything that relies on third-party APIs that change their schemas without warning.

The distinction is not about complexity. A document classifier can be quite sophisticated technically and still be a good custom build candidate, because its failure mode is tolerable: documents wait until someone notices and restarts the process. A customer support routing tool can be architecturally simple and still be a bad custom build candidate, because its failure mode is a customer-facing outage.

The Decision Matrix

Before committing to either direction, run these five questions. They are the ones we should have answered before we shipped.

Question	Custom Build Wins	SaaS Wins
Who maintains it when it breaks at 10pm?	You have a named developer on retainer	Nobody has that answer
How stable are your inputs and outputs?	Same format, same source, every time	Variable data, changing requirements
Is this customer-facing or internal-only?	Internal, async, low stakes	Customer-facing, real-time, SLA matters
Do you need multi-user permissions or audit logs?	No, single operator	Yes, team access or compliance required
What’s your monthly dev retainer already covering?	You have budget and bandwidth	Dev time is already maxed

If you answered “SaaS Wins” on 3 or more rows, do not build. The SaaS price is the cheaper option once you price in your time honestly.

What the Migration Back Actually Cost

Migrating back to Intercom was not painful in terms of the product itself. Intercom’s onboarding is solid, the data export from our custom tool was clean enough, and the feature parity was adequate for our use case.

The real cost was psychological: admitting the build was the wrong call, re-training the team on a new interface, and accounting for the $8,700 in sunk cost.

The lesson is not that the build failed technically. It worked fine. The lesson is that “technically working” and “worth owning” are different thresholds.

A custom tool that requires 8 hours of maintenance per month to save $462 is not a win. It is a part-time job you did not post a listing for.

The Honest Build vs Buy AI Recalculation

When running the math on a custom AI build, use this formula before you start:

Total monthly cost of custom = (API costs) + (expected maintenance hours multiplied by your real hourly rate) + (0.5 multiplied by build cost amortized over 18 months).

The 18-month amortization is intentional. Most internal tools do not survive past 18 months without a rewrite. That is not a knock on the builders. It is a pattern documented in Retool’s State of Internal Tools research. Internal tools get abandoned or rewritten within 12 to 24 months at a high rate, primarily due to changing requirements and maintenance burden.

In our case: $28 API costs plus (7.8 average monthly maintenance hours multiplied by $100) plus (0.5 multiplied by $4,000 divided by 18) equals $28 plus $780 plus $111, for a total of $919/month. Against $490/month for Intercom, the custom build was never the cheaper option once maintenance was priced in.

If your calculated number is within 20% of the SaaS price, take the SaaS. The reliability premium is worth it.

Signals That Predict a Custom Build Will Succeed

Rather than only describing failure modes, it is worth naming the signals that correlate with successful custom AI builds in SMB contexts.

The build succeeds when there is one named owner with explicit time allocated to maintenance in their role. Not “the developer can handle it” but “Sarah owns this tool and has 6 hours per month reserved for it.”

The build succeeds when the failure mode is invisible to customers. Internal async tools that process data in batches can fail quietly and recover quietly. Real-time customer-facing tools cannot.

The build succeeds when the integration surface is small. A tool that calls one API (your own database plus one AI provider) has fewer breaking points than a tool that orchestrates five services. Each additional integration is a maintenance multiplier.

The build succeeds when observability is treated as a first-class deliverable. Teams that instrument their custom tools from the start spend significantly less time on incident response than teams that add monitoring reactively.

The build succeeds when the team has run the full ownership cost calculation before committing, not as a post-hoc justification.

The Bottom Line

Build custom AI tools for narrow, async, internal workflows where you already have developer bandwidth on retainer. Do not build anything customer-facing, real-time, or multi-user unless you are ready to staff it like a product. The migration back to SaaS cost more in time than the original SaaS subscription ever would have, and the lesson was available before we started if we had asked the right questions.

The build vs buy AI decision is not a technical question. It is an operational one. The right answer depends almost entirely on whether your team has the capacity to own what you build, not just ship it.