Hosted or self-hosted AI: how to decide for your business
"Do we have to send our data to OpenAI?" is one of the most common questions in our readiness audits, usually asked by someone who has already been told two contradictory things: that hosted AI is the only practical option, and that serious businesses self-host everything. Both claims are sales pitches. The real answer is a trade-off, it's different for different workloads, and you're allowed to change your mind later.
We run both kinds of setup for clients, so we have no horse in this race. Here's how we'd explain the decision to a friend.
What's the actual difference?
Hosted AI means you call a provider's service (OpenAI, Anthropic, Google) and pay per use; the model runs on their computers. Self-hosted AI means an open model runs on infrastructure you control, your data never leaves your environment, and you pay for the hardware whether it's busy or idle.
That last clause is the heart of the economics. Hosted is a variable cost that starts near zero and grows with usage. Self-hosted is a mostly fixed cost that starts noticeable and barely grows. Low usage favors hosted, heavy usage favors self-hosted, and the crossover point is further out than most people guess, because hosted prices have kept falling.
The trade-off, honestly
| Hosted | Self-hosted | |
|---|---|---|
| Time to start | Same day | Weeks |
| Cost shape | Per use; grows with success | Mostly fixed; predictable |
| Model quality | The best available, updated for you | Open models: genuinely good now, a step behind the frontier |
| Your data | Leaves your environment under the provider's terms | Stays on systems you control |
| Maintenance | The provider's problem | Your problem (or your IT partner's) |
| Provider risk | Price changes, policy changes, deprecations | Largely insulated |
| Compliance fit | Depends on the provider's certifications | Easiest story to tell an auditor |
A few of those rows deserve a closer look.
Data terms are better than the fear, worse than the marketing. The major providers' business tiers contractually commit not to train on your data, and for most businesses that's genuinely sufficient. But "sufficient" depends on what the data is. Customer emails are one thing; patient records, legal files, or anything your own contracts forbid sharing are another. Read your obligations before someone else's terms of service.
Open models stopped being a compromise. A few years ago self-hosting meant accepting visibly worse answers. For focused, well-scoped tasks, answering questions from your documents, classifying email, extracting fields from forms, current open models are more than good enough, and a model that's fine-tuned to your one task can beat a frontier model that's juggling everything. The frontier still wins on hard, open-ended reasoning. Most business workloads aren't that.
Maintenance is the hidden price of control. A self-hosted setup needs patching, monitoring, backups, and someone who notices when a disk fills up at 2am. If you don't have that someone, the savings evaporate into emergencies. This is exactly the work our server management service exists for, and we say more about these unglamorous costs in the infrastructure bill nobody mentions.
When hosted is the right call
Start hosted if you're early, your volume is modest, and your data terms allow it. Speed of learning matters more than anything else at the start of an AI effort, and hosted gets you a working pilot in weeks with no hardware decisions at all.
That covers most small businesses, most of the time, and we're happy to say so even though self-hosting is closer to our roots. Paying a provider a modest monthly bill to avoid owning infrastructure is a perfectly good deal at low volume.
When self-hosting starts to win
Self-hosting starts to win when one of three things is true: your usage bill has grown past the cost of running your own setup, your data legally or contractually can't leave your environment, or a provider dependency has become a business risk you can't accept.
The data reason is the most common trigger for the businesses we work with. Healthcare, legal, finance, and anyone whose client contracts include confidentiality clauses written before AI existed: for them, "the data never leaves our network" isn't a preference, it's the requirement that decides the architecture. The second most common is the bill. A tool that took off internally can turn per-use pricing into a five-figure annual line item, at which point a one-time setup cost plus predictable hosting looks very different.
The part vendors won't tell you: it's not permanent
The decision feels weighty because people treat it as forever. It isn't. The pattern we recommend most often is: prove value on a hosted service, then move the specific workloads that got expensive or sensitive onto your own infrastructure once the numbers justify it.
The catch is that this only works if the system was built to allow it. An assistant wired directly and exclusively to one provider's quirks is a rebuild waiting to happen; one built with a clean seam between "our logic" and "the model" can switch backends in days. We build that seam by default, and if someone else is building for you, it's worth asking them pointedly whether they do. The same seam is what protects you when a provider changes prices, which they have, and will again.
Deciding for your situation
On paper, the decision tree is short: check your data obligations, estimate your volume, be honest about your maintenance capacity, choose, and revisit yearly. In practice the hard part is the estimating, which is why we fold this question into every AI readiness audit rather than answering it in the abstract.
If you're staring at this decision right now, with a compliance question or a hosted bill that's grown teeth, book a call. We'll tell you which side of the line you're on, and "stay hosted, it's fine" is an answer we give a lot.