azyware
Business

Managed AI services vs in-house LLMOps: which to choose and when

EZ
Eazyware
· 7 min read
Quick answer

What is the difference between managed AI services and in-house LLMOps?

Managed AI services mean somebody else runs your AI systems day to day: evals, cost monitoring, prompt regression, model swaps and incidents. In-house LLMOps means your own engineers own that rota. The difference is not capability but who is on call when a model changes behaviour overnight.

Managed AI services mean somebody else runs your AI systems day to day: evals, cost monitoring, prompt regression, model swaps and incident response. In-house LLMOps means your own engineers own that rota. The difference is not capability but who is on call when a model changes behaviour overnight, and how quickly that person can act.

Below is the work itself, spelled out so you can price it either way, followed by the conditions under which each option is genuinely better, the co-sourced arrangement that suits most mid-size companies, and what it costs to move the rota back in-house after a year with a vendor.

What LLMOps actually involves

LLMOps is the operating discipline for systems whose behaviour changes without a code deployment. A conventional web application does what you shipped until you ship again. An AI system can change because the provider updated a model, because your documents were re-indexed, because a prompt was edited, or because users started asking a new kind of question. The LLMOps glossary entry gives the short definition; the work is concrete and repetitive.

Every month somebody has to re-run the evaluation suite, read a sample of real transcripts, review the cost dashboard for a routing change that quietly tripled spend, check latency at the ninety-fifth percentile rather than the mean, triage escalations, update the golden set with new failure cases, patch dependencies, and watch the provider's deprecation notices. Every quarter somebody re-benchmarks the model choice. When a provider retires a model, somebody runs the migration described in handling model deprecations. None of this is glamorous, all of it is load-bearing, and the honest question is only whose calendar it sits in.

The two options compared

DimensionManaged AI servicesIn-house LLMOps
Who holds the pagerThe vendor, under a contracted response timeYour engineers, alongside their other duties
Time to operationalDays; the practice already existsTwo to six months to hire and build the runbook
Cost shapeFixed monthly fee, predictableSalaries and tooling, largely fixed but higher at the floor
Depth of contextGood on AI operations, thinner on your business edge casesDeep on the business, thinner on AI failure patterns at first
Knowledge retentionLeaves with the contract unless handover is contractualStays, until the engineer resigns
CoverageContracted hours, including 24x7 at the top tierWhatever your team can actually sustain
Tooling costIncluded in the feeTracing, eval and dashboard stack on your budget
Best atSteady-state operations and model change managementFast iteration on product behaviour
Reversal costLow if code, prompts and runbooks are yoursHigh if the one person who knows it leaves

When in-house LLMOps is the right call

Build the capability internally when AI is the product rather than a feature of it. If the model behaviour is your differentiation, if your engineers ship prompt and retrieval changes weekly, and if a slow feedback loop through a third party would be the bottleneck, ownership wins. It also wins when data sensitivity means no external party should see production traffic, even redacted, which is common in banking under RBI outsourcing expectations and in hospital systems handling clinical records.

There is a threshold effect worth naming. One AI engineer is not an LLMOps capability; it is a single point of failure with a holiday allowance. The capability starts at roughly two engineers who can cover each other plus a platform engineer who owns the tracing and cost stack. If you cannot fund that, in-house is a decision to be under-covered, not a decision to be independent. The minimum viable version is set out in LLMOps for small teams.

When managed services are the right call

A managed arrangement is the better answer in four situations.

  • AI is one system among many. A support agent and a document pipeline do not justify a standing AI platform team, but they still need the monthly rota run properly.
  • You need coverage you cannot staff. Round-the-clock response with a one-hour target is straightforward to buy and expensive to build for a single system.
  • The system is in steady state. Once behaviour is stable and the work is monitoring, re-running evals and managing model changes, that is exactly the shape a contract handles well.
  • You are pre-hire. Buying the rota for six to twelve months while you recruit is cheaper than shipping unattended and cheaper than a rushed hire.

The honest limit is context. A vendor knows AI failure patterns well and your business edge cases less well, and closing that gap takes a named engineer and regular contact rather than a ticket queue. Ask how it will close before you sign.

The co-sourced middle, which is what most teams need

The arrangement we see working most often is neither pure option. Your engineers own the product behaviour: prompts, retrieval, features, the definition of a correct answer. The vendor owns the operating rota: eval runs, cost and latency monitoring, dependency patching, model migration and out-of-hours response. Roughly half our work is paired with internal teams on exactly this basis, with documented handover written in from the start rather than negotiated at the end.

Two conditions make co-sourcing work rather than muddle. The first is a written split of duties with one named owner per activity, because shared ownership of an on-call rota means nobody owns it. The second is that everything the vendor produces lives in your repository: runbooks, eval sets, dashboards and prompt history. You own code, infrastructure, prompts, model choices and documentation either way, which is what makes the arrangement reversible.

How to decide in an afternoon

  • List every AI system in production and the monthly hours each genuinely needs, honestly rather than optimistically.
  • Name the person who currently reads the cost dashboard. If nobody does, that answers most of this.
  • Decide the response time your business actually needs out of hours, and whether you can staff it for a year.
  • Ask whether model behaviour is your differentiation or your plumbing.
  • Price both options over twenty-four months, not twelve, because the hiring option front-loads cost and the contract does not.
  • Write the exit terms first: what you get back, in what format, within how many days.

What each costs

Our published Care Plans price the managed side directly. Essential is $1,000 or ₹68,000 a month for business-hours cover in IST, an eight-hour response target and ten hours of work. Standard is $2,500 or ₹1,60,000 for 24x5 cover, a four-hour response and twenty-five hours. Enterprise is $5,250 or ₹3,40,000 for 24x7, a one-hour response, sixty hours and a named engineer. The AI system add-on at $750 or ₹40,000 a month covers the AI-specific rota: evals, cost monitoring, prompt regression and re-indexing. Full scope sits on the software maintenance and support page and the pricing page, and what a care plan should cost explains how to read any such contract. The in-house comparison, including the parts that do not appear on a salary line, is laid out in Eazyware vs building an in-house AI team and in-house AI team vs agency vs freelancers.

What it costs to change your mind

Moving from managed to in-house should be a handover, not a migration, and whether it is depends entirely on terms you set at the start. If the runbooks, eval suites, dashboards and prompt history are already in your repository, transition is a few weeks of shadowing: your engineers run the rota with the vendor watching, then alone. If the vendor kept the observability stack in their own tenancy and the eval sets in their own tooling, you are rebuilding rather than receiving, and that is a quarter you did not budget for.

Going the other way, from an under-resourced in-house rota to a contract, is usually faster but exposes how little was written down. Expect the first month to be archaeology. Building on open instrumentation standards reduces both risks, since traces and metrics defined to a public specification move between teams and tools without translation.

When a managed service is the wrong choice

Do not buy operations for a system that has not stabilised. If you are still changing the prompt weekly and arguing about what a correct answer looks like, you are in development and a support contract will measure the wrong things. Do not buy it to avoid building evals either: a vendor can run your evaluation suite, but somebody inside your business still has to decide what correct means, and no contract transfers that. And if your real problem is that the system does not work, no rota fixes it; that is a rebuild conversation. Equally, in-house is the wrong choice when it is really a hiring plan with no funded second engineer, because the risk you have removed is vendor dependency and the risk you have added is one person's notice period.

OpenTelemetry's semantic conventions for generative AI define a vendor-neutral way to record model calls, token usage and errors, which is the practical foundation for keeping your telemetry portable between an in-house team and a managed provider. On our side, LLM observability covers the tracing layer in detail, and the comparison hub collects our other decision guides. If you want the split costed against your actual systems, talk to us.

Keep ownership of what the system should do, and be pragmatic about who runs the rota that keeps it doing it.

Frequently asked questions

What does a managed AI service actually do each month?

▾

Re-runs the evaluation suite, samples real transcripts, monitors cost and latency, patches dependencies, triages escalations, updates the golden set with new failure cases, and manages provider model deprecations. At Eazyware that work sits in a Care Plan with the AI add-on at $750 or ₹40,000 a month on top of the base tier.

Do we lose control of our AI systems with a managed service?

▾

Not if the contract is written properly. You should own the code, infrastructure, prompts, model choices, eval sets, runbooks and dashboards, all held in your own repositories and accounts. If any of those live only in the vendor's tooling, you have bought a dependency rather than a service. Check this before signing.

How many engineers does in-house LLMOps need?

▾

As a working minimum, two AI engineers who can cover each other plus a platform engineer owning tracing, cost dashboards and pipelines. One engineer is a single point of failure rather than a capability. If that team cannot be funded, a co-sourced arrangement usually gives better coverage for less than an under-staffed internal rota.