MLOps for mid-size companies: serving, monitoring, retraining
What should you know about MLOps if you are a mid-size company running one or two models?
MLOps means a serving API, drift and quality monitoring, and a retraining loop with human review, scaled to your team. You do not need a platform team or a feature store product to run two models well; you need those three pieces built plainly, owned by named people, and reviewed on a schedule.
MLOps is the set of practices that keep a trained model useful after the notebook is closed. For a mid-size company that means three things: a way to serve predictions reliably, a way to know when the model has stopped being right, and a way to retrain it and put the new version live without breaking anything. Everything else in the MLOps vendor landscape is optional. This article describes the minimum that works for a company with one to five models, where the tooling can be kept simple, and where it genuinely needs to grow.
What MLOps is for a company that is not Google
Most MLOps writing comes from organisations running hundreds of models with dedicated platform teams. Their problems, such as feature stores shared across teams, multi-tenant training clusters and model registries with approval workflows, are not your problems. Your problem is that the data scientist who built the churn model left, the model is a pickle file on a laptop, and nobody knows whether it still works. MLOps, at your scale, is the discipline that prevents that state. Google's MLOps maturity guide describes levels of automation; the practical aim for most mid-size companies is the middle level, with a pipeline that can be re-run on demand and a monitored serving endpoint, not full continuous training.
The three pieces, and what each needs
| Piece | Minimum that works | Common over-build | Owner |
|---|---|---|---|
| Serving | A versioned container exposing a predict endpoint, with the feature computation inside or in a small adjacent service | Kubernetes-based model mesh for two models | Backend engineer |
| Monitoring | Input distribution drift, prediction distribution drift, and outcome quality once labels mature, on a dashboard someone looks at weekly | Real-time anomaly alerts on every feature that nobody triages | ML engineer plus the business owner |
| Retraining | A reproducible training pipeline run on a schedule or a trigger, producing a challenger evaluated on a forward window and promoted by a person | Fully automatic retraining that ships a worse model at 3am | ML engineer, with sign-off from the business owner |
| Registry and lineage | Every model version stored with its training data snapshot, code commit, metrics and who approved it | A separate governance platform | Whoever runs the pipeline |
Serving: keep the feature logic with the model
The most common serving bug is training-serving skew: features computed one way in the training notebook and another way in the production service. The cure is to put the feature code in one library that both the training pipeline and the serving container import, and to test that a sample of training rows, pushed through the serving path, produces identical feature values. For batch predictions (a nightly churn score, a weekly demand forecast) the serving container is a scheduled job writing to a table. For real-time predictions it is an HTTP endpoint with a latency budget and a fallback, usually the previous prediction or a simple rule, for when the model is unavailable. Tools such as MLflow handle packaging and versioning well enough that you do not need anything heavier.
Model monitoring: three layers, in order of how soon they warn you
Input drift
Compare the distribution of each incoming feature against the training distribution. A sudden shift usually means an upstream data change: a field renamed, a unit changed, a new product category. This is the earliest warning and the cheapest to compute. It fires often, so it feeds a weekly review rather than a pager.
Prediction drift
Compare the distribution of the model's outputs over time. If the churn model suddenly marks twice as many customers at risk, either the world changed or the inputs did. Prediction drift is the signal that gets the business owner's attention, because it shows up in their lists and queues.
Outcome quality
Once labels mature, compute the real metric (precision at threshold, forecast error, resolution rate) on the period the model was live. This is the only measure that says whether the model is still right, and it lags by however long labels take. Model drift, properly speaking, is a decline here. The other two layers exist to warn you before it does.
The retraining pipeline: reproducible, evaluated, gated
A retraining pipeline is a script that can be run from a clean environment and produce the same model from the same data snapshot. It pulls the data as of a cutoff, builds point-in-time features, trains, and evaluates on forward windows using the method described in data leakage: the silent killer of ML projects. The result is a challenger. It is promoted only if it beats the champion on the forward window and a person approves. We do not recommend fully automatic promotion for mid-size teams; the cost of a bad model going live unreviewed is higher than the cost of a weekly ten-minute approval.
- Schedule retraining on the cadence of the label cycle: monthly for most business models, weekly for fast-moving fraud
- Trigger an off-cycle run when outcome quality drops below the agreed floor
- Keep the last three model versions deployable so rollback is a config change
- Record the training data snapshot, code commit and metrics for every version
- Run the challenger in shadow mode for a period before it replaces the champion on high-stakes decisions
Human review is part of the system
Every model in production should have a named business owner who receives the weekly monitoring summary and approves promotions. This is the same stance we take with agents: shadow mode before autonomy, and a person in the loop for anything consequential. The owner does not need to understand the algorithm. They need to see the lists the model produces, compare them to their own judgement, and say when something looks wrong. Their overrides go back into the training data.
Where LLM operations differ
If your "model" is an LLM application rather than a trained classifier, the serving and retraining pieces change shape: you version prompts and retrieval indexes instead of weights, and evaluation is a golden set of questions rather than a forward window. Monitoring is similar in spirit. We cover the LLM side in LLM observability and prompt versioning and evaluation. Many companies end up running both kinds side by side, and the monitoring dashboard should show both.
A worked example
A last-mile logistics operator had a delivery-time prediction model that had been built by a contractor and was served from a script nobody could run again. The contractor's laptop held the only training code. We rebuilt the pipeline from the event data, put the feature logic in a shared library, and packaged the model as a versioned container behind the dispatch API with a rule-based fallback. Monitoring tracked input drift (which caught a depot code change within a day of it happening), prediction drift, and forecast error as deliveries completed. Retraining runs monthly, produces a challenger, and the operations lead approves promotion after reviewing a comparison. The model is now the least fragile part of the platform we describe in the dispatch platform case study.
Team and timeline
Putting an existing model on this footing takes an ML engineer and a backend engineer four to eight weeks, delivered as an AI/ML development engagement from $17,500 / ₹11.2L. If the model also needs rebuilding, plan for the upper end. Ongoing operation, meaning the weekly monitoring review, the monthly retraining run and the promotion decision, fits a Care Plan: Essential at $1,000 / ₹68,000 a month covers one or two models with a monthly cadence; Standard at $2,500 / ₹1,60,000 suits several models or a weekly cycle. Where the data pipeline itself is the problem, our data and analytics applications team builds it first. See the pricing page for the full bands.
Before you start: a checklist
- Locate the training code and data for every model in production, and confirm it can be re-run
- Name a business owner for each model
- Decide batch or real-time serving and the fallback when the model is unavailable
- Agree the outcome metric and its acceptable floor with the owner
- Confirm how long labels take to mature, which sets the retraining cadence
- Choose where model artefacts and metadata live (a registry, or a bucket plus a table)
- Set up the weekly review meeting before the first dashboard exists
Questions clients ask
- Do we need a feature store? Not for a handful of models. A shared feature library and a table of precomputed features are enough until several teams share features across models.
- Should retraining be automatic? Automatic to the point of producing an evaluated challenger, yes. Automatic promotion, no, until you have a year of evidence that the evaluation catches every regression.
- What if the model is a vendor's? You still monitor inputs, outputs and outcomes. You cannot retrain it, but you can detect when it stops being right and escalate.
- How do we handle a model built by someone who left? Treat it as a rebuild with the old model as the baseline to beat. The first step is recovering the training data snapshot.
Related reading
Continue with fraud and anomaly detection for fintech and e-commerce for a model that needs the fastest retraining cycle, total cost of ownership for AI systems for the running-cost view, and what a Care Plan should cost.
Serve it reproducibly, watch it weekly, retrain it on a schedule with a person deciding: that is MLOps at your scale, and it is enough.
Frequently asked questions
What is the difference between MLOps and DevOps?
▾
DevOps manages code that behaves the same on every run. MLOps also manages data and trained artefacts, whose behaviour changes as the world changes, so it adds drift monitoring, retraining and evaluation gates on top of ordinary deployment practice.
How often should a model be retrained?
▾
On the cadence of your label cycle, and sooner if outcome quality drops below the agreed floor. Monthly is typical for churn and demand models; fraud models often need weekly runs because the patterns move faster.
Can a mid-size company do MLOps without a platform team?
▾
Yes. One ML engineer and one backend engineer can put a few models on a serving, monitoring and retraining footing with open-source tooling, and a Care Plan can run the weekly review and monthly retraining afterwards.