Practical notes from the engineers who build agents, copilots and AI products.
Multi-tenant LLM backends isolate prompts, retrieval and logs per tenant and meter usage so features can be billed. The tenant ID travels with every request from the API to the vector store to the trace, and nothing crosses it. Here is the architecture, the isolation points, the metering model and the cost to build.
A multi-agent system splits a workflow across specialised agents, a planner, workers and a reviewer, coordinated by an orchestrator with shared state. It earns its complexity only when a single agent's tool list or context has become unmanageable; before that, one well-built agent is cheaper and easier to evaluate.
Wrap the core with an API, add AI on the stable core and migrate modules in slices scheduled around month-end and audits. The ledger and interest engine stay put until each slice has proven itself in a parallel run, so the lender gets new capability in weeks while migration risk is spread over quarters.
Select by benchmarking candidates on your tasks for accuracy, latency and cost, then design routing so no single vendor locks you in. Public leaderboards rank models on other people's problems; a few hundred of your own labelled examples tell you which model clears your bar at a price you can sustain.
Routing across OpenAI, Anthropic, Google and open-weight models controls cost and risk when any provider changes. Every system we build calls models through one routing layer, with per-task model choices set by evaluation results, so a price change, a deprecation or an outage is a configuration change, not a rewrite.
MLOps means a serving API, drift and quality monitoring, and a retraining loop with human review, scaled to your team. You do not need a platform team or a feature store product to run two models well; you need those three pieces built plainly, owned by named people, and reviewed on a schedule.
Upgrade slice by slice with characterisation tests as the gate, scheduled around the business calendar, with rehearsed rollback. Stage through 7.4, fix deprecations before they become fatal errors, replace removed extensions, and run each slice on the new runtime in parallel before switching traffic.
MCP is a standard for exposing tools to AI agents with scoped permissions, so agents can act on your systems without direct database access. You write one MCP server per system, define each tool's inputs, outputs and scope, and every agent, whichever model runs it, calls the same contract under the same audit.
Minimum LLMOps is tracing, an eval suite on every change, cost dashboards, model pinning and a rollback for prompts. A team of two or three engineers can run all five with open-source tooling and a few days of setup, and without them an LLM application in production is being operated on hope rather than evidence.