Practical notes from the engineers who build agents, copilots and AI products.
Observability traces each request with prompt version, model, latency and token cost so any bad answer can be located in minutes. It makes evals, routing and cost control auditable, and it is a few days of work with tools such as Langfuse or OpenTelemetry. Here is what to record and where teams go wrong.
Forecast from requests per day, tokens per request and model mix; routing and caching usually cut the first estimate by a third or more. Measure real prompts in shadow mode before committing to a number, separate input from output tokens, and revisit the forecast monthly, because model prices and usage both move.
KYC automation extracts and cross-checks fields with confidence scores, queues exceptions and logs every decision for audit. Accuracy is measured per field on your own documents, thresholds decide what a human sees, and the audit record links every accepted value to the page, the model version and the reviewer.
Integrate existing systems through adapters, even those with no API, rather than replacing them in the first phase. A new dispatch platform earns its place by reading orders from the TMS, posting settlements to the ERP and consuming telematics events, while each legacy system keeps doing the job it already does well.
Integrate through scoped wrappers over HIS, LIS and PACS, tested against real message flows, with strict permissions and logging. Each wrapper exposes only the calls a use case needs, speaks the native interface (HL7 v2, FHIR, DICOM or a vendor API) and logs every call; the AI never touches the system directly.
Integrations with finance, identity and messaging are usually the long pole; scope them first and test against real message flows. Platform integration slips when left to the last month, so inventory every interface in week one, build finance and SSO links early, and rehearse end-to-end flows with real records.
Hybrid search combines keyword and vector retrieval with a re-ranker, catching exact terms vectors miss and meaning keywords miss. It is the single largest retrieval improvement most RAG systems can make, costs little to add, and is measured on a golden set rather than argued about.
AI drafts specs, scaffolds, writes tests and boilerplate; engineers own architecture, security and the decisions that matter. We use coding agents at every stage, but every line they produce passes the same review, evaluation and security gates as human-written code, and the engineer who merges it is accountable.
Measure RAG with a golden question set: retrieval recall and precision, answer groundedness and citation accuracy, re-run on every change. The golden set is built from real user questions with verified answers and sources, and metrics are reported per question type so a failing category cannot hide in an average.