Practical notes from the engineers who build agents, copilots and AI products.
Structured outputs and function calling force LLM responses into validated schemas so downstream code never parses prose. The model returns typed fields, the application validates them, and a failure is a countable error, not silent corruption. Here is how each mechanism works and the validation layer you still own.
Stream tokens for interactive tasks, queue long jobs with progress, and never block a request on a slow model call. Those three rules decide whether an LLM feature feels responsive or broken. This article covers the streaming stack, the job queue, progress and cancellation, and what the build takes.
Enterprise buyers require SSO, granular roles and audit logs; building them early is cheaper than losing the deal that needs them. Each one touches the data model, so retrofitting means rewriting authorisation checks across the product, while designing for them on day one costs a few weeks and a handful of decisions.
Introduce a tenant key, isolate data by row or schema, centralise config and billing, migrate customers in waves behind a façade. It succeeds when the tenant key is enforced by the database rather than remembered by developers, and when each wave has a rollback path that keeps the old instance intact until sign-off.
Custom personalisation on your own event data replaces subscriptions with an owned system and an A/B-proven lift. It runs as a small service beside your store, reads events and the catalogue through Shopify's APIs, serves recommendations to your theme or headless front end, and reports lift over a holdout you control.
Ship to a chosen cohort, measure adoption and evals, iterate twice, then release generally with the metric that proves it. AI feature rollout works when the flag controls the model and prompt version as well as the UI, the cohort is chosen for feedback quality, and general release waits for a number, not a date.
Self-host when residency, contracts or volume demand it; open-weight models match public APIs for most retrieval and extraction tasks. If none of those three pressures applies, an API is cheaper and simpler. If one does, plan for GPUs, a serving stack, an evaluation suite and the people to run them.
Patch critical vulnerabilities within days, dependencies monthly, and run scanning in CI so nothing waits for an incident. Add a quarterly slot for framework and runtime upgrades, keep a tested rollback for every patch, and report what was patched and what is outstanding each month so the backlog never goes invisible.
Semantic search interprets a shopper's intent and matches products by meaning, with hybrid retrieval for exact matches. "Something warm for a Bengaluru winter" finds light jackets; "SKU 4471" still finds SKU 4471. The build combines embeddings, keyword search, a re-ranker and filters, scored on real queries.