Practical notes from the engineers who build agents, copilots and AI products.
Crash-free sessions above 99.5% correlate with rating and retention; monitor from the first beta with crash reporting. Track it per release, per device class and per screen, pair it with ANR rate, cold-start time and failed-request rate, and treat any drop after a release as a rollback trigger rather than a ticket.
Features die when they lack context, cannot act or are scoped too broadly; measure weekly active use per account from a beta cohort. The launch spike is not adoption. Adoption is the share of accounts still using the copilot in week six, decided by three design choices before launch and one measurement habit after.
Every copilot action maps to an existing permission-checked API call, so the copilot can never do what the user could not. Add a preview-and-confirm step, a policy gate for sensitive actions, an audit log and an eval suite, and a copilot that writes data becomes as safe as the UI it sits inside. Here is the model.
Conversational banking agents handle balances, card controls and disputes with identity checks and policy-gated actions. Read-only requests are answered after authentication; card actions run inside limits the bank sets; disputes are captured fully and handed to people with a summary, never decided by the agent.
A Slack or Teams bot answers metric questions with charts, cutting the data team's ad-hoc request queue to the hard questions. It works when it uses the dashboards' governed metric definitions, respects each asker's permissions even in channels, replies in-thread with chart and query, and hands off when stuck.
Computer vision reads documents and inspects products or images; modern vision-language models make many cases viable without custom training. The question is no longer whether you have enough labelled images but whether a general model, prompted well and checked with evals, is accurate enough for the decision.
Cold start is solved with content-based and popularity fallbacks that switch to collaborative signals as history accumulates. A cold start recommendation strategy decides what a new user or product gets on the first visit, how fast the system moves to behavioural signals, and how each stage is measured.
Lift-and-shift for speed, re-platform for cost and manageability; decide per module after the assessment, not for the whole system. Most legacy estates end up with a mix: a few modules moved as-is, the ones that hurt re-platformed onto managed services, and a handful retired or replaced outright.
A churn model is only useful if it feeds an action; build it with the retention team, validate on time and act on the scores. A churn prediction model should be defined around a churn event the business can act on early, scored at the cadence the team works to, and judged on customers saved, not accuracy alone.