On-device AI in mobile apps: voice, vision and assistants

On-device AI mobile features are the ones that run on the phone's own chip: wake-word detection, speech-to-text, camera OCR, barcode and document capture, image classification and short-text tasks. Cloud models are the ones that reason, plan and write. The mistake most product teams make is treating this as a choice. It is a routing decision, made per feature and often per request, based on latency, privacy, connectivity and cost. This article explains what runs well on a device today, what does not, how the routing works in a React Native app, and what it costs to build.

Why on-device AI matters for mobile products

Three things change when a model runs on the phone. Latency drops from hundreds of milliseconds to tens, because there is no network round trip. Privacy improves, because the audio, image or text never leaves the device. And the feature works in a lift, a basement warehouse or a rural clinic with no signal. Those three properties are exactly what field, retail and healthcare apps need, and none of them is available from a cloud-only design.

The cost is capability. A model that fits in a few hundred megabytes and runs on a mid-range Android phone cannot reason the way a frontier model can. So the design question is never "on-device or cloud" but "which tasks are small enough to run locally, and how do we hand the rest to the cloud without the user noticing the seam".

On-device versus cloud: a routing table

What runs well on the phone in 2026

Voice

Wake words, voice activity detection and command-length transcription are solved problems on modern phones. Both platforms ship speech APIs, and open-weight speech models in the small size classes run acceptably on devices from the last three or four years. Where teams get into trouble is expecting device transcription to handle a noisy warehouse in three Indian languages. Test with real audio from the real environment before committing to a device-only path, and keep a cloud transcription route available for the hard cases.

Vision

Document capture, OCR, barcode scanning and closed-set classification are the strongest on-device use cases. Platform vision frameworks handle text recognition and rectangle detection; a small custom model handles the labels specific to your business. The pattern that works is device-first capture and pre-processing, then a cloud call only when the document needs understanding rather than reading. That is the shape we used in the KYC document intelligence build: the app captured and checked quality locally, and the extraction ran server-side.

Small language models

Sub-billion and few-billion parameter models now run on flagship phones and, more slowly, on mid-range ones. They are useful for classification, short rewrites, form filling from a sentence, and deciding whether a request needs the cloud at all. They are not useful as a general assistant. Treat them as a fast first layer, not the brain.

Designing the router

The router is a small piece of code that decides, for each request, whether to call a local model or a remote one. It reads four signals: network state and measured latency, battery and thermal state, the sensitivity of the data involved, and the task type. It is model-agnostic on the cloud side, so the remote endpoint can move between OpenAI, Anthropic, Google or a self-hosted open-weight model without the app changing, which is the same routing stance we take in every LLM application.

• Default local for anything in the device column of the table; the cloud is an upgrade, not the baseline
• Fall back silently: if the network call exceeds a timeout, use the local result and mark it as such
• Never send raw audio or camera frames when a local pre-processing step can send text or a crop instead
• Tag every request with its route so analytics can show where quality problems come from
• Keep a per-feature switch server-side so the route can be changed without an app release

React Native AI: the practical stack

React Native does not run models in JavaScript. The model runs in a native module, exposed to the JS layer through a bridge or the newer native module system. On iOS the native side uses Core ML or the platform speech and vision frameworks; on Android it uses the ML Kit and LiteRT family or the NNAPI path. The React Native layer owns the UI, the router and the state. This split keeps one codebase for the product and platform-specific code only where the hardware demands it, which is why we build most React Native mobile products this way rather than going fully native.

Three engineering details decide whether the feature ships well. First, model delivery: bundle the model with the app for the first launch, but download updates on demand so the app store binary does not bloat and the model can improve without a release. Second, threading: inference must never run on the JS thread or the UI freezes. Third, device tiers: define a minimum device class, and degrade gracefully below it by routing to the cloud.

Privacy and compliance

Keeping data on the device is the cleanest privacy story available. Voice never leaves the phone, images are processed and discarded, and only derived text is sent when the cloud is needed. That matters under India's DPDP Act and under GDPR, and it matters more to users than any consent screen. Document the routing so your privacy notice can say precisely what leaves the device and when. Apple's Core ML documentation is a useful primary reference for how on-device inference is packaged on iOS.

Cost: what changes with edge AI

Cloud inference is a per-request cost that scales with users. On-device inference is a build cost that is paid once and then costs nothing per request, which is why an edge AI app can have a much lower running bill than a cloud-only one. The trade is engineering effort: model selection, conversion, quantisation, device testing and the router. High request volume with simple tasks favours the device; low volume with complex tasks favours the cloud. Our pricing page gives the starting points for both the mobile build and the AI features.

A worked example

A field-service company wanted technicians to log a job by talking to the app and photographing the equipment, often in basements with no signal. The first design sent everything to the cloud and failed whenever the network did. The redesign put wake word, command transcription, barcode reading and a damage classifier on the device, queued the structured result, and sent it for a cloud model to draft the report when connectivity returned. Technicians stopped waiting on spinners; the report quality improved because the cloud model now received clean structured input rather than raw audio. The same offline-first thinking shaped the driver app in our logistics work.

Team and timeline

A mobile AI feature set is usually a mobile engineer with native module experience, an ML engineer for model selection and conversion, a designer for the voice and camera flows, and a product owner on your side who can supply real audio and images from the field. Scoping fits a Sprint Zero of ten working days, where we test candidate models on your real data and your real devices before promising anything. A first shippable feature set typically fits a Launch 6 programme; the mobile build starts at $17,500 / ₹11.2L and AI/ML development at $17,500 / ₹11.2L, with the exact figure depending on how many features run locally. After launch, model updates and device compatibility live under a Care Plan.

Before you start: a checklist

• List each AI feature and mark it device, cloud or both, with the reason
• Collect real audio and images from the real environment, including the bad cases
• Define the minimum device class and what happens below it
• Decide what data may leave the device and write it into the privacy notice
• Plan model delivery: bundled, downloaded or both
• Set latency budgets per feature and measure them on mid-range devices, not flagships
• Build the routing switch server-side so routes can change without a release
• Write the evaluation set before the model is chosen, not after

Glossary

• On-device inference: running a model on the phone's CPU, GPU or neural engine rather than a server
• Quantisation: shrinking a model's numeric precision so it fits and runs on a phone
• Router: the code that decides per request whether to use a local or a cloud model
• Wake word: a short phrase that starts listening, detected locally
• Closed-set classification: choosing from a fixed list of labels, which small models do well
• Native module: platform code exposed to React Native for hardware-level work

Related reading

See offline-first mobile apps for the sync side of the same problem, React Native vs Flutter vs native for the framework decision, and multi-model routing for the cloud half of the router.

Put the fast, private, offline tasks on the phone, send judgement to the cloud, and let a router you control decide the rest.