MCUs with NPU for On-Device Speech Recognition: Acoustic Model on STM32N6
The STM32N6 microcontroller with integrated NPU enables real-time acoustic modeling for arbitrary speech recognition. Power draw is just 0.215 W, with a PER of 5.3% on dev_clean, and inference on 500 ms of audio takes only 52 ms. The int8-quantized model runs offline without cloud or internet, perfect for compact devices.
System Architecture for Speech Recognition
The system breaks down into three components to optimize memory and accuracy:
- Acoustic model: Converts raw audio signals into a stream of phonemes. This resource-heavy part runs on the STM32N6 NPU.
- Decoder: Groups phonemes into words using a dictionary and language grammar.
- Rescoring: Refines hypotheses with context to boost the odds of correct results.
We've implemented and tested the acoustic model—the core piece. The decoder and rescoring are in progress; they're lighter on resources.
In demo mode, the model processes arbitrary phrases in real time. An interpreter maps phonemes to words via a lookup table, showing raw phonemes alongside interpreted words. The current setup limits the vocabulary; a full decoder with language modeling will handle any phrases.
Power Consumption and Hardware Specs
During active inference, total power is 0.215 W without tweaks:
| Component | Power Draw |
|----------------------------|------------|
| NPU + Cortex-M55 | 160 mW |
| External memory (Flash + PSRAM) | 45 mW |
| External MCU pins | ~10 mW |
| Total | ~215 mW |
NPU utilization is 10.4%, with Cortex-M55 barely used for mel-spectrograms. Optimizations like core sleep modes, clock throttling, and maxing NPU load are possible. In wake-word → recognition → sleep scenarios, background power is negligible for extended battery life.
Mel-spectrograms from LibriSpeech and onboard mic recordings preserve speech patterns even in noise. Noise filtering is next on the list.
Accuracy Metrics and Performance
Model: 8.5M parameters, trained on LibriSpeech, quantized to int8.
| Metric | Value |
|----------------------|-----------|
| PER (dev_clean) | 5.3% |
| PER (dev_other) | 14.4% |
| Quantization loss (dev_other / dev_clean) | 0.4% / 0.15% |
| NPU inference (500 ms audio) | 52 ms |
| Full latency | 985 ms |
PER (Phone Error Rate) measures phoneme-level errors. Metrics were run on-device: validation dataset through NPU, results via UART. Latency includes a 485 ms "look-ahead" window for predictions.
Comparison with Peers
| Model | Size | PER (test_clean) | WER (dev_clean) |
|------------------------|----------|------------------|-----------------|
| STM32N6 acoustics | 8.5M | 5.51 | — |
| wav2vec 2.0 Base | 95.04M | 5.74 | 6.43 |
| HuBERT Base | 94.68M | 5.41 | 6.42 |
| QuartzNet 5x5 | 6.7M | — | 5.39 |
The STM32N6 model hits 5.51% PER at 1/11th the size of wav2vec/HuBERT. It runs real-time on an MCU using 72 phonemes—a tough feat. Expected WER with decoder: 16–25% on dev_other at 8–16M parameters.
Resources and Scalability
Usage: RAM 18%, NPU 10.4%. Plenty of headroom to 4x the model size or add parallel tasks. Power tweaks will extend battery life further.
In the works:
- Phoneme decoder with probabilistic word selection.
- Language model for context-aware fixes.
- Noise suppression before NPU input.
MCU with NPU Applications
Not for long dictation (needs >100 MB), but beats keyword spotting (KWS): handles arbitrary phrases.
- Smart homes: "Make the living room warmer" → extracts room and setting.
- Data entry: "Blood pressure 132" → number + context.
- Industry: "Speed 20%" at machine tools.
- Healthcare: "Post-meal glucose" on monitors.
- Logistics: "Shelf B12 — 3 boxes".
- Transport: "Head to base" offline.
- Toys: Kids' speech without cloud.
- Assistive tech: Voice control for prosthetics.
Key Takeaways
- STM32N6 acoustic model: 5.3% PER (dev_clean), 0.215 W, real-time.
- 8.5M parameters, int8 quantization with tiny accuracy hit.
- Outshines KWS: arbitrary speech, not just templates.
- Scalable: 18% RAM, 10.4% NPU load.
- Sweet spot: Offline commands and data entry.
— Editorial Team
No comments yet.