Deployment Workflow
Train → Compress → Convert → Flash Pipeline
Gather sensor data that mirrors production conditions: same hardware, same mounting, same environmental variation. For audio, record on the target microphone. For motion, record on the target IMU. Balance classes and augment sparingly.
- Record on target hardware when possible
- Edge Impulse Data Forwarder for live capture
- Minimum 500 samples per class
- Include noise and edge-case samples
- Store raw data before any preprocessing
Choose an architecture sized to the target MCU's RAM and Flash budget. Starting from MobileNet, DS-CNN, or MCUNet is faster than compressing a large model. Use the MCU's RAM budget to back-calculate maximum layer size.
- DS-CNN: depthwise-separable for audio
- MobileNetV1 for image (heavy for MCU)
- MCUNet / ProxylessNAS for MCU-native
- Back-calculate RAM from activation sizes
- Prefer flat architectures (fewer residuals)
Train with INT8 quantization in mind from the start. Use QAT if PTQ accuracy is insufficient. Fix the random seed and log all hyperparameters. Validate on a held-out set that includes out-of-domain samples.
- QAT via TF Model Optimization Toolkit
- PTQ: representative_dataset of 100–1000 samples
- Log: accuracy, loss, confusion matrix
- Test with real INT8 inference via TFLite
- Track model size (KB) alongside accuracy
Export the quantized Keras model to a .tflite flatbuffer, then embed it in firmware as a C byte array. The model is stored in Flash and loaded into a static memory arena at runtime — no filesystem required.
- TFLiteConverter.from_keras_model()
- optimizations = [DEFAULT] for INT8
- xxd -i model.tflite > model_data.h
- model_data.h included in firmware source
- Validate flatbuffer with TFLite Python runtime
Include the model C header and TFLite Micro runtime in the firmware project. Allocate a static tensor arena (aligned byte array) for activations. Use the MicroInterpreter API to run inference.
- Static tensor_arena[N] — no heap needed
- MicroInterpreter with MicroMutableOpResolver
- RegisterOp for only required ops (saves Flash)
- GetInputTensor() → fill → Invoke()
- GetOutputTensor() → read results
Measure actual RAM consumption, Flash usage, and inference latency on the physical device. These differ from simulator estimates. TFLite Micro's benchmark_model and PlatformIO profiling give cycle-accurate measurements.
- benchmark_model via serial output
- RAM: Arena size + interpreter overhead
- Flash: .tflite C array size + runtime
- Latency: DWT cycle counter or timer
- Edge Impulse profiler for quick estimate
Deployment Checklist
| Step | Task | Tool / Command | Pass Criteria |
|---|---|---|---|
| 1 | Collect representative dataset | Edge Impulse Data Forwarder, custom scripts | ≥500 samples/class, balanced |
| 2 | Train baseline model (FP32) | Keras / PyTorch | Accuracy meets threshold on validation set |
| 3 | Apply QAT or PTQ | TF Model Optimization Toolkit | Accuracy drop <2% vs FP32 baseline |
| 4 | Convert to .tflite | tf.lite.TFLiteConverter | .tflite file created, flatbuffer valid |
| 5 | Profile on target MCU | TFLite Micro benchmark_model, Edge Impulse profiler | RAM/Flash within budget, latency < requirement |
| 6 | Generate C header array | xxd -i model.tflite > model_data.h | model_data.h contains valid byte array |
| 7 | Integrate into firmware | TFLite Micro C++ API | Inference runs without memory errors |
| 8 | Run on-device tests | PlatformIO Unity test, custom test suite | Accuracy on held-out set meets threshold |
| 9 | Flash to production device | openocd, bossac, esptool.py | Device boots and runs inference correctly |
| 10 | Plan OTA update path | MCUboot, ESP-IDF OTA, nRF FOTA | Model update deployable without device recall |
Conversion Patterns
Standard path from Keras FP32 model to INT8 TFLite. Requires a representative dataset to calibrate quantization scale factors for activations.
- converter = TFLiteConverter.from_keras_model(model)
- converter.optimizations = [DEFAULT]
- converter.representative_dataset = gen_fn
- converter.target_spec.supported_ops = [INT8]
- tflite_model = converter.convert()
Minimal firmware integration: resolve required ops, allocate a static tensor arena, run interpreter, copy input, invoke, read output.
- #include "tensorflow/lite/micro/micro_interpreter.h"
- MicroMutableOpResolver<N> resolver
- resolver.AddConv2D(); resolver.AddDepthwiseConv2D();
- alignas(16) uint8_t tensor_arena[ARENA_SIZE]
- interpreter.Invoke(); // run inference
OTA Updates & Model Lifecycle
Open-source bootloader for MCUs supporting A/B image slots, cryptographic signature verification, and rollback on failed updates. Used with Zephyr RTOS and supported on STM32, nRF52, ESP32.
- A/B image slots in Flash
- SHA-256 + ECDSA image signing
- Rollback on bad image
- Works with Zephyr, nRF Connect SDK
- Can update model C array as part of firmware
Embed model version and metadata in firmware. Track model ID, training dataset version, and quantization config. Log inference statistics to detect when a model update is needed (concept drift).
- Store model_version in Flash config block
- Hash model data for integrity check
- Report version + confidence via BLE/MQTT
- Monitor accuracy proxy metrics (confidence histogram)
- Trigger OTA when drift detected server-side