ExecuTorch

An open-source runtime and toolchain from the PyTorchPyTorch, prop. n.: The most widely used free software toolkit for building and training neural networks, especially in research; programmers write ordinary Python and it handles the calculus and the graphics chips. project for deploying models on mobile, embedded and edge devices and on personal computers. A model is captured from PyTorch with torch.export as a graph of standardized operators, transformed and optionally quantizedquantization, n.: Storing an AI model's numbers with fewer digits, like rounding prices to the nearest dollar, so the model takes less memory and runs faster, usually with only a small loss of accuracy., partitioned so that supported parts are delegated to hardware back ends — CPU libraries such as XNNPACK, Apple's Core MLCore ML, prop. n.: Apple's software framework that lets apps on iPhones, iPads and Macs run AI models directly on the device, using its own chips instead of sending data to the cloud. and Metal, Qualcomm, MediaTek and Samsung NPUsNPU, abbr.: A small AI chip, or part of a chip, built into phones, laptops and other devices so they can run AI features quickly without draining the battery or using the cloud., Arm accelerators, Vulkan GPUs and others — and serialized as a .pte program file, which a small C++ runtime without any Python dependency loads and executes. Because the whole path stays within PyTorch, developers avoid converting models into another framework's format, a frequent source of errors. ExecuTorch replaced the older PyTorch Mobile, supports language, speech and vision models, and runs on-device features in Meta's apps and smart glasses.

We exported the wake-word model with ExecuTorch and delegated it to the phone's NPU.