Local Wispr Flow

Learn to run a local AI voice dictation app using Gemma 4 for ASR, AST, and generating code from pseudocode, enabling agentic workflows.

Gemma 4 liteRT-LM ASR AST LLM

Overview

Locally running AI voice dictation app with Gemma 4. It does the following:

ASR
AST
Writes perfect code as you describe the pseudo code

Links

https://github.com/vrinda1994/local-wispr-flow
Local, hotkey-triggered macOS dictation tool using Whisper and Gemma.

Tech stack

Gemma 4

Gemma 4 is Google's most capable family of open-weight models, featuring native multimodal support and advanced reasoning under an Apache 2.0 license.

Released on April 2, 2026, Gemma 4 introduces a versatile lineup including the 31B Dense, 26B Mixture-of-Experts (MoE), and the edge-optimized E2B and E4B variants. Built on the same research as Gemini 3, these models move beyond text to natively process vision and audio while supporting context windows up to 256K tokens. The 31B variant currently ranks #3 globally on the Arena AI text leaderboard, outperforming models twenty times its size in complex logic and agentic workflows. With full Apache 2.0 licensing and support for over 140 languages, Gemma 4 provides a transparent foundation for developers to deploy frontier-level intelligence across everything from mobile devices to enterprise workstations.

https://blog.google/technology/google-deepmind/gemma-4-open-models/

View projects
liteRT-LM

LiteRT-LM is a production-ready orchestration layer for deploying high-performance Large Language Models directly to edge devices.

LiteRT-LM serves as the specialized GenAI infrastructure for Google's AI Edge ecosystem, built specifically to handle the complexities of on-device LLM deployment. It manages critical tasks like KV-cache optimization, prompt templating, and function calling while leveraging the foundational LiteRT runtime for hardware acceleration across CPUs, GPUs, and NPUs. Currently powering features in Chrome and the Pixel Watch, it supports a broad range of open models—including Gemma 4, Llama, and Phi-4—delivering up to 1.4x faster GPU performance than legacy TFLite delegates. By utilizing the .litertlm format, developers can implement multi-modal workflows and agentic tool-use capabilities on Android, iOS, and web platforms with minimal latency.

https://ai.google.dev/edge/litert

View projects
ASR

Automatic Speech Recognition (ASR) converts spoken audio into digital text using neural networks like OpenAI’s Whisper or Google’s USM.

ASR technology bridges the gap between human speech and machine processing by leveraging deep learning architectures (specifically Transformers) to transcribe audio in real-time. Modern systems like Whisper handle 680,000 hours of multilingual data to achieve human-level word error rates (WER) across diverse accents and noisy environments. This tech powers everything from Tesla’s voice commands to Zoom’s live captioning, processing raw waveforms into structured data at sub-second latencies. By utilizing large-scale weak supervision, current ASR models eliminate the need for manual fine-tuning, making high-fidelity transcription accessible for global enterprise applications.

https://openai.com/research/whisper

View projects
AST

An Abstract Syntax Tree (AST) is a hierarchical data structure that represents the logical, syntactic structure of source code by stripping away formatting details.

Compilers, interpreters, and static analysis tools rely on ASTs to parse and manipulate source code. By stripping away surface-level syntax (like semicolons, parentheses, and whitespace), an AST maps program logic into a clean tree of nested nodes representing variables, operators, and control flow. This structural representation enables modern development tools (such as ESLint for linting, Babel for transpilation, and static application security testing engines) to programmatically traverse, analyze, and safely transform code at scale.

https://en.wikipedia.org/wiki/Abstract_syntax_tree

View projects
LLM

Large Language Models (LLMs) are deep learning models, built on the Transformer architecture, that process and generate human-quality text and code at scale.

LLMs are a class of foundation models: massive, pre-trained neural networks (often with billions to trillions of parameters) that leverage the self-attention mechanism of the Transformer architecture (introduced in 2017) to predict the next token in a sequence. Trained on vast datasets (e.g., Common Crawl's 50 billion+ web pages), these models—like GPT-4, Gemini, and Claude—acquire predictive power over syntax and semantics. They function as general-purpose sequence models, enabling critical applications such as complex content generation, language translation, and automated code completion (e.g., GitHub Copilot). Their core value: generalizing across diverse tasks with minimal task-specific fine-tuning.

https://en.wikipedia.org/wiki/Large_language_model

View projects

Local Wispr Flow

Related talks