Members-Only
Recent Talks & Demos are for members only
You must be an AI Tinkerers active member to view these talks and demos.
Mining Your Own Agent: A Memory Analytics Layer for Agentic Coding Sessions
This talk introduces Agent Trace Signals, a memory layer for coding agents that captures decision-making, failures, and fixes to improve AI-assisted development.
I run a lot of personal projects through coding agents — Claude Code, Gemini CLI, OpenCode — often several in parallel, switching between them across days or weeks. Every switch has the same cost: I’ve forgotten what I decided, why I decided it, what broke and how I fixed it, and what I already tried that didn’t work. The agent has forgotten too — each session starts from zero, no matter how much was already learned in the last one. That lost signal — the record of how we (human+agent) work together, not just what we built — is what Agent Trace Signals (ATS) exists to capture: a memory and learning layer that turns session history into something both I and the agent can draw on.
The core design choice is a shift in memory design from basic entity tracking to complex behavioral and failure analysis: entities capture nouns, not behavior, so they can tell you what a session touched but not how the agent handled failures or made tradeoffs. A single structured-output LLM call per chunk extracts entities, memories, and behavioral patterns (strategy, recovery, inefficiency) together — over complex heuristic pipelines, that one call replaced four separate passes (regex + GLiNER + a verifier + a memory extractor) and cuts LLM calls by over 50% while producing higher-quality output. From there, every chunk/memory/entity/session gets a semantic embedding, plus sessions get a second, independent “structural” embedding built from raw sampled trace text that captures how a session unfolded, largely independent of topic. HDBSCAN clustering runs on both — consolidating recurring memories across sessions into reusable “memory themes,” and grouping sessions by structural similarity — while a lightweight knowledge graph links sessions through shared entities, shared workspace, and shared consolidated memories. Retrieval fuses lexical (BM25) and semantic search via RRF, and the loop closes through an MCP server that lets any coding agent query its own accumulated history mid-session.
Beyond the personal use case, this ended up producing a set of highly practical architectural patterns for debugging and optimizing autonomous coding agents more generally: getting the right analysis granularity mattered as much as the extraction method — per-exchange scoring and boundary detection produced low precision and high noise on real coding-agent traces and were hard to verify, so per-chunk extraction won out. And the memory taxonomy itself needed the same scrutiny as the pipeline — a “timeless fact” category turned out to be unverifiable from a single chunk and got cut after audits showed it had the worst error rate of any type, while model selection (comparing the gemma3/gemma4 families, olmo3-think, and Gemini 2.5 Flash) came down to real head-to-head runs on real traces, not assumptions.
Built with: Claude Code / Gemini CLI / OpenCode (data source) · Ollama + nomic-embed-text (local embeddings) · Ollama + gemma3:12b (structured extraction) · SQLite + sqlite-vec (storage & vector search) · BM25 (FTS5) + RRF (hybrid retrieval) · Streamlit + Plotly (UI) · HDBSCAN (clustering)
- OllamaDeploy and run open-source Large Language Models (LLMs) like Llama 3 and Mistral locally on your machine: achieve private, cost-effective AI via a simple command-line interface.Ollama is the essential tool for running LLMs locally: consider it the Docker for AI models. It packages complex models and dependencies into a single, easy-to-use application for macOS, Linux, and Windows systems. You get immediate access to models like Gemma 2 and DeepSeek-R1 via a straightforward CLI or REST API. This local-first approach guarantees data privacy and security, eliminating cloud dependency and high API costs. Ollama also optimizes performance on consumer hardware using techniques like quantization, ensuring efficient execution even on standard desktops.
- gemma3:12bGoogle's 12-billion parameter open weights model running locally via Ollama, featuring multimodal vision capabilities and a massive 128K context window.The gemma3:12b model brings Google's highly efficient 12-billion parameter architecture directly to local workstations via Ollama. Built on 12 trillion tokens of training data, this model excels at complex reasoning, code generation, and multilingual tasks across more than 140 languages. It natively supports multimodal inputs (processing both text and images) and leverages a 128K context window to handle massive documents or long-form conversations. Developers can run this balanced model on standard consumer hardware to achieve close to enterprise-grade AI performance without relying on external APIs.
- nomic-embed-textAn open-source, high-performance text embedding model featuring an 8192-token context window and fully auditable training data.Nomic AI developed nomic-embed-text to deliver a highly efficient, open-source alternative to proprietary models. Operating with an expansive 8192-token context window, it outperforms OpenAI's text-embedding-ada-002 and text-embedding-3-small on both short and long-context benchmarks. The model is built on a fully reproducible pipeline (offering complete transparency across model weights, training code, and data curation) and supports specialized task prefixes like search_document and search_query to optimize downstream performance in retrieval-augmented generation (RAG) pipelines.
- sqlite-vecA lightweight, zero-dependency SQLite extension written in C that brings fast, local vector search directly to your embedded database.SQLite-vec is a cross-platform SQLite extension designed to handle vector search locally, eliminating the need for complex, external vector database infrastructure. Written in pure C with zero dependencies, it runs efficiently on everything from mobile devices and Raspberry Pis to web browsers via WebAssembly (WASM). Developers can store embeddings, run K-Nearest Neighbor (KNN) similarity queries, and manage metadata entirely within a single SQLite file using standard SQL syntax. It is an ideal fit for edge AI deployments, local Retrieval-Augmented Generation (RAG) pipelines, and serverless applications that require simple, offline-first vector operations.
- HDBSCANHDBSCAN: A robust, density-based clustering algorithm that finds clusters of varying densities and handles noise effectively.HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise) is a powerful clustering algorithm developed by Dr. Leland McInnes and his team. It extends DBSCAN by converting it into a hierarchical clustering algorithm, then uses a technique to extract a flat partitioning from the hierarchy. This allows HDBSCAN to discover clusters of varying densities in data, unlike algorithms such as k-means. It also inherently handles outliers, marking them as noise rather than forcing them into a cluster. Key advantages include its ability to work with arbitrary distance metrics and its minimal parameterization (primarily requiring only `min_cluster_size`), making it a versatile tool for exploratory data analysis and anomaly detection across diverse datasets.
Compose Email
Loading recent emails...