Building Gemini Local Hub: Stateful Agentic Loops & Real-Time Thought Streaming
Explore building a local AI hub with Gemini. See how to create stateful, low-latency sessions, manage agentic pipelines, and stream real-time model reasoning in Next.js.
Overview
I’ll be doing a live code walkthrough of Gemini Local Hub, a local-first orchestration layer I built around @google/gemini-cli-core. We will dive straight into the Next.js App Router codebase to look at how I transformed stateless LLM CLI interactions into persistent, low-latency “warm” sessions using a Global Singleton pattern. I’ll also demonstrate the architecture behind the asynchronous agentic pipeline (handling both automated “YOLO” tool execution and manual human-in-the-loop validation), and how I split the streaming channels to capture and render real-time model reasoning (“thought blocks”) without polluting the primary UI state.
Video
Transcript
Generated 3 months ago
Summary
Generating a talk summary...
View full transcript
Speaker 0: Hello? Sure. Can you hear me? Alright. Hi.
Speaker 0: So, yeah, I'm Jeff, and I'm, here to cover the, Gemini Local Hub. So I guess just real quick, you know, I mean, how many people here have used, an agent to craft their commit messages in repo? Yeah. Yeah. Right?
Speaker 0: Right. Yeah. Figured. How many people have used agent to, analyze images and run out of tokens? Okay.
Speaker 0: Cool. Cool. Alright. So what I got here is is a basically a wrap around the Jet My CLI. It that allows me to log in with my OAuth token, and take advantage of their request based quotas.
Speaker 0: So what you do is you download, or clone clone the repo. I'll show you that in a second. And then once that's cloned, all you have to do is just copy a couple of files. So I have some scripts that I I wrote. And then the first thing I do is I connect.
Speaker 0: So this is gonna basically tell the hub, like, hello. I am connected to you as a project. And so, basically, the hub is like an API. It's a a Next. Js app that runs, using the Geo Gemini CLI module.
Speaker 0: So once I connect, Motles warm up a session for me. And then, so I have some code changes too. So, like, basically, if I were to commit those, right, I still have a script to do that as well. So commit. Just gonna call it.
Speaker 0: And, obviously, yeah, this is something that I'm sure, you know, you could do through Claude or or whatever. I actually don't use Claude. I'm 1 of those freaks that doesn't use Claude. But, anyway, so this is this is good churn there for a second, to and and maybe, Gemini is is a little slow right now. But you can see it's working here in in the log.
Speaker 0: We'll run on the on this side here. Sometimes it does take a second to to think through things. Ironically, when I was demoing this at home earlier, it was a lot faster. Okay. So that's done.
Speaker 0: So with that feature, I just added actually was a connector to, for this project here. This is a little demo project with DeepSky Observatory. So I do, like, astrophotography and stuff like that. And so right now, all it's gonna do is send an image to the hub and say, identify this. Right?
Speaker 0: So if I say, identify this. Right? It's gonna call the hub. It gives it a base 64 and of the image. And then once Gemini looks at it and assesses it, it'll say, it's the moon.
Speaker 0: It should. Yep. So it says moon. We should've made that font a lot bigger for the demo. I apologize.
Speaker 0: So that's basically the the gist of what it does. You know, there's there's you know, like, it should be able to identify that too. And, you know, I haven't tried identifying multiple items at once, but we could. But, yeah, Orion Nebula. So that's that's a good 1.
Speaker 0: So you haven't seen that 1. By the way, this telescope's really cool. You should get a digital telescope. So yeah. So that's the Gemini Local Hub in a nutshell.
Speaker 0: It's very simple. It's pretty lightweight. It just runs as a Next. Js app on your server or on your on your local. This is the GitHub.
Speaker 0: There's a QR here, code, if you're interested in, checking it out. And that's more or less it. I mean, it's it's it's pretty simple. It's very fast, and I just wanted to make sure that, you know, have at least 1 super fast demo today.
Speaker 1: That's awesome. Thank you very much.
Speaker 0: Any questions?
Speaker 1: What are
Speaker 2: what are these Gemini observing from damages? Is it just the object or what else can it do
Speaker 0: It could do anything that that Gemini can do with image recognition. Right? So, like, if you were to go to, like, the Gemini website and attach an image, like, whatever prompt you give it, it it'll tell you a lot of stuff. Like, so you could have it extract a number of features, identify things. It's pretty cool model.
Speaker 0: It's just just actually using the the 2.5 flash model right now. So, I I know that, like, there's 3.1 flash model up now that might even be better. So, 2.5 flash is just nice because it's it's still pretty good, and and, like, they give you a lot of requests. I'm on a $20 plan. I pretty much never run out of requests.
Speaker 0: So
Speaker 1: Alright. Any other questions? No? Alright. Okay.
Speaker 1: Thanks. Be be be around the house. Yeah.