Making Allies with Adverserial Review

Speaker 0: So let's move HINTS here. Hello Dev everybody. AI, Kevin. Duncan Schouten. I'm from Ben, DC. Speaker 0: It is my 1st time in Seattle, so woah. Speaker 1: It is no surprise that when Speaker 0: you asked a model whether it's doing a good job, it is probably going to say yes. Yen, I am. The problem is that when you're working with the agents, you need a 2nd attention. The run AI. Dan affair is a thing called adversarial review. Speaker 0: It is not rocket science. It is asking someone for a 2nd attention. Specifically an opinion that breaks down the 1st opinion. So what Laes a good E2E War do you look like? I'm not GPT E2E everything re-pings on this slide bottom I want E2E, but what it's pulling out is a few Singh. Speaker 0: Opinion assumptions, specs and code, mismatches, unstated constraints, and there are 3 layers of independence adversarial E2E War it's trying to break the code rather than say I'm a GPT local. This is the prompts AI War adversarial reviewing why am I up here talking about it? Well secure it's actually surprisingly not well used. Models do have some adversarial reviews sense better shares no built in functionality for calling another model to do that dirty work. So this is a prompt that I use. Speaker 0: Don't worry about the prompt excel, it's more about what it's calling out. It's saying check Laes author docs GPT for this and that E2E Ben Real-time AI share the prompt afterwards AI I'll open source a bunch of the stuff in here but it's pointing out a few different things the most important part is that it's calling in an opposing agent if you're working with Claude common codex Vice versa, call in gemini, maybe. If you have a reason for that. Speaker 1: Here we go. Just slower. Just slower. Speaker 0: E2E, many GPT faults here. I I well actually I Shoale the GPT in to call AI from GPT Code and Cloud Codebase E2E. So let's AI through run 1 of these actually E2E like Founding E2E agent implementation. So this is GPT to be a breakneck, not Dev, it is going to be a breakneck video here. I'll try and keep up with my infrastructure for this. Speaker 0: Let's start with creating the spec in Claude Code. This is arguably the most important part where you're defining what the functionality is going to be. I tend to start with a skill called Grow Code Docs by Matt Colcock. The important thing to look at E2E, if you can see it on this GPT, Speaker 1: Joe Speaker 0: E2E see it so I'll talk about City. It's not actually giving a whole lot of content. It's saying I have this general idea for a feature. I John to flesh it out. I give a couple of links to existing attention GPT documentation, that is gonna be E2E only 1. Speaker 0: In this case, the feature is GPT be an SSO module for AI GPT, which we John talk about because no pictures. We take that run it starts asking you questions run by 1 and pulls it apart. You don't have to use GPT testing. It's a really simple thing. You just search, you're asking me questions until we get to a point where run Jake no sense to ask more. Speaker 0: It's gonna ask you clarifying questions adopt your font. It's gonna keep doing this. City keep doing this. It keep doing this till you have a really fleshed out spec that defines what you wanted the code to BDD, Dan that functionality. We talked about E2E, very similar in that regard. Speaker 0: Keeps asking clarifying Speaker 2: questions Speaker 0: run then ultimately once you've got SPEAKER to a good place you've been in the 1st adversarial Review. So I Dan through mine by reviewing the requirements pretty meticulously because this is gonna determine what comes next. I review that spec by hand, Jake edits by hand, and then I pull in a spec review from Run. Codex is gonna go through and see if it can find any GPT, paint, Shoale. It comes back with a good chunk of them. Speaker 0: It says there are a few criticals, a few mediums, a few highs. At that point AI Haskell which you suggest we incorporate and AI. Provide use attention. That is E2E. Ask it to reason about its own modulator. Speaker 0: We'll discuss what to Claude. I'm saying to Claude Code here Lead talk about what's going into it. So adversarial review is not just at the code level it's also run the concept level. Here, I Manya get a parallel review from Fable Tools, because Fable is E2E, really good at Lead, Adverserial review, SWE AI like to Shoale E2E run Gemini. I'm applying Code findings. Speaker 0: You can see the spec, Talk AI you the background. Fable comes back. It actually catches a problem from Codex, and I say, great. Fix it. So now we've gone through multiple layers of review, and the spec is getting firmer and firmer and firmer Martin what I'm actually trying to do. Speaker 0: Excuse me. Like, shares my voice, for example. I apply FableSpecTest, build it into the spec. Now I use my final spec check. Here's where I go Lead and creator test. Speaker 0: I'm Making for the requirements HINTS Making sure that everything is ready to implement. Now AI file plugins CTO for E2E, almost done already. Wow. Okay. Multi Agent, use limitation. Speaker 0: Play it at Track UX E2E. Let's do this. Do it live. Build Dan agent team. I have run GPT. Speaker 0: I'll shares this with you. That plugin Claude adversarial Review in each of those individual agents. The agents are going through the team. Implementation happens. It is duplicated its own firmware E2E over VMs Review on a per agent fail. Speaker 0: Come Track, Making calls to the opposing Models. Track to forth, review, return, problem, fix. You had Codex, evaluates Singh, and now you have a pretty thoroughly reviews PR at the end. I could go and talk about more things, but I don't have CTO. SWE let's focus on questions. Speaker 1: Do you have Agent? New, Sanjay. Yen Joe, Dan you talk to help run search your Staff over CTO AI we're doing questions? Any questions? And the E2E yes. Speaker 1: GPT Going head down there. Allows? Presenter LLM gonna use the adversarial agents ringwormed. I had Tools Shoale of where I would have to balance between time and token usage and also the quality of the stack. How do you balance Staff? Speaker 0: I take advantage Joe full advantage of the generous subscription plans for GPT open-standard the Run. I mean, E2E, it's more about time. I new the front load. I spent most of my time on the SPEAKER planning that I do on the actual Martin. So I will gladly spend multiple days on the spec of War even seeing GPT single line of code being generators. Speaker 1: Yes. Speaker 0: I don't know if that directly answers test, but that's really how I mentally work Speaker 1: through it. Measure AI, cut once. Yes. I guess. Oh, it appears. Speaker 2: And AI them as a, highly limited E2E long Speaker 0: because it can go back and forth. With multiple agents? With multiple models rather? Yeah. Speaker 1: Okay. So I I usually have Speaker 2: it 1, agent on communication and then, you know, and then they can discuss the architecture. Yeah. They go back to food, so I just just ship with 3 AI. And then once if they don't come with a decision, then as I do work with contributing Singh then I kind of do a little kind of shoes, like, how how do Speaker 0: you I mean, Speaker 1: E2E Speaker 0: I Speaker 1: I GPT through adding more of the stuff Speaker 0: that Speaker 1: I cut because we're gonna do it five. But Speaker 0: AI AI-generated you E2E layout 3 or maybe 4 back and forths. Anything more than that and it gets into diminishing returns, you're just GPT, it's gonna find things that aren't really E2E. You use hallucinations. It's much more about the 1st couple of uses, but I find the biggest ROI is building it into the E2E Agent implementation run that it's calling its own review run it really does a pretty solid cycle and it's very hands off. That to me is the the sweet spot. Speaker 0: And it's like I said, I built a GPT for calling Claude Code from Codex run you can do a lot of the kind of stuff that I was showing off in there. I'm happy to talk about it more, but we're gonna be switching over to Rosalia, Speaker 1: hey. Hey, it's Singh, how do you Dev do everyone