Agentic coding at Amazon - Lessons from Amazon and our customers

Speaker 0: AI. Speaker 1: Hello, everyone. Hello. How was dinner? Speaker 0: Good. Good. Speaker 1: We said we need to do some besides pizza AWS the last, Tinkerers Hosting was pizza. Speaker 0: Before Speaker 1: that was pizza. Before that was pizza. So we finally decided to break the mold. So I'm hoping everyone enjoyed their, burritos or their bowls and stuff. So because pizza is great, but sometimes City does get a little old. Speaker 1: So, thank you for coming up. Quick question for Danny of you. Who's your it's your 1st Tinkerers event? Wow. Well, thank you for coming out. Speaker 0: For Speaker 1: those who've never been to our events, we normally do a meetup every quarter, where we do our standard demos. But this time, we, Ron she AWS, for those who don't know, have a mighty presence here in Raleigh. And so, Tinkerers was able to get us hooked up with with Brian and Tech from AWS to give us Kiro and, do his technical seminar. So we're hoping to do more technical seminars, hackathons, a science fair. So we're looking to hopefully do more, but, of course, we have our next meetup in August. Speaker 1: So please be on the lookout for more details for that. Real quick, how many of you are from nPower? nPower students? How many of you know that this space is empower? Right? Speaker 1: So Empower AWS an organization. They're on the 4th Floor. They're an amazing nonprofit. What they do is they specialize in training for community members that are between the ages of 21 to 27 to help give them a Ron traditional path to Out, and we'll get to learn more about some of their students later on. And they also help military vets and their, family who are in transition to help get them into IT. Speaker 1: Sarah and Mick are not here. I thought they could be here, but, they are the ones that provided us the space. So, because this space is scaling, and it's it's it's Date. So we love it. Hands, hopefully, everyone was was able to make it here. Speaker 1: No problem, Trev. Adrian literally just went down his his elevator and got here in stealth mode. He took the stairs. Okay. Oh, there's Speaker 2: hey, Sarah. I was pleading to let everyone in, Speaker 0: but I Speaker 1: think that we might have reached next Okay. Speaker 0: I was just telling Speaker 1: them about nPower, and then you walked in. So do you mind just Speaker 0: Oh, sure. Let me set down how my goodness here. Sorry to Speaker 1: catch off guard there. Speaker 0: Thanks for coming out, y'all. Whoo, Sarah. How are y'all doing tonight? Good. Jonathan, good to see you. Speaker 0: AI don't know if I really need this photo. I Danny, but No. So I'm Sarah Tech. I'm the site director with nPower North Carolina. If you've never heard of well, how many people have heard of Empower? Speaker 0: Okay. Not too many. Speaker 3: Okay. Speaker 0: So we are a tech education nonprofit based out of Brooklyn. Been around for about 25 years, and we expanded into the AI 3 years ago. So we're actually upstairs on the 4th Floor, and we provide tuition free tech education to young adults and military connected individuals who are looking to launch tech careers with no barriers. It's not a catch. We are able to provide out tuition free due to some of our investors and typical, philanthropy. Speaker 0: Truist Hands MetLife are 2 of our biggest, supporters. So we provide entry level education, which is our tech fundamentals program that talk do your comp Paula plus, your tech plus, and helps you kind of break into tech. We're also now recruiting for cloud and cyber. So tech fundamentals is open to 18 to 26 year olds looking to get into the tech space as well as military connected folks at any age. And when we say military connected, that's veterans, veteran spouses, active file Sponsors, and transitioning reservists. Speaker 0: For folks interested in cloud and cyber, you can be 18 plus any age, any affiliation. Tech fundamentals is 20 weeks. Cloud is 22, and cyber is 18 for your Tech plus Hands Linux plus. So we have been around now. We're getting ready to celebrate our 3rd birthday. Speaker 0: We've been doing a lot of community activation in out our space. We've had Jonathan out to do a cool event with our students. You probably saw some of the students from before. These 2026 of our students are here tonight, wonderfully talented. And where's where's my Danny, Carlos? Speaker 0: He's right there. Carlos. Lead. So this is 1 of our alumni who's building so much. You should connect 2026 them on LinkedIn. Speaker 0: He's doing amazing things in the community, supporting his own family's business, doing all the things. But we're always happy to be a convener and a connector. We're always looking for new students, for volunteers, guest speakers, people to support this, early talent development, workforce development. We're really just community builders and strategic partners, and we're open to all. So feel free to connect with us. Speaker 0: AI mean, if you wanna find me on LinkedIn, my last name's Chick. Like the bird, you can't miss me. So thank you so much. Speaker 1: Thanks, Sarah. Speaker 0: Yeah. Speaker 1: Love the shark. Shares. Speaker 0: Oh, no canes. Speaker 1: Because trust me, like, all of us who go to tech Event, right, where we go to AWS things, Jun, all things AI, I expect to see a lot of the AI folks joining our communities and being a part of that. So if this is your 1st tech event, we plan to have you more. So get ready, get Customers, because this is our world, and we're more than happy to have you all be a part of it. So we appreciate you. So I won't, take up too much time. Speaker 1: I wanna bring up Brian. So Brian is local to Raleigh here. He's part of AWS team. We we have Jonathan. We have my man from Kalamazoo originally. Speaker 1: Go Broncos. Street. Speaker 0: Yeah. Speaker 1: Brian. And, where is Who is missing? Where is he? The magician. We have a I met with this earlier. Speaker 1: So If you see a man with an MIT hat and a Stanford shirt, let him know. We've lost him. Speaker 0: I will see him. Yeah. He did that. He did that. He did that. Speaker 0: It's good street magic. Speaker 1: So, we wanna thank you for coming out and and giving us this this seminar. We will be giving out at the end, Kiro credits AWS we wanna make sure you don't just see it, but you have the opportunity to actually go and build afterwards. So, thank you to AWS for the food because they were good enough to sponsor the food, give us the credits to use Esther. And, this is our chance to learn directly from AWS, not just Castro, but what's going on with the software development life cycle AWS talking to Brian, 1 thing you made very clear, you know, the old Sprint file way of doing things was when development took weeks and weeks. Now we're Speaker 3: in Speaker 1: a pace where it's so quick that now he's gonna give us the thoughts on what that software development life cycle should look now in the modern day now that we have such quick adaptation and quick development. So, more than just Carol, but definitely some talk on software Development life cycle. So, thank you, AI. Speaker 0: We're yours. Thank you. Thanks for the opportunity. Alright. I'm terrible with the mic. Speaker 0: I I always kinda drift away, so don't be shy to yell at me if you can't hear me. So again, my name is Brian Beach. I am the tech lead for our developer experience community at Amazon. So if you've worked with Amazon who's who's used AWS? Worked with us? Speaker 0: Us a little bit? Okay. Quite a few of you. If you've ever had a conversation with somebody, a specialist Esther was talking about any of our developer tools, and that's not just the AI HINTS, this is stuff like our CICD tools Hands CloudFormation and CDK PhD and all the other things that developers touch, touch, they were probably part of my community that I run internally. I do all the enablement and activation and train everybody for all the new releases and things that are happening. Speaker 0: I have been spending the vast majority of my time the last couple of years on or Agentic Building on where we are in in the last couple of years and what we AI to call things, what marketing is calling them then, tools for the last few years. So a lot of that is Kiro. Kiro is our developer 2026, that comes in in in an IDE, CLI, and web version Hands also in another very related product called AWS transform, which is very similar, but Raleigh a tool focused on modernization, on taking your old stuff and upgrading it, Drawing that really easy Hands doing it at scale. I'm going to go out of my way to not make this a sales pitch. I'm gonna try really hard not to just show you Kiro. Speaker 0: If anyone wants to be Folder Kiro, great. Come hang out. There's a workshop. The 2nd half of this will be a workshop. You can get hands on. Speaker 0: Hopefully, everyone falls in love with it, and I don't need to do anything to actually sell it to you. What I Danny talk about is everything that we are learning from customers and from Amazon itself internally as we've been rolling these tools out over the last 3, 4 years, 5 years Raleigh Event. We we 1st our 1st tool that we launched was called Coding Whisper. Which Jun the really early days of the code completion automated code completion. It was it was kinda like just predicting the next few tokens in in that era, and this was early 20 22 when that 1st launched. Speaker 0: And I've been a part of this since since the Building, since the very early days of this. So I've watched this kind of evolve and grow from from the early Date. But what AI talk talk about today is the lessons learned from 3 different teams within Amazon. So the 1st of these is called Tech, the Amazon Software builder experience. This is the team that manages all of our SDEs or at least all of the tools that they use and rolls out all the tooling for developers for what is tens of thousands of developers Date Amazon. Speaker 0: Right? There's, AI guess, shares of drivers, developers are probably, like, the next biggest category of people. There's tons and tons of developers. Then there's my group. We are the worldwide specialist organization within AWS. Speaker 0: We are the specialists that are out talking to customers and teaching them how to use these tools, helping them adopt them, helping them roll out, and having a lot of the same conversations that I'll be having. And then the 3rd team that I'm borrowing from here to talk about what we're learning is a team called, AI DLC or the AI driven developer Lead cycle. Kind of this is think of this as AI more focused on the tools. This team is more focused on the thought leadership and the how is the software development life cycle changing Hands how do you adapt your teams and your culture PhD stuff. So I'll pull all those things together successfully here. Speaker 0: I will do 1 or 2 demos throughout. Actually, I have 2 demos prepared as part of this. They're just prerecorded Hands part of the deck. We can demo a little live later during the workshop. And I'll try to Jun use that to talk about how we are adapting our tools, to respond to some of the problems that I'm AI identify here. Speaker 0: But most everything I'm talking about would would, relate to any tool that you're using. AI? If you're using something other than Kiro and and you've got a competitor 2026, City much everything I'm going to talk about AI relate and make sense to you. So, Coding through the agenda Raleigh quick, I'm going to talk a bit about how the SDLC is changing Hands how I see people adapting and changing the way they Workshop. Okay? Speaker 0: Talk talk a little bit about how to get consistent results AWS I hear constantly from teams, from Customers, that they've got small pockets that are doing amazing things and seeing these Tech x productivity improvements using tools, and when they roll it out across the rest of the org, they can't reproduce that success. Hands so, you know, what are we Hosting? What what does that, what does that split look AI, and what are we doing to address it? And then I'll talk a little bit about how job roles are changing. I think everyone probably file Assisted or at least frightened about the impact that this could have Hands start to think about how to reposition yourselves and rolling out what our jobs might look like in the future. Speaker 0: It's early Date. I'm predicting a little bit here, but I've seen a lot of, I've seen the impact on a lot of different teams Date this point. And then I'll also talk about the rest of the tools Hands and how you can adapt the rest of the tools in your SDLC to help. And AI, how to measure impact. I think everyone's trying to figure out how to measure the impact of these tools. Speaker 0: So if the audio works and AI, I'm Join to start with just a quick 3 minute video. This comes from Event, Banner know what re:Invent is? AI don't see a lot of hands rolling up. So re:Invent is our annual conference. We do a couple offering events throughout the year. Speaker 0: We're AI the middle of summit seasons that are happening in a bunch of cities across The U. S. Out the big event AWS in Lead Vegas in November. That's when a bunch of the big launches happen. We usually have 50, 60000 people there, for a huge event in Las Vegas called Event. Speaker 0: And, this is Matt Garman. He's our CEO, the CEO of AWS, talking about, Kiro at the Kiro launch last year. And Raleigh, AI Notes again, I'm not gonna make this about Kiro. I wanna focus in on this team that he's gonna talk about. And dive deeper Speaker 4: into 1 of the stories we've heard in this video AI I think the details are pretty eye opening. Opening. Now this AWS a quote from Anthony, 1 of our distinguished engineers. Anthony Networking on a significant rearchitecture project, and he and the team originally thought that they would need about 30 developers working for 18 months to complete this work. Speaker 0: Now Anthony and the Speaker 4: team were intrigued by the potential of Agentic AI and the potential for it to really supercharge their out. So they decided that they were gonna fully leverage Kiro to deliver the project. It turned out as the team started really digging in and seeing the full potential of agentic tools, it was better than they SPEAKER. And they saw that by leaning in on agentic development, a much smaller team could actually deliver incredible results. Instead of taking 30 developers 18 months to complete the project, they they delivered the entire rearchitecture with only 6 people in 76 days Hands with Kiro. Speaker 4: This is not just the 10 2026 20% efficiency gains that people were seeing with the 1st generation of AI coding tools. This is orders of magnitude more efficiency. Now I think this is a super powerful story, and I've relayed it to a couple of customers over the last month or so. Hands, invariably, I get the question, how PhD they do it? Well, at 1st, it turns out it took the team a little bit of time to fully understand how to best leverage agentic tools. Speaker 4: They started to see, of course, some, efficiency gains right away, but these were honestly a little bit more incremental than transformative. But a few weeks in, they had an moment. They realized that they couldn't keep operating the same way they always operated. They realized to get the most out of the agents Banner changing their workflows, and they wanted to lean Join to the strengths of what the agents were. And then they had to question some of the assumptions they always had about how they wrote software. Speaker 4: The team learned a ton along the way and was able to spot a whole series of new opportunities for how agents could enable teams to ship faster. Speaker 0: The Speaker 4: 1st learning they had, which was how they interacted with these Kiro agents. In the beginning, they would feed the tools Scale tasks to ensure that they got the reliable results Beach, Hands they Paula go back and forth with all Speaker 3: their tools constantly. Speaker 0: But AWS they Speaker 4: learned what the agents were good Hands Web we're not good at, there shares this inflection point shares they moved from babysitting individual tasks HINTS directing broad goal driven outcome. And this is when they saw their velocity on shipping features rapidly accelerate. 2nd then, they thought about moving even faster, and they recognized that they were thinking much too linearly Join assigning tasks to the agent. They realized that the team's velocity was tied to how many concurrent agent Tech they could Jun. And if they can have the agent do more in parallel, they'd go Esther. Speaker 4: So they kept looking for ways to scale out their workloads. Finally, the team observed that as they scaled out, they themselves became the bottlenecks. They had to keep unblocking the agents as they came back because they needed a human intervention or direction. It turns out that the longer they could get these agents to work independently, the better. 1 clear example actually is when the team looked at their commit graphs, not surprisingly, Speaker 3: they Speaker 4: saw their progress stopped when everyone went to sleep. They hypothesized that if the agents could use that time to clear the backlog, the team would be able to wake up in the morning with lots more code to review, and be able to keep moving faster. So we Date back and reflected on these learnings, and we asked ourselves, why can't we have agents that are able to do all of these things? Speaker 2: And Speaker 4: that's why today we're introducing Frontier Agents. Speaker 0: AI. So shares a bunch of stuff that he kind of quickly alludes to. This was early days when we 1st started talking about this story. We've been a lot more open about it. This is a Tech, that's working on a project called Mantle. Speaker 0: This is part of the Bedrock platform that is our model Hosting platform. They were rearchitecting, rewriting a big piece of this Castro year. A bunch of things that he that he Windows loosely alluded to, I'll put maybe a a little bit better numbers 2026. We talk about 10 x productivity improvements. At least measured by commence, this team 800 seeing more like a 15 x productivity improvement from before and after starting to adopt these tools. Speaker 0: This is admittedly, like, you'll see these titles on here are are generally, like, distinguished engineers. These are these people are no joke. Right? They've been there a long time. They're making a ton of money. Speaker 0: They're our top tier guys that they pulled together into this AI team, but they were able to drive tremendous success, compared to themselves and compared to the rest of the organization. And you hear them talk a lot in there about how they broke a lot of things along the way. Right? They had to reinvent the way they worked. They had to reinvent the way they they used tools, the way they structured their teams, and really think about things different. Speaker 0: And that's what I Danny talk about today Hands build on as we go. So let's start with just the basics Hands talk about SDLC. Who here is a professional developer or or supports professional developers, kind of part of the SDLC process. Okay, cool. So most of this successfully will resonate. Speaker 0: AI I'll start this story with kind of a lowly 2 week sprint. AI think everyone's used to this picture. I'm oversimplifying it a little bit here, but generally, Beach beginning of the sprint, day 1, you get together as 800 team Hands you plan the work you're Ron do and you put together some high level designs Hands then the developers go off and they work for maybe 8 days of that sprint Hands then they come together on the last day Amazon as a group. We do some retrospectives Hands showcasing and talk about what we Building and show it off. Generally, group activities Providing individual activities. Speaker 0: Yeah. There's probably some stand ups in here where you get together as a group and and sync. Out, logically, that's roughly how it works. Now forget Web haven't done any AI. Let's pretend AI doesn't exist for the moment Hands I want to do 10 times as much work, AI? Speaker 0: I want to get that Tech improvement in a sprint. Without tools the only way to logically do this is is to start thinking about a much longer HINTS, AI? I've got to go 10 times as long. It looks Building, it's actually only 15 times as long, I couldn't fit another column in there, it looked absurd. Okay, and so you look at a picture like this and it's not even Tech, but you pretty quickly your Windows looking at this and saying this doesn't make any Venue, AI? Speaker 0: I know that I can't plan 20 weeks worth of work in a day. We cannot all get in a room and plan out the next 20 weeks worth of work. That is obviously going to fail. Fail. There is no way that I can wrap my head around what things I'm gonna be doing 19 Hands 20 weeks from now. Speaker 0: That's nuts. I have enough trouble figuring out what the next 2 weeks look like and what's actually gonna fit into a 2 week sprint. This is near impossible. Yet, I I could start to do this. Right? Speaker 0: So so the logical thing is, well, I could do a lot more planning Frontier Hands try to actually plan 20 Workshop of work. AI? This should feel a lot like waterfall. We we learned that this was bad. Web when we went to File and Scrum and the Agile manifesto, it was directly addressing the idea that this is bad, trying to plan this much, even if we had more time to plan, trying to plan this far out, and predict that we're gonna going to get all of these things right so that that 1 can actually work the way we think it's going to work is near impossible. Speaker 0: And so we learned a long time ago that this wasn't making sense. So let's go back to our original picture here, but bring AI in. AI, so we're back to just 2 weeks, but the agents here, this is the Curio logo if anyone hasn't seen it yet, if you haven't actually downloaded Curio out. When they bring this in, I'm effectively doing the last picture, Right? I'm gonna try to get 20 weeks worth of work done. Speaker 0: I want to get a Tech x productivity improvement out of these tools, Hands so I'm gonna try to Ron 20 weeks worth of work into these 2026. And it should become pretty evident that we're actually trying to do the 1st picture that I showed with with all those boxes Join. Hands this is Ron break Danny. This isn't gonna work. You're gonna fail trying to do this. Speaker 0: Esther vast majority of Customers that I work with don't come to that conclusion naturally. Hands they keep doing these 2 week sprints, and they keep sending their developers off with these tools to go do stuff Hands they've planned a tiny fraction of what that developer needs to actually communicate to the tool Hands then they do a terrible job of communicating HINTS the tool and they blame the tool and say hey this isn't working out, AI? These tools aren't working out. I'm not getting what I want. And the reality is what they've done is they've just left all these poor developers out there trying to figure out and predict what the team wants because they haven't done what they needed to do to plan this out. Speaker 0: They haven't spent what would have been 2 weeks worth of work, if you go back to the last picture. And So they're out rolling AI to figure out how to make this work. Hands they're just working with the tool on their Jun, and 6 other developers are working on different parts of the project on their own, and they're all heading off in different directions Hands it quickly turns into a disaster. AI what we start to see people do is to shrink these down. AI logical thing to do is say, AI, I can't plan 20 weeks worth of work, I can only plan about what took shares humans 2 weeks worth of Workshop, so I Date to divide that by 10 Hands I Danny plan about a day's worth of work for this agent as a group. Speaker 0: And then we gotta get back together as a group and review what happened and put it back together. So what we're seeing is naturally the sprint cycles are starting to shares and shorten and shorten Hands go from you know, if you're doing 2 or 3 week sprints, start thinking about how to bring those down as you adopt more and more tools and bring them into the SDLC Hands Windows collapse that. If you're paying attention to that video shares Matt was talking about what the team was doing, this is actually what they're trying to get to. Okay? And this is really aspirational. Speaker 0: I don't know a lot of customers that are doing this. But really what they're talking about is, well, wouldn't it be great if we could just load the tools up AI? File, if we could just build enough of a backlog to keep them busy all day long. And what ends up happening is your humans spend their entire time planning and reviewing. AI humans that are Join to succeed in this are the ones that are already really good at the planning AI, Raleigh good, they're naturally evolving into tech leads Hands doing a lot of tech reviews and code reviews. Speaker 0: Those are the ones that are standing out and really succeeding AWS in the end this is what's really happening. AI is where we want to get to. And again, no one's really here, even the Mantle Tech isn't really here out, I don't think. But that's the Registration. Web just driving these tools constantly. Speaker 0: Okay. So Raleigh quick summary of that. AI? Jun 2 week sprint, we were about 20% planning, 80% dev. We're AI of forced to flip this on its head Jun say now we're going to spend more time planning than Web are executing AWS the execution happens really fast out we still Paula need to be agreed on what we're building Hands that takes some time. Speaker 0: AI what we're seeing is that sprint duration shrinking over time Hands trying to work in shorter, Join in the AI DLC team that I was talking about, they've renamed these bolts. So they don't wanna call them sprints. They talk about them as bolts. It's these really short little Registration, and then we come back and review the work as a group. Okay, the next part of shares, the next thing that comes up is how do I get consistent Presenter? Speaker 0: Hands what we see AWS, generally if you took a team Hands you broke them Join, Danny this is true of just about any organization that I've worked with, you've got over here some early adopters. AI are the people that were always AI the boundaries Join your organization. They shares the ones who were up on a Saturday night playing with new tools, testing new things out. They were the ones who brought these tools HINTS the organization probably before leadership knew they were doing it. Probably there was a negative response to this early on AWS leadership AI really nervous about out stuff you were bringing Join. Speaker 0: But they actually had really good results. AI not Tech x, but really good results. They were showing some great outcomes as a result of them. Eventually the organization gets on board and says 2026 I'm gonna roll that out to the rest of my AI. Hands the rest of the developers AI Coding 1 of 2 2026. Speaker 0: Hey, AI File do it, but I'm not gonna get great results because I really don't understand how they Workshop I haven't been spending nights and weekends trying to figure this stuff out and Lead about City. And if you don't set this up really, really well, these guys are Ron struggle. And then you're gonna find another group of people over here, the Luddites of the world, who just refuse to admit that this is happening. AI? Someone moved your cheese, and I don't want to know about AI, I'm just Join to pretend Date didn't happen and assume it's going to go away real soon. Speaker 0: So what I would encourage you to do AWS, among these people AI your champions. Find the people who are really succeeding, use them to codify the knowledge that they have, and customize the tools for the rest of the group. This is the group who is not going to take the time to customize it. They are not going to go Hands build skills and steering files to make the tools successfully they're June going to use it Jun stick a prompt Ron there exactly as Notes handed to them. AI these ones, AI think investment is wasted shares. Speaker 0: Notes kind of unrecoverable if you refuse to admit that talk of this is happening. So, we've been talking about this for a long time, but it's the same 3 things that we're really trying to accomplish. The 1st 2 are closely related. Esther, Frontier engineering. I have been talking to customers now for 3 and a half, 4 years about prompt engineering and writing a good Ramos, and I've come to the conclusion that no one's ever Ron learn it. Speaker 0: It just seems impossible. Some people got it naturally, right, the the green people in in the last picture. Most people aren't really changing the way they work Hands they're not adapting. The way to make this successful isn't just to, like, craft the perfect Ramos, at least anymore. Maybe in the early days City was. Speaker 0: But now it's just to converse with the agent. Have the conversation, work together, collaborate on the prompt, build it with the Jun, and then let it execute it. But I see some Notes. Most people aren't getting that either. They're still just like 1 Hosting this, throwing prompts in Jun hoping for the best Hands they're excited about the outcomes. Speaker 0: The next 1 AWS context management. AI it have everything that it needs? This is directly related to the prompt, but think of this more like, hey, is this well documented already? Do I have Danny Jun. Md file Join here? Speaker 0: Am I giving the agent what it needs or does it have to go spelunking around the whole code Date to figure out what's going on? AI I managing my context window Web? Or am I compacting City Raleigh inopportune AI because I wasn't paying attention? I'm AI in the middle of a critical moment and it has to compact the current Windows. And then the reality is Notes all developers are good at code reviews. Speaker 0: AI? This is a really unique Scale. This is something that generally would bring you to the top later on in your career. People that would really stand out. The ones that moved into tech lead roles Hands continued Coding up with Jun ones that were good at reviewing Esther people's code. Speaker 0: But not everybody wants to do that. A lot of developers AWS file, AI, I'm not into 2026. I just want to write some code. So Web take those things, and we're heading into the 1st demo of the Kiro Esther, just to warn you. We take that and we apply it to AI Coding. Speaker 0: And 5 coding naturally is this idea that I'm just going to give you a crappy prompt. I'm going to admit that I did that and I'm going to look at the results and then I'm going to give a little corrective action and I'm going to correct you a little bit more, I'm going to correct you a little bit more. AI? And and Event, we'll work our way through this mess and we'll come out the other side of something that that Workshop, something that's kinda like what I had in mind after a whole bunch of corrections. And the reality is you're trying to get a ton of technical debt in AI. Speaker 0: Right? Every 1 of these inflection points is a little piece of technical debt that's probably gonna haunt you later because you're making a bunch of changes. You're working 10 times as fast out you're building technical debt 10 times as fast as well if you're not really cautious with it. And so what we're really promoting Hands what you sort of heard alluded to in the MacGarmin video at the beginning of this is this idea of spectrum Jun development. Don't iterate with Coding. Speaker 0: Iterate on your plan. Spend time working with the Jun. Build your plan together. Then when you have a good plan, execute it. When you've worked the kinks out of the plan. Speaker 0: And you'll see that for for those who have used Kiro L3 Event, AI, when you turn it Ron, it asks you, do you wanna do this in AI mode or do you wanna do this in spec mode? Do you wanna spend time planning up front and then get it AI? Or do you wanna go in vybe mode and just mess around a little bit and eventually get there? So I warned you there'd be a couple of demos in here. This is the 1st piece that Raleigh, this is the critical differentiator AI think with Cura. Speaker 0: Shares a bunch of frameworks for doing this. You can go and get OpenSpec or SPEAKER, but as I said earlier, the yellow people, the ones in the middle of that picture before, they're not gonna go do that. AI? That's what we've realized AWS that they're never gonna take the extra step to go set it up right for successfully so what we're doing with Kiro is just building it right in. So let's just do a really quick demo of this. Speaker 0: I'm Ron build Flappy Kiro, it's actually Flappy Bird out using Kiro. You're gonna do the same thing in the workshop later. Hands so I'm gonna say, Hey, I want to Building read the exact prompt to you if I can see it here because I'm Folder, Building Flappy Kuro game with a Kuro ghost. AI? This is a terrible prompt. Speaker 0: I didn't Jun tell it anywhere near what I should be telling Tech. AI? I June do this in file mode out I'm potentially going to go into spec mode and let it go and turn this prompt into something decent. So PhD 1st thing Kiro AWS going to do is not jump into Coding. It's Join to start working on the plan. Speaker 0: AI? If you're doing a good job with other tools, you're probably already doing AI. You're having a conversation back and forth, iterating on the plan together, working on a document. We've Jun put that in as a guided experience. PhD 1st 1 Lead, hey, are you fixing a bug or working on something new? Speaker 0: Obviously, this is new. I can also skip the requirements. Sometimes you want to go direct to tech design. Not everything needs business requirements. Sometimes you're looking on, like, a performance improvement or just a refactoring that has no business impact at all. Speaker 0: It's just a technical change. Okay. And after, you know, I PhD speed this up a little bit, but after usually about 2 minutes, it comes back with a good plan here. It AWS, alright. There's a high level description at the Jun, and then you get a series of user stories that describe the behavior Hands then a series of exception acceptance criteria that describe what does good look like for this. Speaker 0: And if you're doing a good job, you'll read this and make corrections or ask the agent to make corrections. I'll make you do that in the workshop later. Okay? So you go through this, you spend some time on it, you make sure that this is what you want Hands that everything is expressed the way you expect it to Date. You're much more likely to get to success. Speaker 0: Now, next step here, I am not going to jump directly into coding out, I'm going to go in here and say hey let's work on technical design. Or, again I think it should have said design. Hands it's going to read the requirements back in Notes because I might have changed Jun. I should have changed them. It's very unlikely that it guessed exactly what I wanted from that 1 sentence little terrible prompt that I get City. Speaker 0: And now it goes through Hands then this 1 probably took 4 minutes maybe to go and build. It builds out my technical design AI I know you can't see it Paula, you'll see it Date, But you get a nice architecture diagram describing what are the different pieces of this. Right in here you'll see you've got like a game system, collision detection Assisted, the physics of the game is starting to build out. What is this thing gonna look like? And if we scroll down through this, I think in this case I was doing a AI, Python version of out. Speaker 0: You'll do a JavaScript version later when we're playing. And if you continue down, you'll you'll see this Date, but it does a nice deep technical design. It's at least putting together what Date the interfaces look file. What is the public interface of this application going to look like, Whether those are API or AI? What AWS the what is what are we storing? Speaker 0: What are the tables gonna look like? We're not writing code yet. We're getting really detailed. And then the last thing I'm gonna do is tell it to break this down into a plan. Plan. Speaker 0: So we kind of, remember the 3 things we were struggling with, right? The 1st 1 was all about prompt Jun? We're directly addressing that with this Hands saying here's how to fix it. AI put this whole process on rails, get the design right before we start Networking, we're much more likely to have outcome. The 2nd 1 AWS about Tech Jun. Speaker 0: Well now we've got this really nice context document that's describing everything we need to know Hands we're intentionally breaking this up into small pieces that we can work on Jun at a time with a human in the loop reviewing them. And you Notes, you can see it over on the right, it tackles each Jun of those in a separate context Windows. So it breaks these down into reasonable chunks, things that are nice atomic units, handles them independently 800 you're not Coding up having to compact in the middle of a critical moment when you're about to do this really, you know, you're just getting 2026 the collision detection system and suddenly you gotta compact Hands it doesn't understand anything when you come out of the other side of that which happens all too offering. Cool. I'll address the 3rd bullet when we do the other demo. Speaker 0: So really quickly, there's a lot of confusion about how big a spec should be, this comes up a Jun. This is probably a bit subjective, this is my version of City. Your spec is not your sprint Join. AI team is not getting together and working out the spec. The team is working on a high level design. Speaker 0: They're putting together a Services of features that go into the backlog. The developers taking them and working on it Hands breaking down that and turning it into something that the agent can understand. This is the developer translating those requirements for the agent, not the team kind of designing. It's just way too detailed for for the team to do as a group. And so if you think about it, we often talk about the inner and outer loop of the software developer AI cycle. Speaker 0: The outer loop here is the team working on the sprint plans of what we're gonna do for the next couple of weeks or maybe couple of days if you've already shrunk your your cycles down. And then the inner loop is a developer working with the Jun. And there's probably many of these Banner loops happening with different developers using Kiro, working independently on them. K? And Amazon, there are bolt on things you can add to other tools. Speaker 0: The differentiator here Hands Date 2026 part of the 1st party experience. Okay. So quick summary of this Jun, the team is organizing HINTS into tasks Hands then you're taking each of those tasks and building a spec out of it before you start coding. AI. I didn't show this out you also Ron to be thinking about how to build context for the as is application before you start. Speaker 0: AI done a good job of identifying what that feature looks like. In this Scale, it was only 1 feature because we were starting from scratch, but usually you've got an existing state Date you need to spend some time Building that Frontier really important. Okay. The next question that I get all the time is what's Ron happen with my role? Like how does the job change? Speaker 0: What's happening in the industry? What are the forces being placed on it? And I'm gonna ignore just the, you know, is this just gonna take over the real-world? Jun developers not gonna be needing them? I don't think that's I don't think that's reasonable. Speaker 0: But there are a few specific things we can talk about. So take this we started with this picture. We said, hey. These are group activities. These are individual activities. Speaker 0: When you think about who's in that group, you Date Presenter managers, QA teams, Tech, they all came together 2026 days or 1 day a week kind of on average. Lead then we say well we gotta do this. And suddenly we're calling on those guys all the time. AI changes the way they work, City changes the org structure Raleigh dramatically. You're getting all these benefits as developers, everyone else is struggling to keep up with you. Speaker 0: Hands they're all drowning in it. So talk your product manager. When when the developers were off doing work, when they were writing code, the product manager was running focus sessions, AWS talking to customers about the next feature and testing different ideas with them. There's no time to do that anymore. We're calling them back into a meeting again tomorrow to start talking about the next feature because we built that 1 so fast that they can't keep up. Speaker 0: And so we are starting to see a lot of the traditional roles break down really quickly and blur together. And so we have everyone familiar with 2026 pizza teams? Shares is AI a truly Amazon concept. nPower? AI see some Hands but not many. Speaker 0: Okay. So so we have this kind of fundamental idea at Amazon or have in the past that if your team needs more than 2 pizzas to have a team meeting, it's too big, and we need to split it up. AI? We wanna keep our teams really small. We want AI 8, 10 people in a team. Speaker 0: If it starts to get bigger than that, it's time to think about refactoring, splitting up Notes just the Tech, but the code that they work on. They own an app. We gotta we gotta refactor that app, Tech it into 2, break up the teams, and give them some autonomy so that they can work independently. This is starting to break Danny, And we're starting to blur the roles. So we're starting to move to even smaller teams with the agents taking Partners that work and doing a lot of the dev work. Speaker 0: And then the other rolling starting to blur Hands there's this introduction of this new term called a product engineer. We don't actually use this at at Amazon. This isn't a term that we've introduced, but it's picking up a lot of momentum. We're seeing a big Tech in the use of this idea of a product engineer. Whether or not you use this term or Speaker 3: not, Speaker 0: doesn't really matter. What we're seeing is the same effect everywhere. The the developers who used to depend on a product Banner, let's just take these 2 rolling. Developers depend on a product manager to put together the plan, tell them what to do through focus groups and things. Well, as a developer, I AI go build all of them. Speaker 0: What? Actually, building options is really inexpensive. I can go run 4 agents in parallel and say out build me all 4 of those and I'll Esther. Put them behind a feature flag and we'll figure out which Notes best. I don't need you to go run a focus group to do that. Speaker 0: At the same time, the product manager has out ideas for new HINTS. He doesn't want to wait on the dev team anymore. He doesn't have to. He can start writing code on his own. So we're just naturally seeing these blurring of roles. Speaker 0: And if you think of, like, the the 800 shaped developers of the past, the t is getting really wide Hands the, you know, the AI that's going down is getting really short. The PhD depth that you have in a specific technology is becoming less and less important Hands the breadth of capabilities is really what's starting to expand. And this Event, again, this isn't just Amazon. If you don't know Boris, he's he's AI the lead on AI, which is a competitor Date Tech Kiro. He's saying the same thing. Speaker 0: Right? Tech x developers definitely exist. We see these people, but they're spanning areas of product design, product and business, AI, product and infrastructure. These individual roles are just starting to break down and merge together. We're naturally becoming generalists over time. Speaker 0: AI. So summary of this, AI? Dev accelerates, naturally your planning and review time dramatically increases because you've out to do a lot more planning cycles to keep up with the developers or you're leaving them alone to Jun go guess Date stuff on their Join, which you know is Join to have poor Customers. Hands there's a lot more review, we'll get into that in a minute. AI, these traditional jobs are Ron to start to blur together. Speaker 0: The developers of the future Kiro of become tech Lead slash product managers that are managing fleets of these things. The goal is to run more agents in parallel and run them for longer and longer periods of AI. That's what everybody is trying to do right Jun. And to get the most value out of these things. Okay. Speaker 0: So next 1 here, can the rest of my tools keep up? So PhD developers are going 10 Tech, everyone is going really really fast suddenly, but there's a whole bunch of other things that naturally happened. Are they Ron be able to keep up? Hands we saw the Notes from we saw Anthony Lendori up earlier. He's a good 1 to follow if you're interested in this Scale. Speaker 0: There's a ton of stuff coming out of Amazon, and he's 1 of the leads on out. It. The other guy is Esther out, Joe Mag. He's also on that same team that the CEO was talking about Date the beginning. He was 1 of the other, distinguished engineers that worked on this. Speaker 0: And he wrote a great series of posts out, AI of comparing AI to run a Tech Join in software development with out racing. If you're driving your Camry on the highway, there's not a lot that goes into keeping the car on the road. Lead kind of just gravity gets the job done. When you start running at 200 miles AWS hour, things change. You have to worry a lot about downforce and all the controls that you have in place to keep that car on the ground Date be able to run at those speeds. Speaker 0: And so a bunch of these come up. So if you're running Tech, Web, you're probably generating issues 10 times as often Hands this assumes that the agent AI lot of people would debate that, I've come to the conclusion that the agent is much better than me at this Join, some people feel they're not quite as good as 2026, but let's say on average, AI, you're gonna have about 10 times as many issues, I issues 10 times as often because you're doing 10 times as much at breakneck pace. So you've Ron to be ready and prepared to start putting the controls in place to address this. And humans are very quickly becoming the bottleneck everywhere. We heard the CEO again talking about this at the beginning. Speaker 0: So the 1st 1 is your c I see 800 pipeline. Can this thing keep up? Right? I commit my code. It's gonna go through a bunch of reviews. Speaker 0: Probably the Building in unit test isn't a big deal. If you're doing integration testing and stuff and you're queuing up a bunch of other builds waiting to get in while the 1st 1 finishes Hands suddenly you're doing it 10 times as fast, well, you're gonna be way behind. Right? If I've gotta wait an hour for 1 build to finish before mine June get in, in the Castro, maybe I was committing once or twice a Date. Now suddenly I'm committing 20 times a day. Speaker 0: And so stuff is backing up all the time. If I keep working, the agent's gonna pull off, like, 2 or 3 or 4 days worth of work while it's waiting for that last thing to make it through the CICD pipeline. And now I'm gonna be way out over here and I'm gonna have to rewind a bunch of stuff to go and address the issues that came up. So we wanna we've been talking about shifting left for years. We wanna bring as much as we Castro close to the Jun so that as it's working, as it's building, it is then able to go and run those tests and see those results in real time before it commits. Speaker 0: If you've got to put it in a pipeline and queue it up and back it up, it's gonna you're you're just naturally gonna fail. That's assuming you don't have humans in the loop. If you've got a bunch of human dates in there doing approvals as part of your CICD pipeline as things are moving through, you know, if Joe goes to lunch and you don't get a response for 2 hours and stuff is backing up, you might commit you might commit weeks worth of work before you get the results of the last thing. And that's AWS miserable way to work. I've been in companies that work that way. Speaker 0: It's terrible. Okay. So we're seeing this all over the place. We're seeing this stress in a lot of open source projects right now. This is this is a quote from 1 of the maintainers Ron cURL, around the cURL utility for, you know, essentially doing a web URL. Speaker 0: They are getting so many commits that they are just drowning in them. So So many more people are able to write AI. The developers that are contributing are writing much more code. I have a little feature that I wish Tech did, I wish City did this for 5 shares, well hey, it's not gonna take me in, Jun out asked this tool to go do it and commit it. So they're drowning in it Hands the pipelines are just backing up all over the place, struggling with this. Speaker 0: And it's not Jun of course about the CICD process, shares lots of other things that are breaking Danny, the other big 1 that I'm sure you're thinking about is Hosting. AI, We are very much at or maybe even past the point where I can review the code as fast as this thing can generate. It is near impossible to keep up with it. They're writing so much code so fast that you're delaying the thing. Right? Speaker 0: You've got all this idle AI. You've got GPU's just sitting there doing nothing because you've got humans renewing the code that Hands finished writing. So we've out to get the tooling in place to raise the bar Ron our testing 2026 to take some of that load off the humans. I'm not saying don't review the Coding, I'm saying do a lot of automation so that it makes it easier for the humans to review the code. Right? Speaker 0: Don't misunderstand what I'm saying here. Code is really cheap. In the past, we might have been okay with a 70% code coverage on our unit Esther. That was a reasonable target. Maybe a Lessons organization AWS comfortable with 50% AWS they wanted to focus on feature development and it wasn't worth the time to invest here. Speaker 0: It's effectively free now, investing in unit tests. Everybody should have a really high AI, 99% code coverage rate. That's the 1st Tech, but you Danny also introduce a bunch of other test types. And, the other thing I'd do as a quick demo is, Cura is investing really heavily here, and we've introduced this, this concept of property based Tech. Has anyone ever done property based testing? Speaker 0: This isn't new. I've never actually written a property based Tech. You know, I've read about them occasionally in my career. A property based test, the PBT, is very similar to a unit test. AI? Speaker 0: This is an automated test that you can run. The difference is in a unit test you are usually testing very specific conditions Hands scaling shares a successful Scale, here's an unsuccessful case. The property based test is Join to run a generator to test lots of edge Scale, Ramos fuzz the inputs Hands give you a bunch of inputs to test edge boundary conditions to see how it responds. So take the simplest example would Beach, I'm running a validator for maybe password rules. I have to have a capital letter, AI, I have to have a symbol, I have to have a lower Scale Esther, and I might run 1 good password and 1 bad password. Speaker 0: Then you put your app in the wild Hands, you know, someone from Sweden or someone from Asia puts in a character you've never seen before and your app just blows up and dies. And so you start collecting these and adding new conditions and testing new things that you weren't testing before. Property based tests are Ron generate lots of crazy use cases for those. And then when it finds a problem, it's smart enough to start going through that and AI to figure out what of the many things that I put in this random noise is actually causing that. Let me let me start to test different conditions Hands figure out what's actually happening. Speaker 0: Oh, it's this character, when you see City, that's breaking. So Jun more quick Esther, hopefully you can see this Jun. In this case, no more Flappy Bird. It's Raleigh hard to find a boundary condition in Flappy Bird for inputs anyway. So what we're doing here AWS, this AWS this Software that I'm looking at, this library, is processing markdown documents and, pulling out all the links, cataloging all the links that are in there. Speaker 0: But we wanna be able to canonicalize them, normalize them so that we can find 2026. How many how many times does this particular link occur? And that should be unimportant. If I try to normalize the same URL over and over again AI should get the same results. So I run unit Esther, great, all my unit tests are passing, everything's looking good, all the things that I thought of and wrote are good. Speaker 0: Now let's look at a a property based test variant of this. So I'll run now just the property based Esther, and takes a few seconds longer. Right? This is a little bit slower. And you see it quickly found an issue in here. Speaker 0: Hands then normalized AI Hands said okay, I understand what the issue is. The normalization is not idempotent Hands essentially if I have multiple trailing slashes, the code is only removing 1 slash. It's saying hey, I'm Ron remove extra slashes on the AI, but that results in me having a 2nd slash slash and then 0 slashes. So this is 1 of those edge conditions that was going to break it. You put this out into the wild Hands you're going to get complaints from your customers saying, hey, it's not actually counting things AI, this is the same URL 2 AI, you're not noticing that it's the same. Speaker 0: These are the kinds of things that you can start to pick up by expanding your your test window and looking for new offering things to test for. Cool. Okay. Last piece of this, last thing that I hear from customers all the time AI how do we measure the impact of this? How do we measure the impact that AI is having on my organization? Speaker 0: Hands the 1st thing I Ron to put out is shares a bunch of metrics that come out of the tools, out of the IDEs, AI, Lines of code is 1 of them. Accept rates is 1 of them. Mostly these are equivalent. We've known for a long time that lines of code is useless. Accept rates are quickly becoming more and more useless AWS, made a lot of sense when things were atomic. Speaker 0: You either accepted or declined the thing that it suggested. Now it's way bigger than that. You're getting hundreds of lines of code back in, you're maybe making some modifications. Is that an accept or a decline? Presenter really make sense anymore. Speaker 0: So if we know that the metrics that are available in the IDE aren't going to answer the question, where are we Ron find them? And to do that, we need to instrument a bunch of other things in the organization. AI need to plug into my CICD AI, I need to plug HINTS, AI testing tools, into my security tools, and get a bunch of feedback from all of them. There are great frameworks, DORA and Scale, AWS 2 of them. DORA really focuses on the metrics that are coming out of the CICD AI. Speaker 0: Space builds on that and expands it quite a City. Includes things like just are the developers Danny, what's their sentiment, stuff like that. These are Date, but they're really hard to implement. You gotta go instrument stuff all over the Organizers. It's gonna take a while to do Partners with you, but expect it's gonna take 6 months to get this stuff in place before you can measure it. Speaker 0: And once you do, you are going to be doing this while you're adopting these AI tools Hands the foundation Jun really shaky. You're changing the foundation that you're trying to measure and get a baseline Ron, so things are Coding really quickly. So where do we start? Where does Amazon start? There's a couple of things here. Speaker 0: So the adoption metrics is the 1st 1. For adoption metrics, we started years ago just looking at installations. Paula we looked at was, is the tool installed or not installed on a developers machine? That was a great start Hands then we expanded Danny said, Raleigh, let's move to weekly active users, not just do you have it, but you're actually using it at least once a week. So think of this as AI installations was 2023, 2024 AWS were looking at active Customers. Speaker 0: Last year City was all about active AI touches. Suddenly, everything AWS AI HINTS, right? You've got your IDE with AI in out, but you've also got your CSC AI, Hands all the other tools you touched throughout the day Date AI AI it. So we want to measure all those touches Hands start to look at how that's changing over time. That team I talked about at the beginning, the Mantle team, they're doing about 30 x the average developer of AI touches per day. Speaker 0: They're really using these tools heavily. The 2nd 1 that we do is a bunch of, qualitative feedback. AI the developers happy? How do you feel about it? Do you think you're moving faster? Speaker 0: Regardless Web whether you are or not, there's lots of studies that will kind of disprove some of these. And then the last 1 Ron the important Notes, those impact metrics, things like PR cycle time, how long does it take you from taking an idea all the way through to committing a PR or taking an issue that was reported through to resolution Hands also onboarding time. Right, Building a new developers, it should be a lot easier if you've done a good job of setting up for success and educating these tools on how to do HINTS. They should be able to 2026 talk to speed much more quickly. 2026, we've been doing this for Speaker 3: years Speaker 0: Hands then, go back to late 20 24, Lead of the year, the ASVX team that I talked about at the beginning that drives all of these tools AWS presenting this, all this great feedback Hands reductions in PR times 2026 our board and our leadership, and the response AWS, Beach couldn't care less. None of that stuff you're telling me means anything to AI. I only care about the revenue impact. And so what we've pivoted to start to do recently is this idea of cost to serve. AI to serve is a core Amazon concept, it's been around Folder. Speaker 0: It is Join retail. What does it cost me to serve you a package? What does it cost to get a package to your home and deliver it to you? We've adopted that, or the Tech team has adopted that, in the Castro serve software variant. And essentially what this is is we take all the across to build the application and all the cost to host and run the application and divide it by the number of of deliveries. Speaker 0: How many things were actually delivered? And depending on the app type, delivery might change. But think of it like, hey. This is a deployment in a microservice. Maybe it's just a PR commit in June a large monolith that we don't constantly push to production. Speaker 0: AI? You're paying attention, if you're thinking about this and AI of thinking forward, you can imagine how this would be Beach. So if you're unaware of this idea of good Partners law, AI, as soon as you talk a measure and make it a metric, people are going to start gaming it, it's going to suddenly be a terrible measure the next day. And so the idea of this is people are going to game those metrics. If I focus on PR cycle time Hands that's the divisor, the number of PRs becomes my divisor in my equation, well I just gotta Community a lot more often. Speaker 0: I just commit smaller things much more offering, AI look like a superhero. So AWS Esther stuff that I talked about at the beginning, the DORA metrics, the Scale metrics, they're still Event, they just become your control metrics, what we call tension metrics at Amazon. Those are the things that you're measuring whether people are gaming the Assisted. But this is what we're reporting Ron. This is what's in Jun that's the really important piece of this. Speaker 0: AI. Cool. We're, we're just about there. So let's re recast a couple of the key takeaways of all of this, the things that I want you to walk away from this. Your team structure, the the SDLC structure, your team structure AWS going to have to adapt. Speaker 0: If you are still doing things with these tools, if you're adopting these tools and continuing to do things the way you did it before you had these tools, you're Join to have issues. You are probably asking your developers to do far more work than they have planned for Coding, Jun they have a good Lead idea Hands understanding of Hands that's Join to catch up with you. Paula, the rest of the org is going to struggle to fill up with them Hands you're going to be stressing out your QA team, you're going to be stressing out your security team, you're going to be stressing out your product managers Hands you need to think about how to adapt them Hands bring them into the fold. AI. You're also gonna break all your tooling along the way. Speaker 0: Expect a lot of the other tools to break and not be able to keep up with you. And be really wary of some of these Agentic trap things. Don't get all hung up on, lines of code written by AI because it's meaningless metric in the 1st place. That's not gonna get you anything meaningful. As I said early on, all the stuff I'm talking about is all derived from different teams within Amazon. Speaker 0: All of them have done deep dives on each of these different subjects. So I will give you links. I always have a bitly link that will will link all of these things. You can go see them. I know it's it's hard to get them from here. Speaker 0: I'll throw that up in a minute when we do the workshop. But these are talks from the ASBX team, the AIDLC team, my own team SPEAKER 2026 Event Hands Drawing like that, along with a bunch of science papers and Jun other blog posts that build on these concepts if you wanna read more. And so with that, I think I'm just about Date time if I Banner this Web. Yeah, so talk you, I think you're going to come up and do a little demo of Kiro Hands some of the stuff you've been doing here Kiro? Speaker 1: Yeah, so we have 1 that empowers students, Carlos, who has been building outside of Kiro. And when we announced this, we basically said, hey. We'd like an empowers team to come out and present what they Building. And Sarah said, you gotta meet Carlos. And I'm glad we met Carlos. Speaker 1: And he's gonna go ahead and and show us what we got before Brian goes into this session. Speaker 0: AI. I think you need that 1. Yes. Speaker 1: Alright. This is gonna be my 1st time speaking up here. I've never spoken in Speaker 0: front of a audience. Do it with the mic. Alright. Speaker 1: Thank you. Speaker 0: So Carlos, just while he's setting up, we met him, a week ago to give him the Cura Fritz, and, I was really impressed with the, project 800 showed. He's got a lot of agentic AI experience, and he was doing, like, a really cool stuff with graph based retrieval for his agentic programming tool. So, I think you're gonna have a really good perspective on, some of the AI principles Jun Kiro versus some stuff you've probably used already with, like, quad code and codex and things like that. So, I think he's got a great project. Speaker 1: AI. So, AI, my name AWS Carlos. And so the reason why I built this application was because my dad is in the foreign business. I was working Date 800 Hands nPower helped me actually get to that step. And before that, I was working or I went to that step. Speaker 1: And before AI, I was working or I went to study at June, and that's where I learned a lot about the life cycles of Development. It was City was a boot camp, but it actually helped me learn a lot. I was able to connect with a lot of different people as well. And so AI used a lot of those concepts 2026, pretty much kind of just think through a lot of the issues that, we would have, you know, finding file leads or anything like that. So AI figured out that every county has public APIs for pulling HINTS. Speaker 1: And a lot of general contractors or HVACs or anybody that has to pull a permit, will do that or should do that. And, so I had to come up with this. Basically, it figures out so every county has a separate way of pulling out the APIs. It's free. You don't have to, like, pay anything or anything like that. Speaker 1: But they have to pretty much put in a couple of AI. So a bunch of these, they have to put in the square footage of what they're going to be doing, what part of the phase that they're in for the construction Building. And sorry. I'm losing track right now. But there's this ID as well for all of these clusters that we get. Speaker 1: I can map out to see where they're going to be Hosting from. There's different areas where I'm rolling. So Cary, Raleigh, Date County, Apex, Durham, Chapel Hill, Mooresville, and Hollow Springs. They all are very different for setting up the API. AI, AWS able to help me out with, setting that up. Speaker 1: So I can grab the ID number, and this is going to be Web County. So we go to the Join Community public, we can search out to see where the ID is Date. We can do an exact phrase. And the reason why I don't just use the portal is because it'll throw from 1997 all the way to AI 2026. So just just going through that is Ron take a lot of time. Speaker 1: And I needed a way to be able to go door knock, quicker Web there's gonna be Registration happening. Shares gonna be people, neighbors, seeing that happening. So AI actually was able to get a couple of Join just by doing this. Sometimes you'll be able to see the contractor's phone numbers, emails, or different things like that. And then you also kinda get an estimation of, like, what the general contractor. Speaker 1: But sometimes it's not gonna be accurate because they try to, you know, go a little bit lower or sometimes go a little bit higher. Out you're able to see exactly what the description is and, like, where they're at in that phase. Jun sometimes City wouldn't have worked out for me if they're just beginning. You know, they're just to get to the flooring is kind of, like, 1 of the last steps. So after, like, painting, plumbing, and different things like that, that's where the flooring guy comes in. Speaker 1: That's where I come in. And we do the estimates. AI now, I'm actually working on creating a hardware, system to be able to communicate to the servers that I have at the house. I've been putting on been putting in my business logic in those servers. They have VMs inside of that. Speaker 1: So that way, the business logic and the business, secrets pretty much just get Date in there. Eventually, what I'm hoping to do is create a way for the hardware to communicate to the servers and send out emails to different types of vendors for specific, LVP colors or hardware engineering colors or anything like that. But yeah. So this is what I Hands built. What I also set it out to be was that I didn't wanna drive out to 1 place and then to another place. Speaker 1: If I can cluster them in the same neighborhood, in the same area AI ZIP ZIP code, by street, and that's what Kiro helped me to do. So AI was able to look at the schemas of the APIs, to cluster them exactly where it would be Join PhD I just did a couple of fun other things too shares out can also look up to see where Reddit, people have posted in Bali, Morrisville, and all those different types of areas, just so that way it actually is able to find certain things as well. You're able to actually click on these as well. It'll get to that specific part. Sometimes, like this 1, Date says 7 months ago, but, you know, it's still working. Speaker 1: But the 1 thing that I actually had built to help me do this was the tools. I have been building Kenbun Jun pretty much for, like, the last couple of months. I have open sourced City, so I that's the media thing that I did. Don't worry. This is, what's it Scale? Speaker 1: Encrypted. So but I used this, these tools to help me build some of the systems that I am building, just so just because I kept it kept, not having the same assistants. And yeah. Sorry. So but, yeah, so it's been City, pretty nice to be able to build this secure Hands I'm hoping to keep advancing in this. Speaker 1: I have the way that KenBun is set up is that I have it inside of PhD Docker container on my other PC Web it hosts, the local models, and the database that I have collected for when certain programs have failed, it has rendered, what the mistakes were, and that way so that way, next time it has, like, a similar issue, it can reflect back 2026 that concept, to be able to improve on. So and it's almost exactly what, it follows a plan as kind of what SPEAKER does. It's very similar to that. And you guys can also just see, how it works. So, it works in different systems, and you can attach, it works on HINTS gravity, claud, and you can just, set up an MCP Speaker 3: for it. But, yeah. Speaker 0: Can I ask you a question? Speaker 1: Yeah. Speaker 0: When you were using, Kiro, did you use the spec driven mode, or did you use the vibe AI mode? Speaker 1: So I I wanted to try it out to see what it would Jun, and I used the vibe coating 1 1st. Mhmm. So, yeah, I was able to go so this is the 1 that I had in Speaker 3: Brazil. Speaker 2: Ron another app and then copy and paste over here, or did you just start there? Speaker 1: So I I just started here. So I was I wanted to set up the MCP 1st, for Kidbun because AI have I felt like it helps me a lot. I use anti gravity, mainly, but whenever I AI, like, a situation where the AI hallucinates, because that that was a big problem, I had to create a way to be able to Kiro of, like, isolate that rather than out just code inside of, like, the the files. So when I started to notice that, I had to create a system for that. So I set up a, a Docker container Hands so that the code that the program or the AI, creates is isolated from that. Speaker 1: And then from there, it has, like, another, tool where Date incorporates that code and test it out before it actually changes any of the files. Out Lead see. So AI for here, it says think about tools, right? So AI asked, June Banner to do a full Lead of the code Date, and understand what technology we are using and how the APIs are I just wanted to see out to set up. So initially, I had to set up the MCP. Speaker 1: It went in and PhD a strategy Scale it. There's this tool that I use called Registration Scale repo. So it doesn't so for the scan repo, it takes the, high level files and understands what it could be inside of there Esther than having the context, of the AI be Coding and just completely forget. So initially what it does, it takes all of the, MD files or the files that it can AI, and then it just sorts them out. All the way till, pretty much it captures everything. Speaker 1: So then it followed now let me feed this into Gemini for a deep architecture analysis because the Gemini has a context window of 1000000 tokens. Okay tokens. Okay. So then it indexed a 134 files into 556 semantic chunks. Chunks are pretty much the summaries of whatever the files were. Speaker 1: Okay. So then after I called that, I have this tool called Beach with Gemini. That's where the 1,000,000 context window comes in and is able to grab all that information, critical findings, optimizations, and, best practices. So 1 thing that I have to do is communicate back to the server so that way Tech keeps iterating until a code change gets approved. Once it finally gets approved, then it goes back into the actual files for it to be changed. Speaker 1: Okay. So there's this thing called a Date to hive mind, and that's just basically saving the thinking logic of how it got to certain, how it got to certain, phase where it did get approved, and then that gets saved into the database. Okay. Speaker 3: So alright. Speaker 1: So here AWS another, situation where can we have the Date local use Chrome as well? Hands so each of these, tools, they have, a description of them and how to use them. Everything else is going to be AI of the, documentations. Hands I'm still working on the open source for City, but if you guys ever do decide to get it, just let me know if there Date gonna be some mistakes AI I know that there will be. But 1 thing that I kinda did was inside of, actions, it created a runner. Speaker 1: So it's double checks pretty much some of the things that, the AI puts out. So back when I was getting this, we get a description of what happened Hands the errors that it creates. And then from there, it creates those Tech, that Brian was talking about. And I didn't know about the specs before, with Cairo, so I'm eager to test that out. And, but, yeah, sorry. Speaker 1: That's all I have. Speaker 0: Thank you. Has anyone heard of SAS is Lead? This is a great example of that. I think though watching that, AI think what stands out, 2026 me get my my thing back up here. The early days of I've been I've been at Amazon now for about 12 shares. Speaker 0: And so the early days of this, we talked a lot about how kind of just democratizing cloud was. Right? It used to be a small Esther up Lead could not access anywhere near the infrastructure reliability that a big company could build in their own data center. And the cloud kind of just leveled the playing file. Right? Speaker 0: These small start ups had access to the same thing that the big guys were playing Ron, and City file like the same thing. Right? The stuff that you're building, the things that you're doing, a big company would have access to that kind of data and would would be able to buy an app that would be able to do the segmentation and figure out where opportunities are by scrubbing through public records and stuff like that. That wasn't something that a small business owner could go and build on their Jun. And Notes suddenly Date is. Speaker 0: AI? This is something you can throw together and, really kind of just levels the playing field again, which is so cool, I think. AI, so we're Join to shift gears and, well, as Danny tells me we're not going to do that, to the workshop, right? We're good on that? Okay, cool. Speaker 0: AI wanted to make sure you didn't have anything else planned that I wasn't thinking Ron. Of. AWS that true? Did did anyone not get a code? AI. Speaker 0: So okay. Sounds like there's 1 or 2 issues. We'll out work those out if you're having problems. I didn't go with a QR code. I know it feels like we always put QR codes up, but these are things you're Ron do on your laptop, and then I'm stuck out there saying, like, what am I gonna do with that QR code with my laptop? Speaker 0: And then I'm trying to, like, take a picture of it and transfer it, and it's miserable. So I I thought bit.ly links feel a little dated. They're not that cool. But, hey. I thought this would be the easiest way to communicate it. Speaker 0: If you go to AI, I'm not very creative, it'll take you to this little page with the different links that you'll need. Okay. The 1st Jun, this 1st link will take you to just another page Jun my blog, which are those I Ramos I'd get you the links with all the detail. I do some version of this talk a lot, so those are the ones that were on that last slide. And then I think everyone's found their way to out. Speaker 0: Did everyone who's going to do the workshop get Giro installed? Anyone having trouble raise your hand and we can come unblock it. Cool. And then the last thing we're gonna do, the last link here is just a, GitHub link which will take you here, and this is the Workshop, let me June a little City shares, this is the Raleigh workshop that we're gonna do. AI I'll come back to this in case anyone missed City. Speaker 0: Give people a 2nd to get this up. I'll warn you it usually takes about an hour to actually get to the point where you can play Flappy Hero. If you wanted a criticism of SPEAKER Hands Development, AI, it's that you're gonna spend a little time thinking about your design. I'll slow you down a little out, but that's probably a good thing. We should all slow down a little in this crazy new world. Speaker 0: I'll also say Esther in development is AI overkill for the AI Hero game. If you Event into bytecode and gave it a 2 sentence description of Flappy Kiro you'd come out with something pretty similar to what we're actually Ron Building. But I want you to get the mechanisms without doing something overly complicated. Okay. I'm assuming no one's having any issues. Speaker 0: So this is the Fondikiro Workshop. This is a fairly new version of City, so brush your fingers up everything goes well. Hands, we're Ron go Hands build a Flappy Bird like application. Hands so it'll offer you the Esther Deep AI or Speed Ron. Go to Speed Run since we're a little pressed for time here. Speaker 0: The Deep AI, you're Notes Ron miss anything except that at the end of each phase, Kiro would kind of go through the requirements and make some suggestions for things that it could improve AI really encourage you to spend time reviewing the requirements and design docs and stuff like that. AI, what we're Join to do is go through these 7 Tech. AI, we're just going to start by exploring a little bit and Networking, then we'll go and generate the requirements, file the requirements, build the design, create that task list that you saw, implement the 1st talk, and then run the rest of the tasks out. Okay, AI slow down a little bit while everyone's catching up PhD I do AI of a little bit of a tour here, it's getting kind of absurdly big out I know you can't see this. So when you 1st start this up, you're probably in auto mode. Speaker 0: AI? Auto mode is managing the the model for you and making decisions behind the scenes about what model is probably best for the Workshop Building on the complexity of what you're working on. That's probably the best Esther. It's more than enough for this workshop. It's probably more powerful than you need for this workshop. Speaker 0: If you want though, you can turn this off and go into specific models. So if you want, you could across, like, Opus, you know, we have Date AWS the latest now. For some of these 2, for, like, the Opus series of models, it's 1000000 token Join context window. For some of the others, it's only 2026 Date model depending on the model and what it supports. And for the OPUS ones, you can also set the thinking level. Speaker 0: So if you you can set this AI, I think the default is extra AI, which feels extra slow AI. So you don't want to ratchet it back a little bit depending on the complexity of what I'm working on and how much I actually want it to think. We do have all those controls here. Okay. And you also have a bunch of features for context management that you've probably seen from from other tools where I Jun pull things in and, I can, like, name specific files that I wanna pull in or or tell it what servers to June, things like that. Speaker 0: Tech that AI, I'm assuming everyone's roughly caught up, I am simply going to say, Date it says this right at the very end, when you're Lead, do slash Jun start the workshop. And this will kick off the workshop, and Lessons gonna guide you through. I'll do this with you. But Kiro will Raleigh hold your hand if you just want your credits and AI get out of here. Can't do this at home. Speaker 0: Kiro will guide you through it at home. AI. So out says, hey. What do you want to do? Do you want to do the deep dive or the speed run? Speaker 0: I want to do the speed run. You don't have to type this exactly right. Right? Anything. These tools are super forgiving. Speaker 0: The models are great at figuring out what you meant. Oh, let me go back to auto mode. So I'm doing what everyone else is doing. I want to nPower good best practices here. Alright Hands the 1st thing it's going to do is take you through a little bit of a tour of the features of AI. Speaker 0: Show you around a little bit. We saw a little bit of those earlier out let's take a look at some of the pieces here. AI and to do that I'm Join to come over by the way if you haven't Notes, AI doesn't look familiar, Cura is a clone of Versus Scale. AI shares the same base as Versus Code, Cursor, and City much every other tool in the market that is IDE based. AI should also say that you probably saw this on the site when you were downloading. Speaker 0: This is the IDE version that we're using. There is a CLI version, and there's also a web version where you can do all of this Jun in a web browser. Yeah? Yep. Yep. Speaker 0: Yep. So, yeah, so, start with the Bitly AI. AI tinkerers Date Raleigh. I know we're kind of technically in Durham. And that should take you to this page that has all the links on it. Speaker 0: It's this last 1, it's the GitHub repo. Have you been thinking of some assumptions? Did you ever know how to clone a GitHub repo? Okay. Cool. Speaker 0: Are you helping someone? Help me. Okay. Get it installed. Yeah. Speaker 0: Okay. You got it? Cool. Thank you. Can I take a picture of a link so I can Yeah? Speaker 0: Yeah. Of course. What do you find on the $5? This is the guy that wanted the QR code that I didn't put up. Because I always find them so annoying when I'm on my laptop. Speaker 0: Good. Cool. Speaker 3: Okay. Speaker 0: So, Kiro here is Jun giving me a quick tour of features. Shares are these 2 extra talk, these are the ones we're going to be looking at here just to get a sense of what's available. If you're coming to the Kiro tab, this is where you're going to see things like your specs. We don't have any specs yet, we're going to go build a spec Hands you'll see your specs listed there. Over AI, you're going to be building a spec for features, so you'll probably have a big collection of specs that are part of your project and are checked in. Speaker 0: This is great for documentation and understanding what's shares, how it got there. We're not gonna do any agent hooks Date, but agent hooks are essentially Web something Speaker 1: happens, Speaker 0: I want to kick off a prompt that does something. When we're gonna demo this, what I would usually do is multilingual Hands say, hey. I only know English, but I want you to translate the UI into lots of different languages. Every time I update the English language file, I want you to go Building new roof. And so every time there's a save on what's usually a JSON file for localization, it'll go and kick off the others. Speaker 0: Everyone familiar with skills? Flood skills is kind of a common thing, right? We support flood skills, we also support something called steering, which predates skills. By the way, when I typed slash start workshop, really Paula I was doing was identifying this steering file. There is a steering file here. Speaker 0: Go to the top. Inclusion is manual, which means I have to literally say, Tech forward slash start workshop, and then it just includes the file called start workshop Jun that's what kicked it off. That's what's going to drive the whole experience. And then there's some mcp servers which we are not going to use today. AI. Speaker 0: Cool. It's Join to ask me to say Lead. They don't Date to AI Lead. Anything that prompts it forward. AI I'm going through the workshop City AWS checking these things off saying we did this, we did this These checklists, June in markdown, if you put in a little square brackets, you'll build a little to do list or checklist. Speaker 0: This is a great way to get agents to follow directions. AI sure you have given Jun prompt to any 1 of these tools Hands it does 3 out of AI things you say. It just gives a couple Tech along the way. Putting together a checklist, as silly as this sounds, and saying you must check the checkboxes as you do the things AWS a tremendous help 2026 keeping these things on track Hands helping them to do things iteratively. Okay, so it's just Danny bullet and then shares brackets. Speaker 0: This is what it looks like in text. Hands the agents are Paula Raleigh, really good at following these. Speaker 2: Nope. Speaker 0: I am, I am not using any we we support custom Agentic. You can build a bunch of custom agents and things like that. I'm not using any of them. The the product ships with a bunch of custom agents Hands you can build your Join. I'm not using any of those. Speaker 0: This is really basic. I'm just focusing in on the idea of SPEAKER Hands development here. So I'm just using it Ron of natively. Speaker 2: And this software is, Speaker 0: Yes. AI, it's a derivative of Versus Code, right? Share is basically Versus Code but Yagura is an Amazon product. Okay, so we are going to move into SPEAKER Hands development. Now, this is important that we follow this 1 correctly. Speaker 0: It's writing this prompt for you to 2026 Hands create the Kiro app. I don't want you to 2026 put this right in here. Okay? Because you're kind of giving away the secret, which is we've already read this whole steering document in that tells it all about what the workshop is Hands understands that it's doing a workshop. I don't want it to know that. Speaker 0: It'll tend to jump ahead and do things that are Join to come later Hands you won't get the full experience if you do it here. So just go up and start a new tab Hands in this talk, pick SPEAKER mode and paste that in. AI, Hands it did explain that in the instructions. I want to build a browser based Flappy Hero game. It's a Flappy Bird clone using HTML5. Speaker 0: AI PNG AI is already in the project out. And then it talks about the the gravity Hands we're Ron use the spacebar as our flap to play the game PhD a few other things. Good. So a reasonable Ramos, not great. It'll take this and turn it into a AI requirements document. Speaker 0: So now what I've got is in the 1st tab, this is a AI Coding tab that's driving the Workshop, and then I'm I'm doing the spectra to dev here Agentic AI I don't Ron to talk this with the with the forward knowledge of all the things we're going to do later in the workshop. Otherwise, it will tend to jump ahead and Jun build the whole thing out right away Hands we won't Beach able to make the corrections that we want to Date. Okay, this is obviously a feature since it's new Notes a bug file, there's nothing to fix Speaker 3: yet Speaker 0: Okay, Speaker 2: Hands, Speaker 0: I Ron to start with the requirements documents So feature and then requirements. And all Kiro was telling all telling you all of this in the other tab. Speaker 3: So is this Speaker 2: how it works there? Like, you could you give it a prompt and then it's going to, like, piece you through the process? Speaker 0: Yep. It is gonna go now and start building Speaker 2: out Speaker 0: my specification. Hands you'll see it June. It'll take about 2 minutes for it to kind of reason through all of this. City seems pretty easy, right? Out of the Date, you think of this AWS, Hey, I Jun Date you a prompt, you throw me back shares requirements document, that Assisted. Speaker 0: There's actually a bunch of stuff Speaker 2: happening Speaker 0: behind the scenes beyond just that. We are not simply, leaning on the LLM for that. There is a huge investment here from our, automated reasoning and neurosymbolic AI team that is doing a whole bunch of additional reasoning on top of this spec. AI. Actually, behind the scenes, we're generating multiple iterations of the spec, comparing them, looking for ambiguity, removing ambiguity in them. Speaker 0: So there's shares a lot of churn that's going on right now to get to a good outcome. Hands you see the spec here for AI Kiro. It's built the requirements document. AI the way, this is just a hidden folder in your project. Or it, you know, starts with a out. Speaker 0: It's probably on Jun, not Ramos people's machine. And here's that requirements doc. Alright. So take a minute and kind of read through the requirements doc, get a feel for it. You don't have to read every line of it. Speaker 0: If this were a real world app, you would. This thing's so simple that I'm pretty confident in the requirements doc being a a good outcome. Alright. But this is a series of user stories and associated acceptance Frontier. That's the format for the requirements doc. Speaker 0: Unless you tell it to do something different. You can't override it and say that's not what you want. Speaker 3: Yeah. Speaker 0: So there's a couple of Brian AI think of of how I would approach that if I'm trying to do what I assume you're trying to 2026, which is is get a skill that really follows directions well. AI, so you don't wanna have the the checklist literally in the skill because there's only 1 copy of the Scale, but you can put a template for it and say, hey. I want you to create Speaker 3: this Speaker 0: this checklist with these steps. AI? Add this to my whatever file you're working Ron, and then work through it. In fact, sometimes it skips. It doesn't go through all the skill. Speaker 0: Yeah. Yeah. It's definitely not uncommon that all of these tools will will skip some steps along the way. I've been working with 1 of our partners on, on their install. It's called Kiro nPower. Speaker 0: It's AI the package that installs Folder pieces Networking with them, Notes AI through their install process because it's really complicated with Docker containers and a bunch of complexity. And, yeah, it could it could be a bear sometimes to get that to work. So I'm going on a wild tangent here Jun to answer this question. Don't don't get too bogged down in what I'm about to do. My other my other nerdy passion is, genealogy Hands Ron kind of family tree research. Speaker 0: And so AI have a a skill called PhD genealogist, and 1 of the things he does all the time is create these checklists. AI, I have instructed, I want you for every person in the tree to keep a checklist. You know that he was alive from this period to this period, he should appear in this census, this census, this census, he should have registered in the draft in this war, in this war. I want you to go and create that checklist Event time I identify a new Lessons, once you know his ages, create the checklist Lessons then we will iteratively work at that over time. So, right, the skill is the genealogist Scale that sets this up, but it is constantly building out checklist for me to then go work through later on. Speaker 0: By the way, these things, if anyone else is into genealogy, like, these models are freaking amazing Date doing it. AI, they understand all the contextual awareness of what was happening in that period, in that town, in that AI. It's AI what they know. It would take you months of research to understand Hands, you know, in some foreign country, you've got a relative coming from a, you know, for Ramos of my relatives are Irish, so it's file somewhere in Northern Ireland in some little tiny town Hands it AI file everything that AWS going on there at the AI. It's wild. Speaker 0: It's wild to watch it. Cool. Alright. So back on track. I'm Ron to come back in here and so I'm Jun going to come back to the other tab and say I'm done so that it's guiding me Hands so I will say I'm done. Speaker 0: That Scale, it's literally a it's a offering document Notes a skill in this Castro, but the skill is what's driving this whole experience. You're Ron see it probably do this once in a while. We are literally talking about specs and AI it's like gonna keep saying, hey, you sure you don't wanna switch to spec mode? I'm in I'm in AI mode. No, you don't want to. Speaker 0: File, we're doing that in another tab. It's just it's always kind of advertising its other features. So cool. It came through and said, Raleigh, great. I see you Building the requirements document. Speaker 0: Let's go and add a change to it. AI? I want you to get the experience of going in and iterating and changing these requirements documents, not just taking what it gave you, but thinking through the things. So let's go and add a new feature. So I'm gonna switch back over to my spec tab. Speaker 0: I'm not going to City go Building AI. I'm going to 2026, hey, I want to add a new requirement. Hey, ghosts are AI of intangible. I want my ghosts to be able to phase through walls. So we're playing, you know, if everyone's played Flappy, AWS anyone not played Flappy Bird? Speaker 0: No 1 makes some assumptions about what people know. Okay. So I assumed everybody had played Flappy Bird. Speaker 3: AI. Speaker 0: The ghost, you get the ability to Notes, this is the 1 where you just AI smash on the screen and it makes a little bird go up and you try to get him to go through the gaps. AI so terrible at out. AI do this in Workshop all the AI. I hate this game with a passion. Anyway, the feature we're Ron add AWS, hey you're a Hosting, so once every 5 seconds if I hit the space bar you become intangible and can pass through a wall. Speaker 0: Like, you get, like, 1 free pass and you get through 1 Paula. But then there's a countdown timer Hands you've gotta, like, wait to do it again. So I get kind of a if I'm getting stuck and I'm nervous, I can just hit the space oh, sorry. The shift key and get out of it. So I'm gonna go Ron add that requirement into the mix Esther. Speaker 0: And it's gonna go and update the the requirements document. By the way, I could Date just gone and literally edited the requirements document. AI? If I prefer, if I'm seeing little nitpick things that I wanna change, I can just go change them. You saw Ron the demo before that it read the requirements doc back in before it started working on the design. Speaker 0: Alright. Hands you see it here making some edits to the requirements doc. We'll take a look at what it did in a minute. AI usually will add a whole new AI section and then edit the header with the with the high level descriptions. So I know this is really hard to see on this resolution. Speaker 0: Hands this is talk kind of it made the edits Hands then you see it going through this. I'm going to make it more precise or or some language to that extent. This is usually that kind of automated reasoning nor symbolic AI, additional work that we're doing on top of just asking the LLM. This isn't just passing it to the LLM and taking what 2026 gets back. There's a bunch of other magic that happens in here. Speaker 0: Danny general feedback on the credit process? AWS it painful? No. Is that okay handing them out like that? We used to do it. Speaker 0: We used to do a QR code for this. It was a lot easier. But, we've gone to individual codes now for each individual person. Shares a tendency to, like, stick these up on public forums quick. Before unscrewed bills Yeah. Speaker 0: So we we don't Web don't, like, meter in tokens. We meter in, like, an abstract thing called credits that are, you know, indirectly related to tokens somehow behind the scenes that we don't disclose. What you'll see is AWS you're changing models, you'll see multipliers on the different models for how many what the credit consumption is. The 1 1 credit is bound to the auto mode that you're in by default. If you go to a, you know, a model like an Opus, you'll spend more credits. Speaker 0: Yeah. If you go down to something like an open there's shares a bunch of open weight models and stuff in there Hands, other cheaper models too that are practical of the credit. I'm not gonna comment on that with the camera running. Let's have a beer later Hands I'll talk about it. Alright. Speaker 0: Cool. Alright. So it went through and made some changes. You AWS see it AI Join and did some diagnostic AI it's running linters and stuff like that in here. And, let's go see what actually happened. Speaker 0: So AI, it went through and 1st edited and then you see that AI of the response to neurosymbolic AI doing these ambiguity checks Hands trying to look for vague statements in here. We want to get rid of things that are really vague or contradictory. In the interest of time, I'm going to hit generate tech design because I know that's the next thing it's going to Ron. I'll catch up but I want to keep talking. Web we did these early Ron, it wasn't uncommon to see Hosting, contradictory statements in the in the requirements that the models would generate Event when you were working with a really good model. Speaker 0: Go back maybe a year ago, and you'd see things AI, hey, the, you know, the ghost moves to the left Tech constant speed or the Paula is moving toward the ghost at a constant Beach, some language along those AI, and then a little while later it would talk about talk AI out would get progressively harder and the walls would move faster. It's like those AWS those 2 statements are incorrect. Jun of them says out moves at a constant speed, Jun of them says it gets faster over time as you pass through walls. AI kinds of things, I would probably write it that way 2026 AI, but that's ambiguities, that's something that the model is Join to choke on later Hands AWS probably going to cause you some Join. These checks that we're building in are really looking for those kinds of statements and then trying to reconcile them. Speaker 0: So, the models are getting much better but we're also layering out a bunch of other work on top of it to respond. Cool. So I'm Ron come back and actually do what I should have done before if it'll let me come back Hands say done. AI the way, if anyone went in the the not AI speed run mode out Ron the slower mode, each 1 of these you'll see it read back in the requirements document and make some suggestions just about things we could change Jun to sort of build that muscle of going back in and doing the review and making sure that you're looking. Alright, so now I'm gonna go tell it I already told it to generate design. Speaker 0: Pretend I didn't and I'm going to do it right now. And you see it created the design doc and again it's gonna start working on it. The design takes a couple of minutes. So a few things are happening here. It's going to go through the technical AI, it's Coding to also again analyze the technical design, look for ambiguous statements in there, fix that, but it's also Join to do the work of City property based testing that I talked about a little bit in the earlier talk. Speaker 0: So, that talk a usually takes a couple of minutes in this phase. Speaker 3: And Speaker 0: as I'm doing this, I realized I don't have a final version of this. You can actually see the game that it writes in the end. So you'll have to just run all tasks. When you get home tonight, you'll be able to hit the run all task button and let it build out the rest of the game and play it. Rest of the game and play it. Speaker 3: Yeah. Speaker 0: Of course. Yeah. Yeah. So what I would suggest today, and and I kind of spoke a little bit about this earlier, is you should be thinking 1st about organizing this into tasks that are going to the backlog file you've always Danny. And then you AWS a developer take 1 of those tasks out. Speaker 0: Most frameworks, right, if you're doing, File or Scrum or or any of these, generally says your talk should be about a week. Right? A week of human time. That's probably gonna get collapsed down to a day or half a day using the agent. But you AWS a developer want to take that task out of the backlog Hands then go build the spec with Kiro to Windows build up that context and execute it. Speaker 0: Your peers are hopefully working on something else out of the backlog that you've designed in a way such that those don't have AI coupling between them. Same thing we've always been doing. There's there's a bunch of stuff coming that file start to address more of, like, a collaboration experience Hands, I don't know what that looks like yet or AI any meaningful timelines or anything like that. But we are thinking about, hey. How do we AWS a group also work on the you know, we're we're Web focused on the inner loop in that picture that I showed before shares you're iterating on an individual task. Speaker 0: But the outer loop where we're eliciting those what are the tasks and what is the high level design for this sprint, we don't really have a great solution for that yet. It's not something that we've even tried to tackle to date. That's something, you know, right now you've got the, the Git AWS and the Lessons of the world that are working on that problem shares we're Hardware we are working on the Banner loop problem. I expect we'll all start doing everything eventually. Cool. Speaker 0: Okay. So that's June through and built out our technical design for the game. Hopefully everyone is reviewing shares AWS Web as I talk a little City. Out here you get the overview, AI? It's all that I want HTML, but offering this I intentionally ask asked out to keep this really simple and not go wild with React frameworks and stuff like that. Speaker 0: This is pretty simple. It goes Join AI. Right? You get the very high level design decisions here, and then you've got the architecture. Probably everyone's looks a little bit different AI? Speaker 0: If anyone's gotten this far and is actually following scaling, it's not uncommon that Date this point we're not all in the same place these models Date they all come to different outcomes if you give it the same input multiple times Web don't always get the same output but this is our high level design for the structure of the game Hosting are Join to be organized and then AI alluded to this before but didn't show City, I never scrolled down in that other demo. You start to see it put together, not code, AI not actually writing code yet, but we're putting together some high level structure. What does the core loop look like in this case? Usually you'll see what are the interfaces for the significant classes Join to look like out of these HINTS? How do each of these classes expose themselves to the other classes Event without the implementation? Speaker 0: And then typically shares also some data models, Tech, so you see the data models, how are we going to store talk, how are we scaling AI, the AI scores, the game Date, Hands stuff like that. And then you get down the last part of it is the correctness properties. These are the properties that Date, 2026 know, these are the things that are known to be true of the game that are going to be built as property based Esther. These are the things that we want to test are true using property based tests. We're going 2026 speed Jun out, so so we're not gonna build property based tests because it just takes a little time. Speaker 0: If you've gone the long route, it'll build that out. And you're probably way behind because you're doing a bunch of reviews and stuff and iterating with the model. So I will say done here. Oops. Alright. Speaker 0: And now we're gonna start working Ron the tasks. Hands zoom. If we come back, let me come back into AI mode. Web went from requirements to design Hands now we're going to go Jun build out the task scaling. Break this down into actionable chunks that are organized into atomic units that talk be built independently. Speaker 0: Many of them can be built in parallel. It will make decisions around what it can do in parallel and what it what it needs to be blocked Ron, waiting for other steps to finish. AI remember though the point in all of this is Web know we're terrible at writing good prompts We've adopted this idea of file coding which is I'm just Ron give you a terrible prompt knowingly Hands then make a whole ton of corrections to get you where Web want to go. AI? Rather than iterating on that, we want to be iterating on the plan, building solid requirements, building up a solid AI. Speaker 0: And then, breaking that up into tasks that we can go and work on Hands that will fit into a context window so that we're not constantly overflowing our context window. I think you were talking a little bit about, like, wanting and accessing the million Notes context Windows, AI, you have that with the bigger models, I would really discourage everybody from leaning on large context windows Join bigger models over using the smaller ones. 1 of my coworkers I think, put this really succinctly, which AWS, if you've got a 200 ks context window, you've got like Jun ks of Raleigh smart model Hands 100 ks of kind of dumb. When you move to the million token window, you've got AI 100 ks of smart and like 900 k of dumb. It just gets worse and worse and worse the more crap you pack into that Coding. Speaker 0: The worse the outcomes are getting, the worse the model AWS performing. If you can try to work in in the, you know, in smaller units and break things up into individual tests, you're gonna have much better outcomes Join this AI, long, endless context window that you just keep iterating on. So, I mean, to put this in perspective, our whole design process here, we've used about 5% of that context window. Right? So we're about 50 k Join, or I mean, this is only 200 k context. Speaker 0: I'm just AI close to that. Right? So we shares dented this thing Hands it is now going to encourage me to start working in a new 1. Okay. So at this point it has gone through and organized this into a series of tasks. Speaker 0: The 1st 1 is just going to be setting up the project structure, AI the whole thing, and then we're going to go into implementing the physics and controls. You see the property based tests are disabled because we're going through the speed Ron version of this, so we're keeping this this short and simple. AI, and then we'll go through the AI. Those the pipes are what, the walls that are coming at you that you are trying to fit between, they AI those pipes. That's what it is talking about here. Speaker 0: Closure detection system and scaling. And then, the game state screens. Notes like the start screen and PhD end screen. And Venue usually shares 1 at the end where we go through a bunch of Hosting Hands iterations. So it's broken Danny into AI high level tasks that it thinks are relevant. Speaker 0: I'm just going to come up and say AI just want to run the 1st task. I can come up anytime I want and say, let's go run them Paula. But I don't wanna be that 2026 in the loop. I wanna keep an eye on it. I wanna keep reviewing the work that it's doing. Speaker 0: So I'm gonna say, hey. Let's go start the 1st task and let Kiro tackle this 1. Okay. And now we are finally at the point where we are starting to write some code. Notice we started a new window. Speaker 0: We've got a nice fresh context window that nothing's happened in out, and it is just reading in all of those documents for context. It's pulling in my requirements Tech. It's pulling in my design doc, and then it's gonna go and start building this out. AI. Someone asked about, custom agents and stuff like that. Speaker 0: So far, we've just been iterating with the default agent. Now you're starting to see it call in some, this invoking the spec talk execution. This is City starting to use some of the the custom agents that are built in. AI. Yeah. Speaker 0: Yeah. So if you're changing models in the middle of the of the competition, I actually did that. Right? If you Notes, like, the 1st thing that I started, I realized that I still because I was poking around, Scale had Opus Jun, and that's way overkill, and I switched 2026 out. Out essentially, the way these things work is I'm gonna wildly oversimplify AI. Speaker 0: Out each prompt is, hey. This is what the user AWS asking for. Right? Here's the things that were Ron into context because they had to be, like your skills and your offering files and stuff like that. And here's the prior context. Speaker 0: Right? Here's the whole prior conversation we've Hands, and that just gets packaged and Hands sent back up. Hypothetically, each of these things AWS each call is completely independent. The the model on the other side doesn't care. In reality, there's a bunch of caching that happens. Speaker 0: Server side, You don't always get a cache hit. You're you're not always lucky enough to go back to the same place where your where your stuff is cached. But if you're lucky enough to, it saves you a bunch of time Hands it actually saves a bunch of tokens. You don't have to reprocess all of that. So there's a bunch of work that goes on behind the scenes to try to get you back to where it was. Speaker 0: Obviously, if you change models, you lose that Tech, and you're gonna have to read it in Amazon, but it will totally pick up in every AI. AI. So we got through, looks like we did 1, 2 how many subtasks were there? Looks like 3 subtasks. So it's done 2 of the 3 tasks, building out the scaffolding, and you're starting to see it really write some code now Jun build this out. Speaker 0: AI by the way, when it starts doing these AI Tech like this, these sub agents that are rolling, each of these gets their own context window too, so we've got a lot of Tinkerers around managing the context window really intelligently. Cool. Alright. So it's finished these. We scaffold out the app. Speaker 0: I'm gonna come back and just tell it that head back to back to tell me Raleigh. AI like we're gonna build the whole thing. Cool. So we built out the 1st Jun. I should right now do a Raleigh thorough review of all the code that it just wrote. Speaker 0: This is really trivial code so I'm not terribly worried about it. I'm not going to look in the interest of time. To round this out, I am now just going to say, hey, let's go run all the tasks. So run all the remaining tasks. And I will not run the optional ones. Speaker 0: That'll bring in the property based Tech and and definitely not finish in 12 minutes. This might not even finish in 12 minutes. So it's now Ron go Hands analyze where we're at and figure out what's left to do and go gonna run the whole thing. AI the way, it's built out these these waves at the end shares it goes through and figures out the interdependencies. Logically, wave 1 and 2 are done because we finished all the test Notes, but some of these can be run-in parallel. Speaker 0: Some of these things don't have any dependencies on prior tasks, and then it can go and AI crank through them in parallel and make some decisions about what things get run side by side. So you see, actually, it ran 2 dot 1 and 2 dot 2 together, waited for them to finish, and then kicked off. Oh, sorry. I was running them now. Yeah. Speaker 0: So the Like, who does the Speaker 3: sales people talk? Speaker 0: Yeah. We're well, we are focusing on on the developer community here with this. But more and more oh, globally. No. This is AWS, you know, this is a a product that that Amazon sells to everybody. Speaker 0: And we have different teams that are focused on Jun of the indie developers out in the world Hands then also corporate developers that we're talk 2 different teams within Amazon are talking to. All that said, there are a lot of users that are not developers. Some of them are kind of file the the rest of the the people that contribute to the SDLC, right, the product managers Hands everybody else that I was talking out. But there's also a lot of people using these tools, that have nothing to do with software development at all. Right? Speaker 0: My family tree has nothing to do with software development, but I use this because I'm really comfortable in this environment. We also have other products, right, that are that are not Partners I focused 2026 the developer world. There are other products from Amazon that do this for more of the, you know, office worker. We have a whole, we have a solution called 2026, and there's multiple iterations of Quik that live, like, in your browser or on your desktop Hands things like that. And they can, like, manage your email for you and manage Slack Hands, talk to websites and do a bunch of things for you. Speaker 0: So, like, every morning I come in and and Quick has read my email for me. It says shares, like, the 3 important things. You're getting talk Raleigh crap. These are the things that I think are important for you. And I can say, hey. Speaker 0: Go respond to that, and they'll just do it. AI? So so just like you're writing code, I can do all that stuff in in kind of the the offering world. There wasn't there was this, like, weird interim period where the the non devs were using the dev tools to do this, and now you're seeing all of these tools introduced into the world Date kind of directly address the need of the the Ron developer, which is for us quick. Speaker 2: Can I ask the question because at the beginning of your presentation, you stated your Speaker 0: Yes? Speaker 2: They're the ones that you can quickly grow with, but then you have that middle and then then the last. Beach. Out what are y'all Join to Speaker 0: AI. Don't don't put this on social media. Okay? I'm gonna piss off the marketing team right now. So I think of this like there is a there is also a CLI version of it. Speaker 0: This is AI super Danny path Ron the rails Jun of stuff that we're doing here. If you've looked 2026, like, Raleigh loops and and those kinds of things, that's really pushing boundaries with usually with the CLI. Speaker 1: And also, Speaker 0: Tech. Yeah. Absolutely. And I should say 2026, the, you know, the other thing is the the great thing about the CLI is it's scriptable itself. AI? Speaker 0: It talk run things out you can also run City. Hands that would be an agent calling it or you Danny just run-in headless mode, AI, put it Join a GitHub actionable every time someone opens an issue, it just kicks off PhD AI in in a GitHub action Jun them differently than this or differently than you did a year ago? Speaker 2: Well, definitely different than I did a year ago. I think the the year ago, I shadowed the users. Like, I I shadowed the employees. Speaker 0: Yeah. Speaker 2: Because they implemented AI tools within the company and we encourage them to use them, I watch the patterns on how they use them and, like, the inputs, outputs, and so on and so forth. And then I would build their deliverables or what they're supposed to actually, you know, complete during the day into my agent. AI Speaker 0: we slowly build some confidence in the agent. Speaker 2: Yeah. So we're we're building confident in the agent Speaker 0: and Speaker 2: also in the Speaker 0: AI I love this approach. I'll go Beach. Right? I I mean, I'm I'm kind of fully Drawing Kool Aid here. This is what I do. Speaker 0: Right? I do this all day and talk about this stuff all day. When I was 1st doing cloud file years ago, right, before I came to AWS, I was at KPMG, Danny accounting firm, a very conservative place to work. It wasn't like the 1st early adopters of Lead. And we were doing a lot of data center automation Hands with AI cloud Tech, and we're getting a lot of pushback, like, hey, I don't trust these tools. Speaker 0: I don't want you automating anything in the data Esther. This is too high risk. And the 1st thing we did was we started building out essentially an auditor AI Lead, hey, if automation had done it, it would have done this, this is what your human did, here's the deltas, and we reviewed those every week, And generally what we found was the human made mistakes. Yeah. AI, the automation was right, had we run the automation it would Date been perfect, but that was correct. Speaker 3: Hands Speaker 0: then they were file, okay, we're starting to build some confidence, and then we would say, okay, here's the script that it wrote, it could run this automatically out you're nervous to have your human go run this script that it wrote. And the outcomes improved dramatically over time, we had a lot less issues, we could show that with data Hands then eventually Web said alright, AI turn it, on, let's let this thing happen. And it's essentially the same thing you're doing now. Join the AI variant of that story, Notes, if you're nervous about it, don't jump right into writing Coding, don't ask them to write code, ask them to review the code out the Jun. And let it poke a bunch of holes in it and say, hey, I found a bunch of issues that I wouldn't have done. Speaker 0: I wouldn't have done it AI. Here's why. And you start to really build some confidence in the models. So these things are actually pretty good. Speaker 2: The audience that I deal with doesn't know anything about the code, Speaker 0: so Speaker 2: Yep. I build it and they review the output Hands then they provide me with their feedback Hands then I build that into the code and we try it again, test it out. Speaker 3: Yep. And go from there. Speaker 0: AI I know we're Date time here. We are we are really close. It's almost done. I see Danny packing up. I assume you you wanna wrap up and and turn this back over. Speaker 0: Right? It's it's not a big deal if we don't actually get to run the game. I don't think that is all that terribly important. Oh, cool. Yeah. Speaker 0: Yeah. Bring it up. Come come show us. Often, what I will find with this is is usually this is pretty you get a pretty good version of the game. Sometimes the gravity AWS, like, a little file. Speaker 0: Like, he just falls a little too fast or or a flap makes him go much too high Hands it becomes too hard to play. Speaker 2: AI Speaker 1: changed the UI a little City, Speaker 0: but I I I want you to, but I mean Partners you went the long Ron, it would kind of encourage you through the whole experience to be looking for things to change. Speaker 1: You Speaker 0: need 2 screens. Speaker 1: Yeah. Speaker 0: I have a few. You want it? Oh, I guess you can't you can't play until Yeah. So Speaker 1: and then the the ghost button actually works. Speaker 0: You got it to face them? Yep. You should've pushed it. Yeah. I should've. Speaker 0: Oh, Cool. AI can Speaker 1: only get Castro. Speaker 0: Yeah. Yeah. Speaker 1: And then this is what I June for, like, the MC out tools that you were talking about, like, some of the, skills. So that way, AI could just change the Speaker 2: UI. Speaker 1: Yes. And Speaker 3: Let's Speaker 1: see if Speaker 0: I can So Yours yours looks similar, but you don't have a cool ghost. You just have a cool ghost. No. No. No. Speaker 0: No. And I didn't even know The ghost is cooler. Speaker 2: I built it in a different tool just so I could see what it would look like quickly. Speaker 0: Nice. Speaker 2: And Talk you Speaker 0: for going.