Stop Cloud Waste: AI Checks Costs Before Deploy in AWS

And, so before I go to the demo, I'll try to set up the context here and, the challenge which organizations generally face, when it comes to cloud bids and, how they overcome it and, I mean, how the how they work on the challenge and how, this agent will help in that, process. And, yeah. So before starting, maybe, like, how many of you use public cloud directly, like AWS, Azure, GCP? Okay. So pretty much everyone have it. And so in the like, as you know, like, each 1 of us our own inclination towards, 1 or the other cloud provider. And each 1 of us have our own favorites there, like, the based on the kinds of services they provide. But 1 thing, which, we might agree or we may most of us agrees, is on the surprises which we may get from the bill. Right? And, some even the experienced ones or even the novice ones, like, sometimes there will be spike in the bill, which even can't explain, like, why it happened. Right? So that's a bigger challenge for the larger organizations too, actually. And, they try to overcome this challenge by, following some practices called FinOps practices, basically. And, FinOps practices have, many things like, reporting, forecasting and optimizing. In this demo, we will be discussing about an practice called standardization, which is a pin ops practice, which organizations generally follow. And yeah, so let me just jump into the demo part. And for this demo, we will be assuming like there is an company called Tech Corp, which has an AWS account multiple AWS accounts. And they do have an FinOps team, which has set up the standardization principles. So what does what does this standardization mean? Like, for suppose, on the compute side, the they have apologies like in the DevOp and QA side. They can only, create certain type of resources, certain type of, servers, which are of the lower IDENTIFICATION. While on the production, they're allowed to do some higher configuration servers. Similarly, on the database side, they have a policy like, they can only use PostgreSQL or any open source SQL. While on the production side, they're allowed to do Oracle. So all these are, like, principles which, are part of the standardization Apple’s, which help which help companies to predict the cost, basically. By standardizing, they can control the cost. And similarly, they have a policy for life cycle, s 3 life cycle I mean, sorry, storage life cycle also where, the data in the, volumes have to be retained to only certain period of time because beyond that, it's again waste of money. Like, once the compliance period is over, there is no point in retaining this data. So, all these are like certain, standardization principles which we have defined as part of this organization TechCorp. And I have, documented them in the PDF. Here I have used a PDF for the demo, but in real time it can be a confluence space or any internal knowledge base, which the company has where the FinOps team define these things. And also very important thing is tagging, generally which we do in public cloud, where, as you know, like, in an organization, multiple teams use cloud, same accounts, and, they do a lot of visual share resource sharing. So this tagging will help to track back the, which business unit has created those resources for the billing purposes. So all these Apple’s, standards and principles are documented. And the how, these principles are used is like, these are the recommendations given to all the developers or the operations engineer to follow when they are preparing their infrastructure as code while deploying their resources. So infrastructure as code like people who are not using cloud, I say kind of an declarative language, which will help you to create resources like through code basically. Like instead of doing clicks in the cloud, you can use the ML, JSON or the HCL language based on the like infrastructure code language, which you choose. It can be (Terraform), if it is AWS, it can be what we call cloud formation template in Azure, it is called ARM. So, that based on the choice, you will use the declarative language and define the code to deploy the resources. And so now we have an imaginary company, for this demo the TechCorp, where we have defined all the standardizations as part of the PHENOPS priorities. Now, let us go to the demo part like how this will help theorganization to save the money. So, yeah so I have developed this agent in NA 10, which is a low code and no code tool as you know. And the input for this, I mean the this agent input will be through chat, I mean the the trigger will happen through chat. So here the input, the expected input is the Terraform code, which is developed by the developers (Terraform), they deploy the resources. And the agent, has an OpenEdge chart model access for reasoning. And also, it has access to, Pinecone Vector Store, where I have embedded the the text documentation I showed you before for the standardization. Right? The policies where we define, so that data was embedded into this pine cone vector store, and it also has an embedding open air model. And so the reason for this pine cone vector store and this embedding open air model is we are trying to use retrieval augmented generation here, where we are giving access to our internal company knowledge base for this agent, so that it starts acting as an my FinOps team member or a FinOps agent to give recommendations to the developers. So there is a separate workflow where I have vectorized the data and, like I have got it indexed into the Pinecone Vector Store. I will I will I will show it after this demo part. So, for the demo, I will open the chat. And I already have some code ready where, I am trying to create an S 3 bucket, which is nothing but a blob storage in the cloud. So, I will copy this code. So, if you see I am trying to create a resource bucket called tech cop QA logs, and it has tags, but I will fully avoided the lifecycle policies here. So, let us see like if I pass on this code to the agent, how it would respond. I am giving it the same code in the message. It is the no other prompt, it is just a straightaway code. So it triggers, the agent. You can see the flow actually, like, then in it, and you can it clearly shows the step by step process. Yeah. The workflow executed successfully, and now I got a feedback. Let me make it much more. So in the feedback, I don't know, maybe let me try to make it. So it says me, like, there are some noncompliant items in my code, which shows, like, the policies s 3 bucket life cycle management. And they did, recommend, like, to add the life cycle rules for the s 3 bucket to be created. And it also suggested some cost optimization opportunities and the complaint areas. Like, in this code, what we have what we did is, like, we added the tags, right, which is the complaint. Like, I mean, it's complaint with the FinOps principle, the standardization principle, which they have defined. So we get a summary of this. So how did, this happen? Like, let me show you. If you see, I didn't give a prompt also. I just gave the code. Right? So here in the agent, I have defined somewhere, like, what it has to do once it got the code. Let me open it. I'll show you. See, here in the system messages, I have said clearly to the agent how it has to behave. Like I have mentioned like you are a FinOps adviser to the tech corp, and your role is to review the whatever (Terraform) infrastructure code given to it, and compare it with all the policies which are in the in the pine cone vector d b. And also I have given some strict rules which, the main thing is like I want to avoid hallucinations or avoid, the model to, give me recommendations based on its pre trained data that is not something which we want. We want the model to behave exactly like my, internal team member who has access to, the knowledge basis or the FinOps principles which we define. So, I have to give it a set of rules to exactly behave that way and not use any, like, pretrained data. So those are the rules I have given additionally for this. And, let me close this. And from the flow standpoint, if you observe, like, first it I mean, the input goes to the agent, And then, the prompt along with the code, goes to the embedding model. The embedding model then, chunks it. And then, it then and and that is used in the Pinecone vector store to do a semantic search, and identify the relevant principles in the PHENOPS policies. And then again that gets the context and the prompt gets augmented back to the to the reasoning model here, which then validates the code with the ah FENOPS principles and then give us the feedback, so that is how this agent works. And coming to the vectorization workflow, so this is the vectorization workflow. So, this workflow, it is a trigger like I have already uploaded my, principles in 1 drive right, I am sorry, Google drive. So, it downloads the file if I click it, if I execute this workflow. And then it has access to an embedding model to along with an default data loader. So, the file gets downloaded using this default data loader, I mean gets downloaded here, and it gets uploaded using this default data loader. And then there is a text splitter, which would chunk it, the file, to the data, chunk the chunk of file. And then the embedding model helps to vectorize it and store it in the Pinecone vector store. And this is the Pinecone vector Db where we have these chunks after they are indexed. I will just click on the workflow. There is no output to show here, but it just show you the flow actually. Yeah, it is already indexed, but I just clicked it again. In case if I have refresh the file with some new principles, then if I click So, how does this help actually like to save the cost right. So, now what we have done, see in this agent, we have given our infrastructure as code. And before deployment itself, we are giving the feedback to the developer like what are the principles they have to be complaint of as per the FinOps policies which they have defined within their organization. So, by incorporating this feedback, the developers can edit to their company rules and stick to within the boundaries which they are defined. For. So nowadays, we have a lot of wide quarters also right in any organization. So, I mean, there is a high chance, like, it might generate a code which can, which can lead to deploying some high configuration machines, which are only generally caught it after the deployment after maybe a bill of 1 month or 2 months if they are tracking. So by reviewing the code with this agent, they get the feedback early, and that helps them to, like modify things according to their organization standard. This principle like is called shift left in DevOps, where we try to give feedback early on to the developers. Like, generally, the code is if you if you recollect, like maybe most of the developers would be knowing like, the core will be added with security agents for security vulnerabilities basically static code analysis or dynamic code analysis right, where you get the feedback early on before building actual builds. The same thing we are trying to do here to like here from but from the cost perspective, from the Apple’s principles perspective like we are trying to give the feedback early on to the developers, so that they can make the validright choices, so yeah that is the demo team yeah.