My initial reaction to AI and vibe coding was…here we go again. However, I figured don’t knock it till you try it. So I sat down and started using AI seriously for the last couple of months to see if I could make it work. Mind you I have 30+ years of software engineering and architecture experience so when I say “work” I mean a real, well-designed, maintainable application. I’m also a penetration tester so I am also taking security into account.
The initial reaction is generally “Wow! Look at that! Real code.”
And then you start trying to finish an actual real world application…and you realize all the bugs and problems in the code, and how the AI chatbot will mess up what it already had correct, change the wrong files, try to do risky things it shouldn’t be doing that have nothing to do with what you are trying to accomplish and spin its wheels going in circles. So the goal was - how can I fix that?
So that’s what my research has been about to date. Trying to build that real world application with appropriate security and guardrails in a reasonable amount of time for a reasonable cost.
I’m primarily using AWS Kiro CLI, not to be confused with the AWS Kiro IDE. They are very different animals. And yes, that is confusing. Kiro CLI is like Claude Code. I like using it because it works as good as Claude Code did for me. I can use all the same Anthropic models and switch to a new model by typing /model. I find AWS to have the best security architecture to protect your IP and your liability should use use someone else’s IP that came out of a model. I. know all the tools are trying to one-up each other but I think it’s more about the core of how you use any tool than the whiz-bang features each tool has to offer. But we’ll see how it works out.
I’ve tried Claude Code, I use Google AI Mode for simple scripts and things that I’m not trying to keep proprietary, asked ChatGPT a few questions, and I tried Grok out of the gate (which was not good for programming). I messed around with Facebook’s model in Amazon Bedrock because it is open source. There’s a lot to learn but so far my take is more about harnessing any model than which model is ever so slightly better than the other, and they change a lot.
AWS is also working on their own Nova models. I’m not sure if I get any of them when I’m using “Auto” mode in Kiro which is supposed to automatically select the right model for the job. It seems like that option is less expensive. They also offer what they call frontier models you can build on which sound really interesting - until you see the price. I don’t think I need that at the moment.
My take is it may be more about how you get the models and agents to course correct over actually getting them to be perfect out of the gate or getting the “smartest” model - because at the end of the day they are all prediction engines. That’s what I’ve been tinkering with so far.
I built a framework to help prevent agents from doing risky things immediately upon the first attempt to access AWS credentials when it did not need them and to try to execute commands with sudo. Then I figured out how to add multi-agent capabilities that helped me burn through tokens and supposedly complete over 300 requirements in a couple of days. But the resulting program was not perfect because UIs are difficult for one thing. The application kind of worked but I had to move onto some other things before I had time to fully test it. All those ideas are a work in progress I’ll probably write more about here.
If you want to see what I’ve done to date and catch up on
How and why I prefer working in the command line
Setting up AWS Identity Center for best protection of your IP using AWS AI tools
How to install Kiro CLI
How to vibe code your first program
Why you might choose Amazon’s AI solution over others for IP protections
Kiro custom agent and agent monitoring scripts
You can read about that here:
https://medium.com/cloud-security/artificial-intelligence-2e97415216c0
What I hope to do on this blog in the near future is share some of the things I’ve vibe coded that were actually helpful and what is working. I also have some pretty cool security use cases. You can also find brain dumps on my trials and tribulations on all my social media accounts as I try new things and comment on my successes and failures over there if you want to follow along.
Subscribe for more posts like this on AI and Good Vibes.
—Teri Radichel