A hackbot is the term I’ve settled on when referring to AI systems with hacking abilities. I saw it used in this talk from Blackhat 2023.
With the recent release of the paper LLM Agents can Autonomously Hack Websites (which has its flaws, but I digress) there is finally some official research that agrees with the claims I’ve been making for the past year. Current AI models can autonomously hack given the right setup.
In a previous post, I covered why I think hackbots will outperform humans at some point. This post will be a comprehensive discussion on hackbots. What can a hackbot do? Why do you care? Do they already exist? Etc.
Hackbot Defined
I define hackbot as any automated system that uses AI in a meaningful way in order to find vulnerabilities in hardened production applications (such as those with bug bounty programs). Almost all hackbots today are much more than a simple LLM. They are advanced systems which combine traditional code, existing tools, fuzzers, etc. with AI breakthroughs.
The ways that AI is used in hackbot systems varies a lot, but here are some ways I can see AI being useful:
- Selecting which tool to run
- Selecting which configurations to set for specific tools
- Choosing which wordlists to use
- Guiding the system for which features to hack on
- Knowing which requests or parameters should be tested
- Picking which payloads should be used for testing
- Knowing whether the attacks were successful
- Writing reports
Hero: My Hackbot POC
A few months ago, I wrote a little hackbot proof of concept. In order to inspire and help people see how a hackbot might work, I’ve decided to release it with this blog post. Here’s the link: https://github.com/jthack/hero
The project name hero is from Snowcrash, where the main character (who is a hacker) is named Hero.
It takes a raw http request as input, gives AI the ability to suggest potential vulnerabilities, write the payload, as well as execute and validate the findings. I hope you have fun playing with it or making your own!
Why Do You Care?
The hackbot industry is fascinating and relatively mind-blowing to me for a few reasons:
- If humanity can build a decent hackbot, we could spin up thousands of them and find a massive percentage of vulnerabilities on the internet. That would help us secure it in an impactful way.
- Beyond that, if hackbots work well, they can test code before it goes live, significantly lowering vulnerabilities that make it into production.
- If a company can create a hackbot that can find bounties on bug bounty programs reliably, it’s basically a free-money machine until all the bugs it can find are gone.
- If an enemy nation-state has a highly competent hackbot, that’s a serious national security issue.
Current Hackbots
So, do they already exist? I believe there are many variations on hackbots out there. Most of them are likely stealth mode start-ups or being developed by government agencies.
The four companies I’m aware of are:
I know a bit about each one except Sybil, and I can say that each of them are completely different from one another. They’re tackling the problem from different angles. They have different strengths and weaknesses. Their teams have varied amounts of expertise in AI, hacking, software development, etc.
I think there will be two or three times as many within six months.
My Dreams For The Future of Hackbots
I have a few hopes and dreams for the hackbot industry that I want to put out into the world.
- To OpenAI and other foundation model makers: Please don’t nerf security expertise in your models. GPT4 is already denying me a good amount when using it for hacking. It’s not going to prevent this type of product. The builders will just move to open-source models.
- To bug bounty platforms: What would it look like if you were to ask your hackers for permission to use their reports in training a hacking-specific model?
- To all readers: This is pretty nuanced. LLMs by themselves can’t hack anything. LLMs in a system can hack some things. Over time they will be able to hack more. It’s hard to solve though.
Thanks,
Joseph
Sign up for my email list to know when I post more content like this.
I also post my thoughts on Twitter/X.