Table of contents
25
By Salman Khan, OSCP+, Founder of XHack, SRT (Synack Red Team member)
Read this in 30 seconds: There are now several serious open source AI pentest tools, and the biggest ones have tens of thousands of GitHub stars. Stars tell you what is popular, not what is safe, maintained or accurate. I pulled live GitHub data on 13 November 2026, read each project’s README, and wrote a short script that flags the problems stars hide.
- Strix leads on popularity. With 75,232 stars and an Apache-2.0 licence, it is the most starred of the eight projects, ahead of Shannon at 51,807 and PentAGI at 34,067.
- Licences differ across open source AI pentest tools. Shannon is AGPL-3.0, and CAI combines MIT components with proprietary additions licensed for research only, so read the terms before you build on either.
- One of the eight is archived. CAI says it is no longer under active development, after a run from March 2025 to August 2026.
- These tools execute real exploits. Shannon’s own README says its agents can create users, submit forms and change application state, so use staging and never production.
- Every finding needs a human. Shannon’s README warns that LLM-generated reports can contain weakly supported or incorrect details, and I would apply that to all of them.
The risk with an open source AI pentest tool is rarely the code you read, it is the live system you point it at.
Search for open source AI pentest tools and you get star counts, “awesome” lists and a lot of enthusiasm. What is missing is a plain comparison with the boring facts: licence, last commit, what it needs to run, and what its own documentation says about safety.
This guide gives you that for eight projects, using GitHub’s API and each README as primary sources. It explains what each tool is, where the claims stop being evidence, how to run any of them without hurting yourself, and when a hosted option makes more sense. I run XHack, so I say where we fit. It contains no exploit steps.
An open source AI pentest tool is a project whose code you can read and run yourself, and that uses a language model to plan or perform security testing. In practice that covers four different things, and mixing them up is how people end up comparing the wrong projects.
Two caveats before the comparison. Several of these are open core: Strix also sells a cloud and an enterprise product, and Shannon’s README lists a $50 per active developer per month Pro edition and a custom Enterprise edition next to the open source one. The free version is real, but it is also the top of a sales funnel. And almost all of them need a language model API key, so the software is free while the model usage is not.
Here are eight open source AI pentest tools side by side, using GitHub API data from 13 November 2026. Stars move daily, so treat them as a snapshot.
| Project | Stars | Licence | Language | What it is | Last push |
|---|---|---|---|---|---|
| Strix | 75,232 | Apache-2.0 | Python | Agent team with a Docker sandbox and CI mode; open core | today |
| Shannon | 51,807 | AGPL-3.0 | TypeScript | Source-aware web and API pentester | today |
| PentAGI | 34,067 | MIT | Go | Self-hosted multi-agent system with 20+ tools | 21 days ago |
| PentestGPT | 16,410 | MIT | Python | Drives Claude Code or Codex; classic mode kept | 121 days ago |
| HexStrike AI | 13,559 | MIT | Python | MCP server with 150+ tools and 12+ agents claimed | 101 days ago |
| CAI | 9,857 | Custom | Python | Archived August 2026, research-only terms | 82 days ago |
| PentestAgent | 3,291 | MIT | Python | Agent framework with Docker mode and scoped child agents | 4 days ago |
| ARTEMIS | 550 | Apache-2.0 | Rust | Research agent from the Stanford study | 213 days ago |
These notes come from each project’s own README, so they describe what the maintainers claim, not what I measured. I have not benchmarked any of them.
Strix describes itself as autonomous AI pentesting agents that run your code dynamically and validate findings with proof-of-concept exploits. It runs in a Docker sandbox and needs a language model key, and it can point at a local model through an API base setting. It has a CI mode that scopes pull-request runs to changed files. The README also advertises auto-fix and compliance-ready reports, which are claims I have not tested.
Shannon is a white-box tool: it reads your source code and attacks the running application, and its README says it ties runtime behaviour back to the code. It mounts the repository read-only inside an ephemeral Docker container and supports several model providers, including local models through Ollama, vLLM or LM Studio. Its README says a full run takes roughly 1 to 1.5 hours and may cost model fees, and it says plainly that Shannon is not a replacement for human pentesters.
PentAGI calls itself a self-hosted multi-agent system for autonomous penetration testing, with 20+ professional tools, installed through Docker Compose. It works with more than ten cloud model providers or local models, and its README says it is for authorised testing only. The company behind it, VXControl, describes itself as a security engineering firm.
PentestGPT began as a human-in-the-loop research tool published at USENIX Security 2024. Its README now describes an autonomous mode that drives Claude Code or Codex, with a Docker flow for running it, and says the classic version is preserved. Its last push was 121 days ago, so check that the mode you want is the one being maintained.
HexStrike is an MCP server that lets a model call a large toolbox. Its README claims 150+ security tools and 12+ autonomous agents, and lists approved uses such as authorised penetration tests and bug bounty work within scope. It also states a “CTF success rate” of 89% against a 65% human expert average. I could not find a method for that number in the README, so treat it as marketing until the author shows how it was measured.
PentestAgent is an agent framework for black-box testing with a terminal interface. It can run its tools inside a Docker container, including a Kali image, and lets a running agent spawn child agents with their own scope settings. It needs an API key for a supported model provider.
ARTEMIS is the agent behind the Stanford, Carnegie Mellon and Gray Swan study “Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing” (arXiv 2512.09882). The repository is Apache-2.0 and written in Rust, but it was last updated in April 2026. Use it to understand the research, not as a maintained product.
CAI, the Cybersecurity AI framework from Alias Robotics, is archived. Its README says it is no longer under active development, that the work ran from March 2025 to August 2026, and that the history was squashed into one commit. Its licence combines MIT-licensed components with proprietary additions licensed for research purposes only. It is useful reading, and it is a warning that a popular repository can stop moving overnight.
Stars measure attention, and attention is not accuracy, safety or support. Four things sit behind the star count of any open source AI pentest tool.
Treat every open source AI pentest tool as live ammunition: isolate it, point it at systems you own or have written permission to test, and keep a hand on the stop button. The projects say the same thing in their own words.
Shannon’s README is the clearest. It says exploitation agents can create users, submit forms, change application state and trigger outbound requests, and that you should use sandboxed, staging or local development environments and never production. It also warns against scanning untrusted or adversarial codebases, because a tool that reads source code can be exposed to prompt injection. HexStrike’s README lists unauthorised testing as a thing never to do.
The practical setup is short. Run the tool in a container or a throwaway virtual machine, not on the laptop that holds your keys. Give it a read-only copy of the source where the tool supports that. Set a scope that names your staging hosts, and keep production out of it. Use a model key with a spending limit, and know how to kill the run before you start one.
This short script reads GitHub’s public API and flags archived, stale, unlicensed and copyleft projects. It clones nothing and installs nothing, and it only reads public metadata. I ran it on the eight projects above on 13 November 2026 and the output below is real.
import json
import os
import sys
import urllib.request
from datetime import datetime, timezone
STALE_DAYS = 180
def fetch(repo):
req = urllib.request.Request(f"https://api.github.com/repos/{repo}", headers={"User-Agent": "oss-check"})
if os.environ.get("GITHUB_TOKEN"):
req.add_header("Authorization", "Bearer " + os.environ["GITHUB_TOKEN"])
with urllib.request.urlopen(req, timeout=20) as r:
return json.load(r)
def flags(d, now):
out = []
days = (now - datetime.fromisoformat(d["pushed_at"].replace("Z", "+00:00"))).days
lic = (d.get("license") or {}).get("spdx_id") or "none"
if d["archived"]:
out.append("ARCHIVED")
if days > STALE_DAYS:
out.append("STALE")
if lic in ("none", "NOASSERTION"):
out.append("CHECK-LICENSE")
if lic.startswith(("AGPL", "GPL")):
out.append("COPYLEFT")
return days, lic, out
def main():
now = datetime.now(timezone.utc)
print(f"{'repo':28} {'stars':>7} {'license':12} {'pushed':>8} flags")
for repo in sys.argv[1:]:
d = fetch(repo)
days, lic, f = flags(d, now)
print(f"{d['full_name']:28} {d['stargazers_count']:>7} {lic:12} {str(days) + 'd':>8} {' '.join(f) or 'ok'}")
if __name__ == "__main__":
main()Save it as oss_check.py and run it with the repositories you are weighing:
python3 oss_check.py usestrix/strix KeygraphHQ/shannon vxcontrol/pentagi GreyDGL/PentestGPT 0x4m4/hexstrike-ai GH05TCREW/pentestagent aliasrobotics/cai Stanford-Trinity/ARTEMISrepo stars license pushed flags
usestrix/strix 75232 Apache-2.0 0d ok
KeygraphHQ/shannon 51807 AGPL-3.0 0d COPYLEFT
vxcontrol/pentagi 34067 MIT 21d ok
GreyDGL/PentestGPT 16410 MIT 121d ok
0x4m4/hexstrike-ai 13559 MIT 101d ok
GH05TCREW/pentestagent 3291 MIT 4d ok
aliasrobotics/cai 9857 NOASSERTION 82d ARCHIVED CHECK-LICENSE
Stanford-Trinity/ARTEMIS 550 Apache-2.0 213d STALE
A flag is a question, not a verdict. A stale research repository can still teach you something, and a copyleft licence is fine for internal use. What the script cannot tell you is whether the code is safe, and that still means reading it.
An open source AI pentest tool is free to download, but the rest is not. You pay for model usage, for compute, and for the time you spend setting up, updating and babysitting a project that has no support desk.
Shannon’s README gives the only run-time figure I found: a full run takes roughly 1 to 1.5 hours and may incur model costs depending on pricing and application complexity. None of the other READMEs give a cost per test, so budget with a small trial and a spending cap on the key. If you outgrow the free version, check the commercial editions. Shannon, for example, lists Pro at $50 per active developer per month, and prices change, so confirm on the vendor’s page.
An open source AI pentest tool fits when you want control and can carry the operating burden. It fits less well when you need an accountable person, a report an auditor will accept or a team that is not security-literate.
| Your situation | Open source fits? | Why |
|---|---|---|
| Researcher or bug hunter on authorised targets | Often yes | You want control and are the accountable human |
| Developer testing your own staging app in CI | Yes | White-box tools like Shannon and Strix are built for this |
| Team that needs a signed report for a customer | Rarely alone | You need a named person and evidence an auditor can read |
| Regulated company with a compliance deadline | Support only | Your assessor decides what counts, so ask first |
| No one to maintain it, patch it and watch the model bill | No | Free software still needs an owner |
So yeah, I am biased, it is my company, and I think XHack is the strongest all-in-one answer when you do not want to host an open source AI pentest tool yourself. XHack combines three things most providers sell separately: an autonomous AI agent, a continuous security platform and human testers, all hosted, with accountability attached.
Here is what I can state from our public pages.
Open source gives you control and source you can read, and it hands you the operating risk. If you are weighing both, run an open source AI pentest tool on your own staging app and run XHack on the same one, then compare what each finds and how much of it a human can reproduce. See our broader AI pentest tools guide, the XBOW comparison, the pricing page and our AI VAPT services.
There is no independent ranking, so the honest answer depends on your job. Strix has the most stars, Shannon is built for white-box testing of your own application, and PentAGI is a self-hosted multi-agent system. Pick by licence, maintenance and safety fit, then run a small trial on a system you own.
They are as safe as the environment you run them in. These tools execute real exploits, and Shannon’s README says they can change application state and should not touch production. Run an open source AI pentest tool in a container or throwaway virtual machine, against systems you own or have written permission to test.
Running one on your own systems is generally fine, and running one on systems you do not own or have permission to test is not. This is not legal advice. Get written authorisation, stay inside the agreed scope and check your local law and contracts.
No. Shannon’s README says it is not a replacement for human pentesters, and the Stanford study found agents produced more false positives than humans. An open source AI pentest tool is good for breadth and repetition, and a person still has to confirm findings and own the result.
The code is free, but model API usage, compute and your time are not. Shannon’s README says a full run takes roughly 1 to 1.5 hours and may incur model fees. Some projects, such as Strix and Shannon, also sell commercial editions, so check the vendor’s pricing page.
Look at the last push, whether the repository is archived and who maintains it. The script above flags archived and stale repositories in seconds. CAI shows why this matters, since it was archived in August 2026 with nearly 10,000 stars.
The best open source AI pentest tools are genuinely capable, and the sensible way to use them is the boring way. Check the licence and the last push, run the tool in a sandbox, point it only at systems you are allowed to test, and treat every finding as a lead until a human has reproduced it. Stars will tell you what is trending, and the checks will tell you what is safe to try. If you would rather not run the stack, XHack is the hosted alternative, and the same trial approach works on us too.
Categories
Related articles
