Codex vs Claude Code vs Gemini CLI: Where each AI agent fits in your Bug Bounty workflow

October 16, 2026

Codex vs Claude Code vs Gemini CLI: Bug Bounty lab results compared

We tested Claude Code, Codex and Gemini CLI against blind PortSwigger labs to see where each AI agent fits best in a Bug Bounty workflow.

In these limited, non-equivalent tests, Codex went furthest in autonomous exploit validation, Claude Code was strongest at discovery but needed human help to finish, and Gemini CLI quickly analysed captured traffic but refused active testing. These are practical observations, not a controlled benchmark or permanent model ranking.

Quick verdict: Which tasks does each AI Agent most excel at?

No single agent won every part of the workflow. Based only on the sessions documented in this series:

  • Claude Code was most useful for reconnaissance and vulnerability discovery. It found both client-side and server-side flaws, but a human had to guide or confirm the final exploitation steps
  • Codex produced the most complete active-testing results. It independently finished the shared DOM XSS test and verified administrator access in its JWT test
  • Gemini CLI worked best as a passive analyst. It reviewed captured HTTP traffic quickly and surfaced accurate leads, but active exploitation and final validation remained manual

The practical takeaway is to use these tools –alongside human judgement – according to their respective strengths. We can use an active agent to collect and test evidence, a fast passive reviewer to inspect larger bodies of code or traffic, and our own judgement to decide what is safe, in scope and proven.

How we tested Claude Code, Codex and Gemini CLI

Each agent was evaluated in two PortSwigger mystery labs: one client-side challenge and one server-side challenge. The vulnerability class was hidden, and the prompts prohibited the agent from decoding the Base64 objective embedded in the page. All sessions used Burp Suite traffic, either through active MCP access or passive review of requests and responses we had already captured.

The conditions were not identical:

  • Claude Code used Opus 4.8 for active testing. It received the same minimal attacker prompt in both labs
  • Codex used GPT-5.6 Sol. Personal security skills were disabled, and the disposable labs ran with unrestricted tool approval
  • Gemini CLI used Flash-3.5. It first received the active-testing prompt, refused it and was then asked to analyse captured traffic without interacting with the target

All three agents encountered the same DOM XSS lab, which provides the clearest shared signal. Their server-side labs were different however: H2.TE request smuggling for Claude Code, a JWT authentication bypass for Codex, and Java deserialization for Gemini CLI. The tests also ran at different times and did not use equivalent interaction modes. The results therefore show how each agent behaved in a small set of real sessions.

AI agent lab results at a glance

  • Claude Code with Opus 4.8
    • Testing mode: Active DOM XSS and H2.TE request smuggling tests
    • Observed result: Found both vulnerabilities. The XSS attempt stalled for almost 20 minutes until we identified a missed redirect. The server-side result took 34 minutes and required manual confirmation
    • Best-supported role: Reconnaissance, application reasoning and vulnerability discoveryTesting mode: Active DOM XSS and H2.TE request smuggling tests
  • Codex with GPT-5.6 Sol
    • Testing mode: Active DOM XSS and JWT authentication bypass tests
    • Observed result: Completed the XSS exploit in under four minutes and verified administrator access through a forged JWT in one minute 14 seconds
    • Best-supported role: Active weaponisation, exploitation and evidence validationTesting mode: Active DOM XSS and JWT authentication bypass tests
  • Gemini CLI with Flash-3.5
    • Testing mode: Passive review of DOM XSS and Java deserialization traffic after refusing active testing
    • Observed result: Identified both leads within seconds. XSS payload worked when tested manually, while the deserialization hypothesis required a human-generated payload and out-of-band confirmation
    • Best-supported role: Fast secondary review of captured traffic and codeTesting mode: Passive review of DOM XSS and Java deserialization traffic after refusing active testing

These timings are observations from individual sessions, not repeatable performance benchmarks. Lab difficulty, model version, permissions, prompts and network conditions can all change the result.

Claude Code: strong discovery, human-guided exploitation

In the full Claude Code Bug Bounty test, Opus 4.8 detected the vulnerable client-side JavaScript in the DOM XSS lab within seconds. It correctly identified the postMessage flaw and prepared a payload tailored to the weak URL validation.

The payload was valid, but Claude Code did not complete the delivery workflow. It treated the initial request as if it were enough, missed a redirect and continued to repeat the same unsuccessful process. After almost 20 minutes, we pointed out the redirect. Claude Code then followed it and completed the lab.

The server-side test showed a similar division of labour. Claude Code fingerprinted the application, tested several plausible vulnerability classes and eventually identified an H2.TE request smuggling flaw with response queue poisoning. It supplied a usable Proof-of-Concept strategy after 34 minutes, but we still had to reproduce the request in Burp Suite and confirm that the poisoned queue exposed another user’s session response.

These runs support a narrow conclusion: Claude Code was effective at building an application model and finding promising vulnerability paths, but it did not reliably make the transition from a plausible exploit to independently verified impact. In a real Bug Bounty Program, where a missed state transition can invalidate a report, its output should remain a lead until we reproduce the result.

Codex: Stronger autonomous exploitation and validation

The Codex Bug Bounty test used GPT-5.6 Sol with high reasoning effort and no vulnerability hints.

In the shared DOM XSS lab, Codex found the same vulnerable message handler as Claude Code. It initially took an unnecessarily complicated route, then simplified the payload, delivered it to the victim and checked that the lab status changed to “Solved”. The DOM XSS run took under four minutes.

For the server-side test, Codex authenticated with a normal account, inspected the session JWT and rejected an unsuccessful alg: none attempt. It then inferred that the kid header selected a signing-key path, redirected that path to /dev/null, signed a forged administrator token with a null-byte key and confirmed access to the administration panel. It reached that verified state in one minute and 14 seconds.

Codex showed the strongest autonomous weaponisation and validation in these particular sessions. This does not prove that Codex would outperform Claude Code on any given vulnerability class, model release or target.

Gemini CLI: fast Passive analysis, manual proof

Gemini CLI behaved differently to the two active agents. Even after the prompt identified the lab as authorised and in scope, it refused to interact with the target.

We then changed its role from active tester to HTTP response analyst and asked it to review traffic captured in Burp Suite.

In the DOM XSS lab, Gemini extracted the inline JavaScript within seconds. It noticed the missing origin check, traced untrusted postMessage data into location.href and explained how a javascript: URL could bypass the weak substring validation. It also proposed a working payload, which we delivered and confirmed manually.

In the server-side session, Gemini spotted a Base64-encoded session cookie and recognised the Java serialization magic bytes AC ED 00 05. That was a useful lead, but a serialized object alone does not prove unsafe deserialization or remote code execution. Gemini suggested that a gadget chain could be used. We then generated a serialized gadget using ysoserial and confirmed execution through DNS and HTTP callbacks in Burp Collaborator.

The evidence supports using Gemini CLI as a second pair of eyes when analysing incoming traffic. It was fast at locating suspicious details and explaining why they mattered, but the active requests, exploit delivery and proof of impact came from us.

Which AI agent was best for Bug Bounty hunting overall?

If we had to choose one agent based only on the active lab sessions in this series, Codex produced the most complete end-to-end results. It found the shared DOM XSS and independently verified the exploit, then adapted after a failed JWT hypothesis and demonstrated administrator access in its second lab.

That does not make Codex a universal winner. Claude Code was a capable discovery partner whose reconnaissance and reasoning produced valid vulnerability paths, while Gemini CLI was an effective passive reviewer that surfaced important details from captured traffic within seconds. Their different strengths make a role-based workflow more defensible than a single permanent ranking:

  1. Use Claude Code or Codex to map the application and form testable hypotheses
  2. Use Gemini CLI for a separate passive review of the code and traffic
  3. Require reproducible requests, responses and state changes before accepting any finding
  4. Keep a human responsible for scope, safety, evidence quality and the final report

And remember, only run active agent testing against authorised, in-scope assets. Finally, restrict permissions on uncontrolled targets, avoid techniques that could affect real users, and never treat a plausible LLM explanation as proof of impact.

References & related reading