JMeter Fury Fleet: AI Agents for Smarter Load Testing

Overview

Fury Fleet is my attempt to bring AI agents into the world of JMeter performance testing. Inspired by Feather Wand, I'm building it as a standalone web application using Python, Google ADK, and Streamlit, with a fleet of agents working together to simplify every stage of performance testing.

My goal: make performance testing easier, from creating test plans to understanding test results.

With Fury Fleet, you can:

• Generate JMeter test plans from OpenAPI/Swagger specs.
• Review existing .jmx files and spot issues in the test design.
• Compare API specs with test plans to find endpoints that aren’t covered.
• Analyze JMeter results (.jtl) to understand latency percentiles, error rates, degradation, and possible bottlenecks.

Consider rephrasing to 'My principle: let deterministic code handle data processing, and use AI for interpretation and suggestions.' Consider softening to 'minimizes hallucinated XML and guessing' or clarifying the limits of this approach.

Goals, Scope, and Non-goals

Goals

• A working MVP usable by practicing performance engineers.

In Scope

• HTTP/REST testing
• OpenAPI 3 and Swagger 2 specs
• CSV JTL files
• Single-user browser sessions

Non-goals (for now)

• Running JMeter itself
• Non-HTTP protocols
• XML JTL files
• Persistent history
• Per-user authentication
• Distributed load generation

Architecture

System Context

Layers and Components

Layer Responsibilities

Request Routing

Technology Stack

The Agents

Behavior Notes

• Consider splitting into two sentences or adding an example for clarity.
• script_generator calls parse_openapi, then generate_jmx, then analyze_jmx on its own output as a sanity check. It never claims that the test was executed.
• results_interpreter uses p95 SLA 1000 ms and 1.0% error limit if the user gives none, and says so. It It labels possible causes as hypotheses rather than facts.
• jmx_analyzer It reports only findings returned by the tool and never invents issues.
• coverage_gap It reports the coverage percentage, lists uncovered endpoints grouped by reads and writes, and identifies JMX requests not present in the spec.

Tools Reference

6.1 analyze_jmx (jmx_tools.py)

• analyze_jmx_file is a pure function: parses XML, walks enabled elements, collects thread groups, HTTP samplers, timers, assertions, and memory-heavy listeners.
• analyze_jmx wraps it and caches the result in session state.

Limits: HTTP samplers only; children of a disabled parent are still counted; property expressions like ${__P(threads,50)} are not evaluated.

6.2 parse_openapi and Helpers (openapi_tools.py)

• load_spec: reads file or http(s) URL, JSON or YAML.
• server_info: handles OpenAPI 3 servers and Swagger 2 host, schemes, basePath, including relative server URLs.
• sample_value: builds realistic values from a schema: example, default, enum, allOf/oneOf/anyOf, type-based values. Resolves $ref, stops at depth 5.
• extract_endpoints: flattens the spec into method, path, path params, required query params, JSON body, expected 2xx status.

6.3 generate_jmx (jmx_generator.py)

• Plan is assembled with ElementTree, parameterized for command-line tuning:
  • User Defined Variables: protocol, host, port, token placeholder, path param defaults.
  • HTTP Request Defaults: use ${host}, ${port}, ${protocol}.
  • Header Manager: JSON content type, Authorization Bearer ${token} if security declared.
  • Thread group: driven by ${__P(threads,10)}, ${__P(rampup,30)}, ${__P(duration,120)} with scheduler on.
  • Uniform Random Timer: ${__P(think_ms,500)} and ${__P(think_range_ms,1000)}.
  • One HTTP sampler per endpoint with response-code assertion on expected status.

Example run:

jmeter -n -t output/api.jmx -l results.jtl -Jthreads=10 -Jrampup=30 -Jduration=120

6.4 find_coverage_gaps (coverage_tools.py)

A path template like /users/{id} becomes a regex matching /users/42 and /users/${id}. The spec base path is stripped from sampler paths before matching. If the same JMX was already analyzed in this session, the cached parse is reused.

6.5 analyze_jtl (jtl_tools.py)

Statistics: samples, error percent, average, p50, p90, p95, p99, max, duration, throughput—overall and per label. Timeline is built only if the run lasts at least 10 seconds and has at least 50 samples.

Workflow

The Entire MVP Working, End to End

8.1 User Journey

In plain words: a tester brings an API spec, a plan, or a results file. The app saves each upload to that browser session's workspace, tells the agents the exact paths, and routes the request to the right specialist. The specialist calls deterministic tools for facts; the model explains them. Generated plans come back as downloads, and the loop closes when the plan is run in JMeter and the resulting .jtl is fed back for interpretation.

8.2 How Agents Share State

State lives for the session only. The coverage tool reuses a cached JMX parse only when the cached source_path matches the requested file.

Known Limitations

• HTTP/REST only; other samplers and protocols are ignored by the analyzer.
• CSV JTL only; XML JTL is rejected with a clear error.
• JMX property expressions like ${__P(threads,50)} are not evaluated, and children of a disabled parent are still counted.
• The generator builds a single thread group that covers all endpoints, without simulating user sessions, authentication flows, or realistic multi-step journeys.
• Sessions and files are in memory and on local disk. No history, no multi-instance scaling.
• The agents do not run JMeter and cannot measure anything themselves.