Currentness not reverified

This published article remains available in the archive, but its technical currentness has not yet been reverified. Validate it against current authoritative sources before use.

AI Agent vs. Human with Wireshark: Six Malware PCAPs Put to the Test

This experiment measures how Daybreak Blue at extra-high reasoning investigated six malicious packet captures, where it succeeded, what it missed, and how its work compares with a trained human analyst.

Article Metadata

• Category: AI Security
• Topics: AI Security, Malware Analysis, Network Security, Digital Forensics, Security Tooling
• Source article: Medium — AI Agent vs. Human with Wireshark: Six Malware PCAPs Put to the Test
• Published: 2026-09-18
• Preserved media: 9 unique images—the published cover and all eight infographics.
• Preserved technical blocks: 3 code/configuration blocks.

Ecosystem Fit

This article joins the local 1200km malware-analysis and AI-security archive. Its methodology, measurements, limitations, and repeatable TShark examples are preserved here independently of Medium.

Packet-capture analysis looks deceptively simple when the answer is already known. Find an internal IP, follow a suspicious stream, extract several indicators, and write a short incident summary. Real analysis is harder. The analyst must separate malicious traffic from normal Windows noise, connect network identities to users, avoid blaming shared infrastructure, distinguish successful compromise from mere exposure, and state what encrypted traffic does not prove.

I tested whether an AI coding agent could perform that complete workflow across six malware traffic exercises. The agent had local access to the PCAP files and command-line tools, but the official answers were withheld until the independent findings were written and cryptographically hashed. This created an auditable boundary between original analysis and answer-key comparison.

The result was more useful than a simple “AI versus human” score. The agent processed 251,249 packets, produced six detailed incident reports and a comparative report, recovered all requested values after one correction pass, and exposed several mistakes in the official answer material. It also demonstrated a recognizable AI failure mode: relevant evidence had already been collected, but some of it was not selected during the first interpretation pass.

Scope and safety: These were offline training captures from Malware-Traffic-Analysis.net. Exported scripts and executables were handled as untrusted malware and inspected statically. Nothing extracted from the captures was executed. The timing and quality figures in this article apply only to this experiment; they are not a general benchmark for every model, analyst, PCAP, or incident.

Table of contents

1. Experiment design
2. Model and analysis environment
3. Dataset and workflow
4. Results from the six captures
5. Accuracy and official-answer comparison
6. Time, report size, tokens, and quality
7. Theoretical human-analyst comparison
8. What the agent did well
9. Where the agent failed
10. Reproducing the experiment
11. Limitations
12. Conclusion
13. References
14. Follow My Work

Experiment design

The starting point was the 2026-09-11 “KongTuke Rebuke!” capture. I then selected the next five exercises listed beneath it on the Malware-Traffic-Analysis.net training page:

1. 2026-08-09 — First to Last
2. 2026-02-28 — Easy as 123
3. 2026-01-31 — Lumma in the Room-ah!
4. 2025-06-13 — It’s a trap!
5. 2025-01-22 — Download from fake software site

The experiment had four controls.

First, the agent analyzed local packet evidence before opening the official answer PDFs. Second, each independent report was hashed before comparison. Third, the agent had to distinguish observations, inferences, and unconfirmed claims. Fourth, no parallel subagents were used. One agent performed evidence collection, interpretation, comparison, and report writing.

This ordering matters. If the answer key is opened first, host attribution becomes a lookup exercise. It no longer tests whether the analyst can discover the answer from DHCP, NBNS, Kerberos, SAMR, DNS, HTTP, TLS, and conversation metadata.

The evaluation included more than the questions printed on each exercise page. The agent was also expected to reconstruct infection timelines, identify malicious and suspicious infrastructure, quantify C2 activity, extract and hash transferred objects where possible, document negative findings, recommend response actions, and explain confidence limits.

Model and analysis environment

The configured model was Daybreak Blue, using alias gpt-daybreak-blue-latest, with reasoning effort set to extra high (xhigh). This configuration is experiment metadata supplied by the session operator.

Official OpenAI documentation describes Daybreak Blue as an alias for flagship general-purpose models with safeguards calibrated for defensive cybersecurity work (Daybreak Blue model documentation). Because an alias can resolve to an underlying model that changes over time, the reproducible disclosure is the Daybreak Blue alias and xhigh reasoning setting. The session did not expose an immutable backend snapshot identifier.

The agent combined model reasoning with deterministic local tools:

• capinfos for capture metadata and integrity information;
• TShark for protocol fields, conversations, endpoints, stream extraction, and display filters;
• standard hashing and file-identification tools for exported objects;
• unzip and pdftotext for the publisher’s answer archives;
• a generated shell collector that repeated the same evidence pass across five PCAPs;
• web access only for the exercise pages, downloads, official answers, and references.

This distinction is important. The language model did not internally decode 251,249 raw packets by intuition. It orchestrated protocol-aware tools, inspected structured outputs, formed hypotheses, requested narrower pivots, and converted corroborated results into reports. The TShark manual describes the underlying mechanism: a saved capture can be read with -r, filtered with -Y, and emitted as decoded fields or statistics.

Dataset and workflow

The six captures contained 251,249 packets and 178,964,161 captured bytes in total.

Exercise	Packets	Captured bytes	Capture duration
KongTuke Rebuke	73,779	59,484,402	21m09s
First to Last	22,473	16,007,378	14m00s
Easy as 123	15,512	6,581,991	4h21m29s
Lumma in the Room-ah!	51,181	27,242,609	10m24s
It’s a trap!	48,877	42,888,106	34m32s
Fake software site	39,427	26,759,675	53m22s

The repeatable analysis sequence was:

archive acquisition and hashing
        ↓
capture metadata and protocol hierarchy
        ↓
internal host, MAC, hostname, account, full-name attribution
        ↓
DNS, HTTP, TLS SNI, JA3, endpoints, and conversation triage
        ↓
stream-level timeline and byte/count measurements
        ↓
safe object export, static inspection, and hashing
        ↓
independent incident report and pre-answer checksum
        ↓
official-answer download and comparison
        ↓
uniform correction pass and comparative report

The collector generated TSV files for IP packets, TCP SYNs, DHCP, identity protocols, DNS, HTTP requests and responses, TLS ClientHellos, ICMP, and Windows service protocols. It also generated endpoint, conversation, protocol-hierarchy, and string summaries.

Identity was derived from packet fields instead of guessed from naming conventions. IP/MAC/hostname bindings came from DHCP and local-name protocols. Kerberos exposed account names. SAMR field samr.samr_UserInfo21.full_name exposed directory-backed full names when that traffic was present.

Results from the six captures

KongTuke: ClickFix, operational C2, and AD discovery

The infected system was 10.9.11.135, MAC 08:d4:0c:7a:29:1e, hostname DESKTOP-6T17ZFM, used by Gabriel McDowell (gmcdowell). The browsing chain reached a fake-verification stage and was followed by non-browser TLS, staging infrastructure, cleartext HTTP C2, and Active Directory discovery. This is consistent with ClickFix-style social engineering; MITRE ATT&CK describes user-assisted malicious copy and paste as T1204.004.

The malware generated 99 malicious HTTP requests, including 43 POSTs with 4,926,001 bytes of application data. The primary C2 returned successful HTTP 200 and 204 responses, which proved an operational channel. A 4.447 MB high-entropy POST strongly supported collection or exfiltration, but its content remained encrypted at the application layer.

The endpoint queried domain accounts, groups, computers, DC information, LDAP, SMB, Kerberos, and Group Policy. The report did not claim DCSync because no DsGetNCChanges operation was present. It also found no packet evidence of domain-controller exploitation or lateral movement.

First to Last: FormBook/XLoader traffic across 15 hosts

Two Windows workstations were present, but only 172.16.8.49, DESKTOP-5NLV63K, account rvance, generated the malicious pattern. The endpoint sent 131 patterned HTTP requests—113 POSTs and 18 GETs—with 4,529,609 declared POST-body bytes.

The supplied alerts called the activity FormBook. The official notes called it likely XLoader, a FormBook successor or rebrand. The network evidence supported the combined FormBook/XLoader-style classification but did not independently resolve product naming.

The agent confirmed all six alerted IP addresses and found nine additional fallback host/IP pairs. Its strongest behavioral correlation combined an obsolete Firefox 39 user agent, repeating body sizes, rotating paths, and a consistent POST/GET cadence. It also warned against globally blocking shared CDN IP addresses without hostname context.

Easy as 123: NetSupport RAT hidden on port 443

The victim was 10.2.28.88, DESKTOP-TEYQ2NR, account brolf. It contacted 45.131.214.85:443 for approximately 4h20m37s and sent 264 POST requests with a near-60-second cadence.

Port 443 did not mean TLS. The traffic was cleartext HTTP, identified itself with NetSupport Manager/1.3, and used /fakeurl.htm. NetSupport is legitimate remote-support software, but the direct-to-IP infrastructure, protocol masquerading, long unattended session, and beacon cadence established malicious use as a RAT.

Lumma: browser fingerprinting and follow-up TLS

The victim was 10.1.21.58, DESKTOP-ES9F3ML, account gwyatt. The domain whitepepper.su resolved to 153.92.1.49 and received Chrome and Edge fingerprint submissions through /api/set_agent.

The two cleartext POSTs totaled 15,998 bytes and contained OS, browser, locale, hardware-thread, screen, font, plugin, canvas, WebGL, GPU, and network characteristics. Sixteen TLS sessions to the same infrastructure followed, including substantial server-to-client transfers.

Associated suspicious activity included holiday-forever.cc, communicationfirewall-security.cc, and the earlier whooptm.cyou. The last domain was temporally associated but remained unconfirmed in function. The report did not claim knowledge of the exact stolen data because the later traffic was encrypted.

It’s a trap!: PowerShell staging and rotating HTTP C2

The victim was 10.6.13.133, DESKTOP-5AVE44C, account rgaines. A Microsoft-themed lookalike stage caused PowerShell to download a 5,485-byte script. Static decoding showed that it ran systeminfo, uploaded the output, and evaluated the server response with Invoke-Expression.

The returned second stage was 991,303 bytes of obfuscated PowerShell. A following TLS stream from windows.php.net transferred roughly 33 MB, after which the endpoint began 30-second POSTs across three rotating hostnames.

The capture contained 63 malicious POSTs totaling 1,895,050 declared body bytes. The mechanism proved execution, system discovery, transfer, exfiltration, and resilient web C2. The exact malware family was not recoverable and was left unknown. The exercise publisher explicitly states that no official answers will be posted.

Fake software site: PowerShell, TeamViewer, and three C2 IPs

The victim was 10.1.17.215, DESKTOP-L8C5GSJ, account shutchenson. The likely fake Google Authenticator domain was authenticatoor.org. The chain downloaded a VBScript component and obfuscated PowerShell from 5.252.153.241, then retrieved TeamViewer components and established persistence.

The controller received 594 victim GET requests and served more than 6 MB. Persistent encrypted C2 used 45.125.66.32:2917 and 45.125.66.252:443. Traffic also reached a legitimate TeamViewer relay. The report correctly treated that relay as abused third-party infrastructure rather than an inherently malicious vendor server.

The final C2 answer—5.252.153.241, 45.125.66.32, and 45.125.66.252—matched the official answer exactly. The publisher also declined to name the malware family, supporting the agent’s bounded attribution.

Accuracy and official-answer comparison

Before viewing the answers, the agent supplied 28 of 31 requested values, all correct against packet evidence. Accuracy at the independent freeze was therefore 90.3% for complete question coverage, with no supplied value shown to be wrong by the PCAP.

The three omissions were full names. The agent found the account names rvance, brolf, and gwyatt but initially stated that expanding initials would be an unsupported guess. That caution was correct as a principle, but incomplete as analysis: the captures contained direct SAMR full-name fields. A uniform post-comparison query recovered Raymond Vance, Becka Rolf, and Gabriel Wyatt. Final packet-grounded coverage became 31/31.

Exercise	Independent freeze	After uniform validation	Comparison result
KongTuke	5/5	5/5	Exact official match
First to Last	4/5	5/5	Official misspells packet-proven Raymond as “Ryamond”
Easy as 123	4/5	5/5	Full name recovered; answer list agrees
Lumma	5/6	6/6	Official answer list mistypes the correct internal IP
It’s a trap!	4/4	4/4	No official answer exists
Fake software	6/6	6/6	Exact official match

The answer comparison produced an unexpected result: the official material was not flawless.

• The First-to-Last PDF writes Ryamond Vance; frame 2947 contains Raymond Vance.
• The Lumma answer list writes 10.1.28.58; its own walkthrough and the PCAP show 10.1.21.58.
• The Easy-as-123 PDF once changes the C2 from 45.131.214.85 to 145.131.214.85 and includes a copied caption naming Gabriel Wyatt instead of Becka Rolf.
• The First-to-Last exercise page and PDF disagree about the domain-controller address; the PCAP resolves the discrepancy.

This is a valuable incident-response lesson. An answer key, threat-intelligence report, SIEM alert, or model output is a claim. Packet evidence remains the validation source.

Time, report size, tokens, and quality

Measured time

The experiment did not capture a reliable end-to-end timer for every phase. It would be misleading to reconstruct one afterward. What was measured precisely was the automated multi-pass collection time for each of the five newly downloaded PCAPs.

Exercise	Automated packet-analysis pass
First to Last	35.887 seconds
Easy as 123	26.856 seconds
Lumma in the Room-ah!	82.237 seconds
It’s a trap!	62.884 seconds
Fake software site	55.237 seconds
Total	263.101 seconds (4m23.101s)

The measured interval from the first collector start to the independent five-report freeze was 18m58s. This included overlapping collection, interpretation, targeted pivots, and report drafting. Downloads happened earlier. Official-answer retrieval, comparison, direct SAMR rechecks, and the combined report happened afterward and were not captured by a single end-to-end timer.

The earlier KongTuke run was not timed at the same level. Its file creation window was approximately 55 seconds, but that is not the same as total analysis time and is not presented as such.

Report size

After answer comparisons, the six individual reports contained 5,962 words. The combined comparative report contained another 1,825 words, for a measured experiment corpus of 7,787 words before this article.

Token accounting

Exact input, cached-input, reasoning, tool-output, and generated-output token counts were not exposed to the session. Therefore, the token result for this experiment is not available.

Words cannot be converted into an exact token total. Hashes, IP addresses, domain names, code, Markdown tables, and packet fields tokenize differently from ordinary prose. Reporting a guessed word-to-token conversion would create false precision. Cost also cannot be calculated reliably without the exact model, token categories, and usage record.

Quality score

Quality was self-audited with a 100-point rubric:

• host and task attribution: 20 points;
• malicious mechanism and timeline: 25 points;
• IOC coverage: 20 points;
• evidence boundaries and negative findings: 15 points;
• official comparison: 10 points;
• reproducibility: 10 points.

For the no-answer 2025-06 case, the available 90 points were normalized to 100.

Exercise	Quality	Main deduction
KongTuke	100/100	No material report defect identified
First to Last	95/100	Full-name field missed initially; family label required refinement
Easy as 123	96/100	Full-name field missed initially
Lumma	92/100	Full name and one suspicious domain missed initially
It’s a trap!	98/100 normalized	Exact family remained unknown; no official comparison existed
Fake software site	100/100	All requested values and C2 roles independently recovered

These are transparent internal review scores, not independent scientific ratings.

Theoretical human-analyst comparison

No human analyst performed the same six cases under controlled conditions, so this section is a workload model, not a measured human benchmark.

The comparison assumes a trained SOC or network-forensics analyst who knows Wireshark, TShark, Windows domain protocols, HTTP/TLS triage, and safe malware handling. The analyst has the same PCAPs, no prior access to the answers, and must produce reports of comparable depth—not merely answer the exercise questions.

The modeled work includes acquisition and hashing, broad triage, identity attribution, malicious-stream reconstruction, object extraction, IOC classification, negative-evidence review, report writing, official comparison, and a final cross-case synthesis.

Work item	Theoretical human range
KongTuke deep analysis	2.5–4.0 hours
First to Last	1.0–1.5 hours
Easy as 123	0.5–1.0 hour
Lumma	1.0–1.5 hours
It’s a trap!	2.5–4.0 hours
Fake software site	2.0–3.0 hours
Answer comparison and combined reporting	1.5–2.5 hours
Modeled total	11–17.5 analyst hours

This range is derived from task decomposition, not a published productivity study. A specialist familiar with these exact exercises could be faster. A junior analyst, GUI-only workflow, weak hardware, unfamiliar Windows protocols, or formal evidence requirements could take longer.

Expected human result at equivalent quality

A careful human analyst should target all 31 packet-grounded task values, the major C2 timelines, safe IOC categorization, and the same evidence boundaries. The human would probably have an advantage in noticing that “full name not yet found” should trigger a broader directory-protocol search before report freeze. A human may also be more naturally skeptical of a plausible hostname such as windows.php.net or an official answer containing a typo.

The agent’s advantage was breadth at speed. Once a field set was defined, it applied identical collection logic to every capture, counted requests and bytes exactly, hashed objects consistently, and assembled cross-case comparisons without the fatigue of repeating the same filters.

The fairest conclusion is not that the agent completed 14 hours of human work in 19 minutes. The complete AI experiment was not timed end to end, and model/tool infrastructure performed large amounts of machine work. The defensible conclusion is narrower: the five-case independent analysis batch reached report freeze in 18m58s, while a comparable manual workflow is modeled at multiple analyst-hours.

What the agent did well

It required observable evidence before claiming compromise

The reports did not treat a suspicious domain or HTTP 200 response as automatic code execution. Successful compromise was supported by downstream effects: recurring C2, large uploads, PowerShell stage retrieval, system-information exfiltration, persistence callbacks, RAT beacons, and directory reconnaissance.

It quantified behavior instead of writing vague summaries

Request counts, byte totals, stream durations, response codes, first/last timestamps, and object hashes made the reports reproducible. “The host contacted a bad server” became “264 plaintext POSTs over one 4h20m37s stream on TCP/443.”

It handled shared and legitimate infrastructure carefully

Cloudflare addresses, Node/npm, windows.php.net, and TeamViewer could be misclassified by an automated IOC dump. The reports separated attacker-controlled domains, shared hosting addresses, abused legitimate services, and suspicious-but-unconfirmed infrastructure.

It documented negative evidence

The KongTuke report specifically checked for DsGetNCChanges before discussing DCSync. Across the cases, the agent distinguished authenticated domain access from privilege escalation and stated when lateral movement, exact stolen records, persistence details, or malware-family identity were not proven.

It challenged the answer key

Direct packet validation overruled official typos. That is an important quality signal: comparison was used for validation and error discovery, not blind conformance.

Where the agent failed

Evidence collection was broader than evidence selection

The first pass had already collected identity-protocol and TLS data, but the interpretation missed three SAMR full names and communicationfirewall-security.cc. This was not a parsing limitation. It was an attention and checklist failure.

The corrective action is straightforward: every case should run a mandatory identity field matrix and a sorted unique SNI/DNS review before report freeze.

Caution became premature closure

Refusing to expand rvance into a guessed name was correct. Concluding that the name was not independently provable was not. A stronger phrasing would have been: “The username is proven; the full name remains unresolved pending a SAMR, LDAP, Netlogon, or packet-string sweep.”

Malware naming required refinement

The FormBook alerts supported a network-family label, but the official note preferred XLoader. The final report corrected this to FormBook/XLoader-style traffic and made clear that network behavior did not conclusively resolve branding.

Timing and token instrumentation were incomplete

The experiment measured collector duration and one batch interval but not download time, every reasoning phase, answer comparison, or total token usage. A future benchmark should start a structured event log before the first download and record every phase automatically.

Reproducing the experiment

The experiment produced a full six-PCAP comparative report, an original KongTuke report, and five additional case reports before this article was written.

The following safe, read-only examples show the core approach:

# Capture metadata
capinfos sample.pcap

# Protocol hierarchy and TCP conversations
tshark -n -r sample.pcap -q -z io,phs
tshark -n -r sample.pcap -q -z conv,tcp

# HTTP requests
tshark -n -r sample.pcap -Y 'http.request' -T fields \
  -e frame.number -e frame.time -e ip.src -e ip.dst \
  -e tcp.stream -e http.request.method -e http.host \
  -e http.request.uri -e http.user_agent -e http.content_length

# TLS server names
tshark -n -r sample.pcap -Y 'tls.handshake.type == 1' -T fields \
  -e frame.number -e frame.time -e ip.src -e ip.dst \
  -e tls.handshake.extensions_server_name -e tls.handshake.ja3

# Directory-backed full names
tshark -n -r sample.pcap \
  -Y 'samr.samr_UserInfo21.full_name || netlogon.full_name' \
  -T fields -e frame.number -e ip.src -e ip.dst \
  -e samr.samr_UserInfo21.full_name -e netlogon.full_name

Wireshark and TShark use the same display-filter engine; the official Wireshark filter reference documents the syntax. The examples above show the core fields used by the experiment’s complete evidence collector.

A stronger future benchmark should record:

phase start/end timestamps
model identifier and immutable snapshot
input, cached, reasoning, and output tokens
tool-call duration and output size
independent report checksum
answer-key access timestamp
human analyst experience level and timing
blind scoring by a third evaluator

Limitations

This experiment used one agent run, one host environment, one exercise source, and six captures designed for training. The captures are not representative of every enterprise network, encrypted protocol, packet-loss condition, or adversary.

The quality scores were assigned by the same agent that wrote the reports. They are useful for transparent defect accounting but are not independent evaluation. Likewise, the human comparison is theoretical. No controlled analyst cohort, inter-rater scoring, or statistical analysis was performed.

Official exercise backgrounds influenced high-level family context, because the pages had to be accessed to obtain the downloads. The official answers themselves remained unopened until independent reports were frozen, which protects question-level comparison but does not make the analysis completely blind to exercise titles and alert descriptions.

The configured Daybreak Blue alias and xhigh reasoning effort are known, but the immutable backend snapshot is not. Exact token use, cost, and total end-to-end elapsed time are also unavailable. These missing metrics prevent a scientifically complete cost-performance comparison.

Finally, PCAP evidence has intrinsic limits. TLS and application-layer encryption can prove endpoints, timing, byte transfer, and session behavior without revealing exact commands or stolen content. Network traffic may show an action without identifying the responsible process, persistence mechanism, or complete host impact.

Conclusion

The experiment shows that a tool-using AI agent can perform serious malware traffic analysis when it is required to expose its evidence, preserve uncertainty, and validate conclusions against packet fields. The agent reconstructed six compromises, quantified C2 behavior, extracted and hashed objects, distinguished shared infrastructure from attacker assets, and produced 7,787 words of incident reporting before this article.

Its initial task coverage was 28/31, not perfect. The missing answers were present in the packet data, and one suspicious domain had already been collected but not promoted into the report. That gap matters because it shows why AI-generated analysis still needs checklists, deterministic validation, and human review.

After correction, all 31 requested values were packet-grounded, and the agent identified multiple errors in the official materials. The best result therefore came neither from trusting the model nor trusting the answer key. It came from treating both as hypotheses and returning to the PCAP.

Compared with the theoretical 11–17.5 hours required for a trained human to produce similarly deep work, the measured five-case AI batch was dramatically faster. But speed is not the final lesson. The practical model is AI for exhaustive collection, measurement, correlation, and draft reporting; a human analyst for scope, skepticism, prioritization, and accountability.

References

• Malware-Traffic-Analysis.net — Training Exercises
• 2026-09-11 — KongTuke Rebuke!
• 2026-08-09 — First to Last
• 2026-02-28 — Easy as 123
• 2026-01-31 — Lumma in the Room-ah!
• 2025-06-13 — It’s a trap!
• 2025-01-22 — Download from fake software site
• Wireshark — TShark Manual
• Wireshark — Display Filter Reference
• Wireshark — Capinfos Manual
• MITRE ATT&CK T1204.004 — Malicious Copy and Paste
• OpenAI — Daybreak Blue Model Documentation

Follow My Work

I publish practical cybersecurity research, CTI workflows, detection engineering notes, malware-analysis projects, AI-security research, open-source tools, labs, and technical guides.

• Website — 1200km.com
• Medium — @1200km
• LinkedIn — Andrey Pautov
• GitHub — tools and labs
• Contact — 1200km@gmail.com