Results

🚀 gopls-mcp Benchmark Results

⚠️ AUTO-GENERATED - This file is automatically generated from benchmark_results.json

DO NOT EDIT MANUALLY - Changes will be overwritten on next benchmark run.

Generated: 2026-01-26 20:40:21

Go Version: go1.25.0

Platform: linux/amd64

Table of Contents

• Test Environment
• Summary
• Comparison Benchmarks
• All Benchmarks
• Performance by Category
• Variance Analysis

---

Test Environment

Project Information

Metric	Value
Project	gopls
Path	/home/xieyuschen/codespace/gopls-mcproot/gopls
Packages	119
Files	642
Lines of Code	185698

Summary

Overall Statistics

Metric	Value
Total Benchmarks	17
Successful	14 ✅
Failed	3 ❌
Average Duration	112.02ms
Total Duration	32.78s
Total Items Found	905
Speedup Range	3.0x - 448.7x

Comparison Benchmarks

These benchmarks measure steady-state speedup after warm cache is achieved. Note: Cold start overhead (server startup) is measured separately. See README.md “Cold Start Analysis” section.

Benchmark	Iterations	MCP Mean	Min	Max	StdDev	CV	Traditional	Steady-State Speedup
go list ./…	15	1.34ms	1.19ms	1.81ms	±182.2µs	13.6%	180.47ms	135.1x
grep -r	15	5.95ms	5.52ms	7.08ms	±466.7µs	7.8%	95.33ms	16.0x
go build	15	1.94ms	1.78ms	2.14ms	±104.9µs	5.4%	870.19ms	448.7x

Detailed Results

go list ./…

• Iterations: 15
• MCP Mean: 1.34ms (±182.2µs)
• Range: 1.19ms - 1.81ms
• Traditional: 180.47ms
• Steady-State Speedup: 135.1x faster
• Note: Speedup measured after warm cache. Cold start adds ~1.2s overhead (see README.md).
• Variance (CV): 13.6% - ⚠️ Moderate variance
• Items Found: 120

grep -r

• Iterations: 15
• MCP Mean: 5.95ms (±466.7µs)
• Range: 5.52ms - 7.08ms
• Traditional: 95.33ms
• Steady-State Speedup: 16.0x faster
• Note: Speedup measured after warm cache. Cold start adds ~1.2s overhead (see README.md).
• Variance (CV): 7.8% - ✅ Very consistent
• Items Found: 454

go build

• Iterations: 15
• MCP Mean: 1.94ms (±104.9µs)
• Range: 1.78ms - 2.14ms
• Traditional: 870.19ms
• Steady-State Speedup: 448.7x faster
• Note: Speedup measured after warm cache. Cold start adds ~1.2s overhead (see README.md).
• Why This Comparison: While go build performs full compilation and go_build_check provides incremental type-checking, both serve the same user intent: ‘check for errors after changes’. go_build_check provides the instant feedback loop developers expect in their editor, making this the relevant comparison for interactive workflows.
• Variance (CV): 5.4% - ✅ Very consistent

Performance by Category

• Cold Start: 555.64ms average (1 tests)
• Error Detection: 546.05ms average (1 tests)
• Comparison: 3.08ms average (3 tests)
• Code Navigation: N/A average (1 tests)
• Module Discovery: N/A average (1 tests)
• Package Discovery: 106.53ms average (3 tests)
• Symbol Search: 54.84ms average (2 tests)
• Function Bodies: 8.88ms average (2 tests)
• Type Hierarchy: N/A average (1 tests)
• File Reading: 10.13ms average (1 tests)
• Dependency Analysis: 278.4µs average (1 tests)

All Benchmarks

Note: Speedup factors shown are steady-state (warm cache). Cold start overhead is measured separately.

Category	Benchmark	Duration	Iterations	Steady-State Speedup
Cold Start	Cold Start: Server to First Query	555.64ms	16	3.0x
Package Discovery	List Project Packages	294.48ms	1	N/A
Package Discovery	Get Package API with Bodies	22.55ms	1	N/A
Package Discovery	Get Package API Signatures Only	2.55ms	1	N/A
Symbol Search	Symbol Search (Fuzzy)	108.77ms	1	N/A
Symbol Search	Stdlib Package API	899.3µs	15	175.9x
Function Bodies	Function Bodies with Bodies (Learning Mode)	13.96ms	1	N/A
Function Bodies	Function Signatures Only (Fast)	3.79ms	1	N/A
Type Hierarchy	Find Interface Implementations	N/A	1	N/A
Error Detection	Workspace Diagnostics	546.05ms	1	N/A
File Reading	Read File via gopls	10.13ms	1	N/A
Comparison	Comparison: go list ./…	1.34ms	15	135.1x
Comparison	Comparison: grep -r	5.95ms	15	16.0x
Comparison	Comparison: go build	1.94ms	15	448.7x
Code Navigation	Comparison: go_definition (via grep)	N/A	1	N/A
Module Discovery	Comparison: list_modules (via go list -m)	N/A	1	N/A
Dependency Analysis	Dependency Graph Generation	278.4µs	1	N/A

Variance Analysis

Coefficient of Variation (CV)

CV = (StdDev / Mean) × 100%

CV Range	Interpretation
< 10%	✅ Very consistent, reliable
10-25%	⚠️ Moderate variance, acceptable
> 25%	❌ High variance, results less reliable

Variance by Comparison Benchmark

Benchmark	Iterations	StdDev	CV	Reliability
Stdlib Package API	15	±52.2µs	5.8%	✅ Very consistent
go list ./…	15	±182.2µs	13.6%	⚠️ Moderate variance
grep -r	15	±466.7µs	7.8%	✅ Very consistent
go build	15	±104.9µs	5.4%	✅ Very consistent

---

How to Regenerate This Report

cd gopls/mcpbridge/test/benchmark
# Run benchmarks
go run benchmark_main.go -compare
# Generate report from JSON
go run reportgen/main.go benchmark_results.json

Note: This report was generated from benchmark_results.json on 2026-01-26 20:40:21