Project report / GitHub evidence

Procedural Memory Benchmark Model Card

这是可索引项目报告证据页:它保留 Procedural Memory Benchmark Model Card 的源材料入口、机制线索和限制提醒;正文仍需 reader/editor 与 academic public-copy review 后才能当作最终结论引用。

Procedural Memory Benchmark Model Card

FieldValue
Repositoryqpiai/Proced_mem_bench
CategoryProcedural Memory Retrieval Benchmark
Stars / forks snapshot6 / 3
LanguagePython
LicenseApache-2.0
Raw captureraw-github/qpiai_proced_mem_bench.md
Updated byhourly public metadata update, 2026-05-29 04:05 +0800

1. Role in Self Evolve

Proced_mem_bench benchmarks procedural memory retrieval for language agents across ALFWorld trajectories and upcoming OSWorld scenarios. It matters because self-evolving agents need repeatable harness control, measurable feedback loops, and reusable skill procedures before claiming stable improvement.

2. Working Principle

task trajectory corpus -> procedural retrieval methods -> LLM-as-judge plus IR metrics -> benchmark reports for procedural memory quality

3. Evidence Path

web-observed GitHub page showed 6 stars, 3 forks, 7 commits, Apache-2.0 license, and README framing as a standardized procedural-memory benchmark. Shell GitHub API access remained blocked by DNS and local gh auth was invalid, so this card treats the snapshot as web-observed rather than API-verified.

4. Teaching Use

Use this card to explain Procedural Memory Retrieval Benchmark: it shows how harness/runtime/benchmark layers convert agent behavior into reproducible and auditable engineering workflows.

5. Limits

The repository was not cloned in this iteration; no benchmark run, plugin install, workflow execution, or agent loop experiment was executed. Counts and claims are visible public-page/search signals unless independently revalidated later.