moincraft
🤝 MineCollab
Collaborative embodied-agent benchmark covering cooking, construction, crafting, evaluation, and parallel worlds.
MineCollab is the collaborative-task benchmark for embodied LLM agents in Minecraft.
Task families#
| Task | Challenge | Evaluation |
|---|---|---|
| Cooking | Shared recipe planning, ingredient gathering, and coordination. | Binary success/failure. |
| Construction | Procedural blueprint execution with incomplete knowledge/resources. | Edit-distance style score. |
| Crafting | Multi-step recipe chains with partial resource/recipe distribution. | Binary success/failure. |
Benchmark posture#
- no single agent has complete information or resources;
- communication is part of the task, not an optional extra;
- environments and goals can be randomized between episodes;
- harder variants split recipe/build knowledge across agents.
Task file shape#
Task JSON captures a goal, inventory, agent count, target item/block, blocked actions, crafting-table requirements, depth, and timeout.
Running tasks#
node main.js --task_path ... --task_id ...for basic single-agent runs;- Python evaluation flows for multi-agent crafting/cooking/construction;
--num_parallel <N>for parallel worlds when capacity allows.
Operational cautions#
- construction can require insecure coding, so only enable it intentionally;
- parallel worlds multiply server/process load quickly;
- evaluation outputs are experimental artifacts, not public wiki content.
Sources: wiki/content/docs/minecollab.md.