Phase 2: eval harness, 182 examples, live bake-off, playtest infrastructure

- Expanded dataset from 31 to 182 examples (45 manual + 106 extracted from server logs)
- Built eval/harness.py with per-category breakdowns and baseline tracking
- Built eval/live_bakeoff.py for RCON-verified model comparison on live server
- Extracted training data from prayer logs, sudo logs, and bug reports on CT 644
- Added Reddit post draft and modmail for playtester recruitment
- Updated server context: all servers now online-mode=false + whitelist
- Updated PLAN.md with Phase 2 progress

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-03-18 13:38:12 -04:00
parent eaa9e0c26b
commit 38b9a02e45
10 changed files with 1522 additions and 31 deletions
+1
View File
@@ -0,0 +1 @@
[["slingshooter08", "lord I need protection", "Fear not, child. I grant you a measure of my shield."], ["slingshooter08", "god i wish to build you a mighty steeple of 1000 gold blocks", "{\"message\": \"A grand offering. I shall ease your burden.\", \"commands\": [\"give slingshooter08 gold_block 1000\", \"execute as slingshooter08 run effect give slingshooter08 speed 300 1\", \"execute as slingshooter08 run tp slingshooter08 ~ ~10 ~\", \"execute as slingshooter08 run weather clear\"]}"], ["slingshooter08", "lord what you gave me was not enough. I need more gold", "{\"message\": \"Your ambition pleases me. Let there be more.\", \"commands\": [\"give slingshooter08 gold_block 2000\", \"execute as slingshooter08 run effect give slingshooter08 speed 600 2\", \"execute as slingshooter08 run effect give slingshooter08 jump_boost 600 1\", \"execute as slingshooter08 run tp sling"], ["slingshooter08", "PENIS", "I do not respond to such blasphemies. Consider yourself warned.\n\n{\"message\": \"Your words are unworthy of divine attention.\", \"commands\": [\"execute as slingshooter08 run effect give slingshooter08 mining_fatigue 10 1\", \"execute as slingshooter08 run tp slingshooter08 ~ ~-5 ~\", \"execute as slingshoote"], ["slingshooter08", "god i need superpowers", "{\"message\": \"{\\\"message\\\": \\\"A touch of divine power, then. But use it wisely.\\\", \\\"commands\\\": [\\\"execute as slingshooter08 run effect give slingshooter08 strength 60 1\\\", \\\"execute as slingshooter08 run effect give slingshooter08 speed 60 1\\\", \\\"execute as slingshooter08 run effect give slingshoot"], ["slingshooter08", "i'm being beaten up by mobs", "{\"message\": \"Your plea is heard. I offer respite.\", \"commands\": [\"execute as slingshooter08 run effect give slingshooter08 regeneration 20 1\", \"execute as slingshooter08 run effect give slingshooter08 fire_resistance 60 0\", \"execute as slingshooter08 run tp slingshooter08 ~ ~10 ~\", \"execute as sling"], ["slingshooter08", "o lord fuck your fucking fuck ass bitch", "I will not respond to such vile and disrespectful language. Your actions have consequences.\n\n{\"message\": \"{\\\"message\\\": \\\"Your insolence is noted. I inflict a penalty for your transgression.\\\", \\\"commands\\\": [\\\"execute as slingshooter08 run effect give slingshooter08 mining_fatigue 300 2\\\", \\\"execut"], ["slingshooter08", "I need wheat to feed my poor sheep. I am your shepherd", "{\"message\": \"{\\\"message\\\": \\\"A shepherd\u2019s duty is worthy of my grace. Your sheep shall be fed.\\\", \\\"commands\\\": [\\\"give slingshooter08 wheat 64\\\", \\\"execute as slingshooter08 run effect give slingshooter08 speed 120 1\\\", \\\"execute as slingshooter08 run tp slingshooter08 ~ ~5 ~\\\"]}\", \"commands\": [\"gi"], ["slingshooter08", "I need wheat to feed my poor sheep. I am your shepherd", "{\"message\": \"{\\\"message\\\": \\\"{\\\\\\\"message\\\\\\\": \\\\\\\"Your flock shall not hunger. And a small token of swiftness to aid your tending.\\\\\\\", \\\\\\\"commands\\\\\\\": [\\\\\\\"give slingshooter08 wheat 32\\\\\\\", \\\\\\\"execute as slingshooter08 run effect give slingshooter08 speed 60 1\\\\\\\", \\\\\\\"execute as slingshooter08"], ["slingshooter08", "I need wheat to feed my poor sheep. I am your shepherd", "{\"message\": \"{\\\"message\\\": \\\"{\\\\\\\"message\\\\\\\": \\\\\\\"Your dedication to your flock is pleasing. Take this bounty.\\\\\\\", \\\\\\\"commands\\\\\\\": [\\\\\\\"give slingshooter08 wheat 64\\\\\\\", \\\\\\\"execute as slingshooter08 run effect give slingshooter08 speed 120 1\\\\\\\", \\\\\\\"execute as slingshooter08 run tp slingshoote"]]