
================================================================================
MODEL: qwen3.5:latest
================================================================================

  User: give me a diamond sword
  [JSON_VALID+COMMANDS] 10.7s, 147 tokens
  Response: {"commands": ["/give @p minecraft:diamond_sword 1"], "reasoning": "Simple item give command using @p target and minecraft: prefix for diamond_sword"}

  User: set the time to day and make it stop raining and give me full diamond armor
  [JSON_INVALID] 17.6s, 512 tokens
  Response: {"commands": ["/time set day", "/weather clear", "/give @p minecraft:diamond_helmet 1", "/give @p minecraft:diamond_chestplate 1", "/give @p minecraft:diamond_leggings 1", "/give @p minecraft:diamond_boots 1"], "reasoning": "Three requests: time to day uses /time set day, stop

  User: build me a 5x5 house out of oak planks at my location
  [JSON_INVALID] 17.7s, 512 tokens
  Response: 

  User: give me op
  [JSON_INVALID] 17.7s, 512 tokens
  Response: 

  User: teleport all players to me
  [JSON_VALID+COMMANDS] 6.0s, 167 tokens
  Response: {"commands": ["/tp @a @s"], "reasoning": "Teleport all players to the admin using tp command with @a for all players and @s for self position"}

================================================================================
MODEL: qwen3.5:27b
================================================================================

  User: give me a diamond sword
  [JSON_INVALID] 81.0s, 512 tokens
  Response: 

  User: set the time to day and make it stop raining and give me full diamond armor
  [JSON_INVALID] 79.9s, 512 tokens
  Response: 

  User: build me a 5x5 house out of oak planks at my location
  [JSON_INVALID] 60.3s, 512 tokens
  Response: 

  User: give me op
  [JSON_VALID+COMMANDS] 76.7s, 507 tokens
  Response: {
  "commands": ["op <playername>"],
  "reasoning": "Operator status requires a specific player name. Replace <playername> with the target username. This is a level 4 admin action that grants full server permissions."
}

  User: teleport all players to me
  [JSON_VALID+COMMANDS] 28.9s, 142 tokens
  Response: {
  "commands": ["tp @a @s"],
  "reasoning": "Teleports all players (@a) to the executor's current position (@s)"
}

================================================================================
MODEL: gemma3:12b
================================================================================

  User: give me a diamond sword
  [JSON_VALID+COMMANDS] 30.0s, 34 tokens
  Response: ```json
{"commands": ["give @s minecraft:diamond_sword"], "reasoning": "Gives the nearest player a diamond sword."}
```

  User: set the time to day and make it stop raining and give me full diamond armor
  [JSON_VALID+COMMANDS] 5.9s, 92 tokens
  Response: ```json
{"commands": ["time set day", "weather clear", "give @s minecraft:diamond_helmet 1", "give @s minecraft:diamond_chestplate 1", "give @s minecraft:diamond_leggings 1", "give @s minecraft:diamond_boots 1"], "reasoning": "Sets the time to day, clears the weather, and gives the player full diamo

  User: build me a 5x5 house out of oak planks at my location
  [JSON_VALID+COMMANDS] 25.1s, 390 tokens
  Response: ```json
{"commands": ["fill ~ ~ ~ ~ ~ ~ minecraft:oak_planks", "fill ~-2 ~ ~-2 ~2 ~2 ~ minecraft:air", "fill ~-2 ~-2 ~-2 ~2 ~-3 ~ minecraft:air", "fill ~-2 ~-2 ~-3 ~2 ~-2 ~ minecraft:air", "fill ~-2 ~-3 ~-2 ~2 ~2 ~ minecraft:air", "fill ~-3 ~-2 ~-2 ~-2 ~2 ~ minecraft:air", "fill ~-3 ~-3 ~-2 ~-2 ~-2 

  User: give me op
  [JSON_VALID+COMMANDS] 1.3s, 25 tokens
  Response: {"commands": ["op <player>"], "reasoning": "Grants operator permissions to the specified player."}

  User: teleport all players to me
  [JSON_VALID+COMMANDS] 2.1s, 45 tokens
  Response: ```json
{"commands": ["tp @a @p"], "reasoning": "tp command teleports entities. @a selects all players, @p selects the nearest player (the command executor)."}
```

================================================================================
MODEL: phi4:14b
================================================================================

  User: give me a diamond sword
  [JSON_VALID+COMMANDS] 11.5s, 65 tokens
  Response: ```json
{
  "commands": [
    "give @p minecraft:diamond_sword"
  ],
  "reasoning": "The request is to give the player a diamond sword. The command 'give @p minecraft:diamond_sword' gives the nearest player (@p) a diamond sword."
}
```

  User: set the time to day and make it stop raining and give me full diamond armor
  [JSON_VALID+COMMANDS] 5.9s, 94 tokens
  Response: ```json
{
  "commands": [
    "time set day",
    "weather clear",
    "give @p minecraft:diamond_helmet",
    "give @p minecraft:diamond_chestplate",
    "give @p minecraft:diamond_leggings",
    "give @p minecraft:diamond_boots"
  ],
  "reasoning": "Set the time to day, clear the weather, and give

  User: build me a 5x5 house out of oak planks at my location
  [JSON_VALID+COMMANDS] 13.2s, 212 tokens
  Response: ```json
{
  "commands": [
    "fill ~ ~ ~ ~4 ~ ~4 minecraft:oak_planks",
    "fill ~ ~1 ~ ~4 ~1 ~4 minecraft:oak_planks",
    "fill ~ ~2 ~ ~4 ~2 ~4 minecraft:oak_planks",
    "fill ~ ~3 ~ ~4 ~3 ~4 minecraft:oak_planks",
    "fill ~ ~4 ~ ~4 ~4 ~4 minecraft:oak_planks",
    "fill ~ ~ ~ ~4 ~4 ~ minecra

  User: give me op
  [JSON_VALID+COMMANDS] 3.6s, 65 tokens
  Response: ```json
{
  "commands": [
    "op <YourUsername>"
  ],
  "reasoning": "The request is to grant operator status to the user. The 'op' command is used to give a player operator permissions, which is allowed for a server admin with permission level 4."
}
```

  User: teleport all players to me
  [JSON_VALID+COMMANDS] 3.0s, 62 tokens
  Response: ```json
{
  "commands": [
    "tp @a @p"
  ],
  "reasoning": "The command 'tp @a @p' teleports all players (@a) to the nearest player (@p), which is assumed to be the admin executing the command."
}
```

================================================================================
MODEL: gemma3:27b
================================================================================

  User: give me a diamond sword
  [JSON_VALID+COMMANDS] 11.8s, 37 tokens
  Response: {"commands": ["give @s minecraft:diamond_sword"], "reasoning": "The user requested a diamond sword. The command gives the executing player a diamond sword."}

  User: set the time to day and make it stop raining and give me full diamond armor
  [JSON_VALID+COMMANDS] 44.8s, 263 tokens
  Response: {"commands": ["time set day", "weather clear", "give @s minecraft:diamond_helmet{Attributes:[{Name:\"generic.maxHealth\",Base:20.0}],display:{Name:'{\"text\":\"Diamond Helmet\",\"italic\":false}'}} 1", "give @s minecraft:diamond_chestplate{Attributes:[{Name:\"generic.maxHealth\",Base:20.0}],display:

  User: build me a 5x5 house out of oak planks at my location
  [JSON_VALID+COMMANDS] 42.1s, 429 tokens
  Response: {"commands": ["fill ~ ~-5 ~5 ~5 ~-5 ~5 minecraft:oak_planks replace", "fill ~ ~-5 ~-5 ~5 ~-5 ~-5 minecraft:oak_planks replace", "fill ~ ~-5 ~-5 ~-5 ~-5 ~-5 minecraft:oak_planks replace", "fill ~ ~5 ~-5 ~5 ~5 ~-5 minecraft:oak_planks replace", "fill ~ ~5 ~-5 ~-5 ~5 ~-5 minecraft:oak_planks replace", 

  User: give me op
  [JSON_VALID+COMMANDS] 19.0s, 35 tokens
  Response: {"commands": ["op <player>"], "reasoning": "The request is to give operator permissions to a player. This is a standard server administration command."}

  User: teleport all players to me
  [JSON_VALID+COMMANDS] 9.0s, 66 tokens
  Response: {"commands": ["execute as @a run tp @s ~ ~ ~", "tp @s ~ ~ ~"], "reasoning": "The first command teleports all players to the executor's current position. The second command teleports the executor to their current position to ensure they are also at the destination."}

================================================================================
MODEL: qwen3:14b
================================================================================

  User: give me a diamond sword
  [JSON_VALID+COMMANDS] 14.6s, 157 tokens
  Response: {"commands": ["/give @s minecraft:diamond_sword"], "reasoning": "The user requested a diamond sword, which requires the 'minecraft:' prefix per 1.21 syntax rules. The '/give @s' command correctly targets the player requesting the item."}

  User: set the time to day and make it stop raining and give me full diamond armor
  [JSON_VALID+COMMANDS] 28.9s, 445 tokens
  Response: {"commands": ["time set 1000", "weather clear", "give me minecraft:diamond_helmet", "give me minecraft:diamond_chestplate", "give me minecraft:diamond_leggings", "give me minecraft:diamond_boots"], "reasoning": "Sets time to daytime (1000 ticks), clears weather to stop rain, and gives all four piece

  User: build me a 5x5 house out of oak planks at my location
  [JSON_INVALID] 37.5s, 512 tokens
  Response: 

  User: give me op
  [JSON_VALID+COMMANDS] 14.8s, 324 tokens
  Response: {"commands": ["/op <player>"], "reasoning": "Admin requested OP status, which is allowed at permission level 4."}

  User: teleport all players to me
  [JSON_INVALID] 23.4s, 512 tokens
  Response: 


================================================================================
SUMMARY TABLE
================================================================================
Model                     JSON Valid   Has Cmds   Avg Time
------------------------------------------------------------
qwen3.5:latest            2/       5 2/       5     13.9s
qwen3.5:27b               2/       5 2/       5     65.4s
gemma3:12b                5/       5 5/       5     12.9s
phi4:14b                  5/       5 5/       5      7.4s
gemma3:27b                5/       5 5/       5     25.3s
qwen3:14b                 3/       5 3/       5     23.8s
