Difference between revisions of "Data Generators"

From wiki.vg
Jump to navigation Jump to search
(Create draft article)
 
m (→‎Registries report: add missing comma in JSON)
 
(10 intermediate revisions by 4 users not shown)
Line 7: Line 7:
 
<pre>
 
<pre>
 
java -cp minecraft_server.jar net.minecraft.data.Main
 
java -cp minecraft_server.jar net.minecraft.data.Main
 +
</pre>
 +
 +
Starting in 1.18, this command is used instead due to the server jar bundling libraries instead of merging them into a single file:
 +
 +
<pre>
 +
java -DbundlerMainClass=net.minecraft.data.Main -jar minecraft_server.jar
 
</pre>
 
</pre>
  
Line 66: Line 72:
 
  | {{no}} || {{no}} || {{yes}} || {{no}}
 
  | {{no}} || {{no}} || {{yes}} || {{no}}
 
  |}
 
  |}
 +
 +
=== Packets report ===
 +
A report that contains all packets used by both the server and client. This file stores both a resource location and numeric id for each packet used in each state of the protocol. The presence of both a resource location and the numeric id makes it easier to identify the new packet IDs sent by both the server and client as resource locations are more human readable and are less likely to change between versions.
 +
The report is saved in <code>reports/packets.json</code>
 +
 +
At the root of the packets report are a key for each state of the protocol, currently these are "handshake", "status", "login", "configuration" and "play". Each state key maps to an object with two keys, "clientbound" and "serverbound" storing the clientbound and serverbound packets of each state respectively, these two keys map to an object acting as a map of all packets indexed by a unique resource location. Similar to other reports the packets are encoded as an object with the singular key "protocol_id" which is the numeric ID used over the protocol.
 +
 +
It should be noted that the "handshake" object is the only state object without a "clientbound" key as this state doesn't have any packets that are sent to the client, this may be important to note when attempting to parse the report.
 +
 +
Example of the top of this report:
 +
 +
<syntaxhighlight lang="json>
 +
{
 +
  "configuration": {
 +
    "clientbound": {
 +
      "minecraft:cookie_request": {
 +
        "protocol_id": 0
 +
      },
 +
      "minecraft:custom_payload": {
 +
        "protocol_id": 1
 +
      },
 +
      "minecraft:custom_report_details": {
 +
        "protocol_id": 15
 +
      },
 +
      "minecraft:disconnect": {
 +
        "protocol_id": 2
 +
      },
 +
...
 +
</syntaxhighlight>
  
 
=== Blocks report ===
 
=== Blocks report ===
  
A report that contains all block states in the global palette.  Notably, this contains numeric IDs as used on the protocol, so it can be used to get the proper IDs for a version.  The report is saved in <code>reports/blocks.json</code>.
+
A report that contains all block states in the block state registry.  Notably, this contains numeric IDs as used on the protocol, so it can be used to get the proper IDs for a version.  The report is saved in <code>reports/blocks.json</code>.
  
 
A few sample entries:
 
A few sample entries:
Line 113: Line 148:
 
This report does not contain numeric block IDs, only state IDs.  Block IDs only remain in use on the Block Action packet; they can be inferred by the position in the report.
 
This report does not contain numeric block IDs, only state IDs.  Block IDs only remain in use on the Block Action packet; they can be inferred by the position in the report.
  
{{warning|Due to [https://bugs.mojang.com/browse/MC-131740 MC-131740], the blocks report will delete its results if run twice in the same location.}}
+
{{warning|Due to [https://bugs.mojang.com/browse/MC-131740 MC-131740], the blocks report will delete its results if run twice in the same location. (This was fixed in 18w50a)}}
  
 
=== Items report ===
 
=== Items report ===
 +
 +
{{Warning|Replaced in 18w50a with the registries report}}
  
 
The items report contains a list of all items, and their network IDs.
 
The items report contains a list of all items, and their network IDs.
Line 133: Line 170:
  
 
{{warning|Due to [https://bugs.mojang.com/browse/MC-131740 MC-131740], the items report will delete its results if run twice in the same location.}}
 
{{warning|Due to [https://bugs.mojang.com/browse/MC-131740 MC-131740], the items report will delete its results if run twice in the same location.}}
 +
 +
=== Registries report ===
 +
 +
{{Warning|Added in 18w50a}}
 +
 +
Contains IDs from all registries, including block and item IDs.
 +
 +
<syntaxhighlight lang="json">
 +
{
 +
  "minecraft:sound_event": {
 +
    "protocol_id": 0,
 +
    "entries": {
 +
      "minecraft:ambient.cave": {
 +
        "protocol_id": 0
 +
      },
 +
      "minecraft:ambient.underwater.enter": {
 +
        "protocol_id": 1
 +
      }
 +
    }
 +
  },
 +
  "minecraft:fluid": {
 +
    "default": "minecraft:empty",
 +
    "protocol_id": 1,
 +
    "entries": {
 +
      "minecraft:empty": {
 +
        "protocol_id": 0
 +
      },
 +
      "minecraft:flowing_water": {
 +
        "protocol_id": 1
 +
      }
 +
    }
 +
  },
 +
  "minecraft:mob_effect": {
 +
    "protocol_id": 2,
 +
    "entries": {
 +
      "minecraft:speed": {
 +
        "protocol_id": 1
 +
      },
 +
      "minecraft:slowness": {
 +
        "protocol_id": 2
 +
      }
 +
    }
 +
  }
 +
}
 +
</syntaxhighlight>
 +
 +
The key within the root object is the registry's own ID, and the <code>protocol_id</code> there is the ID for the registry over the network (is this used?).  <code>default</code> indicates a default value to be used when one is needed or an unknown value is used.  <code>entries</code> is the actual content of the registry; keys in it are the name of the element and <code>protocol_id</code> is the actual relevant ID.
 +
 +
The different registries (ordered by their protocol ID):
 +
 +
{| class="wikitable"
 +
! Name
 +
! Meaning
 +
|-
 +
| <code>minecraft:sound_event</code>
 +
| Sounds and their numeric IDs, for the sound effect packet.
 +
|-
 +
| <code>minecraft:fluid</code>
 +
| Fluids.
 +
|-
 +
| <code>minecraft:mob_effect</code>
 +
| Actual potion effects.  Includes effects such as nausea, absorption, and saturation that don't have actual potions associated with them.
 +
|-
 +
| <code>minecraft:block</code>
 +
| Blocks.  Note that the block ID is only used in a few places and is separate from the block state ID.
 +
|-
 +
| <code>minecraft:enchantment</code>
 +
| Enchantments.
 +
|-
 +
| <code>minecraft:entity_type</code>
 +
| Entities.
 +
|-
 +
| <code>minecraft:item</code>
 +
| Items.
 +
|-
 +
| <code>minecraft:potion</code>
 +
| IDs for potion items.  Note that these have "long" and "strong" variants, as would be seen on brewing.
 +
|-
 +
| <code>minecraft:carver</code>
 +
|
 +
|-
 +
| <code>minecraft:surface_builder</code>
 +
|
 +
|-
 +
| <code>minecraft:feature</code>
 +
|
 +
|-
 +
| <code>minecraft:decorator</code>
 +
|
 +
|-
 +
| <code>minecraft:biome</code>
 +
| Biome IDs.
 +
|-
 +
| <code>minecraft:block_state_provider_type</code>
 +
|
 +
|-
 +
| <code>minecraft:block_placer_type</code>
 +
|
 +
|-
 +
| <code>minecraft:foliage_placer_type</code>
 +
|
 +
|-
 +
| <code>minecraft:trunk_placer_type</code>
 +
|
 +
|-
 +
| <code>minecraft:tree_decorator_type</code>
 +
|
 +
|-
 +
| <code>minecraft:particle_type</code>
 +
| Particle effect IDs.
 +
|-
 +
| <code>minecraft:biome_source_type</code>
 +
|
 +
|-
 +
| <code>minecraft:block_entity_type</code>
 +
| Block/tile entity IDs.
 +
|-
 +
| <code>minecraft:chunk_generator_type</code>
 +
|
 +
|-
 +
| <code>minecraft:dimension_type</code>
 +
| Overworld/Nether/End.
 +
|-
 +
| <code>minecraft:motive</code>
 +
| Painting images.
 +
|-
 +
| <code>minecraft:custom_stat</code>
 +
| Stats that are not associated directly with using a block or item.
 +
|-
 +
| <code>minecraft:chunk_status</code>
 +
| States a chunk can be in during generation.
 +
|-
 +
| <code>minecraft:structure_feature</code>
 +
| Various generated structures.
 +
|-
 +
| <code>minecraft:structure_piece</code>
 +
| IDs for components of structures themselves; see {{Minecraft Wiki|Generated structures data file format}} (possibly outdated).  These IDs are extremely terse; as an example "nesclt" means something like "''ne''ther fortress ''s''mall ''c''orridor ''l''eft ''t''urn.
 +
|-
 +
| <code>minecraft:rule_test</code>
 +
|
 +
|-
 +
| <code>minecraft:pos_rule_test</code>
 +
|
 +
|-
 +
| <code>minecraft:structure_processor</code>
 +
|
 +
|-
 +
| <code>minecraft:structure_pool_element</code>
 +
|
 +
|-
 +
| <code>minecraft:menu</code>
 +
| Different [[inventory]] windows.
 +
|-
 +
| <code>minecraft:recipe_type</code>
 +
| Not used for anything over the network or in recipe JSON files.  Internally indicates the device with which a recipe can be used.  For instance, <code>minecraft:crafting</code>.
 +
|-
 +
| <code>minecraft:recipe_serializer</code>
 +
| Indicates the format of a recipe when serialized to JSON or over the network; also controls the actual behavior.  For instance, <code>minecraft:crafting_shaped</code> and <code>minecraft:crafting_shapeless</code>.
 +
|-
 +
| <code>minecraft:attribute</code>
 +
| Attributes for mobs and players.
 +
|-
 +
| <code>minecraft:stat_type</code>
 +
| Statistics category.
 +
|-
 +
| <code>minecraft:villager_type</code>
 +
| Villager types, i.e. the type/biome of the village they came from.
 +
|-
 +
| <code>minecraft:villager_profession</code>
 +
| Professions used by villagers.
 +
|-
 +
| <code>minecraft:point_of_interest_type</code>
 +
|
 +
|-
 +
| <code>minecraft:memory_module_type</code>
 +
|
 +
|-
 +
| <code>minecraft:sensor_type</code>
 +
|
 +
|-
 +
| <code>minecraft:schedule</code>
 +
|
 +
|-
 +
| <code>minecraft:activity</code>
 +
|
 +
|}
  
 
=== Commands report ===
 
=== Commands report ===
Line 138: Line 361:
 
This report contains the [[Command Data|Command Graph]].  The format matches information described in that article, except that <code>name</code> is the key for an object, and suggestions are not present in the report.
 
This report contains the [[Command Data|Command Graph]].  The format matches information described in that article, except that <code>name</code> is the key for an object, and suggestions are not present in the report.
  
{{warning|Due to [https://bugs.mojang.com/browse/MC-131740 MC-131740], the commands report will delete its results if run twice in the same location.}}
+
{{warning|Due to [https://bugs.mojang.com/browse/MC-131740 MC-131740], the commands report will delete its results if run twice in the same location. (This was fixed in 18w50a)}}
  
 
=== NBT converters ===
 
=== NBT converters ===

Latest revision as of 12:42, 5 October 2024

Data Generators are a tool added in 1.13 that can export reports about various parts of the game, and additionally are used internally by Mojang to generate various json files.

Command line

The main entry point for data generators is net.minecraft.data.Main instead of net.minecraft.client.main.Main or net.minecraft.server.MinecraftServer. This is present on both the client and the server, but since the server includes all needed libraries to run it while the client has them in separate jars, it is much easier to use the server. Since this isn't the main class, the game must be ran using a different command line than normal:

java -cp minecraft_server.jar net.minecraft.data.Main

Starting in 1.18, this command is used instead due to the server jar bundling libraries instead of merging them into a single file:

java -DbundlerMainClass=net.minecraft.data.Main -jar minecraft_server.jar

There are several arguments:

--help
Show the help menu (same as running with no arguments)
--input
Specify an input folder for the converters. Can be specified multiple times, as in --input folder1 --input folder2.
--output
Output folder (defaults to generated)
--client
--dev
--reports
--server
Specifies which generators to run; see the table below.
--all
Same as --client --dev --reports --server.

Generators

There are several different generators, which are controlled via the arguments above.

Name --client --dev --reports --server
SNBT -> NBT Yes No No Yes
Fluid Tags No No No Yes
Block Tags No No No Yes
Item Tags No No No Yes
Recipes No No No Yes
Advancements No No No Yes
NBT to SNBT No Yes No No
Block List No No Yes No
Item List No No Yes No
Command List No No Yes No

Packets report

A report that contains all packets used by both the server and client. This file stores both a resource location and numeric id for each packet used in each state of the protocol. The presence of both a resource location and the numeric id makes it easier to identify the new packet IDs sent by both the server and client as resource locations are more human readable and are less likely to change between versions. The report is saved in reports/packets.json

At the root of the packets report are a key for each state of the protocol, currently these are "handshake", "status", "login", "configuration" and "play". Each state key maps to an object with two keys, "clientbound" and "serverbound" storing the clientbound and serverbound packets of each state respectively, these two keys map to an object acting as a map of all packets indexed by a unique resource location. Similar to other reports the packets are encoded as an object with the singular key "protocol_id" which is the numeric ID used over the protocol.

It should be noted that the "handshake" object is the only state object without a "clientbound" key as this state doesn't have any packets that are sent to the client, this may be important to note when attempting to parse the report.

Example of the top of this report:

{
  "configuration": {
    "clientbound": {
      "minecraft:cookie_request": {
        "protocol_id": 0
      },
      "minecraft:custom_payload": {
        "protocol_id": 1
      },
      "minecraft:custom_report_details": {
        "protocol_id": 15
      },
      "minecraft:disconnect": {
        "protocol_id": 2
      },
...

Blocks report

A report that contains all block states in the block state registry. Notably, this contains numeric IDs as used on the protocol, so it can be used to get the proper IDs for a version. The report is saved in reports/blocks.json.

A few sample entries:

{
  "minecraft:air": {
    "states": [
      {
        "id": 0,
        "default": true
      }
    ]
  },
  "minecraft:oak_sapling": {
    "properties": {
      "stage": [
        "0",
        "1"
      ]
    },
    "states": [
      {
        "properties": {
          "stage": "0"
        },
        "id": 21,
        "default": true
      },
      {
        "properties": {
          "stage": "1"
        },
        "id": 22
      }
    ]
  }
}

The key of the object is the block ID. Within it, the properties tag contains all properties within the state, where the key within that is the property name and the value is an array of allowed property values. This tag will not be present if there are no properties. The states tag is a list of different states for the block. Within each state, the properties tag maps each property to the value on that state (which, again, will not be present if there are no properties); the id tag gives the numeric state ID (as used over the network); and the default tag indicates if the state is the default one for the block (there should be only one such state).

This report does not contain numeric block IDs, only state IDs. Block IDs only remain in use on the Block Action packet; they can be inferred by the position in the report.

Warning.png Due to MC-131740, the blocks report will delete its results if run twice in the same location. (This was fixed in 18w50a)

Items report

Warning.png Replaced in 18w50a with the registries report

The items report contains a list of all items, and their network IDs.

{
  "minecraft:air": {
    "protocol_id": 0
  },
  "minecraft:stone": {
    "protocol_id": 1
  }
}

Note that item IDs are no longer related to block IDs or block state IDs. They all start out in the same sequence, but some blocks do not have items and items do not include block states (by design).

Warning.png Due to MC-131740, the items report will delete its results if run twice in the same location.

Registries report

Warning.png Added in 18w50a

Contains IDs from all registries, including block and item IDs.

{
  "minecraft:sound_event": {
    "protocol_id": 0,
    "entries": {
      "minecraft:ambient.cave": {
        "protocol_id": 0
      },
      "minecraft:ambient.underwater.enter": {
        "protocol_id": 1
      }
    }
  },
  "minecraft:fluid": {
    "default": "minecraft:empty",
    "protocol_id": 1,
    "entries": {
      "minecraft:empty": {
        "protocol_id": 0
      },
      "minecraft:flowing_water": {
        "protocol_id": 1
      }
    }
  },
  "minecraft:mob_effect": {
    "protocol_id": 2,
    "entries": {
      "minecraft:speed": {
        "protocol_id": 1
      },
      "minecraft:slowness": {
        "protocol_id": 2
      }
    }
  }
}

The key within the root object is the registry's own ID, and the protocol_id there is the ID for the registry over the network (is this used?). default indicates a default value to be used when one is needed or an unknown value is used. entries is the actual content of the registry; keys in it are the name of the element and protocol_id is the actual relevant ID.

The different registries (ordered by their protocol ID):

Name Meaning
minecraft:sound_event Sounds and their numeric IDs, for the sound effect packet.
minecraft:fluid Fluids.
minecraft:mob_effect Actual potion effects. Includes effects such as nausea, absorption, and saturation that don't have actual potions associated with them.
minecraft:block Blocks. Note that the block ID is only used in a few places and is separate from the block state ID.
minecraft:enchantment Enchantments.
minecraft:entity_type Entities.
minecraft:item Items.
minecraft:potion IDs for potion items. Note that these have "long" and "strong" variants, as would be seen on brewing.
minecraft:carver
minecraft:surface_builder
minecraft:feature
minecraft:decorator
minecraft:biome Biome IDs.
minecraft:block_state_provider_type
minecraft:block_placer_type
minecraft:foliage_placer_type
minecraft:trunk_placer_type
minecraft:tree_decorator_type
minecraft:particle_type Particle effect IDs.
minecraft:biome_source_type
minecraft:block_entity_type Block/tile entity IDs.
minecraft:chunk_generator_type
minecraft:dimension_type Overworld/Nether/End.
minecraft:motive Painting images.
minecraft:custom_stat Stats that are not associated directly with using a block or item.
minecraft:chunk_status States a chunk can be in during generation.
minecraft:structure_feature Various generated structures.
minecraft:structure_piece IDs for components of structures themselves; see Generated structures data file format (possibly outdated). These IDs are extremely terse; as an example "nesclt" means something like "nether fortress small corridor left turn.
minecraft:rule_test
minecraft:pos_rule_test
minecraft:structure_processor
minecraft:structure_pool_element
minecraft:menu Different inventory windows.
minecraft:recipe_type Not used for anything over the network or in recipe JSON files. Internally indicates the device with which a recipe can be used. For instance, minecraft:crafting.
minecraft:recipe_serializer Indicates the format of a recipe when serialized to JSON or over the network; also controls the actual behavior. For instance, minecraft:crafting_shaped and minecraft:crafting_shapeless.
minecraft:attribute Attributes for mobs and players.
minecraft:stat_type Statistics category.
minecraft:villager_type Villager types, i.e. the type/biome of the village they came from.
minecraft:villager_profession Professions used by villagers.
minecraft:point_of_interest_type
minecraft:memory_module_type
minecraft:sensor_type
minecraft:schedule
minecraft:activity

Commands report

This report contains the Command Graph. The format matches information described in that article, except that name is the key for an object, and suggestions are not present in the report.

Warning.png Due to MC-131740, the commands report will delete its results if run twice in the same location. (This was fixed in 18w50a)

NBT converters

There are two NBT-related data generators. One converts "SNBT" (stringified NBT - the same NBT format used in commands) to NBT; the other converts NBT into SNBT. Both look for files of the appropriate extension (.nbt or .snbt) in the input folder(s), and output them at the same relative location in the output folder.

Warning.png Due to MC-131740, the NBT to SNBT converter will delete its results if run twice in the same location.

Tag providers

The tag providers are used to create tag JSON files based off of data stored in code. As such, they aren't directly useful to people who are not writing mods. They can be used by subclassing, overriding the appropriate methods, and changing the data within.

Advancement provider

The advancement provider is used to generate advancement JSON files based off of data stored in code. It works off of nested consumers, where one layer of consumers contains the actual data (split between the different advancement categories), and that consumer takes a consumer that is actually responsible for saving the advancement. It can be used by subclassing only.

Recipes provider

The recipe JSON file creates recipes based on data in code. It can't be used directly.