Difference between revisions of "User:Pokechu22/Chunk Format"

From wiki.vg
Jump to navigation Jump to search
(Stub)
 
(Copy original for diff)
Line 1: Line 1:
 
'''v2 of [[SMP Map Format]]''', work in progress.
 
'''v2 of [[SMP Map Format]]''', work in progress.
 +
 +
This article describes in additional detail the format of the [[Protocol#Chunk Data|Chunk Data]] packet.
 +
 +
== Concepts ==
 +
 +
* Chunk Section: a 16×16×16 area, sometimes also called chunk.
 +
* Chunk Column: 16 chunk sections aligned vertically (totaling 16×256×16).
 +
 +
== Packet structure ==
 +
 +
{| class="wikitable"
 +
! Packet ID
 +
! State
 +
! Bound To
 +
! Field Name
 +
! Field Type
 +
! Notes
 +
|-
 +
|rowspan="9"| 0x20
 +
|rowspan="9"| Play
 +
|rowspan="9"| Client
 +
| Chunk X
 +
| Int
 +
| Block coordinate divided by 16, rounded down
 +
|-
 +
| Chunk Z
 +
| Int
 +
| Block coordinate divided by 16, rounded down
 +
|-
 +
| Ground-Up Continuous
 +
| Boolean
 +
| This is true if the packet represents all chunk sections in this vertical chunk column, where the Primary Bit Mask specifies exactly which chunk sections are included, and which are air.
 +
|-
 +
| Primary Bit Mask
 +
| VarInt
 +
| Bitmask with bits set to 1 for every 16×16×16 chunk section whose data is included in Data. The least significant bit represents the chunk section at the bottom of the chunk column (from y=0 to y=15).
 +
|-
 +
| Size
 +
| VarInt
 +
| Size of Data in bytes, plus size of Biomes in bytes if present
 +
|-
 +
| Data
 +
| Array of [[SMP Map Format|Chunk Section]]
 +
| The length of the array is equal to the number of bits set in Primary Bit Mask. Chunks are sent bottom-to-top, i.e. the first chunk, if sent, extends from Y=0 to Y=15.
 +
|-
 +
| Biomes
 +
| Optional Byte Array
 +
| Only if Ground-Up Continuous: biome array, byte per XZ coordinate, 256 bytes total.
 +
|-
 +
| Number of block entities
 +
| VarInt
 +
| Length of the following array
 +
|-
 +
| Block entities
 +
| Array of [[NBT|NBT Tag]]
 +
| All block entities in the chunk.  Use the x, y, and z tags in the NBT to determine their positions.
 +
|}
 +
 +
== Format ==
 +
 +
A Chunk Section is defined in terms of other [[data types]]. A Chunk Section consists of the following fields:
 +
 +
{| class="wikitable"
 +
|-
 +
! Field Name
 +
! Field Type
 +
! Notes
 +
|-
 +
| Bits Per Block
 +
| Unsigned Byte
 +
| Determines how many bits are used to encode a block. Note that not all numbers are valid here. This also changes whether the palette is present.
 +
|-
 +
| Palette Length
 +
| VarInt
 +
| Length of the following array. May be 0, in which case the following palette is not sent.
 +
|-
 +
| Palette
 +
| Optional Array of VarInt
 +
| Mapping of block state IDs in the global palette to indices of this array
 +
|-
 +
| Data Array Length
 +
| VarInt
 +
| Number of longs in the following array
 +
|-
 +
| Data Array
 +
| Array of Long
 +
| Compacted list of 4096 indices pointing to state IDs in the Palette
 +
|-
 +
| Block Light
 +
| Byte Array
 +
| Half byte per block
 +
|-
 +
| Sky Light
 +
| Optional Byte Array
 +
| Only if in the Overworld; half byte per block
 +
|}
 +
 +
In half-byte arrays, two values are packed into each byte. Even-indexed items are packed into the ''low bits'', odd-indexed into the ''high bits''.
 +
 +
Data Array, Block Light, and Sky Light are given for each block with increasing x coordinates, within rows of increasing z coordinates, within layers of increasing y coordinates.
 +
 +
The Data Array, although varying in length, will never be padded due to the number of blocks being evenly divisible by 64, which is the number of bits in a long.
 +
 +
There are several values that can be used for the bits per block value. In most cases, invalid values will be interpreted as a different value when parsed by the Notchian client, meaning that chunk data will be parsed incorrectly if you use an invalid bits per block.  Servers must make sure that the bits per block value is correct.
 +
 +
* up to 4: Blocks are encoded as 4 bits. The palette is used and sent.
 +
* 5 to 8: Blocks are encoded with the given number of bits. The palette is used and sent.
 +
* 9 and above: The palette is not sent. Blocks are encoded by their whole ID in the global palette, with bits per block being set as the base 2 logarithm of the number of block states, rounded up. For the current vanilla release, this is 13 bits per block.
 +
 +
The global palette encodes a block as 13 bits. It uses the {{Minecraft Wiki|Data values#Block IDs|block ID}} for the first 9 bits, and the block damage value for the last 4 bits. For example, Diorite (block ID <code>1</code> for <code>minecraft:stone</code> with damage <code>3</code>) would be encoded as <code>000000001 0011</code>. If a block is not found in the global palette (either due to not having a valid damage value or due to not being a valid ID), it will be treated as air.
 +
 +
If Minecraft Forge is installed and a sufficiently large number of blocks are added, the bits per block value for the global palette will be increased to compensate for the increased ID count.  This increase can go up to 16 bits per block (for a total of 4096 block IDs; when combined with the 16 damage values, there are 65536 total states).  You can get the number of blocks with the "Number of ids" field found in the [[Minecraft Forge Handshake#RegistryData|RegistryData packet in the Forge Handshake]].
 +
 +
The data array stores several entries within a single long, and sometimes overlaps one entry between multiple longs.  For a bits per block value of 13, the data is stored such that bits 1 through 13 are the first entry, 14 through 26 are the second, and so on.  Note that bit 1 is the ''least'' significant bit in this case, not the most significant bit.  The same behavior applies when a value stretches between two longs: for instance, block 5 would be bits 53 through 64 of the first long and then bit 65 of the second long.
 +
 +
=== Example ===
 +
 +
13 bits per block, using the global palette.
 +
 +
The following two longs would represent...
 +
 +
<code>1001880C0060020</code> =
 +
<code><span style="outline: solid 2px hsl(160, 90%, 60%); outline-left-style: dashed">00000001</span><span style="outline: solid 2px hsl(160, 90%, 70%)">0000</span><span style="outline: solid 2px hsl(120, 90%, 60%)">000000011</span><span style="outline: solid 2px hsl(120, 90%, 70%)">0001</span><span style="outline: solid 2px hsl(80, 90%, 60%)">000000011</span><span style="outline: solid 2px hsl(80, 90%, 70%)">0000</span><span style="outline: solid 2px hsl(40, 90%, 60%)">000000011</span><span style="outline: solid 2px hsl(50, 90%, 70%)">0000</span><span style="outline: solid 2px hsl(0, 90%, 65%)">000000010</span><span style="outline: solid 2px hsl(0, 90%, 75%)">0000</span></code><br>
 +
<code>200D0068004C020</code> = <code><span style="outline: solid 2px rgb(60%, 60%, 60%); outline-left-style: dashed">0000001</span><span style="outline: solid 2px rgb(70%, 70%, 70%)">0000</span><span style="outline: solid 2px hsl(320, 90%, 60%)">000001101</span><span style="outline: solid 2px hsl(320, 90%, 70%)">0000</span><span style="outline: solid 2px hsl(280, 90%, 60%)">000001101</span><span style="outline: solid 2px hsl(280, 90%, 75%)">0000</span><span style="outline: solid 2px hsl(240, 90%, 65%)">000000001</span><span style="outline: solid 2px hsl(240, 90%, 75%)">0011</span><span style="outline: solid 2px hsl(200, 90%, 65%)">000000001</span><span style="outline: solid 2px hsl(200, 90%, 70%)">0000</span><span style="outline: solid 2px hsl(160, 90%, 60%); outline-right-style: dashed">0</span></code>
 +
 +
9 blocks, with the start of a 10th (that would be finished in the next long).
 +
 +
#Grass, <span style="border: solid 2px hsl(0, 90%, 65%)">2</span>:<span style="border: solid 2px hsl(0, 90%, 75%)">0</span>
 +
#Dirt, <span style="border: solid 2px hsl(40, 90%, 60%)">3</span>:<span style="border: solid 2px hsl(40, 90%, 70%)">0</span>
 +
#Dirt, <span style="border: solid 2px hsl(80, 90%, 60%)">3</span>:<span style="border: solid 2px hsl(80, 90%, 70%)">0</span>
 +
#Coarse dirt, <span style="border: solid 2px hsl(120, 90%, 60%)">3</span>:<span style="border: solid 2px hsl(120, 90%, 70%)">1</span>
 +
#Stone, <span style="border: solid 2px hsl(160, 90%, 60%)">1</span>:<span style="border: solid 2px hsl(160, 90%, 70%)">0</span>
 +
#Stone, <span style="border: solid 2px hsl(200, 90%, 60%)">1</span>:<span style="border: solid 2px hsl(200, 90%, 70%)">0</span>
 +
#Diorite, <span style="border: solid 2px hsl(240, 90%, 65%)">1</span>:<span style="border: solid 2px hsl(240, 90%, 75%)">3</span>
 +
#Gravel, <span style="border: solid 2px hsl(280, 90%, 65%)">13</span>:<span style="border: solid 2px hsl(280, 90%, 75%)">0</span>
 +
#Gravel, <span style="border: solid 2px hsl(320, 90%, 60%)">13</span>:<span style="border: solid 2px hsl(320, 90%, 70%)">0</span>
 +
#Stone, <span style="border: solid 2px rgb(60%, 60%, 60%)">1</span>:<span style="border: solid 2px rgb(70%, 70%, 70%)">0</span> (or potentially emerald ore, <span style="border: solid 2px rgb(60%, 60%, 60%)">129</span>:<span style="border: solid 2px rgb(70%, 70%, 70%)">0</span>)
 +
 +
== Implementations ==
 +
 +
* [https://github.com/Steveice10/MCProtocolLib/blob/4ed72deb75f2acb0a81d641717b7b8074730f701/src/main/java/org/spacehq/mc/protocol/data/game/chunk/BlockStorage.java#L42 Java, 1.9]
 +
* [https://github.com/SpockBotMC/SpockBot/blob/0535c31/spockbot/plugins/tools/smpmap.py#L144-L183 Python, 1.9]
 +
* [https://github.com/Protryon/Osmium/blob/fdd61b9/MinecraftClone/src/ingame.c#L512-L632 C, 1.9]
 +
* [https://github.com/Protryon/Basin/blob/master/basin/src/packet.c#L1124 C, 1.11.2]
 +
 +
The following implement the [http://wiki.vg/index.php?title=SMP_Map_Format&oldid=7164 previous] (before 1.9) format:
 +
 +
* [https://github.com/GlowstoneMC/Glowstone/blob/d3ed79ea7d284df1d2cd1945bf53d5652962a34f/src/main/java/net/glowstone/GlowChunk.java#L640 Java, 1.8]
 +
* [https://github.com/barneygale/smpmap Python, 1.4]
 +
* [https://github.com/PrismarineJS/prismarine-chunk Node.js, 1.8]

Revision as of 15:40, 19 January 2017

v2 of SMP Map Format, work in progress.

This article describes in additional detail the format of the Chunk Data packet.

Concepts

  • Chunk Section: a 16×16×16 area, sometimes also called chunk.
  • Chunk Column: 16 chunk sections aligned vertically (totaling 16×256×16).

Packet structure

Packet ID State Bound To Field Name Field Type Notes
0x20 Play Client Chunk X Int Block coordinate divided by 16, rounded down
Chunk Z Int Block coordinate divided by 16, rounded down
Ground-Up Continuous Boolean This is true if the packet represents all chunk sections in this vertical chunk column, where the Primary Bit Mask specifies exactly which chunk sections are included, and which are air.
Primary Bit Mask VarInt Bitmask with bits set to 1 for every 16×16×16 chunk section whose data is included in Data. The least significant bit represents the chunk section at the bottom of the chunk column (from y=0 to y=15).
Size VarInt Size of Data in bytes, plus size of Biomes in bytes if present
Data Array of Chunk Section The length of the array is equal to the number of bits set in Primary Bit Mask. Chunks are sent bottom-to-top, i.e. the first chunk, if sent, extends from Y=0 to Y=15.
Biomes Optional Byte Array Only if Ground-Up Continuous: biome array, byte per XZ coordinate, 256 bytes total.
Number of block entities VarInt Length of the following array
Block entities Array of NBT Tag All block entities in the chunk. Use the x, y, and z tags in the NBT to determine their positions.

Format

A Chunk Section is defined in terms of other data types. A Chunk Section consists of the following fields:

Field Name Field Type Notes
Bits Per Block Unsigned Byte Determines how many bits are used to encode a block. Note that not all numbers are valid here. This also changes whether the palette is present.
Palette Length VarInt Length of the following array. May be 0, in which case the following palette is not sent.
Palette Optional Array of VarInt Mapping of block state IDs in the global palette to indices of this array
Data Array Length VarInt Number of longs in the following array
Data Array Array of Long Compacted list of 4096 indices pointing to state IDs in the Palette
Block Light Byte Array Half byte per block
Sky Light Optional Byte Array Only if in the Overworld; half byte per block

In half-byte arrays, two values are packed into each byte. Even-indexed items are packed into the low bits, odd-indexed into the high bits.

Data Array, Block Light, and Sky Light are given for each block with increasing x coordinates, within rows of increasing z coordinates, within layers of increasing y coordinates.

The Data Array, although varying in length, will never be padded due to the number of blocks being evenly divisible by 64, which is the number of bits in a long.

There are several values that can be used for the bits per block value. In most cases, invalid values will be interpreted as a different value when parsed by the Notchian client, meaning that chunk data will be parsed incorrectly if you use an invalid bits per block. Servers must make sure that the bits per block value is correct.

  • up to 4: Blocks are encoded as 4 bits. The palette is used and sent.
  • 5 to 8: Blocks are encoded with the given number of bits. The palette is used and sent.
  • 9 and above: The palette is not sent. Blocks are encoded by their whole ID in the global palette, with bits per block being set as the base 2 logarithm of the number of block states, rounded up. For the current vanilla release, this is 13 bits per block.

The global palette encodes a block as 13 bits. It uses the block ID for the first 9 bits, and the block damage value for the last 4 bits. For example, Diorite (block ID 1 for minecraft:stone with damage 3) would be encoded as 000000001 0011. If a block is not found in the global palette (either due to not having a valid damage value or due to not being a valid ID), it will be treated as air.

If Minecraft Forge is installed and a sufficiently large number of blocks are added, the bits per block value for the global palette will be increased to compensate for the increased ID count. This increase can go up to 16 bits per block (for a total of 4096 block IDs; when combined with the 16 damage values, there are 65536 total states). You can get the number of blocks with the "Number of ids" field found in the RegistryData packet in the Forge Handshake.

The data array stores several entries within a single long, and sometimes overlaps one entry between multiple longs. For a bits per block value of 13, the data is stored such that bits 1 through 13 are the first entry, 14 through 26 are the second, and so on. Note that bit 1 is the least significant bit in this case, not the most significant bit. The same behavior applies when a value stretches between two longs: for instance, block 5 would be bits 53 through 64 of the first long and then bit 65 of the second long.

Example

13 bits per block, using the global palette.

The following two longs would represent...

1001880C0060020 = 0000000100000000000110001000000011000000000001100000000000100000
200D0068004C020 = 0000001000000000110100000000011010000000000001001100000000100000

9 blocks, with the start of a 10th (that would be finished in the next long).

  1. Grass, 2:0
  2. Dirt, 3:0
  3. Dirt, 3:0
  4. Coarse dirt, 3:1
  5. Stone, 1:0
  6. Stone, 1:0
  7. Diorite, 1:3
  8. Gravel, 13:0
  9. Gravel, 13:0
  10. Stone, 1:0 (or potentially emerald ore, 129:0)

Implementations

The following implement the previous (before 1.9) format: