Parsing Deadlock Demo Files

Recently, I’ve been writing a tool to parse Deadlock demo files in my free time.¹ One of the biggest blockers that has slowed development is that no-one has written a complete - or even really up-to-date guide on parsing Valve’s esoteric demo file format for Deadlock. As I’m working on this tool on and off, I decided I might as well document my progress here and hopefully build a more up-to-date guide, to help me (and maybe others) in the future.

The Demo File

Deadlock games are recorded to a .dem file, which represents all of the data necessary to playback a complete replay of the game at a later time. The file is structured as a series of protobuf messages which can be decoded using the .proto files found here. Before we get into how we do that, let’s look at a high-level overview of how the file is structured:

• 8-byte header - this is some Valve metadata for demo files that can be safely ignored
• 4-byte little-endian integer - represents a byte offset for the CDemoFileInfo protobuf message (which contains summary info on the match)
• a series of messages which contain the following elements:
  • Metadata - contains info about the upcoming protobuf message:
    • command - varint indicating message type
    • tick - varint indicating replay time
    • size - varint indicating byte length of the message
  • Message - binary blob of data representing the protobuf message to be deserialized. (note: can be compressed)

Now, using the info above, if we parse the header and the metadata of the first message for any Deadlock demo file we should see that the command for the first message is 1. If that’s the case we can continue to dive deeper into the parsing.

Handling Compression

Except there’s one further thing we need to discuss: compression. Deadlock demo files use Google’s Snappy to compress data for certain long messages. Often, we’ll run into a message where the command is seemingly way too high of a number. We can OR the number with 01110000 to indicate whether the message is compressed or not. If we determine it’s compressed we need to use a relevant Snappy decompression library to decompress the message’s binary blob.

CDemoPacket && CDemoSignonPacket

These two messages only contain one field: data.

message CDemoPacket {
    optional bytes data = 3;
}

So what is contained in the data? Well data is a binary stream of data which contains messages of the following format:

• Message Type - a ubit which represents a protobuf message.
• Size - a varint indicating the byte length of the message.

But hang on a second, what’s a ubit? A ubit is essentially a variable length unsigned integer found only in Source 2 demo files. To read a ubit, we read the first 6 bits, then use the top two of those 6 bits to determine how many additional bits to read, as shown in the below table.

Top 2 bits	Additional bits to read
00	0
01	4
10	8
11	28

Then, we can simply read the size and decode the message using the extracted message type.