Stratum Mining Block Headers - A Worked Example

Anybody spending any significant amount of time on the internet lately knows that tulip bulbs, er, I mean, cryptocurrencies are a hot topic. As such, it seems like there are a lot of projects out there looking to implement related technologies.

Stratum mining is one of those technologies. It is currently the in-vogue replacement for the older 'get-work' protocol, and everyone seems to want to write their own implementation. Unfortunately, the official documentation is severely lacking in any kind of worked example, a weakness I hope to rectify here.

This is not intended be a complete document on the ins and outs of the communication aspect of Stratum, that is something I feel like the official documentation does well. This is merely intended to show a worked example of generating a block header based on information provided to a Stratum client by a Stratum server. Think of  it as the 'back of the book' answer for a homework problem. Also note, this is for a Scrypt based coin, though the process should be the same for double SHA-256 based currencies.

If it helps you, you can send me a Bitcoin tip so maybe I can afford the fancy ramen this week: 18ymUR1XyZ68NCzjrzYtMRP2ztk8LjAgBL




Assume we see the following from our server after we subscribe:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

{ "result": [ [ "mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f" ], "60021014", 4 ], "id": 1, "error": null } { "params": [ 512 ], "id": null, "method": "mining.set_difficulty" } { "params": [ "5c04", "da0dadb0eda4381df442bde08d23d54d7d371d5ce7af3ee716bd2a7e017eacb8", "01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff2a03700a08062f503253482f04953f1a5308", "102f776166666c65706f6f6c2e636f6d2f0000000001d07e582a010000001976a9145d8f33b0a7c94c878d572c40cbff22a49268467d88ac00000000", [ "50a4a386ab344d40d29a833b6e40ea27dab6e5a79a2f8648d3bc0d1aa65ecd3f", "7952ecc836fb104f41b2cb06608eeeaa6d1ca2fe4391708fb13bb10ccf8da179", "9400ec6453aac577fb6807f11219b4243a3e50ca6d1c727e6d05663211960c94", "c11a630fa9332ab51d886a47509b5cbace844316f4fc52b493359b305fd489ae", "85891e7c5773f234d647f1d5fca7fbcabb59b261322d16c0ae486ccf5143383d", "faa26bbc17f99659f64136bea29b3fc8d772b339c52707d5f2ccfe1195317f43" ], "00000002", "1b10b60e", "531a3f95", false ], "id": null, "method": "mining.notify" }

That is a gross wall of text. Let's pick out the important bits.

From the first message:

Extra Nonce 1: 0x60021014
Extra Nonce 2 Size: 4

From the set difficulty message:

Difficulty: 512

From the job message:

Job ID: 0x5c04
Previous Block Hash: 0xda0dadb0eda4381df442bde08d23d54d7d371d5ce7af3ee716bd2a7e017eacb8
Coin Base 1: 0x01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff2a03700a08062f503253482f04953f1a5308
Coin Base 2:  0x102f776166666c65706f6f6c2e636f6d2f0000000001d07e582a010000001976a9145d8f33b0a7c94c878d572c40cbff22a49268467d88ac00000000
Merkle Branch:  [0x50a4a386ab344d40d29a833b6e40ea27dab6e5a79a2f8648d3bc0d1aa65ecd3f, 0x7952ecc836fb104f41b2cb06608eeeaa6d1ca2fe4391708fb13bb10ccf8da179, 0x9400ec6453aac577fb6807f11219b4243a3e50ca6d1c727e6d05663211960c94, 0xc11a630fa9332ab51d886a47509b5cbace844316f4fc52b493359b305fd489ae, 0x85891e7c5773f234d647f1d5fca7fbcabb59b261322d16c0ae486ccf5143383d, 0xfaa26bbc17f99659f64136bea29b3fc8d772b339c52707d5f2ccfe1195317f43]
Version:  0x00000002
Bits:  0x1b10b60e
Time:  0x531a3f95

Now, in your code, you have to do something with all this. But first, an aside. I highly recommend you keep all values given to you in hex as hex values until you absolutely need them as something else. If you convert them to ints, you have to remember the required length of all of them when building the block header (if you convert them to octal, you should see a therapist). Additionally, it keeps diagnostic output as not-confusing as possible. At the end of the day, it's your choice, but you have been warned.

At this point, we are ready to calculate extra nonce 2. We are given the required length of extra nonce 2 in bytes (4), technically, extra nonce 2 is allowed to be any valid 32 bit number, but if you are like every other Stratum implementation on the planet, you'll use 0x00000000.

We can now calculate our coinbase. This is covered in the Stratum documentation. Simply calculate the double SHA-256 of the concatenation  of Coin Base 1 + Extra Nonce 1 + Extra Nonce 2 + Coin Base 2. The result should be the following:

Coin Base: 0x0ae66bdd8cf4ff954e4b0cbd91ba0a8d1038712e33a1a15baae46f413449371f

Next we need the Merkle root. Since we have branches, we use the method of calculation described in the documentation. If we were given an empty array for the Merkle root calculation, we would use the coin base as the Merkle root. For this example, the answer is:

Merkle Root: 0x43c4345eb9ad9135836f5c31b697f62429c1be08d55906ff407852adfba680a5

At this point, we can calculate the block header. Let's calculate the header of a solution for this share, the necessary things for the solution block header are:

Solution Nonce: 0x6acb01a0
Solution Time: 0x531a3f95
Padding: 0x000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000

Now, you can put together the block header as shown in the documentation, concatenate the following (the Merkle Root has its endianness preserved, ALL OTHER COMPONENTS MUST BE CONVERTED TO LITTLE ENDIAN):

Version + Previous Hash + Merkle Root + Solution Time + Bits + Solution Nonce + Padding

Block Header: 0x02000000b0ad0dda1d38a4ede0bd42f44dd5238d5c1d377de73eafe77e2abd16b8ac7e0143c4345eb9ad9135836f5c31b697f62429c1be08d55906ff407852adfba680a5953f1a530eb6101ba001cb6a800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000280

Hashing this header (remember, Scrypt based) gives:

0x0000005e2e75336501c003c45e7605128642ef78405eb7322862119a8e3c250a

Which we can see is less than the target:

0x0000007fff800000000000000000000000000000000000000000000000000000

If you calculated the target yourself from the difficulty, you may have noticed it doesn't match the target used above (you probably got something like 0x00000000007fff80000000000000000000000000000000000000000000000000). That's because in this example we're mining Litecoin. As noted in the Litecoin wiki: "Even if officially difficulty is defined the same way as for Bitcoin, for historical reasons the value is sometimes multiplied by 65536 (216). This is in part due to the fact that early versions of cgminer did not support non-integer difficulties, and 2-16 was the lowest share difficulty used by Litecoin pools." That's why the value shown here is larger.

Some implementations of the hashing algorithm may implement the padding for you, in which case you would use the message sans padding:

0x02000000b0ad0dda1d38a4ede0bd42f44dd5238d5c1d377de73eafe77e2abd16b8ac7e0143c4345eb9ad9135836f5c31b697f62429c1be08d55906ff407852adfba680a5953f1a530eb6101ba001cb6a

Extra credit: WTF is with the padding?

The official  documentation just gives us the padding with the promise "the rest is padding to uint512 and it is always the same". Depending on the implementation of the hashing algorithm you're using, you may find you don't even need the padding! So seriously, why the padding?

The answer comes from our friends at the NSA. You see, when they aren't laughing at the inappropriate selfies you are sending your mistress, they are developing cryptography standards. SHA-256 is one of them. The answer for the padding comes from the NIST publication (caution PDF link) describing SHA-256. On page 18, under preprocessing:

"Pad the message in the usual way: Suppose the length of the message M, in bits, is l. Append the bit '1' to the end of the message, and then k zero bits, where k is the smallest non-negative solution to the equation l+1+k (is congruent with) 896 mod 1024. To this append the 128-bit block which is equal  to the number l written in binary..."

This isn't as gnarly as it probably sounds if you are unfamiliar with cryptography. Besides, this is "the usual way", and surely you  have other messages you need to pad in "the usual way", right? Right...? Bueller...?

Basically, SHA-256 operates on 512 bit blocks. Our block header, without padding, is 160 hex characters, which gives an l of 640 bits. We need to get to 1024 bits so we have two, even 512 bit blocks. 1 + 128 bits are already spoken for by the spec and our header is another 640 bits, so we need 1024 - (1 + 128 + 640) = 255 bits of 0s (or more simply, 255 0s) between our  1, and the 128 bit block which is equal to the number l written in binary. Our l is 640, or 0x280. We need to expand that to 128 bits, so prepend 29 0s to that hex number.

So, in binary, we have a 1, followed by our 255 0s, 118 more 0s from the leading 0s of l, then '101', and then 7 more 0s. Or in hex:

0x800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000280

While this is the padding used in out block header, this doesn't match the padding we were given, but convert it to little endian:

0x000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000

Which should look familiar.  And just to check, it is 96 hex characters long, which is 384 bits, which, when added to the 640 bits we already had, gives us a 1024  bit message, which can be evenly split into two 512 bit blocks for SHA-256.

Of course, it gets converted to little endian again when assembling the block header. So we wind up going full circle.

Extra, extra credit: Does a 1, followed by 255 0s, followed by 118 more 0s, '101', and 7 more 0s, really give the above?

For funsies, in Python:

>>> int('1' + '0'*255 + '0'*118 + '101' + '0'*7, 2)

Gives you the integer value, which you can compare against the non-little-endian padding calculated above, if you wish.

>>> int('1' + '0'*255 + '0'*118 + '101' + '0'*7, 2) == int('800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000280', 16)

True