Introducing the JS Bitcoin Benchmark

bitcoinbench

[This is a mirror of a post I made over at the BitGo Company blog]

Recently we started work here at BitGo to implement BIP38. BIP38 is a solid proposal for a standard way to encrypt a bitcoin private key with a passcode. Since we have that functionality, we’d like to use a standard, interoperable format rather than create our own. The key premise behind BIP38 is to use an algorithm which is resistant to large-scale bot-net style attacks. This is accomplished using scrypt, which is intentionally resource intensive. If you’re a user, and just doing a single password try, BIP38 is tolerable, despite taking 2-3 seconds for a single decrypt operation. But if you’re a hacker, trying to brute force crack a password (e.g. guess every possible combination), taking 2 seconds per guess makes it cost-prohibitive.

Since, BitGo is implemented almost entirely in Javascript, we were able to include an open source implementation of BIP38 into our version of bitcoinjs-lib (available on github). But we ran into a snag. While a high-end machine running Chrome can execute a single BIP38 encryption in about 5 seconds, other browsers do not fare as well, with some taking as much as 60 seconds to do a single operation! Our first instinct was that our implementation is flawed. But after some investigation we concluded that while it certainly can be optimized, we weren’t going to get to an acceptable user-waitable performance. And since Chrome was able to do it while other browsers were as much as 12x slower, we strongly believe that the fault here lies mostly in the javascript virtual machine implementations rather than our javascript code. We verified that other sites that do BIP38 are equally slow (e.g. http://bit2factor.org/). And so, this led us to do a fair amount of research into the javascript performance of our product. And soon we had constructed a small benchmark which encompasses many of the crypto algorithms included in bitcoin:

  • bitcoin address creation
  • elliptic curve signing and verification
  • SHA
  • Scrypt
  • address chaining
  • AES

The source code for the benchmark is available on github as part of the bitcoinjs-lib project mentioned above. Or, if you just want to run the benchmark yourself, you can check it out here: https://bitgo.com/bitcoinbenchmark

As for test results, it looks about like this on my windows laptop:

  • Chrome32: 97.5
  • Firefox26: 78.2
  • IE11: 27.1

And on a Mac Mini running Mountain Lion:

  • Chrome32: 131
  • Firefox26: 111
  • Safari6.1.1: 105

I believe this benchmark is differentiated from other benchmarks, the relative difference between browsers using the this benchmark is different than the relative difference of the same browsers using other benchmarks. Using either the sunspider benchmark (http://ie.microsoft.com/testDrive/Benchmarks/SunSpider/Default.html) or Google’s Octane, or Firefox’s Kraken Benchmark, you might think IE is only ~25% slower than other browsers. But as you can see in this benchmark, IE is about 3 times slower on these bitcoin related operations.

I hope others find this benchmark useful. Unlike many benchmarks, which can test contrived cases, these tests really do reflect the core algorithms of bitcoin. If we can make these run super fast in Javascript, we can make great bitcoin software that runs in your browser.

To my friends in the browser teams of Microsoft, Firefox, and Google – the bitcoin community will be very happy if you adopt this benchmark to help move JS forward in 2014!

Please don’t hesitate to send us feedback or jump in and contribute to the effort!