CHANGELOG: reth, It's Fast Now!
It's amazing how being employed completely sucks the life out of a personal website. All the energy you previously expended on the website, both productive and social, is now spent at work.
Last June I joined the Paradigm team as a developer working on reth, the rust-based ethereum execution client.
Since then Paradigm has turned into Tempo, which is building a blockchain of its own using reth as an SDK, but this did not change my day-to-day work significantly; I'm still mainly focused on reth itself, and specifically on making it go fast.
One of the perks of this job is that everything I do is open-source; you can just look at my Github page and see what me and the rest of the team is working on. Going in I thought I would take advantage of this and continue using this website the way I always have: to chronicle the things I'm working on and what's on my mind, just now with my work instead of personal/open-source projects. What's happened in reality is that I just don't have the energy for that, and the reth project moves so fast that it would be a real chore to keep up with it in blog form anyway. Today I happened to have the time and energy for a quick update, so here we are.
Trie Guy
My primary directive is: make reth fast. There are multiple ethereum client implementations out there, and they are all competing on many different metrics. The metric we have been focused on for the last 6 months or so is payload validation time, ie the time it takes for a client in the network to receive a block and say "yes, this block is good". For ethereum this involves a couple of steps:
- Decode the block, and all of its many many fields
- Do some pre-validation on those fields.
- Execute the transactions in the block. This produces a set of state updates, ie the keys/values which were changed by the transactions.
- Validate the block's new state root hash.
That final step, the state root hash, has been my primary focus since I joined. Ethereum's entire state, all account information and the key/value pairs which make up each account's storage slots, is encoded into a single giant trie structure, which is merkle-ized and reduced down to a single hash deterministically. If a single bit of a single storage slot is changed, then the state root hash changes. By including the state root hash in each block, every ethereum client can verify that when they've executed a block's transactions the resulting state, across all accounts in the network, is the same as all other clients.
The state root is interesting because there's many different ways to approach the problem of recomputing it and validating it, given just a delta of key/values, without having to actually traverse the entire keyspace of all accounts for every block (which would be impossible to do in the 12 seconds available between blocks). One day I'll write a more detailed post with the actual details of all this, because it is super interesting, but today is not that day.
Suffice it to say that I've been focused on the state root for the last 8 months or so. Of all the niche things I could have become a domain expert in, the ethereum Merkle-Patricia Trie is on the higher end of niche-ness. Even so, I'm very proud of the work I've done so far; perhaps the most proud of any work I've ever done! It's been challenging, it's pushed me to the limits of my ability, but it's also been extremely satisfying to see the results.
reth: Is It Fast?
When I started at Paradigm reth was probably the second or third fastest client, measuring by new payload validation speed. As of a few weeks ago, reth is unequivocally the fastest.
Timing measurements by EthPandaOps, taken across standardized reference hardware, per-block per-client, and counting which client wins on each block.
What that chart is saying is that for 84% of blocks produced in the last week, reth validated them faster than any other client.
The entire team pulled extremely hard to make this happen, especially in the last couple of months, across all areas of the client. For my part, I rewrote a core piece of the state root calculation machinery from scratch. The new version unlocked a new way forward in terms of in-memory caching of some key data which previously wasn't worthwhile to cache, and with some further rewiring we were able to drop the disk overhead of state root calculations by something like 50%. The result of this was a 10% to 20% speedup, though possibly more as we're seeing that the cache takes a few hours to properly warm up, and we never measured the difference with a fully warmed cache.
And we're still not done! I'm working on a rewrite of another piece of the state root machinery right now which should unlock some further gains. And from there we're already talking about how we can improve (rewrite? implement from scratch??) the database portion itself.
Working on performance is a bit like speedrunning: no matter how fast you get, there's always a little more to go, and there's always someone else right on your tail keeping you from letting off the gas.
So that's the update. reth is fast, it's getting faster, and working on it is extremely engaging. Perhaps too engaging. My side-projects aren't dead, Micropelago still gets some love on evenings and weekends, but between work and having bought a house (we bought a house!) there hasn't been much spare gas in the tank for joy riding. One day we'll be back though.
This site is a mirror of my gemini capsule. The equivalent gemini page can be found here, and you can learn more about gemini at my 🚀 What is Gemini? page.