You may have seen a new channel appear on the XYMcity Discord server recently called #rosetta-impl and wondered what it was all about. I picked up on Rosetta discussions a few weeks back before the channel was active and thought I better write an article about it, this got delayed as I had a lot of other things on. I also thought it would be a challenge for me to write as I have no prior knowledge about what it was so the prospect of doing it wasn’t super appealing either 😆.

Seeing @xembook‘s Twitter post and accompanying Japanese article yesterday (featured in the Symbol Blog news update) I thought I should pull my finger out, do some research and write this up.

So here goes, a Symbol Blog article about Rosetta 😬 I am not sure how this will turn out but I will try my best!

DISCLAIMER: I put this article together myself from what I picked up on the Discord channel – if I have taken anything out of context or made mistakes I apologize. Just let me know and I will update the text!

What is Rosetta?

Rosetta was launched by Coinbase in June 2020. They describe it as an “open-source specification and set of tools that make integrating with blockchains simpler, faster, and more reliable”.   Read more…

Rosetta standardizes and structures deployment, communication, and data formats. Instead of having to write custom parsing code for every blockchain, applications are able to directly read on-chain data and construct transactions without having to deal directly with the native blockchain codebase. This makes it way easier for developers and exchanges, simplifies the writing and maintenance of code.

As of December 2020, there were over 25 projects with Rosetta implementations including Bitcoin and Ethereum, but this number is continuing to grow.

Read further…

Why do we want a Rosetta implementation?

Well first off, the easier the team make it for people to interact with the Symbol blockchain the better! If it can be integrated into other platforms without the hassle of having to spend a long time learning all of the technical details and writing bespoke code to interact with the blockchain then it has to be a good thing. If we make it easier for developers and exchanges to work with Symbol then we will see more people using it.

I believe that it might now a requirement for Coinbase to have a Rosetta implementation of a blockchain before they will list it’s tokens on their exchange. I can’t find a direct source for this but it has been mentioned on Discord in the past(?). It seems like it will certainly boost the chances of acceptance on the Coinbase Asset Hub anyway and it seems that maybe the team have had some contact with Coinbase about this already:

We all want a Coinbase listing for XEM and XYM right? Well, it looks like the pirates may be paving the way for this to happen 🥳

Rosetta Roadmap

As far as I can see from reading through the #rosetta-impl channel the plan is to make a Rosetta implementation for NIS1 first and then move on to Symbol. I was a bit surprised by this but apparently, it is much easier to do it for NIS1 than for Symbol. There are complications though as it seems that NIS1 might have to be made open source before integration with Rosetta happens.

I will definitely keep my eyes open and ears to the ground to see how this progresses 😊

Symbol Rosetta Issues

Well, maybe this is a good thing. I think that the plan was to make NIS1 open source anyway and it is well known that this is a requirement for a Coinbase listing. I still don’t really know what the future plans for NIS1 are. If it is planned to move over to be a subChain of Symbol then shouldn’t we be pushing a Symbol implementation? I would love to see XYM listed on Coinbase before (or at the same time as) XEM 😁.

I guess that the thinking is that a NEM Rosetta implementation will be a good learning exercise before moving on to Symbol.

The team has started work already and there is now a GitHub page up. I feel like we are still in the early stages though. From the Discord discussion and this pull request, I think that the team has opted for a TypeScript implementation and there is no current Rosetta SDK available (it seems only a GO SDK is published?) so I guess it means writing it from scratch.

I will definitely keep my eyes open and ears to the ground to see how this progresses 😊

Symbol Rosetta issues

So I just found this excellent technical summary on HackMD written by fboucquez. I really didn’t need to write this article at all did I?! 😆

As I mentioned above (based on reading Discord discussions) Symbol is a more complex beast than NIS1. Apparently, a Symbol node would have to exist in the same Docker file/container as the Rosetta implementation.

Rather than write this in my own, dumbed-down way I am going to rip off a whole chunk of the HackMD article here (apologies to Fernando but thank you for writing this!) and please let me know if you would rather I didn’t steal your words! 😁

Rosetta implementation with a built-in node

If having a separate container/compose running a Symbol node is not an option, we would need to squash a Symbol node next to the Rosetta implementation in the same Docker file/container.

These Rosetta examples do that, running a node and a Rosetta server next to each other.

Rosetta Bitcoin
Rosetta Ethereum

These are some initial options for a built-in node:

Full dual node

Squashing Catapult, Broker, RocksDB, Mongo, Recovery, Rest, and the Rosetta implementation (new) into one large container. Too big, too many points of failure. Rest needs to be extended adding or modifying some endpoints as they are not Rosetta friendly. e.g. Block by hash, or transaction hashes of a block.

API only Node

Squashing Catapult, RocksDB, Recovery, and the Rosetta implementation (new) into one large container. The Rosetta implementation would query RocksDB and talk to Server API directly. We most likely need a RocksDB query/indexer service/component (new). Unsure how hard is to query and index RocksDB.

Api only node + light rest

Squashing Catapult, RocksDB, Recovery, Ligh Rest (new), and the Rosetta implementation (new) into one large container. The Rosetta implementation would talk to a Light Rest service. The Light Rest implementation would query RocksDB and talk to the Server API. A light rest service could be used for regular node deployment, not just Rosetta. We most likely need a RocksDB query/indexer service/component (new).

Catapult extension

A catapult C++ extension that exposes the Rossetta implementation by querying the RocksDB directly.

Custom/Go catapult client

A brand new Catapult client that exposes the Rosetta endpoints. E.g. a GO catapult client that uses the Rosetta GO SDK.

Endpoint Mappings

TODO. In this section, I would like to talk about how the 23 Rossetta endpoints can be mapped to Symbol data and operations.

Future research, questions, TODOs:

• • How symbol specific transactions like metadata, restrictions, etc can be implemented in Rossetta?
  • How Symbol specific entities like restriction, metadata, etc can be used in Rosetta? Can a built-in node vote, harvest and collect harvesting rewards?
  • How to create a single image for both testnet and mainnet nodes. How to setup the node configuration in the image, especially around dynamic/provided keys or customizations.
  • Can a Symbol/Nis1-Rosetta be a first-class citizen?
  • Would regular node admins want to run those kinds of nodes?
  • What if we create explorers and wallets that understand Rosetta for our networks or any other Rosetta compatible subchain? How can Rosetta be extended by adding extra Symbol/Nis1 specific endpoints (like /node/unlockedaccount)?
  • Understand how mempool and subchain works in Rosetta.
  • Storage prunning and how can we delete old blocks.

Conclusion

Well, that’s my attempt at writing an article about Rosetta. It is very NEM/Symbol focussed so if you actually want to go and have a look at what Rosetta can do and the finer details of how it works then I suggest checking out the Rosetta website and technical specifications. It looks as though implementation is a priority for the team and you are going to hear a lot more about this over the coming weeks and months.

I feel like there are a lot of projects on the go at the moment and it is hard to keep up with it all. I love the new transparent way of working and I hope to bring you some more articles in the future piecing together the projects and progress of the Pirate Crew. I just hope that they are not spreading themselves too thinly!