27 Jun Interacting with the Symbol blockchain (by a dummy)

By XHarvesting

So, I thought that I would write a quick post about my first baby steps interacting with the Symbol blockchain outside of the Symbol Wallet.

I run a Symbol node and was curious about the number of accounts and the amount of XYM that were harvesting on each of the nodes on the network. At the time I don’t think that there were any other resources providing this information so I thought I would have a play and see if I could work these numbers out for myself. I have some programming skills (although it’s all in Perl and I am very rusty) and I am pretty proficient on the Linux command line so I took the quick and dirty approach to coding and had a go.

Breaking down the problem

As with all programming exercises the first obstacle was to work out what I needed to do in order to get all of the information that I needed and solve the problem. Then I needed to write some code to achieve this.

Obtaining all mainnet node addresses

First, I needed to find a list of all nodes on the Symbol mainnet. Luckily this information is available from multiple sources. First one I found was on the SymbolNodes site (which is a really useful resource!). Here I could do a simple wget command in my Perl script to download the page and parse the html to extract all of the node addresses.

Getting a list of all accounts harvesting on each node

Next I needed to get a list of all accounts that were harvesting on each of the nodes. This again was easy since you can retrieve this information directly from each of the API nodes:

http://[node address]:3000/node/unlockedaccount/

Here are the accounts harvesting on my node http://xymharvesting.net:3000/node/unlockedaccount/

It was easy enough to query each of the nodes and store these values but as you will notice is that they do not look like Symbol account addresses. They are in fact public keys of the harvesting accounts. We therefore need to convert these public keys into account addresses.

Converting between public key and account addresses

How do we do this? One way is to use the symbol-cli where you can easily convert between the two:

symbol-cli converter publicKeyToAddress -n MAIN_NET -p [publicKey]

So using an example from one of the accounts above we can obtain the account address from the public key returned from the unlockedaccounts:

$ symbol-cli converter publicKeyToAddress -n MAIN_NET -p F87A6DE69C37D4BAF6D08DC7D9C55F2DE08B6059628944464779309342763163

NBLY24-QOGHX5-PNCJFF-5MJOUL-N477SI-SQQM5R-ZPY

So our account address for this public key is NBLY24-QOGHX5-PNCJFF-5MJOUL-N477SI-SQQM5R-ZPY.

Great! So we can: 1) grab all node addresses 2) get all accounts harvesting on each of the nodes 3) convert the public keys of each account to the account addresses. We are nearly there aren’t we? Well no, not quite.. I remembered that delegated harvesting involves the use of a remote account which has a zero balance:

So in the example above we have managed to find a list of remote account addresses that are harvesting on the node. This will show us the number of harvesting accounts on each node but not their balance.

Linking proxy/remote accounts to main harvesting account addresses

In order to link the remote account addresses harvesting on each node to their main accounts holding the XYM balances we can again use the symbol-cli. I am not sure if this is a good way to achieve this task but it seems to work for me.. I figured since the remote account has no balance we should be able to find the single transaction which links it with its main account.

$ symbol-cli transaction search --address  NBLY24-QOGHX5-PNCJFF-5MJOUL-N477SI-SQQM5R-ZPY
┌────────────────────────────────────────────────────────────────────────────────────┐
│                                 AGGREGATE_COMPLETE                                 │
├─────────────────┬──────────────────────────────────────────────────────────────────┤
│ Hash:           │ 8F0BA4490AA88266A658E83D690B3E2E374FEA147A83BEAAA23E70F7D0E3B956 │
├─────────────────┼──────────────────────────────────────────────────────────────────┤
│ Max fee:        │ 1,000,000                                                        │
├─────────────────┼──────────────────────────────────────────────────────────────────┤
│ Height (Block): │ 1,197                                                            │
├─────────────────┼──────────────────────────────────────────────────────────────────┤
│ Index:          │ 5                                                                │
├─────────────────┼──────────────────────────────────────────────────────────────────┤
│ Network type:   │ MAIN_NET                                                         │
├─────────────────┼──────────────────────────────────────────────────────────────────┤
│ Deadline:       │ 2021-03-17 11:31:39.108                                          │
├─────────────────┼──────────────────────────────────────────────────────────────────┤
│ Signer:         │ NANZVL-ZJTVXA-K2772C-AETNCF-XL7FRV-TJNGYA-H4Q                    │
└─────────────────┴──────────────────────────────────────────────────────────────────┘

OK, so now we have the signer of the transaction with the remote account: NANZVL-ZJTVXA-K2772C-AETNCF-XL7FRV-TJNGYA-H4Q – this will be the main account harvesting on the node.

Getting the balance of an account

We now have the address of the main account linked to the remote harvesting account and this account should contain information on balance and importance. Again we can get this information using the symbol-cli:

$ symbol-cli account info -a NANZVL-ZJTVXA-K2772C-AETNCF-XL7FRV-TJNGYA-H4Q
⠼ Processing
Account Information
┌───────────────────┬──────────────────────────────────────────────────────────────────┐
│ Property          │ Value                                                            │
├───────────────────┼──────────────────────────────────────────────────────────────────┤
│ Address           │ NANZVL-ZJTVXA-K2772C-AETNCF-XL7FRV-TJNGYA-H4Q                    │
├───────────────────┼──────────────────────────────────────────────────────────────────┤
│ Address Height    │ 1175                                                             │
├───────────────────┼──────────────────────────────────────────────────────────────────┤
│ Public Key        │ E8BB7C7BC4FD5C0BBA04FCDC7726581347040B3E855DEF416435E1A25825AC57 │
├───────────────────┼──────────────────────────────────────────────────────────────────┤
│ Public Key Height │ 1197                                                             │
├───────────────────┼──────────────────────────────────────────────────────────────────┤
│ Importance        │ 544103437486                                                     │
├───────────────────┼──────────────────────────────────────────────────────────────────┤
│ Importance Height │ 286560                                                           │
└───────────────────┴──────────────────────────────────────────────────────────────────┘ 
Balance Information
┌──────────────────┬─────────────────┬─────────────────┬───────────────────┐
│ Mosaic Id        │ Relative Amount │ Absolute Amount │ Expiration Height │
├──────────────────┼─────────────────┼─────────────────┼───────────────────┤
│ 5AFC20E233ECDAF8 │ 10,000          │ 10000000000     │ Never             │
├──────────────────┼─────────────────┼─────────────────┼───────────────────┤
│ 5329A81F32850CF9 │ 9,000           │ 9000000000      │ Never             │
├──────────────────┼─────────────────┼─────────────────┼───────────────────┤
│ 6BED913FA20223F8 │ 573,777.073     │ 573777072664    │ Never             │
└──────────────────┴─────────────────┴─────────────────┴───────────────────┘ 
Is cosignatory of
┌───────────────────────────────────────────────┐
│ Address                                       │
├───────────────────────────────────────────────┤
│ NAPTWL-PJG3LA-VKUT3Y-6CHCLK-BL6DAR-ZZK6EZ-34A │
└───────────────────────────────────────────────┘

You can see that by querying the main account we can find the account balance of mosaic ID 6BED913FA20223F8 (XYM) as well as the account importance.

This was a lot more longwinded than I had expected but within a couple of hours I had scripted something that repeated this task for every account harvesting on every node listed on SymbolNodes and answered my original questions:

1. How many accounts were harvesting on each of the Symbol mainnet nodes
2. What was the balance of all accounts harvesting on each node

What did I learn?

As you can see using the symbol-cli probably isn’t the best way to do this for a large-scale automated approach! Using a Symbol SDK is going to be more efficient for bulk querying. As I said at the beginning of this article it I took a quick and dirty approach. It took maybe two hours to work out how to get the data and to write the script and then another 7 hours(!) to run it over 1500+ nodes and 9000+ harvesting accounts 😊

Although this is a very simple approach and I am not doing anything complicated here, it was nice to actually delve a little deeper into Symbol and get a feel for what you can do. Even as a complete novice I managed to answer the question that I was interested in as a node owner in a matter of hours. As with all experiments you get a sense of achievement when you solve the problem and for this reason alone I highly recommend delving a little deeper into Symbol and seeing what it (and you) can do.

As a side note, if anyone would like to contribute a how to guide for getting started writing code that utilises Symbol, maybe with some simple exercises I would love to post this on the blog (and work through it myself). I feel that there is a ton of information out there but it is not that accessible to absolute beginners.

Anyway, if you are interested in what I have written about today, I still produce these stats every week on a Friday evening (UTC) and post on Twitter (follow me @XHarvesting). I also have additional data on amount harvested per node and account on my website xymharvesting.com. Getting the data also allowed me to play around with some visualisations illustrating the size of the Symbol network and the relative distribution of funds/importance across different nodes. There’s lots more that I would like to do in terms of data analysis if I can ever find the time. Maybe I will talk about this in a future blog post!

I also experimented with interactive plots (here is an example from my node):