16 Dec HOW THE TRANSFER IS MADE

Background

In the previous article, we looked at adding a simple transaction to Catapult. That will provide some additional context to this post, so you should read it if you haven’t already! 🤓

In that article you might have seen some terms like Notification, Observer, Validator (oh my!) and wondered what that mean. Now we’re going to dive into each of them in more detail and learn how Catapult actually transfers tokens from one account to another.

Architecture Overview

In this post, we will be focusing on the starred parts of the diagram. 💪

┌─────────────────┐          ┌─────────────────┐
│                 │          │                 │    1 *   ┌──────────────┐
│ Pipeline        ◄──────────┤  Block          ◄──────────┤ Consumer     │
│                 │          │  Disruptor      │          └──────────────┘
│                 ◄──┐     ┌►│                 │
└─────────▲───────┘  │     │ └─────────────────┘
          │          │     │ ┌─────────────────┐
          │1         └─────┼─┤                 │    1 *   ┌──────────────┐
          │*               │ │  Transaction    ◄──────────┤ Consumer     │
┌---------┴----------┐     │ │  Disruptor      │          └──────────────┘
│ Transaction Plugin │     │ │                 ◄─┐
│  ****************  │     │ └─────────────────┘ │
│  * Observers    *  │     │                     │
│  ****************  │     │  ┌───────────────┐  │ 
│  ****************  │     └──┤ Packet        ├──┘
│  * Validators   *  │        │ Handlers      │
│  ****************  │        └───────▲───────┘
│  ┌--------------┐  │                │
│  │ Transactions │  │       ┌────────┴────────┐
│  │ Definitions  │  │       │ NETWORK TRAFFIC │
│  └--------------┘  │       └─────────────────┘
└--------------------┘

Notifications

Notifications are the smallest unit that can be processed by the Core Processing Pipeline.

Every notification shares the same header, which is composed of a type and size.

/// Basic notification.
struct PLUGIN_API_DEPENDENCY Notification {
public:
    /// Creates a new notification with \a type and \a size.
    Notification(NotificationType type, size_t size)
            : Type(type)
            , Size(size)
    {}

public:
    /// Notification type.
    NotificationType Type;

    /// Notification size.
    size_t Size;
};

There are common notification types as well. Importantly, a notification type is itself composed of a channel, facility and a code.

channel Each notification can be raised for validators and/or observers. Be patient, we’ll get to those soon. Due to the way rollback works, any notifications raised on the observer channel must be fully stack allocated. Notifications raised only on the validator channel can use heap allocation, but it is preferable to avoid this.

facility Each major component and plugin in Catapult is assigned a facility code. This allows notifications to easily be grouped by source.

code Locally unique identifier relative to facility.

Together this construction looks like:

/// Makes a notification type given \a channel, \a facility and \a code.
constexpr NotificationType MakeNotificationType(NotificationChannel channel, FacilityCode facility, uint16_t code) {
    return static_cast<NotificationType>(
            static_cast<uint32_t>(channel) << 24 //    01..08: channel
            | static_cast<uint32_t>(facility) << 16 // 09..16: facility
            | code); //                                16..32: code
}

You might think this looks familiar to how transaction types are constructed. If you do, you’re right! 🎊 If you don’t, study more 🤓

BalanceTransferNotification

A balance transfer notification is published when a token needs to be transfered from one account to another. In this article, we’ll use a slightly simplified version of the notifications. Specifically, we’ll remove the base class that it shares with BalanceDebitNotification and drop the dynamic fee handling (used for calculating mosaic and namespace rental fees). With those simplifications in mind, the notification looks like:

/// Notifies a balance transfer from sender to recipient.
struct BalanceTransferNotification : public Notification {
public:
    /// Matching notification type.
    static constexpr auto Notification_Type = Core_Balance_Transfer_Notification;

public:
    /// Creates a notification around \a sender, \a recipient, \a mosaicId and \a amount.
    BalanceTransferNotification(
            const ResolvableAddress& sender,
            const ResolvableAddress& recipient,
            UnresolvedMosaicId mosaicId,
            catapult::Amount amount)
            : Notification(Notification_Type, sizeof(BalanceTransferNotification))
            , Sender(sender)
            , MosaicId(mosaicId)
            , Amount(amount)
            , Recipient(recipient)
    {}

public:
    /// Sender.
    ResolvableAddress Sender;

    /// Mosaic id.
    UnresolvedMosaicId MosaicId;

    /// Amount.
    catapult::Amount Amount;

    /// Recipient.
    ResolvableAddress Recipient;
};

The fields should be straightforward. The notification can be thought of an instruction to transfer Amount units of MosaicId mosaic from Sender to Recipient.

Notification_Type is important because it will dictate what channels the notification gets raised on.

Ok, we’re done with notifications. That was the easy part. 😅

Core_Balance_Transfer_Notification

The notification type Core_Balance_Transfer_Notification is defined by the following declaration:

/// Mosaic was transferred between two accounts.
DEFINE_CORE_NOTIFICATION(Balance_Transfer, 0x0003, All);

which expands into:

/// Defines a core notification type with \a DESCRIPTION, \a CODE and \a CHANNEL.
#define DEFINE_CORE_NOTIFICATION(DESCRIPTION, CODE, CHANNEL) DEFINE_NOTIFICATION_TYPE(CHANNEL, Core, DESCRIPTION, CODE)

which continuing down the rabbit hole leads to:

/// Defines a notification type given \a CHANNEL, \a FACILITY, \a DESCRIPTION and \a CODE.
#define DEFINE_NOTIFICATION_TYPE(CHANNEL, FACILITY, DESCRIPTION, CODE) \
    constexpr auto FACILITY##_##DESCRIPTION##_Notification = model::MakeNotificationType( \
            (model::NotificationChannel::CHANNEL), \
            (model::FacilityCode::FACILITY), \
            CODE)

and that function is finally calling the MakeNotificationType function referenced above!

If we manually run the preprocessor we get:

constexpr auto Core_Balance_Transfer_Notification = model::MakeNotificationType(
        model::NotificationChannel::All,
        model::FacilityCode::Core,
        0x0003);

We can lookup NotificationChannel::All (0xFF) and FacilityCode::Core (0x43) and see that the value is calculated in MakeNotificationType as follows:

return static_cast<NotificationType>(
        0xFF << 24 //   01..08: channel
        | 0x43 << 16 // 09..16: facility
        | 0x0003); //.   16..32: code

which is simply 0xFF 43 0003 (channel, facility, code)!

Alice, time to leave Wonderland! 🐰 Aren’t macros fun?!? 😈

Validators

Catapult has two types of validators: stateless and stateful validators.

Stateless validators inspect things independent of state (like signatures) and are run in parallel by the StatelessValidationConsumer. The validation function only accepts a notification and looks roughly like this:

/// Validates a single \a notification.
template<typename TNotification>
virtual ValidationResult validate(const TNotification& notification) const = 0;

Stateful validators, as you might guess, are dependent on the global blockchain state and will pass or fail depending on its current values. For example, when A has 100 XYM transferring 90 XYM is valid but when A only has 80 XYM it is not. Consequently, these need to be run sequentially in the BlockChainSyncConsumer. Since the stateful validators are dependent on state, they are passed a ValidationContext through which the state is accessible. The validation function looks roughly like this:

/// Validates a single \a notification with contextual information \a context.
template<typename TNotification>
virtual ValidationResult validate(const TNotification& notification, const ValidatorContext& context) const = 0;

You might have noticed the use of the word “roughly” when describing the validate functions above. That’s because the real implementation relies on some template trickery NotificationValidator. 😬

BalanceTransferValidator

Before transferring tokens from a sender to a recipient, we want to ensure the sender has sufficient tokens and won’t end up with a negative balance. These checks occur in the balance transfer validator. Due to the dark magic of macros, CheckBalance is effectively the validate call referenced above.

Simplifying with some inlining and dropping support for dynamic fee calculations, we end up with something like this:

template<typename TNotification>
ValidationResult CheckBalance(const TNotification& notification, const ValidatorContext& context) {
    const auto& cache = context.Cache.sub<cache::AccountStateCache>();

    Amount amount;
    auto mosaicId = context.Resolvers.resolve(notification.MosaicId);
    const auto& senderAddress = notification.Sender.resolved(context.Resolvers);
    if (FindAccountBalance(cache, senderAddress, mosaicId, amount)) {
        if (amount >= notification.Amount)
            return ValidationResult::Success;
    }

    return Failure_Core_Insufficient_Balance;
}

The stateful validator is accepting a BalanceTransferNotification and a ValidationContext. First, we retrieve the AccountStateCache from the ValidatorContext. Next we use FindAccountBalance to check that the sender exists and retrieve its current balance. Finally, we check that the sender’s balance (amount) is at least the transfer amount (notification.Amount). If all checks pass, we return success (ValidationResult::Success). Otherwise, we return a failure (Failure_Core_Insufficient_Balance).

For interested readers, the following declaration is what converts CheckBalance into a stateful NotificationValidator:

DEFINE_STATEFUL_VALIDATOR_WITH_TYPE(BalanceTransfer, BalanceTransferNotification, CheckBalance<BalanceTransferNotification>)

Exploring that is beyond the scope of this article.

We have one more section to go, you can do it! 💪

Observers

Now that we’ve established that the transfer can be made, we need to actually make it.

Catapult has observers for this. Observers accept a notification and the current state and modify the state. The observer function should look familiar to the stateless validator validate function if you’re not hallucinating.

/// Notifies the observer with \a notification and observer \a context.
virtual void notify(const TNotification& notification, ObserverContext& context) const = 0;

There’s no template magic here. 😢You can find that exact line in NotificationObserver.

Due to the current rollback implementation, every observer needs to be able to commit and rollback every notification. In other words, the following must always hold:

S = ...
S' = observe(N, S, commit)
S'' = observe(N, S', rollback)
assert(S'' == S)

BalanceTransferObserver

After all that, we can finally move tokens. The balance transfer observer is the observer that does that. Due to the darkmagic of macros, the lambda in DEFINE_OBSERVER is effectively the observe call referenced above.

Simplifying by dropping support for dynamic fee calculations, we end up with something like this:

DEFINE_OBSERVER(BalanceTransfer, model::BalanceTransferNotification, [](
        const model::BalanceTransferNotification& notification,
        const ObserverContext& context) {
    auto& cache = context.Cache.sub<cache::AccountStateCache>();
    auto senderIter = cache.find(notification.Sender.resolved(context.Resolvers));
    auto recipientIter = cache.find(notification.Recipient.resolved(context.Resolvers));

    auto& senderState = senderIter.get();
    auto& recipientState = recipientIter.get();

    auto effectiveAmount = notification.Amount;
    auto mosaicId = context.Resolvers.resolve(notification.MosaicId);
    if (NotifyMode::Commit == context.Mode)
        Transfer(senderState, recipientState, mosaicId, effectiveAmount);
    else
        Transfer(recipientState, senderState, mosaicId, effectiveAmount);
})

The observer is accepting a BalanceTransferNotification and an ObserverContext. First, we retrieve the sender and recipient account states from the ValidatorContext and resolve the mosaic id.

If we’re in Commit mode, we’re moving tokens from the sender to the recipient. If we’re in Rollback mode, we’re doing the opposite and moving from the recipient to the sender in order to restore the original state.

Notice that we don’t need any error checking here because the validators already checked everything!

That’s it, you got through the whole article! 👏 I hope you have learned something interesting. For your time, here’s a gold star ⭐.