XRPL Plugins

[mvadari]

Updated version: XLS 42d: XRPL Plugins

XRPL Plugins

Problem

Currently, very few people have the knowhow to build new features for rippled, such as new transaction types. This limits ecosystem growth and makes feature development less decentralized.

The guiding question of this proposal: How can we make it easier for developers to modify rippled to build sidechains?

There are two main ideas behind this project:

• Make it easier to build new C++ transaction types, so developers don’t need to intimately know rippled in order to add new features.
• Make transaction type development language-agnostic, so developers don’t need to intimately know C++ in order to modify rippled to add new features.

This project will be useful for several groups of devs:

• Sidechain developers
• Researchers (who typically aren't C++ experts) experimenting with new features
• Other devs who want to contribute to the community by writing a new rippled feature

Overview

The general idea of this proposal is to be able to add new plugin features (transaction types, ledger objects, RPC requests) without needing to recompile rippled or needing to know anything about C++. Any dev can just take rippled out-of-the-box, follow a few steps with their language of choice, and write a new feature.

The C++ architecture will be as follows:

1. A C++ shared static library that is...
2. Called by a C++ program that is written by the user and contains all of the transaction logic, which is...
3. Compiled as a shared dynamic library (.so/.dll/.dylib), which is...
4. Called by rippled at runtime, without needing to recompile rippled

We will initially implement this project in 4 languages: C++, Python, Rust and Javascript.

The non-C++ architecture (with Python as an example) will be as follows:

1. A C/C++ shared library (with Python wrappers to convert it to a Python library) that is...
2. called (as a Python library) by a Python module that is written by the user and contains all of the transaction logic, which is...
3. Compiled as a shared library (.so/.dll/.dylib) via some wrapper C++ code (which has the Python interpreter embedded in it), which is...
4. called by rippled (C++) at runtime

This proposal is only for sidechains and experimental features, and recommends that mainnet features are still written in C++.

Note: The rest of this document currently only covers transactors and ledger objects, not RPC requests. There will be a separate design for that at a later point in time, as it would also be useful to have plugin versions of those features. It will follow the same general idea.

Background

Anatomy of a Transactor

The term transactor refers to the code in rippled that processes a transaction.

Every transactor is mainly made up of 5 functions:

• preflight: What is everything that you can check about the validity of the transaction without needing to check the current ledger state?
  • This method is run by the node that receives the submitted transaction before it is broadcast across the network to all the other nodes and validators, so any processing that happens here instead of later makes the network more efficient.
• calculateBaseFee: Calculate the fee that the transaction needs to pay (usually this is just inherited from the base transactor, but e.g. escrows need higher fees)
• preclaim: What is everything that you can check about the validity of the transaction with read-only access to the ledger state (within reason, since you don’t want to duplicate too much work between preclaim and doApply)?
• doApply: Do a few sanity checks, and actually try to apply the transaction (if it fails, then you can throw another error)
• makeTxConsequences: Used when an account has multiple transactions queued to estimate whether it’ll be able to pay the fees for all of them

There are several other methods that transactors call, but they only rarely need to be modified.

Other parts of a transaction include:

• The name of a transaction
• The params of a transaction (TxFormats)
• The transaction type ID (TxType)
• The flags for the transaction (TxFlags)
• The SFields in a transaction
• The types of the SFields in a transaction (Serialized Types)

Anatomy of a Ledger Object

Ledger objects are much simpler, since they are only data structures and don't have any functionality.

The parts of a ledger object are:

• The name of the ledger object
• The params of a ledger object (LedgerFormats)
• The unique type of a ledger object (LedgerEntryType)

Implementation Details

The implementation details are still being worked out in prototypes. Once they are more solidified, a full XLS spec will be written and published.

Example Plugins

Here are a few fully-functional transactor examples ported from existing rippled functionality:

Python

SetRegularKey

tx_name = "PythonSetRegularKey"
tx_type = 30 # the next available transaction type number
tx_format = [
    (sfRegularKey, soeOPTIONAL),
    (sfTicketSequence, soeOPTIONAL)
]

def preflight(ctx):
    if (preflight1ret := preflight1(ctx)) != tesSUCCESS:
        return preflight1ret

    if ctx.tx.getFlags() & tfUniversalMask:
        print("Malformed transaction: Invalid flags set.")
        return temINVALID_FLAG

    if ctx.rules.enabled(fixMasterKeyAsRegularKey) and \
        ctx.tx.isFieldPresent(sfRegularKey) and \
            ctx.tx[sfRegularKey] == ctx.tx[sfAccount]:
        return temBAD_REGKEY

    return preflight2(ctx)

def preclaim(ctx):
    return tesSUCCESS

def doApply(ctx, _mPriorBalance, _mSourceBalance):
    account = ctx.tx[sfAccount]
    sle = ctx.view().peek(accountKeylet(account))
    # skip weird fee stuff
    if ctx.tx.isFieldPresent(sfRegularKey):
        sle[sfRegularKey] = ctx.tx[sfRegularKey]
    else:
        # Account has disabled master key and no multi-signer signer list.
        if sle.isFlag(lsfDisableMaster) and not ctx.view().peek(signersKeylet(account)):
            return tecNO_ALTERNATIVE_KEY

        del sle[sfRegularKey]
    return tesSUCCESS

Note: this transactor ignores the case where a user's first SetRegularKey is free, for the purpose of simplicity and readability.

Original C++ transactor, for reference

Rust

SetRegularKey

struct RustSetRegularKey;

impl Transactor for RustSetRegularKey {
    fn pre_flight(ctx: PreflightContext) -> NotTEC {
        let preflight1 = preflight1(&ctx);
        if preflight1 != TEScodes::tesSUCCESS {
            return preflight1;
        }

        if ctx.tx().flags() & TF_UNIVERSAL_MASK != 0 {
            return TEMcodes::temINVALID_FLAG.into();
        }

        let sf_regular_key = SField::sf_regular_key();
        if ctx.rules().enabled(&Feature::fix_master_key_as_regular_key()) &&
            ctx.tx().is_field_present(&sf_regular_key) &&
            ctx.tx().get_account_id(&sf_regular_key) == ctx.tx().get_account_id(&SField::sf_account()) {
            return TEMcodes::temBAD_REGKEY.into();
        }

        preflight2(&ctx)
    }

    fn pre_claim(ctx: PreclaimContext) -> TER {
        TEScodes::tesSUCCESS.into()
    }

    fn do_apply<'a>(ctx: &'a mut ApplyContext<'a>, m_prior_balance: XrpAmount, m_source_balance: XrpAmount) -> TER {
        let tx = &ctx.tx;
        let view = &mut ctx.view;
        let app = &mut ctx.app;
        let base_fee = ctx.base_fee;

        let account = tx.get_account_id(&SField::sf_account());
        let maybe_sle = view.peek(&Keylet::account(&account));

        if maybe_sle.is_none() {
            return TEFcodes::tefINTERNAL.into()
        } else {
            let mut sle = maybe_sle.unwrap();
            if minimum_fee(app, base_fee, view.fees(), view.flags()) == XrpAmount::of_drops(0).unwrap() {
                sle.set_flag(LedgerSpecificFlags::lsfPasswordSpent);
            }

            let sf_regularkey = &SField::sf_regular_key();
            if tx.is_field_present(sf_regularkey) {
                sle.set_field_account(sf_regularkey, &tx.get_account_id(sf_regularkey));
            } else {
                if sle.is_flag(LedgerSpecificFlags::lsfDisableMaster) && !view.peek(&Keylet::signers(&account)).is_none() {
                    return TECcodes::tecNO_ALTERNATIVE_KEY.into();
                }

                sle.make_field_absent(sf_regularkey);
            }
            return TEScodes::tesSUCCESS.into();
        }
    }

    fn tx_format() -> Vec<SOElement> {
        vec![
            SOElement {
                field_code: SField::sf_regular_key().code(),
                style: SOEStyle::soeOPTIONAL,
            },
            SOElement {
                field_code: SField::sf_ticket_sequence().code(),
                style: SOEStyle::soeOPTIONAL,
            },
        ]
    }
}

#[no_mangle]
pub fn preflight(ctx: &rippled::PreflightContext) -> NotTEC {
    RustSetRegularKey::pre_flight(PreflightContext::new(ctx))
}

#[no_mangle]
pub fn preclaim(ctx: &rippled::PreclaimContext) -> TER {
    RustSetRegularKey::pre_claim(PreclaimContext::new(ctx))
}

#[no_mangle]
pub unsafe fn calculateBaseFee(view: &rippled::ReadView, tx: &rippled::STTx) -> XRPAmount {
    RustSetRegularKey::calculate_base_fee(ReadView::new(view), STTx::new(tx)).into()
}

#[no_mangle]
pub fn doApply(mut ctx: Pin<&mut rippled::ApplyContext>, mPriorBalance: rippled::XRPAmount, mSourceBalance: rippled::XRPAmount) -> TER {
    RustSetRegularKey::do_apply(&mut ApplyContext::new(&mut ctx.as_mut()), mPriorBalance.into(), mSourceBalance.into())
}

#[no_mangle]
pub fn getTxType() -> u16 {
    31
}

Original C++ transactor, for reference

Adding Additional Language Support

Only two pieces would need to be modified to add support for an additional language:

• Wrappers around the C/C++ library code to make it a native package in the new language
• Writing the C API that provides C-style function pointers from a dynamic library for the plugin transactors (for higher-level languages a C++ wrapper layer may be necessary)

Current Source Code

You can find the current version of the plugin transactor code in this repo. Note that it is still in the prototype phase and thus the code is incomplete, very messy, and sparsely documented.

Questions for the Community

1. Do you find this interesting or useful?
2. Is there anything missing in here that you would like to see?
3. Do you have any concerns about this feature?

[jsardello]

I think plugins would be a great way to prove out amendments in tandem to sidechains. One proposed amendment to the XRPL that most seem to agree upon is some sort of fee escalation for memo size. However, nobody can land the plane on what that implementation specifically should look like.

Seems to me like a plugin framework would allow for several different fee escalation variants to be stood up and experimented with.

[mvadari]

Fee escalation might not be doable with this API - its main usecase is new transaction types/ledger object types. You could probably experiment by building a new dummy transactor that charges fees based on memo size, though.

[AlexanderBuzz]

Looks super interesting!

[xVet]

I see great potential here especially for drafting new functionalities and making proposal makings more accessible for the community!

[shortthefomo]

• Do you find this interesting or useful?
  Yup looking really forward to getting to try this out, specifically having ability to write evernode contracts with possibly a specific token type it's extremely useful 👍🏻
• Is there anything missing in here that you would like to see?
  Yes for me so far there is no JS or I would be trying this out already
• Do you have any concerns about this feature?
  Mainly what current public ledger can this be deployed to? Is there a public ledger that one can test this on?

[mvadari]

Javascript is definitely in the pipeline 🙂

There's no public ledger this can be deployed to. The only way to test it out would be to set up my branch of rippled locally and run it on standalone. It's a node/validator-level feature, and every node and validator in the network would need to run the same plugin transactors for it to work.

[intelliot]

Should this conversation be closed in favor of the new spec? #116

[mvadari]

Closed in favor of #116