Technical Deep Dive into the Solana Transaction Settlement Lifecycle
From a transaction leaving your wallet to getting settled on the blockchain within seconds, this article covers how settlements happen in the land of Solana. Sit back, get your technical glasses on to enjoy some glass-chewing
What is a transaction, and what does it look like?
Transactions are a set of instructions that are bundled together to process or make up for a change in the Solana global state.
To be precise, a transaction might consist of not just the instructions but the set of accounts(array) to read/write along with one or more signatures (permission)
These transactions(tx) can be an NFT Mint, a token transfer, or any activity on the blockchain, and an instruction (ix) is the smallest contiguous unit of execution logic in a program. Transactions are limited to 1232 bytes.
Here is an example of a transaction that interacts with a program to bid an NFT.
https://solscan.io/tx/T8Yi6Wd5BdF9pECbdFYUaScrAUSLifECaZAKfxT3YNBzDaZVtxSsK4TrYSNyuUhaEfNV4AbgUtBGoennLhWsjhx
One can verify the signer and all the instructions with a set of accounts to change or modify the state.
Client Side Processing
The moment you decide to swap a token or approve a change in a game, such as buying or selling an asset or purchasing an NFT, it goes through certain steps before being received by the validator using the RPC. There are two methods via which the transaction is received by the validator using the RPC.
Methods of Transactions Processing
The transaction can be sent to the leaders either programmatically or from the end user in two ways.
Using the standard sendTransaction JSON-RPC method to the RPC Server
Forwarding directly to the leader node via TPU Client
While both methods are accepted; however, RPC is the preferred one. Both methods send the transactions over the UDP to the current and next leader. Until the transaction is processed by a leader, there is no record of the transaction outside of what the client and the relaying RPC nodes are aware of. In the case of a TPU client, rebroadcast and leader forwarding is handled entirely by the client software.
Forwarding through RPC Nodes
Submitting a transaction from your wallet along with the recent block hash to prepare a transaction message goes directly to the RPC node, which forwards it to the current block producer/ the leader node that validates and commits the transaction in their produced block. This happens during each leader slot which lasts around 400 ms, the given time for them to commit the transaction. The block hash is used to verify the last entry ID of the block so that it lands on time.
A slot is equivalent to sequentially ordered and non-overlapping, roughly equal real-world time as per Proof of History. One can use solanabeach.io or validators. app to verify/ look at the current leader node and upcoming leader schedule.
Solana uses QUIC (a custom raw UDP-based protocol) to pass transactions between RPC nodes and the current leader.QUIC is designed for fast asynchronous communication like UDP but with sessions and flow control like TCP by Google is the default transaction ingestion protocol from 1.13.4 release.
Since the leader schedule in Solana is known well in advance (before an epoch), This is targeted towards the leader’s TPU (Transaction processing Unit), whose process is split into 5 different phases.
These are
Fetch Stage
The fetch stage batches input from a UDP socket and sends it to a channel.
SigVerify Stage
The SigVerify stage verifies signatures on packets that come under the tpu module and implements the Transaction Processing Unit, a multi-stage transaction processing pipeline that pushes it over to the banking stage.
This has two different pipelines to run different checks on votes and transactions to avoid DOS on the vote pipeline.
Banking Stage
The banking stage processes Transaction messages. It is intended to be used
to construct a software pipeline. The stage uses all available CPU cores and
can do its processing in parallel with signature verification on the GPU.
PoH Service
The poh_service module implements a service that records the passing of
"ticks"(smaller than a slot), a measure of time in the PoH stream.
Broadcast stage
This stage is responsible for broadcasting entries into the network. These entries are converted to shreds and passed onto other network nodes through the turbine.
Commitment Status
There are three specific commitment statuses:
Processed - This is the initial stage of the commitment process. When a transaction is submitted to the Solana network, it is first processed by the network's nodes. This involves validating the transaction's correctness and ensuring it adheres to the network's rules. Once processed, the transaction is included in a block and broadcast to the network
.
Confirmed - At this stage, the transaction has been included in a block and voted on by a supermajority of the Solana nodes. This means that a significant portion of the network has acknowledged the transaction's validity and its inclusion in the blockchain.
Finalized - This is the highest level of commitment and indicates that the transaction is permanently recorded on the blockchain. A block must have reached a maximum lockout period to reach the finalized state. This means that a sufficiently large number of blocks have been added to the blockchain after the block containing the transaction. This ensures that the transaction has been distributed across a large enough portion of the network and is unlikely to be reversed due to any errors or malicious attacks.
Although each level of commitment depends on the network congestion, type, transaction volume etc.., the difference almost takes a very short amount of time (in seconds ) on Solana.
Retrying Transactions
In the case of Network congestion, the RPC node fails to rebroadcast the transaction to the leader. To an end-user, it may appear as if their transaction disappears entirely. While RPC nodes are equipped with a generic rebroadcasting algorithm, application developers can also develop their own custom rebroadcasting logic.
Beyond your wallets and client-side processing
Things that should be known once about your ecosystem, what happens after your transaction has been received by the validator? Well lets have a look into it.
Gulfstream: Solana’s Mempooless transaction forwarding protocol
A mempool is where all the transactions wait to be processed by the network. Anyone can query the mempool transactions on Ethereum with various JSON-RPC methods available for different execution clients, which are waiting for validators to pick them up and put them in a block.
While being in the queue, anyone can pay extra fees to prioritize themselves to get their transactions done, thereby incentivizing validators which costs much more than just gas for an ETH transaction to get through earlier.
Not just costs, but mempool also makes up for a lot of MEV opportunities that don't make sense to a normal user transacting on the blockchain. This leads to more failed transactions as well.
While there is no global ordering of queues for transaction execution on Solana, unlike Ethereum, there is local ordering in each multi-threaded queue. (Each thread has its own independent queue of transactions waiting for execution. )
Each program on Solana can utilize 12M CUs(compute Units) out of the total 48 CUs allocated per block, which prevents any single program from taking over the blockspace. This prevents gas wars and hotspots on the chain.
Turbine
The Turbine concept in Solana introduces a method that makes it easy for the leader or chosen validator to add a new block, and to share the context with other validators for confirmation. Turbine uses something called "shreds," where the given block is broken down into pieces. These pieces, along with the use of Reed-Solomon erasure coding, are then shared with validators hierarchically in multiple layers. The topmost layer consists of the highest-staked validators. Therefore, these staked validators are the first ones to receive the shreds, and then they are passed on to the other validators. The existing validators that have received the shreds relay them further, following the provided structure.
Fee Markets:-
While the number of unconfirmed transactions depends on the supply and demand of blockspace, networks can become highly unusable with really high gas fees. Eth folks have seen the huge spike in gas during the Otherside BAYC mint and various other events for the whole chain.
Also, to avoid spamming and congestion on a low-fee network like Solana, Local Fee markets were introduced in the late summer of 22. This ensured that one particular event wouldn't affect the network's usability, and this was pretty much seen as successful when Mad Lads Mint and Helium’s transition to Solana happened simultaneously without affecting anyone using the network. No spike in failed transactions or high gas fees for the network except for high fees for that particular program’s limited blockspace.
Sealevel Runtime
We all know Solana uses accounts to store state. i.e. to write data onto the blockchain.
One such speciality is the Sealevel runtime that parallel processes multiple transactions at the same time compared to EVM’s single-threaded runtime. This happens as the accounts that are to be read and written are mentioned before processing the transactions, and conflicting/overlapping transactions execute concurrently based on the historical time using Proof of History
Why are failed transactions shown in the explorers?
These are transactions that use computation but fail in programming logic/errors.
Base fees and Priority fees (Local and global)
Priority fees take the stage when the demand for blockspace is high. When the demand is low, most of the low CU transactions get through easily, but the transactions that pay higher fees for relative CUs get priority in transaction ordering.
In the case of high network demand (transacting on the chain - global fee markets), the base fee increases for all the transactions. Reducing account CU limits might prevent hotspots in that case, but it comes with a lot of advantages and disadvantages too.
Multiple teams on Solana are working together to increase the blockspace, which seems like an ideal condition to avoid high fees on hotspots and global fees
Conclusion
Understanding transaction settlements can give one a good overview of all the engineering lifts that Solana does to outstand itself from all the other existing blockchains, and it has been improving everyday as well.
Many teams have been working their best to improve the network, and the upcoming clients like Firedancer and Zig are pretty interesting to look into if you want to explore more.
To more transactions at the speed of light 🥂⚡
References:-
https://twitter.com/7LayerMagik/status/1615569374647287808