Event & Data Extraction
In this chapter, we will delve into the techniques and best practices for extracting events and data within your Substreams. Having learned about the building blocks of a Substreams package, the various data types available, and how to build modules, we will now focus on adding functionality to our modules that enables efficient extraction of pertinent data.
This chapter aims to provide a more in-depth look at how to capture and process data from the blockchain, utilizing the powerful features of Substreams. While we have touched on some of these approaches in previous chapters, this section will offer a more focused and detailed exploration of the methods used for event and data extraction.
Extracting an Event
In this section, we will explore how to extract a single type of event using the turbofish syntax and the .events helper function. This method, exposed on the Block type, enables processing of specific events emitted by contracts during a block.
Adding ABI and Generating Rust Bindings
To begin, ensure that the ABI for the contract you are interested in is added to the /abi directory of your codebase. You must also update the build.rs file to generate the corresponding Rust bindings for this ABI. These bindings provide access to a wealth of functionality related to the ABI, including event extraction.
Using the .events. Helper Function
The .events helper function on the Block type simplifies the extraction of events. The syntax for using this function is as follows:
block.events::<EventTypeFromABI>(&[&CONTRACT_TO_EXTRACT_EVENTS_FROM])Where EventTypeFromABI is the type of event from the generated rust bindings, and CONTRACT_TO_EXTRACT_EVENTS_FROM is the address of the contract that emits the event in .
This line processes a single type of event from a given contract in a block, returning an iterator of these events that were emitted during the block from the specified contract.
Here is a more complete example, similar to the code snippet you might recognize from the previous chapter:
const UNISWAP_V3_FACTORY: [u8; 20] = hex!("1f98431c8ad98523631ae4a59f267346ea31f984");
#[substreams::handlers::map]
fn map_pools_created(blk: eth::Block) -> Result<Pools, substreams::errors::Error> {
Ok(Pools {
pools: blk
// Extract PoolCreated events from the Uniswap V3 Factory contract
.events::<PoolCreated>(&[&UNISWAP_V3_FACTORY])
.map(|(event, log)| Pool {
address: Hex::encode(event.pool),
token0: Hex::encode(event.token0),
token1: Hex::encode(event.token1),
created_at_tx_hash: Hex(&log.receipt.transaction.hash).to_string(),
created_at_block_number: blk.number,
created_at_timestamp: blk.timestamp_seconds(),
log_ordinal: log.ordinal(),
})
.collect(),
})
}Let’s break down this module:
blk.events::<PoolCreated>(&[&UNISWAP_V3_FACTORY]):- This line uses the turbofish syntax to specify the type of event (
PoolCreated) we want to extract. It filters the events in the block to include only those emitted by theUNISWAP_V3_FACTORYcontract. Note that the contract address passed to the function is of type[u8; 20].
- This line uses the turbofish syntax to specify the type of event (
- Performing logic on the filtered events:
- Once we have extracted the relevant events, we can perform logic on these. In our case we are using
.map(|(event, log)| Pool {...}), this maps each matched event and corresponding log to a new Pool struct, extracting and converting the necessary details such as the pool address, tokens, transaction hash, block number, timestamp, and log ordinal.
- Once we have extracted the relevant events, we can perform logic on these. In our case we are using
- Creating Result Object:
Ok(Pools { pools: ... }): This initializes a Pools object to store the created pools and returns it as the result of the function. This is the output of the map function, and can be used as an input to subsequent modules in our Substreams package.
Extracting Multiple Events
What happens when we need to extract multiple types of events from a contract? This is a common requirement, and addressing it involves a different approach.
While you could use the .events function for each event type, this approach would be inefficient. Each call to .events loops through the logs in the block, extracting each type of event one at a time, which can lead to significant performance overhead when processing large volumes of data.
A more efficient method is to loop through the logs in the block once and check against multiple event types within this loop. This approach ensures that each log is processed only once, reducing the computational overhead and improving the performance of your Substreams.
Below is a code example demonstrating how to extract multiple types of events within a single loop. Note that we use if let Some on the result of a match and decode function instead of the turbofish syntax.
#[substreams::handlers::map]
pub fn map_pool_events(
blk: eth::Block,
pool_store: StoreGetProto<Pool>,
) -> Result<PoolEvents, substreams::errors::Error> {
let mut pool_events = PoolEvents::default();
// Iterate through each transaction in the block
for trx in blk.transactions() {
// Iterate through each log within the transaction
for (log, _) in trx.logs_with_calls() {
let pool_address = Hex::encode(&log.address);
// Check if the pool address exists in the pool store
if let Some(pool) = pool_store.get_last(&pool_address) {
// Attempt to match and decode the Swap event
if let Some(swap) = abi::pool_contract::events::Swap::match_and_decode(&log) {
pool_events.events.push(PoolEvent {
r#type: Some(pool_event::Type::SwapEvent(pool_event::SwapEvent {
sender: Hex::encode(&swap.sender),
recipient: Hex::encode(&swap.recipient),
amount0: swap.amount0.to_string(),
amount1: swap.amount1.to_string(),
tick: swap.tick.to_string(),
})),
pool_address: pool.address.clone(),
tx_hash: Hex::encode(&trx.hash),
block_number: blk.number,
timestamp: blk.timestamp_seconds(),
log_ordinal: log.ordinal,
});
}
// Attempt to match and decode the Mint event
if let Some(mint) = abi::pool_contract::events::Mint::match_and_decode(&log) {
pool_events.events.push(PoolEvent {
r#type: Some(pool_event::Type::MintEvent(pool_event::MintEvent {
sender: Hex::encode(&mint.sender),
// other fields...
})),
pool_address: pool.address.clone(),
// other fields...
});
}
// Attempt to match and decode the Burn event
if let Some(burn) = abi::pool_contract::events::Burn::match_and_decode(&log) {
pool_events.events.push(PoolEvent {
r#type: Some(pool_event::Type::BurnEvent(pool_event::BurnEvent {
tick_lower: burn.tick_lower.to_string(),
// other fields...
})),
pool_address: pool.address.clone(),
// other fields...
});
}
}
}
}
Ok(pool_events)
}Let’s break down this module:
-
Initialize the Result Object:
let mut pool_events = PoolEvents::default();initializes a PoolEvents object to store the extracted events.
-
Loop Through Transactions and Logs:
for trx in blk.transactions()iterates through each transaction in the block.for (log, _) in trx.logs_with_calls()iterates through each log within the transaction, along with the associated call data. We have opted to use_here to ignore the call data, as it is not needed.
-
Check for Pool Address:
if let Some(pool) = pool_store.get_last(&pool_address)checks if the pool address is present in the pool store.
-
Match and Decode Events:
if let Some(swap) = abi::pool_contract::events::Swap::match_and_decode(&log)attempts to match and decode theSwapevent from the log.if let Some(mint) = abi::pool_contract::events::Mint::match_and_decode(&log)attempts to match and decode theMintevent from the log.if let Some(burn) = abi::pool_contract::events::Burn::match_and_decode(&log)attempts to match and decode theBurnevent from the log.
-
Create Event Objects:
- For each matched event, creates a corresponding
PoolEventobject, extracting and converting the necessary data.
- For each matched event, creates a corresponding
-
Return the Result Object:
Ok(pool_events)returns the PoolEvents object containing all extracted events.
Extracting Data from Function Calls
Sometimes, functions in smart contracts do not emit events, yet you might still want to extract information related to these function calls. Fortunately, similar to the events available in ABI bindings, we also have access to contract functions and can match and decode them by passing a call in. This capability allows us to extract pertinent data directly from function calls within transactions.
Consider a scenario where you want to extract data from the EnableFeeAmount function call in a Uniswap factory contract. Below is a code snippet demonstrating how to do this:
for trx in blk.transactions() {
for call in trx.calls() {
if let Some(enable_fee_amount_call) = abi::factory_contract::functions::EnableFeeAmount::match_and_decode(&call) {
return Some(FeeAmountEnabledCall {
fee: enable_fee_amount_call.fee.to_string(),
tick_spacing: enable_fee_amount_call.tick_spacing.to_string(),
});
}
}
}In this code:
- We iterate through each transaction in the block using
for trx in blk.transactions(). - For each transaction, we further iterate through all calls using
for call in trx.calls(). - We then use
if let Some(enable_fee_amount_call) = abi::factory_contract::functions::EnableFeeAmount::match_and_decode(&call)to check if the current call matches theEnableFeeAmountfunction signature and decode its parameters if it does. - If a matching function call is found, it returns a
FeeAmountEnabledCallstruct containing the extracted data (feeandtick_spacing).
The process of extracting data from function calls parallels the syntax used when extracting events from logs. For example:
- Events Extraction:
if let Some(burn) = abi::pool_contract::events::Burn::match_and_decode(&log) {- Function Call Extraction:
if let Some(enable_fee_amount_call) = abi::factory_contract::functions::EnableFeeAmount::match_and_decode(&call) {In both cases, the match_and_decode function on the ABI binding is used to identify and decode specific interactions, whether they are events in logs or function calls in transactions.
This method enhances your ability to capture relevant data from function calls, even when no events are emitted.