Data Conversion

Converting Addresses

In Substreams, converting addresses between their byte representation (Vec<u8>) and their hexadecimal string representation is a common task. The Hex utility from the substreams module simplifies these conversions. Below, we’ll go through each conversion method with detailed explanations and code examples.

Bytes to String

To convert an address from its byte representation (Vec<u8> or [u8; 20]) to a hexadecimal string, you can use Hex::encode. This method is straightforward and is commonly used when you need to save or log addresses as strings.

use substreams::Hex;
 
// Example byte address
let address_bytes: [u8; 20] = [31, 152, 67, 28, 138, 217, 133, 35, 99, 26, 228, 165, 159, 38, 115, 70, 234, 49, 249, 132]
 
// Convert bytes to string
let address_string = Hex::encode(&address_bytes);

String to Bytes

To convert a hexadecimal string back to its byte representation, use Hex::decode. This method returns a Result which we can unwrap.

use substreams::Hex;
 
// Example hexadecimal address string
let address_string = "1f98431c8ad98523631ae4a59f267346ea31f984";
 
// Convert string to bytes
let address_bytes = Hex::decode(address_string).unwrap();

Safely Converting from String to Bytes

When converting from a string to bytes, it’s important to handle errors safely. You can use pattern matching or the Result handling methods to ensure your code is robust against invalid input.

Using if let for Error Handling

use substreams::Hex;
 
let address_string = "1f98431c8ad98523631ae4a59f267346ea31f984";
 
if let Ok(address_bytes) = Hex::decode(address_string) {
    substreams::log::debug!("Successfully converted to bytes: {:?}", address_bytes);
    // Do something with `address_bytes`
}

Using match for Comprehensive Error Handling

use substreams::Hex;
 
let address_string = "1f98431c8ad98523631ae4a59f267346ea31f984";
 
match Hex::decode(address_string) {
    Ok(address_bytes) => {
        substreams::log::debug!("Successfully converted to bytes: {:?}", address_bytes);
        // Do something with `address_bytes`
    }
    Err(err) => {
        substreams::log::debug!("Error decoding address: {:?}", err);
    }
}

BigInt & BigDecimal

As we are dealing with on-chain data in our Substreams, we will often have to handle BigInt and BigDecimal values.

In Substreams, BigInt and BigDecimal values can’t be directly stored in protobuf messages, so conversion to and from strings is necessary when passing data between modules. The BigInt and BigDecimal types come from use rust substreams::scalar::{BigInt, BigDecimal}.

Converting `BigInt` or `BigDecimal` to `String`

To convert a BigInt or BigDecimal to a string, you can use the to_string method. This is straightforward and commonly used when you need to serialize these values for protobuf messages.

use substreams::scalar::{BigInt, BigDecimal};
 
// Example BigInt
let big_int = BigInt::from(1);
 
// Convert BigInt to string
let big_int_string = big_int.to_string();
 
// Example BigDecimal
let big_decimal = BigDecimal::from(1.23);
 
// Convert BigDecimal to string
let big_decimal_string = big_decimal.to_string();

Converting `String` to `BigInt` or `BigDecimal`

To convert a String back to a BigInt or BigDecimal, use the from_str method. This method returns a Result, so you can handle potential errors gracefully.

Safely Using match for Error Handling

use substreams::scalar::{BigInt, BigDecimal};
use std::str::FromStr;
 
// Example BigInt string
let big_int_string = "1";
 
match BigInt::from_str(big_int_string) {
    Ok(big_int) => {
        // Do something with `big_int`
    }
    Err(err) => {
        substreams::log::debug!("Error converting to BigInt: {:?}", err);
    }
};
 
// Example BigDecimal string
let big_decimal_string = "1.23";
 
match BigDecimal::from_str(big_decimal_string) {
    Ok(big_decimal) => {
        // Do something with `big_decimal`
    }
    Err(err) => {
        substreams::log::debug!("Error converting to BigDecimal: {:?}", err);
    }
};

Using `unwrap_or_else` for Safe Conversion

use substreams::scalar::{BigInt, BigDecimal};
use std::str::FromStr;
 
// Example BigInt string
let big_int_string = "1";
 
let big_int = BigInt::from_str(big_int_string).unwrap_or_else(|_| {
    substreams::log::debug!!("Failed to parse BigInt");
    // Provide default value if appropriate
    BigInt::zero()
});
 
// Example BigDecimal string
let big_decimal_string = "1.23";
 
let big_decimal = BigDecimal::from_str(big_decimal_string).unwrap_or_else(|_| {
    substreams::log::debug!!("Failed to parse BigDecimal");
    // Provide default value if appropriate
    BigDecimal::zero()
});

Converting `BigInt` to `BigDecimal`

When dealing with large numbers on the blockchain, especially in contexts such as token balances or amounts, it is often necessary to convert between BigInt and BigDecimal. This is because BigInt represents integers while BigDecimal can represent fractional values. For example, token balances are often stored as integers (BigInt) in the smallest unit (like wei for Ether), but you may want to represent these values as decimals (BigDecimal) for readability and usability, or when calculating USD values of token amounts.

A common case is converting token amounts using the token’s decimal precision.

use substreams::scalar::{BigInt, BigDecimal};
 
// Example BigInt representing a token amount
let amount_big_int = BigInt::from(100000);
let decimals = 18;
 
// Convert BigInt to BigDecimal using the token's decimals
let amount_big_decimal = amount_big_int.to_decimal(decimals);

In this example:

amount_big_int represents a token amount in the smallest unit (e.g., wei for Ether).
decimals represents the token’s decimal precision (e.g., 18 for Ether).
amount_big_decimal is the human-readable decimal representation of the token amount.

Date & Time

In Substreams, you can utilize the Block or Clock types to work with date and time. Both types provide access to the timestamp, which represents the number of seconds since the Unix epoch (1970-01-01T00:00:00Z). This timestamp can be used to calculate day or hour IDs.

You can retrieve the timestamp from a Block or Clock object using the timestamp method. The seconds method returns the number of seconds as an i64, representing UTC time since the Unix epoch.

// Using Block to get the timestamp
let block_timestamp = block.timestamp().seconds();
 
// Using Clock to get the timestamp
let clock_timestamp = clock.timestamp().unwrap().seconds;

Once you have the timestamp, you can use it to calculate the day or hour ID since the Unix epoch. This is useful for aggregating data by day or hour.

// Example timestamp in seconds
let block_timestamp = block.timestamp().seconds();
 
// Calculate the day ID since Unix epoch
let day_id = timestamp_seconds / 86400; // 86400 seconds in a day
 
// Calculate the hour ID since Unix epoch
let hour_id = timestamp_seconds / 3600; // 3600 seconds in an hour

Data Types in Substreams Modules