Welcome to Arch Network

What is Arch Network?
Arch Network is a computation environment that enhances Bitcoin's capabilities by enabling complex operations on Bitcoin UTXOs through its specialized virtual machine. Unlike Layer 2 solutions, Arch Network provides a native computation layer that works directly with Bitcoin's security model.
Choose Your Path ๐
๐ Deploy First
Get your first smart contract running on Arch Network as quickly as possible
- Download CLI and deploy a program in 15 minutes
- Use our pre-configured development environment
- Perfect for developers wanting to try Arch Network
๐๏ธ Run a Validator
Set up and run your own validator node on the Arch Network
- Set up Bitcoin Core and Titan
- Configure and run a validator node
- Perfect for those wanting to participate in network security
Network Options
๐ง Regtest
Local development environment with instant block confirmation. Perfect for development and testing.
๐งช Testnet
Test network with real Bitcoin testnet integration. For testing in a live environment.
Key Features
Bitcoin-Native
Direct integration with Bitcoin through UTXO management
Computation Environment
Execute complex programs within the Arch VM
Program Development
Write programs in Rust to interact with Bitcoin UTXOs
Security
Leverages Bitcoin's proven security guarantees through multi-signature validation
Developer Tools
Complete development environment with CLI tools and explorer
Prerequisites
- Node.js v19+ (installation guide)
- Rust (latest stable)
- Docker for local development
- Basic understanding of Bitcoin UTXOs
Core Architecture
How Arch Works
- Network Layer
- Network Architecture
- Bootnode: Network discovery and peer management
- Leader Node: Transaction coordination
- Validator Nodes: Program execution
- Bitcoin Integration
- UTXO Management
- Transaction tracking
- State anchoring
- Ownership validation
- RPC Integration
- Bitcoin node communication
- Transaction submission
- Network synchronization
- Computation Layer
- Programs
- Transaction Processing
- Message validation
- State updates
- UTXO management
๐ Reference Documentation
Need Help?
๐ Quick Start Guide
Welcome to Arch Network! Let's get your first program running in under 15 minutes.
๐ฏ What You'll Build
graph LR A[Your Program] -->|Deploy| B[Local Validator] B -->|Execute| C[Arch Network] classDef default fill:#f8f9fa,stroke:#dee2e6,stroke-width:2px,rx:10px,ry:10px classDef program fill:#ff6b81,stroke:#ff4757,stroke-width:2px,rx:10px,ry:10px classDef validator fill:#2ed573,stroke:#26ae60,stroke-width:2px,rx:10px,ry:10px classDef network fill:#ffd700,stroke:#f4c430,stroke-width:2px,rx:10px,ry:10px class A program class B validator class C network linkStyle default stroke:#a4b0be,stroke-width:2px
โฑ๏ธ Time Estimate
- Total time: ~15 minutes
- Active time: ~10 minutes
- Waiting time: ~5 minutes (during installations)
๐ Quick Setup
1. Install CLI (2 minutes)
Download the appropriate binary for your system from the latest releases page:
curl -L -o cli https://github.com/Arch-Network/arch-node/releases/latest/download/cli-aarch64-apple-darwin
chmod +x cli
sudo mv cli /usr/local/bin/
After installation, verify it works:
cli --version
2. Start Local Validator (1 minute)
Start the local validator:
# Start a local validator
cli validator start \
--network-mode devnet \
--rpc-bind-port 9002 \
--titan-rpc-endpoint titan-node.dev.aws.archnetwork.xyz \
--titan-rpc-port 18443 \
--titan-rpc-username bitcoin \
--titan-rpc-password 428bae8f3c94f8c39c50757fc89c39bc7e6ebc70ebf8f618
3. Clone Example Project (2 minutes)
# Get the starter example
git clone https://github.com/Arch-Network/arch-examples
cd arch-examples/examples/helloworld
4. Build and Deploy (5 minutes)
First Time Setup
If this is your first time building Arch programs, install the required dependencies:
macOS Dependencies
# Install Rust if not already installed
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Install Solana CLI tools
sh -c "$(curl -sSfL https://release.solana.com/v1.18.18/install)"
Linux Dependencies
# Install Rust if not already installed
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Install build essentials
sudo apt-get update && sudo apt-get install -y build-essential
# Install Solana CLI tools
sh -c "$(curl -sSfL https://release.solana.com/v1.18.18/install)"
Build and deploy the program:
# Build the program
cargo build-sbf
# Deploy to your local validator
cli deploy ./target/deploy/helloworld.so
๐ฎ Test Your Deployment
Once deployed, you can interact with your program:
# Show program information
cli show <PROGRAM_ADDRESS>
# Get block information
cli get-block <BLOCK_HASH>
๐ Ready for Testnet?
When you're ready to deploy to testnet:
cli validator start --network-mode testnet
cli deploy ./target/deploy/helloworld.so --network-mode testnet
๐ Next Steps
- Modify the Hello World program
- Create a fungible token
- Build a Runes swap application
- Set up a full validator node
๐ Need Help?
- Join our Discord for real-time support
- Check the Troubleshooting Guide
- Browse the FAQ
๐๏ธ Running an Arch Network Validator
Welcome to the validator setup guide! This comprehensive guide will walk you through setting up a full Arch Network validator node, including all required components. As a validator, you'll be an integral part of the network's security and computation infrastructure.
๐ฏ What You'll Build
graph TD A[Bitcoin Core] -->|Blockchain Data| B[Titan] B -->|Efficient Queries| C[Validator Node] C -->|Participate in| D[Arch Network] D -->|Secure| E[Bitcoin Network] classDef default fill:#f8f9fa,stroke:#dee2e6,stroke-width:2px,rx:10px,ry:10px classDef bitcoin fill:#ffd700,stroke:#f4c430,stroke-width:2px,rx:10px,ry:10px classDef titan fill:#4a90e2,stroke:#357abd,stroke-width:2px,rx:10px,ry:10px classDef validator fill:#2ed573,stroke:#26ae60,stroke-width:2px,rx:10px,ry:10px classDef arch fill:#ff6b81,stroke:#ff4757,stroke-width:2px,rx:10px,ry:10px class A,E bitcoin class B titan class C validator class D arch linkStyle default stroke:#a4b0be,stroke-width:2px
๐ก Understanding Your Role
As a validator, you will:
- Execute smart contracts and validate transactions
- Participate in network consensus
- Help secure the Bitcoin integration
- Earn rewards for your contribution
๐ System Requirements
Before starting, ensure you have:
- 4+ CPU cores
- 16GB+ RAM
- 100GB+ SSD storage
- Stable internet connection
- Linux (Ubuntu 20.04+ or similar) or macOS (12.0+)
๐บ๏ธ Setup Overview
-
Bitcoin Core Setup (30-45 minutes)
- Install dependencies
- Build from source
- Configure for your network
-
Titan Setup (15-20 minutes)
- Build our custom fork
- Configure for your network
-
Validator Setup (10-15 minutes)
- Install Arch Network CLI
- Configure validator node
- Join the network
Total estimated time: 1-1.5 hours
๐ฏ What We're Building
graph TD A[Your dApp] -->|Interacts with| B[Local Validator] B -->|Queries| C[Titan] C -->|Reads| D[Bitcoin Core] D -->|Manages| E[Local Blockchain] classDef default fill:#f8f9fa,stroke:#dee2e6,stroke-width:2px,rx:10px,ry:10px classDef dapp fill:#ff6b81,stroke:#ff4757,stroke-width:2px,rx:10px,ry:10px classDef validator fill:#2ed573,stroke:#26ae60,stroke-width:2px,rx:10px,ry:10px classDef titan fill:#4a90e2,stroke:#357abd,stroke-width:2px,rx:10px,ry:10px classDef bitcoin fill:#ffd700,stroke:#f4c430,stroke-width:2px,rx:10px,ry:10px classDef blockchain fill:#a4b0be,stroke:#747d8c,stroke-width:2px,rx:10px,ry:10px class A dapp class B validator class C titan class D bitcoin class E blockchain linkStyle default stroke:#a4b0be,stroke-width:2px
๐งฉ Understanding the Components
Bitcoin Core ๐ฆ
- Your personal Bitcoin node
- Manages a local blockchain in regtest mode
- Perfect for development - create test Bitcoin at will!
Titan โก
- Lightning-fast Bitcoin data indexer
- Makes blockchain queries super efficient
- Essential for real-time dApp responses
๐ Progress Tracker
- Install Bitcoin Core dependencies
- Build Bitcoin Core
- Configure Bitcoin Core
- Test Bitcoin Core
- Build Titan
- Configure Titan
- Test the full stack
1. ๐๏ธ Bitcoin Core Setup
1.1 Installing Dependencies
macOS
# Install required dependencies via Homebrew
brew install automake boost ccache git libevent libnatpmp libtool \
llvm miniupnpc pkg-config python qrencode qt@5 sqlite zeromq
Ubuntu/Debian Linux
# Install required dependencies
sudo apt-get update && sudo apt-get install -y \
automake autotools-dev bsdmainutils build-essential ccache \
clang gcc git libboost-dev libboost-filesystem-dev \
libboost-system-dev libboost-test-dev libevent-dev \
libminiupnpc-dev libnatpmp-dev libsqlite3-dev libtool \
libzmq3-dev pkg-config python3 qtbase5-dev qttools5-dev \
qttools5-dev-tools qtwayland5 systemtap-sdt-dev
RHEL/Fedora Linux
# Install required dependencies
sudo dnf install -y automake boost-devel ccache clang gcc git \
libevent-devel libnatpmp-devel libtool make miniupnpc-devel \
pkg-config python3 qt5-qtbase-devel qt5-qttools-devel \
sqlite-devel systemtap-sdt-devel zeromq-devel
1.2 ๐ญ Building Bitcoin Core
# Clone Bitcoin Core
git clone https://github.com/bitcoin/bitcoin.git
cd bitcoin
# Switch to latest stable version
git checkout v28.0
# Prepare the build system
./autogen.sh
# Configure the build
./configure
# Build Bitcoin Core (this might take 30-45 minutes)
make -j$(nproc) # Uses all available CPU cores
# Install the binaries
sudo make install
1.3 โ๏ธ Bitcoin Core Configuration
Create your configuration directory:
macOS
mkdir -p ~/Library/'Application Support'/Bitcoin
CONFIG_DIR=~/Library/'Application Support'/Bitcoin
Linux
mkdir -p ~/.bitcoin
CONFIG_DIR=~/.bitcoin
Create and edit your configuration file:
cat > "$CONFIG_DIR/bitcoin.conf" << 'EOF'
# ๐ Network Settings
server=1
regtest=1
txindex=1
prune=0
# ๐ Security (Change these values in production!)
rpcuser=bitcoin
rpcpassword=bitcoinpass
# ๐ง Performance
dbcache=150
maxmempool=100
# ๐ Development Settings
fallbackfee=0.001
maxtxfee=0.002
[regtest]
rpcbind=0.0.0.0
rpcport=18443
wallet=testwallet
EOF
1.4 ๐ Launch Bitcoin Core
# Start Bitcoin Core in regtest mode
bitcoind -regtest -daemon
# Verify it's running
bitcoin-cli -regtest getblockchaininfo
System Requirements
Welcome to the Arch Network development guide. This page will walk you through setting up your development environment with all necessary dependencies. Please follow each section carefully to ensure a smooth setup process.
Overview
Before you begin development with Arch Network, you'll need to install and configure the following tools:
Requirement | Minimum Version | Description |
---|---|---|
Rust | Latest stable | Core development language |
C++ Compiler | gcc/clang | Required for native builds |
Solana CLI | v1.18.18 | Solana development tools |
Arch Network CLI | Latest | Arch Network development toolkit |
Detailed Installation Guide
1. Install Rust
Rust is the primary development language for Arch Network programs.
# Install Rust using rustup
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Verify installation
rustc --version
cargo --version
๐ก Note: Make sure you're using the stable channel throughout this book.
2. C++ Compiler Setup
MacOS Users
The C++ compiler comes pre-installed with Xcode Command Line Tools. Verify with:
gcc --version
If not installed, run:
xcode-select --install
Linux Users (Debian/Ubuntu)
Install the required compiler tools:
sudo apt-get update
sudo apt-get install gcc-multilib build-essential jq
3. Install Solana CLI
The Solana CLI is required for program compilation and deployment.
sh -c "$(curl -sSfL https://release.anza.xyz/v2.1.13/install)"
โ ๏ธ Important Notes:
- Add Solana to your PATH as instructed after installation
Troubleshooting Solana Installation
If you installed Rust through Homebrew and encounter cargo-build-sbf
issues:
- Remove existing Rust installation:
rustup self uninstall
- Verify removal:
rustup --version # Should show "command not found"
- Perform clean Rust installation:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
- Reinstall Solana:
sh -c "$(curl -sSfL https://release.solana.com/v1.18.18/install)"
4. Install Arch Network CLI
The Arch Network CLI provides essential development tools and a local development environment.
# Download the appropriate CLI binary for your architecture from:
# https://github.com/Arch-Network/arch-node/releases/latest
# For example, for macOS with Apple Silicon:
curl -L -o cli https://github.com/Arch-Network/arch-node/releases/latest/download/cli-aarch64-apple-darwin
chmod +x cli
sudo mv cli /usr/local/bin/
# Verify installation
cli --version
Features
The Arch Network CLI provides:
- Local validator node for development
- Program deployment and management
- Account and transaction management
- Block data and program logging
- Group key and network controls
Need Help?
- Check our Troubleshooting Guide
- Join our Discord dev-chat for community support
- Review the Arch Network CLI repo documentation
- Ensure all version requirements are met
How to configure the local validator with Bitcoin Testnet4
This guide is intended for those wishing to view logs from their programs while benefitting from being connected to Bitcoin testnet4 and therefore gaining access to ordinals/runes helper tools.
Table of Contents:
Config
First, you'll need to configure the network settings for connecting to Bitcoin testnet4. The new CLI accepts these parameters directly when starting the validator.
Note: We have redacted our Bitcoin node password to prevent abuse; contact us if you need this, otherwise provide your own node credentials and use the below as a reference.
When starting the validator, you'll need to include the following parameters:
cli validator start --network-mode testnet \
--titan-rpc-endpoint titan-node.test.aws.archnetwork.xyz \
--titan-rpc-port 49332 \
--titan-rpc-username bitcoin \
--titan-rpc-password redacted \
--titan-rpc-wallet testwallet
Local validator
Note: You can start a local validator using the Arch Network CLI tool.
Additionally, you can download pre-built binaries from the arch-node releases page.
Be sure to download the local validator binary, not the regular validator.
Run the local validator
Use the CLI command to run the local validator. You'll need to have Docker installed and running.
cli validator start --network-mode testnet
The validator logs can be viewed easily within the Docker desktop dashboard.
Note: You can also run the standalone local validator binary where the logs will be streamed to
stdout
unless otherwise redirected.
Steps for running standalone validator binary:
-
Download the appropriate binary as well as the
system_program.so
file from arch-node releases page. -
Store the
system_program.so
file within a new directory called/ebpf
.Your directory structure should resemble the following:
tmp/ โโ ebpf/ โ โโ system_program.so โโ local_validator
-
Run the binary and pass the relevant flags dependening on your target network.
RUST_LOG=info \ ./local_validator \ --network-mode testnet \ --rpc-bind-ip 127.0.0.1 \ --rpc-bind-port 9002 \ --titan-rpc-endpoint titan-node.test.aws.archnetwork.xyz \ --titan-rpc-port 49332 \ --titan-rpc-username bitcoin \ --titan-rpc-password redacted
Help commands
This section includes some helpful material when needing to restart the node state or better ensure our infrastructure is operational before proceeding.
Arch node
The below commands can be used to assist with running the Local validator.
Start fresh
By removing the /.arch_data
directory, we can wipe the state and effective start the node again from genesis (block: 0).
rm -rf .arch_data && RUST_LOG=info \
./local_validator \
--network-mode testnet \
--rpc-bind-ip 127.0.0.1 \
--rpc-bind-port 9002 \
--titan-rpc-endpoint titan-node.test.aws.archnetwork.xyz \
--titan-rpc-port 49332 \
--titan-rpc-username bitcoin \
--titan-rpc-password redacted
Pulse check
This cURL
command will allow us to ensure that our Local validator is up and running correctly. We can use this to effective get a pulse check on the node which is helpful for debugging.
curl -vL POST -H 'Content-Type: application/json' -d '
{
"jsonrpc":"2.0",
"id":1,
"method":"is_node_ready",
"params":[]
}' \
http://localhost:9002/
Log assistance
Ordinarily, the arch-node logs will flood your terminal screen (or the Docker logs). This is less than idea when needing to review them carefully, so you can also direct the stdout
to a file for later reading.
Here's an example of how to do this:
rm -rf .arch_data && RUST_LOG=info \
./local_validator \
--network-mode testnet \
--rpc-bind-ip 127.0.0.1 \
--rpc-bind-port 9002 \
--titan-rpc-endpoint titan-node.test.aws.archnetwork.xyz \
--titan-rpc-port 49332 \
--titan-rpc-username bitcoin \
--titan-rpc-password redacted \
> node-logs.txt
Then you can tail
the output and view the logs as they stream in.
tail -f node-logs.txt
Deploy + interact
Now that everything is setup correctly, we can now deploy our program and begin interacting with it. The deploy step will prove everything works correctly.
cli deploy --network-mode testnet
And if you are running the local validator binary directly from the command-line, set the --rpc-url
flag to specify your validator endpoint:
cli deploy --network-mode testnet --rpc-url http://localhost:9002
We hope this guide has been helpful, but as always, feel free to ask question within our Discord dev-chat or submit issues within out public-issues repo.
Validator Staking Guide
This guide will walk you through the process of staking ARCH tokens to become a validator on the Arch Network. As a validator, you'll be an integral part of the network's security and computation infrastructure.
Prerequisites
๐ฅ๏ธ System Requirements
Component | Minimum | Recommended |
---|---|---|
CPU | 4+ cores | 8+ cores |
RAM | 16GB | 32GB |
Storage | 100GB SSD | 500GB+ SSD |
Network | 100Mbps | 1Gbps+ |
OS | Ubuntu 20.04+ / macOS 12.0+ | Latest LTS |
๐ ARCH Tokens
Contact the Arch Network team for current staking requirements, including:
- Minimum stake amounts
- Lockup periods
- Commission rates
Validator Responsibilities
๐ Transaction Processing
- Execute programs in Arch VM
- Validate transaction signatures
- Process Bitcoin-related transactions
- Maintain transaction history
๐ค Consensus Participation
- Participate in ROAST protocol
- Contribute to threshold signing
- Coordinate transaction finality
- Verify state transitions
๐ State Management
- Track UTXO states
- Validate Bitcoin operations
- Maintain state consistency
- Verify network state
Setup & Configuration
1. Install Arch Network CLI
macOS - Apple Silicon
curl -L -o cli https://github.com/Arch-Network/arch-node/releases/latest/download/cli-aarch64-apple-darwin
chmod +x cli
sudo mv cli /usr/local/bin/
macOS - Intel
curl -L -o cli https://github.com/Arch-Network/arch-node/releases/latest/download/cli-x86_64-apple-darwin
chmod +x cli
sudo mv cli /usr/local/bin/
Linux - x86_64
curl -L -o cli https://github.com/Arch-Network/arch-node/releases/latest/download/cli-x86_64-unknown-linux-gnu
chmod +x cli
sudo mv cli /usr/local/bin/
Linux - ARM64
curl -L -o cli https://github.com/Arch-Network/arch-node/releases/latest/download/cli-aarch64-unknown-linux-gnu
chmod +x cli
sudo mv cli /usr/local/bin/
Verify installation:
cli --version
2. Configure Bitcoin Node Access
๐ก Remote Node (Recommended)
Regtest/Development:
--bitcoin-rpc-endpoint bitcoin-node.dev.aws.archnetwork.xyz \
--bitcoin-rpc-port 18443 \
--bitcoin-rpc-username bitcoin \
--bitcoin-rpc-password your_password \
--bitcoin-rpc-wallet testwallet
Testnet:
--bitcoin-rpc-endpoint bitcoin-node.test.aws.archnetwork.xyz \
--bitcoin-rpc-port 49332 \
--bitcoin-rpc-username bitcoin \
--bitcoin-rpc-password your_password \
--bitcoin-rpc-wallet testwallet
๐ฅ๏ธ Local Node
For advanced users who want full control. See our Bitcoin Node Setup Guide.
Local Regtest Configuration:
--bitcoin-rpc-endpoint 127.0.0.1 \
--bitcoin-rpc-port 18443 \
--bitcoin-rpc-username your_username \
--bitcoin-rpc-password your_password \
--bitcoin-rpc-wallet regtest
Local Testnet Configuration:
--bitcoin-rpc-endpoint 127.0.0.1 \
--bitcoin-rpc-port 18332 \
--bitcoin-rpc-username your_username \
--bitcoin-rpc-password your_password \
--bitcoin-rpc-wallet testnet
Local Mainnet Configuration:
--bitcoin-rpc-endpoint 127.0.0.1 \
--bitcoin-rpc-port 8332 \
--bitcoin-rpc-username your_username \
--bitcoin-rpc-password your_password \
--bitcoin-rpc-wallet mainnet
3. Start Your Validator
cli validator start \
--network-mode mainnet \
--titan-rpc-endpoint your_endpoint \
--titan-rpc-port your_port \
--titan-rpc-username your_username \
--titan-rpc-password your_password \
--titan-rpc-wallet your_wallet
Monitoring & Maintenance
๐ Health Checks
# Node status
cli validator status
# Performance metrics
cli validator metrics
๐ Sync Management
# Check sync status
cli validator sync-status
# Force resync if needed
cli validator resync
Understanding Staking in Arch Network
๐ What is Staking?
Staking in Arch Network is fundamentally different from traditional Proof of Stake systems. Instead of using staking for consensus, Arch Network uses staked validators to participate in the ROAST protocol for secure Bitcoin transaction signing.
flowchart TB subgraph Staking["Staking Process"] direction TB V[Validator Node] -->|1. Stakes ARCH| N[Network] N -->|2. Assigns Share| DKG[Distributed Key] DKG -->|3. Participates in| ROAST[ROAST Protocol] end subgraph Validation["Transaction Validation"] direction TB TX[Transaction] -->|1. Submitted| L[Leader] L -->|2. Distributes| VS[Validator Set] VS -->|3. Execute & Sign| R[Results] R -->|4. Aggregate| BTC[Bitcoin Network] end style Staking fill:#f3e5f5,stroke:#4a148c style Validation fill:#e8f5e9,stroke:#1b5e20
๐ค Solana vs. Arch Network: Validator Comparison
Feature | Solana | Arch Network |
---|---|---|
Consensus Role | Validators vote on blocks and produce blocks when selected as leader | Validators execute transactions and sign Bitcoin transactions using threshold signatures |
Economic Model | Block rewards + transaction fees | Transaction fees + commission from Bitcoin operations |
Selection Mechanism | Stake-weighted leader selection | Stake-weighted participation in threshold signing committee |
Performance Metrics | Vote signing speed, block production, uptime | Transaction execution correctness, signing participation, uptime |
Slashing Conditions | Double signing, unavailability | Malicious signing, transaction manipulation attempts |
Hardware Requirements | High-end CPU, 128GB+ RAM, 2TB+ NVMe | 4+ CPU cores, 16GB+ RAM, 100GB+ SSD |
๐ From Solana to Arch: Operational Transition Guide
If you're an experienced Solana validator operator, here's what you need to know about running an Arch Network validator:
โ๏ธ Technical Setup
- Lower Hardware Requirements: Arch Network requires less powerful hardware than Solana
- Bitcoin RPC Access: Validators need Bitcoin node access (remote or local)
- Key Management: Different key structure focusing on distributed key generation
- Monitoring: Focus on signing participation rather than block production
๐ฐ Economic Considerations
- Staking Return Model: Fee-based with transaction execution rewards
- Reward Distribution: Based on stake proportion and signing participation
- Commission Structure: Set during validator configuration
- Lockup Periods: Network-defined based on security requirements
๐ Operational Differences
- Signing vs. Voting: Focus on correct transaction execution and signing
- Performance Metrics: Measured by signing participation and availability
- Updates: Less frequent than Solana's rapid release cycle
- Network Bandwidth: Lower requirements due to different architecture
๐ฃ๏ธ Onboarding Process
- Registration: Complete validator registration through the network portal
- Stake Deposit: Transfer ARCH tokens to the validator staking contract
- Configuration: Set up your validator with proper Bitcoin node access
- Key Generation: Participate in distributed key generation ceremony
- Activation: Begin participation after stake activation period
๐ Staking Economics
Validator Requirements
- Minimum Stake: Contact Arch Network team for current requirements
- Lockup Period: Network-defined based on security requirements
- Uptime Requirement: High availability expected for signing participation
- Performance Bonding: Stake acts as bond for correct behavior
Reward Structure
- Base Rewards: From transaction fees distributed proportionally to stake
- Signing Rewards: Additional rewards for participating in threshold signing
- Commission: Set percentage of rewards retained by validator
- Distribution Frequency: Continuous as transactions are processed
๐ ROAST Protocol Integration
The ROAST (Robust Asynchronous Schnorr Threshold) protocol enables validators to collectively sign Bitcoin transactions:
sequenceDiagram participant C as Client participant L as Leader participant V as Validators participant B as Bitcoin Network C->>L: 1. Submit Transaction L->>V: 2. Distribute to Validators V->>V: 3. Execute in Arch VM V->>L: 4. Sign Results L->>B: 5. Submit to Bitcoin
๐ก๏ธ Security Model
flowchart LR subgraph Security["Security Layers"] direction TB UTXO[UTXO Validation] -->|Verifies| Own[Ownership] Own -->|Ensures| State[State Consistency] State -->|Commits to| BTC[Bitcoin] end subgraph Threshold["Threshold Signing"] direction TB Val[Validators] -->|t-of-n| Sign[Signature] Sign -->|ROAST| Agg[Aggregation] Agg -->|Submit| Final[Final Transaction] end style Security fill:#e1f5fe,stroke:#01579b style Threshold fill:#fff3e0,stroke:#e65100
Key Features
- Distributed key generation for secure signing
- Threshold signature scheme (t-of-n) for fault tolerance
- Bitcoin-based finality guarantees
- Automatic malicious node detection
Understanding Arch Programs
This guide analyzes a simple Hello World program to introduce the core concepts of Arch program development. We'll break down each component of the program to understand how Arch programs work.
Program Overview
The Hello World program is a simple smart contract that stores a greeting message for a given name. While simple, it demonstrates the key concepts of Arch program development including:
- Program structure and entrypoints
- Account management
- State updates
- Bitcoin transaction handling
- Fee management
Let's break down each part of the program.
1. Program Dependencies
use arch_program::{
account::AccountInfo,
bitcoin::{self, absolute::LockTime, transaction::Version, Transaction},
entrypoint,
helper::add_state_transition,
input_to_sign::InputToSign,
msg,
program::{
get_account_script_pubkey, get_bitcoin_block_height, next_account_info,
set_transaction_to_sign,
},
program_error::ProgramError,
pubkey::Pubkey,
transaction_to_sign::TransactionToSign,
};
use borsh::{BorshDeserialize, BorshSerialize};
The program starts by importing necessary dependencies:
AccountInfo
: Provides access to account data and metadatabitcoin
: Core Bitcoin types and functionality for transaction handlingentrypoint
: Macro for registering the program's entry pointmsg
: Logging functionality for debuggingborsh
: Serialization/deserialization for program data
2. Program Entry Point
entrypoint!(process_instruction);
Every Arch program needs a single entry point that the runtime will call. The entrypoint!
macro registers our process_instruction
function as this entry point. This tells the Arch runtime which function to call when our program is invoked.
3. Program Parameters
pub fn process_instruction(
_program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8],
) -> Result<(), ProgramError>
The entry point function takes three key parameters:
program_id
: The public key of our program (unused in this example)accounts
: Array of accounts the instruction will operate oninstruction_data
: Serialized instruction parameters
These parameters provide everything our program needs to execute: the context (program_id), the accounts it can read/write, and the instruction-specific data.
4. Custom Error Handling
Programs should define their own error types to provide clear and specific error messages. Here's how to implement custom errors in your Arch program:
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum HelloWorldError {
/// The provided name is too long (max 50 chars)
NameTooLong,
/// The provided name contains invalid characters
InvalidName,
/// Insufficient account balance for fees
InsufficientFees,
/// Invalid fee transaction format
InvalidFeeTransaction,
}
impl From<HelloWorldError> for ProgramError {
fn from(e: HelloWorldError) -> Self {
ProgramError::Custom(match e {
HelloWorldError::NameTooLong => 1001,
HelloWorldError::InvalidName => 1002,
HelloWorldError::InsufficientFees => 1003,
HelloWorldError::InvalidFeeTransaction => 1004,
})
}
}
Using custom errors in your program:
fn validate_name(name: &str) -> Result<(), HelloWorldError> {
if name.len() > 50 {
return Err(HelloWorldError::NameTooLong);
}
if !name.chars().all(|c| c.is_alphanumeric() || c.is_whitespace()) {
return Err(HelloWorldError::InvalidName);
}
Ok(())
}
pub fn process_instruction(
_program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8],
) -> Result<(), ProgramError> {
let params: HelloWorldParams = borsh::from_slice(instruction_data)
.map_err(|_| ProgramError::InvalidInstructionData)?;
validate_name(¶ms.name)?; // Custom error will be converted to ProgramError
// ... rest of the implementation
}
Best practices for error handling:
- Define descriptive error variants that clearly indicate what went wrong
- Use unique error codes in the 1000+ range to avoid conflicts with system errors
- Implement proper error conversion to
ProgramError
- Add documentation comments for each error variant
- Consider grouping related errors into separate enums if your program grows large
5. Program State
#[derive(Debug, Clone, BorshSerialize, BorshDeserialize)]
pub struct HelloWorldParams {
/// The name to say hello to
pub name: String,
/// Raw Bitcoin transaction for fees
pub tx_hex: Vec<u8>,
}
This structure defines the data format our program expects. It includes:
name
: The name to include in the greetingtx_hex
: A serialized Bitcoin transaction for paying fees
The BorshSerialize
and BorshDeserialize
traits allow us to convert this structure to and from bytes for storage and transmission.
6. Program Logic
Let's break down the main program logic into steps:
Account Validation
if accounts.len() != 1 {
return Err(ProgramError::Custom(501));
}
let account_iter = &mut accounts.iter();
let account = next_account_info(account_iter)?;
assert!(account.is_writable);
assert!(account.is_signer);
The program first performs several important validations:
- Checks that exactly one account is provided
- Ensures the account is writable (can be modified)
- Verifies the account is a signer (authorized to make changes)
These checks are crucial for security and proper program execution.
State Management
let params: HelloWorldParams = borsh::from_slice(instruction_data).unwrap();
let new_data = format!("Hello {}", params.name);
let data_len = account.data.try_borrow().unwrap().len();
if new_data.as_bytes().len() > data_len {
account.realloc(new_data.len(), true)?;
}
account.data.try_borrow_mut().unwrap().copy_from_slice(new_data.as_bytes());
The state management section:
- Deserializes the instruction parameters into our
HelloWorldParams
structure - Creates the greeting message
- Checks if the account has enough space for our data
- Reallocates space if needed
- Stores the greeting in the account's data
Bitcoin Transaction Handling
let fees_tx: Transaction = bitcoin::consensus::deserialize(¶ms.tx_hex).unwrap();
let mut tx = Transaction {
version: Version::TWO,
lock_time: LockTime::ZERO,
input: vec![],
output: vec![],
};
add_state_transition(&mut tx, account);
tx.input.push(fees_tx.input[0].clone());
This section handles the Bitcoin transaction aspects:
- Deserializes the fee transaction provided by the user
- Creates a new Bitcoin transaction for our state update
- Adds the state transition to the transaction
- Includes the fee input to pay for the transaction
Transaction Signing
let tx_to_sign = TransactionToSign {
tx_bytes: &bitcoin::consensus::serialize(&tx),
inputs_to_sign: &[InputToSign {
index: 0,
signer: account.key.clone(),
}],
};
set_transaction_to_sign(accounts, tx_to_sign)
Finally, the program prepares the transaction for signing:
- Serializes the Bitcoin transaction
- Creates a signing request specifying which inputs need to be signed
- Submits the transaction to be signed by the Arch runtime
Next Steps
Now that you understand the basic structure of an Arch program, you can:
- Learn about more complex account management in the Program documentation
- Explore cross-program invocation for program composition
- Study the Bitcoin transaction lifecycle in our docs
- Build more complex programs using these fundamentals
For practical examples, check out our other guides:
Writing Your First Arch Program
This guide will walk you through creating your first Arch program from scratch. We'll build a simple counter program that demonstrates the core concepts of Arch development while providing hands-on experience with the development workflow.
Prerequisites
Before starting, ensure you have:
- Rust and Cargo installed
- CLI installed and configured (see Quick Start Guide)
- A running validator node
- Basic understanding of Arch concepts
Project Setup
- Create a new project directory:
mkdir my-counter-program
cd my-counter-program
- Initialize a new Rust project:
cargo init --lib
- Add necessary dependencies to
Cargo.toml
:
[package]
name = "my-counter-program"
version = "0.1.0"
edition = "2021"
[dependencies]
arch-program = { git = "https://github.com/Arch-Network/arch-program" }
borsh = "0.10.3"
[lib]
crate-type = ["cdylib"]
Writing the Program
Let's create a simple program that:
- Stores a counter in an account
- Can increment the counter
- Can decrement the counter
- Can reset the counter
- Define our program's state structure in
src/lib.rs
:
#![allow(unused)] fn main() { use arch_program::{ account::AccountInfo, entrypoint, msg, program_error::ProgramError, pubkey::Pubkey, }; use borsh::{BorshDeserialize, BorshSerialize}; #[derive(BorshSerialize, BorshDeserialize, Debug)] pub struct CounterAccount { pub count: u64, } #[derive(BorshSerialize, BorshDeserialize, Debug)] pub enum CounterInstruction { Increment, Decrement, Reset, } }
- Implement the program logic:
#![allow(unused)] fn main() { entrypoint!(process_instruction); pub fn process_instruction( _program_id: &Pubkey, accounts: &[AccountInfo], instruction_data: &[u8], ) -> Result<(), ProgramError> { // Get the account to store counter data let account_iter = &mut accounts.iter(); let counter_account = next_account_info(account_iter)?; // Verify account is writable if !counter_account.is_writable { return Err(ProgramError::InvalidAccountData); } // Deserialize the instruction let instruction = CounterInstruction::try_from_slice(instruction_data)?; // Get current counter value let mut counter = match CounterAccount::try_from_slice(&counter_account.data.borrow()) { Ok(data) => data, Err(_) => CounterAccount { count: 0 }, }; // Process the instruction match instruction { CounterInstruction::Increment => { counter.count = counter.count.checked_add(1).ok_or(ProgramError::Custom(1))?; msg!("Counter incremented to {}", counter.count); } CounterInstruction::Decrement => { counter.count = counter.count.checked_sub(1).ok_or(ProgramError::Custom(2))?; msg!("Counter decremented to {}", counter.count); } CounterInstruction::Reset => { counter.count = 0; msg!("Counter reset to 0"); } } // Serialize and save the counter counter.serialize(&mut &mut counter_account.data.borrow_mut()[..])?; Ok(()) } }
Building the Program
- Build your program:
cargo build-sbf
This will create a compiled program at target/deploy/my_counter_program.so
Deploying the Program
- Deploy your program using the CLI:
cli deploy target/deploy/my_counter_program.so
Save the Program ID output - you'll need it to interact with your program.
Creating a Counter Account
Before we can use our counter, we need to create an account to store its state:
cli create-account <PROGRAM_ID> 8
Save the account address - you'll need it to interact with your counter.
Testing Your Program
You can now interact with your program using the CLI:
- Increment the counter:
cli invoke <PROGRAM_ID> <ACCOUNT_ADDRESS> --data 00
- Decrement the counter:
cli invoke <PROGRAM_ID> <ACCOUNT_ADDRESS> --data 01
- Reset the counter:
cli invoke <PROGRAM_ID> <ACCOUNT_ADDRESS> --data 02
Next Steps
Now that you've created your first program, you can:
- Add more features to the counter program
- Learn about cross-program invocation
- Explore more complex Program Examples
- Study the Understanding Arch Programs guide for deeper insights
Program Examples
In this section, we'll provide a few guides that can step through constructing an Arch program, as well as deploying and interacting with your program.
Table of Contents:
How to create a fungible token
This guide walks through how to implement the Fungible Token Standard program, part of the Arch Program Library, or APL.
Table of Contents:
Description
The Fungible Token Standard program provides a consistent interface for implementing fungible tokens on Arch. As with all programs within the APL, this program is predeployed and is tested against the Arch runtime.
The source code can be found within the arch-examples repo.
Logic
If you haven't already read How to write an Arch program, we recommend starting there to get a basic understanding of the program anatomy before going further.
Implementation
Deploy
Although the Fungible Token Standard program is part of the APL, and is there predeployed by the validators, for local testing, we can deploy it ourselves. Move to Mint if you'd like to skip this step.
To demonstrate a deploy, we'll reference: deploy.rs
We make use of try_deploy_program
, a helper function from the ebpf-counter example to deploy our program.
pub const ELF_PATH: &str = "./program/target/sbf-solana-solana/release/fungible-token-standard-program.so";
fn deploy_standard_program() {
let program_pubkey =
try_deploy_program(ELF_PATH, PROGRAM_FILE_PATH, "Fungible-Token-Standard").unwrap();
println!(
"Deployed Fungible token standard program account id {:?}!",
program_pubkey.serialize()
);
...
}
Mint
To mint tokens, we must supply a few pieces of information:
- Owner
- Supply
- Ticker
- Decimals
This data gets stored in the InitializeMintInput
struct, which will be used to generate a new instance of the Fungible Token Standard.
#[derive(Debug, Clone, BorshSerialize, BorshDeserialize)]
pub struct InitializeMintInput {
owner: [u8; 32],
supply: u64, // in lowest denomination
ticker: String,
decimals: u8,
}
To demonstrate a mint, we'll reference: tests_mint.rs
We initialize a new instance of InitializeMintInput
and pass in the necessary data. In the below case, our owner account will create the token "SPONK," with a total supply of 1,000,000, which will have only a single decimal, meaning it is divisible by 1.
// deploy.rs
let mint_input = InitializeMintInput::new(
mint_account_pubkey.serialize(),
1000000,
"SPONK".to_string(),
1,
);
We then serialize mint_input
so that we can pass it as instruction_data
within an Instruction which then gets submitted to the deployed Fungible Token Standard program.
let mut instruction_data = vec![0u8];
mint_input
.serialize(&mut instruction_data)
.expect("Couldnt serialize mint input");
let initialize_mint_instruction = Instruction {
program_id: program_pubkey.clone(),
accounts: vec![AccountMeta {
pubkey: mint_account_pubkey,
is_signer: true,
is_writable: true,
}],
data: instruction_data,
};
Next, we build a transaction using build_transaction
and then submit the transaction with build_and_send_block
, both helper function from the ebpf-counter example.
let transaction = build_transaction(
vec![mint_account_keypair],
vec![initialize_mint_instruction],
);
let block_transactions = build_and_send_block(vec![transaction]);
We fetch the result of the transaction with fetch_processed_transactions
helper function (ebpf-counter) and then obtain the mint details by passing the Pubkey of the token owner.
let processed_transactions = fetch_processed_transactions(block_transactions).unwrap();
assert!(matches!(
processed_transactions[0].status,
Status::Processed
));
let mint_details = get_mint_info(&mint_account_pubkey).expect("Couldnt deserialize mint info");
println!("Mint account {:?}", mint_account_pubkey.serialize());
Transfer
To demonstrate a transfer, we'll reference: tests_transfer.rs
We obtain a mint_account_pubkey
, made possible by using the try_create_mint_account
helper function. We pass true
as this is a one-time mint event and this will generate a new keypair and Pubkey.
This step will actually create a new token with the following details:
- Supply: 1,000,000
- Ticker: "ARCH"
- Decimals: 2
- Mint Price: 1000 sats
let mint_account_pubkey = try_create_mint_account(true).unwrap();
We then fetch the token mint details with get_mint_info
.
let previous_mint_details = get_mint_info(&mint_account_pubkey).unwrap();
Now, let's provision our two accounts: the sender and the receiver.
// sending account
let (first_account_owner_key_pair, first_account_owner_pubkey, _first_account_owner_address) =
generate_new_keypair();
let first_balance_account_pubkey = create_balance_account(
&first_account_owner_pubkey,
first_account_owner_key_pair,
&mint_account_pubkey,
&program_pubkey,
)
.unwrap();
// receiving account
let (second_account_owner_key_pair, second_account_owner_pubkey, _second_account_owner_address) =
generate_new_keypair();
let second_balance_account_pubkey = create_balance_account(
&second_account_owner_pubkey,
second_account_owner_key_pair,
&mint_account_pubkey,
&program_pubkey,
)
.unwrap();
We then procure funds for the sending account. In this case, we'll mint 10 tokens.
let mint_amount = 10u64;
let mint_instruction = mint_request_instruction(
&mint_account_pubkey,
&program_pubkey,
&first_balance_account_pubkey,
&first_account_owner_pubkey,
mint_amount,
)
.unwrap();
We utilize the transfer_request_instruction
helper function to generate a transfer Instruction.
let transfer_instruction = transfer_request_instruction(
&mint_account_pubkey,
&program_pubkey,
&first_balance_account_pubkey,
&first_account_owner_pubkey,
&second_balance_account_pubkey,
mint_amount,
)
.unwrap();
We build the transaction by passing in the newly created transfer Instruction as well as the keypair of the sending account, necessary for authorizing the fund transfer.
let transfer_transaction = build_transaction(
vec![first_account_owner_key_pair],
vec![transfer_instruction],
);
Next, we then submit the transaction with build_and_send_block
and then fetch the processed transaction to get the result.
let block_transactions = build_and_send_block(vec![transfer_transaction]);
let processed_transactions = fetch_processed_transactions(block_transactions).unwrap();
assert!(matches!(
processed_transactions[0].status,
Status::Processed
));
Balance check
In order to check the token balance of an account, we'll make use of the get_balance_account
function and pass in the account we are looking to query the balance of; in the below example, we'll fetch the balances of both the sending and receiving accounts.
let resulting_sender_balance = get_balance_account(&first_balance_account_pubkey).unwrap();
let resulting_receiver_balance = get_balance_account(&second_balance_account_pubkey).unwrap();
assert_eq!(resulting_receiver_balance.current_balance, mint_amount);
assert_eq!(resulting_sender_balance.current_balance, 0);
How to write an oracle program
This guide walks through the innerworkings of an oracle program as well as details how oracle data can be utilized by other programs on Arch Network.
Table of Contents:
Description
Two important aspects of understanding how this oracle example is implemented within Arch:
- The oracle is a program that updates an account which holds the data
- No cross-program invocation occurs since only the account is updated and read from versus this being another program that gets interacted with from another program
The source code can be found within the arch-examples repo.
Flow
- Project deploys oracle program
- Project creates state account that the oracle program will control in order to write state to it
- Projects submit data to the oracle state account by submitting instructions to the oracle program
- Programs include oracle state account alongside their program instructions in order to use this referenced data stored in the oracle state account within their program
- Projects submit instructions to oracle program periodically to update oracle state account with fresh data
Logic
If you haven't already read How to write an Arch program, we recommend starting there to get a basic understanding of the program anatomy before going further.
We'll look closely at the logic block contained within the update_data
handler.
pub fn update_data(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8],
) -> Result<(), ProgramError> {
let account_iter = &mut accounts.iter();
let oracle_account = next_account_info(account_iter)?;
assert!(oracle_account.is_signer);
assert_eq!(instruction_data.len(), 8);
...
}
First, we'll iterate over the accounts that get passed into the function, which includes the newly created state account that will be responsible for managing the oracle's data.
We then assert that the oracle state account has the appropriate authority to be written to and update what it stores within its data field. Additionally, we assert that the data we wish to update the account with is at least a certain number of bytes.
let data_len = oracle_account.data.try_borrow().unwrap().len();
if instruction_data.len() > data_len {
oracle_account.realloc(instruction_data.len(), true)?;
}
Next, we calculate the length of the new data that we are looking to store in the account and reallocate memory to the account if the new data is larger than the data currently existing within the account. This step is important for ensuring that there is no remaining, stale data stored in the account before adding new data to it.
oracle_account
.data
.try_borrow_mut()
.unwrap()
.copy_from_slice(instruction_data);
msg!("updated");
Ok(())
Lastly, we store the new data that is passed into the program via the instruction to the state account for management, thus marking the end of the oracle update process.
Implementation
Let's look at an example implementation of this oracle program. This includes:
- Create oracle project
- Deploy program
- Create a state account
- Update the state account
- Read from the state account
Create oracle project
First, we'll need to create a new project to hold our oracle logic.
# Create a new directory for your oracle project
mkdir oracle
cd oracle
# Initialize a Rust project
cargo init --lib
Note: The new CLI does not currently have a project creation command. We'll manually set up our project structure.
You'll need to create and edit the following files:
Cargo.toml
- Add dependencies for your oracle programsrc/lib.rs
- Implement the oracle program logic
Example program files can be found in the arch-examples repo.
Deploy program
After the project is created, the program is written and the Cargo.toml
is set with the proper dependencies, we can deploy the program.
# Build the program
cargo build-sbf
# Deploy the program
cli deploy target/deploy/oracle.so
During the deployment, a new account is created for the deployed program logic and set to be executable, marking it as a Program rather than a data Account.
Create state account
From the deployment output, you should obtain the program_id
. We can use this program_id
to create a state account that is owned and updated by the program.
The oracle state account can then be read from by any program in order to retrieve the associated oracle data.
# The new CLI may not have direct account creation functionality
# You'll need to use an RPC call to create the account
# For example, using curl:
curl -X POST http://localhost:9002 \
-H "Content-Type: application/json" \
-d '{
"jsonrpc":"2.0",
"id":1,
"method":"sendTransaction",
"params":[{
"signature":"your_signature",
"message":{
"accountKeys":["your_pubkey", "your_program_id"],
"instructions":[{
"programId":"system_program_id",
"accounts":["your_pubkey", "new_account_pubkey"],
"data":"encoded_create_account_data"
}]
}
}]
}'
Note: The above is a simplified example. You'll need to properly construct, sign, and encode your transaction according to the Arch Network protocol.
In this step, the account is created and ownership is transferred to the program. This allows the program to update the account's data field which holds state for the program.
Update the state account
Now that we have created an account and the oracle program has authority to update it, we now want to update the data that the account holds.
In order to update the data stored in the account, we simply need to make a transaction that includes the data that we wish to update the oracle state account to hold, and submit this within the context of an instruction.
As an example, below we have a sample rust program that we'll use to fetch the Bitcoin fees from the mempool.space API and store this fee data in our oracle state account that was created during deployment.
Note: The below is a rust program and is not an Arch program.
The call to update the oracle state account can be written in any programming language as it is simply an RPC call. For sake of continuity, we're using rust along with methods from both the
program
andsdk
crates.
use bitcoincore_rpc::{Auth, Client};
let mut old_feerate = 0;
let body: Value = reqwest::blocking::get("https://mempool.space/api/v1/fees/recommended").unwrap().json().unwrap();
let feerate = body.get("fastestFee").unwrap().as_u64().unwrap();
if old_feerate != feerate {
let (txid, instruction_hash) = sign_and_send_instruction(
Instruction {
program_id: program_pubkey.clone(),
accounts: vec![AccountMeta {
pubkey: caller_pubkey.clone(),
is_signer: true,
is_writable: true
}],
data: feerate.to_le_bytes().to_vec()
},
vec![caller_keypair],
).expect("signing and sending a transaction should not fail");
let processed_tx = get_processed_transaction(NODE1_ADDRESS, txid.clone()).expect("get processed transaction should not fail");
println!("processed_tx {:?}", processed_tx);
println!("{:?}", read_account_info(NODE1_ADDRESS, caller_pubkey.clone()));
old_feerate = feerate;
}
Read from the state account
Below is an example of a different program (we'll call this app-program) that would like to access the oracle data.
Essentially, what happens here is that when we pass an instruction into our app-program, we must also include the oracle state account alongside any other account that we need for the app-program. In this way, the oracle state account is now in-scope and its data can be read from.
Building Your First Bitcoin Runes Swap Application
Welcome to this hands-on tutorial! Today, we're going to build a decentralized application that enables users to swap Bitcoin Runes tokens on the Arch Network. By the end of this lesson, you'll understand how to create a secure, trustless swap mechanism for Runes tokens.
Class Prerequisites
Before we dive in, please ensure you have:
- Completed the environment setup
- A basic understanding of Bitcoin Integration
- Familiarity with Rust programming language
- Your development environment ready with the Arch Network CLI installed
Lesson 1: Understanding the Basics
What are Runes?
Before we write any code, let's understand what we're working with. Runes is a Bitcoin protocol for fungible tokens, similar to how BRC-20 works. Each Rune token has a unique identifier and can be transferred between Bitcoin addresses.
What are we building?
We're creating a swap program that will:
- Allow users to create swap offers ("I want to trade X amount of Rune A for Y amount of Rune B")
- Enable other users to accept these offers
- Let users cancel their offers if they change their mind
- Ensure all swaps are atomic (they either complete fully or not at all)
Lesson 2: Setting Up Our Project
Let's start by creating our project structure. Open your terminal and run:
# Create a new directory for your project
mkdir runes-swap
cd runes-swap
# Initialize a new Rust project
cargo init --lib
# Your project structure should look like this:
# runes-swap/
# โโโ Cargo.toml
# โโโ src/
# โ โโโ lib.rs
Lesson 3: Defining Our Data Structures
Now, let's define the building blocks of our swap program. In programming, it's crucial to plan our data structures before implementing functionality.
use arch_program::{
account::AccountInfo,
entrypoint,
msg,
program_error::ProgramError,
pubkey::Pubkey,
utxo::UtxoMeta,
borsh::{BorshDeserialize, BorshSerialize},
};
/// This structure represents a single swap offer in our system
#[derive(BorshSerialize, BorshDeserialize, Debug)]
pub struct SwapOffer {
// Unique identifier for the offer
pub offer_id: u64,
// The public key of the person creating the offer
pub maker: Pubkey,
// The Rune ID they want to give
pub rune_id_give: String,
// Amount of Runes they want to give
pub amount_give: u64,
// The Rune ID they want to receive
pub rune_id_want: String,
// Amount of Runes they want to receive
pub amount_want: u64,
// When this offer expires (in block height)
pub expiry: u64,
// Current status of the offer
pub status: OfferStatus,
}
Let's break down why we chose each field:
offer_id
: Every offer needs a unique identifier so we can reference it latermaker
: We store who created the offer to ensure only they can cancel itrune_id_give/want
: These identify which Runes are being swappedamount_give/want
: The quantities of each Rune in the swapexpiry
: Offers shouldn't live forever, so we add an expiration
Lesson 4: Implementing the Swap Logic
Now that we understand our data structures, let's implement the core swap functionality. We'll start with creating an offer:
fn process_create_offer(
accounts: &[AccountInfo],
instruction: SwapInstruction,
) -> Result<(), ProgramError> {
// Step 1: Get all the accounts we need
let account_iter = &mut accounts.iter();
let maker = next_account_info(account_iter)?;
let offer_account = next_account_info(account_iter)?;
// Step 2: Verify the maker has the Runes they want to swap
if let SwapInstruction::CreateOffer {
rune_id_give,
amount_give,
rune_id_want,
amount_want,
expiry
} = instruction {
// Security check: Ensure the maker owns enough Runes
verify_rune_ownership(maker, &rune_id_give, amount_give)?;
// Step 3: Create and store the offer
let offer = SwapOffer {
offer_id: get_next_offer_id(offer_account)?,
maker: *maker.key,
rune_id_give,
amount_give,
rune_id_want,
amount_want,
expiry,
status: OfferStatus::Active,
};
store_offer(offer_account, &offer)?;
}
Ok(())
}
Understanding the Create Offer Process
- First, we extract the accounts passed to our program
- We verify that the maker actually owns the Runes they want to trade
- We create a new
SwapOffer
with an Active status - Finally, we store this offer in the program's state
Lesson 5: Testing Our Program
Testing is crucial in blockchain development because once deployed, your program can't be easily changed. Let's write comprehensive tests for our swap program.
#[cfg(test)]
mod tests {
use super::*;
use arch_program::test_utils::{create_test_account, create_test_pubkey};
/// Helper function to create a test offer
fn create_test_offer() -> SwapOffer {
SwapOffer {
offer_id: 1,
maker: create_test_pubkey(),
rune_id_give: "RUNE1".to_string(),
amount_give: 100,
rune_id_want: "RUNE2".to_string(),
amount_want: 200,
expiry: 1000,
status: OfferStatus::Active,
}
}
#[test]
fn test_create_offer() {
// Arrange: Set up our test accounts
let maker = create_test_account();
let offer_account = create_test_account();
// Act: Create an offer
let result = process_create_offer(
&[maker.clone(), offer_account.clone()],
SwapInstruction::CreateOffer {
rune_id_give: "RUNE1".to_string(),
amount_give: 100,
rune_id_want: "RUNE2".to_string(),
amount_want: 200,
expiry: 1000,
},
);
// Assert: Check the result
assert!(result.is_ok());
// Add more assertions here to verify the offer was stored correctly
}
}
Understanding Our Test Structure
We follow the "Arrange-Act-Assert" pattern:
- Arrange: Set up the test environment and data
- Act: Execute the functionality we're testing
- Assert: Verify the results match our expectations
Lesson 6: Implementing Offer Acceptance
Now let's implement the logic for accepting an offer. This is where atomic swaps become crucial:
fn process_accept_offer(
accounts: &[AccountInfo],
instruction: SwapInstruction,
) -> Result<(), ProgramError> {
// Step 1: Get all required accounts
let account_iter = &mut accounts.iter();
let taker = next_account_info(account_iter)?;
let maker = next_account_info(account_iter)?;
let offer_account = next_account_info(account_iter)?;
if let SwapInstruction::AcceptOffer { offer_id } = instruction {
// Step 2: Load and validate the offer
let mut offer = load_offer(offer_account)?;
require!(
offer.status == OfferStatus::Active,
ProgramError::InvalidAccountData
);
require!(
offer.offer_id == offer_id,
ProgramError::InvalidArgument
);
// Step 3: Verify the taker has the required Runes
verify_rune_ownership(taker, &offer.rune_id_want, offer.amount_want)?;
// Step 4: Perform the atomic swap
// Transfer Runes from maker to taker
transfer_runes(
maker,
taker,
&offer.rune_id_give,
offer.amount_give,
)?;
// Transfer Runes from taker to maker
transfer_runes(
taker,
maker,
&offer.rune_id_want,
offer.amount_want,
)?;
// Step 5: Update offer status
offer.status = OfferStatus::Completed;
store_offer(offer_account, &offer)?;
}
Ok(())
}
Understanding Atomic Swaps
An atomic swap ensures that either:
- Both transfers complete successfully, or
- Neither transfer happens at all
This is crucial for preventing partial swaps where one party could lose their tokens.
Lesson 7: Implementing Offer Cancellation
Finally, let's implement the ability to cancel offers:
fn process_cancel_offer(
accounts: &[AccountInfo],
instruction: SwapInstruction,
) -> Result<(), ProgramError> {
let account_iter = &mut accounts.iter();
let maker = next_account_info(account_iter)?;
let offer_account = next_account_info(account_iter)?;
if let SwapInstruction::CancelOffer { offer_id } = instruction {
// Load the offer
let mut offer = load_offer(offer_account)?;
// Security checks
require!(
offer.maker == *maker.key,
ProgramError::InvalidAccountData
);
require!(
offer.status == OfferStatus::Active,
ProgramError::InvalidAccountData
);
require!(
offer.offer_id == offer_id,
ProgramError::InvalidArgument
);
// Update offer status
offer.status = OfferStatus::Cancelled;
store_offer(offer_account, &offer)?;
}
Ok(())
}
Deploying Your Runes Swap Program
After you've written and tested your program, it's time to deploy it to the Arch Network:
# Build the program
cargo build-sbf
# Deploy the program to the Arch Network
cli deploy target/deploy/runes_swap.so
Make sure you have a validator node running before deployment:
# Start a local validator
cli validator start
Conclusion
Congratulations! You've built a complete Runes swap program. This program demonstrates several important blockchain concepts:
- Atomic transactions
- State management
- Security checks
- Program testing
Remember to always:
- Test thoroughly before deployment
- Consider edge cases
- Implement proper error handling
- Add detailed documentation
Next Steps
To further improve your program, consider adding:
- A UI for interacting with the swap program
- More sophisticated offer matching
- Order book functionality
- Price oracle integration
- Additional security features
Questions? Feel free to ask in the comments below!
How to Build a Bitcoin Lending Protocol
This guide walks through building a lending protocol for Bitcoin-based assets (BTC, Runes, Ordinals) on Arch Network. We'll create a decentralized lending platform similar to Aave, but specifically designed for Bitcoin-based assets.
Prerequisites
Before starting, ensure you have:
- Completed the environment setup
- A basic understanding of Bitcoin Integration
- Familiarity with Rust programming language
- Your development environment ready with the Arch CLI installed
System Overview
Basic User Flow
flowchart TD subgraph Depositing A[User wants to lend] -->|1. Deposits BTC| B[Lending Pool] B -->|2. Receives interest| A end subgraph Borrowing C[User needs loan] -->|3. Provides collateral| B B -->|4. Lends BTC| C C -->|5. Repays loan + interest| B end style A fill:#b3e0ff style B fill:#98FB98 style C fill:#b3e0ff
Safety System
flowchart LR subgraph "Price Monitoring" direction TB A[Price Oracle] -->|1. Updates prices| B[Health Checker] end subgraph "Health Check" direction TB B -->|2. Monitors positions| C[User Position] C -->|3. If position unsafe| D[Liquidator] end style A fill:#FFB6C1 style B fill:#FFB6C1 style C fill:#b3e0ff style D fill:#FFB6C1
Simple Example
Let's say Alice wants to borrow BTC and Bob wants to earn interest:
-
Bob (Lender)
- Deposits 1 BTC into pool
- Earns 3% APY interest
-
Alice (Borrower)
- Provides 1.5 BTC as collateral
- Borrows 1 BTC
- Pays 5% APY interest
-
Safety System
- Monitors BTC price
- Checks if Alice's collateral stays valuable enough
- If BTC price drops too much, liquidates some collateral to protect Bob's deposit
Architecture Overview
Our lending protocol consists of several key components:
1. Pool Accounts
Pool accounts are the core of our lending protocol. They serve as liquidity pools where users can:
- Deposit Bitcoin-based assets (BTC, Runes, Ordinals)
- Earn interest on deposits
- Borrow against their collateral
- Manage protocol parameters
Each pool account maintains:
- Total deposits and borrows
- Interest rates and utilization metrics
- Collateral factors and liquidation thresholds
- Asset-specific parameters
The pool account manages both state and UTXOs:
- State Management: Tracks deposits, withdrawals, and user positions
- UTXO Management:
- Maintains a collection of UTXOs for the pool's Bitcoin holdings
- Manages UTXO creation for withdrawals
- Handles UTXO consolidation for efficient liquidity management
2. Price Oracle
Track asset prices for liquidation calculations
3. User Positions
User positions track all user interactions with the lending pools:
- Active deposits and their earned interest
- Outstanding borrows and accrued interest
- Collateral positions and health factors
- Liquidation thresholds and warnings
Each user can have multiple positions across different pools, and the protocol tracks:
- Position health through real-time monitoring
- Collateralization ratios
- Interest accrual
- Liquidation risks
Core Data Structures
#[derive(BorshSerialize, BorshDeserialize)]
pub struct LendingPool {
pub pool_pubkey: Pubkey,
pub asset_type: AssetType, // BTC, Runes, Ordinals
pub total_deposits: u64,
pub total_borrows: u64,
pub interest_rate: u64,
pub utilization_rate: u64,
pub liquidation_threshold: u64,
pub collateral_factor: u64,
pub utxos: Vec<UtxoMeta>,
pub validator_signatures: Vec<Signature>,
pub min_signatures_required: u32,
}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct UserPosition {
pub user_pubkey: Pubkey,
pub pool_pubkey: Pubkey,
pub deposited_amount: u64,
pub borrowed_amount: u64,
pub collateral_amount: u64,
pub last_update: i64,
}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct InterestRateModel {
pub base_rate: u64,
pub multiplier: u64,
pub jump_multiplier: u64,
pub optimal_utilization: u64,
}
// Additional helper structures for managing positions
#[derive(BorshSerialize, BorshDeserialize)]
pub struct PositionHealth {
pub health_factor: u64,
pub liquidation_price: u64,
pub safe_borrow_limit: u64,
}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct PoolMetrics {
pub total_value_locked: u64,
pub available_liquidity: u64,
pub utilization_rate: u64,
pub supply_apy: u64,
pub borrow_apy: u64,
}
Custom Scoring and Risk Management
LTV (Loan-to-Value) Scoring System
flowchart TD subgraph Core_Factors[Core Factors] P1[Transaction History] P2[Asset Quality] P3[Market Volatility] P4[Position Size] end subgraph User_Metrics[User Metrics] P5[Account History] P6[Repayment Record] P7[Portfolio Health] end subgraph Market_Context[Market Context] P8[Market Conditions] P9[Price Impact] P10[Network Status] end Core_Factors --> SC[Scoring Engine] User_Metrics --> SC Market_Context --> SC SC --> WF[Weight Calculation] WF --> NM[Risk Normalization] NM --> LTV[Final LTV Ratio] style Core_Factors fill:#e1f3d8 style User_Metrics fill:#fff7e6 style Market_Context fill:#e6f3ff style SC fill:#f9f9f9 style WF fill:#f9f9f9 style NM fill:#f9f9f9 style LTV fill:#d4edda
Health Score Monitoring
sequenceDiagram participant User participant HealthMonitor participant PriceOracle participant LiquidationEngine participant Market loop Every Block PriceOracle->>HealthMonitor: Update Asset Prices HealthMonitor->>HealthMonitor: Calculate Health Score alt Health Score < Threshold HealthMonitor->>LiquidationEngine: Trigger Liquidation LiquidationEngine->>User: Lock Account LiquidationEngine->>Market: List Assets Market-->>LiquidationEngine: Asset Sale Complete LiquidationEngine->>User: Update Position else Health Score >= Threshold HealthMonitor->>User: Position Safe end end
Liquidation Process
stateDiagram-v2 [*] --> Monitoring Monitoring --> Warning: Health Score Declining Warning --> AtRisk: Below Warning Threshold AtRisk --> Liquidation: Below Critical Threshold Liquidation --> Step1: Lock Account Step1 --> Step2: List Assets Step2 --> Recovery: Asset Sale Recovery --> [*]: Position Cleared Warning --> Monitoring: Health Restored AtRisk --> Warning: Health Improved
Custom Scoring Implementation
#[derive(BorshSerialize, BorshDeserialize)]
pub struct UserScore {
pub historical_data_score: u64,
pub asset_quality_score: u64,
pub market_volatility_score: u64,
pub position_size_score: u64,
pub account_age_score: u64,
pub liquidation_history_score: u64,
pub repayment_history_score: u64,
pub cross_margin_score: u64,
pub portfolio_diversity_score: u64,
pub market_condition_score: u64,
pub collateral_quality_score: u64,
pub platform_activity_score: u64,
pub time_weighted_score: u64,
pub price_impact_score: u64,
pub network_status_score: u64,
}
pub fn calculate_ltv_ratio(score: &UserScore) -> Result<u64> {
// Weighted calculation of LTV based on all scoring parameters
let weighted_score = calculate_weighted_score(score)?;
let normalized_score = normalize_score(weighted_score)?;
// Convert normalized score to LTV ratio
let ltv_ratio = convert_score_to_ltv(normalized_score)?;
// Apply market condition adjustments
let adjusted_ltv = apply_market_adjustments(ltv_ratio)?;
Ok(adjusted_ltv)
}
pub fn monitor_health_score(
ctx: Context<HealthCheck>,
position: &UserPosition,
score: &UserScore,
) -> Result<()> {
let health_score = calculate_health_score(position, score)?;
if health_score < CRITICAL_THRESHOLD {
trigger_full_liquidation(ctx, position)?;
lock_account(ctx.accounts.user_account)?;
} else if health_score < WARNING_THRESHOLD {
emit_warning(ctx.accounts.user_account)?;
}
Ok(())
}
pub fn trigger_full_liquidation(
ctx: Context<Liquidation>,
position: &UserPosition,
) -> Result<()> {
// Step 1: Lock the account
lock_account(ctx.accounts.user_account)?;
// Step 2: Calculate current position value
let position_value = calculate_position_value(position)?;
// Step 3: List assets on marketplace
list_assets_for_liquidation(
ctx.accounts.marketplace,
position.assets,
position_value,
)?;
// Step 4: Monitor recovery process
start_recovery_monitoring(ctx.accounts.recovery_manager)?;
Ok(())
}
## Health Score Calculation
The health score is calculated using a combination of factors:
```rust,ignore
pub fn calculate_health_score(
position: &UserPosition,
score: &UserScore,
) -> Result<u64> {
// 1. Calculate base health ratio
let base_health = calculate_base_health_ratio(
position.collateral_value,
position.borrowed_value,
)?;
// 2. Apply user score modifiers
let score_adjusted_health = apply_score_modifiers(
base_health,
score,
)?;
// 3. Apply market condition adjustments
let market_adjusted_health = apply_market_conditions(
score_adjusted_health,
&position.asset_type,
)?;
// 4. Apply time-weighted factors
let final_health_score = apply_time_weights(
market_adjusted_health,
position.last_update,
)?;
Ok(final_health_score)
}
Liquidation Implementation
The two-step liquidation process is implemented as follows:
pub struct LiquidationConfig {
pub warning_threshold: u64,
pub critical_threshold: u64,
pub recovery_timeout: i64,
pub minimum_recovery_value: u64,
}
pub fn handle_liquidation(
ctx: Context<Liquidation>,
config: &LiquidationConfig,
) -> Result<()> {
// Step 1: Asset Recovery
let recovery_listing = create_recovery_listing(
ctx.accounts.marketplace,
ctx.accounts.user_position,
config.minimum_recovery_value,
)?;
// Step 2: Monitor Recovery
start_recovery_monitoring(
recovery_listing,
config.recovery_timeout,
)?;
// Lock account until recovery complete
lock_user_account(ctx.accounts.user_account)?;
Ok(())
}
Implementation Steps
1. Initialize Lending Pool
First, we'll create a function to initialize a new lending pool:
pub fn initialize_lending_pool(
ctx: Context<InitializeLendingPool>,
asset_type: AssetType,
initial_interest_rate: u64,
liquidation_threshold: u64,
collateral_factor: u64,
) -> Result<()> {
let lending_pool = &mut ctx.accounts.lending_pool;
lending_pool.pool_pubkey = ctx.accounts.pool.key();
lending_pool.asset_type = asset_type;
lending_pool.total_deposits = 0;
lending_pool.total_borrows = 0;
lending_pool.interest_rate = initial_interest_rate;
lending_pool.utilization_rate = 0;
lending_pool.liquidation_threshold = liquidation_threshold;
lending_pool.collateral_factor = collateral_factor;
Ok(())
}
// Initialize pool metrics
pub fn initialize_pool_metrics(
ctx: Context<InitializePoolMetrics>,
) -> Result<()> {
let pool_metrics = &mut ctx.accounts.pool_metrics;
pool_metrics.total_value_locked = 0;
pool_metrics.available_liquidity = 0;
pool_metrics.utilization_rate = 0;
pool_metrics.supply_apy = 0;
pool_metrics.borrow_apy = 0;
Ok(())
}
2. Manage User Positions
Functions to handle user position management:
pub fn create_user_position(
ctx: Context<CreateUserPosition>,
pool_pubkey: Pubkey,
) -> Result<()> {
let user_position = &mut ctx.accounts.user_position;
user_position.user_pubkey = ctx.accounts.user.key();
user_position.pool_pubkey = pool_pubkey;
user_position.deposited_amount = 0;
user_position.borrowed_amount = 0;
user_position.collateral_amount = 0;
user_position.last_update = Clock::get()?.unix_timestamp;
Ok(())
}
pub fn update_position_health(
ctx: Context<UpdatePositionHealth>,
) -> Result<()> {
let position = &ctx.accounts.user_position;
let pool = &ctx.accounts.lending_pool;
let health = &mut ctx.accounts.position_health;
// Calculate health factor based on current prices and positions
let collateral_value = calculate_collateral_value(
position.collateral_amount,
pool.asset_type,
)?;
let borrow_value = calculate_borrow_value(
position.borrowed_amount,
pool.asset_type,
)?;
health.health_factor = calculate_health_factor(
collateral_value,
borrow_value,
pool.collateral_factor,
)?;
health.liquidation_price = calculate_liquidation_price(
position.borrowed_amount,
position.collateral_amount,
pool.liquidation_threshold,
)?;
health.safe_borrow_limit = calculate_safe_borrow_limit(
collateral_value,
pool.collateral_factor,
)?;
Ok(())
}
3. Pool and Position Utilities
Helper functions for managing pools and positions:
// Calculate the utilization rate of a pool
pub fn calculate_utilization_rate(pool: &LendingPool) -> Result<u64> {
if pool.total_deposits == 0 {
return Ok(0);
}
Ok((pool.total_borrows * 10000) / pool.total_deposits)
}
// Calculate the health factor of a position
pub fn calculate_health_factor(
collateral_value: u64,
borrow_value: u64,
collateral_factor: u64,
) -> Result<u64> {
if borrow_value == 0 {
return Ok(u64::MAX);
}
Ok((collateral_value * collateral_factor) / (borrow_value * 10000))
}
// Update pool metrics
pub fn update_pool_metrics(
pool: &LendingPool,
metrics: &mut PoolMetrics,
) -> Result<()> {
metrics.total_value_locked = pool.total_deposits;
metrics.available_liquidity = pool.total_deposits.saturating_sub(pool.total_borrows);
metrics.utilization_rate = calculate_utilization_rate(pool)?;
// Update APY rates based on utilization
let (supply_apy, borrow_apy) = calculate_apy_rates(
metrics.utilization_rate,
pool.interest_rate,
)?;
metrics.supply_apy = supply_apy;
metrics.borrow_apy = borrow_apy;
Ok(())
}
4. Deposit Assets
Create a deposit function to allow users to provide liquidity:
pub fn deposit(
ctx: Context<Deposit>,
amount: u64,
btc_txid: [u8; 32],
vout: u32,
) -> Result<()> {
let pool = &mut ctx.accounts.lending_pool;
let user_position = &mut ctx.accounts.user_position;
// Verify the UTXO belongs to the user
require!(
validate_utxo_ownership(
&UtxoMeta {
txid: btc_txid,
vout,
amount,
},
&ctx.accounts.user.key()
)?,
ErrorCode::InvalidUTXO
);
// Create deposit account to hold the UTXO
invoke(
&SystemInstruction::new_create_account_instruction(
btc_txid,
vout,
pool.pool_pubkey,
),
&[ctx.accounts.user.clone(), ctx.accounts.pool.clone()]
)?;
// Update pool state
pool.total_deposits = pool.total_deposits
.checked_add(amount)
.ok_or(ErrorCode::MathOverflow)?;
// Update user position
user_position.deposited_amount = user_position.deposited_amount
.checked_add(amount)
.ok_or(ErrorCode::MathOverflow)?;
// Update utilization metrics
update_utilization_rate(pool)?;
Ok(())
}
5. Borrow Assets
Implement borrowing functionality:
pub fn borrow(
ctx: Context<Borrow>,
amount: u64,
collateral_utxo: UtxoMeta,
) -> Result<()> {
let pool = &mut ctx.accounts.lending_pool;
let borrower_position = &mut ctx.accounts.user_position;
// Verify collateral UTXO ownership
require!(
validate_utxo_ownership(
&collateral_utxo,
&ctx.accounts.borrower.key()
)?,
ErrorCode::InvalidCollateral
);
// Check collateral requirements
require!(
is_collateral_sufficient(borrower_position, pool, amount)?,
ErrorCode::InsufficientCollateral
);
// Create collateral account
invoke(
&SystemInstruction::new_create_account_instruction(
collateral_utxo.txid,
collateral_utxo.vout,
pool.pool_pubkey,
),
&[ctx.accounts.borrower.clone(), ctx.accounts.pool.clone()]
)?;
// Create borrow UTXO for user
let mut btc_tx = Transaction::new();
add_state_transition(&mut btc_tx, ctx.accounts.pool);
// Set transaction for validator signing
set_transaction_to_sign(
ctx.accounts,
TransactionToSign {
tx_bytes: &bitcoin::consensus::serialize(&btc_tx),
inputs_to_sign: &[InputToSign {
index: 0,
signer: pool.pool_pubkey
}]
}
);
// Update states
pool.total_borrows = pool.total_borrows
.checked_add(amount)
.ok_or(ErrorCode::MathOverflow)?;
borrower_position.borrowed_amount = borrower_position.borrowed_amount
.checked_add(amount)
.ok_or(ErrorCode::MathOverflow)?;
update_utilization_rate(pool)?;
update_interest_rate(pool)?;
Ok(())
}
6. Liquidation Logic
Implement liquidation for underwater positions:
pub fn liquidate(
ctx: Context<Liquidate>,
repay_amount: u64,
) -> Result<()> {
let pool = &mut ctx.accounts.lending_pool;
let liquidated_position = &mut ctx.accounts.liquidated_position;
// Check if position is liquidatable
require!(
is_position_liquidatable(liquidated_position, pool)?,
ErrorCode::PositionNotLiquidatable
);
// Calculate liquidation bonus
let bonus = calculate_liquidation_bonus(repay_amount, pool.liquidation_threshold)?;
// Process liquidation
process_liquidation(
pool,
liquidated_position,
repay_amount,
bonus,
)?;
Ok(())
}
Testing
Create comprehensive tests for your lending protocol:
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_initialize_lending_pool() {
// Test pool initialization
}
#[test]
fn test_deposit() {
// Test deposit functionality
}
#[test]
fn test_borrow() {
// Test borrowing
}
#[test]
fn test_liquidation() {
// Test liquidation scenarios
}
}
Security Considerations
- Collateral Safety: Implement strict collateral requirements and regular position health checks
- Price Oracle Security: Use reliable price feeds and implement safeguards against price manipulation
- Interest Rate Model: Ensure the model can handle extreme market conditions
- Access Control: Implement proper permission checks for all sensitive operations
- Liquidation Thresholds: Set appropriate thresholds to maintain protocol solvency
Next Steps
-
Implement additional features:
- Flash loans
- Multiple collateral types
- Governance mechanisms
-
Deploy and test on testnet:
- Monitor pool performance
- Test liquidation scenarios
- Validate interest rate model
-
Security audit:
- Contract review
- Economic model analysis
- Risk assessment
Process Descriptions
1. Pool Initialization Process
The pool initialization process involves several steps:
%%{init: { 'theme': 'base', 'themeVariables': { 'fontSize': '16px'}, 'flowchart': { 'curve': 'basis', 'nodeSpacing': 50, 'rankSpacing': 50, 'animation': { 'sequence': true, 'duration': 1000, 'ease': 'linear', 'diagramUpdate': 200 } } }}%% graph LR A[Admin] -->|Create Pool| B[Initialize Pool Account] B -->|Set Parameters| C[Configure Pool] C -->|Initialize Metrics| D[Create Pool Metrics] D -->|Enable Oracle| E[Connect Price Feed] E -->|Activate| F[Pool Active] classDef default fill:#f9f9f9,stroke:#333,stroke-width:2px; classDef active fill:#4a9eff,color:white,stroke:#3182ce,opacity:0; classDef complete fill:#98FB98,stroke:#333;
- Admin creates a new pool account
- Pool parameters are set (interest rates, thresholds)
- Pool metrics are initialized
- Price oracle connection is established
- Pool is activated for user operations
2. Deposit and Borrow Flow
The lending and borrowing process follows this sequence:
graph TD A[User] -->|Deposit Assets| B[Lending Pool] B -->|Create Position| C[User Position] C -->|Calculate Capacity| D[Borrow Limit] D -->|Enable Borrowing| E[Borrow Assets] E -->|Update Metrics| F[Pool Metrics] F -->|Adjust Rates| G[Interest Rates]
Key steps:
- User deposits assets into the pool
- System creates or updates user position
- Calculates borrowing capacity based on collateral
- Enables borrowing up to the limit
- Updates pool metrics and interest rates
3. Health Monitoring System
Continuous health monitoring process:
graph TD A[Price Oracle] -->|Update Prices| B[Position Valuation] B -->|Calculate Ratios| C[Health Check] C -->|Evaluate| D{Health Factor} D -->|>1| E[Healthy] D -->|<1| F[At Risk] F -->|<Threshold| G[Liquidatable] G -->|Notify| H[Liquidators]
The system:
- Continuously monitors asset prices
- Updates position valuations
- Calculates health factors
- Triggers liquidations when necessary
Withdrawal Process
The withdrawal process in our lending protocol involves two key components:
- State management through program accounts
- Actual BTC transfer through UTXOs
#[derive(BorshSerialize, BorshDeserialize)]
pub struct WithdrawRequest {
pub user_pubkey: Pubkey,
pub pool_pubkey: Pubkey,
pub amount: u64,
pub recipient_btc_address: String,
}
pub fn process_withdrawal(
ctx: Context<ProcessWithdraw>,
request: WithdrawRequest,
) -> Result<()> {
let pool = &mut ctx.accounts.lending_pool;
let user_position = &mut ctx.accounts.user_position;
// 1. Validate user position
require!(
user_position.deposited_amount >= request.amount,
ErrorCode::InsufficientBalance
);
// 2. Check pool liquidity
require!(
pool.available_liquidity() >= request.amount,
ErrorCode::InsufficientLiquidity
);
// 3. Find available UTXOs from pool
let selected_utxos = select_utxos_for_withdrawal(
&pool.utxos,
request.amount
)?;
// 4. Create Bitcoin withdrawal transaction
let mut btc_tx = Transaction::new();
// Add inputs from selected UTXOs
for utxo in selected_utxos {
btc_tx.input.push(TxIn {
previous_output: OutPoint::new(utxo.txid, utxo.vout),
script_sig: Script::new(),
sequence: Sequence::MAX,
witness: Witness::new(),
});
}
// Add withdrawal output to user's address
let recipient_script = Address::from_str(&request.recipient_btc_address)?
.script_pubkey();
btc_tx.output.push(TxOut {
value: request.amount,
script_pubkey: recipient_script,
});
// Add change output back to pool if needed
let total_input = selected_utxos.iter()
.map(|utxo| utxo.amount)
.sum::<u64>();
if total_input > request.amount {
btc_tx.output.push(TxOut {
value: total_input - request.amount,
script_pubkey: get_account_script_pubkey(&pool.pool_pubkey),
});
}
// 5. Set transaction for validator signing
set_transaction_to_sign(
ctx.accounts,
TransactionToSign {
tx_bytes: &bitcoin::consensus::serialize(&btc_tx),
inputs_to_sign: &selected_utxos.iter()
.enumerate()
.map(|(i, _)| InputToSign {
index: i as u32,
signer: pool.pool_pubkey,
})
.collect::<Vec<_>>()
}
);
// 6. Update pool state
pool.total_deposits = pool.total_deposits
.checked_sub(request.amount)
.ok_or(ErrorCode::MathOverflow)?;
// 7. Update user position
user_position.deposited_amount = user_position.deposited_amount
.checked_sub(request.amount)
.ok_or(ErrorCode::MathOverflow)?;
// 8. Remove spent UTXOs from pool
pool.utxos.retain(|utxo| !selected_utxos.contains(utxo));
Ok(())
}
fn select_utxos_for_withdrawal(
pool_utxos: &[UtxoMeta],
amount: u64,
) -> Result<Vec<UtxoMeta>> {
let mut selected = Vec::new();
let mut total_selected = 0;
for utxo in pool_utxos {
if total_selected >= amount {
break;
}
// Verify UTXO is still valid and unspent
validate_utxo(utxo)?;
selected.push(utxo.clone());
total_selected += utxo.amount;
}
require!(
total_selected >= amount,
ErrorCode::InsufficientUtxos
);
Ok(selected)
}
Architecture Overview
Core Components
graph TB subgraph "Arch Network" VM[Arch VM<br/>eBPF-based] BTC[Bitcoin Integration] subgraph "Validator Network" L[Leader Node] V1[Validator Node 1] V2[Validator Node 2] V3[Validator Node ...] B[Bootnode] end VM --> BTC L --> V1 L --> V2 L --> V3 B --> V1 B --> V2 B --> V3 end
Arch VM
The Arch Virtual Machine (VM) is built on eBPF technology, providing a secure and efficient environment for executing programs.
Key features:
- ๐ Manages program execution
- โก Handles state transitions
- ๐ฏ Ensures deterministic computation
- ๐ Provides syscalls for Bitcoin UTXO operations
Bitcoin Integration
Arch Network interacts directly with Bitcoin through:
- ๐ผ Native UTXO management
- โ Transaction validation
- ๐ Multi-signature coordination
- ๐ State commitment to Bitcoin
Validator Network
The validator network consists of multiple node types that work together:
Node Types
Node Type | Primary Responsibilities |
---|---|
Leader Node | โข Coordinates transaction signing โข Submits signed transactions to Bitcoin โข Manages validator communication |
Validator Nodes | โข Execute programs in the Arch VM โข Validate transactions โข Participate in multi-signature operations โข Maintain network state |
Bootnode | โข Handles initial network discovery โข Similar to Bitcoin DNS seeds โข Helps new nodes join the network |
Transaction Flow
sequenceDiagram participant C as Client participant L as Leader participant V as Validators participant B as Bitcoin Network C->>L: 1. Submit Transaction L->>V: 2. Distribute to Validators V->>V: 3. Execute in Arch VM V->>L: 4. Sign Results L->>B: 5. Submit to Bitcoin
Security Model
Arch Network implements a robust multi-layered security model that directly leverages Bitcoin's security guarantees:
1. UTXO Security
-
๐ Ownership Verification
- Public key cryptography using secp256k1
- BIP322 message signing for secure ownership proofs
- Double-spend prevention through UTXO consumption tracking
-
๐ State Management
- State anchoring to Bitcoin transactions
- Atomic state transitions with rollback capability
- Cross-validator state consistency checks
2. Transaction Security
pub struct SecurityParams {
pub min_confirmations: u32, // Required Bitcoin confirmations
pub signature_threshold: u32, // Multi-sig threshold
pub timelock_blocks: u32, // Timelock requirement
pub max_witness_size: usize // Maximum witness data size
}
- ๐ Multi-signature Validation
- ROAST protocol for distributed signing
- Threshold signature scheme (t-of-n)
- Malicious signer detection and removal
- Binding factor verification for signature shares
3. Network Security
-
๐ Validator Selection
pub struct ValidatorSet { pub validators: Vec<ValidatorInfo>, pub threshold: u32 }
- Stake-weighted validator participation
- Dynamic threshold adjustment
- Automatic malicious node detection
-
๐ก๏ธ State Protection
- Multi-stage transaction verification
- Bitcoin-based finality guarantees
- State root commitment to Bitcoin
- Mandatory signature verification for all state changes
4. Best Practices
-
โ UTXO Management
- Minimum 6 confirmations for finality
- Comprehensive UTXO validation
- Double-spend monitoring
- Reorg handling for UTXO invalidation
-
๐ Transaction Processing
- Full signature verification
- Input/output validation
- Proper error handling
- Network partition handling
Network Architecture
Arch Network operates as a distributed system with different types of nodes working together to provide secure and efficient program execution on Bitcoin. This document details the network's architecture and how different components interact.
Network Overview
flowchart TB subgraph Core["Core Components"] direction TB BN[Bitcoin Network] Boot[Bootnode] end subgraph Leader["Leader Node Services"] direction LR TC[Transaction\nCoordination] MS[MultiSig\nAggregation] end subgraph Validators["Validator Network"] direction TB V1[Validator 1] V2[Validator 2] V3[Validator 3] VN[Validator N] end BN --> LN[Leader Node] Boot --> LN LN --> TC LN --> MS LN --> V1 LN --> V2 LN --> V3 LN --> VN %% Styling classDef core fill:#e1f5fe,stroke:#01579b classDef leader fill:#fff3e0,stroke:#e65100 classDef validators fill:#f3e5f5,stroke:#4a148c class BN,Boot core class TC,MS leader class V1,V2,V3,VN validators
Node Types
1. Bootnode
The bootnode serves as the network's entry point, similar to DNS seeds in Bitcoin:
- Handles initial network discovery
- Maintains whitelist of valid validators
- Coordinates peer connections
- Manages network topology
flowchart LR subgraph Bootnode["Bootnode Services"] direction TB PR[Peer Registry] WL[Validator Whitelist] end NN[New Node] VN[Validator Network] %% Connections NN <--> PR PR <--> VN WL -.-> PR %% Styling classDef bootnode fill:#e1f5fe,stroke:#01579b classDef external fill:#f5f5f5,stroke:#333 classDef connection stroke-width:2px class PR,WL bootnode class NN,VN external
Configuration:
cargo run -p bootnode -- \
--network-mode localnet \
--p2p-bind-port 19001 \
--leader-peer-id "<LEADER_ID>" \
--validator-whitelist "<VALIDATOR_IDS>"
2. Leader Node
The leader node coordinates transaction processing and Bitcoin integration:
flowchart TB %% Main Components BN[Bitcoin Network] LN[Leader Node] VN[Validator Network] PE[Program Execution] %% Leader Node Services subgraph Leader["Leader Node Services"] direction LR TC[Transaction\nCoordination] MS[Multi-sig\nAggregation] end %% Connections BN <--> LN LN --> Leader TC --> VN MS --> VN VN --> PE %% Styling classDef bitcoin fill:#f7931a,stroke:#c16c07,color:white classDef leader fill:#fff3e0,stroke:#e65100 classDef validator fill:#f3e5f5,stroke:#4a148c classDef execution fill:#e8f5e9,stroke:#1b5e20 class BN bitcoin class LN,TC,MS leader class VN validator class PE execution
Key responsibilities:
- Transaction coordination
- Multi-signature aggregation
- Bitcoin transaction submission
- Network state management
3. Validator Nodes
Validator nodes form the core of the network's computation and validation:
flowchart TB subgraph ValidatorNode["Validator Node"] direction TB subgraph Execution["Execution Layer"] direction LR VM["Arch VM\nExecution"] SV["State\nValidation"] end NP["Network Protocol"] P2P["P2P Network"] %% Connections within validator VM --> NP SV --> NP NP --> P2P end %% Styling classDef validator fill:#f3e5f5,stroke:#4a148c,stroke-width:2px classDef execution fill:#e8f5e9,stroke:#1b5e20,stroke-width:2px classDef network fill:#e3f2fd,stroke:#0d47a1,stroke-width:2px class ValidatorNode validator class VM,SV execution class NP,P2P network
Types:
-
Full Validator
- Participates in consensus
- Executes programs
- Maintains full state
-
Lightweight Validator
- Local development use
- Single-node operation
- Simulated environment
Network Communication
P2P Protocol
The network uses libp2p for peer-to-peer communication:
pub const ENABLED_PROTOCOLS: [&str; 2] = [
ArchNetworkProtocol::STREAM_PROTOCOL,
ArchNetworkProtocol::VALIDATOR_PROTOCOL,
];
// Protocol versions
pub const PROTOCOL_VERSION: &str = "/arch/1.0.0";
pub const VALIDATOR_VERSION: &str = "/arch/validator/1.0.0";
Message Types
-
Network Messages
pub enum NetworkMessage { Discovery(DiscoveryMessage), State(StateMessage), Transaction(TransactionMessage), }
-
ROAST Protocol Messages
pub enum RoastMessage { KeyGeneration(KeyGenMessage), Signing(SigningMessage), Aggregation(AggregationMessage), }
Network Modes
1. Devnet
- Local development environment
- Single validator setup
- Simulated Bitcoin interactions
- Fast block confirmation
2. Testnet
- Test environment with multiple validators
- Bitcoin testnet integration
- Real network conditions
- Test transaction processing
3. Mainnet
- Production network
- Full security model
- Bitcoin mainnet integration
- Live transaction processing
Security Model
1. Validator Selection
pub struct ValidatorInfo {
pub peer_id: PeerId,
pub pubkey: Pubkey,
pub stake: u64,
}
pub struct ValidatorSet {
pub validators: Vec<ValidatorInfo>,
pub threshold: u32,
}
2. Transaction Security
- Multi-signature validation using ROAST protocol
- Threshold signing (t-of-n)
- Bitcoin-based finality
- Double-spend prevention
3. State Protection
pub struct StateUpdate {
pub block_height: u64,
pub state_root: Hash,
pub bitcoin_height: u64,
pub signatures: Vec<Signature>,
}
Monitoring and Telemetry
1. Node Metrics
pub struct NodeMetrics {
pub peer_id: PeerId,
pub network_mode: ArchNetworkMode,
pub bitcoin_block_height: u64,
pub arch_block_height: u64,
pub peers_connected: u32,
pub transactions_processed: u64,
pub program_count: u32,
}
2. Network Health
pub struct NetworkHealth {
pub validator_count: u32,
pub active_validators: u32,
pub network_tps: f64,
pub average_block_time: Duration,
pub fork_count: u32,
}
3. Monitoring Endpoints
/metrics
- Prometheus metrics/health
- Node health check/peers
- Connected peers/status
- Network status
Best Practices
1. Node Operation
- Secure key management
- Regular state verification
- Proper shutdown procedures
- Log management
2. Network Participation
- Maintain node availability
- Monitor Bitcoin integration
- Handle network upgrades
- Backup critical data
3. Development Setup
- Use lightweight validator for testing
- Monitor resource usage
- Handle network modes properly
- Implement proper error handling
Bitcoin Integration
Arch Network provides direct integration with Bitcoin, enabling programs to interact with Bitcoin's UTXO model while maintaining Bitcoin's security guarantees. This document details how Arch Network integrates with Bitcoin.
Architecture Overview
flowchart TB subgraph BN[Bitcoin Network] BNode[Bitcoin Node] end subgraph TC[Titan Client] TCNode[Titan Client] end subgraph AN[Arch Network] LN[Leader Node\nBitcoin Integration] subgraph VN[Validator Network] P1[Program 1] PN[Program N] end end BNode <--> TCNode TCNode <--> LN LN <--> VN P1 --- PN
Core Components
1. UTXO Management
Arch Network manages Bitcoin UTXOs through a specialized system:
flowchart LR subgraph UTXO[Bitcoin UTXO] TxID[TransactionID] OutIdx[OutputIndex] end subgraph Account[Arch Account] Meta[UTXOMeta] State[ProgramState] end TxID --> Meta OutIdx --> Meta Meta --- State
// UTXO Metadata Structure
pub struct UtxoMeta {
pub txid: [u8; 32], // Transaction ID
pub vout: u32, // Output index
pub amount: u64, // Amount in satoshis
pub script_pubkey: Vec<u8>, // Output script
pub confirmation_height: Option<u32>, // Block height of confirmation
}
// UTXO Account State
pub struct UtxoAccount {
pub meta: UtxoMeta,
pub owner: Pubkey,
pub delegate: Option<Pubkey>,
pub state: Vec<u8>,
pub is_frozen: bool,
}
Key operations:
// UTXO Operations
pub trait UtxoOperations {
fn create_utxo(meta: UtxoMeta, owner: &Pubkey) -> Result<()>;
fn spend_utxo(utxo: &UtxoMeta, signature: &Signature) -> Result<()>;
fn freeze_utxo(utxo: &UtxoMeta, authority: &Pubkey) -> Result<()>;
fn delegate_utxo(utxo: &UtxoMeta, delegate: &Pubkey) -> Result<()>;
}
2. Bitcoin RPC Integration
flowchart LR AP[Arch\nProgram] RPC[Bitcoin RPC\nInterface] BN[Bitcoin\nNode] Config[Configuration] Network[Bitcoin\nNetwork] AP --> RPC RPC --> BN AP --> Config Config --> RPC BN --> Network style AP fill:#f9f9f9,stroke:#333,stroke-width:2px style RPC fill:#f9f9f9,stroke:#333,stroke-width:2px style BN fill:#f9f9f9,stroke:#333,stroke-width:2px style Config fill:#f9f9f9,stroke:#333,stroke-width:2px style Network fill:#f9f9f9,stroke:#333,stroke-width:2px
Programs can interact with Bitcoin through RPC calls:
// Bitcoin RPC Configuration
pub struct BitcoinRpcConfig {
pub endpoint: String,
pub port: u16,
pub username: String,
pub password: String,
pub wallet: Option<String>,
pub network: BitcoinNetwork,
pub timeout: Duration,
}
// RPC Interface
pub trait BitcoinRpc {
fn get_block_count(&self) -> Result<u64>;
fn get_block_hash(&self, height: u64) -> Result<BlockHash>;
fn get_transaction(&self, txid: &Txid) -> Result<Transaction>;
fn send_raw_transaction(&self, tx: &[u8]) -> Result<Txid>;
fn verify_utxo(&self, utxo: &UtxoMeta) -> Result<bool>;
}
Transaction Flow
sequenceDiagram participant Program participant Leader participant Validator participant Bitcoin Program->>Leader: Create UTXO Leader->>Validator: Validate Validator->>Leader: Sign Leader->>Bitcoin: Submit TX Bitcoin-->>Program: Confirmation
1. Transaction Creation
// Create new UTXO transaction
pub struct UtxoCreation {
pub amount: u64,
pub owner: Pubkey,
pub metadata: Option<Vec<u8>>,
}
impl UtxoCreation {
pub fn new(amount: u64, owner: Pubkey) -> Self {
Self {
amount,
owner,
metadata: None,
}
}
pub fn with_metadata(mut self, metadata: Vec<u8>) -> Self {
self.metadata = Some(metadata);
self
}
}
2. Transaction Validation
// Validation rules
pub trait TransactionValidation {
fn validate_inputs(&self, tx: &Transaction) -> Result<()>;
fn validate_outputs(&self, tx: &Transaction) -> Result<()>;
fn validate_signatures(&self, tx: &Transaction) -> Result<()>;
fn validate_script(&self, tx: &Transaction) -> Result<()>;
}
3. State Management
// State transition
pub struct StateTransition {
pub previous_state: Hash,
pub next_state: Hash,
pub utxos_created: Vec<UtxoMeta>,
pub utxos_spent: Vec<UtxoMeta>,
pub bitcoin_height: u64,
}
Security Model
1. UTXO Security
- Ownership verification through public key cryptography
- Double-spend prevention through UTXO consumption
- State anchoring to Bitcoin transactions
- Threshold signature requirements
2. Transaction Security
// Transaction security parameters
pub struct SecurityParams {
pub min_confirmations: u32,
pub signature_threshold: u32,
pub timelock_blocks: u32,
pub max_witness_size: usize,
}
3. Network Security
- Multi-signature validation
- Threshold signing (t-of-n)
- Bitcoin-based finality
- Cross-validator consistency
Error Handling
1. Bitcoin Errors
pub enum BitcoinError {
ConnectionFailed(String),
InvalidTransaction(String),
InsufficientFunds(u64),
InvalidUtxo(UtxoMeta),
RpcError(String),
}
2. UTXO Errors
pub enum UtxoError {
NotFound(UtxoMeta),
AlreadySpent(UtxoMeta),
InvalidOwner(Pubkey),
InvalidSignature(Signature),
InvalidState(Hash),
}
Best Practices
1. UTXO Management
- Always verify UTXO ownership
- Wait for sufficient confirmations
- Handle reorganizations gracefully
- Implement proper error handling
2. Transaction Processing
- Validate all inputs and outputs
- Check signature thresholds
- Maintain proper state transitions
- Monitor Bitcoin network status
3. Security Considerations
- Protect private keys
- Validate all signatures
- Monitor for double-spend attempts
- Handle network partitions
ROAST and FROST Consensus
This section explores Arch's consensus mechanism, which combines ROAST (Robust Asynchronous Schnorr Threshold Signatures) and FROST (Flexible Round-Optimized Schnorr Threshold Signatures) to create a secure, efficient, and highly scalable approach to distributed consensus that's perfectly suited for Bitcoin-based smart contracts.
Implementation Status
The consensus mechanism implementation has made significant progress, particularly in the core cryptographic components:
-
Implemented Components
- Complete Distributed Key Generation (DKG) protocol using FROST-secp256k1
- Two-round DKG process with package handling
- Network message protocol for DKG coordination
- State management and status tracking
- Integration with network layer
- Error handling and recovery mechanisms
-
In Progress
- Additional ROAST protocol components
- Advanced state management features
- Performance optimizations
- Extended monitoring and telemetry
The subsequent sections describe both the implemented features and the complete protocol design.
Core Implementation Details
Distributed Key Generation (DKG)
#![allow(unused)] fn main() { // Core DKG message types for network coordination pub enum DKGMessage { StartDKG { message: String }, Round1Package { package: round1::Package }, Round2Package { package: round2::Package }, DKGStatus(DKGStatusMessage), } // DKG state management pub enum DKGStatus { Pending(String), Ongoing(String), Failed(String, String), Finished(String), NetworkCompleted(String), } }
The DKG implementation provides:
- Two-round key generation protocol
- Secure package exchange between validators
- State tracking and synchronization
- Failure recovery and error handling
TL;DR
Arch's consensus mechanism combines ROAST and FROST to provide a robust, Bitcoin-native consensus solution. Validators participate in a threshold signature scheme where blocks are produced by designated leaders and finalized through collective signing. The system maintains both safety and liveness through careful economic incentives and cryptographic guarantees, while ensuring complete compatibility with Bitcoin's Schnorr signature scheme.
Block Production Process
1. Leader Selection
graph TD A[Epoch Start] --> B[Calculate Leader Schedule] B --> C[Distribute Schedule to Validators] C --> D[Leaders Prepare for Slots] D --> E[Next Leader Takes Turn] E --> F[Block Production] F --> G[Block Distribution] G --> E
The block production process begins with leader selection:
- Each epoch (fixed time period) has a predetermined leader schedule
- Leaders are selected based on their stake weight
- The schedule is deterministic and known to all validators
- Multiple backup leaders are selected for fault tolerance
2. Transaction Collection and Verification
graph LR A[Transaction Pool] --> B[Leader] C[Mempool] --> B B --> D[Transaction Verification] D --> E[Block Formation] E --> F[Block Proposal]
When a validator becomes the leader:
- Collects pending transactions from the mempool
- Verifies transaction signatures and validity
- Orders transactions based on priority and fees
- Prepares them for inclusion in the next block
3. Block Formation
sequenceDiagram participant L as Leader participant B as Block participant V as Validators L->>B: Create Block Header L->>B: Add Transactions L->>B: Add State Updates L->>B: Sign Block L->>V: Broadcast Block
The block structure includes:
- Previous block reference
- Timestamp
- Transaction merkle root
- UTXO state updates
- Leader's signature
Consensus Process
1. Block Validation
graph TD A[Receive Block] --> B[Verify Leader] B --> C[Verify Signatures] C --> D[Execute Transactions] D --> E[Verify UTXO States] E --> F[Vote Decision]
When validators receive a new block:
- Verify the block producer is the designated leader
- Validate all transaction signatures
- Execute transactions and verify UTXO states
- Check for any consensus rule violations
2. UTXO-Based State Management
graph TD A[Transaction] --> B[UTXO Validation] B --> C[State Update] C --> D[Bitcoin Transaction] D --> E[Validator Signatures] B --> F[Ownership Verification] B --> G[Double-spend Check] B --> H[Confirmation Check] C --> I[Account Updates] C --> J[Program State] C --> K[UTXO Set Changes]
Arch's unique approach to state management leverages Bitcoin's UTXO model while extending it for smart contract functionality:
UTXO State Tracking
#![allow(unused)] fn main() { pub struct UtxoState { pub meta: UtxoMeta, // UTXO identification pub status: UtxoStatus, // Current UTXO status pub owner: Pubkey, // UTXO owner pub created_at: i64, // Creation timestamp pub spent_at: Option<i64>, // Spend timestamp if spent } pub enum UtxoStatus { Pending, // Waiting for confirmations Active, // Confirmed and spendable Spent, // UTXO has been consumed Invalid, // UTXO was invalidated (e.g., by reorg) } }
State Transition Process
-
UTXO Validation
- Verify UTXO existence on Bitcoin
- Check confirmation requirements (typically 6+)
- Validate ownership and spending conditions
- Prevent double-spending attempts
-
State Updates
- Atomic account data modifications
- Program state transitions
- UTXO set updates
- Cross-validator state consistency
-
Bitcoin Integration
- State anchoring to Bitcoin transactions
- Threshold signature aggregation
- Transaction finality through Bitcoin confirmations
- Reorg handling and state rollbacks
Security Properties
-
Ownership Verification
- Public key cryptography using secp256k1
- BIP322 message signing for ownership proofs
- Threshold signature requirements
-
Double-spend Prevention
- UTXO consumption tracking
- Cross-validator consistency checks
- Bitcoin-based finality guarantees
-
State Protection
- Atomic state transitions
- Rollback capability for reorgs
- State root commitments
- Multi-stage verification
Performance Optimizations
- UTXO caching for frequent access
- Batch processing of state updates
- Parallel transaction validation
- Efficient UTXO lookup mechanisms
This UTXO-based approach provides several advantages:
- Direct compatibility with Bitcoin's security model
- Natural support for atomic operations
- Clear ownership and state transition rules
- Built-in protection against double-spending
- Simplified state verification and rollback
3. FROST Signing Process
sequenceDiagram participant V1 as Validator 1 participant V2 as Validator 2 participant V3 as Validator 3 participant C as Consensus V1->>C: Share 1 V2->>C: Share 2 V3->>C: Share 3 C->>C: Aggregate Shares C->>All: Final Signature
The FROST signing process involves:
- Each validator generates their partial signature
- Signatures are shared among the threshold group
- Partial signatures are aggregated into a final signature
- The aggregated signature is verified against the group public key
4. ROAST Enhancement Layer
graph TD A[Block Proposal] --> B[FROST Signing] B --> C[ROAST Protocol] C --> D[Asynchronous Consensus] D --> E[Block Finalization] E --> F[Chain Extension]
ROAST transforms FROST into a production-ready consensus mechanism by adding several crucial enhancements:
Asynchronous Operation Guarantees
sequenceDiagram participant V1 as Validator 1 participant V2 as Validator 2 participant V3 as Validator 3 participant N as Network Note over V1,N: Validator 1 experiences delay V2->>N: Sign Share (t=1) V3->>N: Sign Share (t=1) V1->>N: Sign Share (t=3) Note over N: Protocol continues despite delay N->>V1: Aggregate & Finalize N->>V2: Aggregate & Finalize N->>V3: Aggregate & Finalize
Unlike traditional consensus mechanisms that require strict synchronization:
- Validators can participate in signing rounds without tight timing constraints
- The protocol progresses even when some validators are temporarily delayed
- Network partitions and varying message delivery times are handled gracefully
- No assumptions about network synchrony are required for safety
Byzantine Fault Tolerance
graph TD A[Validator Set] --> B[Honest Majority] A --> C[Byzantine Nodes] B --> D[Valid Signatures] C --> E[Detection System] E --> F[Isolation] F --> G[Protocol Progress]
ROAST maintains safety and liveness even in the presence of malicious actors:
- Tolerates up to f Byzantine validators where f < n/3
- Malicious behavior is detected and isolated
- Signature shares from Byzantine validators can be identified and excluded
- The protocol remains secure even if Byzantine validators:
- Submit invalid signature shares
- Attempt to sign conflicting blocks
- Try to delay or prevent consensus
- Collude with other malicious validators
Leader Rotation Mechanism
sequenceDiagram participant VS as Validator Set participant L1 as Leader 1 participant L2 as Leader 2 participant L3 as Leader 3 Note over VS: Round r VS->>L1: Select Leader Note over L1: Leader Timeout/Failure VS->>L2: Backup Leader Takes Over Note over L2: Successful Block Note over VS: Round r+1 VS->>L3: New Leader Selection
ROAST implements a robust leader rotation system that:
- Deterministically selects leaders based on stake weight and randomness
- Automatically rotates leaders to prevent centralization
- Provides backup leaders in case of primary leader failure
- Ensures fair distribution of block production opportunities
- Maintains progress even when leaders fail or misbehave
Liveness Guarantees
graph TD A[Network State] --> B{Leader Active?} B -->|Yes| C[Normal Operation] B -->|No| D[Backup Leader] C --> E[Progress] D --> E E --> F{Sufficient Signatures?} F -->|Yes| G[Block Finalization] F -->|No| H[Continue Collection] H --> F
ROAST ensures the network continues to make progress through several mechanisms:
-
View Synchronization
- Validators maintain a consistent view of network state
- Recovery procedures for missed blocks or state updates
- Automatic resynchronization after network partitions
-
Failure Recovery
- Automatic detection of failed validators
- Seamless transition to backup leaders
- Recovery from temporary network failures
- Rejoining procedures for validators that fall behind
-
Progress Conditions
- Guaranteed block finalization when sufficient honest validators participate
- No single validator can prevent progress
- Continued operation during validator churn
- Resilient to temporary network issues
-
Deadlock Prevention
- No waiting for specific validators
- Timeout mechanisms for unresponsive participants
- Alternative paths for consensus when optimal path fails
- Dynamic adjustment of protocol parameters
These enhancements make ROAST particularly well-suited for production environments where:
- Network conditions are unpredictable
- Validators may join or leave the network
- Malicious actors may attempt to disrupt consensus
- High availability and reliability are required
Fork Resolution
graph TD A[Fork Detection] --> B[Weight Calculation] B --> C[Heaviest Chain] C --> D[Switch Decision] D --> E[Chain Reorganization]
When forks occur:
- Validators identify competing chains
- Calculate the weight of each fork based on stake
- Apply the heaviest-chain rule
- Coordinate chain reorganization if needed
Understanding FROST
FROST is a threshold signature scheme that enables a group of participants to collectively generate Schnorr signatures. This foundational protocol is crucial for Arch's consensus mechanism because it provides a way to achieve distributed agreement while maintaining compatibility with Bitcoin's native signature scheme.
Key Components
- Distributed Key Generation: Validators collectively participate in a process that generates a shared public key while keeping individual private key shares separate and secure.
- Threshold Signatures: The system requires a specific number of validators (t-of-n) to cooperate in order to produce valid signatures, balancing security with fault tolerance.
- Share Management: Each validator maintains their own private key share, contributing to the system's security through distribution of trust.
- Signature Aggregation: Multiple partial signatures are combined into a single Schnorr signature that's indistinguishable from a standard single-signer signature.
Benefits of FROST
-
Enhanced Security
- No single validator can compromise the system
- Distributed trust model eliminates single points of failure
- Cryptographic guarantees of signature validity
-
Bitcoin Compatibility
- Native integration with Bitcoin's Schnorr signature scheme
- No additional on-chain overhead
- Seamless interaction with Bitcoin's transaction validation
-
Efficiency
- Constant-size signatures regardless of validator count
- Optimized communication patterns
- Reduced blockchain space usage
ROAST: Enhancing FROST for Production
While FROST provides the cryptographic foundation, ROAST adds crucial properties needed for real-world deployment in adversarial environments. ROAST transforms FROST from a theoretical protocol into a production-ready consensus mechanism.
Key Enhancements
-
Asynchronous Operation
- Validators can participate without strict timing requirements
- Resilient to network delays and partitions
- Maintains liveness in real-world conditions
-
Robustness Against Attacks
- Continues operating even with malicious participants
- Detects and handles various forms of validator misbehavior
- Provides provable security guarantees
-
Leader Selection
- Efficient and fair leader rotation mechanism
- Prevents centralization of power
- Maintains system progress even if leaders fail
-
Liveness Guarantees
- Ensures forward progress under adverse conditions
- Handles validator churn gracefully
- Recovers automatically from temporary failures
Arch's Novel Implementation
Arch's implementation of ROAST/FROST represents a significant innovation in the blockchain space, particularly for Bitcoin-based smart contract platforms.
Unique Features
-
Bitcoin-Native Design
- Optimized for Bitcoin's specific constraints and capabilities
- Leverages Bitcoin's security model
- Minimizes on-chain footprint
-
Smart Contract Integration
- Seamless combination with programmable logic
- Maintains Bitcoin's security guarantees
- Enables complex decentralized applications
-
Scalable State Management
- Efficient handling of state transitions
- Parallel transaction processing where possible
- Optimized validator resource usage
-
Economic Security
- Carefully designed incentive structure
- Slashing conditions for misbehavior
- Aligned validator and network interests
Performance Characteristics
- Throughput: High transaction processing capacity without sacrificing decentralization
- Latency: Optimized confirmation times while maintaining security
- Resource Usage: Efficient use of network and computational resources
- Scalability: Linear scaling with validator count for most operations
Security Considerations
Threat Model
- Byzantine Validators: System remains secure with up to f Byzantine validators (where f < n/3)
- Network Adversaries: Resilient against various network-level attacks
- Cryptographic Security: Based on well-studied cryptographic assumptions
Security Properties
-
Safety
- No conflicting transactions can be confirmed
- Cryptographic guarantees of transaction finality
- Protection against double-spending
-
Liveness
- System continues to make progress
- Recovers from temporary failures
- Handles validator set changes
-
Fault Tolerance
- Continues operating with partial validator failures
- Graceful degradation under attack
- Automatic recovery mechanisms
Future Directions
The ROAST/FROST consensus mechanism in Arch provides a solid foundation for future enhancements:
-
Scalability Improvements
- Research into further optimization of signature aggregation
- Investigation of layer-2 scaling solutions
- Exploration of parallel processing techniques
-
Security Enhancements
- Ongoing cryptographic research
- Additional protection against emerging threats
- Enhanced monitoring and detection systems
-
Feature Extensions
- Support for more complex smart contract patterns
- Enhanced cross-chain interoperability
- Advanced state management techniques
Further Reading
Academic Papers and Research
FROST (Flexible Round-Optimized Schnorr Threshold Signatures)
- FROST: Flexible Round-Optimized Schnorr Threshold Signatures - The original FROST paper by Chelsea Komlo and Ian Goldberg
- Two-Round Threshold Schnorr Signatures with FROST - An optimized two-round variant of FROST
- Implementing FROST - Reference implementation by the Zcash Foundation
ROAST (Robust Asynchronous Schnorr Threshold Signatures)
- ROAST: Robust Asynchronous Schnorr Threshold Signatures - The foundational ROAST paper
- Practical Threshold Signatures for Bitcoin - Implementation insights for Bitcoin-based threshold signatures
Threshold Cryptography and Consensus
- A Survey of Distributed Consensus Protocols for Blockchain Networks - Comprehensive overview of consensus mechanisms
- Threshold Signatures: The Future of Consensus? - Analysis of threshold signatures in consensus protocols
- Schnorr Multi-Signatures and Applications - Foundational work on Schnorr multi-signatures
Technical Resources
Implementation Guides
- BIP 340: Schnorr Signatures for secp256k1 - Bitcoin Improvement Proposal for Schnorr signatures
- Implementing Threshold Signatures - Technical guide on threshold signature implementation
- Multi-Party Computation for Distributed Key Generation - Reference implementation of distributed key generation
Security Analysis
- Security Analysis of Threshold Signature Schemes - Comprehensive security analysis
- Formal Verification of FROST - Formal security proofs for FROST
- Byzantine Fault Tolerance in Distributed Systems - Analysis of BFT in consensus protocols
Community Resources
- FROST Working Group - Community working group on FROST implementation
- Bitcoin Dev Mailing List - Discussions on threshold signatures in Bitcoin
Conclusion
The combination of ROAST and FROST in Arch represents a significant advancement in Bitcoin-based smart contract platforms. This consensus mechanism enables sophisticated applications while maintaining the security and decentralization principles that make Bitcoin valuable. Through careful design and implementation, Arch has created a system that is not just theoretically sound but practically deployable and scalable for real-world applications.
Program
A program is a special kind of account that contains executable eBPF bytecode, denoted by the Account.is_executable: true
field. This allows an account to receive arbitrary instruction data via a transaction to be processed by the runtime.
Every program is stateless, meaning that it can only read/write data to other accounts and that it cannot write to its own account; this, in-part, is how parallelized execution is made possible (see State for more info).
๐ก Additionally, programs can send instructions to other programs which, in turn, receive instructions and thus extend program composability further. This is known as cross-program invocation (CPI) and will be detailed in future sections.
Components:
1. Entrypoint
Every Arch program includes a single entrypoint used to invoke the program. A handler function, often named process_instruction
, is then used to handle the data passed into the entrypoint.
These parameters are required for every instruction to be processed._
use arch_program::entrypoint;
entrypoint!(process_instruction);
pub fn process_instruction(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8],
) -> Result<(), ProgramError> {
// Program logic here
}
2. Instruction
The instruction_data
is deserialized after being passed into the entrypoint. From there, if there are multiple instructions, a match
statement can be utilized to point the logic flow to the appropriate handler function previously defined within the program which can continue processing the instruction.
3. Process Instruction
If a program has multiple instructions, a corresponding handler function should be defined to include the specific logic unique to the instruction.
4. State
Since programs are stateless, a "data" account is needed to hold state for a user. This is a non-executable account that holds program data.
If a program receives instruction that results in a user's state being altered, the program would manage this user's state via a mapping within the program's logic. This mapping would link the user's pubkey with a data account where the state would live for that specific program.
The program will likely include a struct to define the structure of its state and make it easier to work with. The deserialization of account data occurs during program invocation. After an update is made, state data gets re-serialized into a byte array and stored within the data
field of the account.
UTXO (Unspent Transaction Output)
UTXOs (Unspent Transaction Outputs) are fundamental to Bitcoin's transaction model and serve as the foundation for state management in Arch Network. Unlike account-based systems that track balances, UTXOs represent discrete "coins" that must be consumed entirely in transactions.
Core Concepts
What is a UTXO?
- A UTXO represents an unspent output from a previous transaction
- Each UTXO is uniquely identified by a transaction ID (txid) and output index (vout)
- UTXOs are immutable - they can only be created or spent, never modified
- Once spent, a UTXO cannot be reused (prevents double-spending)
Role in Arch Network
- UTXOs anchor program state to Bitcoin's security model
- They provide deterministic state transitions
- Enable atomic operations across the network
- Allow for provable ownership and state validation
UTXO Structure
The UtxoMeta
struct encapsulates the core UTXO identification data:
use arch_program::utxo::UtxoMeta;
use bitcoin::Txid;
#[derive(Debug, Clone, PartialEq)]
pub struct UtxoMeta {
pub txid: [u8; 32], // Bitcoin transaction ID (32 bytes)
pub vout: u32, // Output index in the transaction
}
impl UtxoMeta {
/// Creates a new UTXO metadata instance
pub fn new(txid: [u8; 32], vout: u32) -> Self {
Self { txid, vout }
}
/// Deserializes UTXO metadata from a byte slice
/// Format: [txid(32 bytes)][vout(4 bytes)]
pub fn from_slice(data: &[u8]) -> Self {
let mut txid = [0u8; 32];
txid.copy_from_slice(&data[0..32]);
let vout = u32::from_le_bytes([
data[32], data[33], data[34], data[35]
]);
Self { txid, vout }
}
}
UTXO Lifecycle
1. Creation Process
Creating a UTXO with Bitcoin RPC
use bitcoincore_rpc::{Auth, Client as RpcClient, RpcApi};
use bitcoin::{Amount, Address};
use arch_program::pubkey::Pubkey;
// Initialize Bitcoin RPC client
let rpc = RpcClient::new(
"http://localhost:18443", // Bitcoin node RPC endpoint
Auth::UserPass(
"user".to_string(),
"pass".to_string()
)
).expect("Failed to create RPC client");
// Generate a new account address
let account_address = Pubkey::new_unique();
let btc_address = Address::from_pubkey(&account_address);
// Create UTXO by sending Bitcoin
// Parameters explained:
// - address: Destination Bitcoin address
// - amount: Amount in satoshis (3000 sats = 0.00003 BTC)
// - comment: Optional transaction comment
// - replaceable: Whether the tx can be replaced (RBF)
let txid = rpc.send_to_address(
&btc_address,
Amount::from_sat(3000),
Some("Create Arch UTXO"), // Comment
None, // Comment_to
Some(true), // Replaceable
None, // Fee rate
None, // Fee estimate mode
None // Avoid reuse
)?;
// Wait for confirmation (recommended)
rpc.wait_for_confirmation(&txid, 1)?;
Creating an Arch Account with UTXO
use arch_program::{
system_instruction::SystemInstruction,
pubkey::Pubkey,
transaction::Transaction,
};
// Create new program account backed by UTXO
let account_pubkey = Pubkey::new_unique();
let instruction = SystemInstruction::new_create_account_instruction(
txid.try_into().unwrap(),
0, // vout index
account_pubkey,
// Additional parameters like:
// - space: Amount of space to allocate
// - owner: Program that owns the account
);
// Build and sign transaction
let transaction = Transaction::new_signed_with_payer(
&[instruction],
Some(&payer.pubkey()),
&[&payer],
recent_blockhash
);
2. Validation & Usage
Programs must implement proper UTXO validation:
fn validate_utxo(utxo: &UtxoMeta) -> Result<(), ProgramError> {
// 1. Verify UTXO exists on Bitcoin
let btc_tx = rpc.get_transaction(&utxo.txid)?;
// 2. Check confirmation count
if btc_tx.confirmations < MIN_CONFIRMATIONS {
return Err(ProgramError::InsufficientConfirmations);
}
// 3. Verify output index exists
if utxo.vout as usize >= btc_tx.vout.len() {
return Err(ProgramError::InvalidVout);
}
// 4. Verify UTXO is unspent
if is_spent(utxo) {
return Err(ProgramError::UtxoAlreadySpent);
}
Ok(())
}
3. State Management
// Example UTXO state tracking
#[derive(Debug)]
pub struct UtxoState {
pub meta: UtxoMeta,
pub status: UtxoStatus,
pub owner: Pubkey,
pub created_at: i64,
pub spent_at: Option<i64>,
}
#[derive(Debug)]
pub enum UtxoStatus {
Pending, // Waiting for confirmations
Active, // Confirmed and spendable
Spent, // UTXO has been consumed
Invalid, // UTXO was invalidated (e.g., by reorg)
}
Best Practices
-
Validation
- Always verify UTXO existence on Bitcoin
- Check for sufficient confirmations (recommended: 6+)
- Validate ownership and spending conditions
- Handle Bitcoin reorgs that might invalidate UTXOs
-
State Management
- Implement robust UTXO tracking
- Handle edge cases (reorgs, conflicting txs)
- Consider implementing UTXO caching for performance
- Maintain accurate UTXO sets for your program
-
Security
- Never trust client-provided UTXO data without verification
- Implement proper access controls
- Consider timelock constraints for sensitive operations
- Monitor for suspicious UTXO patterns
-
Performance
- Batch UTXO operations when possible
- Implement efficient UTXO lookup mechanisms
- Consider UTXO consolidation strategies
- Cache frequently accessed UTXO data
Error Handling
Common UTXO-related errors to handle:
pub enum UtxoError {
NotFound, // UTXO doesn't exist
AlreadySpent, // UTXO was already consumed
InsufficientConfirmations, // Not enough confirmations
InvalidOwner, // Unauthorized attempt to spend
Reorged, // UTXO invalidated by reorg
InvalidVout, // Output index doesn't exist
SerializationError, // Data serialization failed
}
Related Topics
- Account Model - How UTXOs relate to Arch accounts
- Program State - Using UTXOs for program state
- System Program - Core UTXO operations
Account Guide
Navigation: Reference โ Program โ Account Guide
For the core account structure and data types, see Account Structure.
Accounts are the fundamental building blocks for state management and program interaction in Arch Network. They serve as containers for both program code and state data, bridging the gap between Bitcoin's UTXO model and modern programmable state machines.
Note: For detailed documentation on core system functions used to interact with accounts (like
invoke
,new_create_account_instruction
,add_state_transition
, andset_transaction_to_sign
), see System Functions.
flowchart TD A[Account] --> B[Program Account] A --> C[Data Account] A --> D[Native Account] B --> E[Executable Code] C --> F[Program State] C --> G[UTXOs] D --> H[System Operations] style A fill:#f9f,stroke:#333,stroke-width:2px style B fill:#f5f5f5,stroke:#666 style C fill:#f5f5f5,stroke:#666 style D fill:#f5f5f5,stroke:#666
Core Concepts
Account Fundamentals
Every account in Arch Network is uniquely identified by a public key (pubkey) and contains four essential components:
pub struct Account {
/// The program that owns this account
pub owner: Pubkey,
/// Number of lamports assigned to this account
pub lamports: u64,
/// Data held in this account
pub data: Vec<u8>,
/// Whether this account can process instructions
pub executable: bool,
}
Component Details:
-
Owner (Pubkey)
- Controls account modifications
- Determines which program can modify data
- Can be transferred to new programs
- Required for all accounts
-
Lamports (u64)
- Native token balance
- Used for:
- Transaction fees
- Rent payments
- State storage costs
- Program execution fees
-
Data (Vec
) - Flexible byte array for state storage
- Common uses:
- Program code (if executable)
- Program state
- UTXO metadata
- Configuration data
- Size determined at creation
-
Executable Flag (bool)
- Determines if account contains program code
- Immutable after deployment
- Controls instruction processing capability
flowchart LR A[Account Creation] --> B[Initial State] B --> C[Runtime Operations] C --> D[State Updates] D --> E[Account Closure] subgraph Lifecycle A -. Initialize .-> B B -. Process Instructions .-> C C -. Modify State .-> D D -. Cleanup .-> E end style A fill:#f9f,stroke:#333,stroke-width:2px style E fill:#9ff,stroke:#333,stroke-width:2px style Lifecycle fill:#f5f5f5,stroke:#666,stroke-width:1px
Account Types & Use Cases
1. Program Accounts
Program accounts contain executable code and form the backbone of Arch Network's programmable functionality.
// Example program account creation
let program_account = SystemInstruction::CreateAccount {
lamports: rent.minimum_balance(program_data.len()),
space: program_data.len() as u64,
owner: bpf_loader::id(), // BPF Loader owns program accounts
executable: true,
data: program_data,
};
Key characteristics:
- Immutable after deployment
- Owned by BPF loader
- Contains verified program code
- Processes instructions
2. Data Accounts
Data accounts store program state and user data. They're highly flexible and can be structured to meet various needs.
// Example data structure for a game account
#[derive(BorshSerialize, BorshDeserialize)]
pub struct GameAccount {
pub player: Pubkey,
pub score: u64,
pub level: u8,
pub achievements: Vec<Achievement>,
pub last_played: i64,
}
// Creating a data account
let game_account = SystemInstruction::CreateAccount {
lamports: rent.minimum_balance(size_of::<GameAccount>()),
space: size_of::<GameAccount>() as u64,
owner: game_program::id(),
executable: false,
data: Vec::new(), // Will be initialized by program
};
Common use cases:
- Player profiles
- Game state
- DeFi positions
- NFT metadata
- Configuration settings
3. UTXO Accounts
Special data accounts that bridge Bitcoin UTXOs with Arch Network state.
#[derive(BorshSerialize, BorshDeserialize)]
pub struct UtxoAccount {
pub meta: UtxoMeta,
pub owner: Pubkey,
pub delegate: Option<Pubkey>,
pub state: UtxoState,
pub created_at: i64,
pub last_updated: i64,
pub constraints: Vec<UtxoConstraint>,
}
// Example UTXO account creation
let utxo_account = SystemInstruction::CreateAccount {
lamports: rent.minimum_balance(size_of::<UtxoAccount>()),
space: size_of::<UtxoAccount>() as u64,
owner: utxo_program::id(),
executable: false,
data: Vec::new(),
};
Account Interactions
Account interactions in Arch Network are facilitated through a set of core system functions. These functions handle everything from account creation to state transitions and are documented in detail in System Functions. Below are common patterns for account interactions:
1. Creation Patterns
// 1. Basic account creation
pub fn create_basic_account(
payer: &Keypair,
space: u64,
owner: &Pubkey,
) -> Result<Keypair, Error> {
let account = Keypair::new();
let rent = banks_client.get_rent().await?;
let lamports = rent.minimum_balance(space as usize);
let ix = system_instruction::create_account(
&payer.pubkey(),
&account.pubkey(),
lamports,
space,
owner,
);
let tx = Transaction::new_signed_with_payer(
&[ix],
Some(&payer.pubkey()),
&[payer, &account],
recent_blockhash,
);
banks_client.process_transaction(tx).await?;
Ok(account)
}
// 2. PDA (Program Derived Address) creation
pub fn create_pda_account(
program_id: &Pubkey,
seeds: &[&[u8]],
space: u64,
) -> Result<Pubkey, Error> {
let (pda, bump) = Pubkey::find_program_address(seeds, program_id);
let ix = system_instruction::create_account(
&payer.pubkey(),
&pda,
lamports,
space,
program_id,
);
// Include the bump seed for deterministic address
let seeds_with_bump = &[&seeds[..], &[&[bump]]].concat();
let signer_seeds = &[&seeds_with_bump[..]];
invoke_signed(&ix, &[payer, pda], signer_seeds)?;
Ok(pda)
}
2. State Management
// Example of managing account state
pub trait AccountState: Sized {
fn try_from_slice(data: &[u8]) -> Result<Self, Error>;
fn try_serialize(&self) -> Result<Vec<u8>, Error>;
fn load(account: &AccountInfo) -> Result<Self, Error> {
Self::try_from_slice(&account.data.borrow())
}
fn save(&self, account: &AccountInfo) -> Result<(), Error> {
let data = self.try_serialize()?;
let mut account_data = account.data.borrow_mut();
account_data[..data.len()].copy_from_slice(&data);
Ok(())
}
}
// Implementation example
impl AccountState for GameAccount {
fn update_score(&mut self, new_score: u64) -> Result<(), Error> {
self.score = new_score;
self.last_played = Clock::get()?.unix_timestamp;
Ok(())
}
}
3. Cross-Program Invocation (CPI)
// Example of one program calling another
pub fn process_instruction(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8],
) -> ProgramResult {
// Deserialize accounts
let account_info_iter = &mut accounts.iter();
let source_info = next_account_info(account_info_iter)?;
let dest_info = next_account_info(account_info_iter)?;
let system_program = next_account_info(account_info_iter)?;
// Create CPI context
let cpi_accounts = Transfer {
from: source_info.clone(),
to: dest_info.clone(),
};
let cpi_program = system_program.clone();
let cpi_ctx = CpiContext::new(cpi_program, cpi_accounts);
// Perform cross-program invocation
transfer(cpi_ctx, amount)?;
Ok(())
}
Security Considerations
1. Access Control
fn verify_account_access(
account: &AccountInfo,
expected_owner: &Pubkey,
writable: bool,
) -> ProgramResult {
// Check account ownership
if account.owner != expected_owner {
return Err(ProgramError::IncorrectProgramId);
}
// Verify write permission if needed
if writable && !account.is_writable {
return Err(ProgramError::InvalidAccountData);
}
// Additional checks...
Ok(())
}
2. Data Validation
fn validate_account_data<T: AccountState>(
account: &AccountInfo,
validate_fn: impl Fn(&T) -> bool,
) -> ProgramResult {
// Load and validate account data
let data = T::load(account)?;
if !validate_fn(&data) {
return Err(ProgramError::InvalidAccountData);
}
Ok(())
}
Best Practices
1. Account Management
- Always validate account ownership before modifications
- Use PDAs for deterministic addresses
- Implement proper error handling
- Close unused accounts to reclaim rent
2. Data Safety
- Validate all input data
- Use proper serialization
- Handle account size limits
- Implement atomic operations
3. Performance
- Minimize account creations
- Batch operations when possible
- Use appropriate data structures
- Cache frequently accessed data
4. Upgrades
- Plan for version management
- Implement migration strategies
- Use flexible data structures
- Document state changes
Common Patterns
1. Account Initialization
pub fn initialize_account<T: AccountState>(
program_id: &Pubkey,
account: &AccountInfo,
initial_state: T,
) -> ProgramResult {
// Verify account is uninitialized
if !account.data_is_empty() {
return Err(ProgramError::AccountAlreadyInitialized);
}
// Set account owner
account.set_owner(program_id)?;
// Initialize state
initial_state.save(account)?;
Ok(())
}
2. Account Updates
pub fn update_account<T: AccountState>(
account: &AccountInfo,
update_fn: impl FnOnce(&mut T) -> ProgramResult,
) -> ProgramResult {
// Load current state
let mut state = T::load(account)?;
// Apply update
update_fn(&mut state)?;
// Save updated state
state.save(account)?;
Ok(())
}
3. Account Closure
pub fn close_account(
account: &AccountInfo,
destination: &AccountInfo,
) -> ProgramResult {
// Transfer lamports
let dest_starting_lamports = destination.lamports();
**destination.lamports.borrow_mut() = dest_starting_lamports
.checked_add(account.lamports())
.ok_or(ProgramError::Overflow)?;
**account.lamports.borrow_mut() = 0;
// Clear data
account.data.borrow_mut().fill(0);
Ok(())
}
Related Topics
- UTXOs - How UTXOs integrate with accounts
- Programs - Programs that own and modify accounts
- Instructions - How to interact with accounts
Instructions and Messages
Instructions and messages are fundamental components of Arch's transaction processing system that enable communication between clients and programs. They form the basis for all state changes and interactions within the Arch network.
Instructions
An instruction is the basic unit of program execution in Arch. It contains all the information needed for a program to execute a specific operation. Instructions are processed atomically, meaning they either complete entirely or have no effect.
Structure
pub struct Instruction {
/// Program ID that executes this instruction
pub program_id: Pubkey,
/// Accounts required for this instruction
pub accounts: Vec<AccountMeta>,
/// Instruction data
pub data: Vec<u8>,
}
Components:
- Program ID: The pubkey of the program that will process the instruction
- Accounts: List of accounts required for the instruction, with their metadata
- Instruction Data: Custom data specific to the instruction, typically serialized using Borsh or another format
Account Metadata
pub struct AccountMeta {
pub pubkey: Pubkey,
pub is_signer: bool,
pub is_writable: bool,
}
pubkey
: The account's public keyis_signer
: Whether the account must sign the transactionis_writable
: Whether the account's data can be modified
Messages
A message is a collection of instructions that form a transaction. Messages ensure atomic execution of multiple instructions, meaning either all instructions succeed or none take effect.
Structure
pub struct Message {
/// List of account keys referenced by the instructions
pub account_keys: Vec<Pubkey>,
/// Recent blockhash
pub recent_blockhash: Hash,
/// List of instructions to execute
pub instructions: Vec<CompiledInstruction>,
}
Components:
- Account Keys: All unique accounts referenced across instructions
- Recent Blockhash: Used for transaction uniqueness and timeout
- Instructions: List of instructions to execute in sequence
Instruction Processing Flow:
-
Client creates an instruction with:
- Program ID to execute the instruction
- Required accounts with appropriate permissions
- Instruction-specific data (serialized parameters)
-
Instruction(s) are bundled into a message:
- Multiple instructions can be atomic
- Account permissions are consolidated
- Blockhash is included for uniqueness
-
Message is signed to create a transaction:
- All required signers must sign
- Transaction size limits apply
- Fees are calculated
-
Transaction is sent to the network:
- Validated by validators
- Processed in parallel when possible
- Results are confirmed
-
Program processes the instruction:
- Deserializes instruction data
- Validates accounts and permissions
- Executes operation
- Updates account state
Best Practices:
-
Account Validation
- Always verify account ownership
- Check account permissions
- Validate account relationships
-
Data Serialization
- Use consistent serialization format (preferably Borsh)
- Include version information
- Handle errors gracefully
- Validate data lengths
-
Error Handling
- Return specific error types
- Provide clear error messages
- Handle all edge cases
- Implement proper cleanup
Cross-Program Invocation (CPI)
Instructions can invoke other programs through CPI, enabling composability:
-
Create new instruction for target program:
- Specify program ID
- Include required accounts
- Prepare instruction data
-
Pass required accounts:
- Include all necessary accounts
- Set proper permissions
- Handle PDA derivation
-
Invoke using
invoke
orinvoke_signed
:- For regular accounts:
invoke
- For PDAs:
invoke_signed
- Handle return values
- For regular accounts:
-
Handle results:
- Check return status
- Process any returned data
- Handle errors appropriately
Security Considerations:
-
Account Verification
- Verify all account permissions
- Check ownership and signatures
- Validate account relationships
- Prevent privilege escalation
-
Data Validation
- Sanitize all input data
- Check buffer lengths
- Validate numerical ranges
- Prevent integer overflow
-
State Management
- Maintain atomic operations
- Handle partial failures
- Prevent race conditions
- Ensure consistent state
Common Patterns:
-
Initialization
- Create necessary accounts
- Set initial state
- Assign proper ownership
-
State Updates
- Validate permissions
- Update account data
- Maintain invariants
-
Account Management
- Close accounts when done
- Manage PDAs properly
Syscalls
A syscall is a function that can be used to obtain information from the underlying virtual machine.
// Used for cross-program invocation (CPI)
// Invokes a cross-program call
define_syscall!(fn sol_invoke_signed_rust(instruction_addr: *const u8, account_infos_addr: *const u8, account_infos_len: u64) -> u64);
// Sets the data to be returned for the cross-program invocation
define_syscall!(fn sol_set_return_data(data: *const u8, length: u64));
// Returns the cross-program invocation data
define_syscall!(fn sol_get_return_data(data: *mut u8, length: u64, program_id: *mut Pubkey) -> u64);
// Arch
// Validates and sets up transaction for being signed
define_syscall!(fn arch_set_transaction_to_sign(transaction_to_sign: *const TransactionToSign));
// Retrieves raw Bitcoin transaction from RPC and copies into memory buffer
define_syscall!(fn arch_get_bitcoin_tx(data: *mut u8, length: u64, txid: &[u8; 32]) -> u64);
// Retrieves the multi-sig public key and copies into memory buffer
define_syscall!(fn arch_get_network_xonly_pubkey(data: *mut u8) -> u64);
// Validates ownership of a Bitcoin UTXO against a public key
define_syscall!(fn arch_validate_utxo_ownership(utxo: *const UtxoMeta, owner: *const Pubkey) -> u64);
// Generates a Bitcoin script public key and copies into memory buffer
define_syscall!(fn arch_get_account_script_pubkey(script: *mut u8, pubkey: *const Pubkey) -> u64);
// Retrieves the latest Bitcoin block height
define_syscall!(fn arch_get_bitcoin_block_height() -> u64);
// logs
// Prints the hexidecimal representation of a string slice to stdout
define_syscall!(fn sol_log_(message: *const u8, len: u64));
// Prints 64-bit values represented as hexadecimal to stdout
define_syscall!(fn sol_log_64_(arg1: u64, arg2: u64, arg3: u64, arg4: u64, arg5: u64));
// Prints the hexidecimal representation of a public key to stdout
define_syscall!(fn sol_log_pubkey(pubkey_addr: *const u8));
// Prints the base64 representation of a data array to stdout
define_syscall!(fn sol_log_data(data: *const u8, data_len: u64));
Nodes
Let's introduce the nodes that comprise the Arch Network stack in greater detail.

All signing is coordinated by the leader. Ultimately, the leader submits signed Bitcoin transactions to the Bitcoin network following program execution.
This node represents a generic node operated by another party. It performs the validator role and has a share in the network's distributed signing key. The leader node passes transactions to validator nodes to validate and sign. After enough signatures have been collected (a threshold has been met), the leader can then submit a fully signed Bitcoin transaction to the Bitcoin network.
The validator node also runs the eBPF virtual machine and executes the transactions asynchronously alongside the other validator nodes in the network.
This validator is a lightweight server that only serves as an RPC for developers to get up and running quickly with the least amount of overhead. It simulates a single-node blockchain environment that is meant for efficient, rapid development.
Note: the Lightweight Validator node uses the same image as the Validator node though operates singularly for maximum efficiency. You can start a lightweight validator using the
cli validator start
command.
More can be read about the Arch Network architecture in our docs.
RPC
Interact with Arch nodes directly with the JSON RPC API via the HTTP methods.
HTTP Methods
Interact with Arch nodes directly with the JSON RPC API via this list of available HTTP methods.
Note: For client-side needs, use the @saturnbtcio/arch-sdk library as an interface for the RPC methods to interact with an Arch node. Alternatively, the new Arch Network CLI tool provides convenient commands to interact with the network.
Endpoint
Default port: 9002
- http://localhost:9002
Request Format:
To make a JSON-RPC request, send an HTTP POST
request with a Content-Type: application/json
header.
The JSON request data should contain 4 fields:
jsonrpc: <string>
- set to "2.0."id: <number>
- a unique client-generated identifying integer.method: <string>
- a string containing the method to be invoked.params: <array>
- a JSON array of ordered parameter values.
Response Format:
The response output will be a JSON object with the following fields:
jsonrpc: <string>
- matching the value set in the request.id: <number>
- matching the value set in the request.result: <array|boolean|number|object|string>
- requested data, success confirmation or boolean flag.
CLI Alternative
Many of these RPC methods can be accessed through the Arch Network CLI tool with simpler commands:
# Start a local validator
cli validator start
# Deploy a program
cli deploy <ELF_PATH>
# Show program information
cli show <PROGRAM_ADDRESS>
# Confirm transaction status
cli confirm <TX_ID>
# Get block information
cli get-block <BLOCK_HASH>
# Get current block height
cli get-block-height
# Log program messages in a transaction
cli log-program-messages <TX_ID>
# Get group key
cli get-group-key <PUBKEY>
# Change account owner
cli change-owner <ACCOUNT_ADDRESS> <NEW_OWNER>
For more details on the CLI tool, download the latest version from the Arch Network CLI releases page.
Processed Transaction
A processed transaction is a custom data type that contains a runtime transaction, a status, denoting the result of executing this runtime transaction, as well as a collection of Bitcoin transaction IDs.
#[derive(Clone, Debug, Deserialize, Serialize, BorshDeserialize, BorshSerialize)]
pub enum Status {
Processing,
Processed,
}
#[derive(Clone, Debug, Serialize, Deserialize, BorshSerialize, BorshDeserialize)]
pub struct ProcessedTransaction {
pub runtime_transaction: RuntimeTransaction,
pub status: Status,
pub bitcoin_txids: Vec<String>,
}
System Program
The Arch System Program is Arch's core program. This program contains a set of variants that can be thought-of as native functionality that can be used within any Arch program.
The System Program creates new accounts, assigns accounts to owning programs, marks accounts as executable, and writes data to the accounts.
In order to make calls to the System Program, the following mapping can help you point to the correct functionality.
index | method |
---|---|
0 | CreateAccount |
1 | WriteBytes |
2 | MakeExecutable |
3 | AssignOwnership |
CreateAccount
Index: 0
Create a new account.
Below, within the Instruction data
field, we find a local variable instruction_data
that contains vec![0]
, the correct index for making a call to SystemProgram::CreateAccount
.
let instruction_data = vec![0];
let instruction = Instruction {
program_id: Pubkey::system_program(),
accounts: vec![AccountMeta {
pubkey,
is_signer: true,
is_writable: true,
}],
data: instruction_data,
}
MakeExecutable
Index: 2
Sets the account as executable, marking it as a program.
Below, within the Instruction data
field, we find a local variable instruction_data
that contains vec![2]
, the correct index for making a call to SystemProgram::MakeExecutable
.
let instruction_data = vec![2];
let instruction = Instruction {
program_id: Pubkey::system_program(),
accounts: vec![AccountMeta {
pubkey,
is_signer: true,
is_writable: true,
}],
data: instruction_data,
}
We can proceed to confirm that the program is executable with read_account_info which returns an AccountInfoResult that gets parsed to obtain the is_executable
value.
assert!(
read_account_info("node_url", program_pubkey)
.unwrap()
.is_executable
);
Troubleshooting
FAQ
Resources
Bitcoin mempool and blockchain explorer
- mempool.space - Arch Regtest
- Bitcoin mempool and block explorer. This mempool.space instance monitors the regtest Bitcoin blockchain being used to run and validate all examples in this repo.
- Solana CLI
- Solana Local Development Guide