This project demonstrates how to use solana rust APIs to write solana on-chain program(aka contract) and off-chain client program in rust via a simple counter example.
The project comprises of:
- An on-chain counter program
- A rust client that can send a "Increament" counter message to the on-chain program and get the current reading.
The following dependencies are required to build and run this example:
- Install Rust v1.60.0 or later from https://rustup.rs/
- Install Solana v1.10.8 or later from https://docs.solana.com/cli/install-solana-cli-tools
If this is your first time using Rust, these Installation Notes might be helpful.
If you're on Windows, it is recommended to use WSL to run these commands
- Set CLI config url to localhost cluster
solana config set --url localhost
or
solana config set -ul
- Create CLI Keypair
If this is your first time using the Solana CLI, you will need to generate a new keypair:
solana-keygen newThis example connects to a local Solana cluster by default.
Start a local Solana cluster:
solana-test-validatorIf you want start with a clean slate after couple of trials, you can do:
solana-test-validator --resetNote: You may need to do some system tuning (and restart your computer) to get the validator to run
On-chain deployed program's logs can be viewed by launcing a separate terminal and firing the following command:
solana logsNote: For logging messages inside the on-chain program, we should use the
msg!macro.
Go inside the 'solana_counter_program' directory if not already done:
cd solana_counter_program
cargo build-bpfsolana program deploy target/deploy/program.socargo runValues will differ!
Connecting to cluster...http://localhost:8899
Connection to cluster established
Cluster node solana version 1.10.5
Counter account B6rWFbQ4pmb4pvcZstFCjLXffZSaqqn6c8fdXzpK3WSX already exists. Owner program: HGsPi7r4MEeUSC74vzx9qCqJvuuBb3AcjNc5MrtEjCGu
Binary address 8cRrhLjJ7sSbSa1kuaShq2Ywu1otyRhkNwTQ3E1Bqr4T
Fee for message 5000
Counter value 1
- Ensure you've started the local cluster, build the on-chain program and deployed the program to the cluster.
solana config set --url d
solana program deploy target/deploy/program.sosolana config set -ut
solana program deploy target/deploy/program.soNote: You may not have required SOL balance to deploy and run transactions in devnet or testnet. To request SOL into your account do an airdrop:
solana balancesolana airdrop 1cargo runThe following image shows the project layout. We are making use of cargo workspaces.
- program - this is the on-chain counter program
- client - this is the rust client program that invokes the program to increament the counter.
- common - this crate contains the enum/structs shared by both program and client.
For experimentation, tweaking files under the program folder would require rebuild and redeployment.
Now when you rerun cargo run, you should see the results of your changes.
The client is a rust cli program with a main function.
The main function does following five things:
Client
creates an instance of RpcClient in its get_rpc_client methhod. This sets up a Http client to the solana network that is picked up from ~/.config/solana/cli/config.yml. Once the client has been setup - we can start interacting with solana network for things like querying about accounts, sending transactions, getting cluster related information and much more. The exhaustive list can be found here.
The json_rpc_url entry in the config.yaml(/.config/solana/cli/config.yml) file gets configured via the following command:
solana config set --url localhost[devnet, testnet etc]Solana on-chain programs are stateless and immutable(which is different from upgradable - we can keep modifying and deploying a program again and again so long as we don't supply the --final flag to solana program deploy program.so or don't use solana deploy program.so - which sets up 'BPFLoader2111111111111111111111111111111111' as program owner instead of BPFLoaderUpgradeab1e11111111111111111111111 and does not allow us to upgrade the program further unless we specify different program address). Also, looking at the program crate_type, we see that crate-type is "cdylib", "lib". We can ommit the "lib" type that would work just fine. "cdylib" produces a .so file in linux and .dll file in windows. These are shared libraries - they do not maintain state across invocations! Where then we store state in solana? In accounts. If a program in solana wants to persist state, it would have to make use of accounts that it owns.
Also, programs themselves are stored in accounts - they are marked as executable. For more information about account see here.
Note: There is limit to how much storage space(currently 10MB) an account can have. Space incurs cost. Incurred cost is paid via rent. An account can be rent exempt if it maintains atleast two years worth of rent as balance in its account. See more here. On-chain programs are expected to be rent exempt otherwise they would be purged from the chain. Amount of lamports required for an account to be rent exempt can be calculated programmatically or via the cli as shown:
solana rent 1000 [in bytes]On entry to the account setup process, we retrieve the payer pubkey(i.e. pubkey from ~/.config/solana/id.json), then look for the program id(pubkey from ./target/deploy/program-keypair.json). If the program has not been built - account set up would fail fast.
Next, we construct the counter account pubkey based on payer pubkey, seed and the program id(owner of the account) and make a rpc call to the chain to retrieve the account. Successful retrieval of the account results in early exit from this call because required counter account already exists and we have nothing to setup.
We proceed to setup the account if it does not already exist. We calculate the minimum balance that would be required for the counter account to stay rent exempt based on the how much space it would maintain in its data field. This data field will hold the serialzed bytes of Counter struct. This struct has a count field of type u64 - which is 8 bytes long. The count struct derives borsh BorshSerialize and BorshDeserialize traits here - rendering it capable of being serialized to and from a byte slice. We calculate size of a Counter struct here.
We fetch minimum required lamports balance for the counter account here. This amount would be deducted from the payer's account when we execute the create account transaction later.
After this - we proceed to construct the system instruction for creating the counter account in this section. We pass the lamports amount, space and owner(program id) along with other relevant fields.
Next, we query the latest blockhash from the solana network. This is a measure of how long ago the client has seen the network state and used by the network to accept/reject transaction.
We query network again to find out the required fee for the transaction message - this is the amount for executing transaction on the network passsing the message and the blockhash retrieved in the previous step.
We sum up the minimum rent exemption lamports and transaction cost(fee_for_message) and do ourselves a lamports airdrop. Airdrop request would not hit the network if we pass an environment variable named 'skip_airdrop' with value set to some non-empty value(for experimentation!) or the payer account has sufficient lamports to provide for the transaction cost and minimum rent exemption amount required for the counter account to stay afloat(aka rent free!).
At the end, we send our account setup transaction across to the network and We get back a transaction signature! We can make use of the signature to find out the transaction status, if we want. We are ignoring the returned signature here(Have not seen it fail & retry would muddy this learning exercise!).
Deployment verification starts by checking for the existence program keypair that must have been generated at the program build phase. If keypair can not be found - the program exits with appropriate error message.
We try to retrieve the program account corresponding to the pubkey of the program keypair - here the intent being two pronged - to verify that the program has been deployed to the chain and it, indeed, is executable.
Now here is a catch - we can load the program account and check for executable flag on it and decide whether to proceed further or not. But this alone is not sufficient - because programs owned by the upgradable bpf loader maybe closed(solana program close program_id) - and it will still report the program as being executable.
This is not the case with bpf loader - It does not allow closing a deployed program.
Programs owned by upgradable loader store their executable bits in a seprate account which can be seen below:
We can query the program data account (underlined red in the image) and it will spit out a huge pile of hexadecimal numbers. When we close a program - it is this program data account that gets wiped out - but program account still says it is executable - which is not very helpful. That is why we try to retrieve the program data account where actual program byte codes are stored - in the case that program is owned by upgradeable bpf loader and deployed on-chain program may have been closed.
Note:
solana program deploy program.so- deployes to upgradable loader andsolana deploy program.so- deploys to bpf loader. Programs owned by bpf loader are are not upgradeable and store progrm byte code in the program account itself.
Here we submit a transaction to our on-chain counter program to increament the counter value that is maintained in its owned account.
Usual steps like loading payer keypair, program id, querying for latest blockhash and fee for message etc happen in appropriate places - but one thing to note here is that we are packing an enum defined here with the instruction.
To invoke a solana on-chain program - we send a Transaction, which contains a message and the message encaptulates one or more instructions within it. We see that instruction construct has a data field within it - which is a Vec of bytes. We can send any data specific to our program so long as the program knows how to deserialize and handle it - solana runtime is agnostic about the format of data that an instruction carries but it exposes useful APIs for constructing instructions from both borsh and bincode serializable types. Borsh is preferred because of its stable specification. Bincode is mentioned as being computationally expensive and not to have had a published spec in solana documentations but now bincode encoding spec can be found here.
As said - solana runtime does not care what data we pack inside an instruction as long as our on-chain program is able to deserialize and decipher it. It is not mandatory to use borsh or bincode - we can,very well, invent our own serialization mechanism if we want and make use of this API to construct an instruction, embed it in a message, submit a transaction that carries the message to the network. During execution, solana runtime will faithfully make available the packed data in the instruction to the program that the instruction was created for - in the form of byte array.
In any case, we did not wnat to re-invent the wheel instead use borsh serialization and deserialization here and here. We pack our application specific custom data(which is an enum with just one variant) here and solana runtime makes that data available to our program here and we reconstruct our enum variant here. Its also mandatory that we pass along accounts that our program reads or modifies during its execution. Our program increaments the counter value in the counter account that it owns. Hence we pass that information in a AccountMeta struct marking that as writable. Passing accounts that an on-chain program touches during its execution lets solana runtime parallelize transactions leading to faster execution time.
Note: This line is commented out. Its clones the instruction and packs it twice inside the message. What will happen if we uncomment this line and comment out the above line? Check that out!
Each time we run our client program - it increaments the count field inside the counter account owned by our on-chain program. We load the counter account here - deserialize the data field of the account into Counter struct and print out the count fields value.
To write an on-chain solana program - primarily we need to follow these steps:
- Provide a function whose type signature matches this. Here
program_idin the function signature is, of course, program pubkey. We can change this program id to some other id of our liking(Usingsolana-keygen grindto generate a vanity keypair and passing that as--program-idduring deployment). The second parameteraccounts- is the consolidated shared list of all the accounts that all instructions(embedded inside the message that a transaction contains) read and/or write to. They appear in the order that AccountMeta structs are added to an instruction and the order that instructions are added to the containing message. All accounts(the type AccountInfo that appears in the function signature - is a runtime construct - it does not stay physically in disc) should all have their writable/readable/signer attributes set. In our case, while we are constructing the counter accountAccountMetahere - we are saying that its not a singer but writable because the program writes to it while increamenting the counter value(AccountMeta::new- creates a writable account). - Decorate the implementation with the entrypoint macro. As we can see - net effect of invoking this macro is that our implemention function get embedded inside an external
cfunction calledentrypoint. Also, this externalcfunction has gotno_mangleannotation defined - which means compiler will keep its name as it is. - Define
no-entrypointfeature - During on-chain program development we might depend on other crates for many useful APIs that they might provide. But they might have their own entrypoints as we do. So there is the issue of entrypoint collisions. Since there can not be multiple entrypoints at runtime - we need to take care to exclude or include entrypoint sections as needed during the compilation phase. Wich is why we define the entrypoint feature here. If, let say, someone is developing there own on-chain program and wants use some API from our crate - but wants to exclude our entrypoint - they would add a sectionprogram = { version = "0.1.0", path = "../program",features = [ "no-entrypoint" ] }to their Cargo.toml. Read more here. - Build the on-chain program- During the build process the program is compiled to Berkeley Packet Filter(BPF) bytecode and stored as an Executable and Linkable Format ELF shared object locally. A program keypair is also generated - this generated keypair's pubkey becomes the default program_id.
- Deploy the program to the network - Solana CLI breaks up compiled program byte code into smaller chunks(due to restricted transaction size) and sends the chunks to an intermediate on-chain buffer account in a series of transactions. Once transmission is complete and verified, a final transaction instruction moves the intermediate buffered content to program's data account. This completes a new deployment or a program upgrade. As usual, transaction costs are deducted from payer's account. See this excellent post for more info.
End note: If we start the validator in a clean state(
solana-test-validator --reset) and run - for _ in {0..99}; do cargo run; done - from two terminals - we must observe counter value as 200. But we don't get! What gives?
Hint: Check all the entries in ~/.config/solana/cli/config.yml
End note: If we use
solana deploy program.so[path to .so]to deploy our program - then the deployed program is owned by the bpf loader. We get a randomly generated program id. If that is the case - while running the client program - we need to pass an environment variable namedprogram_idwith value set generated id. Refer here.