json-rpc/docs/client_implementation_guide.md
To implement a client connecting to Diem JSON-RPC APIs, you need consider the followings:
Any JSON-RPC 2.0 client should be able to work with Diem JSON-RPC APIs. Diem JSON-RPC APIs extend to JSON-RPC 2.0 Spec for specific use case, check Diem Extensions for details, we will discuss more about them in Error Handling section.
A simplest way to validate your client works is connecting it to Testnet(https://testnet.diem.com/v1). For some query blockchain methods like get_currencies or get_metadata, you don't need anything else other than a HTTP client to get back response from server. Try out get_currencies example on Testnet, and this can be the first query blockchain API you implement for your client.
When you need a test submit transaction method, like peer to peer transfer coins, you will need accounts created for both sender and receiver. Technically it is a transaction (with creating account script) that needs to be submitted and executed, but creating account script is permitted to special accounts, and Testnet does not publish these accounts' private key, thus you can't do it on your own.
Instead, we created a service named Faucet for anyone who wants to create an account and mint coins on Testnet.
Please follow Testnet Faucet Service to implement mint coins for testing accounts, then you are ready to test submit a peer to peer transfer transaction.
All the methods prefixed with get_ listed at here are designed for querying Diem blockchain data.
You may start with implementing get_currencies, which is the simplest API that does not require any arguments and always responds to the same result on Testnet.
When you implement get_account, you can use the following 3 static account address to test the API against with Testnet:
| account name | address hex-encoded string | description |
|---|---|---|
| root account address | 0000000000000000000000000A550C18 | A special root account, stores important global resource information like all currencies info |
| core code address | 00000000000000000000000000000001 | A special code account, we will need it for submit transaction, it stores currency code type info |
| designed dealer address | 000000000000000000000000000000DD | A special account for minting coins on Testnet, checkout Testnet Faucet Service for more info |
As the above account addresses are static on Testnet, it is convenient for you to test against them for get_account method. However, if you implemented Testnet Faucet Service, you can create your own testing account for testing on Testnet.
Similarly, we can test our get_account_transaction implementation with root account address.
We need call get_account and get_account_transaction when we implement and test Submit Transaction method. So you should at least implement and confirm these two methods are working as expected.
To implement submitting a transaction, you may follow the following steps:
The following diagram shows the sequence of submit and wait for a peer to peer transaction executed successfully:
A local account holds secrets of an onchain account: the private key. Maintaining the local account or keeping the secure of private key is out of a Diem client's scope. In this guide, we use Diem Swiss Knife to generate local account keys:
# generate test keypair
cargo run -p swiss-knife -- generate-test-ed25519-keypair
{
"error_message": "",
"data": {
"diem_account_address": "a74fd7c46952c497e75afb0a7932586d",
"diem_auth_key": "459c77a38803bd53f3adee52703810e3a74fd7c46952c497e75afb0a7932586d",
"private_key": "cd9a2c90296a210249128ae3c908611637b2e00efd4986670e252abf3fabd1a9",
"public_key": "447fc3be296803c2303951c7816624c7566730a5cc6860a4a1bd3c04731569f5"
}
}
To run this by yourself, clone https://github.com/diem/diem.git, and run
./scripts/dev_setup.shto setup dev env. You can run the command in the above example at the root directory of diem codebase.
Now we have a local account address and keys, we can start to prepare a transaction. In this guide we use peer to peer transfer as an example, others will be similar except some scripts can only be submitted by specific accounts.
There are several development tools available for you:
Here we give an example of how to create and sign transactions with option 1 in Java. Please follow the guide at Transaction Builder Generator to generate code into your project.
Example: create and sign a transaction that transfers 12 Coint1 coins from account1 to account2.
ChainId testNetChainID = new ChainId((byte) 2); // Testnet chain id is static value
String currencyCode = "XUS";
String account1_address = "a74fd7c46952c497e75afb0a7932586d";
String account1_public_key = "447fc3be296803c2303951c7816624c7566730a5cc6860a4a1bd3c04731569f5";
String account1_private_key = "cd9a2c90296a210249128ae3c908611637b2e00efd4986670e252abf3fabd1a9";
String account2_address = "5b9f7691937732eedfbe4f194275247b";
long amountToTransfer = coins(12);
// step 1: create transaction script:
TypeTag currencyCodeMoveResource = new TypeTag.Struct(new StructTag(
bytesToAddress(hexToBytes("00000000000000000000000000000001")), // 0x1 is core code account address
new Identifier(currencyCode),
new Identifier(currencyCode),
new ArrayList<>()
));
Script script = Helpers.encode_peer_to_peer_with_metadata_script( // Helpers.encode_xxx is code generated by transaction builder generator
currencyCodeMoveResource,
hexToAddress(account2_address),
amountToTransfer,
new Bytes(new byte[]{}),
new Bytes(new byte[]{})
);
// step 2: get current submitting transaction account sequence number.
Account account1Data = client.getAccount(account1_address);
// step 3: create RawTransaction
RawTransaction rt = new RawTransaction(
hexToAddress(account1_address),
account1Data.sequence_number,
new TransactionPayload.Script(script),
coins(1), // maxGasAmount
0L, // gasUnitPrice, you can always set gas unit price to zero on Testnet. At launch, gas unit price can be zero in most of time. Only during high congestion, you may specify a gas price.
currencyCode,
System.currentTimeMillis()/1000 + 30, // expirationTimestampSecs, expire after 30 seconds
testNetChainID
);
byte[] rawTxnBytes = toBCS(rt);
You can find imports and util functions code here.
The following code does signing transaction:
// sha3 hash "DIEM::RawTransaction" bytes first, then concat with raw transaction bytes to create a message for signing.
byte[] hash = concat(sha3Hash("DIEM::RawTransaction".getBytes()), rawTxnBytes);
// [bouncycastle](https://www.bouncycastle.org/)'s Ed25519Signer
Ed25519Signer signer = new Ed25519Signer();
byte[] privateKeyBytes = hexToBytes(account1_private_key);
signer.init(true, new Ed25519PrivateKeyParameters(privateKeyBytes, 0));
signer.update(hash, 0, hash.length);
byte[] sign = signer.generateSignature();
SignedTransaction st = new SignedTransaction(rt, new TransactionAuthenticator.Ed25519(
new Ed25519PublicKey(new Bytes(hexToBytes(account1_public_key))),
new Ed25519Signature(new Bytes(sign))
));
String signedTxnData = bytesToHex(toBCS(st));
For more details related to Diem crypto, please checkout Crypto Spec.
When you implement above logic, you may extract createRawTransaction and createSignedTransaction methods and use the following data to confirm their logic is correct:
import com.novi.bcs.BcsSerializer;
import com.novi.serde.Bytes;
import com.novi.serde.Serializer;
import org.bouncycastle.crypto.params.Ed25519PrivateKeyParameters;
import org.bouncycastle.crypto.signers.Ed25519Signer;
import org.diem.stdlib.Helpers;
import org.diem.types.*;
import java.io.IOException;
import java.util.ArrayList;
// ......
public static long coins(long n) {
return n * 1000000;
}
public static AccountAddress hexToAddress(String hex) {
return bytesToAddress(hexToBytes(hex));
}
static AccountAddress bytesToAddress(byte[] values) {
assert values.length == 16;
Byte[] address = new Byte[16];
for (int i = 0; i < 16; i++) {
address[i] = Byte.valueOf(values[i]);
}
return new AccountAddress(address);
}
public static byte[] hexToBytes(String hex) {
return BaseEncoding.base16().decode(hex.toUpperCase());
}
public static String bytesToHex(byte[] bytes) {
return BaseEncoding.base16().encode(bytes);
}
public static String bytesToHex(Bytes bytes) {
return bytesToHex(bytes.content());
}
public static byte[] toBCS(RawTransaction rt) throws Exception {
Serializer serializer = new BcsSerializer();
rt.serialize(serializer);
return serializer.get_bytes();
}
public static byte[] toBCS(SignedTransaction rt) throws Exception {
Serializer serializer = new BcsSerializer();
rt.serialize(serializer);
return serializer.get_bytes();
}
public static byte[] sha3Hash(byte[] data) {
SHA3.DigestSHA3 digestSHA3 = new SHA3.Digest256();
return digestSHA3.digest(data);
}
public static byte[] concat(byte[] part1, byte[] part2) {
byte[] ret = new byte[part1.length + part2.length];
System.arraycopy(part1, 0, ret, 0, part1.length);
System.arraycopy(part2, 0, ret, part1.length, part2.length);
return ret;
}
public static String addressToHex(AccountAddress address) {
byte[] bytes = new byte[16];
for (int i = 0; i < 16; i++) {
bytes[i] = Byte.valueOf(address.value[i]);
}
return bytesToHex(bytes);
}
After extracting out creating and signing transaction logic, the JSON-RPC submit method API itself is simple with hex-encoded signed transaction serialized string.
Assuming we have the API implemented by client like other get methods, we call submit with the signedTxnData to send the transaction to the server.
client.submit(signedTxnData);
After the transaction is submitted successfully, we need to wait for it to be executed and validate the execution result.
We can call get_account_transaction to find the transaction by account address and sequence number. If transaction has not been executed yet, server responses null:
public Transaction waitForTransaction(String address, @Unsigned long sequence, String transactionHash,
@Unsigned long expirationTimeSec, int timeout) throws DiemException {
Calendar calendar = Calendar.getInstance();
calendar.add(Calendar.SECOND, timeout);
Date maxTime = calendar.getTime();
while (Calendar.getInstance().getTime().before(maxTime)) {
Transaction accountTransaction = getAccountTransaction(address, sequence, true);
if (accountTransaction != null) {
if (!accountTransaction.getHash().equalsIgnoreCase(transactionHash)) {
throw new DiemException(
String.format("found transaction, but hash does not match, given %s, but got %s",
transactionHash, accountTransaction.getHash()));
}
if (!txn.getVmStatus() != null && "executed".equalsIgnoreCase(accountTransaction.getVmStatus().getType())) {
throw new DiemTransactionExecutionFailedException(
String.format("transaction execution failed: %s", accountTransaction.getVmStatus()));
}
return accountTransaction;
}
try {
Thread.sleep(200);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
throw new DiemWaitForTransactionTimeoutException(
String.format("transaction not found within timeout period: %d (seconds)", timeout));
}
In above example code, we do the following 2 validations after the transaction showed up:
SignedTransaction. This makes sure the transaction we got is the one we submitted.We also should have a wait timeout for the case if the transaction is dropped some where for unknown reason.
There are four general types errors you need consider:
Distinguish above four types errors can help application developer to decide what to do with each different type error:
Other than general error handling, another type of error that client / application should pay attention to is server side stale response. This type problem happens when a Full Node is out of sync with the Diem network, or you connected to a sync delayed Full Node in a cluster of Full Nodes. To prevent these problems, we need:
diem_ledger_version and diem_ledger_timestampusec (see Diem Extensions) for clients to validate and track server side data freshness.Once the above basic function works, you have a minimum client ready for usage. To make a production quality client, please checkout our Client CHECKLIST.