Skip to main content

KSYNC

# KSYNC

Rapidly sync validated blocks and snapshots from KYVE to every Tendermint based Blockchain Application

Installation

To install the latest version of KSYNC, run the following command:

go install github.com/KYVENetwork/ksync/cmd/ksync@latest

To verify the installation simply run ksync version. To build from source visit the repository on GitHub.

What is KSYNC?

KSYNC is a tool capable of syncing blocks and state-sync snapshots from the decentralized KYVE data lake directly into Cosmos blockchain nodes. With KSYNC Cosmos validators don't need to wait for peers in order to block-sync and they don't need to search for trusted app hashes if they want to state-sync. Furthermore, state-syncing to historical heights up to genesis are possible.

Why KSYNC?

There is no incentive to keep historical blockchain data in Cosmos, therefore finding peers with historical blocks is difficult. Furthermore state-sync snapshots, which promise syncs in minutes, sometimes simply do not work due to other nodes not having them enabled or p2p connections breaking. Additionally it is only possible to state-sync to the current live height.

  • KSYNC can block-sync cosmos chains from genesis up to live height, which might be the only option for archival node runners in the future since historical blocks are harder each day to come by.
  • KSYNC can state-sync cosmos chains from genesis up to live height for certain intervals (usually not bigger than 10,000). State-syncing from historical heights was impossible before and could only be accomplished with centralized providers having huge backups of the entire data directory.
  • KSYNC can checkout any blockheight from genesis up to live height within minutes with height-sync (combination of state- and block-sync). This was only possible with huge efforts and resources before.

Usage

Depending on what you want to achieve with KSYNC there are three sync modes available. A quick summary of what they do and when to use them can be found below:

DescriptionRecommendation
BLOCK-SYNCSyncs blocks from the node's current height up to a specified target height.Generally recommended for archival node runners, who want to have a full node containing all blocks.
STATE-SYNCApplies a state-sync snapshot to the node. After the snapshot is applied, the node can continue block-syncing from the applied snapshot height.Generally recommended for new node runners, who want to join a network in minutes without wanting to sync the entire blockchain.
HEIGHT-SYNCFinds the quickest way out of state-sync and height-sync to get to the specified target height.Generally recommended for users who want to check out a historical state within minutes at the specified target height for analysis.

Check availability

In order to use KSYNC, the data has to be made available by KYVE in the first place. To get an overview of all supported sources simply run:

ksync info

This provides an overview of all chains that are supported by KSYNC using the validated Mainnet data. The listed source name can be used as --source=<source-name> in several commands.

To get an overview of all supported chains using the validated data of the KYVE Testnet Kaon, simply run:

ksync info --chain-id="kaon-1"

BLOCK-SYNC

Syncing to latest available height

Depending on your current node height (can be also 0 if you start syncing from genesis) you can sync up to the latest height available by the storage pool. KSYNC will automatically exit once that height is reached.

ksync block-sync --binary="/path/to/<binaryd>" --source=<source-name>

Syncing to specified target height

Depending on your current node height (can be also 0 if you start syncing from genesis) you can sync up to your desired target height. KSYNC will automatically exit once that height is reached.

ksync block-sync --binary="/path/to/<binaryd>" --source=<source-name> --target-height=<height>

Syncing from genesis with Cosmovisor

If you want to sync from genesis it is often recommended to sync with Cosmovisor so you don't have to manually upgrade everytime. For this reason KSYNC also supports the Cosmovisor. You can simply instead of using the binaryd use the cosmovisor as binary.

ksync block-sync --binary="/path/to/cosmovisor" --source=<source-name>

It is important to note that this command alone will not automatically upgrade, KSYNC will always exit once an upgrade is reached. To automatically upgrade you need to specify a systemd process in order to restart the process. During the restart cosmovisor will pick the upgrade binary and KSYNC can continue to block-sync. The systemd config file should look like this:

[Unit]
Description=KSYNC deamon supervising the ksync sync process
After=network-online.target

[Service]
User=ec2-user
WorkingDirectory=$HOME
ExecStart=$HOME/ksync block-sync --binary="/path/to/cosmovisor" --source=<source> -y
Restart=always
RestartSec=10s
LimitNOFILE=infinity
Environment="DAEMON_NAME=<binary>"
Environment="DAEMON_HOME=$HOME/.<binary>"
Environment="DAEMON_ALLOW_DOWNLOAD_BINARIES=false"
Environment="DAEMON_RESTART_AFTER_UPGRADE=false"
Environment="DAEMON_LOG_BUFFER_SIZE=512"
Environment="UNSAFE_SKIP_BACKUP=true"

[Install]
WantedBy=multi-user.target

Example

Sync Archway from genesis up to live height and upgrade automatically.

To state-sync Archway you have to download and set up the correct Archway binary. To sync from genesis the version v1.0.1 has to be used. You can download them here or build them from source: https://github.com/archway-network/archway

Verify installation with:

./archwayd version
1.0.1

After the installation, init the config:

./archwayd init <your-moniker> --chain-id archway-1

Download the genesis:

wget -qO- https://github.com/archway-network/networks/raw/main/archway/genesis/genesis.json.gz | zcat > ~/.archway/config/genesis.json

Install or download the Cosmovisor binary from here and create the following directories:

mkdir -p "${HOME}"/.archway/cosmovisor/genesis/bin
mkdir "${HOME}"/.archway/cosmovisor/upgrades

Then copy the archwayd binary to the genesis/bin folder:

cp "${GOPATH}"/bin/archwayd "${HOME}"/.archway/cosmovisor/genesis/bin

Furthermore, you can already create and download the upgrade binaries like this:

mkdir -p "${HOME}"/.archway/cosmovisor/upgrades/v2.0.0/bin
cp "${GOPATH}"/bin/archwayd_v2 "${HOME}"/.archway/cosmovisor/upgrades/v2.0.0/bin

mkdir -p "${HOME}"/.archway/cosmovisor/upgrades/v4.0.0/bin
cp "${GOPATH}"/bin/archwayd_v4 "${HOME}"/.archway/cosmovisor/upgrades/v4.0.0/bin

Now you can create the systemd service file with:

sudo nano /etc/systemd/system/ksync.service

and add the following content by making sure to change the <your-user>, <path-to-ksync>, <path-to-archway> and <path-to-cosmovisor> with your values:

[Unit]
Description=ksync
After=network-online.target

[Service]
User=<your-user>
ExecStart=/<path-to-ksync>/ksync block-sync --binary="<path-to-cosmovisor>/cosmovisor" --source=archway -y
Restart=always
RestartSec=3
LimitNOFILE=4096
Environment="DAEMON_NAME=archwayd"
Environment="DAEMON_HOME=/<path-to-archway>/.archway"
Environment="DAEMON_ALLOW_DOWNLOAD_BINARIES=false"
Environment="DAEMON_RESTART_AFTER_UPGRADE=false"
Environment="DAEMON_LOG_BUFFER_SIZE=512"
Environment="UNSAFE_SKIP_BACKUP=true"

[Install]
WantedBy=multi-user.target

You can now reload the systemctl daemon:

sudo -S systemctl daemon-reload

and enable KSYNC as a service:

sudo -S systemctl enable ksync

You can now start KSYNC by executing:

sudo systemctl start ksync

Make sure to check that the service is running by executing:

sudo journalctl -u ksync -f

STATE-SYNC

Syncing to latest available snapshot height

You can state-sync a node if it has no height (either node has to be just initialized or reset with ksync unsafe-reset-all) to the latest available snapshot archived by the pool with the following command. If the storage pool has synced with the live height this can be used to rapidly join this network.

ksync state-sync --binary="/path/to/<binaryd>" --source=<source-name>

Syncing to specified snapshot height

You can state-sync a node if it has no height (either node has to be just initialized or reset with ksync unsafe-reset-all) to your desired target height. The target height has to be the exact height of the archived snapshot. If the specified height can not be found it prints out the nearest available snapshot height you can use.

ksync state-sync --binary="/path/to/<binaryd>" --source=<source-name> --target-height=<height>

Example

Use state-sync to sync your Archway node with validated KYVE data to height 135,000:

To state-sync Archway you have to download and set up the correct Archway binary. To sync from genesis the version v1.0.1 has to be used. You can download them here or build them from source: https://github.com/archway-network/archway

Verify installation with:

./archwayd version
1.0.1

After the installation, init the config:

./archwayd init <your-moniker> --chain-id archway-1

Download the genesis:

wget -qO- https://github.com/archway-network/networks/raw/main/archway/genesis/genesis.json.gz | zcat > ~/.archway/config/genesis.json

Now that the binary is properly installed, KSYNC can already be started:

ksync state-sync --binary="/path/to/archwayd" --source="archway" --target-height=135000

HEIGHT-SYNC

Syncing to latest available block height

You can height-sync a node if it has no height (either node has to be just initialized or reset with ksync unsafe-reset-all) to the latest available height. This is especially useful for joining a new network if the user wants to join as quick as possible.

ksync height-sync --binary="/path/to/<binaryd>" --source=<source-name>

Syncing to specified target height

You can height-sync a node if it has no height (either node has to be just initialized or reset with ksync unsafe-reset-all) to your desired target height. The target height can be any height (but the block data pool must have archived it), then it will use available state-sync snapshots and block-sync to get to the target height as quickly as possible

ksync height-sync --binary="/path/to/<binaryd>" --source=<source-name> --target-height=<height>

Example

Use height-sync to sync your Archway node with validated KYVE data to height 249,238:

To height-sync Archway you have to download and set up the correct Archway binary. To sync from genesis the version v1.0.1 has to be used. You can download them here or build them from source: https://github.com/archway-network/archway

Verify installation with:

./archwayd version
1.0.1

After the installation, init the config:

./archwayd init <your-moniker> --chain-id archway-1

Download the genesis:

wget -qO- https://github.com/archway-network/networks/raw/main/archway/genesis/genesis.json.gz | zcat > ~/.archway/config/genesis.json

Now that the binary is properly installed, KSYNC can already be started:

ksync state-sync --binary="/path/to/archwayd" --source="archway" --target-height=249238

Settings

Metrics

You can enable useful metrics through the --metrics flag for all syncing commands. By default, it's exposed on http://localhost:8080/metrics and you can specify a custom port with --metrics-port.

The exposed metrics include the following information:

{
"latest_block_hash": "A6C59D5F7487B95B32B71EB97F8FE0EE7BE7B512044FC53B6C4A706594167AF9",
"latest_app_hash": "6BF3787314EC5C1B8FF08334193A31EF562CFE6700C3E6B604C31FD053F7FAF4",
"latest_block_height": "180",
"latest_block_time": "2021-06-18T22:03:40.861352885Z",
"earliest_block_hash": "C8DC787FAAE0941EF05C75C3AECCF04B85DFB1D4A8D054A463F323B0D9459719",
"earliest_app_hash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855",
"earliest_block_height": "1",
"earliest_block_time": "2021-06-18T17:00:00Z",
"catching_up": true
}

Overwrite default home

KSYNC uses by default the default home path of the binary. If the home directory lies under another path, every command takes the --home path, in order to overwrite the default path.

Example

Use another home path for the binary:

ksync block-sync --binary="/path/to/osmosisd" --home="/my/custom/home/.osmosis" --source="osmosis" --target-height=42000

Overwrite chain endpoint

KSYNC retrieves the metadata including checksums from the KYVE blockchain. By default, KSYNC uses the following endpoints:

These can be changed by adding the flag --chain-rest.

tip

You can find all official KYVE chain endpoints here.

Example

Use the endpoint of your own KYVE node:

ksync block-sync --binary="/path/to/osmosisd" --chain-rest="https://api-us-1.kyve.network" --source="osmosis" --target-height=42000

Overwrite storage provider endpoint

For KSYNC to actually sync blocks and snapshots the data has to be retrieved from the storage provider. By default, KSYNC uses the following endpoints:

These can be changed by adding the flag --storage-rest.

tip

You can check which storage providers KSYNC uses by visiting KYVE webapp. There the storage provider is listed for each pool.

Example

Use another gateway for Bundlr:

ksync block-sync --binary="/path/to/osmosisd" --storage-rest="https://gateway.irys.xyz" --source="osmosis" --target-height=42000

Overwrite Pool IDs

To use specific Pool IDs for any kind of syncing process, replace --source=<source-name> with block-pool-id=<id> and/or --snapshot-pool-id=<id>.

Example

Use Archway Pool IDs for HEIGHT-SYNC:

ksync height-sync --binary="/path/to/archwayd" --block-pool-id=2 --snapshot-pool-id=4

Backups

Even with the right setup and careful maintenance, it's possible to encounter app-hash errors or other unexpected problems that can lead to node collisions and resyncs from Genesis. Especially when you're dealing with syncing an archival node, it's a good idea to create periodic backups of the node's data.

KSYNC offers precisely this option for creating backups, which is generally recommended for archival node runners using block-sync. There are two different methods to utilize this:

1. BLOCK-SYNC-Backups

With block-sync, nodes can be synced by KSYNC from any height up to the latest height available by the storage pool. Backups can be created automatically at an interval, with the following parameters:

--binary               string   'path to binary (e.g. ~/osmosisd)'
--backup-interval int 'block interval to write backups of data directory (set 0 to disable backups)'
--backup-keep-recent int 'number of latest backups to be keep (0 to keep all backups)'
--backup-compression string 'compression type used for backups ("tar.gz","zip"), if not compression given the backup will be stored uncompressed'
--backup-dest string 'path where backups should be stored [default = ~/.ksync/backups]'

When the specified backup-interval is reached (height % backup-interval = 0), KSYNC temporarily pauses the sync process and creates a backup. These backups are duplicates of the node's data directory (e.g. ~/.osmosisd/data). If compression is enabled (e.g. using --backup-compression="tar.gz"), the backup is compressed and the original uncompressed version is deleted after successful compression in a parallel process.

Usage

Because backups are disabled by default, it's only required to set backup-interval, whereas the other flags are optional. Since the creation of a backup takes steadily longer as the data size grows, it is recommended to choose an interval of more than 20000 blocks.

Example command to run block-sync with compressed backups:

ksync block-sync --binary="/path/to/<binaryd>" --source=<source-name> --target-height=<height>
--backup-interval=50000 --backup-compression="tar.gz"

2. Backup-Command

The backup functionality can of course also be used with a standalone command. In this case everything runs in one process where the following flags can be used:

--binary               string   'path to binary (e.g. ~/osmosisd)'
--backup-keep-recent int 'number of latest backups to be keep (0 to keep all backups)'
--backup-compression string 'compression type used for backups ("tar.gz","zip"), if not compression given the backup will be stored uncompressed'
--backup-dest string 'path where backups should be stored [default = ~/.ksync/backups]'
Usage
ksync backup --binary="/path/to/osmosisd" --compression="tar.gz"

For KYVE Protocol Validators

This section includes all commands used by KYVE Protocol Validators to participate in state-sync data pools.

info

If you are not a KYVE protocol validator or do not intend to run as a KYVE protocol validator you can skip this section

SERVE-SNAPSHOTS

This command is essential for running as a protocol node in a state-sync pool since this will serve the snapshots to the protocol node. Basically, KSYNC will sync the blocks with block-sync and waits for the ABCI app to create the snapshots, once created they are exposed over a REST API server which the protocol node can then query.

To start with default settings serve the snapshots with:

ksync serve-snapshots --binary="/path/to/<binaryd>" --source=<source-name>

Once you see that KSYNC is syncing blocks you can open https://localhost:7878/list_snapshots. In the beginning it should return an empty array, but after the first snapshot height is reached (check the interval in the data pool settings) you should see a first snapshot object in the response.

Changing snapshot api server port

You can change the snapshot api server port with the flag --snapshot-port=<port>

Enabling metrics server and manage port

You can enable a metrics server running by default on http://localhost:8080/metrics by add the flag --metrics. Furthermore, can you change the port of the metrics server by adding the flag --metrics-port=<port>

Manage pruning

By default, pruning is enabled. That means that all blocks, states and snapshots prior to the snapshot pool height are automatically, deleted, saving a lot of disk space. If you want to disable it add the flag --pruning=false.

tip

If you want to keep all of your snapshots but still want to prune everything else you can run with --pruning and --keep-snapshots

Contribution & Low-level technical documentation

You can find the source code of KSYNC, together with contribution guidelines and a detailed technical documentation in the README of KSYNC's repository here: https://github.com/KYVENetwork/ksync