The KYVE Data Pipeline enables easy import of KYVE data into any data warehouse or destination supported by Airbyte. With the ELT format, data analysts and engineers can now confidently source KYVE data without worrying about its validity or reliability.
The sections that follow were created to help you build up a local testbed for testing KYVE's DataPipeline.
To experiment with the ELT pipeline via Airbyte, you must ensure that your PC has been configured correctly.
For Windows users, the Windows Subsystem for Linux 2 (WSL2) must be installed. You must be running Windows 10 version 2004 or higher (build 19041 or higher), or Windows 11 to use the commands in this tutorial.
Open PowerShell or the Windows Command Prompt in
administratormode, then enter the following command to install WSL2:
Note: In case the installation command returns "This application requires the Windows Subsystem for Linux Optional Component", you should run:
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux
Restart your machine after installation.
To ensure that WSL2 has been installed successfully, run the following command:
wsl -l -v
NAME STATE VERSION
* Ubuntu Running 2
After you've set up WSL2 successfully, install
Docker desktop on Windowsby following the steps here.
We will be utilizing
brew in order to install Docker Desktop and
docker-compose on Mac. In order to install
please follow the instructions listed here.
You are now ready to install Docker Desktop and
docker-compose. Run the following command on your terminal:
brew install docker docker-compose