Welcome back to our "Build & deploy" series! Here’s a quick recap of what we’ve covered so far:
- Build & deploy 1: Installation, first project and environment.
- Build & deploy 2: Develop your first pipelines in a workflow.
- Build & deploy 3: Manage your Apache Hop project with the Git integration.
- Build & deploy 4: Upgrade Apache Hop version without losing your configurations
In this post, we'll revisit the steps we covered in the first post of the "Build & deploy" series—creating a project and setting up an environment—but this time, we'll be using the Hop Conf command-line tool instead of the Hop GUI.
What you’ll learn in this post
- How to install Apache Hop on your preferred platform.
- Creating and structuring your first Apache Hop project using the Hop Conf tool.
- Setting up and managing your first environment using the Hop Conf tool.
Let’s get started!
Apache Hop
Apache Hop is designed to be easy to set up across various operating systems, whether you're on Windows, Linux, or macOS.
Supported operating systems
- Windows 7 or higher
- Linux (x86_64, ARM)
- macOS
The only prerequisite is having a Java Runtime Environment installed.
Suggested Java runtimes
- Oracle Java Runtime
- Microsoft OpenJDK
- OpenJDK
Steps to install Apache Hop
- Download Apache Hop: Head over to the Apache Hop website and download the latest version.
- Unzip the file: Once downloaded, unzip the file to a location of your choice.
- Navigate to the directory: Navigate to your newly unzipped Apache Hop directory. Here, you'll find everything you need to start using Hop.
Command overview
- hop-gui.bat (Windows) / hop-gui.sh (Linux, MacOS): Launches the Hop graphical interface.
hop-gui.bat
./hop-gui.sh
- hop-run.bat (Windows) / hop-run.sh (Linux, MacOS): Runs workflows or pipelines from the command line.
hop-run.bat
./hop-run.sh
- hop-server.bat (Windows) / hop-server.sh (Linux, MacOS): Starts a Hop Server instance.
hop-server.bat
./hop-server.sh
Creating a project via Hop Conf
But in this video, we’ll be creating and configuring our project using the Hop Conf command line tool, similar to how you would in the GUI.
Command to create a project
First, we’ll create a new project. For this, open your terminal and run the following command:
./hop-conf.sh --project-create --project=my-hop-project \
--project-home=/path/to/your/project
This command creates a project named "my-hop-project".
You should get an output message like this.
Creating project 'project'
Project 'project' was created for home folder:
/path/to/your/project
Configuration file for project 'my-hop-project' was saved to:
file:/path/to/your/project/project-config.json
Creating an environment via Hop Conf
Next, let’s set up our first environment. An environment in Apache Hop represents the stage of your project, such as Development or Testing.
Command to create an environment
To create an environment using Hop Conf, use this command:
./hop-conf.sh --environment-create \
--environment dev-env \
--environment-project my-hop-project \
--environment-purpose Development \
--environment-config-files="/path/to/your/env-variables.json"
To add a second config file:
./hop-conf.sh --environment-modify \
--environment dev-env \
--environment-project my-hop-project \
--environment-purpose Development \
--environment-config-files="/path/to/your/env-variables2.json"
This command creates an environment named dev-env for the my-hop-project with a purpose set to Development. Make sure to replace /path/to/your/env-variables.json with the path to your configuration file.
Setting environment variables
Once your environment is created, you can set variables for it. Use the following command to add variables:
./hop-conf.sh --config-file "/path/to/your/env-variables.json" \
--config-file-set-variables \
"DB_HOSTNAME=localhost, DB_PASSWORD=abcd"
This command updates your environment configuration file with the specified variables.
Remove a configuration file
If you need to remove a config file from an environment, you can use the command --environment-modify and specify the config files you want to keep for the environment.
For example, let’s say we have 2 config files that are used in "dev-env". The files are env-variables.json and env-connections.json.
./hop-conf.sh --environment-modify \
--environment dev-env \
--environment-project my-hop-project \
--environment-purpose Development \
--environment-config-files=/path/to/your/env-variables.json, \
/path/to/your/env-connections.json
If we need to remove "env-connections.json" from the environment setup, we run the following command:
./hop-conf.sh --environment-modify \
--environment dev-env \
--environment-project my-hop-project \
--environment-purpose Development \
--environment-config-files=/path/to/your/env-variables.json
Note that we only specify the config file env-variables.json in --environment-config-files instead of the 2 files comma separated.
Deleting an environment
If you ever need to delete an environment, you can use this command:
./hop-conf.sh --environment-delete --environment dev-env
This removes the "dev-env" environment from your Hop configuration file.
You should get an output message like this.
Conclusion
And that’s it! You’ve successfully installed Apache Hop, created your first project, and set up your environment using the Hop Conf command line tool. It’s as straightforward as doing it through the Hop GUI.
Don't miss the video below for a step-by-step walkthrough of the entire process!
Stay connected
If you have any questions or run into issues, contact us and we’ll be happy to help.
Build & deploy 5: First project and environment using Hop Conf