28 Sep 2022 Installing Apache NiFi on Windows
Apache NiFi is a very popular data movement tool, widely used by many companies around the world to move data across different systems. Born as an Apache Open Source project, it was packaged by Hortonworks in its Data Flow platform, and then by Cloudera in their own version of Data Flow when the two companies merged.
Today, it is available in the Cloudera stack both for on-premise and public clusters, but the open-source version still remains a valid option for those who want to experiment, play, or simply run it in non-production environments.
While typically run on Linux machines, it is perfectly feasible to run it on Windows, and in certain cases this might be an easier and more immediate option. If you just want to run it for simple flows, or if you want to build and test your pipelines without worrying about incurring additional costs from your cloud provider before the flow is deployed to production, then you can do so perfectly well from your personal or work Windows laptop, without needing to install Linux virtual machines and so on.
We have written plenty of blog posts about NiFi usage in certain enterprise scenarios (have a look at our blog here or simply google “clearpeaks nifi”), but today we want to get back to the roots and guide you through the straightforward process of installing it on your local Windows machine. There are two main steps that we need to perform, installing a valid JDK (Java Development Kit), and downloading then starting NiFi.
Let’s take a look at both of them in more detail.
Installing a valid JDK
In this section we’ll assume that you do not have a running version of the JDK. On a Windows machine, it’s easy to check: go to Start > Control Panel > Programs, and see if Java is installed. Alternatively, you can open a command line and type “javac”. If the command is not recognised, chances are that you do not have a JDK (although it might depend on how your system variables are set up, as we will see in a bit).
If you already have a JDK, no problem; NiFi will most probably work with it and you can move on to the next step.
To download a JDK, go to the Oracle website. In our case, we decided to install JDK 8, as it is one of the most used versions and usually works well with common software.
Follow this link, and scroll down until you find the Windows x64 installer; click on the link on the right and download the exe file.
Figure 1: The JDK download page on the Oracle website
After downloading the file, double-click on it and follow the wizard. There’s no need to change anything, so just confirm and click Next whenever requested, and wait for the installation to complete. Remember the name of the folder where the JDK will be installed (probably something like “C:\Program Files\Java”).
Once the installation has finished, you need to edit your system variables so your laptop will be able to find the JDK location when required. Go to the Start Menu, look for Edit the system environment variables, and open the System Properties tab. There, at the bottom, click on Environment Variables.
Click on New at the bottom of the tab that opens to add a new system variable. We called ours JAVA_HOME, with the value where the JDK was installed (in our case, C:\Program Files\Java\jdk1.8.0_331).
Figure 2: Creating the JAVA_HOME system variable
Then scroll down the list of existing variables until you find Path; open it, and at the bottom add a new line, using the same path as before, but this time with the “\bin” suffix (so in our case, “C:\Program Files\Java\jdk1.8.0_331\bin”).
Figure 3: Editing the Path system variable
Now confirm the changes and close the tab.
At this point, the JDK should have been installed and detected by your machine; to confirm this, open a command line and type “javac”. If the output looks like the one in the picture below, then you are all set!
Figure 4: Checking the JDK on your console
Downloading and Starting NiFi
Now that we have a working JDK, we can move on to downloading and installing NiFi.
To download NiFi, go to this link (or simply google “download nifi”).
Figure 5: NiFi download page
The version we want ends with “bin.zip” (the binary file zip folder). Click on it, and you are taken to another page where you can actually download the file (using the first link).
Figure 6: Binary zip file download link
Once the zip file has been downloaded, just extract it. A simple and effective choice is to unzip the folder in C:\, and you should end up with a path like C:\nifi-1.16.3, in which you should already see a bin directory.
At this point, we should check out the NiFi documentation to better understand the next steps.
At this link you’ll see that you are now required to navigate to that bin folder and double-click on the run-nifi.bat file. This will start the NiFi process in your machine. Keep this tab open as long as you want to use NiFi!
Figure 7: Starting the NiFi process
If you scroll down the documentation page a bit, you’ll also see how NiFi has already created a user for us: simply open the logs/nifi-app.log, look for “Generated”, and you should see both the username and password. The documentation also explains how to change those credentials, should you wish to.
Finally, you can open your NiFi UI. As the documentation shows, it is actually secured and running on HTTPS, on port 8443. Depending on your DNS/hosts file setting, you might need to use one of these URLs to open it:
If everything has gone according to plan, you should be prompted with the login page, as in the picture below:
Figure 8: NiFi login
Enter your credentials (the default ones or the new ones, if you changed them) and the NiFi UI will appear!
Figure 9: NiFi UI
Conclusion
Now you have a running NiFi instance to experiment with and test your flows – and it’s free! You can save some money before moving the flows to your production environment (for example, if you are running Data Flow in CDP Public Cloud).
As we said before, we have already written a lot about NiFi (check out our blog posts), but we thought that it would make your life easier if you had a quick and concise guide just to run NiFi locally, without having to worry about clusters, cloud resources, or Linux VMs.
At ClearPeaks, we are experts in NiFi and so much more. If you have any questions about what you’ve just read, or simply need some help with your Big Data, BI or ML architecture, don’t hesitate to contact us and we’ll be happy to help.
