This is an updated guide on installation of Apache NiFi. Version 1.21.0 is the latest. It was built by NSA to automate the flow of data between their systems. Nifi is a Java based project and executes within a JVM. The biggest advantage is that, it has a web based interface to create, monitor and control data flow. You’ll get a good documentation on limits here:
1 | https://wiki.archlinux.org/title/Limits.conf |
Now, the steps to install Apache Nifi.
1 2 | apt update apt upgrade |
We will edit /etc/security/limits.conf :
---
1 | nano /etc/security/limits.conf |
Find the entries and change the values like this :
1 2 3 4 5 6 7 | * hard nofile 50000 * soft nofile 50000 * hard nproc 10000 * soft nproc 10000 |
We also need to run :
1 2 | sudo sysctl -w net.ipv4.ip_local_port_range="10000 65000" sudo sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait="1" |
Your distribution may need to edit /etc/security/limits.d/90-nproc.conf :
1 | * soft nproc 10000 |
Edit /etc/sysctl.conf to add the following line :
1 | vm.swappiness = 0 |
After saving, reboot the system.
How To Install Apache NiFi On Ubuntu 16.04 LTS
We are showing minimal way to run Apache NiFi. You’ll get the latest version from here :
1 | https://nifi.apache.org/download.html |
Apache NiFi needs a Java runtime environment and a user with sudo privilege. In the example, we will use the user name nifi. The command adduser will automatically create the user, initial group, and home directory. Run these commands one by one :
1 2 3 | adduser nifi id nifi ls -lad /home/nifi/ |
Now we will set the user for sudo permission to allow to be in suoders list :
1 | echo 'nifi ALL=(ALL) ALL' >> /etc/sudoers |
Type exit to exit the session and run command SSH with that username :
1 | ssh nifi@your-IP-addresss |
After login, type :
1 | sudo su |
Now you’ll become root user. Now, you may be in need in future to close the way for nifi to SSH. Open this file :
1 | nano /etc/ssh/sshd_config |
You’ll add entry of kafka to SSH if needed by having syntax like this :
1 | DenyUsers nifi |
save the file. Run these commands :
1 2 | /etc/init.d/sshd restart sudo service ssh restart |
During initial installation do not do the changes in /etc/ssh/sshd_config. You’ll install the software by SSH to the system as any permitted user and run su nifi to become nifi.
Next we will install Java. There are two ways, one is using oracle-java8 :
1 2 3 | sudo add-apt-repository -y ppa:webupd8team/java sudo apt update apt install oracle-java8-installer -y |
Another is the default Java Run Time provided by apt :
1 | apt install default-jre |
Follow the way you want. Verify that JDK 8 is installed properly
1 | java -version |
Now NiFi specific steps :
1 2 3 4 5 6 7 8 9 10 11 12 13 | su nifi cd ~ ## wget latest nifi ## change the values of x,y in version number tar xvzf nifi-x.y.0-bin.tar.gz rm nifi-x.y.0-bin.tar.gz ls -al # for compatibility with other tutorials ln -s ~/nifi-1.y.0/ /opt/nifi-1.y.0/ ln -s ~/nifi-1.y.0/ /opt/nifi-latest # chown chown -h nifi. /opt/nifi-latest chown -R nifi. ~/nifi-1.y.0/ |
The main configuration is nifi.properties file :
1 | nano /opt/nifi-latest/conf/nifi.properties |
The default 8080 is used by Tomcat.
Set JAVA_HOME :
1 2 3 4 5 6 | nano .bash_profile # Add, change x.y.x to real ## export JAVA_HOME=/etc/alternatives/java_sdk_x.y.x_openjdk # or export JAVA_HOME=/usr/lib/jvm/java-8-oracle source ~./bashrc |
Start/Stop Apache NiFi :
1 2 | /opt/nifi-latest/bin/nifi.sh install /opt/nifi-latest/bin/nifi.sh start |
Open a browser window and verify that you can access NiFi :
1 | http://IP-Address:8080/nifi |
You really should use official guides :
1 2 | https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#how-to-install-and-start-nifi https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html |
