The original project behind Apache NiFi was created by NSA! Of course it will be efficient in tracking and can have vulnerabilities, at least older than 1.3 versions had. Version 1.3.0 is latest at the time of publication of this guide. At present NSA has nothing to do with Apache NiFi and it has been another Big Data tool in Apache Software Foundation’s collection. It can have vulnerabilities. Apache NiFi Enables Automation of Real Time Data Flow Between Systems. Here Is How To Install Apache NiFi On Ubuntu 16.04 LTS on Cloud Server.Official website is :
1 | https://nifi.apache.org |
We have to increase the limits. What we will do, wiki is here :
1 | https://wiki.archlinux.org/index.php/Limits.conf |
Run :
---
1 2 | apt update apt upgrade |
We will edit /etc/security/limits.conf :
1 | nano /etc/security/limits.conf |
Find the entries and change the values like this :
1 2 3 4 5 6 7 | * hard nofile 50000 * soft nofile 50000 * hard nproc 10000 * soft nproc 10000 |
We also need to run :
1 2 | sudo sysctl -w net.ipv4.ip_local_port_range="10000 65000" sudo sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait="1" |
Your distribution may need to edit /etc/security/limits.d/90-nproc.conf :
1 | * soft nproc 10000 |
Edit /etc/sysctl.conf to add the following line :
1 | vm.swappiness = 0 |
After saving, reboot the system.
How To Install Apache NiFi On Ubuntu 16.04 LTS
Apache NiFi can only made to run, can be fully installed, can be integrated with other Big Data analytics tools. We are showing minimal way to run Apache NiFi. You’ll get the latest version from here :
1 | https://nifi.apache.org/download.html |
Apache NiFi needs a Java runtime environment and a user with sudo privilege. In the example, we will use the user name nifi. The command adduser will automatically create the user, initial group, and home directory. Run these commands one by one :
1 2 3 | adduser nifi id nifi ls -lad /home/nifi/ |
Now we will set the user for sudo permission to allow to be in suoders list :
1 | echo 'nifi ALL=(ALL) ALL' >> /etc/sudoers |
Type exit to exit the session and run command SSH with that username :
1 | ssh nifi@your-IP-addresss |
After login, type :
1 | sudo su |
Now you’ll become root user. Now, you may be in need in future to close the way for nifi to SSH. Open this file :
1 | nano /etc/ssh/sshd_config |
You’ll add entry of kafka to SSH if needed by having syntax like this :
1 | DenyUsers nifi |
save the file. Run these commands :
1 2 | /etc/init.d/sshd restart sudo service ssh restart |
During initial installation do not do the changes in /etc/ssh/sshd_config. You’ll install the software by SSH to the system as any permitted user and run su nifi to become nifi.
Next we will install Java. There are two ways, one is using oracle-java8 :
1 2 3 | sudo add-apt-repository -y ppa:webupd8team/java sudo apt update apt install oracle-java8-installer -y |
Another is the default Java Run Time provided by apt :
1 | apt install default-jre |
Follow the way you want. Verify that JDK 8 is installed properly
1 | java -version |
Now NiFi specific steps :
1 2 3 4 5 6 7 8 9 10 11 12 | su nifi cd ~ wget http://redrockdigimark.com/apachemirror/nifi/1.3.0/nifi-1.3.0-bin.tar.gz tar xvzf nifi-1.3.0-bin.tar.gz rm nifi-1.3.0-bin.tar.gz ls -al # for compatibility with other tutorials ln -s ~/nifi-1.3.0/ /opt/nifi-1.3.0/ ln -s ~/nifi-1.3.0/ /opt/nifi-latest # chown chown -h nifi. /opt/nifi-latest chown -R nifi. ~/nifi-1.3.0/ |
The main configuration is nifi.properties file :
1 | nano /opt/nifi-latest/conf/nifi.properties |
The default 8080 is used by Tomcat and many others.
Set JAVA_HOME :
1 2 3 4 5 6 | nano .bash_profile # Add, change x.y.x to real ## export JAVA_HOME=/etc/alternatives/java_sdk_x.y.x_openjdk # or export JAVA_HOME=/usr/lib/jvm/java-8-oracle source ~./bashrc |
Start/Stop Apache NiFi :
1 | /opt/nifi-latest/bin/nifi.sh start |
Open a browser window and verify that you can access NiFi :
1 | http://IP-Address:8080/nifi |
You really should use official guides :
1 2 | https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#how-to-install-and-start-nifi https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html |
