One of the most frequent things I've been asked for is why Fl0wer does not have a web interface. In today's world, it has become a standard to provide a web interface for every kind of software, even those that does not really need it.

Being a veteran Unix sysadmin, and developing code in C & Python, you can easily imagine the hate relationship that I have with web technologies that are mainstream (or, at least, are trendy) but are either badly documented or over documented and the moment you get enough skill to start coding they are already old, and are replaced with something new that is completely different . It's simple, I'm not a web developer and you want a web interface, so, how to solve this issue ?

The answer is pretty easy, and has always been part of the Fl0wer design concept from its inception and is present since the first version: integrate.

Flower makes a big use of the JSON standard (all dialog between the daemon and the client is based on JSON, and most of the CLI tools output JSON documents) and can obviously output flows in JSON format, but again, since I want Fl0wer to be compatible with whatever tool you use, Comma Separated Value (CSV) output is something that simplifies things a lot. Simple (one row per flow), "greppable" , easily compressible with enormous space gains, it fits the job.

Having been busy writing multithreaded C code, measuring its performance (microseconds are very very important !), debugging and all things required to make a great product, I never had so much time to play with all the tools available on the market, and in these days I finally found some time to take a look at the ELK stack, and added the CSVFULL output format in Fl0wer 1.2. I played with ELK some years ago, finding it interesting, but still not very ready for production use, so I decided to give it a try with the most recent version, and I was astonished about the improvements.

Setting up an ELK stack on your system (yes, you need fast CPUs and lots of memory) is pretty easy and it takes less than an hour of work, so this is what I did.

My Journey to ELK

Install latest Java 8, ELK is written in Java and requires Java 8. I use it with Java 8U131 from Oracle and works well.

My laptop runs on a Xubuntu 14.04, so I followed these instructions, refer to your favorite distribution instructions for your platform.

Once Java 8 is installed, go to the ELK site and download:

  1. Elasticsearch
  2. Logstash
  3. Kibana

I've used 5.4.3 version but they're now at 5.50, it should work the same or better.

Having a Debian based version, I got the Debian packages and installed them with a rude dpkg -i.

You should install and start them in the above said order (but don't start them now).

Once you installed all the packages, setup your /opt/fl0wer/etc/fl0wer.conf file setting the storage_format as CSV or CSVFULL (it depends on the detail level you want, I set up CSVFULL to experiment, if you want to use CSV you will have to make refinement to the logstash configuration).

The flow of the whole thing is:

Flower -> logs in CSV to /opt/fl0wer/data/netflowdata.csv

Logstash -> reads continuously (in "tail" mode) from /opt/fl0wer/data/netflowdata.csv, parses the fields and sends the data to Elasticsearch

Elasticsearch -> receives the data from Logstash and stores into its indexes

Kibana -> provides the web dashboard and allows you to create all the visualizations you like

Before starting the daemon, add the logstash user to the fl0wer group and check that the logstash user can access the /opt/fl0wer/data/netflowdata.csv file. Note: IMHO it is a bug in logstash, but it seems that if you don't give the "others" group read permission to the netflowdata.csv, logstash won't work. So a chmod o+r /opt/fl0wer/data/netflowdata.csv command should fix the thing.

Now start elasticsearch and check if it works. In my configuration I set it up to listen on localhost, just review the configuration file in /etc/elasticsearch/elasticsearch.yml. It should listen on TCP port 9200.

So, once you started the Elasticsearch engine and the Fl0wer daemon, and it starts logging in /opt/fl0wer/data/netflowdata.csv, we need to configure logstash to feed the Fl0wer logs to Elasticsearch. I always had a bad relationship with logstash, but adding support for CSV was pretty easy.

So, let's create a configuration file in /etc/logstash/conf.d/fl0wer.conf and put this stuff into it:

        path => "/opt/fl0wer/data/netflowdata.csv"
        start_position => "beginning"
        sincedb_path => "/opt/fl0wer/data/"
filter {
     csv {
           separator => ","
                         "SRC_FQDN","DST_FQDN","USERNAME","NEXTHOP_ADDRESS" ]
  elasticsearch {
    action => "index"
    codec => json
    hosts => [ "localhost:9200" ]
    index => "fl0wer"

This should be pretty clear to understand:

input: get your input from this file, and if/when logstash stops reading, save the point in "

filter: use a csv filter and these are the fields

output: send to the local Elasticsearch istance in the "fl0wer" index.

Review the configuration in /etc/logstash/logstash.yml and start logstash using /etc/init.d/logstash start (or whatever your distro uses to start services) checking for errors in the logs in the /var/log/logstash directory.

Once logstash started, it should read data from Fl0wer and pump it into the Elasticsearch Index.

Now it's time to start Kibana. Review the /etc/kibana/kibana.yml configuration file and fire up an /etc/init.d/kibana start.

Once started, it should be listening on port 5601.

Start Chrome/Firefox/yourfavouritemodernbrowserwithhtml5support and go to the Kibana webpage.

It will ask you what Index it should use, simply use "fl0wer*" and let it refresh the fields. If logstash is working correctly, it should show you with the fields it will be parsing:

The Elasticseach Index was found !The Elasticseach Index was found !

If everything is ok, you should now be presented with the "Discover" Page, which is something like this:

We're getting data !We're getting data !

If this works, you're setup and Elastic is indexing correctly. You can start playing with the Timelion ( a wonderful tool created for web monekys like me ) and create your favourite fancy charts using Elastic query language, which is pretty simple and immediate:

Timelion !Timelion !

Once you decide what to chart, save the interested expressions as Kibana dashboard panel and add them to your dashboard. Just to play a bit for testing, I come out with something like this:

Example DashboardExample Dashboard

With recent fixes, Fl0wer is able to output over 70000 FPS in CSVFULL (that's why microseconds are important !)

The rest is ordinary Elasticsearch stuff, but I'm sure you can craft nicer and more useful charts than me :-D