Netflow/IPFIX basic concepts

Netflow and IPFIX are industry standards that summarizes the IP network traffic between two devices, sending the summary to an analyzing device. Explaining it in plain words, you can consider a Netflow/IPFIX stream as a phone call in a telephone bill. There is the IP address of the calling device, the IP address of the called device, the start time and end time of the conversation, the type of traffic, and the amount of bytes and packets transferred. The content of the conversation is not reported at all. In this way the protocol is very light and you can trace all the incoming and outgoing traffic from the device that exports the stream.

The following picture describes the basic concept behind Netflow/IPFIX:

Netflow/IPFIX Basic example

Netflow/IPFIX Basic example

As you can see in this simple example, the Fl0wer collector receives Netflow data from a Router or UTM device and stores it to disk, allowing information querying from a console using the provided tools or integration with your favourite Analytics.

In this example, only the traffic crossing the Router/UTM is tracked, all the communication on the “Company Network” switch is not tracked, since the Router/UTM device has visibility only of traffic “crossing” it or directed to it.

A more complex (and realistic schema) could be this one:

Fl0wer has no limits on the number of Router/UTM devices, and it is surprising to see how many vendors and software solutions export data in one of the supported Netflow/IPFIX formats.

As a quick and incomplete list you can get Netflow data from:

Routers/UTM: Cisco, Juniper, Fortinet, Mikrotik, Ubiquiti, Huawei, Sophos

Firewalls: CheckPoint GAIA, Palo Alto Networks, opnSense, pfSense, Linux ipt_NETFLOW.

Servers: openvswitch, softflowd on Linux, *BSD, etc.

Virtualization Platforms: VMWare ESXi >= 5 (with distributed vSwitch), Citrix (integrated openvswitch), Proxmox (using softflowd or openvswitch), OpenStack Neutron (integrated openvswitch).

Fl0wer has been tested with most of them, revealing new and interesting perspectives for data analysis and security applications.

Conceptually, Fl0wer receives its input data (Netflow or IPFIX) on UDP port 2056 (it can be obviously changed if needed) and listens for CLI-tools or GUI-Application requests on port 7443 (this can also be changed).

Incoming flows are instantly processed and decoded, submitted to its internal analyzer, then stored if required. The storage of flows is done using a separate thread for buffering, and when a buffer has to be stored, it is processed for all other things that are not done in real-time.