Select Page

Recently, we saw the release of Faucet v1.1, which included many new features including multi-datapath support for a single controller instance. It’s time to take a deep dive into the inner workings of Faucet and explore just what makes this controller app tick. I hope you will enjoy this article and find it informative. I know I learned a lot writing it!

There is a lot more in Faucet than I can cover in a single article, so I will be primarily covering the core functionality as implemented by its use of tables and flow entries installed in the switches it controls. I am focusing less on the code this time as this article is more for those who will be working on applications working aside Faucet, in which case knowledge of its tables and flow entries is more important and I’ve already covered the Ryu API in the past, and also for those wishing to use Faucet in their network, in which case knowing the packet pipeline can help in understanding how Faucet works and how it can be configured. If you are looking for a tutorial on installing Faucet v1.1, I covered that in my last tutorial, Faucet in a Virtual Infrastructure.

Faucet Packet Flow and Network Logic

This article builds on the concepts taught in the Understanding the Ryu API: Reimagining Simple Switch article. Specifically, it already covers the idea of keeping state (such as which hosts are learned) in the switch by utilizing multiple tables in such a way that it prevents sending packets to the controller once something is learned, but still notifying the controller if something changes (such as a host changing ports). The basic table structure used in the Reimagined Simple Switch is based heavily on Faucet and covers basic L2 learning without VLAN or other advanced switch features implemented in Faucet. Please review that article before continuing here.

Table Overview

Below is a Table Pipeline chart with all the tables Faucet uses and potential actions at each table.

Faucet Packet Flow - Possible Actions and Paths

Faucet Packet Flow – Possible Actions and Paths

 

  • VLAN (vlan– The main entry point for packets processed by Faucet’s flow entries. It includes default drop flows for traffic that switches generally shouldn’t process or forward. It also contains antispoof flow entries for Faucet’s control plane MAC address. Finally, as the name suggests, it pushes VLAN tags on untagged packets for the ports that have a “native” or untagged VLAN assigned to it. Most packets are forwarded to the ACL or Ethernet Source tables.
  • ACL (acl– This includes flow entries created from the ACL rules defined in Faucet’s YAML file. The ACLs are processed for each port (if the ACL is assigned to it) before moving on to the learning process. This is where your overriding flow entries for your custom applications would reside.
  • Ethernet Source (eth_src) – This table keeps track of what host MACs have already been learned. When an unlearned host send traffic through the switch, a table-miss flow sends the headers to Faucet for learning. Both learned and unlearned source traffic are usually forwarded directly to the Ethernet Destination table, but this table can also contain flow entries to send ARP and Neighbor Discovery packets to the controller to ensure the L3 learning component works properly. Lastly, if too many unlearned hosts are learned in a short period of time on a VLAN, Faucet can optionally tell the switch to stop sending it packets for that VLAN. This helps reduce load on the controller during a DDoS attack or bringing up the network in a clean state.
  • IPv4 FIB (ipv4_fib), IPv6 FIB (ipv6_fib) – FIB stands for Forwarding Information Base. If the Faucet configuration includes IPv4 or IPv6 routes, this table will contain the flow entries required to route matching packets sent from the Ethernet Source table. This also includes flow entries for allowing the controller to directly process ICMP packets sent to the controller IP. The routes are configured through the Faucet VLAN configuration and optionally BGP.
  • Ethernet Destination (eth_dst) – Receives packets from the Ethernet Source table and resolved routes from the IPv4 and IPv6 FIB tables. Contains learned destinations with flows that output a packet on the learned port and any mirrored ports, if configured. If a destination is not known, the packet is sent to the Flood table.
  • Flood (flood) – Rather than using OpenFlow’s built in OFPP_FLOOD port, Faucet ensures packets are only ever flooded to ports that are configured to receive traffic for the specific VLAN. This contains the flow entries to ensure this more secure flooding and also supports flooding on mirrored ports, if configured.

Example Packet Flow

As you can see above, there are quite a few actions that can be taken across all the tables. Thankfully, for the most common scenario, only three or four tables are used and the only packet modifications used might be to push or pop a VLAN tag. For these examples, I’ve configured Mininet to emulate a single switch with four ports and configured Faucet to match that topology and assigned all ports to VLAN 10. These ports are configured for untagged traffic, so VLAN tags will be automatically added to incoming packets and removed from outgoing packets. Although switches used in production will often use a lot more ports, just having four will make it easy to demonstrate some of the basic inner workings of the Faucet pipeline.

Example – Known Source and Destination

In the chart below, we have a packet flow where the controller (and switch) have already learned the source and destination addresses.

 

Faucet Packet Flow - Learned Source and Destination

Faucet Packet Flow – Learned Source and Destination

  1. VLAN Table – In this case, packets arrive on the VLAN table (as all packets do) and our native VLAN is pushed on to the packet. This ensures that further processing is always performed on a VLAN tagged packet. In our example, there are no ACLs assigned for the in port, so the VLAN table forwards the packet directly to the Ethernet Source table.
  2. Ethernet Source Table – The packet is processed by the Ethernet Source table where it will match a flow entry with the in port and Ethernet Source address (Source MAC) of the packet along with its VLAN tag. If these conditions were not matched, the controller would have been notified in this process, which is explained in Learning a New Host below. In either case, the packet is forwarded directly to the Ethernet Destination table.
  3. Ethernet Destination Table – The packet is lastly processed by the Ethernet Destination where it matches a flow entry based on the Ethernet Destination address (Destination MAC) of the packet and its VLAN tag. The flow entry causes the VLAN tag to be removed (because it will be exiting on an untagged port) and then finally output on the learned port for that destination address.

Here is a graphical representation of the table flow entries in use on a four port example OVS switch as emulated by Mininet. For the full flow entries, see Flow Entry Listing (for the Curious).

Faucet Packet Flow with OVS and Mininet - Learned Source and Destination

Faucet Packet Flow with OVS and Mininet – Learned Source and Destination

Learning a New Host

At startup, however, none of the hosts are learned just yet. In the case where one unlearned host sends packets to another unlearned host, the Ethernet Source table will send a partial copy of the packet to the controller so it can learn the port that the first unlearned host can be reached and add flow entries to the Ethernet Source and Ethernet Destination tables accordingly. This is so the controller is not contacted again about the now-learned host (unless that host moves) and packets destined to that host will output to the learned port, rather than flooding to all ports for that VLAN.

Faucet Packet Flow - Unlearned Source and Destination

Faucet Packet Flow – Unlearned Source and Destination

  1. VLAN Table – As before, the VLAN tag is pushed on to the packet coming from an untagged port and sent to the Ethernet Source table.
  2. Ethernet Source Table – Since the sending host is not yet learned, the packet will not match any flow entries except for the table-miss entry. This entry sends a partial copy of the packet to the controller to trigger a learning event and the packet is directly sent to the Ethernet Destination table. It is important to note that packets are not queued and do not wait for the controller to respond.
  3. Ethernet Destination Table – As the destination host is also unknown, the packet will fail to match on any Ethernet Destination address (Destination MAC). Again, the table-miss flow is triggered, this time sending the packet directly to the Flood table.
  4. Flood Table – This is the last chance a packet has to being output on a port. The packet will match based on the in port and VLAN tag. The actions will include removing the VLAN tag before outputting on untagged ports and the packet is finally output on every port assigned to the packet’s VLAN, except for the in port of the packet.

Here is a graphical representation of the table flow entries in use on a four port example OVS switch as emulated by Mininet.

Faucet Packet Flow with OVS and Mininet - Unlearned Source and Destination

Faucet Packet Flow with OVS and Mininet – Unlearned Source and Destination

The Flood table includes several entries for each in port matching several broadcast and multicast addresses with a high priority, but even if those addresses are not matched (such as a standard unicast address), the packet is still flooded to all ports assigned to that VLAN save for the in port. For more details on the higher priority flow entries, see Flow Entry Listing (for the Curious) below.

When the second host sends a reply back to the first, the host learning event is triggered like before, but the packet is not flooded since the destination is already learned. After both hosts are learned, then the packet flow will resemble the first chart (Learned Source and Destination)

Faucet Packet Flow - Unlearned Source, Learned Destination

Faucet Packet Flow – Unlearned Source, Learned Destination

Flow Entry Listing (for the Curious)

In writing these examples, I ran Faucet v1.1 configured with a single switch and four hosts, emulated by Mininet. While running Faucet, I also ran Ryu’s OFCTL REST application to pull the flow entries from the switch (see Interactive Ryu with Postman for more info). Since I skipped some flow entries for brevity above, here are the full flow entry listings both before and after host learning has occurred:

Before Learning (Default State)

This is the state of the switch after Faucet is started and it has installed its default flows. Data in parentheses explain or convert the value directly before it.

Table 0 (VLAN)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
0 9099 01:80:c2:00:00:00 drop
0 9099 01:00:0c:cc:cc:cd drop
0 9099 0x88CC drop
0 9001 0e:00:00:00:00:01 drop
0 9001 ff:ff:ff:ff:ff:ff
0 9000 0 (none) 1 PUSH_VLAN:33024 (0x8100); SET_FIELD: {vlan_vid:4106 (0x1000 + 10d)}; GOTO_TABLE:2
0 9000 0 (none) 2 PUSH_VLAN:33024 (0x8100); SET_FIELD: {vlan_vid:4106 (0x1000 + 10d)}; GOTO_TABLE:2
0 9000 0 (none) 3 PUSH_VLAN:33024 (0x8100); SET_FIELD: {vlan_vid:4106 (0x1000 + 10d)}; GOTO_TABLE:2
0 9000 0 (none) 4 PUSH_VLAN:33024 (0x8100); SET_FIELD: {vlan_vid:4106 (0x1000 + 10d)}; GOTO_TABLE:2
0 0 drop

Table 1 (ACL)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
1 0 drop

Table 2 (Ethernet Source)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
2 9000 OUTPUT:CONTROLLER; GOTO_TABLE:5
2 0 drop

Table 3 (IPv4 FIB)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
3 0 drop

Table 4 (IPv6 FIB)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
4 0 drop

Table 5 (Ethernet Destination)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
5 9000 GOTO_TABLE:6
5 0 drop

Table 6 (Flood)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
6 9008 ff:ff:ff:ff:ff:ff 10 1 POP_VLAN; OUTPUT:2; OUTPUT:3; OUTPUT:4
6 9008 ff:ff:ff:ff:ff:ff 10 2 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9008 ff:ff:ff:ff:ff:ff 10 3 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9008 ff:ff:ff:ff:ff:ff 10 4 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:3
6 9006 33:33:00:00:00:00/ff:ff:00:00:00:00 10 1 POP_VLAN; OUTPUT:2; OUTPUT:3; OUTPUT:4
6 9006 33:33:00:00:00:00/ff:ff:00:00:00:00 10 2 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9006 33:33:00:00:00:00/ff:ff:00:00:00:00 10 3 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9006 33:33:00:00:00:00/ff:ff:00:00:00:00 10 4 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:3
6 9004 01:00:5e:00:00:00/ff:ff:ff:00:00:00 10 1 POP_VLAN; OUTPUT:2; OUTPUT:3; OUTPUT:4
6 9004 01:00:5e:00:00:00/ff:ff:ff:00:00:00 10 2 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9004 01:00:5e:00:00:00/ff:ff:ff:00:00:00 10 3 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9004 01:00:5e:00:00:00/ff:ff:ff:00:00:00 10 4 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:3
6 9002 01:80:c2:00:00:00/01:80:c2:00:00:00 10 1 POP_VLAN; OUTPUT:2; OUTPUT:3; OUTPUT:4
6 9002 01:80:c2:00:00:00/01:80:c2:00:00:00 10 2 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9002 01:80:c2:00:00:00/01:80:c2:00:00:00 10 3 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9002 01:80:c2:00:00:00/01:80:c2:00:00:00 10 4 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:3
6 9000 10 1 POP_VLAN; OUTPUT:2; OUTPUT:3; OUTPUT:4
6 9000 10 2 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9000 10 3 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9000 10 4 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:3
6 0 drop

After Learning (Mininet pingall)

This is the state of the switch after pingall was performed in the Mininet CLI. Data in parentheses explain or convert the value directly before it. Lines in bold indicate flows that were added.

Table 0 (VLAN)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
0 9099 01:80:c2:00:00:00 drop
0 9099 01:00:0c:cc:cc:cd drop
0 9099 0x88CC drop
0 9001 0e:00:00:00:00:01 drop
0 9001 ff:ff:ff:ff:ff:ff
0 9000 0 (none) 1 PUSH_VLAN:33024 (0x8100); SET_FIELD: {vlan_vid:4106 (0x1000 + 10d)}; GOTO_TABLE:2
0 9000 0 (none) 2 PUSH_VLAN:33024 (0x8100); SET_FIELD: {vlan_vid:4106 (0x1000 + 10d)}; GOTO_TABLE:2
0 9000 0 (none) 3 PUSH_VLAN:33024 (0x8100); SET_FIELD: {vlan_vid:4106 (0x1000 + 10d)}; GOTO_TABLE:2
0 9000 0 (none) 4 PUSH_VLAN:33024 (0x8100); SET_FIELD: {vlan_vid:4106 (0x1000 + 10d)}; GOTO_TABLE:2
0 0 drop

Table 1 (ACL)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
1 0 drop

Table 2 (Ethernet Source)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
2 9098   00:00:00:00:00:01 10   1   GOTO_TABLE:5
2 9098   00:00:00:00:00:02 10   2   GOTO_TABLE:5
2 9098   00:00:00:00:00:03 10   3   GOTO_TABLE:5
2 9098   00:00:00:00:00:04 10   4   GOTO_TABLE:5
2 9000 OUTPUT:CONTROLLER; GOTO_TABLE:5
2 0 drop

Table 3 (IPv4 FIB)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
3 0 drop

Table 4 (IPv6 FIB)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
4 0 drop

Table 5 (Ethernet Destination)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
5 9001 00:00:00:00:00:01   10       POP_VLAN; OUTPUT:1
5 9001 00:00:00:00:00:02   10       POP_VLAN; OUTPUT:2
5 9001 00:00:00:00:00:03   10       POP_VLAN; OUTPUT:3
5 9001 00:00:00:00:00:04   10       POP_VLAN; OUTPUT:4
5 9000 GOTO_TABLE:6
5 0 drop

Table 6 (Flood)

Table Priority dl_dst dl_src dl_vlan dl_type in_port actions
6 9008 ff:ff:ff:ff:ff:ff 10 1 POP_VLAN; OUTPUT:2; OUTPUT:3; OUTPUT:4
6 9008 ff:ff:ff:ff:ff:ff 10 2 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9008 ff:ff:ff:ff:ff:ff 10 3 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9008 ff:ff:ff:ff:ff:ff 10 4 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:3
6 9006 33:33:00:00:00:00/ff:ff:00:00:00:00 10 1 POP_VLAN; OUTPUT:2; OUTPUT:3; OUTPUT:4
6 9006 33:33:00:00:00:00/ff:ff:00:00:00:00 10 2 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9006 33:33:00:00:00:00/ff:ff:00:00:00:00 10 3 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9006 33:33:00:00:00:00/ff:ff:00:00:00:00 10 4 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:3
6 9004 01:00:5e:00:00:00/ff:ff:ff:00:00:00 10 1 POP_VLAN; OUTPUT:2; OUTPUT:3; OUTPUT:4
6 9004 01:00:5e:00:00:00/ff:ff:ff:00:00:00 10 2 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9004 01:00:5e:00:00:00/ff:ff:ff:00:00:00 10 3 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9004 01:00:5e:00:00:00/ff:ff:ff:00:00:00 10 4 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:3
6 9002 01:80:c2:00:00:00/01:80:c2:00:00:00 10 1 POP_VLAN; OUTPUT:2; OUTPUT:3; OUTPUT:4
6 9002 01:80:c2:00:00:00/01:80:c2:00:00:00 10 2 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9002 01:80:c2:00:00:00/01:80:c2:00:00:00 10 3 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9002 01:80:c2:00:00:00/01:80:c2:00:00:00 10 4 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:3
6 9000 10 1 POP_VLAN; OUTPUT:2; OUTPUT:3; OUTPUT:4
6 9000 10 2 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9000 10 3 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:4
6 9000 10 4 POP_VLAN; OUTPUT:1; OUTPUT:2; OUTPUT:3
6 0 drop

Table Functions In Depth

There are more features in Faucet than just a simple L2 learning switch. It supports some more advanced features usually only found in high-end legacy switches and many of these features are implemented to make it easier to migrate from a legacy L2 network to an OpenFlow controlled network. From a packet flow perspective, these features are implemented in the six tables I listed earlier. Here, I’ll be covering each table in depth and how they interact with each other.

VLAN

Default Table ID
0
Receives From Tables
N/A (main entry point)
Sends To Tables
ACL Table (if ACLs configured for port)
Ethernet Source Table (if ACLs not configured for port)
Match Conditions
in_port, vlan_vid, eth_src, eth_dst, eth_type

One of the key features of Faucet is its ability natively support VLANs and the port isolation it provides. Each port on a Faucet-controlled switch can be assigned to any number of tagged VLANs and a single “native” VLAN. Like any other VLAN-capable switch, these ports will only send and receive traffic from the VLANs assigned to them. If a on the switch receives a packet with a VLAN tag that is not associated with that port, the switch will drop that packet. Similarly, these ports will only send (and accept) broadcast and multicast traffic from the VLANs that are assigned for that port. These features are primarily implemented using flow entries in the VLAN table.

Here is an overview of functions that the VLAN table is used to help implement:

  • Stop packets that should not be forwarded or processed by the switch (LLDP, STP BPDU, Broadcast Sources)
  • Provide an anti-spoof mechanism for Faucet’s control plane MAC address
  • Tag untagged packets received on ports with native VLANs configured, and allow that packet to continue to be processed in the pipeline
  • Allow tagged packets received on ports to continue to be processed by the pipeline if the packet’s VLAN ID matches one of the allowed VLANs for the incoming port
  • Drop incoming packets on ports that are configured to output mirrored traffic
  • Mirror incoming traffic from the current incoming port

By default, the VLAN table will have a table ID of 0, or the first table in the datapath. This is where incoming packets are first processed and the entry point into the Faucet pipeline. Being the first table to process packets, Faucet takes advantage of this to immediately stop processing certain types of packets. These high-priority default flow entries drop broadcast sources, LLDP packets and STP BDPUs. Further, a flow entry is added that serves as an anti-spoof mechanism for Faucet’s control plane MAC address. As packets from this MAC address are always sent via OpenFlow Packet Out commands, any traffic incoming on any port that claims to be from this MAC address should not be forwarded.

The primary purpose of this table, however, is to validate that incoming packets contain valid and allowed VLAN tags and, optionally, tag packets that arrive without a VLAN tag if a native VLAN is configured on the arriving port. This ensures that, anywhere else in the pipeline, the packets will always have a valid VLAN tag with the exception of mirror ports. If the mirror port is requesting traffic from a port with a native VLAN, the packet is sent out the mirror port before a VLAN tag is pushed on to the packet. This differs from packets that are mirrored for outputting on the port, which will always include a VLAN ID (see the Ethernet Destination and Flood table descriptions) even if the actual output port has the VLAN tag popped. It is important to note this difference when using mirror ports in Faucet 1.1 using ports with native VLANs.

ACL

Default Table ID
1
Receives From Tables
VLAN Table
Sends To Tables
Ethernet Source Table
Match Conditions
in_port, vlan_vid, (any)

The ACL (Access Control List) table is much simpler to explain than the other tables, but provides one of Faucets most useful features. In Faucet, you can configure multiple ACLs as required for your security (or other) purposes. Each port can have a single ACL assigned to it, but the same ACL can be used on multiple ports. It is important to note that the ACL rules are only applied on packets arriving at the switch on the port that is configured to use the ACL. It does not process the ACL for traffic output from the switch to the host (or other switch) on that port. This is not actually much of an issue, especially if you configure all ports that may have untrusted traffic to use an ACL. Since ACLs can match on just about any OpenFlow field, you can still have custom rules for traffic destined to specific hosts via L2 and/or L3 addresses.

ACLs support the following base actions for any match condition:

  • Mirror matched packets to a specific port
  • Output matched packets on a specific port and stop further processing
    • Modify destination MAC address
    • Push a VLAN tag
  • Allow matched packets to continue on in the Faucet pipeline
  • Drop matched packets

ACLs can match packets on any conditions supported by Ryu’s ofctl.to_match() function for OpenFlow 1.3.

From a flow entry standpoint, ACLs are essentially direct flow entries added for each configured port. The only real limits are the defined base actions, though this is a design decision that actually works quite well. The ACLs can be written without having to give much thought on the internals of OpenFlow and Faucet handles adding these flow entries for each port. This allows converting ACLs written for legacy switches pretty easy. In cases where advanced actions are needed, it is easy enough to write a custom Ryu application, but it is generally rare to need more than the base actions when writing ACLs in the first place. It’s even possible to implement some pretty complex custom packet routing using only the “Output” action.

Ethernet Source

Default Table ID
2
Receives From Tables
VLAN Table, ACL Table
Sends To Tables
Ethernet Destination Table, IPv4 FIB Table, IPv6 FIB Table
Match Conditions
in_port, vlan_vid, eth_src, eth_dst, eth_type, ip_proto, icmpv6_type, ipv6_nd_target, arp_tpa, ipv4_src

The Ethernet Source table is primarily used to track hosts that have a learned MAC/VLAN/Port combination. The table-miss flow entry (the default action) instructs the switch to send a partial copy of the incoming packet to the controller while also sending the full packet to the forwarding table. The forwarding table is usually the Ethernet Destination table, but the packet may instead be sent to the IPv4 or IPv6 FIB tables for more processing before reaching the Ethernet Destination table or being handled manually by the controller. See the section on the IPv4 and IPv6 FIB tables for more information on Faucet’s optional L3 processing.

Having the controller receive every packet is very compute intensive for both the controller and the switch, but the controller must receive some packets in order to learn host locations. When a host is learned by the controller receiving the partial packet copy, the controller installs flow entries in the Ethernet Source and Ethernet Destination tables. Specifically where the Ethernet Source table is concerned, a flow entry is added with a higher priority that matches on the source MAC address, the VLAN ID, and the port that the packet arrived on. This flow entry instructs the packet to go directly to the Ethernet Destination table without sending a copy to the controller. So long as packets from the same MAC address and VLAN arrive on the learned port, the controller is no longer sent traffic for that combination. However, if the incoming port changes, this flow entry will not have its match conditions met and the packet will fall through to the table-miss entry that sends the packet to the controller. This allows for the efficient learning of a new host’s location (i.e. a host being moved to a different port). Upon learning the new host location, the previous location is unlearned and flow entry removed followed by new entries for the new location.

The flow entries added by the L2 learning process on both the Ethernet Source and Ethernet Destination tables have timeouts attached to them. This allows old hosts to be removed and relearned over time. The flow entries added to the Ethernet Source table have a Hard Timeout, so any entries added are guaranteed to expire after a configured amount of time (by default, 5 minutes). Using a hard timeout here also ensures that the Ethernet Source flow entry is removed before the Ethernet Destination flow entry, which uses an Idle Timeout.

The Ethernet Source table also helps implement the following additional (but optional) features:

  • Directly forward ARP and Neighbor Discovery (Solicit and Advert) packets for the controller to the controller via Packet-In, where the controller can respond directly using Packet-Out to ensure that the appropriate reply contains Faucet’s internal MAC address (0e:00:00:00:00:01 in Faucet v1.1).
  • Forward IP packets destined to the controller internal MAC address to the IPv4 or IPv6 tables.
  • Limit the amount of hosts learned on a VLAN for DDoS mitigation by installing flow entries that implement a cooldown period when the limit is reached.

IPv4 FIB, IPv6 FIB

Default Table ID
3 (IPv4), 4 (IPv6)
Receives From Tables
Ethernet Source Table
Sends To Tables
Ethernet Destination Table
Match Conditions
vlan_vid, eth_type, ip_proto, ipv4_src (IPv4), ipv4_dst (IPv4), icmpv6_type (IPv6), ipv6_dst (IPv6)

Faucet supports optional IPv4 and IPv6 routing controlled by BGP. The FIB tables for IPv4 and IPv6 tables are separate, but perform essentially the same functions, so they are combined in this description. These primary functions are to forward ICMP and ICMPv6 packets for the controller to the controller via Packet-In and modify packet data for IPv4 and IPv6 configured routes.

As mentioned in the Ethernet Source table description, IP packets destined to the controller are sent through the IPv4 or IPv6 FIB tables. While ARP and Neighbor Discovery packets were sent to the controller (via Packet-In) from the Ethernet Source table, ICMP and ICMPv6 Echo Requests are sent to the controller (also via Packet-In) through the IPv4 and IPv6 FIB tables respectfully. The controller will build responses to these requests and send them directly using Packet-Out messages. As before, the responses contain a source MAC set to Faucet’s internal MAC address.

Secondly, routes can be configured in Faucet’s VLAN definition and will result in flow entries added the appropriate FIB table. These flow entries will match on the EtherType, destination IP address, and the VLAN ID. The actions will modify the source MAC to match Faucet’s internal MAC address (0e:00:00:00:00:01), modify the destination MAC to the route’s gateway MAC address, decrement the TTL and finally send the packet to the Ethernet Destination Table.

Finally, routes can be configured on the fly through the use of BGP Speakers on each VLAN configured with a BGP Router ID. The BGP Speaker implementation is provided by Ryu with Faucet providing a BGP route handler to modify the FIB tables as needed. Statically configured routes in Faucet’s configuration file will automatically be added as prefixes to the speaker. It is important to note that Faucet v1.1 does not support multiple datapaths sharing a single BGP speaker VLAN with those datapaths controlled by the same Faucet instance, though this may be implemented in a future release. This does not affect statically defined routes in the VLAN configuration without BGP enabled.

Ethernet Destination

Default Table ID
5
Receives From Tables
Ethernet Source Table, IPv4 FIB Table, IPv6 FIB Table
Sends To Tables
Flood Table
Match Conditions
vlan_vid, eth_dst

This is generally the final stop for any packet destined to a learned host. The Ethernet Destination table includes flow entries for all learned MAC address and VLAN combinations, and their associated port. As explained in the Ethernet Source table description, flow entries are added here during learning events triggered by the controller receiving Packet-Ins from the switch.

For learned hosts, each flow entry will match on the Ethernet Destination and the VLAN ID. The action will output the packet to the learned port for that host. If there is another port configured to mirror traffic on the learned port, the packet will immediately be output on the mirror port. If the output port has a native VLAN assigned (untagged), the VLAN tag will be popped off if the VLAN matches the native VLAN. However, if a mirror port is configured, the packet will be sent to the mirror port before the pop VLAN action is performed even if a native VLAN is configured.

As also mentioned in the Ethernet Source table description, Ethernet Destination flow entries include an Idle Timeout. So long as traffic is sent to the learned destination, the flow entry will remain in the Ethernet Destination table unless removed specifically by Faucet, such as during a host relearning event.

If no other flow entries match on the destination and VLAN IDs, then the packet is sent to the Flood table.

Flood

Default Table ID
6
Receives From Tables
Ethernet Destination Table
Sends To Tables
None
Match Conditions
in_port, vlan_vid, eth_dst

One way Faucet maintains VLAN security is by ensuring that packets that must be flooded are only flooded on ports assigned to that VLAN.

For every combination of physical port and VLANs assigned to them, flow entries are added to match the input port of the packet and the VLAN ID. The actions will include outputting the packet on each port except for the input port. If ports include native VLANs that match the flow entry’s VLAN ID, packets will be output first to any ports that do not include the native VLAN, pop the VLAN tag, and then output on the ports that do include the native VLAN. Mirror actions are also honored and are performed before all other actions. As with the Ethernet Destination table, mirror ports will always include the VLAN tag on a flood event even if the monitored port has a native VLAN.

When configuring many VLANs on a switch with many ports, it is important to note the associated increase in flow entries and the number of actions in each entry. Considering that Faucet adds four flow entries (to cover 802.x, IPv4/6 multicast and ethernet broadcasts) in addition to the entry just matching on the In Port and VLAN ID, there will be five flow entries per In-Port and VLAN combination. In a best case scenario using a switch with only four ports with only a single native VLAN configured, this will result in 20 flow entries plus the default drop entry for a total of 21. However, on a 48 port Top-of-Rack switch, the flow count will be 241 (48 ports * 1 vlan * 5 entries + 1). This, of course, is not that many flows either, but it is good to know what Faucet will require when putting it in deployment. For example, on an aggregator switch with hundreds of VLANs configured, there may well be tens of thousands of flow entries, many of which will require masked matching (fixed prefix for IPv4/6 multicast, full mask for 802.x) which requires TCAM or software matching, and this may place limits on which switches you can choose for the scenario you are in.

Another thing to note, but one that is much easier to mitigate and will hopefully be resolved in a future version of Faucet, is the number of actions attached to each flow entry. When flooding on a VLAN, the matched flow entry will have a number of actions equal to the number of ports assigned to that VLAN minus 1 (all but the receiving port), plus optional mirror and VLAN pop actions. Using the example of a 48 port ToR switch again with all ports in a single native VLAN, this requires 1 Pop VLAN action and 47 Output actions. Some hardware switches have a limited amount of actions that can be assigned to a flow entry. Thankfully, these actions can be moved into Group Table entries and reduce the number of actions required per flow entry significantly so this shouldn’t be a limitation for long.

In both cases, the number of actions and flow entries can be significantly reduced by being as specific as possible in Faucet’s configuration by ensuring that VLANs are only assigned to ports that actually need to service that traffic. Not only does this make it easier on the switches, but also helps provide better security.

Summary

We would like to thank REANNZ, ONF, and their contributors for the Faucet controller. The full source code for Faucet v1.1 can be viewed on REANNZ Faucet repository.

We hope that this article has provided you with new insights on the Faucet pipeline. I could not cover how all the features work in this article, but I am interested in reading any feedback. What areas would you like to see expanded on? Are there any network topologies or features you would like to see implemented utilizing Faucet? Please let us know in the comments below.

Share This