Select Page

Table of Contents

Now that we’ve explored Ryu’s API and its example SimpleSwitch, let’s move on to a new switch Ryu app written from scratch. SimpleSwitch Reimagined (SS2) is Inside OpenFlow’s example L2 switch controller application for Ryu that expands on what we’ve learned about the Ryu API while starting to use one of the most basic features of OpenFlow 1.3 and above that Ryu’s SimpleSwitch ignored: multi-table flow pipelines. Using multiple tables allows a controller application to implement more logic and state in the switch itself, reducing the load on the controller. This is especially useful where packets are routed at line rate in hardware OpenFlow switches or even software switches controlled by a remote controller. SS2 is also designed to keep the packets flowing with as little latency as possible during the learning process by taking advantage of multiple instructions in a single flow entry. These are simple features supported in modern OpenFlow switches (both hardware and software) that make designing your network pipeline much easier and fun!

SimpleSwitch Reimagined is heavily influenced by Faucet, an Open Source commercial-grade OpenFlow controller application for Ryu. Our purpose was not to rewrite Faucet, but to write an easy-to-understand controller application for learning more advanced OpenFlow concepts. As a result, SS2 does not include some of Faucet’s features such as VLAN learning, but is written to be very easy to follow and to teach useful design patterns for Ryu controller applications and OpenFlow in general. This tutorial will run through the core of SS2, explaining the thinking and design behind every bit of code and hopefully provide you with ideas on writing your own controller application. The structure and concepts employed by SS2 will also help introduce the logic behind Faucet when we explore it in the near future.

Downloadable Content – SimpleSwitch Reimagined Source Code and Tests

This includes the full source code in a ready to run package. Some of the utility code not related to OpenFlow or SDN is not listed in this article, so the full source code archive is required to follow along with the examples.

Download SimpleSwitch Reimagined v0.1.2

For convenience, the source code here can also be viewed on GitHub.

Related Articles and Downloadable Content

This article is a culmination of all the previous articles in the Core OpenFlow Track. Before starting through this tutorial, we suggest you at least run through the following articles:

You will also need the following free downloadable content to follow along with the examples:

If you already followed along with the previous tutorials, you should already have these packages. The only one you may need to download again is the Inside OpenFlow Postman Collection if you downloaded it before this article was completed.

Overview

SimpleSwitch Reimagined (SS2) is a Ryu controller application utilizing multi-table and multi-instruction support features available in OpenFlow 1.3 and above. SS2 is written with DRY principles in mind and tries to keep network logic and implementation separated to allow for easy code browsing and modification. It is meant as a platform that can be built on in your own projects and is under a standard MIT license.

Table Layout and High Level Logic

SimpleSwitch Reimagined utilizes several tables in each datapath (switch) which allow some state and logic to reside solely on that switch. This logic and state can both allow the controller to immediately learn of topology changes while also reducing load on the controller. Allowing the switch to do this introduces some interesting opportunities. Since the controller doesn’t have to keep track of which hosts are learned and is only bothered when the switch doesn’t recognize source host address, the learning process could theoretically occur on multiple high-availability controllers without the need for shared state. That is really an advanced topic or later, but keep in mind that by moving as much as we can over to the switch, we can add parallelization and high availability to our network logic and routing. The flow entries and tables employed here allows an initialized switch to function even if the controller goes offline for a few minutes.

Further, the table and flow entry structure allow for expansion by other controller applications without interfering with the core L2 learning component of SS2. This is accomplished by tagging all flow entries entered by SS2 Core with a cookie identifier. When a switch attaches itself to the controller, SS2 will delete all flows in all tables it uses that match that cookie. This allows, for example, a separate controller application to add higher priority flow entries to some of these tables and have those entries remained untouched.

SS2 Core utilizes four tables: the ACL table, L2 Filter table, ETH_SRC table, and ETH_DST table.

SimpleSwitch OpenFlow Multi-Table Flow

ACL Table

Internally table_acl, this is the main entry table for all packets by being on table 0. The only flow entry is a table-miss entry that forwards all traffic to the L2 Filter table. As the name suggests, it can be used for high priority ACL flow entries. This table can also be used as a “registration” table with medium priority flow entries passing traffic to other tables controlled outside SS2. These topics will be covered in a later article.

L2 Filter Table

Internally table_l2_switch, this table contains flow entries that drop packets that should not be forwarded by a compliant L2 Hub or Switch. This includes dropping LLDP, STP BPDU, and broadcast sources. This mimics some of Faucets default drop flows. The table-miss flow entry forwards traffic to the ETH_SRC table.

ETH_SRC Table

Internally table_eth_src, this table is solely used to keep the state of learned devices and their associated physical ports. With timeouts that are guaranteed to be triggered before the ETH_DST table, all entries (other than table-miss) represent learned MAC/Port pairs that can be successfully routed in the next table. The entries include a match on the source MAC and port and instruct the switch to forward the packet to the next table. Essentially, these entries prevent the table-miss entry from being triggered for packets from already learned hosts.

If the packet fails to match any of these entries, the table-miss entry instructs the switch to send the first 256 bytes of the packet from an unlearned host to the controller for learning. At the same time, it forwards the packet to the next table as with the learned entries above. The table-miss entry in this table is the only entry added by SS2 that causes a packet to be sent to the controller.

By matching on both the eth_src of the packet and the in_port, if a host is moved from one port to another, that packet will fail to match the learned entry and trigger a new learning event.

ETH_DST Table

Internally table_eth_dst, this table handles the final routing of a packet. It contains some high priority entries that handle multicast and broadcast destination addresses, such as 802.x, IPv4/IPv6 multicast, and ethernet broadcast. These entries cause the packet to be flooded to all ports and mimics Faucet’s default behavior.

This table also includes entries that represent all learned destinations. SS2 Core adds entries to this table and the ETH_SRC table on learning a new destination. The entries here match on the eth_dst of the packet and instruct the switch to output the packet on the learned port.

Finally, the table-miss entry is triggered when the location of the destination is unknown. It instructs the switch to flood the packet to all ports. When the destination responds to the packets, that host will eventually be learned. While it is possible to flood much earlier in the pipeline (such as in the L2 Filter table), flooding this late allows the controller to learn destinations when hosts send discovery or other broadcast packets such as ARP.

Packet Flow and Learning

SimpleSwitch Reimagined (SS2) will set up switches with default flow entries that allow the switch to immediately act as an L2 Hub. As packets from unlearned hosts are mirrored to the controller, additional flow entries are added to mark that host as learned and provide the correct routing of packets to the learned port.

Default Flow Entries

SS2 OpenFlow Multi-Table Packet Flow - Defaults or Learning Disabled

When the datapath (switch) is first added to the controller, any previous flows added by the SS2 core application are deleted by matching on the configured cookie value. This ensures other flows are not touched when the controller resets the learning state for the switch. This also means that any previously learned destinations are also removed from the switch. In this clean state, any new packets will follow workflow:

  1. A new packet is received and processed through table 0, which is the ACL Table for SS2.
    • The packet is processed by the table-miss flow and immediately goes to the L2 Filter table.
  2. The packet is processed by the L2 Filter table.
    • The packet is dropped if it matches any of the following conditions:
      • Packet has an ethertype of LLDP
      • Packet has destination MAC address associated with STP BPDUs
      • Packet has a source MAC address of the ethernet broadcast address
    • All other packets are sent to the ETH_SRC table.
  3. The packet is processed by the ETH_SRC table. Since the switch is a clean state, no learned destinations exist.
    • The packet is processed by the table-miss flow.
      • Packet headers (first 256 bytes of the packet) are sent to the controller for learning.
      • The complete packet is sent to the ETH_DST table.
  4. The packet is processed by the ETH_DST table. Since the switch is in a clean state, no learned destinations exist.
    • The packet is flooded to all ports if it matches any of the following conditions:
      • Packet destination address matches any of the following addresses/masks
        • 01:80:c2:00:00:00 (mask: 01:80:c2:00:00:00) – 802.x
        • 01:00:5e:00:00:00 (mask: ff:ff:ff:00:00:00) – IPv4 Multicast
        • 33:33:00:00:00:00 (mask: ff:ff:00:00:00:00) – IPv6 Multicast
        • ff:ff:ff:ff:ff:ff (mask: ff:ff:ff:ff:ff:ff) – Ethernet Broadcast
    • All other packets are processed by the table-miss flow, which floods the packet.

Learned Flow Entries

SS2 OpenFlow Multi-Table Packet Flow with Learned Hosts

When the controller receives packets (sent by the table-miss flow on the ETH_SRC table), two flow entries are added to two different tables. A flow matching the in-port and source address of the learned host is placed in the ETH_SRC table to prevent the table-miss flow from triggering as long as the host remains on the learned port, or the timeout is reached. Another flow matching the destination address to the address of the learned host with an action to forward the packet to the learned port is added to the ETH_DST table. Any new packets follow the same workflow as above with the following exceptions:

While the packet is processed by the ETH_SRC table, the packet source is checked against the learned ports and addresses. So long as packets from the previously learned host continue to arrive on the expected port, the packet is immediately sent to the ETH_DST table. If the packet arrives on an unexpected port, the table-miss flow is triggered and the packet headers are sent to the controller for the new host location to be learned. When relearning a host, the previous flow entries added for the host are deleted before new ones are added.

While the packet is processed by the ETH_DST table, the packet destination is checked against the learned addresses. If the destination address is learned, the packet is directly output on the learned port.

Architecture Summary

Now that we’ve gone through all this detail on where the packets go, let’s talk about why. Here are the major points:

  • Faster Data Flow in All Cases – All traffic cases have associated rules in the switch that allow traffic to be passed without waiting for the controller to respond. Not only is this faster even with a software switch, it allows for line-rate processing in hardware switches. The controller can eventually respond with flow entries that optimize the process (preventing flooding of unicast packets).
  • Moved All Required State to the Switch – The flow entries in the switch are set up in a way that allows it to only bother the controller when there is a topology change or an unlearned host is seen. Essentially, it knows when it should ask for help.
  • Controller Can be Made Stateless – This major point is easy to overlook, but is one of the most powerful features the controller has. Since the switch contains all the required state to know when it absolutely needs to bother the controller, the controller doesn’t need to keep track of learned hosts except for a very short cache to ensure it processes a host only once before the switch implements the new learned flow entries. The system would even work without that caching and nothing breaks if the controller inserts the learned flow entries multiple times (the process is idempotent).
  • Controller Can be Made Parallelizable – Since the controller is stateless and the same incoming packets will always result in the same learned flows, there is nothing preventing the controller to be expanded to multiple processes. This doesn’t only apply L2 learning. There are many network functions that can be virtualized in the switch and be applied without the controller needing to know the state of the network. At most, the controllers may need to keep some common configuration, but that information would only be changed when actual changes to the network logic are needed and network state would not be stored there.

By changing the way we look at a software defined network and how it is controlled, we can create some pretty awesome SDN applications.

Testing with Mininet and Postman

Now that we’ve covered the high-level flow of packets through the switch as instructed by SS2, let’s give SS2 a test run.

Setup

First, load your virtual development workspace as set up in Creating a Development Workspace. Any Linux environment should work so long as Ryu 4.3+ and Mininet 2.3+ are installed and available globally. Download and extract the SimpleSwitch Reimagined downloadable content. The package will extract with the version number in the directory name. The screenshots below show version v0.1.2-noacl. The version available for download at the link above may be a later version, but these instructions should still work if you replace the version number as needed.

Inside OpenFlow Screenshot - Extracting SimpleSwitch Reimagined

Running Unit Tests

Before continuing with the manual examples, run the unittests to ensure your environment is set up correctly. These instructions are in the README.md file in the root of the package. If you see tests fail (such as all tests in test_l2_learning.py), check the errors and ensure that Ryu and Mininet are properly installed. As per the README instructions, the tests must be run with sudo as Mininet requires root permissions.

Inside OpenFlow Screenshot - SS2 Unittests

Testing with the Datacenter Topology

Now that we know the SS2 app can function correctly, let’s make sure we have the proper Mininet topologies. We will be using the configurable datacenter topology as developed in Custom Mininet Topologies and Introducing Atom. The package containing all the topologies from that article are available from the Example Mininet Topologies downloadable content. Following the Custom Mininet Topologies article, we will assume these are available at ~/ofworkspace/mininet-topologies.

Open two terminals side-by-side. On one, change to the directory with the Mininet topologies. In the other, change to the directory with the SS2 application.

Run ryu-manager with the SS2 app (ss2.core) in the terminal you are using for Ryu. Also include the REST API application so we can poke around later (ryu.app.ofctl_rest).

Inside OpenFlow Screenshot - Starting Ryu with SS2 and OFCTL REST

In the terminal for Mininet, start the configurable datacenter topology using the default values for the topology configuration and specifying to auto-set MACs and use a remote controller:

Inside OpenFlow Screenshot - Mininet Startup with Datacenter Topology

Test connectivity by issuing the pingall command in the Mininet console. There should be no dropped packets.

Inside OpenFlow Screenshot - Mininet Datacenter Topology pingall

Retrieving Flow Entries with Postman

Now that our SimpleSwitch Reimagined controller application is tested and running, let’s take a peek at the tables to make sure they follow the design above. To easily run through this section, download the latest version of the Inside OpenFlow Postman Collection. If you already registered and downloaded this before, you can log in and download without going through the checkout process. Please note that all downloadable content here is free to download. You may also want to run through Interactive Ryu with Postman to review the usage of Postman.

Go ahead and start Postman. We will be using the requests under the “Article: Simple Switch Reimagined” folder of the “Inside OpenFlow” collection. With SS2 and Mininet running, select the “Get all switches” request and click the blue Send button. If you are not using our Postman collection, submit a GET request to http://localhost:8080/stats/switches. You should see a total of five datapath IDs listed. They may be listed out of order, but should include 1 (s1, aggregate switch), 17 (s1r1), 33 (s1r2), 49 (s1r3), and 65 (s1r4). This is assuming that Mininet is running with the Configurable Datacenter topology with the default configuration as started in “Testing with the Datacenter Topology” section above.

Inside OpenFlow Screenshot - Postman Requesting Ryu REST API Datapath List

Depending on how much time has elapsed, the learned host flow entries may or may not exist in the tables in the switches. When Mininet creates the switch environment, SS2 Core automatically learns the hosts because of various broadcast messages sent from the hosts on startup. Let’s ensure a clean start by restarting Ryu with the SS2 Core application as above, but without restarting Mininet. Ryu can be stopped by pressing Ctrl+C in Ryu’s console, then press the up arrow key to select the previous shell command (which started that instance of Ryu) and press enter.

Inside OpenFlow Screenshot - Restarting Ryu for a Clean Switch State

Since the hosts stay active, they should not send any additional packets until we ping them, and having the switches reconnect to a freshly restarted SS2 controller will cause any existing SS2 flow entries to be removed and replaced with the defaults.

Now bring Postman back up and run the “Get aggregate switch flows” request (/stats/flow/1). This will query Ryu for all flow entries on the aggregate switch (DPID 1) and should result in a total of 12 flow entries, which are the default flow entries added to the four tables used by SS2 Core.

We can drill down by requesting flow entries only in a certain table. Run the “Get ETH_DST Table Flows (aggregate switch)” request (POST {"table_id":103} to /stats/flow/1). This should return five flow entries, four of which match our default broadcast behavior for this table (ETH_DST) given certain destination MAC addresses. The remaining flow entry is the table-miss flow with a low priority that will flood the packet to all ports (highlighted below).

Inside OpenFlow Screenshot - Viewing Default Flows in Postman

Let’s force a learning event by running the pingall command in the Mininet console. Running the Postman request again will show several new flow entries, each with an idle timeout. One such flow is highlighted below. Since we are querying the aggregate switch, several MAC addresses will be associated with each port representing the top-of-rack switch that the host is on. In this case, if a packet has a destination MAC address of 00:00:00:00:00:0b, it will be sent out port 3.

Inside OpenFlow Screenshot - Showing Learned Destination in Postman (ETH_SRC)

Looking at the associated flow in the ETH_SRC table shows that the switch will expect packets with a source address of 00:00:00:00:00:0b to also show up on port 3. If a packet arrives with that source address, but on a different port, then this flow entry will not match and the packet will fall to the table-miss flow entry, which triggers a learning event in the controller.

Inside OpenFlow Screenshot - Showing Learned Destination Flow ETH_DST

Simulating a Host Move

There are many cases within a network as to why a host will move between switches. The most common might be the migration of a VM from one hypervisor to another, even across the datacenter. Wireless devices that switch access points while the device is moving is another example.

We can simulate a host move in Mininet by disconnecting the existing host interface, then creating a new host interface with the same MAC address on a new port and, in this case, a different switch. We will move h1r1 from s1r1 to s1r4 (move the first host on the first top-of-rack switch to the top-of-rack switch on the last rack). Here is what we will be doing:

  1. Delete all interfaces on h1r1 (first host on the TOR switch in the first rack). This simulates disconnecting the cable.
  2. Create a new link between s1r4 and h1r1 (TOR on last rack and host on first rack). This creates a new port on s1r4 and a new interface on h1r1 (see below) and a virtual link between them.
  3. Set the MAC address of the new interface on h1r1 to the original MAC address of the deleted interface, which was 00:00:00:00:00:01.
  4. Configure the IP address on the new interface on h1r1 to the original IP address of the deleted interface, which was 10.0.0.1/8.
  5. Instruct s1r4 to attach the new virtual link to the OVS instance running on that virtual switch.
A Better Way?

It would be nice to just be able to disconnect the virtual link between h1r1 and s1r1 and create a new one between h1r1 and s1r4 without deleting or creating interfaces, but I have not found a way to do this in Mininet. If you know of a way, please leave a comment below. I would be very interested to know!

In order to perform these steps, we will need to run some Python commands in Mininet. Thankfully, the CLI provided by Mininet allows us to do just that. The commands we will be using are very similar to the ones used in the unittests provided with SS2, so if you would like to learn more, check those out.

The learned rules have a timeout of five minutes by default, so all of the following will have to occur within that five minutes in order for the test to be valid. Let’s prepare by switching to Postman and loading and running the “Get Dest Flow for h1r1 (aggregate switch)” request. It may or may not show a flow entry. Perform a quick pingall in Mininet and run the request again if there are no flow entries listed. Don’t worry, the timer hasn’t started yet. You should see something like this:

Inside OpenFlow Screenshot - Ryu REST API Filter for Learned Destination Flow

We are querying the aggregate switch, which routes traffic between all the top-of-rack switches. The host h1r1 is currently connected to the TOR switch on that rack and that switch is connected to port 1 of the aggregate switch. The ports are similarly numbered for the other racks, so rack 2 is on port 2 and so on. When the move is finished, this entry should output to port 4 since the host will be moved to rack 4.

Performing the Move

Stop the running instances of Mininet and Ryu. Start Ryu back up first, then Mininet.

Inside OpenFlow Screenshot - Restarting Mininet and Ryu for simulated host move

Run the pingall command on Mininet and run the “Get Dest Flow for h1r1 (aggregate switch)” request again. You should see something like this:

Inside OpenFlow Screenshot - Verify Learned Destination before Host Move

In Mininet, run the following command as one line:

This performs the steps we listed above for emulating a host move. Specifically:

  • px – Instructs Mininet to run the rest of the line as Python code.
  • h1r1.deleteIntfs() – Deletes all interfaces on h1r1. The only interface is the one connected to the TOR switch on rack 1
  • link=net.addLink(s1r4, h1r1, addr2="00:00:00:00:00:01") – This registers and sets up a new link between the TOR switch on rack 4 and h1r1. In order to do this, new interfaces are created on both the virtual host and virtual switch, then linked. The addr2 keyword argument instructs Mininet to preconfigure the MAC address for the virtual host. This is the MAC address of the original interface deleted in the first instruction.
  • link.intf2.config(ip="10.0.0.1/8") – This instructs the second interface on the new link to be configured with the specified IP address. This allows the L3 ping to function.
  • s1r4.attach(link.intf1) – This instructs the virtual switch to attach the newly created interface on the switch. Otherwise the virtual host that runs the OVS instance will contain the new interface, but the OVS instance will not be configured to use it as a port.

Inside OpenFlow Screenshot - Mininet Host Move to Different Switch

Run pingall again. No packets should be dropped.

Inside OpenFlow Screenshot - Verify Host Move in Mininet

Send the Postman request again. The actions should change to output the packet on port 4, which links to the TOR switch on rack 4 and the new location of the host.

Inside OpenFlow Screenshot - Verify New Learned Destination in Postman

For more information on how to use this method in automated tests, see the unittests for SS2 in test_l2_learning.py. As an additional exercise, try running the same steps above simulating the host move using Ryu’s provided SimpleSwitch application by running ryu-manager ryu.app.simple_switch_13.

Notes on Production Implementations

The reason why the pingall command works perfectly the first time is due to the order that Mininet chooses to run individual ping commands. h1r1 is the first host to ping another host. Since it sends traffic first and from an unexpected location, the new location of the host is learned before the target host has a chance to respond. If another host was to try to ping h1r1 first and the moved host has not sent any traffic yet, the switch will still try to route packets to the old location. This is true for any L2 learning switch. Thankfully, most operating systems will send broadcast traffic of some sort when an interface is connected to a switch, so the learning time should be minimal even in this case.

When using orchestration software to live migrate VMs between hypervisors, you may want to have that software contact the controller application to unlearn (or flood) the packets that should be received by the moving VM with a higher priority flow entry until the migration is complete. You could even write your integration software to be more specific and ensure that the traffic is mirrored to just the old and new location. With the appropriate amount of thought and design in your datacenter or product, you can ensure that migrated VMs can have minimal to no packet loss through the life of the migration, meeting any high availability requirements.

Feature Comparison

Let’s see how we did against the SimpleSwitch application provided by Ryu.

Feature Ryu’s SS IOF’s SS2
Ryu’s SimpleSwitch Features
Learn Host Location yes yes
Insert Flow Entries for Learned Addresses yes yes
Detect when a Host Moves yes (on flood) yes
Flood Multicast and Broadcast Traffic yes yes
Function on Switches with only a Single Table yes no
Controller Access to Learned Addresses yes no[1]
Great Tool for Introducing Controller Applications yes no[2]
Additional SimpleSwitch Reimagined Design Goals
Great Tool for Teaching Multi-Table Flows and Offloading State and Logic no yes
Clear Flow Entries when Switch Connects no yes[3]
Delete and Update Flow Entries on Host Move no yes
Handle Broadcast and Multicast on Switch no yes
Easily Expandable and Configurable no yes
  1. Except for a temporary host cache to prevent relearning the same address in a short amount of time while the switch flow entries are updated. Long term storage is not required by SS2 Core.
  2. Ryu’s SimpleSwitch implements an L2 switch in under 100 SLOC entirely in a single module. SS2 Core is meant as an intermediate instruction tool and requires hundreds of SLOC across several modules.
  3. Only flows added by SS2 Core, identified by a cookie ID.

All in all, I think we met our goals fairly well. We have a great educational tool that serves as the next logical step in learning how to design and implement your own controller applications. To be clear, both Ryu’s SimpleSwitch and Inside OpenFlow’s SimpleSwitch Reimagined have different goals and are both very useful. It would be much more difficult to explain our SimpleSwitch Reimagined without dissecting and understanding Ryu’s SimpleSwitch first.

SimpleSwitch Reimagined In Detail

We’ve seen what SS2 can do. Now let’s see how it is organized and how the different parts work together.

Terminology and Icon Legend

Any reference to a variable in this article should be prefixed with an icon that represents its type. It may optionally be followed by a reference to the class that the variable is an instance of. Here is a list of the icons in use, with an example name:

  • ryu.ofproto.ether – A Module, which in Python is usually defined by an entire file such as ryu/ofproto/ether.py for ryu.ofproto.ether, or ryu/ofproto/__init__.py for ryu.ofproto.
  • ryu.util.round_up – A Function, a defined block of code that can be called with or without arguments and produce a return value. Generally available globally under a module, but Functions can be defined inside any code block and are subject to normal Python namespacing and scoping rules. Functions belonging as a direct ancestor to a Module are generally shared utilities, helpers, or Decorators.
  • ryu.ofproto.ofproto_v1_3_parser.OFPSwitchFeatures – A Class, a declared shared namespace object that may be Called to instantiate any number of Instance objects. Usually, Classes are used to provide a framework for a specific type of object, potentially including Methods (Class Functions) that an instance, or client of the instance, can use to provide common behavior.
  • self.mac_to_port – A public Data Attribute, a reference to another object, usually referenced as a Class or Instance Variables, but the target object may be anything. In this article, Attributes are followed by the Class that was used to instantiate the Object referenced by the Attribute, if known.
  • self.add_flow – A public Method, a Function defined in a Class and available to Instances along with the Class itself.
  • self._reserved – A private Data Attribute
  • self.__init__ – A private Method
  • int – A native type
  • ev – An argument passed into a Function or Method
  • parser – A local variable
  • True – A Python primitive value

Application Structure

The SimpleSwitch Reimagined (SS2) application is split into several modules that each handles their own areas of concern. Here is a listing of modules and classes in SS2:

  • ss2src – The SS2 application package itself
    • appsrc – Base methods for SS2 RyuApp classes
    • configsrc – The configuration loader and parser for any SS2 app
    • coresrc – The SS2 Core controller application and contains the high-level switch logic
    • utilsrc – Utilities for SS2

By splitting our core application into several modules and enforcing a separation between our high-level logic and low-level actions, we are able to make the code much easier to follow. Three of the more important modules are documented below for reference. Further down the article, we will get into the details of each method.

ss2.core.SS2Core

ss2.core.SS2Coresrc provides the core L2 learning logic. This is also the RyuAppsrc, doc subclass specified when starting SS2 via the ryu-manager. There are a number of methods here, starting with the event handlers:

Next, helper methods are defined for the instance of SS2Coresrc run by Ryu. All of these methods return a list of datapath messages that are sent to the switch to implement the required flow entries. These include the method called from the event handlers above and further methods that break down the process into more simple steps:

  • add_datapathsrc – Returns messages needed to ss2.app.SS2App.clean_all_flowssrc and self.add_default_flowssrc for the current datapath.
  • learn_sourcesrc – Returns messages needed to unlearn_sourcesrc in case the source MAC address was already learned with flow entries still in the switch, add_eth_src_flowsrc entries instructing the switch to prevent sending further packets from this source MAC and port to the controller, and add_eth_dst_flowsrc entries that will appropriately forward packets to this port when sent to this MAC address.
  • unlearn_sourcesrc – Returns messages that delete existing flow entries (SS2App.flowdelsrc) for previously learned host locations that may exist in the switch, both in the ETH_SRC and ETH_DST tables, then add a SS2App.barrier_requestsrc to ensure these flow deletion messages are processed completely before processing any further flow additions or modifications.
  • add_eth_src_flowsrc – Returns messages that add a flow entry (SS2App.flowmodsrc) that will SS2App.matchsrc the packet’s eth_src and in_port and instruct the switch to SS2App.goto_tablesrc ETH_SRC.
  • add_eth_dst_flowsrc – Returns messages that add a flow entry (SS2App.flowmodsrc) that will SS2App.matchsrc on the packet’s eth_dst and instruct the switch to SS2App.apply_actionssrc of SS2App.action_outputsrc on the current learned port.
  • add_default_flowssrc – Returns messages that create the appropriate flow entries to instruct the switch on the default packet pipeline expected by SS2. This includes adding a flow entry (SS2App.flowmodsrc) in the ACL table that forwards all packets that miss any other match condition in that table (table-miss) to the L2 Filter Table. Flow entries are added to the L2 Filter Table to automatically drop LLDP, STP BPDU, and broadcast source packets, then forward all remaining packets to the ETH_SRC table. A table-miss flow entry is added to the ETH_SRC table that forwards the first part of the packet to the controller for learning and forwards the whole packet to the next table, ETH_DST. Finally, flow entries are added to the ETH_DST table that automatically floods packets for certain destinations (802.x, IPv4/IPv6 multicast and ethernet broadcast) along with a table-miss flow entry that floods packets that do not match any learned host destinations.

ss2.app.SS2App

ss2.app.SS2Appsrc acts as a mixin to provide helper methods that make interfacing with the Ryu API easier. These are called from the SS2 Core application above by including SS2Appsrc as a secondary superclass to SS2Coresrc. Like above, some of these methods will return a list of datapath messages, but most are used to generate message segments when building a flow entry. These helper methods are kept in a separate class so it can be used in other SS2 modules in the future.

ss2.config

ss2.configsrc provides functions to load the default SS2 configuration (defaults.cfg) and custom configuration if provided (ss2.cfg in the current working directory). It provides a number of functions available to the rest of SS2 so parameters like table IDs and priority numbers can be changed as needed for your local installation. Further, it provides an easy-to-read way to apply this configuration in code. We won’t get into any more detail than what is listed here as its function is not specific to OpenFlow or Ryu, but it does illustrate the flexibility that a configuration system can provide. It will be used extensively in the SS2ACL add-on component for SS2.

  • DEFAULT_CONFIGsrcstr – Points to the location of the default configuration file.
  • DEFAULT_FILESsrclist<str> – List of potential configuration files to load in the current working directory (CWD).
  • AttrDictsrc – Subclasses dict to allow instances to get values either by the standard method (config["priority_high"]) or by accessing keys as attributes (config.priority_high). All configuration dictionaries are returned as AttrDictsrc instances.
  • get_parsersrc – Get the ConfigParserdoc instance with the default configuration and local configuration files already loaded.
  • read_configsrc – Reads the specified files into a ConfigParserdoc instance (using get_parsersrc), gets configuration items from the specified section (defaults to “Core”, uses get_sectionsrc) and returns a AttrDictsrc instance loaded with those configuration items (uses parse_types).
  • get_sectionsrc – Uses the ConfigParserdoc instance to read a specific section of the configuration file, applying defaults along the way. Returns a list of all items from the appropriate default sections and the section requested in order of defaults to most specific and may contain duplicates (later items override previous items).
  • get_subsectionssrc – Gets a list of subsections for the specified section (used later in SS2ACL for individual ACL rules).
  • parse_typessrc – Converts a list of items into an AttrDictsrc instance while coercing the string values returned by ConfigParserdoc into float, int, and bool where possible. Configuration keys with dot-seperated attributes (foo.bar) are automatically converted into nested AttrDictsrc instances.

Code Internals – SS2Core

If you are still interested in how these actions are performed, continue reading below where we explain each method in detail. This first section covers the high-level logic in SS2Coresrc

Core Application, Registration and Initialization

ss2.core.SS2Coresrc provides the core L2 learning logic. This is also the RyuAppsrc, doc subclass specified when starting SS2 via the ryu-manager.

This application specifies that it is compatible with OpenFlow 1.3. When the application is initialized, configuration is read and a host cache initialized. The host cache is used later to deal with packet-in floods on high-speed flows from an unlearned host.

Event Handler for New Switches

As with simpleswitch_13.py, SS2 registers a handler for ryu.controller.ofp_event.EventOFPSwitchFeaturessrc, doc, which is triggered any time a new switch is added to the controller. This calls add_datapathsrc and sends the returned list of messages to the datapath (switch).

Event Handler for Incoming Packets, Packet Dissection

SS2 also registers a handler for ryu.controller.ofp_event.EventOFPPacketInsrc, doc. Packets sent to the controller via Packet-In messages are parsed to retrieve the ethernet headers. Checks are performed to ensure the host was not recently learned (flowmod flood protection). If the host should be learned, the packet source address and in-port are passed to learn_sourcesrc and the returned list of messages are sent to the switch.

Main Logic and Helper Methods

Much of the rest of the code for SS2Coresrc is written to be self explanatory. Many helper methods are used to break down tasks into more manageable chunks. All of these methods return lists of composed messages ready to send to the switch.

Adding a Datapath to the Controller

When a datapath (switch) is added to the controller, add_datapathsrc is called from switch_features_handlersrc to first clear all SS2-related flows from the datapath and then add the default flows required for the expected packet pipeline.

Learning the Host’s Location

Called by the packet_in_handlersrc, learn_sourcesrc aggregates the messages required to instruct the switch to unlearn a previous location for the host (if required), then add flows to the ETH_SRC and ETH_DST tables for the host’s new learned location.

Unlearning a Previous Host Location

Before adding new flow entries to instruct the switch of the learned location of a host, any previous flows for that host need to be removed. This is the function of unlearn_sourcesrc. As the pipeline is designed so that the switch keeps track of which hosts are learned, the controller may not know if a host was previously learned and, if so, on a different physical port. To resolve this, flow modification messages are sent to delete any flows in the ETH_SRC and ETH_DST tables that match on the MAC address of the current packet being processed by the controller, followed by a barrier request. The barrier request ensures that the flow deletions occur completely before further flows are added.

Adding Default Flow Entries

In order to set up the expected packet pipeline for the switch, several flows must be added when the switch connects to the controller. These flow entries are generated by add_default_flowssrc.

ACL Table Entry

The first flow entry is added to the ACL table. Currently, the ACL table (table 0 by default) only forwards packets to the L2 Filter table, but future components to SS2 (such as SS2ACL to be featured in a future article), or even other applications entirely, could use this table to register custom checks on traffic before it is handled by the L2 switch logic.

L2 Filter Entries

Next, packet drop rules are added to the L2 Filter table. These flow entries are like ACL entries in that they will drop packets that mean certain criteria. In this case, we are ensuring that certain packets that should not be forwarded by compliant L2 Hubs and Switches are appropriately dropped.

ETH_SRC Entries

The ETH_SRC Table simply needs a single table-miss flow entry that will used if a packet comes from an unlearned host or a host that has changed location on the network. This entry will send the beginning of the packet to the controller for learning and forward the full packet to the next table, ETH_DST.

ETH_DST Table Entries

The ETH_DST Table performs the actual forwarding of packets. Since it is possible to determine which packets should be flooded based on its destination, let’s get a few out of the way. Flow entries are added to flood common multicast and broadcast traffic with a high priority. Following that, a table-miss flow is added to flood any packets to unlearned hosts.

Finally, all of the messages generated are returned back to add_datapathsrc

Adding Learned Host Marker Flow Entries

Since we want the state of which destination addresses are learned in the switch itself, a flow entry needs to be added to the ETH_SRC table when a host’s destination address is learned. We give it a hard timeout so the flow is guaranteed to be removed from the switch before the ETH_DST table entry, which uses an idle_timeout.

Using a timeout also ensures that flow entries are not kept in the switch for destinations that no longer exist. When working with a large virtualization environment in this network, MAC address may not be recycled while VMs are torn down and brought up depending on the VM technology used. This is especially true for private clouds that use automatic orchestration and load balancing based on changing compute requirements throughout the day. Since the ETH_DST table entries use idle timeouts, traffic should not be interrupted for previously learned destinations while the relearning process is triggered by the hard timeout.

Adding Learned Host Destination Flow Entries

This is similar to adding the entry to ETH_SRC above except the action instructs the packet to be finally sent out on the appropriate physical port. An idle timeout is used here so learned destinations can continue to receive traffic directly while the switch and the controller are in the process of relearning the destination’s location if it was moved or that destination host has not responded in some time. The flow added here will eventually be deleted when a destination is learned again and replaced with a fresh flow entry.

Code Internals – SS2App

We previously mentioned that we plan on having more SS2 applications available, especially one providing customized ACL support. To make coding future apps easier, the helper methods were split off into its own class that is mixed into the application’s main class. Without this abstraction, writing something as simple as the SS2Core.add_eth_dst_flowsrc from above would look something like this:

The SS2Appsrc class does not subclass RyuAppsrc, doc so it isn’t accidentily registered as an application itself. Instead, is meant to be specified alongside RyuAppsrc, doc in a class’s superclass list.

Sending Messages to the Datapath

Although the datapath API has a built in send_msgsrc, doc method, this helper method allows sending a list of several messages to the datapath. Since the SS2 core application aggregates all the messages required to implement the requested operations before sending them out, this makes the core application code easier to read by removing the for-loop boilerplate in the event handlers.

The @staticmethoddoc decorator simply means that the method does not require access to the class instance data. As a result, the self argument is not specified in the method and Python is able to make additional optimizations. It is recommended that any method that does not require access to self should be written as a static method.

Building Apply Actions Messages

This helper makes generating a OFPInstructionActionssrc, doc message easier to read and is used in many places in SS2Coresrc.

Building Output Actions

This helper is a wrapper for OFPActionOutputsrc, doc which generates an action that instructs the switch to output the packet the specified port.

Building Goto Table Instructions

This is a simple wrapper around OFPInstructionGotoTablesrc, doc.

Building Match Conditions

In several of these helper methods in SS2Appsrc, a pattern of including additional keyword arguments (such as in_port) is seen even if not directly defined in the wrapped class. These arguments are primary listed in the code to allow static analysis tools such as autocomplete functions in IDEs to present the developer with common developer options. In these cases, the argument defaults to None and the arguments passed to the wrapped class only includes the additional keyword arguments if they were specified by the calling function (see line #58).

In the case of OFPActionOutputsrc, doc further above, the static analysis tools could easily see the port and max_len arguments, but more complex message builders like this one (OFPMatchsrc, doc) don’t bother listing all available match options in the method definition and instead uses a keyword argument mapping (**kwargs). Although this makes sense from Ryu’s perspective where there are too many options to list, static analysis tools can’t determine which keyword arguments are valid. As a compromise, these wrappers include the most common options used by this application, but can often still include additional keyword arguments if needed.

Building Barrier Request Messages

This is a simple wrapper around OFPBarrierRequestsrc, doc. Barrier Requests allow synchronous actions on OpenFlow switches. This is used in SS2Core.unlearn_sourcesrc to ensure flow entries are deleted completely before allowing further flow modifications to occur. The docstring below gives a full explanation.

Getting a List of All Tables Used by the Application

all_ss2_tablessrc checks the configuration of the current application (the instance that subclassed SS2Appsrc, such as SS2Coresrc) to get a list of all table numbers in use. This also marks where the static methods ends and the instance methods begin (requiring access to self).

This is also the first method that references the applications configuration, which must exist under self.config as an instance of ss2.config.AttrDictsrc. This is loaded by the __init__ method of the application. See SS2Core.__init__src for an example.

Creating Flow Modification Messages

This is a very useful helper method and the reason why I chose to split the helper methods into its own class. flowmodsrc generates an OFPFlowModsrc, doc message which automatically includes the current application’s OpenFlow cookie, which identify flows as being part of this application in a multi-application environment. This means that the application doesn’t need to specify the cookie when calling flowmodsrc and Flow Mod messages that would normally delete all flow entries in a table end up only deleting the flow entries that are a part of this application specifically. This creates a transparent way for the application to deal with its own flow entries while ensuring other application’s flow entries are not accidentally removed or modified.

Several of the more common keyword arguments are specified in the method definition to aid autocomplete tools while developing code, but also allows any keyword arguments that OFPFlowModsrc, doc supports.

Deleting Flow Entries

flowdelsrc is a wrapper for flowmodsrc that automatically assigns the Flow Mod command to OFPFC_DELETEsrc on OFPP_ANYsrc out port (unless specified) and in OFPG_ANYsrc out group. This allows for very easy-to-read flow deletion message generation code.

Cleaning All Flows for this Application

Here we see a use for the all_ss2_tablessrc method above in this clean_all_flowssrc method definition. It iterates through all tables used by this application and calls flowdelsrc with no priority or match condition. As explained in Creating Flow Modification Messages above, only flow entries with this application’s cookie are deleted.

 

What’s Next

We are going to take a break from instructing on the Ryu API and instead introduce Faucet, which inspired this controller application. Don’t worry, though, we will continue developing on SimpleSwitch Reimagined, including the addition of configurable ACLs, both at startup and at runtime through a REST interface! We also plan on adding Mirror support and exploring Intrusion Detection Systems (IDS). Sign up to our mailing list to keep updated with the latest articles!

Code Listing

The full source code for SimpleSwitch Reimagined is available for download for free. The code can also be browsed at GitHub. The version used in this article is v0.1.2-noacl.

 

Share This