diff --git a/CONTRIBUTORS b/CONTRIBUTORS new file mode 100644 index 0000000..5c6120b --- /dev/null +++ b/CONTRIBUTORS @@ -0,0 +1,39 @@ +Mininet Contributors + +Mininet is an open source project and we gratefully acknowledge +the many contributions to the project! If you have contributed +code into the project and are not on this list, please let us know +or send a pull request. + +Contributors include: + +Mininet Core Team + +Bob Lantz +Brandon Heller +Nikhil Handigol +Vimal Jeyakumar +Brian O'Connor + +Additional Mininet Contributors + +Gustavo Pantuza Coelho Pinto +Ryan Cox +Shaun Crampton +David Erickson +Glen Gibb +Andrew Ferguson +Eder Leao Fernandes +Vitaly Ivanov +Rich Lane +Murphy McCauley +James Page +Angad Singh +Piyush Srivastava +Ed Swierk +Isaku Yamahata + +Thanks also to everyone who has submitted issues and pull +requests on github, and to our friendly mininet-discuss +mailing list! + diff --git a/INSTALL b/INSTALL index d3e4695..8771678 100644 --- a/INSTALL +++ b/INSTALL @@ -2,7 +2,7 @@ Mininet Installation/Configuration Notes ---------------------------------------- -Mininet 2.0.0 +Mininet 2.1.0 --- The supported installation methods for Mininet are 1) using a @@ -15,7 +15,7 @@ like to contribute an installation script, we would welcome it!) 1. Easiest "installation" - use our pre-built VM image! The easiest way to get Mininet running is to start with one of our - pre-built virtual machine images from + pre-built virtual machine images from Boot up the VM image, log in, and follow the instructions on the Mininet web site. @@ -42,7 +42,7 @@ like to contribute an installation script, we would welcome it!) sudo rm /usr/local/bin/ovs* sudo rm /usr/local/sbin/ovs* -3. Native installation from source on Ubuntu 11.10+ +3. Native installation from source on Ubuntu 12.04+ If you're reading this, you've probably already done so, but the command to download the Mininet source code is: @@ -56,7 +56,7 @@ like to contribute an installation script, we would welcome it!) `install.sh` is a bit intrusive and may possibly damage your OS and/or home directory, by creating/modifying several directories - such as `mininet`, `openflow`, `oftest`, `pox`, or `noxcosre`. + such as `mininet`, `openflow`, `oftest`, `pox`, etc.. Although we hope it won't do anything completely terrible, you may want to look at the script before you run it, and you should make sure your system and home directory are backed up just in case! @@ -81,6 +81,11 @@ like to contribute an installation script, we would welcome it!) This takes about 4 minutes on our test system. + You can change the directory where the dependencies are installed using + the -s flag. + + mininet/util/install.sh -s -a + 4. Creating your own Mininet/OpenFlow tutorial VM Creating your own Ubuntu Mininet VM for use with the OpenFlow tutorial @@ -108,7 +113,7 @@ like to contribute an installation script, we would welcome it!) modules (e.g. tun and ofdatapath for the reference kernel implementation) must be loaded. - * Python, `bash`, `ping`, `iperf`, etc.` + * Python, `bash`, `ping`, `iperf`, etc. * Root privileges (required for network device access) diff --git a/LICENSE b/LICENSE index 704d157..de9b391 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ -Mininet 2.0.0 License +Mininet 2.1.0 License -Copyright (c) 2012 Open Networking Laboratory +Copyright (c) 2013 Open Networking Laboratory Copyright (c) 2009-2012 Bob Lantz and The Board of Trustees of The Leland Stanford Junior University diff --git a/Makefile b/Makefile index c0c53f2..c989e61 100644 --- a/Makefile +++ b/Makefile @@ -1,6 +1,6 @@ MININET = mininet/*.py TEST = mininet/test/*.py -EXAMPLES = examples/*.py +EXAMPLES = mininet/examples/*.py MN = bin/mn BIN = $(MN) PYSRC = $(MININET) $(TEST) $(EXAMPLES) $(BIN) diff --git a/README.md b/README.md index 724d5fa..3c965a9 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ Mininet: Rapid Prototyping for Software Defined Networks *The best way to emulate almost any network on your laptop!* -Version 2.0.0 +Version 2.1.0 ### What is Mininet? @@ -67,32 +67,32 @@ Mininet includes: `mn -c` -### New features in 2.0.0 +### New features in 2.1.0 -Mininet 2.0.0 is a major upgrade and provides -a number of enhancements and new features, including: +Mininet 2.1.0 provides a number of bug fixes as well as +several new features, including: -* "Mininet-HiFi" functionality: +* Convenient access to Mininet() as a dict of nodes +* X11 tunneling (wireshark in Mininet hosts, finally!) +* Accurate reflection of the Mininet() object in the CLI +* Automatically detecting and adjusting resource limits +* Automatic cleanup on failure of the `mn` command +* Preliminary support for running OVS in user space mode +* Preliminary support (`IVSSwitch()`) for the Indigo Virtual Switch +* support for installing the OpenFlow 1.3 versions of the reference + user switch and NOX from CPqD +* The ability to import modules from mininet.examples - * Link bandwidth limits using `tc` (`TCIntf` and `TCLink` classes) +We have provided several new examples (which can easily be +imported to provide useful functionality) including: - * CPU isolation and bandwidth limits (`CPULimitedHost` class) +* Modeling separate control and data networks +* Connecting Mininet hosts the internet (or a LAN) using NAT +* Creating per-host custom directories using bind mounts -* Support for Open vSwitch 1.4+ (including Ubuntu OVS packages) - -* Debian packaging (and `apt-get install mininet` in Ubuntu 12.10) - -* First-class Interface (`Intf`) and Link (`Link`) classes for easier - extensibility - -* An upgraded Topology (`Topo`) class which supports node and link - customization - -* Man pages for the `mn` and `mnexec` utilities. - -[Since the API (most notably the topology) has changed, existing code -that runs in Mininet 1.0 will need to be changed to run with Mininet -2.0. This is the primary reason for the major version number change.] +Note that examples contain experimental features which might +"graduate" into mainline Mininet in the future, but they should +not be considered a stable part of the Mininet API! ### Installation @@ -116,19 +116,19 @@ Mininet mailing list, `mininet-discuss` at: ### Contributing -Mininet is an open-source project and is currently hosted at -. You are encouraged to download the code, -examine it, modify it, and submit bug reports, bug fixes, feature -requests, and enhancements! +Mininet is an open source project and is currently hosted +at . You are encouraged to download +the code, examine it, modify it, and submit bug reports, bug fixes, +feature requests, new features and other issues and pull requests. +Thanks to everyone who has contributed to the project +(see CONTRIBUTORS for more info!) Best wishes, and we look forward to seeing what you can do with Mininet to change the networking world! ### Credits -The Mininet Team: +The Mininet 2.1.0 Team: * Bob Lantz -* Brandon Heller -* Nikhil Handigol -* Vimal Jeyakumar +* Brian O'Connor diff --git a/debian/control b/debian/control index e2297ae..b89b3b3 100644 --- a/debian/control +++ b/debian/control @@ -17,10 +17,13 @@ Architecture: any Depends: openvswitch-switch, telnet, + socat, + iperf, + cgroup-bin, ${misc:Depends}, ${python:Depends}, ${shlibs:Depends} -Recommends: iperf, openvswitch-controller, socat +Recommends: openvswitch-controller Description: Process-based network emulator Mininet is a network emulator which uses lightweight virtualization to create virtual networks for rapid diff --git a/debian/copyright b/debian/copyright index 4c9fd63..5b43112 100644 --- a/debian/copyright +++ b/debian/copyright @@ -3,7 +3,7 @@ Upstream-Name: mininet Source: https://github.com/mininet/mininet Files: * -Copyright: 2012 Open Networking Laboratory, +Copyright: 2012-2013 Open Networking Laboratory, 2009-2012 Bob Lantz, 2009-2012 The Board of Trustees of the Leland Stanford Junior University diff --git a/examples/README b/examples/README.md similarity index 76% rename from examples/README rename to examples/README.md index 7b3f795..4be5564 100644 --- a/examples/README +++ b/examples/README.md @@ -1,4 +1,3 @@ - Mininet Examples ======================================================== @@ -7,114 +6,114 @@ Mininet's Python API. ======================================================== -### baresshd.py: +#### baresshd.py: This example uses Mininet's medium-level API to create an sshd process running in a namespace. Doesn't use OpenFlow. -### consoles.py: +#### consoles.py: This example creates a grid of console windows, one for each node, and allows interaction with and monitoring of each console, including graphical monitoring. -### controllers.py: +#### controllers.py: This example creates a network with multiple controllers, by -using a custom Switch() subclass. +using a custom `Switch()` subclass. -### controllers2.py: +#### controllers2.py: This example creates a network with multiple controllers by creating an empty network, adding nodes to it, and manually starting the switches. -### controlnet.py: +#### controlnet.py: This examples shows how you can model the control network as well as the data network, by actually creating two Mininet objects. -### cpu.py: +#### cpu.py: This example tests iperf bandwidth for varying CPU limits. -### emptynet.py: +#### emptynet.py: This example demonstrates creating an empty network (i.e. with no topology object) and adding nodes to it. -### hwintf.py: +#### hwintf.py: This example shows how to add an interface (for example a real hardware interface) to a network after the network is created. -### limit.py: +#### limit.py: This example shows how to use link and CPU limits. -### linearbandwidth.py: +#### linearbandwidth.py: This example shows how to create a custom topology programatically by subclassing Topo, and how to run a series of tests on it. -### miniedit.py: +#### miniedit.py: This example demonstrates creating a network via a graphical editor. -### multiping.py: +#### multiping.py: This example demonstrates one method for monitoring output from multiple hosts, using `node.monitor()`. -### multipoll.py: +#### multipoll.py: This example demonstrates monitoring output files from multiple hosts. -### multitest.py: +#### multitest.py: This example creates a network and runs multiple tests on it. -### nat.py: +#### nat.py: This example shows how to connect a Mininet network to the Internet using NAT. It also answers the eternal question "why can't I ping -google?" +`google.com`?" -### popen.py: +#### popen.py: This example monitors a number of hosts using `host.popen()` and `pmonitor()`. -### popenpoll.py: +#### popenpoll.py: This example demonstrates monitoring output from multiple hosts using -the `node.popen()` interface (which returns Popen objects) and `pmonitor()`. +the `node.popen()` interface (which returns `Popen` objects) and `pmonitor()`. -### scratchnet.py, scratchnetuser.py: +#### scratchnet.py, scratchnetuser.py: These two examples demonstrate how to create a network by using the lowest- level Mininet functions. Generally the higher-level API is easier to use, but scratchnet shows what is going on behind the scenes. -### simpleperf.py: +#### simpleperf.py: A simple example of configuring network and CPU bandwidth limits. -### sshd.py: +#### sshd.py: -This example shows how to run an sshd process in each host, allowing -you to log in via ssh. This requires connecting the Mininet data network +This example shows how to run an `sshd` process in each host, allowing +you to log in via `ssh`. This requires connecting the Mininet data network to an interface in the root namespace (generaly the control network already lives in the root namespace, so it does not need to be explicitly connected.) -### tree1024.py: +#### tree1024.py: This example attempts to create a 1024-host network, and then runs the CLI on it. It may run into scalability limits, depending on available -memory and sysctl configuration (see `INSTALL`.) +memory and `sysctl` configuration (see `INSTALL`.) -### treeping64.py: +#### treeping64.py: This example creates a 64-host tree network, and attempts to check full -connectivity using ping, for different switch/datapath types. +connectivity using `ping`, for different switch/datapath types. diff --git a/examples/__init__.py b/examples/__init__.py new file mode 100644 index 0000000..4b2e73c --- /dev/null +++ b/examples/__init__.py @@ -0,0 +1,4 @@ +""" +Mininet Examples +See README for details +""" diff --git a/examples/baresshd.py b/examples/baresshd.py index a714edb..081a812 100755 --- a/examples/baresshd.py +++ b/examples/baresshd.py @@ -2,6 +2,7 @@ "This example doesn't use OpenFlow, but attempts to run sshd in a namespace." +import sys from mininet.node import Host from mininet.util import ensureRoot @@ -27,6 +28,10 @@ f.write( 'Welcome to %s at %s\n' % ( h1.name, h1.IP() ) ) f.close() print "*** Running sshd" -h1.cmd( '/usr/sbin/sshd -o "Banner /tmp/%s.banner"' % h1.name ) +cmd = '/usr/sbin/sshd -o "Banner /tmp/%s.banner"' % h1.name +# add arguments from the command line +if len( sys.argv ) > 1: + cmd += ' ' + ' '.join( sys.argv[ 1: ] ) +h1.cmd( cmd ) print "*** You may now ssh into", h1.name, "at", h1.IP() diff --git a/examples/bind.py b/examples/bind.py new file mode 100755 index 0000000..2c317cf --- /dev/null +++ b/examples/bind.py @@ -0,0 +1,197 @@ +#!/usr/bin/python + +""" +bind.py: Bind mount prototype + +This creates hosts with private directories as desired. +""" + +from mininet.net import Mininet +from mininet.node import Host +from mininet.cli import CLI +from mininet.util import errFail, quietRun, errRun +from mininet.topo import SingleSwitchTopo +from mininet.log import setLogLevel, info, debug + +from os.path import realpath +from functools import partial + + +# Utility functions for unmounting a tree + +MNRUNDIR = realpath( '/var/run/mn' ) + +def mountPoints(): + "Return list of mounted file systems" + mtab, _err, _ret = errFail( 'cat /proc/mounts' ) + lines = mtab.split( '\n' ) + mounts = [] + for line in lines: + if not line: + continue + fields = line.split( ' ') + mount = fields[ 1 ] + mounts.append( mount ) + return mounts + +def unmountAll( rootdir=MNRUNDIR ): + "Unmount all mounts under a directory tree" + rootdir = realpath( rootdir ) + # Find all mounts below rootdir + # This is subtle because /foo is not + # a parent of /foot + dirslash = rootdir + '/' + mounts = [ m for m in mountPoints() + if m == dir or m.find( dirslash ) == 0 ] + # Unmount them from bottom to top + mounts.sort( reverse=True ) + for mount in mounts: + debug( 'Unmounting', mount, '\n' ) + _out, err, code = errRun( 'umount', mount ) + if code != 0: + info( '*** Warning: failed to umount', mount, '\n' ) + info( err ) + + +class HostWithPrivateDirs( Host ): + "Host with private directories" + + mnRunDir = MNRUNDIR + + def __init__(self, name, *args, **kwargs ): + """privateDirs: list of private directories + remounts: dirs to remount + unmount: unmount dirs in cleanup? (True) + Note: if unmount is False, you must call unmountAll() + manually.""" + self.privateDirs = kwargs.pop( 'privateDirs', [] ) + self.remounts = kwargs.pop( 'remounts', [] ) + self.unmount = kwargs.pop( 'unmount', True ) + Host.__init__( self, name, *args, **kwargs ) + self.rundir = '%s/%s' % ( self.mnRunDir, name ) + self.root, self.private = None, None # set in createBindMounts + if self.privateDirs: + self.privateDirs = [ realpath( d ) for d in self.privateDirs ] + self.createBindMounts() + # These should run in the namespace before we chroot, + # in order to put the right entries in /etc/mtab + # Eventually this will allow a local pid space + # Now we chroot and cd to wherever we were before. + pwd = self.cmd( 'pwd' ).strip() + self.sendCmd( 'exec chroot', self.root, 'bash -ms mininet:' + + self.name ) + self.waiting = False + self.cmd( 'cd', pwd ) + # In order for many utilities to work, + # we need to remount /proc and /sys + self.cmd( 'mount /proc' ) + self.cmd( 'mount /sys' ) + + def mountPrivateDirs( self ): + "Create and bind mount private dirs" + for dir_ in self.privateDirs: + privateDir = self.private + dir_ + errFail( 'mkdir -p ' + privateDir ) + mountPoint = self.root + dir_ + errFail( 'mount -B %s %s' % + ( privateDir, mountPoint) ) + + def mountDirs( self, dirs ): + "Mount a list of directories" + for dir_ in dirs: + mountpoint = self.root + dir_ + errFail( 'mount -B %s %s' % + ( dir_, mountpoint ) ) + + @classmethod + def findRemounts( cls, fstypes=None ): + """Identify mount points in /proc/mounts to remount + fstypes: file system types to match""" + if fstypes is None: + fstypes = [ 'nfs' ] + dirs = quietRun( 'cat /proc/mounts' ).strip().split( '\n' ) + remounts = [] + for dir_ in dirs: + line = dir_.split() + mountpoint, fstype = line[ 1 ], line[ 2 ] + # Don't re-remount directories!!! + if mountpoint.find( cls.mnRunDir ) == 0: + continue + if fstype in fstypes: + remounts.append( mountpoint ) + return remounts + + def createBindMounts( self ): + """Create a chroot directory structure, + with self.privateDirs as private dirs""" + errFail( 'mkdir -p '+ self.rundir ) + unmountAll( self.rundir ) + # Create /root and /private directories + self.root = self.rundir + '/root' + self.private = self.rundir + '/private' + errFail( 'mkdir -p ' + self.root ) + errFail( 'mkdir -p ' + self.private ) + # Recursively mount / in private doort + # note we'll remount /sys and /proc later + errFail( 'mount -B / ' + self.root ) + self.mountDirs( self.remounts ) + self.mountPrivateDirs() + + def unmountBindMounts( self ): + "Unmount all of our bind mounts" + unmountAll( self.rundir ) + + def popen( self, *args, **kwargs ): + "Popen with chroot support" + chroot = kwargs.pop( 'chroot', True ) + mncmd = kwargs.get( 'mncmd', + [ 'mnexec', '-a', str( self.pid ) ] ) + if chroot: + mncmd = [ 'chroot', self.root ] + mncmd + kwargs[ 'mncmd' ] = mncmd + return Host.popen( self, *args, **kwargs ) + + def cleanup( self ): + """Clean up, then unmount bind mounts + unmount: actually unmount bind mounts?""" + # Wait for process to actually terminate + self.shell.wait() + Host.cleanup( self ) + if self.unmount: + self.unmountBindMounts() + errFail( 'rmdir ' + self.root ) + + +# Convenience aliases + +findRemounts = HostWithPrivateDirs.findRemounts + + +# Sample usage + +def testHostWithPrivateDirs(): + "Test bind mounts" + topo = SingleSwitchTopo( 10 ) + remounts = findRemounts( fstypes=[ 'nfs' ] ) + privateDirs = [ '/var/log', '/var/run' ] + host = partial( HostWithPrivateDirs, remounts=remounts, + privateDirs=privateDirs, unmount=False ) + net = Mininet( topo=topo, host=host ) + net.start() + info( 'Private Directories:', privateDirs, '\n' ) + CLI( net ) + net.stop() + # We do this all at once to save a bit of time + info( 'Unmounting host bind mounts...\n' ) + unmountAll() + + +if __name__ == '__main__': + unmountAll() + setLogLevel( 'info' ) + testHostWithPrivateDirs() + info( 'Done.\n') + + + + diff --git a/examples/consoles.py b/examples/consoles.py index 53fc400..3197ca1 100755 --- a/examples/consoles.py +++ b/examples/consoles.py @@ -418,7 +418,8 @@ class ConsoleApp( Frame ): count = len( consoles ) self.setOutputHook( self.updateGraph ) for console in consoles: - #sometimes iperf -sD doesn't return, so we run it in the background instead + # Sometimes iperf -sD doesn't return, + # so we run it in the background instead console.node.cmd( 'iperf -s &' ) i = 0 for console in consoles: diff --git a/examples/controllers.py b/examples/controllers.py index ac86429..d3694bc 100755 --- a/examples/controllers.py +++ b/examples/controllers.py @@ -15,8 +15,8 @@ setLogLevel( 'info' ) # Two local and one "external" controller (which is actually c0) # Ignore the warning message that the remote isn't (yet) running -c0 = Controller( 'c0' ) -c1 = Controller( 'c1' ) +c0 = Controller( 'c0', port=6633 ) +c1 = Controller( 'c1', port=6634 ) c2 = RemoteController( 'c2', ip='127.0.0.1' ) cmap = { 's1': c0, 's2': c1, 's3': c2 } @@ -28,7 +28,8 @@ class MultiSwitch( OVSSwitch ): topo = TreeTopo( depth=2, fanout=2 ) net = Mininet( topo=topo, switch=MultiSwitch, build=False ) -net.controllers = [ c0, c1 ] +for c in [ c0, c1 ]: + net.addController(c) net.build() net.start() CLI( net ) diff --git a/examples/controlnet.py b/examples/controlnet.py index 3d5dd9d..d5a8781 100755 --- a/examples/controlnet.py +++ b/examples/controlnet.py @@ -10,10 +10,12 @@ control network to control the data network. Since we're using UserSwitch on the data network, it should correctly fail over to a backup controller. -We also hack/subclass the CLI slightly so it can talk to -both the control and data networks. +We also use a Mininet Facade to talk to both the +control and data networks from a single CLI. """ +from functools import partial + from mininet.net import Mininet from mininet.node import Controller, UserSwitch from mininet.cli import CLI @@ -30,31 +32,62 @@ class DataController( Controller ): "Ignore spurious error" pass - -class CLI2( CLI ): - "CLI that can talk to two networks" +class MininetFacade( object ): + """Mininet object facade that allows a single CLI to + talk to one or more networks""" - def __init__( self, *args, **kwargs ): - "cnet: second network" - self.cnet = kwargs.pop( 'cnet' ) - CLI.__init__( self, *args, **kwargs ) + def __init__( self, net, *args, **kwargs ): + """Create MininetFacade object. + net: Primary Mininet object + args: unnamed networks passed as arguments + kwargs: named networks passed as arguments""" + self.net = net + self.nets = [ net ] + list( args ) + kwargs.values() + self.nameToNet = kwargs + self.nameToNet['net'] = net - def updateVars( self ): - "Update variables to include cnet" - cnet = self.cnet - nodes2 = cnet.controllers + cnet.switches + cnet.hosts - self.nodelist += nodes2 - for node in nodes2: - self.nodemap[ node.name ] = node - self.locals[ 'cnet' ] = cnet - self.locals.update( self.nodemap ) - - def cmdloop( self, *args, **kwargs ): - "Patch to add cnet if needed" - if 'cnet' not in self.locals: - self.updateVars() - CLI.cmdloop( self, *args, **kwargs ) + def __getattr__( self, name ): + "returns attribute from Primary Mininet object" + return getattr( self.net, name ) + def __getitem__( self, key ): + "returns primary/named networks or node from any net" + #search kwargs for net named key + if key in self.nameToNet: + return self.nameToNet[ key ] + #search each net for node named key + for net in self.nets: + if key in net: + return net[ key ] + + def __iter__( self ): + "Iterate through all nodes in all Mininet objects" + for net in self.nets: + for node in net: + yield node + + def __len__( self ): + "returns aggregate number of nodes in all nets" + count = 0 + for net in self.nets: + count += len(net) + return count + + def __contains__( self, key ): + "returns True if node is a member of any net" + return key in self.keys() + + def keys( self ): + "returns a list of all node names in all networks" + return list( self ) + + def values( self ): + "returns a list of all nodes in all networks" + return [ self[ key ] for key in self ] + + def items( self ): + "returns (key,value) tuple list for every node in all networks" + return zip( self.keys(), self.values() ) # A real control network! @@ -79,40 +112,40 @@ class ControlNetwork( Topo ): # Make it Happen!! -setLogLevel( 'info' ) +def run(): + "Create control and data networks, and invoke the CLI" + + info( '* Creating Control Network\n' ) + ctopo = ControlNetwork( n=4, dataController=DataController ) + cnet = Mininet( topo=ctopo, ipBase='192.168.123.0/24', controller=None ) + info( '* Adding Control Network Controller\n') + cnet.addController( 'cc0', controller=Controller ) + info( '* Starting Control Network\n') + cnet.start() -info( '* Creating Control Network\n' ) -ctopo = ControlNetwork( n=4, dataController=DataController ) -cnet = Mininet( topo=ctopo, ipBase='192.168.123.0/24', build=False ) -info( '* Adding Control Network Controller\n') -cnet.addController( 'cc0' ) -info( '* Starting Control Network\n') -cnet.build() -cnet.start() -dataControllers = cnet.hosts[ : -1 ] # ignore 'root' node - -info( '* Creating Data Network\n' ) -topo = TreeTopo( depth=2, fanout=2 ) -# UserSwitch so we can easily test failover -net = Mininet( topo=topo, switch=UserSwitch, build=False ) -info( '* Adding Controllers to Data Network\n' ) -net.controllers = dataControllers -net.build() -info( '* Starting Data Network\n') -net.start() - -CLI2( net, cnet=cnet ) - -info( '* Stopping Data Network\n' ) -net.stop() - -info( '* Stopping Control Network\n' ) -# dataControllers have already been stopped -cnet.hosts = list( set( cnet.hosts ) - set( dataControllers ) ) -cnet.stop() + info( '* Creating Data Network\n' ) + topo = TreeTopo( depth=2, fanout=2 ) + # UserSwitch so we can easily test failover + sw = partial( UserSwitch, opts='--inactivity-probe=1 --max-backoff=1' ) + net = Mininet( topo=topo, switch=sw, controller=None ) + info( '* Adding Controllers to Data Network\n' ) + for host in cnet.hosts: + if isinstance(host, Controller): + net.addController( host ) + info( '* Starting Data Network\n') + net.start() + mn = MininetFacade( net, cnet=cnet ) + CLI( mn ) + info( '* Stopping Data Network\n' ) + net.stop() + info( '* Stopping Control Network\n' ) + cnet.stop() +if __name__ == '__main__': + setLogLevel( 'info' ) + run() diff --git a/examples/emptynet.py b/examples/emptynet.py index 9f57855..afcf6a8 100755 --- a/examples/emptynet.py +++ b/examples/emptynet.py @@ -27,8 +27,8 @@ def emptyNet(): s3 = net.addSwitch( 's3' ) info( '*** Creating links\n' ) - h1.linkTo( s3 ) - h2.linkTo( s3 ) + net.addLink( h1, s3 ) + net.addLink( h2, s3 ) info( '*** Starting network\n') net.start() diff --git a/examples/hwintf.py b/examples/hwintf.py index e5960d9..1e010fd 100755 --- a/examples/hwintf.py +++ b/examples/hwintf.py @@ -5,7 +5,7 @@ This example shows how to add an interface (for example a real hardware interface) to a network after the network is created. """ -import re +import re, sys from mininet.cli import CLI from mininet.log import setLogLevel, info, error @@ -28,7 +28,10 @@ def checkIntf( intf ): if __name__ == '__main__': setLogLevel( 'info' ) - intfName = 'eth1' + # try to get hw intf from the command line; by default, use eth1 + intfName = sys.argv[ 1 ] if len( sys.argv ) > 1 else 'eth1' + info( '*** Connecting to hw intf: %s' % intfName ) + info( '*** Checking', intfName, '\n' ) checkIntf( intfName ) diff --git a/examples/multiping.py b/examples/multiping.py index 3bd231c..88667d4 100755 --- a/examples/multiping.py +++ b/examples/multiping.py @@ -23,13 +23,8 @@ def chunks( l, n ): def startpings( host, targetips ): "Tell host to repeatedly ping targets" - targetips.append( '10.0.0.200' ) - targetips = ' '.join( targetips ) - # BL: Not sure why loopback intf isn't up! - host.cmd( 'ifconfig lo up' ) - # Simple ping loop cmd = ( 'while true; do ' ' for ip in %s; do ' % targetips + @@ -63,6 +58,8 @@ def multiping( netsize, chunksize, seconds): # Start pings for subnet in subnets: ips = [ host.IP() for host in subnet ] + #adding bogus to generate packet loss + ips.append( '10.0.0.200' ) for host in subnet: startpings( host, ips ) diff --git a/examples/multitest.py b/examples/multitest.py index bcb40f7..b50acb2 100755 --- a/examples/multitest.py +++ b/examples/multitest.py @@ -22,7 +22,7 @@ if __name__ == '__main__': info( "*** Initializing Mininet and kernel modules\n" ) OVSKernelSwitch.setup() info( "*** Creating network\n" ) - network = Mininet( TreeTopo( depth=2, fanout=2 ), switch=OVSKernelSwitch) + network = Mininet( TreeTopo( depth=2, fanout=2 ), switch=OVSKernelSwitch ) info( "*** Starting network\n" ) network.start() info( "*** Running ping test\n" ) diff --git a/examples/popenpoll.py b/examples/popenpoll.py index c581c27..33478ae 100755 --- a/examples/popenpoll.py +++ b/examples/popenpoll.py @@ -23,7 +23,7 @@ def pmonitorTest( N=3, seconds=10 ): endTime = time() + seconds for h, line in pmonitor( popens, timeoutms=500 ): if h: - print '%s: %s' % ( h.name, line ), + print '<%s>: %s' % ( h.name, line ), if time() >= endTime: for p in popens.values(): p.send_signal( SIGINT ) diff --git a/examples/sshd.py b/examples/sshd.py index 2bedb9c..3c94359 100755 --- a/examples/sshd.py +++ b/examples/sshd.py @@ -16,6 +16,8 @@ demonstrates: - running server processes (sshd in this case) on hosts """ +import sys + from mininet.net import Mininet from mininet.cli import CLI from mininet.log import lg @@ -68,4 +70,6 @@ def sshd( network, cmd='/usr/sbin/sshd', opts='-D' ): if __name__ == '__main__': lg.setLogLevel( 'info') net = TreeNet( depth=1, fanout=4, switch=OVSKernelSwitch ) - sshd( net ) + # get sshd args from the command line; default: -D + opts = ' '.join( sys.argv[ 1: ] ) if len( sys.argv ) > 1 else '-D' + sshd( net, opts=opts ) diff --git a/examples/test/runner.py b/examples/test/runner.py new file mode 100755 index 0000000..4939e08 --- /dev/null +++ b/examples/test/runner.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python + +""" +Run all mininet.examples tests + -v : verbose output + -quick : skip tests that take more than ~30 seconds +""" + +import unittest +import os +import sys +from mininet.util import ensureRoot +from mininet.clean import cleanup + +def runTests( testDir, verbosity=1 ): + "discover and run all tests in testDir" + # ensure root and cleanup before starting tests + ensureRoot() + cleanup() + # discover all tests in testDir + testSuite = unittest.defaultTestLoader.discover( testDir ) + # run tests + unittest.TextTestRunner( verbosity=verbosity ).run( testSuite ) + +if __name__ == '__main__': + # get the directory containing example tests + testDir = os.path.dirname( os.path.realpath( __file__ ) ) + verbosity = 2 if '-v' in sys.argv else 1 + runTests( testDir, verbosity ) diff --git a/examples/test/test_baresshd.py b/examples/test/test_baresshd.py new file mode 100755 index 0000000..d708761 --- /dev/null +++ b/examples/test/test_baresshd.py @@ -0,0 +1,62 @@ +#!/usr/bin/env python + +""" +Tests for baresshd.py +""" + +import unittest +import pexpect +from time import sleep +from mininet.clean import cleanup, sh + +class testBareSSHD( unittest.TestCase ): + + opts = [ '\(yes/no\)\?', 'Welcome to h1', 'refused', pexpect.EOF, pexpect.TIMEOUT ] + + def connected( self ): + "Log into ssh server, check banner, then exit" + p = pexpect.spawn( 'ssh 10.0.0.1 -i /tmp/ssh/test_rsa exit' ) + while True: + index = p.expect( self.opts ) + if index == 0: + p.sendline( 'yes' ) + elif index == 1: + return True + else: + return False + + def setUp( self ): + # verify that sshd is not running + self.assertFalse( self.connected() ) + # create public key pair for testing + sh( 'rm -rf /tmp/ssh' ) + sh( 'mkdir /tmp/ssh' ) + sh( "ssh-keygen -t rsa -P '' -f /tmp/ssh/test_rsa" ) + sh( 'cat /tmp/ssh/test_rsa.pub >> /tmp/ssh/authorized_keys' ) + # run example with custom sshd args + cmd = ( 'python -m mininet.examples.baresshd ' + '-o AuthorizedKeysFile=/tmp/ssh/authorized_keys ' + '-o StrictModes=no' ) + sh( cmd ) + + def testSSH( self ): + "Simple test to verify that we can ssh into h1" + result = False + # try to connect up to 3 times; sshd can take a while to start + for _ in range( 3 ): + result = self.connected() + if result: + break + else: + sleep( 1 ) + self.assertTrue( result ) + + def tearDown( self ): + # kill the ssh process + sh( "ps aux | grep 'ssh.*Banner' | awk '{ print $2 }' | xargs kill" ) + cleanup() + # remove public key pair + sh( 'rm -rf /tmp/ssh' ) + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_bind.py b/examples/test/test_bind.py new file mode 100755 index 0000000..dcc9fbf --- /dev/null +++ b/examples/test/test_bind.py @@ -0,0 +1,66 @@ +#!/usr/bin/env python + +""" +Tests for bind.py +""" + +import unittest +import pexpect + +class testBind( unittest.TestCase ): + + prompt = 'mininet>' + + def setUp( self ): + self.net = pexpect.spawn( 'python -m mininet.examples.bind' ) + self.net.expect( "Private Directories: \[([\w\s,'/]+)\]" ) + self.directories = [] + # parse directories from mn output + for d in self.net.match.group(1).split(', '): + self.directories.append( d.strip("'") ) + self.net.expect( self.prompt ) + self.assertTrue( len( self.directories ) > 0 ) + + def testCreateFile( self ): + "Create a file, a.txt, in the first private directory and verify" + fileName = 'a.txt' + directory = self.directories[ 0 ] + path = directory + '/' + fileName + self.net.sendline( 'h1 touch %s; ls %s' % ( path, directory ) ) + index = self.net.expect( [ fileName, self.prompt ] ) + self.assertTrue( index == 0 ) + self.net.expect( self.prompt ) + self.net.sendline( 'h1 rm %s' % path ) + self.net.expect( self.prompt ) + + def testIsolation( self ): + "Create a file in two hosts and verify that contents are different" + fileName = 'b.txt' + directory = self.directories[ 0 ] + path = directory + '/' + fileName + contents = { 'h1' : '1', 'h2' : '2' } + # Verify file doesn't exist, then write private copy of file + for host in contents: + value = contents[ host ] + self.net.sendline( '%s cat %s' % ( host, path ) ) + self.net.expect( 'No such file' ) + self.net.expect( self.prompt ) + self.net.sendline( '%s echo %s > %s' % ( host, value, path ) ) + self.net.expect( self.prompt ) + # Verify file contents + for host in contents: + value = contents[ host ] + self.net.sendline( '%s cat %s' % ( host, path ) ) + self.net.expect( value ) + self.net.expect( self.prompt ) + self.net.sendline( '%s rm %s' % ( host, path ) ) + self.net.expect( self.prompt ) + + # TODO: need more tests + + def tearDown( self ): + self.net.sendline( 'exit' ) + self.net.wait() + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_controllers.py b/examples/test/test_controllers.py new file mode 100755 index 0000000..b9b93d9 --- /dev/null +++ b/examples/test/test_controllers.py @@ -0,0 +1,48 @@ +#!/usr/bin/env python + +""" +Tests for controllers.py and controllers2.py +""" + +import unittest +import pexpect + +class testControllers( unittest.TestCase ): + + prompt = 'mininet>' + + def connectedTest( self, name, cmap ): + "Verify that switches are connected to the controller specified by cmap" + p = pexpect.spawn( 'python -m %s' % name ) + p.expect( self.prompt ) + # but first a simple ping test + p.sendline( 'pingall' ) + p.expect ( '(\d+)% dropped' ) + percent = int( p.match.group( 1 ) ) if p.match else -1 + self.assertEqual( percent, 0 ) + p.expect( self.prompt ) + # verify connected controller + for switch in cmap: + p.sendline( 'sh ovs-vsctl get-controller %s' % switch ) + p.expect( 'tcp:([\d.:]+)') + actual = p.match.group(1) + expected = cmap[ switch ] + self.assertEqual( actual, expected ) + p.expect( self.prompt ) + p.sendline( 'exit' ) + p.wait() + + def testControllers( self ): + c0 = '127.0.0.1:6633' + c1 = '127.0.0.1:6634' + cmap = { 's1': c0, 's2': c1, 's3': c0 } + self.connectedTest( 'mininet.examples.controllers', cmap ) + + def testControllers2( self ): + c0 = '127.0.0.1:6633' + c1 = '127.0.0.1:6634' + cmap = { 's1': c0, 's2': c1 } + self.connectedTest( 'mininet.examples.controllers2', cmap ) + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_controlnet.py b/examples/test/test_controlnet.py new file mode 100755 index 0000000..91631f2 --- /dev/null +++ b/examples/test/test_controlnet.py @@ -0,0 +1,47 @@ +#!/usr/bin/env python + +""" +Test for controlnet.py +""" + +import unittest +import pexpect + +class testControlNet( unittest.TestCase ): + + prompt = 'mininet>' + + def testPingall( self ): + "Simple pingall test that verifies 0% packet drop in data network" + p = pexpect.spawn( 'python -m mininet.examples.controlnet' ) + p.expect( self.prompt ) + p.sendline( 'pingall' ) + p.expect ( '(\d+)% dropped' ) + percent = int( p.match.group( 1 ) ) if p.match else -1 + self.assertEqual( percent, 0 ) + p.expect( self.prompt ) + p.sendline( 'exit' ) + p.wait() + + def testFailover( self ): + "Kill controllers and verity that switch, s1, fails over properly" + count = 1 + p = pexpect.spawn( 'python -m mininet.examples.controlnet' ) + p.expect( self.prompt ) + lp = pexpect.spawn( 'tail -f /tmp/s1-ofp.log' ) + lp.expect( 'tcp:\d+\.\d+\.\d+\.(\d+):\d+: connected' ) + ip = int( lp.match.group( 1 ) ) + self.assertEqual( count, ip ) + count += 1 + for c in [ 'c0', 'c1' ]: + p.sendline( '%s ifconfig %s-eth0 down' % ( c, c) ) + p.expect( self.prompt ) + lp.expect( 'tcp:\d+\.\d+\.\d+\.(\d+):\d+: connected' ) + ip = int( lp.match.group( 1 ) ) + self.assertEqual( count, ip ) + count += 1 + p.sendline( 'exit' ) + p.wait() + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_cpu.py b/examples/test/test_cpu.py new file mode 100755 index 0000000..2547784 --- /dev/null +++ b/examples/test/test_cpu.py @@ -0,0 +1,38 @@ +#!/usr/bin/env python + +""" +Test for cpu.py +""" + +import unittest +import pexpect +import sys + +class testCPU( unittest.TestCase ): + + prompt = 'mininet>' + + @unittest.skipIf( '-quick' in sys.argv, 'long test' ) + def testCPU( self ): + "Verify that CPU utilization is monotonically decreasing for each scheduler" + p = pexpect.spawn( 'python -m mininet.examples.cpu' ) + opts = [ '([a-z]+)\t([\d\.]+)%\t([\d\.]+)', pexpect.EOF ] + scheds = [] + while True: + index = p.expect( opts, timeout=600 ) + if index == 0: + sched = p.match.group( 1 ) + cpu = float( p.match.group( 2 ) ) + bw = float( p.match.group( 3 ) ) + if sched not in scheds: + scheds.append( sched ) + previous_bw = 10 ** 4 # 10 GB/s + self.assertTrue( bw < previous_bw ) + previous_bw = bw + else: + break + + self.assertTrue( len( scheds ) > 0 ) + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_emptynet.py b/examples/test/test_emptynet.py new file mode 100755 index 0000000..0d4d01d --- /dev/null +++ b/examples/test/test_emptynet.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python + +""" +Test for emptynet.py +""" + +import unittest +import pexpect + +class testEmptyNet( unittest.TestCase ): + + prompt = 'mininet>' + + def testEmptyNet( self ): + "Run simple CLI tests: pingall (verify 0% drop) and iperf (sanity)" + p = pexpect.spawn( 'python -m mininet.examples.emptynet' ) + p.expect( self.prompt ) + # pingall test + p.sendline( 'pingall' ) + p.expect ( '(\d+)% dropped' ) + percent = int( p.match.group( 1 ) ) if p.match else -1 + self.assertEqual( percent, 0 ) + p.expect( self.prompt ) + # iperf test + p.sendline( 'iperf' ) + p.expect( "Results: \['[\d.]+ .bits/sec', '[\d.]+ .bits/sec'\]" ) + p.expect( self.prompt ) + p.sendline( 'exit' ) + p.wait() + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_hwintf.py b/examples/test/test_hwintf.py new file mode 100755 index 0000000..20d08d3 --- /dev/null +++ b/examples/test/test_hwintf.py @@ -0,0 +1,63 @@ +#!/usr/bin/env python + +""" +Test for hwintf.py +""" + +import unittest +import pexpect +import re +from mininet.log import setLogLevel +from mininet.net import Mininet +from mininet.node import Node +from mininet.link import Link, Intf + +class testHwintf( unittest.TestCase ): + + prompt = 'mininet>' + + def setUp( self ): + self.h3 = Node( 't0', ip='10.0.0.3/8' ) + self.n0 = Node( 't1', inNamespace=False ) + Link( self.h3, self.n0 ) + self.h3.configDefault() + + def testLocalPing( self ): + "Verify connectivity between virtual hosts using pingall" + p = pexpect.spawn( 'python -m mininet.examples.hwintf %s' % self.n0.intf() ) + p.expect( self.prompt ) + p.sendline( 'pingall' ) + p.expect ( '(\d+)% dropped' ) + percent = int( p.match.group( 1 ) ) if p.match else -1 + self.assertEqual( percent, 0 ) + p.expect( self.prompt ) + p.sendline( 'exit' ) + p.wait() + + def testExternalPing( self ): + "Verify connnectivity between virtual host and virtual-physical 'external' host " + p = pexpect.spawn( 'python -m mininet.examples.hwintf %s' % self.n0.intf() ) + p.expect( self.prompt ) + # test ping external to internal + expectStr = '(\d+) packets transmitted, (\d+) received' + m = re.search( expectStr, self.h3.cmd( 'ping -v -c 1 10.0.0.1' ) ) + tx = m.group( 1 ) + rx = m.group( 2 ) + self.assertEqual( tx, rx ) + # test ping internal to external + p.sendline( 'h1 ping -c 1 10.0.0.3') + p.expect( expectStr ) + tx = p.match.group( 1 ) + rx = p.match.group( 2 ) + self.assertEqual( tx, rx ) + p.expect( self.prompt ) + p.sendline( 'exit' ) + p.wait() + + def tearDown( self ): + self.h3.terminate() + self.n0.terminate() + +if __name__ == '__main__': + setLogLevel( 'warning' ) + unittest.main() diff --git a/examples/test/test_limit.py b/examples/test/test_limit.py new file mode 100755 index 0000000..db9eb33 --- /dev/null +++ b/examples/test/test_limit.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python + +""" +Test for limit.py +""" + +import unittest +import pexpect +import sys + +class testLimit( unittest.TestCase ): + + @unittest.skipIf( '-quick' in sys.argv, 'long test' ) + def testLimit( self ): + "Verify that CPU limits are within a 2% tolerance of limit for each scheduler" + p = pexpect.spawn( 'python -m mininet.examples.limit' ) + opts = [ '\*\*\* Testing network ([\d\.]+) Mbps', + '\*\*\* Results: \[([\d\., ]+)\]', + pexpect.EOF ] + count = 0 + bw = 0 + tolerance = 2 + while True: + index = p.expect( opts ) + if index == 0: + bw = float( p.match.group( 1 ) ) + count += 1 + elif index == 1: + results = p.match.group( 1 ) + for x in results.split( ',' ): + result = float( x ) + self.assertTrue( result < bw + tolerance ) + self.assertTrue( result > bw - tolerance ) + else: + break + + self.assertTrue( count > 0 ) + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_linearbandwidth.py b/examples/test/test_linearbandwidth.py new file mode 100755 index 0000000..d3c1144 --- /dev/null +++ b/examples/test/test_linearbandwidth.py @@ -0,0 +1,44 @@ +#!/usr/bin/env python + +""" +Test for linearbandwidth.py +""" + +import unittest +import pexpect +import sys + +class testLinearBandwidth( unittest.TestCase ): + + @unittest.skipIf( '-quick' in sys.argv, 'long test' ) + def testLinearBandwidth( self ): + "Verify that bandwidth is monotonically decreasing as # of hops increases" + p = pexpect.spawn( 'python -m mininet.examples.linearbandwidth' ) + count = 0 + opts = [ '\*\*\* Linear network results', + '(\d+)\s+([\d\.]+) (.bits)', + pexpect.EOF ] + while True: + index = p.expect( opts, timeout=600 ) + if index == 0: + previous_bw = 10 ** 10 # 10 Gbits + count += 1 + elif index == 1: + n = int( p.match.group( 1 ) ) + bw = float( p.match.group( 2 ) ) + unit = p.match.group( 3 ) + if unit[ 0 ] == 'K': + bw *= 10 ** 3 + elif unit[ 0 ] == 'M': + bw *= 10 ** 6 + elif unit[ 0 ] == 'G': + bw *= 10 ** 9 + self.assertTrue( bw < previous_bw ) + previous_bw = bw + else: + break + + self.assertTrue( count > 0 ) + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_multiping.py b/examples/test/test_multiping.py new file mode 100755 index 0000000..ff0571c --- /dev/null +++ b/examples/test/test_multiping.py @@ -0,0 +1,48 @@ +#!/usr/bin/env python + +""" +Test for multiping.py +""" + +import unittest +import pexpect +from collections import defaultdict + +class testMultiPing( unittest.TestCase ): + + def testMultiPing( self ): + """Verify that each target is pinged at least once, and + that pings to 'real' targets are successful and unknown targets fail""" + p = pexpect.spawn( 'python -m mininet.examples.multiping' ) + opts = [ "Host (h\d+) \(([\d.]+)\) will be pinging ips: ([\d\. ]+)", + "(h\d+): ([\d.]+) -> ([\d.]+) \d packets transmitted, (\d) received", + pexpect.EOF ] + pings = defaultdict( list ) + while True: + index = p.expect( opts ) + if index == 0: + name = p.match.group(1) + ip = p.match.group(2) + targets = p.match.group(3).split() + pings[ name ] += targets + elif index == 1: + name = p.match.group(1) + ip = p.match.group(2) + target = p.match.group(3) + received = int( p.match.group(4) ) + if target == '10.0.0.200': + self.assertEqual( received, 0 ) + else: + self.assertEqual( received, 1 ) + try: + pings[ name ].remove( target ) + except: + pass + else: + break + self.assertTrue( len( pings ) > 0 ) + for t in pings.values(): + self.assertEqual( len( t ), 0 ) + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_multipoll.py b/examples/test/test_multipoll.py new file mode 100755 index 0000000..12321ea --- /dev/null +++ b/examples/test/test_multipoll.py @@ -0,0 +1,38 @@ +#!/usr/bin/env python + +""" +Test for multipoll.py +""" + +import unittest +import pexpect + +class testMultiPoll( unittest.TestCase ): + + def testMultiPoll( self ): + "Verify that we receive one ping per second per host" + p = pexpect.spawn( 'python -m mininet.examples.multipoll' ) + opts = [ "\*\*\* (h\d) :" , + "(h\d+): \d+ bytes from", + "Monitoring output for (\d+) seconds", + pexpect.EOF ] + pings = {} + while True: + index = p.expect( opts ) + if index == 0: + name = p.match.group( 1 ) + pings[ name ] = 0 + elif index == 1: + name = p.match.group( 1 ) + pings[ name ] += 1 + elif index == 2: + seconds = int( p.match.group( 1 ) ) + else: + break + self.assertTrue( len( pings ) > 0 ) + # make sure we have received at least one ping per second + for count in pings.values(): + self.assertTrue( count >= seconds ) + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_multitest.py b/examples/test/test_multitest.py new file mode 100755 index 0000000..09172c5 --- /dev/null +++ b/examples/test/test_multitest.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python + +""" +Test for multitest.py +""" + +import unittest +import pexpect + +class testMultiTest( unittest.TestCase ): + + prompt = 'mininet>' + + def testMultiTest( self ): + "Verify pingall (0% dropped) and hX-eth0 interface for each host (ifconfig)" + p = pexpect.spawn( 'python -m mininet.examples.multitest' ) + p.expect( '(\d+)% dropped' ) + dropped = int( p.match.group( 1 ) ) + self.assertEqual( dropped, 0 ) + ifCount = 0 + while True: + index = p.expect( [ 'h\d-eth0', self.prompt ] ) + if index == 0: + ifCount += 1 + elif index == 1: + p.sendline( 'exit' ) + break + p.wait() + self.assertEqual( ifCount, 4 ) + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_nat.py b/examples/test/test_nat.py new file mode 100755 index 0000000..8e49ed6 --- /dev/null +++ b/examples/test/test_nat.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python + +""" +Test for nat.py +""" + +import unittest +import pexpect +from mininet.util import quietRun + +destIP = '8.8.8.8' # Google DNS + +class testNAT( unittest.TestCase ): + + prompt = 'mininet>' + + @unittest.skipIf( '0 received' in quietRun( 'ping -c 1 %s' % destIP ), + 'Destination IP is not reachable' ) + def testNAT( self ): + "Attempt to ping an IP on the Internet and verify 0% packet loss" + p = pexpect.spawn( 'python -m mininet.examples.nat' ) + p.expect( self.prompt ) + p.sendline( 'h1 ping -c 1 %s' % destIP ) + p.expect ( '(\d+)% packet loss' ) + percent = int( p.match.group( 1 ) ) if p.match else -1 + p.expect( self.prompt ) + p.sendline( 'exit' ) + p.wait() + self.assertEqual( percent, 0 ) + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_popen.py b/examples/test/test_popen.py new file mode 100755 index 0000000..c7f83f0 --- /dev/null +++ b/examples/test/test_popen.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python + +""" +Test for popen.py and popenpoll.py +""" + +import unittest +import pexpect + +class testPopen( unittest.TestCase ): + + def pingTest( self, name ): + "Verify that there are no dropped packets for each host" + p = pexpect.spawn( 'python -m %s' % name ) + opts = [ "<(h\d+)>: PING ", + "<(h\d+)>: (\d+) packets transmitted, (\d+) received", + pexpect.EOF ] + pings = {} + while True: + index = p.expect( opts ) + if index == 0: + name = p.match.group(1) + pings[ name ] = 0 + elif index == 1: + name = p.match.group(1) + transmitted = p.match.group(2) + received = p.match.group(3) + # verify no dropped packets + self.assertEqual( received, transmitted ) + pings[ name ] += 1 + else: + break + self.assertTrue( len(pings) > 0 ) + # verify that each host has gotten results + for count in pings.values(): + self.assertEqual( count, 1 ) + + def testPopen( self ): + self.pingTest( 'mininet.examples.popen' ) + + def testPopenPoll( self ): + self.pingTest( 'mininet.examples.popenpoll' ) + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_scratchnet.py b/examples/test/test_scratchnet.py new file mode 100755 index 0000000..31739b8 --- /dev/null +++ b/examples/test/test_scratchnet.py @@ -0,0 +1,27 @@ +#!/usr/bin/env python + +""" +Test for scratchnet.py +""" + +import unittest +import pexpect + +class testScratchNet( unittest.TestCase ): + + opts = [ "1 packets transmitted, 1 received, 0% packet loss", pexpect.EOF ] + + def pingTest( self, name ): + "Verify that no ping packets were dropped" + p = pexpect.spawn( 'python -m %s' % name ) + index = p.expect( self.opts ) + self.assertEqual( index, 0 ) + + def testPingKernel( self ): + self.pingTest( 'mininet.examples.scratchnet' ) + + def testPingUser( self ): + self.pingTest( 'mininet.examples.scratchnetuser' ) + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_simpleperf.py b/examples/test/test_simpleperf.py new file mode 100755 index 0000000..7d44c9c --- /dev/null +++ b/examples/test/test_simpleperf.py @@ -0,0 +1,53 @@ +#!/usr/bin/env python + +""" +Test for simpleperf.py +""" + +import unittest +import pexpect +import re +import sys +from mininet.log import setLogLevel +from mininet.net import Mininet +from mininet.node import CPULimitedHost +from mininet.link import TCLink + +from mininet.examples.simpleperf import SingleSwitchTopo + +class testSimplePerf( unittest.TestCase ): + + @unittest.skipIf( '-quick' in sys.argv, 'long test' ) + def testE2E( self ): + "Run the example and verify ping and iperf results" + p = pexpect.spawn( 'python -m mininet.examples.simpleperf' ) + # check ping results + p.expect( "Results: (\d+)% dropped", timeout=120 ) + loss = int( p.match.group( 1 ) ) + self.assertTrue( loss > 0 and loss < 100 ) + # check iperf results + p.expect( "Results: \['([\d\.]+) .bits/sec", timeout=480 ) + bw = float( p.match.group( 1 ) ) + self.assertTrue( bw > 0 ) + p.wait() + + def testTopo( self ): + """Import SingleSwitchTopo from example and test connectivity between two hosts + Note: this test may fail very rarely because it is non-deterministic + i.e. links are configured with 10% packet loss, but if we get unlucky and + none or all of the packets are dropped, the test will fail""" + topo = SingleSwitchTopo( n=4 ) + net = Mininet( topo=topo, host=CPULimitedHost, link=TCLink ) + net.start() + h1, h4 = net.get( 'h1', 'h4' ) + # have h1 ping h4 ten times + expectStr = '(\d+) packets transmitted, (\d+) received, (\d+)% packet loss' + output = h1.cmd( 'ping -c 10 %s' % h4.IP() ) + m = re.search( expectStr, output ) + loss = int( m.group( 3 ) ) + net.stop() + self.assertTrue( loss > 0 and loss < 100 ) + +if __name__ == '__main__': + setLogLevel( 'warning' ) + unittest.main() diff --git a/examples/test/test_sshd.py b/examples/test/test_sshd.py new file mode 100755 index 0000000..04bd184 --- /dev/null +++ b/examples/test/test_sshd.py @@ -0,0 +1,61 @@ +#!/usr/bin/env python + +""" +Test for sshd.py +""" + +import unittest +import pexpect +from time import sleep +from mininet.clean import sh + +class testSSHD( unittest.TestCase ): + + opts = [ '\(yes/no\)\?', 'refused', 'Welcome|\$|#', pexpect.EOF, pexpect.TIMEOUT ] + + def connected( self, ip ): + "Log into ssh server, check banner, then exit" + # Note: this test will fail if "Welcome" is not in the sshd banner + # and '#'' or '$'' are not in the prompt + p = pexpect.spawn( 'ssh -i /tmp/ssh/test_rsa %s' % ip, timeout=10 ) + while True: + index = p.expect( self.opts ) + if index == 0: + print p.match.group(0) + p.sendline( 'yes' ) + elif index == 1: + return False + elif index == 2: + p.sendline( 'exit' ) + p.wait() + return True + else: + return False + + def setUp( self ): + # create public key pair for testing + sh( 'rm -rf /tmp/ssh' ) + sh( 'mkdir /tmp/ssh' ) + sh( "ssh-keygen -t rsa -P '' -f /tmp/ssh/test_rsa" ) + sh( 'cat /tmp/ssh/test_rsa.pub >> /tmp/ssh/authorized_keys' ) + cmd = ( 'python -m mininet.examples.sshd -D ' + '-o AuthorizedKeysFile=/tmp/ssh/authorized_keys ' + '-o StrictModes=no' ) + # run example with custom sshd args + self.net = pexpect.spawn( cmd ) + self.net.expect( 'mininet>' ) + + def testSSH( self ): + "Verify that we can ssh into all hosts (h1 to h4)" + for h in range( 1, 5 ): + self.assertTrue( self.connected( '10.0.0.%d' % h ) ) + + def tearDown( self ): + self.net.sendline( 'exit' ) + self.net.wait() + # remove public key pair + sh( 'rm -rf /tmp/ssh' ) + +if __name__ == '__main__': + unittest.main() + diff --git a/examples/test/test_tree1024.py b/examples/test/test_tree1024.py new file mode 100755 index 0000000..fbc82c5 --- /dev/null +++ b/examples/test/test_tree1024.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python + +""" +Test for tree1024.py +""" + +import unittest +import pexpect +import sys + +class testTree1024( unittest.TestCase ): + + prompt = 'mininet>' + + @unittest.skipIf( '-quick' in sys.argv, 'long test' ) + def testTree1024( self ): + "Run the example and do a simple ping test from h1 to h1024" + p = pexpect.spawn( 'python -m mininet.examples.tree1024' ) + p.expect( self.prompt, timeout=6000 ) # it takes awhile to set up + p.sendline( 'h1 ping -c 1 h1024' ) + p.expect ( '(\d+)% packet loss' ) + percent = int( p.match.group( 1 ) ) if p.match else -1 + p.expect( self.prompt ) + p.sendline( 'exit' ) + p.wait() + self.assertEqual( percent, 0 ) + +if __name__ == '__main__': + unittest.main() diff --git a/examples/test/test_treeping64.py b/examples/test/test_treeping64.py new file mode 100755 index 0000000..ae02afc --- /dev/null +++ b/examples/test/test_treeping64.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python + +""" +Test for treeping64.py +""" + +import unittest +import pexpect +import sys + +class testTreePing64( unittest.TestCase ): + + prompt = 'mininet>' + + @unittest.skipIf( '-quick' in sys.argv, 'long test' ) + def testTreePing64( self ): + "Run the example and verify ping results" + p = pexpect.spawn( 'python -m mininet.examples.treeping64' ) + p.expect( 'Tree network ping results:', timeout=6000 ) + count = 0 + while True: + index = p.expect( [ '(\d+)% packet loss', pexpect.EOF ] ) + if index == 0: + percent = int( p.match.group( 1 ) ) if p.match else -1 + self.assertEqual( percent, 0 ) + count += 1 + else: + break + self.assertTrue( count > 0 ) + +if __name__ == '__main__': + unittest.main() diff --git a/mininet/cli.py b/mininet/cli.py index 5de9beb..3ca190f 100644 --- a/mininet/cli.py +++ b/mininet/cli.py @@ -323,14 +323,13 @@ class CLI( Cmd ): corresponding IP addrs.""" first, args, line = self.parseline( line ) - if not args: - return - if args and len(args) > 0 and args[ -1 ] == '\n': - args = args[ :-1 ] - rest = args.split( ' ' ) if first in self.mn: + if not args: + print "*** Enter a command for node: %s " % first + return node = self.mn[ first ] + rest = args.split( ' ' ) # Substitute IP addresses for node names in command rest = [ self.mn[ arg ].defaultIntf().updateIP() if arg in self.mn else arg @@ -341,7 +340,7 @@ class CLI( Cmd ): node.sendCmd( rest, printPid=( not builtin ) ) self.waitForNode( node ) else: - error( '*** Unknown command: %s\n' % first ) + error( '*** Unknown command: %s\n' % line ) # pylint: enable-msg=R0201 diff --git a/mininet/examples b/mininet/examples new file mode 120000 index 0000000..a6573af --- /dev/null +++ b/mininet/examples @@ -0,0 +1 @@ +../examples \ No newline at end of file diff --git a/mininet/net.py b/mininet/net.py index beaf1bd..4994f9e 100755 --- a/mininet/net.py +++ b/mininet/net.py @@ -102,7 +102,7 @@ from mininet.util import macColonHex, ipStr, ipParse, netParse, ipAdd from mininet.term import cleanUpScreens, makeTerms # Mininet version: should be consistent with README and LICENSE -VERSION = "2.0.0" +VERSION = "2.1.0" class Mininet( object ): "Network emulation with hosts spawned in network namespaces." @@ -209,9 +209,19 @@ class Mininet( object ): def addController( self, name='c0', controller=None, **params ): """Add controller. controller: Controller class""" + # Get controller class if not controller: controller = self.controller - controller_new = controller( name, **params ) + # Construct new controller if one is not given + if isinstance(name, Controller): + controller_new = name + # Pylint thinks controller is a str() + # pylint: disable=E1103 + name = controller_new.name + # pylint: enable=E1103 + else: + controller_new = controller( name, **params ) + # Add new controller to net if controller_new: # allow controller-less setups self.controllers.append( controller_new ) self.nameToNode[ name ] = controller_new @@ -230,31 +240,31 @@ class Mininet( object ): return self.getNodeByName( *args ) # Even more convenient syntax for node lookup and iteration - def __getitem__( self, *args ): + def __getitem__( self, key ): """net [ name ] operator: Return node(s) with given name(s)""" - return self.getNodeByName( *args ) + return self.nameToNode[ key ] def __iter__( self ): - "return iterator over nodes" - #or dow we want to iterate of the keys i.e. node.name like a dict + "return iterator over node names" for node in chain( self.hosts, self.switches, self.controllers ): yield node.name def __len__( self ): "returns number of nodes in net" - return len( self.hosts ) + len( self.switches ) + len( self.controllers ) + return ( len( self.hosts ) + len( self.switches ) + + len( self.controllers ) ) def __contains__( self, item ): "returns True if net contains named node" - return item in self.keys() + return item in self.nameToNode def keys( self ): "return a list of all node names or net's keys" - return list( self.__iter__() ) + return list( self ) def values( self ): "return a list of all nodes or net's values" - return [ self[name] for name in self.__iter__() ] + return [ self[name] for name in self ] def items( self ): "return (key,value) tuple list for every node in net" @@ -307,7 +317,7 @@ class Mininet( object ): info( '*** Creating network\n' ) - if not self.controllers: + if not self.controllers and self.controller: # Add a default controller info( '*** Adding controller\n' ) classes = self.controller diff --git a/mininet/node.py b/mininet/node.py index a737845..3d6dc0e 100644 --- a/mininet/node.py +++ b/mininet/node.py @@ -183,7 +183,8 @@ class Node( object ): def terminate( self ): "Send kill signal to Node and clean up after it." - os.kill( self.pid, signal.SIGKILL ) + if self.shell: + os.kill( self.pid, signal.SIGKILL ) self.cleanup() def stop( self ): @@ -1050,6 +1051,8 @@ class OVSSwitch( Switch ): def stop( self ): "Terminate OVS switch." self.cmd( 'ovs-vsctl del-br', self ) + if self.datapath == 'user': + self.cmd( 'ip link del', self ) self.deleteIntfs() OVSKernelSwitch = OVSSwitch @@ -1140,7 +1143,7 @@ class Controller( Node ): "installed." ) listening = self.cmd( "echo A | telnet -e A %s %d" % ( self.ip, self.port ) ) - if 'refused' not in listening: + if 'Connected' in listening: servers = self.cmd( 'netstat -atp' ).split( '\n' ) pstr = ':%d ' % self.port clist = servers[ 0:1 ] + [ s for s in servers if pstr in s ] @@ -1235,6 +1238,7 @@ class RemoteController( Controller ): "Warn if remote controller is not accessible" listening = self.cmd( "echo A | telnet -e A %s %d" % ( self.ip, self.port ) ) - if 'refused' in listening: + if 'Connected' not in listening: warn( "Unable to contact the remote controller" " at %s:%d\n" % ( self.ip, self.port ) ) + diff --git a/mininet/test/test_hifi.py b/mininet/test/test_hifi.py index e881f3a..c9d288b 100755 --- a/mininet/test/test_hifi.py +++ b/mininet/test/test_hifi.py @@ -4,13 +4,15 @@ Test creation and pings for topologies with link and/or CPU options.""" import unittest +from functools import partial from mininet.net import Mininet -from mininet.node import OVSKernelSwitch, UserSwitch, IVSSwitch +from mininet.node import OVSSwitch, UserSwitch, IVSSwitch from mininet.node import CPULimitedHost from mininet.link import TCLink from mininet.topo import Topo from mininet.log import setLogLevel +from mininet.util import quietRun # Number of hosts for each test N = 2 @@ -29,9 +31,12 @@ class SingleSwitchOptionsTopo(Topo): host = self.addHost('h%s' % (h + 1)) self.addLink(host, switch) +# Tell pylint not to complain about calls to other class +# pylint: disable=E1101 class testOptionsTopoCommon( object ): - "Verify ability to create networks with host and link options (common code)." + """Verify ability to create networks with host and link options + (common code).""" switchClass = None # overridden in subclasses @@ -39,7 +44,8 @@ class testOptionsTopoCommon( object ): "Generic topology-with-options test runner." mn = Mininet( topo=SingleSwitchOptionsTopo( n=n, hopts=hopts, lopts=lopts ), - host=CPULimitedHost, link=TCLink, switch=self.switchClass ) + host=CPULimitedHost, link=TCLink, + switch=self.switchClass ) dropped = mn.run( mn.ping ) self.assertEqual( dropped, 0 ) @@ -105,7 +111,8 @@ class testOptionsTopoCommon( object ): REPS = 1 lopts = { 'loss': LOSS_PERCENT, 'use_htb': True } mn = Mininet( topo=SingleSwitchOptionsTopo( n=N, lopts=lopts ), - host=CPULimitedHost, link=TCLink, switch=self.switchClass ) + host=CPULimitedHost, link=TCLink, + switch=self.switchClass ) # Drops are probabilistic, but the chance of no dropped packets is # 1 in 100 million with 4 hops for a link w/99% loss. dropped_total = 0 @@ -121,16 +128,28 @@ class testOptionsTopoCommon( object ): hopts = { 'cpu': 0.5 / N } self.runOptionsTopoTest( N, hopts=hopts, lopts=lopts ) -class testOptionsTopoOVSKernel( testOptionsTopoCommon, unittest.TestCase ): - "Verify ability to create networks with host and link options (OVS kernel switch)." - switchClass = OVSKernelSwitch +# pylint: enable=E1101 +class testOptionsTopoOVSKernel( testOptionsTopoCommon, unittest.TestCase ): + """Verify ability to create networks with host and link options + (OVS kernel switch).""" + switchClass = OVSSwitch + +@unittest.skip( 'Skipping OVS user switch test for now' ) +class testOptionsTopoOVSUser( testOptionsTopoCommon, unittest.TestCase ): + """Verify ability to create networks with host and link options + (OVS user switch).""" + switchClass = partial( OVSSwitch, datapath='user' ) + +@unittest.skipUnless( quietRun( 'which ivs-ctl' ), 'IVS is not installed' ) class testOptionsTopoIVS( testOptionsTopoCommon, unittest.TestCase ): - "Verify ability to create networks with host and link options (IVS switch)." + "Verify ability to create networks with host and link options (IVS)." switchClass = IVSSwitch +@unittest.skipUnless( quietRun( 'which ofprotocol' ), + 'Reference user switch is not installed' ) class testOptionsTopoUserspace( testOptionsTopoCommon, unittest.TestCase ): - "Verify ability to create networks with host and link options (Userspace switch)." + "Verify ability to create networks with host and link options (UserSwitch)." switchClass = UserSwitch if __name__ == '__main__': diff --git a/mininet/test/test_nets.py b/mininet/test/test_nets.py index 027bdd4..159ba34 100755 --- a/mininet/test/test_nets.py +++ b/mininet/test/test_nets.py @@ -4,13 +4,17 @@ Test creation and all-pairs ping for each included mininet topo type.""" import unittest +from functools import partial from mininet.net import Mininet from mininet.node import Host, Controller -from mininet.node import UserSwitch, OVSKernelSwitch, IVSSwitch +from mininet.node import UserSwitch, OVSSwitch, IVSSwitch from mininet.topo import SingleSwitchTopo, LinearTopo from mininet.log import setLogLevel +from mininet.util import quietRun +# Tell pylint not to complain about calls to other class +# pylint: disable=E1101 class testSingleSwitchCommon( object ): "Test ping with single switch topology (common code)." @@ -25,23 +29,36 @@ class testSingleSwitchCommon( object ): def testSingle5( self ): "Ping test on 5-host single-switch topology" - mn = Mininet( SingleSwitchTopo( k=5 ), self.switchClass, Host, Controller ) + mn = Mininet( SingleSwitchTopo( k=5 ), self.switchClass, Host, + Controller ) dropped = mn.run( mn.ping ) self.assertEqual( dropped, 0 ) +# pylint: enable=E1101 + class testSingleSwitchOVSKernel( testSingleSwitchCommon, unittest.TestCase ): "Test ping with single switch topology (OVS kernel switch)." - switchClass = OVSKernelSwitch + switchClass = OVSSwitch +class testSingleSwitchOVSUser( testSingleSwitchCommon, unittest.TestCase ): + "Test ping with single switch topology (OVS user switch)." + switchClass = partial( OVSSwitch, datapath='user' ) + +@unittest.skipUnless( quietRun( 'which ivs-ctl' ), 'IVS is not installed' ) class testSingleSwitchIVS( testSingleSwitchCommon, unittest.TestCase ): "Test ping with single switch topology (IVS switch)." switchClass = IVSSwitch +@unittest.skipUnless( quietRun( 'which ofprotocol' ), + 'Reference user switch is not installed' ) class testSingleSwitchUserspace( testSingleSwitchCommon, unittest.TestCase ): "Test ping with single switch topology (Userspace switch)." switchClass = UserSwitch +# Tell pylint not to complain about calls to other class +# pylint: disable=E1101 + class testLinearCommon( object ): "Test all-pairs ping with LinearNet (common code)." @@ -53,14 +70,24 @@ class testLinearCommon( object ): dropped = mn.run( mn.ping ) self.assertEqual( dropped, 0 ) +# pylint: enable=E1101 + + class testLinearOVSKernel( testLinearCommon, unittest.TestCase ): "Test all-pairs ping with LinearNet (OVS kernel switch)." - switchClass = OVSKernelSwitch + switchClass = OVSSwitch +class testLinearOVSUser( testLinearCommon, unittest.TestCase ): + "Test all-pairs ping with LinearNet (OVS user switch)." + switchClass = partial( OVSSwitch, datapath='user' ) + +@unittest.skipUnless( quietRun( 'which ivs-ctl' ), 'IVS is not installed' ) class testLinearIVS( testLinearCommon, unittest.TestCase ): "Test all-pairs ping with LinearNet (IVS switch)." switchClass = IVSSwitch +@unittest.skipUnless( quietRun( 'which ofprotocol' ), + 'Reference user switch is not installed' ) class testLinearUserspace( testLinearCommon, unittest.TestCase ): "Test all-pairs ping with LinearNet (Userspace switch)." switchClass = UserSwitch diff --git a/mininet/topo.py b/mininet/topo.py index 148011d..2ab455c 100644 --- a/mininet/topo.py +++ b/mininet/topo.py @@ -250,9 +250,9 @@ class LinearTopo(Topo): self.n = n if n == 1: - genHostName = lambda i,j: 'h%s' % i + genHostName = lambda i, j: 'h%s' % i else: - genHostName = lambda i,j: 'h%ss%d' % (j,i) + genHostName = lambda i, j: 'h%ss%d' % (j, i) lastSwitch = None @@ -261,8 +261,6 @@ class LinearTopo(Topo): switch = self.addSwitch('s%s' % i) # Add hosts to switch for j in irange(1, n): - hostNum = (i-1)*n + j - #host = self.addHost('h%s' % hostNum) host = self.addHost(genHostName(i, j)) self.addLink(host, switch) # Connect switch to previous diff --git a/mininet/util.py b/mininet/util.py index 05df136..385cdb7 100644 --- a/mininet/util.py +++ b/mininet/util.py @@ -1,6 +1,6 @@ "Utility functions for Mininet." -from mininet.log import output, info, error, warn +from mininet.log import output, info, error, warn, debug from time import sleep from resource import getrlimit, setrlimit, RLIMIT_NPROC, RLIMIT_NOFILE @@ -361,16 +361,17 @@ def sysctlTestAndSet( name, limit ): if '/' not in name: name = '/proc/sys/' + name.replace( '.', '/' ) #read limit - f = open( name, 'r+' ) - oldLimit = f.readline() - if type( limit ) is int: - #compare integer limits before overriding - if int( oldLimit ) < limit: - f.write( "%d" % limit ) - else: - #overwrite non-integer limits - f.write( limit ) - f.close() + with open( name, 'r' ) as readFile: + oldLimit = readFile.readline() + if type( limit ) is int: + #compare integer limits before overriding + if int( oldLimit ) < limit: + with open( name, 'w' ) as writeFile: + writeFile.write( "%d" % limit ) + else: + #overwrite non-integer limits + with open( name, 'w' ) as writeFile: + writeFile.write( limit ) def rlimitTestAndSet( name, limit ): "Helper function to set rlimits" @@ -381,24 +382,30 @@ def rlimitTestAndSet( name, limit ): def fixLimits(): "Fix ridiculously small resource limits." - rlimitTestAndSet( RLIMIT_NPROC, 8192 ) - rlimitTestAndSet( RLIMIT_NOFILE, 16384 ) - #Increase open file limit - sysctlTestAndSet( 'fs.file-max', 10000 ) - #Increase network buffer space - sysctlTestAndSet( 'net.core.wmem_max', 16777216 ) - sysctlTestAndSet( 'net.core.rmem_max', 16777216 ) - sysctlTestAndSet( 'net.ipv4.tcp_rmem', '10240 87380 16777216' ) - sysctlTestAndSet( 'net.ipv4.tcp_wmem', '10240 87380 16777216' ) - sysctlTestAndSet( 'net.core.netdev_max_backlog', 5000 ) - #Increase arp cache size - sysctlTestAndSet( 'net.ipv4.neigh.default.gc_thresh1', 4096 ) - sysctlTestAndSet( 'net.ipv4.neigh.default.gc_thresh2', 8192 ) - sysctlTestAndSet( 'net.ipv4.neigh.default.gc_thresh3', 16384 ) - #Increase routing table size - sysctlTestAndSet( 'net.ipv4.route.max_size', 32768 ) - #Increase number of PTYs for nodes - sysctlTestAndSet( 'kernel.pty.max', 20000 ) + debug( "*** Setting resource limits\n" ) + try: + rlimitTestAndSet( RLIMIT_NPROC, 8192 ) + rlimitTestAndSet( RLIMIT_NOFILE, 16384 ) + #Increase open file limit + sysctlTestAndSet( 'fs.file-max', 10000 ) + #Increase network buffer space + sysctlTestAndSet( 'net.core.wmem_max', 16777216 ) + sysctlTestAndSet( 'net.core.rmem_max', 16777216 ) + sysctlTestAndSet( 'net.ipv4.tcp_rmem', '10240 87380 16777216' ) + sysctlTestAndSet( 'net.ipv4.tcp_wmem', '10240 87380 16777216' ) + sysctlTestAndSet( 'net.core.netdev_max_backlog', 5000 ) + #Increase arp cache size + sysctlTestAndSet( 'net.ipv4.neigh.default.gc_thresh1', 4096 ) + sysctlTestAndSet( 'net.ipv4.neigh.default.gc_thresh2', 8192 ) + sysctlTestAndSet( 'net.ipv4.neigh.default.gc_thresh3', 16384 ) + #Increase routing table size + sysctlTestAndSet( 'net.ipv4.route.max_size', 32768 ) + #Increase number of PTYs for nodes + sysctlTestAndSet( 'kernel.pty.max', 20000 ) + assert False + except: + warn( "*** Error setting resource limits. " + "Mininet's performance may be affected.\n" ) def mountCgroups(): "Make sure cgroups file system is mounted" diff --git a/setup.py b/setup.py index 9cee655..346f525 100644 --- a/setup.py +++ b/setup.py @@ -20,19 +20,19 @@ setup( description='Process-based OpenFlow emulator', author='Bob Lantz', author_email='rlantz@cs.stanford.edu', - packages=find_packages(exclude='test'), + packages=[ 'mininet', 'mininet.examples' ], long_description=""" Mininet is a network emulator which uses lightweight virtualization to create virtual networks for rapid prototyping of Software-Defined Network (SDN) designs - using OpenFlow. http://openflow.org/mininet + using OpenFlow. http://mininet.org """, classifiers=[ "License :: OSI Approved :: BSD License", "Programming Language :: Python", - "Development Status :: 2 - Pre-Alpha", + "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", - "Topic :: Internet", + "Topic :: System :: Emulators", ], keywords='networking emulator protocol Internet OpenFlow SDN', license='BSD', diff --git a/util/install.sh b/util/install.sh index 02f146f..d7921e6 100755 --- a/util/install.sh +++ b/util/install.sh @@ -10,15 +10,15 @@ set -e set -o nounset # Get directory containing mininet folder -MININET_DIR="$( cd -P "$( dirname "${BASH_SOURCE[0]}" )/../.." && pwd )" +MININET_DIR="$( cd -P "$( dirname "${BASH_SOURCE[0]}" )/../.." && pwd -P )" # Set up build directory, which by default is the working directory # unless the working directory is a subdirectory of mininet, # in which case we use the directory containing mininet -BUILD_DIR=$PWD -case $PWD in +BUILD_DIR="$(pwd -P)" +case $BUILD_DIR in $MININET_DIR/*) BUILD_DIR=$MININET_DIR;; # currect directory is a subdirectory - *) BUILD_DIR=$PWD;; + *) BUILD_DIR=$BUILD_DIR;; esac # Location of CONFIG_NET_NS-enabled kernel(s) @@ -281,7 +281,7 @@ function wireshark { export WIRESHARK=/usr/include/wireshark scons # libwireshark0/ on 11.04; libwireshark1/ on later - WSDIR=`ls -d /usr/lib/wireshark/libwireshark* | head -1` + WSDIR=`find /usr/lib -type d -name 'libwireshark*' | head -1` WSPLUGDIR=$WSDIR/plugins/ sudo cp openflow.so $WSPLUGDIR echo "Copied openflow plugin to $WSPLUGDIR" @@ -645,13 +645,16 @@ function all { printf "with Fedora's kernel (3.10) and openvswitch (1.10.0) packages.\n" exit 3 fi - echo "Running all commands..." + echo "Installing all packages except for -eix (doxypy, ivs, nox-classic)..." kernel mn_deps - mn_dev + # Skip mn_dev (doxypy/texlive/fonts/etc.) because it's huge + # mn_dev of wireshark ovs + # We may add ivs once it's more mature + # ivs # NOX-classic is deprecated, but you can install it manually if desired. # nox pox diff --git a/util/m b/util/m index 786dcce..860eeb0 100755 --- a/util/m +++ b/util/m @@ -33,4 +33,12 @@ if [ -d "$cgroup" ]; then cg="-g $host" fi -exec sudo mnexec -a $pid $cg $cmd +# Check whether host should be running in a chroot dir +rootdir="/var/run/mn/$host/root" +if [ -d $rootdir -a -x $rootdir/bin/bash ]; then + cmd="'cd `pwd`; exec $cmd'" + cmd="chroot $rootdir /bin/bash -c $cmd" +fi + +cmd="exec sudo mnexec -a $pid $cg $cmd" +eval $cmd diff --git a/util/vm/build.py b/util/vm/build.py index eb54c94..79ec1dd 100755 --- a/util/vm/build.py +++ b/util/vm/build.py @@ -6,16 +6,17 @@ build.py: build a Mininet VM Basic idea: prepare - - download cloud image if it's missing - - write-protect it + -> create base install image if it's missing + - download iso if it's missing + - install from iso onto image build - -> create cow disk for vm + -> create cow disk for new VM, based on base image -> boot it in qemu/kvm with text /serial console -> install Mininet test - -> make codecheck + -> sudo mn --test pingall -> make test release @@ -23,59 +24,98 @@ Basic idea: -> shrink-wrap VM -> upload to storage - -Notes du jour: - -- our infrastructure is currently based on 12.04 LTS, so we - can't rely on cloud-localds which is only in 12.10+ - -- as a result, we should download the tar image, extract it, - and boot with tty0 as the console (I think) - -- and we'll manually add the mininet user to it - -- and use pexpect to interact with it on the serial console - -Something to think about: - -Maybe download the cloud image and customize it so that -it is an actual usable/bootable image??? - - """ - import os -from os import stat -from stat import ST_MODE -from os.path import exists, splitext -from sys import exit, argv +from os import stat, path +from stat import ST_MODE, ST_SIZE +from os.path import abspath +from sys import exit, stdout, argv, modules +import re from glob import glob -from urllib import urlretrieve -from subprocess import check_output, call, Popen, PIPE +from subprocess import check_output, call, Popen from tempfile import mkdtemp, NamedTemporaryFile -from time import time +from time import time, strftime, localtime import argparse +from distutils.spawn import find_executable +import inspect + +pexpect = None # For code check - imported dynamically # boot can be slooooow!!!! need to debug/optimize somehow TIMEOUT=600 +# Some configuration options +# Possibly change this to use the parsed arguments instead! + +LogToConsole = False # VM output to console rather than log file +SaveQCOW2 = False # Save QCOW2 image rather than deleting it +NoKVM = False # Don't use kvm and use emulation instead +Branch = None # Branch to update and check out before testing + VMImageDir = os.environ[ 'HOME' ] + '/vm-images' -ImageURLBase = { +Prompt = '\$ ' # Shell prompt that pexpect will wait for + +isoURLs = { + 'precise32server': + 'http://mirrors.kernel.org/ubuntu-releases/12.04/' + 'ubuntu-12.04.3-server-i386.iso', + 'precise64server': + 'http://mirrors.kernel.org/ubuntu-releases/12.04/' + 'ubuntu-12.04.3-server-amd64.iso', + 'quantal32server': + 'http://mirrors.kernel.org/ubuntu-releases/12.10/' + 'ubuntu-12.10-server-i386.iso', + 'quantal64server': + 'http://mirrors.kernel.org/ubuntu-releases/12.10/' + 'ubuntu-12.10-server-amd64.iso', 'raring32server': - 'http://cloud-images.ubuntu.com/raring/current/' - 'raring-server-cloudimg-i386', + 'http://mirrors.kernel.org/ubuntu-releases/13.04/' + 'ubuntu-13.04-server-i386.iso', 'raring64server': - 'http://cloud-images.ubuntu.com/raring/current/' - 'raring-server-cloudimg-amd64' + 'http://mirrors.kernel.org/ubuntu-releases/13.04/' + 'ubuntu-13.04-server-amd64.iso', + 'saucy32server': + 'http://mirrors.kernel.org/ubuntu-releases/13.10/' + 'ubuntu-13.10-server-i386.iso', + 'saucy64server': + 'http://mirrors.kernel.org/ubuntu-releases/13.10/' + 'ubuntu-13.10-server-amd64.iso', } +LogStartTime = time() +LogFile = None + +def log( *args, **kwargs ): + """Simple log function: log( message along with local and elapsed time + cr: False/0 for no CR""" + cr = kwargs.get( 'cr', True ) + elapsed = time() - LogStartTime + clocktime = strftime( '%H:%M:%S', localtime() ) + msg = ' '.join( str( arg ) for arg in args ) + output = '%s [ %.3f ] %s' % ( clocktime, elapsed, msg ) + if cr: + print output + else: + print output, + # Optionally mirror to LogFile + if type( LogFile ) is file: + if cr: + output += '\n' + LogFile.write( output ) + LogFile.flush() + + def run( cmd, **kwargs ): "Convenient interface to check_output" - print cmd + log( '-', cmd ) cmd = cmd.split() + arg0 = cmd[ 0 ] + if not find_executable( arg0 ): + raise Exception( 'Cannot find executable "%s";' % arg0 + + 'you might try %s --depend' % argv[ 0 ] ) return check_output( cmd, **kwargs ) @@ -84,324 +124,707 @@ def srun( cmd, **kwargs ): return run( 'sudo ' + cmd, **kwargs ) +# BL: we should probably have a "checkDepend()" which +# checks to make sure all dependencies are satisfied! + def depend(): - "Install packagedependencies" - print '* Installing package dependencies' - packages = ( ) + "Install package dependencies" + log( '* Installing package dependencies' ) run( 'sudo apt-get -y update' ) run( 'sudo apt-get install -y' ' kvm cloud-utils genisoimage qemu-kvm qemu-utils' ' e2fsprogs ' ' landscape-client' - ' python-setuptools' ) + ' python-setuptools mtools' ) run( 'sudo easy_install pexpect' ) def popen( cmd ): "Convenient interface to popen" - print cmd + log( cmd ) cmd = cmd.split() return Popen( cmd ) def remove( fname ): - "rm -f fname" - return run( 'rm -f %s' % fname ) + "Remove a file, ignoring errors" + try: + os.remove( fname ) + except OSError: + pass - -def imageURL( image ): - "Return base URL for VM image" - return ImageURLBase[ image ] +def findiso( flavor ): + "Find iso, fetching it if it's not there already" + url = isoURLs[ flavor ] + name = path.basename( url ) + iso = path.join( VMImageDir, name ) + if not path.exists( iso ) or ( stat( iso )[ ST_MODE ] & 0777 != 0444 ): + log( '* Retrieving', url ) + run( 'curl -C - -o %s %s' % ( iso, url ) ) + if 'ISO' not in run( 'file ' + iso ): + os.remove( iso ) + raise Exception( 'findiso: could not download iso from ' + url ) + # Write-protect iso, signaling it is complete + log( '* Write-protecting iso', iso) + os.chmod( iso, 0444 ) + log( '* Using iso', iso ) + return iso -def imagePath( image ): - "Return base pathname for VM image files" - url = imageURL( image ) - fname = url.split( '/' )[ -1 ] - path = os.path.join( VMImageDir, fname ) - return path - - -def fetchImage( image, path=None ): - "Fetch base VM image if it's not there already" - if not path: - path = imagePath( image ) - tgz = path + '.tar.gz' - disk = path + '.img' - kernel = path + '-vmlinuz-generic' - floppy = path + '-floppy' - if exists( disk ) and exists( kernel ): - print '* Found', disk, 'and', kernel - # Detect race condition with multiple builds - perms = stat( disk )[ ST_MODE ] & 0777 - if perms != 0444: - raise Exception( 'Error - %s is writable ' % disk + - '; are multiple builds running?' ) - else: - dir = os.path.dirname( path ) - run( 'mkdir -p %s' % dir ) - if not os.path.exists( tgz ): - url = imageURL( image ) + '.tar.gz' - print '* Retrieving', url - urlretrieve( url, tgz ) - print '* Extracting', tgz - run( 'tar -C %s -xzf %s' % ( dir, tgz ) ) - # Write-protect disk image so it remains pristine; - # We will not use it directly but will use a COW disk - print '* Write-protecting disk image', disk - os.chmod( disk, 0444 ) - return disk, kernel - - -def addTo( file, line ): - "Add line to file if it's not there already" - if call( [ 'sudo', 'grep', line, file ] ) != 0: - call( 'echo "%s" | sudo tee -a %s' % ( line, file ), shell=True ) - - -def disableCloud( bind ): - "Disable cloud junk for disk mounted at bind" - print '* Disabling cloud startup scripts' - modules = glob( '%s/etc/init/cloud*.conf' % bind ) - for module in modules: - path, ext = splitext( module ) - override = path + '.override' - call( 'echo manual | sudo tee ' + override, shell=True ) - - -def addMininetUser( nbd ): - "Add mininet user/group to filesystem" - print '* Adding mininet user to filesystem on device', nbd - # 1. We bind-mount / into a temporary directory, and - # then mount the volume's /etc and /home on top of it! - mnt = mkdtemp() - bind = mkdtemp() - srun( 'mount %s %s' % ( nbd, mnt ) ) - srun( 'mount -B / ' + bind ) - srun( 'mount -B %s/etc %s/etc' % ( mnt, bind ) ) - srun( 'mount -B %s/home %s/home' % ( mnt, bind ) ) - def chroot( cmd ): - "Chroot into bind mount and run command" - call( 'sudo chroot %s ' % bind + cmd, shell=True ) - # 1a. Add hostname entry in /etc/hosts - addTo( bind + '/etc/hosts', '127.0.1.1 mininet-vm' ) - # 2. Next, we delete any old mininet user and add a new one - chroot( 'deluser mininet' ) - chroot( 'useradd --create-home mininet' ) - print '* Setting password' - call( 'echo mininet:mininet | sudo chroot %s chpasswd -c SHA512' - % bind, shell=True ) - # 2a. Add mininet to sudoers - addTo( bind + '/etc/sudoers', 'mininet ALL=NOPASSWD: ALL' ) - # 2b. Disable cloud junk - disableCloud( bind ) - chroot( 'sudo update-rc.d landscape-client disable' ) - # 2c. Add serial getty - print '* Adding getty on ttyS0' - chroot( 'cp /etc/init/tty1.conf /etc/init/ttyS0.conf' ) - chroot( 'sed -i "s/tty1/ttyS0/g" /etc/init/ttyS0.conf' ) - # 3. Lastly, we umount and clean up everything - run( 'sync' ) - srun( 'umount %s/home ' % bind ) - srun( 'umount %s/etc ' % bind ) - srun( 'umount %s' % bind ) - srun( 'umount ' + mnt ) - run( 'rmdir ' + bind ) - run( 'rmdir ' + mnt ) - # 4. Just to make sure, we check the filesystem - srun( 'e2fsck -y ' + nbd ) - - -def connectCOWdevice( cow ): - "Attempt to connect a COW disk and return its nbd device" - print '* Checking for unused /dev/nbdX device ', +def attachNBD( cow, flags='' ): + """Attempt to attach a COW disk image and return its nbd device + flags: additional flags for qemu-nbd (e.g. -r for readonly)""" + # qemu-nbd requires an absolute path + cow = abspath( cow ) + log( '* Checking for unused /dev/nbdX device ' ) for i in range ( 0, 63 ): nbd = '/dev/nbd%d' % i - print i, # Check whether someone's already messing with that device if call( [ 'pgrep', '-f', nbd ] ) == 0: continue - # Fails without -v for some annoying reason... + srun( 'modprobe nbd max-part=64' ) + srun( 'qemu-nbd %s -c %s %s' % ( flags, nbd, cow ) ) print - srun( 'qemu-nbd -c %s %s' % ( nbd, cow ) ) return nbd raise Exception( "Error: could not find unused /dev/nbdX device" ) -def disconnectCOWdevice( nbd ): +def detachNBD( nbd ): + "Detatch an nbd device" srun( 'qemu-nbd -d ' + nbd ) -def makeCOWDisk( image ): - "Create new COW disk for image" - disk, kernel = fetchImage( image ) - cow = NamedTemporaryFile( prefix=image + '-', suffix='.qcow2', - dir='.' ).name - print '* Creating COW disk', cow - run( 'qemu-img create -f qcow2 -b %s %s' % ( disk, cow ) ) - print '* Resizing COW disk and file system' - run( 'qemu-img resize %s +8G' % cow ) - srun( 'modprobe nbd max-part=64') - nbd = connectCOWdevice( cow ) - srun( 'e2fsck -y ' + nbd ) - srun( 'resize2fs ' + nbd ) - addMininetUser( nbd ) - disconnectCOWdevice( nbd ) - return cow, kernel +def extractKernel( image, flavor, imageDir=VMImageDir ): + "Extract kernel and initrd from base image" + kernel = path.join( imageDir, flavor + '-vmlinuz' ) + initrd = path.join( imageDir, flavor + '-initrd' ) + if path.exists( kernel ) and ( stat( image )[ ST_MODE ] & 0777 ) == 0444: + # If kernel is there, then initrd should also be there + return kernel, initrd + log( '* Extracting kernel to', kernel ) + nbd = attachNBD( image, flags='-r' ) + print srun( 'partx ' + nbd ) + # Assume kernel is in partition 1/boot/vmlinuz*generic for now + part = nbd + 'p1' + mnt = mkdtemp() + srun( 'mount -o ro %s %s' % ( part, mnt ) ) + kernsrc = glob( '%s/boot/vmlinuz*generic' % mnt )[ 0 ] + initrdsrc = glob( '%s/boot/initrd*generic' % mnt )[ 0 ] + srun( 'cp %s %s' % ( initrdsrc, initrd ) ) + srun( 'chmod 0444 ' + initrd ) + srun( 'cp %s %s' % ( kernsrc, kernel ) ) + srun( 'chmod 0444 ' + kernel ) + srun( 'umount ' + mnt ) + run( 'rmdir ' + mnt ) + detachNBD( nbd ) + return kernel, initrd -def boot( cow, kernel, tap ): +def findBaseImage( flavor, size='8G' ): + "Return base VM image and kernel, creating them if needed" + image = path.join( VMImageDir, flavor + '-base.qcow2' ) + if path.exists( image ): + # Detect race condition with multiple builds + perms = stat( image )[ ST_MODE ] & 0777 + if perms != 0444: + raise Exception( 'Error - %s is writable ' % image + + '; are multiple builds running?' ) + else: + # We create VMImageDir here since we are called first + run( 'mkdir -p %s' % VMImageDir ) + iso = findiso( flavor ) + log( '* Creating image file', image ) + run( 'qemu-img create -f qcow2 %s %s' % ( image, size ) ) + installUbuntu( iso, image ) + # Write-protect image, also signaling it is complete + log( '* Write-protecting image', image) + os.chmod( image, 0444 ) + kernel, initrd = extractKernel( image, flavor ) + log( '* Using base image', image, 'and kernel', kernel ) + return image, kernel, initrd + + +# Kickstart and Preseed files for Ubuntu/Debian installer +# +# Comments: this is really clunky and painful. If Ubuntu +# gets their act together and supports kickstart a bit better +# then we can get rid of preseed and even use this as a +# Fedora installer as well. +# +# Another annoying thing about Ubuntu is that it can't just +# install a normal system from the iso - it has to download +# junk from the internet, making this house of cards even +# more precarious. + +KickstartText =""" +#Generated by Kickstart Configurator +#platform=x86 + +#System language +lang en_US +#Language modules to install +langsupport en_US +#System keyboard +keyboard us +#System mouse +mouse +#System timezone +timezone America/Los_Angeles +#Root password +rootpw --disabled +#Initial user +user mininet --fullname "mininet" --password "mininet" +#Use text mode install +text +#Install OS instead of upgrade +install +#Use CDROM installation media +cdrom +#System bootloader configuration +bootloader --location=mbr +#Clear the Master Boot Record +zerombr yes +#Partition clearing information +clearpart --all --initlabel +#Automatic partitioning +autopart +#System authorization infomation +auth --useshadow --enablemd5 +#Firewall configuration +firewall --disabled +#Do not configure the X Window System +skipx +""" + +# Tell the Ubuntu/Debian installer to stop asking stupid questions + +PreseedText = """ +d-i mirror/country string manual +d-i mirror/http/hostname string mirrors.kernel.org +d-i mirror/http/directory string /ubuntu +d-i mirror/http/proxy string +d-i partman/confirm_write_new_label boolean true +d-i partman/choose_partition select finish +d-i partman/confirm boolean true +d-i partman/confirm_nooverwrite boolean true +d-i user-setup/allow-password-weak boolean true +d-i finish-install/reboot_in_progress note +d-i debian-installer/exit/poweroff boolean true +""" + +def makeKickstartFloppy(): + "Create and return kickstart floppy, kickstart, preseed" + kickstart = 'ks.cfg' + with open( kickstart, 'w' ) as f: + f.write( KickstartText ) + preseed = 'ks.preseed' + with open( preseed, 'w' ) as f: + f.write( PreseedText ) + # Create floppy and copy files to it + floppy = 'ksfloppy.img' + run( 'qemu-img create %s 1440k' % floppy ) + run( 'mkfs -t msdos ' + floppy ) + run( 'mcopy -i %s %s ::/' % ( floppy, kickstart ) ) + run( 'mcopy -i %s %s ::/' % ( floppy, preseed ) ) + return floppy, kickstart, preseed + + +def archFor( filepath ): + "Guess architecture for file path" + name = path.basename( filepath ) + if '64' in name: + arch = 'x86_64' + elif 'i386' in name or '32' in name: + arch = 'i386' + else: + log( "Error: can't discern CPU for file name", name ) + exit( 1 ) + return arch + + +def installUbuntu( iso, image, logfilename='install.log' ): + "Install Ubuntu from iso onto image" + kvm = 'qemu-system-' + archFor( iso ) + floppy, kickstart, preseed = makeKickstartFloppy() + # Mount iso so we can use its kernel + mnt = mkdtemp() + srun( 'mount %s %s' % ( iso, mnt ) ) + kernel = path.join( mnt, 'install/vmlinuz' ) + initrd = path.join( mnt, 'install/initrd.gz' ) + if NoKVM: + accel = 'tcg' + else: + accel = 'kvm' + cmd = [ 'sudo', kvm, + '-machine', 'accel=%s' % accel, + '-nographic', + '-netdev', 'user,id=mnbuild', + '-device', 'virtio-net,netdev=mnbuild', + '-m', '1024', + '-k', 'en-us', + '-fda', floppy, + '-drive', 'file=%s,if=virtio' % image, + '-cdrom', iso, + '-kernel', kernel, + '-initrd', initrd, + '-append', + ' ks=floppy:/' + kickstart + + ' preseed/file=floppy://' + preseed + + ' console=ttyS0' ] + ubuntuStart = time() + log( '* INSTALLING UBUNTU FROM', iso, 'ONTO', image ) + log( ' '.join( cmd ) ) + log( '* logging to', abspath( logfilename ) ) + params = {} + if not LogToConsole: + logfile = open( logfilename, 'w' ) + params = { 'stdout': logfile, 'stderr': logfile } + vm = Popen( cmd, **params ) + log( '* Waiting for installation to complete') + vm.wait() + if not LogToConsole: + logfile.close() + elapsed = time() - ubuntuStart + # Unmount iso and clean up + srun( 'umount ' + mnt ) + run( 'rmdir ' + mnt ) + if vm.returncode != 0: + raise Exception( 'Ubuntu installation returned error %d' % + vm.returncode ) + log( '* UBUNTU INSTALLATION COMPLETED FOR', image ) + log( '* Ubuntu installation completed in %.2f seconds' % elapsed ) + + +def boot( cow, kernel, initrd, logfile ): """Boot qemu/kvm with a COW disk and local/user data store cow: COW disk path kernel: kernel path - tap: tap device to connect to VM + logfile: log file for pexpect object returns: pexpect object to qemu process""" # pexpect might not be installed until after depend() is called global pexpect import pexpect - if 'amd64' in kernel: - kvm = 'qemu-system-x86_64' - elif 'i386' in kernel: - kvm = 'qemu-system-i386' + arch = archFor( kernel ) + if NoKVM: + accel = 'tcg' else: - print "Error: can't discern CPU for image", cow - exit( 1 ) - cmd = [ 'sudo', kvm, - '-machine accel=kvm', + accel = 'kvm' + cmd = [ 'sudo', 'qemu-system-' + arch, + '-machine accel=%s' % accel, '-nographic', '-netdev user,id=mnbuild', '-device virtio-net,netdev=mnbuild', '-m 1024', '-k en-us', '-kernel', kernel, + '-initrd', initrd, '-drive file=%s,if=virtio' % cow, - '-append "root=/dev/vda init=/sbin/init console=ttyS0" ' ] + '-append "root=/dev/vda1 init=/sbin/init console=ttyS0" ' ] cmd = ' '.join( cmd ) - print '* STARTING VM' - print cmd - vm = pexpect.spawn( cmd, timeout=TIMEOUT ) + log( '* BOOTING VM FROM', cow ) + log( cmd ) + vm = pexpect.spawn( cmd, timeout=TIMEOUT, logfile=logfile ) return vm -def interact( vm ): - "Interact with vm, which is a pexpect object" - prompt = '\$ ' - print '* Waiting for login prompt' +def login( vm ): + "Log in to vm (pexpect object)" + log( '* Waiting for login prompt' ) vm.expect( 'login: ' ) - print '* Logging in' + log( '* Logging in' ) vm.sendline( 'mininet' ) - print '* Waiting for password prompt' + log( '* Waiting for password prompt' ) vm.expect( 'Password: ' ) - print '* Sending password' + log( '* Sending password' ) vm.sendline( 'mininet' ) - print '* Waiting for login...' - vm.expect( prompt ) - print '* Sending hostname command' - vm.sendline( 'hostname' ) - print '* Waiting for output' - vm.expect( prompt ) - print '* Fetching Mininet VM install script' - vm.sendline( 'wget ' - 'https://raw.github.com/mininet/mininet/master/util/vm/' - 'install-mininet-vm.sh' ) - vm.expect( prompt ) - print '* Running VM install script' - vm.sendline( 'bash install-mininet-vm.sh' ) - print '* Waiting for script to complete... ' - # Gigantic timeout for now ;-( - vm.expect( 'Done preparing Mininet', timeout=3600 ) - print '* Completed successfully' - vm.expect( prompt ) - print '* Testing Mininet' + log( '* Waiting for login...' ) + + +def sanityTest( vm ): + "Run Mininet sanity test (pingall) in vm" vm.sendline( 'sudo mn --test pingall' ) - if vm.expect( [ ' 0% dropped', pexpect.TIMEOUT ], timeout=30 ): - print '* Sanity check succeeded' + if vm.expect( [ ' 0% dropped', pexpect.TIMEOUT ], timeout=45 ) == 0: + log( '* Sanity check OK' ) else: - print '* Sanity check FAILED' - vm.expect( prompt ) - print '* Making sure cgroups are mounted' + log( '* Sanity check FAILED' ) + + +def coreTest( vm, prompt=Prompt ): + "Run core tests (make test) in VM" + log( '* Making sure cgroups are mounted' ) vm.sendline( 'sudo service cgroup-lite restart' ) vm.expect( prompt ) vm.sendline( 'sudo cgroups-mount' ) vm.expect( prompt ) - print '* Running make test' + log( '* Running make test' ) vm.sendline( 'cd ~/mininet; sudo make test' ) + # We should change "make test" to report the number of + # successful and failed tests. For now, we have to + # know the time for each test, which means that this + # script will have to change as we add more tests. + for test in range( 0, 2 ): + if vm.expect( [ 'OK', 'FAILED', pexpect.TIMEOUT ], timeout=180 ) == 0: + log( '* Test', test, 'OK' ) + else: + log( '* Test', test, 'FAILED' ) + + +def examplesquickTest( vm, prompt=Prompt ): + "Quick test of mininet examples" + vm.sendline( 'sudo apt-get install python-pexpect' ) vm.expect( prompt ) - print '* Shutting down' + vm.sendline( 'sudo python ~/mininet/examples/test/runner.py -v -quick' ) + + +def examplesfullTest( vm, prompt=Prompt ): + "Full (slow) test of mininet examples" + vm.sendline( 'sudo apt-get install python-pexpect' ) + vm.expect( prompt ) + vm.sendline( 'sudo python ~/mininet/examples/test/runner.py -v' ) + + +def checkOutBranch( vm, branch, prompt=Prompt ): + vm.sendline( 'cd ~/mininet; git fetch; git pull --rebase; git checkout ' + + branch ) + vm.expect( prompt ) + vm.sendline( 'sudo make install' ) + + +def interact( vm, prompt=Prompt ): + "Interact with vm, which is a pexpect object" + login( vm ) + log( '* Waiting for login...' ) + vm.expect( prompt ) + log( '* Sending hostname command' ) + vm.sendline( 'hostname' ) + log( '* Waiting for output' ) + vm.expect( prompt ) + log( '* Fetching Mininet VM install script' ) + vm.sendline( 'wget ' + 'https://raw.github.com/mininet/mininet/master/util/vm/' + 'install-mininet-vm.sh' ) + vm.expect( prompt ) + log( '* Running VM install script' ) + vm.sendline( 'bash install-mininet-vm.sh' ) + vm.expect ( 'password for mininet: ' ) + vm.sendline( 'mininet' ) + log( '* Waiting for script to complete... ' ) + # Gigantic timeout for now ;-( + vm.expect( 'Done preparing Mininet', timeout=3600 ) + log( '* Completed successfully' ) + vm.expect( prompt ) + log( '* Testing Mininet' ) + runTests( vm ) + log( '* Shutting down' ) vm.sendline( 'sync; sudo shutdown -h now' ) - print '* Waiting for EOF/shutdown' + log( '* Waiting for EOF/shutdown' ) vm.read() - print '* Interaction complete' + log( '* Interaction complete' ) def cleanup(): "Clean up leftover qemu-nbd processes and other junk" - call( 'sudo pkill -9 qemu-nbd', shell=True ) + call( [ 'sudo', 'pkill', '-9', 'qemu-nbd' ] ) def convert( cow, basename ): """Convert a qcow2 disk to a vmdk and put it a new directory basename: base name for output vmdk file""" - dir = mkdtemp( prefix=basename, dir='.' ) - vmdk = '%s/%s.vmdk' % ( dir, basename ) - print '* Converting qcow2 to vmdk' + vmdk = basename + '.vmdk' + log( '* Converting qcow2 to vmdk' ) run( 'qemu-img convert -f qcow2 -O vmdk %s %s' % ( cow, vmdk ) ) return vmdk -def build( flavor='raring-server-amd64' ): - "Build a Mininet VM" +# Template for OVF - a very verbose format! +# In the best of all possible worlds, we might use an XML +# library to generate this, but a template is easier and +# possibly more concise! +# Warning: XML file cannot begin with a newline! + +OVFTemplate = """ + + + + + +Virtual disk information + + + +The list of logical networks + +The nat network + + + +A Mininet Virtual Machine (%s) +mininet-vm + +Virtual hardware requirements + +hertz * 10^6 +Number of Virtual CPUs +1 virtual CPU(s) +1 +3 +1 + + +byte * 2^20 +Memory Size +%dMB of memory +2 +4 +%d + + +0 +scsiController0 +SCSI Controller +scsiController0 +4 +lsilogic +6 + + +0 +disk1 +ovf:/disk/vmdisk1 +11 +4 +17 + + +2 +true +nat +E1000 ethernet adapter on nat +ethernet0 +12 +E1000 +10 + + +0 +usb +USB Controller +usb +9 +23 + + + + +""" + + +def generateOVF( name, diskname, disksize, mem=1024 ): + """Generate (and return) OVF file "name.ovf" + name: root name of OVF file to generate + diskname: name of disk file + disksize: size of virtual disk in bytes + mem: VM memory size in MB""" + ovf = name + '.ovf' + filesize = stat( diskname )[ ST_SIZE ] + # OVFTemplate uses the memory size twice in a row + xmltext = OVFTemplate % ( diskname, filesize, disksize, name, mem, mem ) + with open( ovf, 'w+' ) as f: + f.write( xmltext ) + return ovf + + +def qcow2size( qcow2 ): + "Return virtual disk size (in bytes) of qcow2 image" + output = check_output( [ 'file', qcow2 ] ) + assert 'QCOW' in output + bytes = int( re.findall( '(\d+) bytes', output )[ 0 ] ) + return bytes + + +def build( flavor='raring32server' ): + "Build a Mininet VM; return vmdk and vdisk size" + global LogFile start = time() - cow, kernel = makeCOWDisk( flavor ) - print '* VM image for', flavor, 'created as', cow - with NamedTemporaryFile( - prefix='mn-build-%s-' % flavor, suffix='.log', dir='.' ) as logfile: - print '* Logging results to', logfile.name - vm = boot( cow, kernel, logfile ) - vm.logfile_read = logfile - interact( vm ) - # cow is a temporary file and will go away when we quit! - # We convert it to a .vmdk which can be used in most VMMs - vmdk = convert( cow, basename=flavor ) - print '* Converted VM image stored as', vmdk + date = strftime( '%y%m%d-%H-%M-%S', localtime()) + dir = 'mn-%s-%s' % ( flavor, date ) + try: + os.mkdir( dir ) + except: + raise Exception( "Failed to create build directory %s" % dir ) + os.chdir( dir ) + LogFile = open( 'build.log', 'w' ) + log( '* Logging to', abspath( LogFile.name ) ) + log( '* Created working directory', dir ) + image, kernel, initrd = findBaseImage( flavor ) + basename = 'mininet-' + flavor + volume = basename + '.qcow2' + run( 'qemu-img create -f qcow2 -b %s %s' % ( image, volume ) ) + log( '* VM image for', flavor, 'created as', volume ) + if LogToConsole: + logfile = stdout + else: + logfile = open( flavor + '.log', 'w+' ) + log( '* Logging results to', abspath( logfile.name ) ) + vm = boot( volume, kernel, initrd, logfile ) + interact( vm ) + size = qcow2size( volume ) + vmdk = convert( volume, basename=basename ) + if not SaveQCOW2: + log( '* Removing qcow2 volume', volume ) + os.remove( volume ) + log( '* Converted VM image stored as', abspath( vmdk ) ) + ovf = generateOVF( diskname=vmdk, disksize=size, name=basename ) + log( '* Generated OVF descriptor file', ovf ) end = time() elapsed = end - start - print '* Results logged to', logfile.name - print '* Completed in %.2f seconds' % elapsed - print '* %s VM build DONE!!!!! :D' % flavor - print + log( '* Results logged to', abspath( logfile.name ) ) + log( '* Completed in %.2f seconds' % elapsed ) + log( '* %s VM build DONE!!!!! :D' % flavor ) + os.chdir( '..' ) + + +def runTests( vm, tests=None, prompt=Prompt ): + "Run tests (list) in vm (pexpect object)" + if not tests: + tests = [ 'sanity', 'core' ] + testfns = testDict() + for test in tests: + if test not in testfns: + raise Exception( 'Unknown test: ' + test ) + log( '* Running test', test ) + fn = testfns[ test ] + fn( vm ) + vm.expect( prompt ) + + +def bootAndRunTests( image, tests=None ): + """Boot and test VM + tests: list of tests (default: sanity, core)""" + bootTestStart = time() + basename = path.basename( image ) + image = abspath( image ) + tmpdir = mkdtemp( prefix='test-' + basename ) + log( '* Using tmpdir', tmpdir ) + cow = path.join( tmpdir, basename + '.qcow2' ) + log( '* Creating COW disk', cow ) + run( 'qemu-img create -f qcow2 -b %s %s' % ( image, cow ) ) + log( '* Extracting kernel and initrd' ) + kernel, initrd = extractKernel( image, flavor=basename, imageDir=tmpdir ) + if LogToConsole: + logfile = stdout + else: + logfile = NamedTemporaryFile( prefix=basename, + suffix='.testlog', delete=False ) + log( '* Logging VM output to', logfile.name ) + vm = boot( cow=cow, kernel=kernel, initrd=initrd, logfile=logfile ) + prompt = '\$ ' + login( vm ) + log( '* Waiting for VM boot and login' ) + vm.expect( prompt ) + if Branch: + checkOutBranch( vm, branch=Branch ) + vm.expect( prompt ) + log( '* Running tests' ) + runTests( vm, tests=tests ) + # runTests eats its last prompt, but maybe it shouldn't... + log( '* Shutting down' ) + vm.sendline( 'sudo shutdown -h now ' ) + log( '* Waiting for shutdown' ) + vm.wait() + log( '* Removing temporary dir', tmpdir ) + srun( 'rm -rf ' + tmpdir ) + elapsed = time() - bootTestStart + log( '* Boot and test completed in %.2f seconds' % elapsed ) + + +def buildFlavorString(): + "Return string listing valid build flavors" + return 'valid build flavors: ( %s )' % ' '.join( sorted( isoURLs ) ) + + +def testDict(): + "Return dict of tests in this module" + suffix = 'Test' + trim = len( suffix ) + fdict = dict( [ ( fname[ : -trim ], f ) for fname, f in + inspect.getmembers( modules[ __name__ ], + inspect.isfunction ) + if fname.endswith( suffix ) ] ) + return fdict + + +def testString(): + "Return string listing valid tests" + return 'valid tests: ( %s )' % ' '.join( testDict().keys() ) def parseArgs(): "Parse command line arguments and run" - parser = argparse.ArgumentParser( description='Mininet VM build script' ) - parser.add_argument( '--depend', action='store_true', - help='Install dependencies for this script' ) - parser.add_argument( '--list', action='store_true', - help='list valid build flavors' ) - parser.add_argument( '--clean', action='store_true', + global LogToConsole, NoKVM, Branch + parser = argparse.ArgumentParser( description='Mininet VM build script', + epilog=buildFlavorString() + ' ' + + testString() ) + parser.add_argument( '-v', '--verbose', action='store_true', + help='send VM output to console rather than log file' ) + parser.add_argument( '-d', '--depend', action='store_true', + help='install dependencies for this script' ) + parser.add_argument( '-l', '--list', action='store_true', + help='list valid build flavors and tests' ) + parser.add_argument( '-c', '--clean', action='store_true', help='clean up leftover build junk (e.g. qemu-nbd)' ) + parser.add_argument( '-q', '--qcow2', action='store_true', + help='save qcow2 image rather than deleting it' ) + parser.add_argument( '-n', '--nokvm', action='store_true', + help="Don't use kvm - use tcg emulation instead" ) + parser.add_argument( '-i', '--image', metavar='image', default=[], + action='append', + help='Boot and test an existing VM image' ) + parser.add_argument( '-t', '--test', metavar='test', default=[], + action='append', + help='specify a test to run' ) + parser.add_argument( '-b', '--branch', metavar='branch', + help='For an existing VM image, check out and install' + ' this branch before testing' ) parser.add_argument( 'flavor', nargs='*', - help='VM flavor to build (e.g. raring32server)' ) - args = parser.parse_args( argv ) + help='VM flavor(s) to build (e.g. raring32server)' ) + args = parser.parse_args() if args.depend: depend() if args.list: - print 'valid build flavors:', ' '.join( ImageURLBase ) + print buildFlavorString() if args.clean: cleanup() - flavors = args.flavor[ 1: ] - for flavor in flavors: - if flavor not in ImageURLBase: - parser.print_help() + if args.verbose: + LogToConsole = True + if args.nokvm: + NoKVM = True + if args.branch: + Branch = args.branch + for flavor in args.flavor: + if flavor not in isoURLs: + print "Unknown build flavor:", flavor + print buildFlavorString() + break # try: build( flavor ) # except Exception as e: - # print '* BUILD FAILED with exception: ', e + # log( '* BUILD FAILED with exception: ', e ) # exit( 1 ) - if not ( args.depend or args.list or args.clean or flavors ): + for image in args.image: + bootAndRunTests( image, tests=args.test ) + if not ( args.depend or args.list or args.clean or args.flavor + or args.image ): parser.print_help() + if __name__ == '__main__': parseArgs()