Cisco's OTV - Overlay Transport Virtualization

I was taking a look at Cisco's OTV capability they are rolling out on the Nexus gear and my first impression was - wow!

The gist of it is: OTV is a new feature of the Nexus OS operating system that encapsulates Layer 2 Ethernet traffic within IP packets, allowing Ethernet traffic from a local area network (LAN) to be tunneled over an IP network to create a “logical data center” spanning several data centers in different locations. OTV technology will be supported in Cisco’s Nexus 7000 in April 2010, and existing Nexus customers can deploy OTV through a software upgrade.


Cisco says its overlay approach makes OTV easier to implement than using a dark fiber route or MultiProtocol Label Switching (MPLS) over IP to move workloads between facilities.

“Moving workloads between data centers has typically involved complex and time-consuming network design and configurations,” said Ben Matheson, senior director, global partner marketing, VMware. “VMware VMotion can now leverage Cisco OTV to easily and cost-effectively move data center workloads across long distances, providing customers with resource flexibility and workload portability that span across geographically dispersed data centers.

“This represents a significant advancement for virtualized environments by simplifying and accelerating long-distance workload migrations,” Matheson added.

My opinion on why this is important centers around Layer 2. Layer 2 is where peering happens. Peering allows companies to move data around without paying for it. Bandwidth providers agree to pass traffic from one network to another via a cross connection between the two networks. Instead of buying a point to point OC-192 between data centers, a company would colocate IT gear in a data center on a peering point, buy a port on a peering Exchange like Any2, and cross connect to other networks who can move traffic around on other networks at Gig and 10 Gig (and up) speeds. The connections are in Layer 2.

A pertinent example would go like this:

An online gaming company colocates 50 cabinets in One Wilshire or 900 N. Alameda. They buy a port on the Any2 Exchange and set up 5 cross connections to different networks who peer at One Wilshire and 900 N. Alameda - Level 3, AboveNet, Tata, Vodaphone, and NTT as an example. As they expand into their global footprint, they can move VM's - workloads and game play - around from one data center to another using the cross connects, and not have to have large pipes, point to point, from one facility to another.

Another example would be a US based space agency I have done some work with, has containers that house a cloud offering (OS) in them. One of their satellites takes 100 pictures of the rigs of Saturn one morning and needs to distribute those massive images to thousands of constituents worldwide. In the past they may have purchased multiple 10 Gig pipes from their Center to a handful of hubs they interconnect with. Big money for big bandwidth which they need. Using this OTV technology, they buy a fat pipe from their Center to 55 S. Market in San Jose (3 miles or so), buy a port on an exchange that peers there, and now they can move those photos, videos, etc. to their other hubs who use the same peering exchange and not have to pay for the bandwidth between 55 S. Market in San Jose, CA and say Chicago - 2167 miles. This pays for the deployment of other containers where there is 100% green power that is cheap, can use peering to expand the footprint's network, and if a better spot becomes available, they move the container after offloading the workload to another container or two.

This is one of those game changing technologies in how people can deliver compute to customers. For large scale deployments, especially those that must be green and use wind generated power or nuclear generated power, this is a huge advantage. You know have the ability to drive physical and virtual movement of workload based on criteria other than - is there (do we have) a data center there.

I will be watching this solution closely.