modular models

I have spoken to a number of users in private enterprise and government over the past 3 years who had expressed interest in looking at modular data center solutions. The three main reasons that these discussions never culminated in a modular model of deployment were that the solutions were uproven, they were a niche play, and there was no third party who had the skillsets required to put a deal together that included site selection, site prep, purchase of site, modular units, and ongoing operation. A close fourth was that there was no available sales data from the manufacturers.

The 'niche' has changed a lot and I personally have seen hundreds in operation - all at single user sites. These single users can tap into their corporate real estate teams, their facilities teams, and their IT organizations to figure it out.

• What about the broader market/multi tenant marketplace? Where's the beef?
• Where do companies turn to get real estate expert  who knows how to source land for a data center, talk electricity voltages and network topography, dabbles in construction and site prep, and has a track record with data center operations?
• Who can actually do an assessment of the vendors and see who knows their shit, and who is just spinning the latest marketing brochure?

I know many real estate professionals and I work with a handful that understand data centers, I know and work with electrical contractors and manufacturers who excel at that piece, and have walked through several manufacturers solutions from around the world that are deployed at customer sites, and being a data center operator I understand the nuances of running a facility. So is it a matter of everyone wants to just focus on their ingredient vs. be the chef, or is it something else?

I love that the space is maturing. It truly is about time.

By my own research, modular data centers and containers (yes they are different), cost 50-60% less than a traditional build. They also cost 50% less to operate. I have the numbers to prove this out. So the industry just got half as expensive to get into, it's half the cost to operate. I have watched an announcement from SKANSKA with deep intrigue, because it is a BIG signal to me that they get it, and the industry is in fact changing.

The announcement - I saw it at Data Center Knowledge - is important because it means a builder gets it, and they know what I know - the writing is on the wall for traditional data center builds because of cost if nothing else. 'The modular design will serve as the nucleus for an ambitious plan for build a series of colocation facilities around the United States, which Skanska will own and operate'. So here we have a construction company getting into the data center business because they know that they can deliver - and operate - a facility for HALF the cost. Looks like Nepholo has figured it out too.

Pay attention folks, the industry just blinked...

Maturity in the Container/modular space

2011 was an interesting year for the container data center space. There were several new entrants (modular), a maturing in the industry of where they fit into the grand scheme of data centers and an evolution by the existing manufacturers into the next generations of their initial offerings.

I will list the vendors that I know about and have met with and personally verified that their offering is in fact the real deal. This is a list and not a ranking by the way. If you want my personal professional opinion on this then track me down off the record:

CirraScale (f/k/a/ Verari)
HP
Dell
Blade Room
I/O

SmartCube
Elliptical Mobile Solutions (EMS)
SGI/Rackable

I am sure I will get calls from the vendors who will read the post and make 'corrections' to my positioning and commentary but I am vendor NEUTRAL and I have met with, seen, and in some cases sold one of these so my experiences and comments are based on seeing, smelling, and experiencing the products and the people who work for the companies who make them. Everyone has their own secret sauce, I get it. I also get that I am one of the few people on the planet who has not worked for a vendor who has any longevity in covering and paying attention to the space, so my comments are solely mine, as are my opinions.

The overall maturity in the space has gone from containers, to a true modular design, with the biggest difference being that modular units are the needed hybrid between the container and the traditional stick built data center. Essentially they are a data center built in a factory, vs. a data center retrofitted on site.

The bigger issue in adoption of this class of data center solution, I believe, is that companies have no single organization to turn to for evaluating if their needs are better served by a container, a modular design, or a traditional data center. The skillsets needed to evaluate the solutions require IT, facilities, real estate, electrical systems, cooling, site prep/construction, permitting, and finance skills. Since many large customers have people with these skillsets it is arguably easier to tap into the knowledge base, however politics can derail things quickly. That and an IBM container salesperson does not get paid to tout the HP Ecopod's features and benefits so it is vendor lock from the get go and that scares the sh*t out of most CTO/CIO's. The don't run just ___________ gear.

Let's get real for a moment - if the vendors wanted to eat their own dog food (or wear their own outfits), they would. HP would never build another data center, nor would IBM, I/O, or anyone else with the offering of the physical and operational components under one roof. They would be consolidating in droves out of their facilities into containers and that would would be a clear and consistent path forward. It would also be the smartest thing financially they could do.

The only issue is that none of their larger customers runs just that vendor's gear to support their business so it's a religious sell to get someone to convert, or you're taking an oval peg and jamming it in a round hole.

I have spent a fair amount of time looking hard at modular solutions - very different from a container- since it is close to a traditional data center by accepting 19 or 24 inch racks of ANY manufacturers hardware that you would put in a stick built facility, the biggest difference is that the data center is built in a factory so there is nothing to tour ahead of time in most cases. For some people this is a deal breaker, for others it doesn't matter since the absolute and measurable consistency of product, cost, and financial aspects trump the inability to walk through something.

As for the financial aspects of all of these NSB (non-stick-built) solutions, a modular data center is TOUGH to beat. In one recent study I saw the cost of a retrofit was ~$22M per megawatt. This is 4x the capital cost of the same power footprint of a modular solution. 400% and that is not a typo. On the OpEx (power/cooling/operation) side the modular designs deliver a PUE of 1.2 consistently. Knowing what I know about air flow and pimping the data center floor with different efficiency gaining tricks of the trade, I would think you get lower. Contrast that with a 2.0-3.0 PUE in traditional and older facilities you can cut your costs in half.

One of the most compelling examples I have thought about was based on a 20 megawatt proposed project. That's 20,000 Kw - plus a greenfield build cost of ~$2,000/foot. That's likely a 180,000 square foot size building (conservatively). So using back of the napkin math, that's roughly a $360 million dollar project that has to be COMMITTED to up front. Yes it would be built and financed in phases, but who wants a $360M obligation on their financial statement today. Especially when 20MW of a modular solution can be delivered for roughly $160M. No - not a typo. That is $200M less off the top and out of the gate. Operationally it's 14.2M in electricity (in North Carolina - a marketed cheap power state) for a 2.0 PUE facility vs. $8.5M/year for a 1.2 PUE modular. Thats $6M/year less. So dollars in over a 10 year period is $500M vs. $245M. I am no mathemetician but that's a big difference.

If you would like to discuss in any detail - ping me. I have sold a container to NASA, consulted on RFI and RFP documents for intelligence and military applications, and am truly vendor neutral. I am  running out of reasons to believe that the modular solutions are anything BUT the way to deliver data centers. Technically, financially, and environmentally. I love this evolution.

Others are taking note as referenced by a study Digital Realty Trust paid for to take the temperature of the data center growth patterns. The interesting nugget was that 41% of companies surveyed were looking at containers and/or modular solutions. Smart move.

email

Why are containerized modular data centers the new Black now?

What took you so long?

I was at Data Center Dynamics in Wasington DC on Tuesday and you may have thought it was Modular Dynamics instead. It seemed as if the data center world finally came around to what I have been seeing, studying, deploying, and writing about since 2008. That's not to infer that the modular solutions have gone mainstream yet, but the conceptualization has matured a lot as expected.

HP had a scale model of the EcoPod which was cool to see since up to that point I had only seen Powerpoint slides about it and have yet to see the real thing. PDI was showing off their modular solution as well which will be watched closely as they have some patent infringement allegations they are fending off. ActivePower had their PowerHouse scale model on display as well and given the coziness between HP and ActivePower in recently announced deployments I could easily see them sharing a booth at future trade events.

Bytegrid was the only data center owner operator who was able to speak to containers at the event - and did openly talk to several people about the good the bad and the stuff that trips you up on deployment and had specifics to back up the discussions. The three other owner operators there - GigaPark, QTS, and Powerloft weren't talking about supporting modular solutions.

Based on what I saw, acceptance has increased dramatically, demand is rising right along with it, and the Achilles heel that was there three years ago is there today still - being able to answer 'So where can I put one of these things?' with something other than 'Wherever you want'. ByteGrid is on to something and will be watched.

Why Cisco needs to buy Verari, and why Dell IBM, Rackable, and HP will fight for it

I have been covering the Verari story for the past several weeks, talking with folks close to the situation every other day. I keep coming back to Why Cisco Needs to buy Verari:

1. Protect IP
2. Support the brand
3. Have a container play in the market
4. Maintain a level of credibility in a market they said they were entering - servers

The other reason they should make an offer is because they have a financial backer who will do the deal with them and pony up 50-60% of the cash.

So why would Cisco want to protect the IP? The technology is solid, Verari has customers, and if they don't Cisco must now go after another hardware manufacturer to secure solid (and deployed) server technology to make good on its announcement that it was going after HP's server business to get market share inside the data center.

There is also being in a position to address what a data center is since the move is clearly going towards modularity and container deployments in very key areas. If Cisco doesn't have a story, their competitors craft deals so Cisco can't play. Ouch.

Support the brand - Verari, when it comes out of the ashes, will be recapitalized with Carlyle, Citibank, Sierra Ventures and others out of the picture. In other words the company has a running head start in that it has customers, a pipeline, a brand, production, the whole nine yards supporting the brand. Cisco believed in it enough to do an OEM deal, now they get to put a Verari server in a pillow case for a data center technology beat down of the competition.

Have a container play in the market. For people who follow this blog and colleagues in the data center industry know I have been a strong proponent of containerized data centers for about two years - since before there was a lot of buzz about them anyway. Having a container play is absolutely crucial - why? Because companies who offer high performance computing, serve civilian and defense agencies, are in the oil and gas business, and cloud computing companies are all looking at and/or purchasing them. There was a question on Linked In a few weeks ago - 'why havent we heard about all these container deployments?' and there are a couple of answers both of which point back to 'we could tell you but then we'd have to kill you'. People don't want their strategic advantage broadcast to the world. Nor does the military. So containers are crucial - not for companies who need a couple of cabinets, but for companies who need hundreds of cabinets - containers are cheaper and more secure hands down.

Maintain a level of credibility - this one is crucial. I cannot and do not see companies today making large announcements that create seismic events in certain markets, pull back after a few months and say - Ha! Just kidding!. Cisco could buy Sun's server business and let Oracle ravage the software side, and that's about it. There are slim pickens for proven server technology out there where ther companies don't already have a relationship with IBM, Dell, and HP - Cisco's partners and in many circles now - the enemy.

Cisco solves a lot of problems for short money with this deal. On the flip side Cisco's competitors can create a lot of problems for Cisco for short money - real and in the press - by buying Verari. They keep Cisco out of the Server Hardware Club for a few more months, which is a lifetime in this business. and with the tech refreshes that need to happen in the next 12-24 months, a misstep or delay means a few hundred million are at stake.

This one will be fun to watch...

I thought a niche was a small market...

Containerized data centers remain niche players according to IBM...

I got the heads up on this article from my Google Alerts and it was a head scratcher. IBM, who makes containers, was saying that containerized data centers are still a niche play. Steve Sams, VP of site and facilities for IBM, said containerized data centers are not for everyone and won't be for a long time.

"Most major companies conclude that data center containers are not for them, and I think they're right," he said. "I never think it's going to be a huge market."

Is IBM trying to sell these, or is this a ploy a la Sun Microsystems to sell more servers in a bigger box?

He goes on to say - "But Sams said that in general, he doesn't think containers are "the best way to build 320 square feet of data center."

"Ninety-nine percent of our customers are never going to install thousands of servers at a time," he said.

Someone in marketing at IBM may be to differ. I know I do.

Why would an organization NOT want to spend less money on delivering its IT resources? I can't name one company I have spoken to in the past 18 months that says 'We don't care what it costs, we'll do things the most expensive and inefficient way possible.'

The biggest issue I see with container adoption is that there are few places to plug and play. Microsoft and Google and Amazon are single tenant operations with custom built facilities to support their compute and storage infrastructure. Ninety nine percent of companies who buy IT gear don't have their budgets and their expertise to do this on their own.

Couple the budget issue with the fact that traditional data center operators do not have capital to build on spec, and when the container manufacturers are touting that customers can have 4000 square feet of data center deployable - including servers, OS, and racks - in 8 weeks, data center operators can't build out their infrstaructure fast enough to give container buyers a place to land them.

So a company must then navigate facilities issues, zoning and permitting, siting, redundancy, in short - it's a new data center project for their data center in a box.

I personally believe that when there is a vendor agnostic place to plug them in, it will increase data center container sales. Who in their right mind wouldn't want a PUE of les than 1.3, breathtaking capacity ordered, installed, tested, and delivered in 8 weeks, a quickly depreciable asset, and one that can be leased 100% with tech refreshes built in?

IBM must have quite a list if only 1% of their cutomers thinks that greener, more efficient, highly dense, and fiscally responsible solution is a way to go.

Containers/PODS/Data Center in a box - the ultimate hedge?

There has been a ton of stuff going on in the world of technology this summer and I thought it was about time I shared the sum of my discussions this summer and see what other folks are thinking about.

I just spoke to a good friend of mine who is headed to work at Stratus Technologies - the ultra high availability server company and we were chatting about the electricity issues companies are faced with as they virtualize their infrastructure.

Cabinet draws increase when you virtualize - especially with blades - which means electricity USE increases as well. If you use and need more of anything than what you have, you will have a shortage unless you get more. Duh. You can have a 100,000 square foot facility but if it has 5MW available and your new virtualized design requires 10MW, even though you can now put the entire data center into 100 cabinets - doesn't matter the size of the glass my friend - when you take the last sip you take the last sip.

Other topics that have been fun to watch...

Markey-Waxman bill HR2454 that got passed in the House and is up for debate in the Senate next session. Talking about taxing carbon footprints of companies. I am sure taxing carbon profiles of humans is around the corner - especially when they see the tab for Social Security, but I digress...

The Waxman-Markey bill is 1400 pages. I have read a fraction of it. It lays out a cap and trade program whereby companies get an allocation of carbon they can emit, and anything over that they pay a HUGE tax on, or go out to a soon to be created public market (Goldman Sachs is the lead developer) and buy other companies' carbon surplus.

So if I am a data center operator and I like coal and diesel generated electricity because it is reliable and cheap but dirtier than a mud wrestling prostitute, then I will get a hefty tax bill because I emit more carbon than the government thinks I should. OR if I can find an organic, wind, methane, gerbil wheel generating source of electricity data center that emits a tablespoon of carbon each year, I can cut a deal with them to buy their surplus of carbon allowance and keep right on chugging with my diesel and coal.

This, like many other government programs is rearranging deck chairs on the Titanic - there is no real change to the outcome, but we can feel good about appearing to have addressed the outcome.

The other side of this is the cap program. Reduce your carbon footprint period. No trading credits/surplus/etc. reduce your emissions or we will tax you out of business. This has legs, in my opinion and speaks to what is good business, will drive innovation, and help the East Coast and Europeans have cleaner air. If we can get China and India to help out - sweet!

The other discussion I had more than a few times was around data center inventory and the lack of it. It has been leading to higher prices in a down economy, the fact that there is little inventory in the wholesale space - 20,000 square feet and up, and that there will be a tightness felt over the next few years until debt markets can open up, get financing flowing for new projects and space becomes available.

The last discussion area that has heated up even more in the past 60 days has been around containers. There is a LOT more activity out there as people come to realize how slick this solution is, and why it's a hedge against carbon tax (efficiency goes up with power consumption), you can get them quick, so when budgets open up, you can wait to buy as long as possible and get another few dozen petabytes of storage or a few thousand cores up and running in 2 months, and if you pl,ug them into green power and grey water - well, now you're talking extra credit.

The one thing that I am watching closely is based on a question I heard on a recent call - 'Whay are these container deployments fire drills every time?' to which I responded, 'Because they can be deployed faster than a traditional data center EVERY time. Customers know this, it's the data center companies that need to step it up and have inventory ready or the process in place to deploy it more quickly than they are used to.

There is also the issue of zoning too - cities and towns don't know what to make of the containers, especially the ones outside. They see them as 'occupied temporary structures'. To me that definition is a refrigerator box with a schizophrenic uncle and his three imaginary friends in it, not a standard container that there are several thousand of in Newark and Long Beach. Aside from the fact that they are 'occupied' maybe 2 days a year, if that.

Anyway, containers are getting much more street cred and are the ultimate hedge against taxes, portability, density, with the upside of efficiency gains and fast depreciation.

Data Center Containers - a black licorice product?

I sincerely hope Tier 1 does a Data Center Transformation Summit twice a year. The one I was at yesterday was excellent and was topped off by dinner with the rock stars over at Horizon Data Center Solutions in Reston, VA. Then this morning our press release hit the wires which was great to wake up to.

One theme I saw again and again was people either 'get' containers or they dont. Chris Crosby over at Digital Realty Trust doesn't like them. Nor does the CTO at Equinix, Dave Pickut. Chris had a presentation that was full of inconsistencies that I will attribute to DLR being a REIT vs. a data center owner/operator. REITs view the container as competition - it's not real estate so they're not interested because they can't sell it. Given Mike Manos' move over there and his work in deploying containers for Microsoft makes me wonder if DLR will shed its REIT status and broaden its offerings. Nah. A REIT is a REIT and they have a lot of customers who rely on them for space.


Equinix (and others) don't like them, I believe, for two reasons -

1. They house high density equipment, up to 27 Kw per cabinet. The standard is 4.8 Kw, which means you need to allocate more floor space for air flow and that's floor space you cant sell to the 4.8 Kw cabinet customers.

2. The data centers they are in can't support them from a deployment perspective. Something new they can't support.

Here is why I believe they cannot ignore the data center container (DCC) market - the efficiency is unprecedented for the draw, the footprint, and the cost. You will save $6,000 per cabinet per year putting it in a container on the PUE gains alone (Power Utilization Efficiency). When all you need is a 600 AMP feed and a 4" water pipe connected to a water source of 65 degress, you're done.

So what's holding the proliferation back?

Basically 18 months ago before the economy really tanked, companies refreshed a lot of servers and other IT gear. They bought a lot of VMWare, and stood up a lot of new stuff. New stuff that was leased. On a 36 month lease. We are 18 or so months into a lease cycle for a lot of companies that have their hands tied and can't move to a DCC if they wanted to.

So what I see starting to happen is that companies are giving DCCs a look because bonuses are being tied to reducing carbon emissions, consuption, and doing more with renewable energy sources. DCC's will give them a healthy bonus. That tied with being able to lease extreme density gear self contained and from PO to plug in of 10 weeks tops - look out.

So right now the DCC's are a black licorice product - you either love em or you hate 'em. I believe more people will be loving them in the next 18-24 months because it's green ($) for them to do it.

A Penny Per GB of Storage

One of the comments I got back from Verari was that their solution is capable of 13 Petabytes of storage, going well beyond the 600 TB I used as a benchmark.

So since a single Petabyte is 1024 TB, you basically double the storage for the sake of math and keeping it in line with my earlier benchmark of 600TB:

Petabyte = 1024 Terabytes

The new computations then look like this:

At ~$200 KW, you’re looking at $120,000 month in base rent for a 600kw solution ($200*600)

So for 13.3M gigabytes you’ll pay about 130,000 per month (keeping numbers round) which means you’re at one cent per gig on cost. And more efficient and that is with the drives fired up 100% of the time.

Microsoft needs to implement these to archive email at these prices... Or Live Apps data.

Mark

600 Trillion Bytes per box - ROI of Pod Container Storage

I read recently that the IRS stores 15 to 20 Terabytes of data for each year of tax returns.

If the number is 20 Terabytes, 600/20 = 30 years of tax records storage is possible with a single container/POD. Since they go back 10 years for most practical applications of retrieval this means, at least conceptually, that the IRS and any other entity that stores records electronically (lawyers) will pay for 66% of records they seldom or never need to access for legal, operational, or other reasons.

For other organizations (banks, brokerages) it is 4-7 years where records need to be accessed on a 'regular' basis and after four years the trend falls off a cliff and after 7 it's hello abyss. So let's look at what a company might pay to store all this data and then let's look at what they would save if they thought differently about how they actually implemented their storage both in dollars and in energy efficiency.

1024 Gigabytes (GB) = 1 Terabyte (TB). I will round it to 1000 to keep it simple.
I will also assume that the cost per GB is $.50 - 50 cents US, so cost per TB is $500 USD. So that means in raw Capex you're at $300,000 ($500*600) for a container of raw storage.

Let's look at Opex now. We have power, cooling (subset of power), and space.

Power for a container is 350-600kw. If I use 600 or the max draw/densities I have seen, then 1TB = 1 KW of power for keeping the equipment fired up. At $200/kw or $200/Tb you're at $120,000 per month in base rent. Here is where the containers leave the data centers in the dust - PUE.

A data center PUE (Power Utilization Efficiency) looks at how much additional power is needed to power the cooling and facility in which the computing equipment is stored. Most data centers are at 1.7 for a PUE, meaning for every Kw of power needed to power the computers, an additional .7 Kw is needed for cooling it and ultimately powering the chillers, the A/C, etc.

POD Containers are 1.08 to 1.3. What does this really mean?

That you will save .4 to .6 per Kw in a container per month in power alone.

In real dollars that means per TB of storage, saving .5 Kw (mean average) equates to cutting my Opex by about a 1/3 over a traditional data center FOR THE SAME FUNCTION - storage of data.

So we lop off $40K/ month, which means that over a year we save $480K ($500k to round it). This covers the cost of the storage ($300k) and then some. You can look at it as free storage plus $200K, or a substantial reduction in Opex, or $200K to do other things with.

Bottom line is a POD/container pays for itself in under a year, is FAR more efficient than a data center day to day and that's not only good for our bottom lines, but it's good for our environment, and is responsible computing.

I will take a look at an even smarter solution for older records using MAID - Massive Arrays of Idle Disks.

Want help crunching your numbers? Email me -

mark.macauley@gmail.com