Tuesday, October 18, 2011

The Shape Of The New Storage Market

Part of my role here at EMC is to spend time with what we call the "industry influencers" -- the press, analysts, and the consultants who ostensibly shape public perceptions.

When it comes to EMC's storage business, they collectively seem to be fascinated by one minor market development or another -- mostly it's about a specific technology and specific vendors.  Only a few seem to have an appreciation for the big, secular trends that are driving the transformation of storage technology -- and the vendors that play.

Although EMC doesn't have an official talk track on "the changing shape of the storage industry", I do have a personal one.

And, if you're interested, I'd like to share the back story of how I put all of the day-to-day discussion into a broader framework.

It's Not About Storage, It's About Information I suppose a good starting point is that storage is merely a receptacle for information.  It's the information that matters.

We're all familiar with the bold statements around explosive information growth.  I paint information growth as the inevitable consequence of a shift to an information economy -- one where information (and value-added around information) becomes the prime motivator of value.

Of all the fundamental IT infrastructure technologies, only storage is persistent.  Networks don't retain information, nor do CPUs.  To me, this means that storage growth is on a fundamentally different market growth trajectory than, say, bandwidth or processor speed.  And any big market attracts a lot of attention.

The reason I call this out is simple: way too often people who get close to the topic can lose sight of what this stuff is actually being used for :)

Big Idea #1 -- The Fundamental Storage Technologies Are Changing
At the storage device level, I think most people realize we're in the midst of two fundamental storage media changes.  Tape is rapidly being replaced by disk for backup and archive, and flash is the process of replacing both spinning disks and DRAM-based storage caches.

The key to both transitions is not hardware, but intelligent software that makes extracting the benefit from these new technologies transparent for their users.  In the EMC portfolio, examples would be DataDomain's operating system for backup, and FAST for primary storage.

The way that these technologies are being assembled are changing as well.  Industry standard components and building blocks (think Intel) are now the norm vs. proprietary chips and architectures.  Traditional scale up storage architectures are quickly giving way to scale-out that start smaller and get linearly get bigger and faster.

Storage technology isn't just about isolated arrays anymore, either.  The storage architectural domain now projects upwards into the server and hypervisor (think server-based flash storage and path management) as well as outwards across meaningful distance (think replication and geographic optimization, for example).

Many people have noticed that the classic lines between "what is a storage array" and "what is a server" are starting to blur.  The parts are the same, the roles assigned are becoming largely arbitrary.  Consider, for example, EMC's recent demos of virtual machines being VMotioned to an Isilon array.  Or VMware's "soft" storage array that runs as a virtual machine.

And that's going to continue.  The result?  "Storage" become something that needs to get done in the infrastructure.  And it won't necessarily be done by a well-defined box to point to.

Big Idea #2 -- Storage Integration And Convergence Matters
People love to talk about storage as an isolated topic, but that's getting much harder and harder to do.  Why?  Customers are demanding integration and convergence for all the right reasons -- and in a dizzying number of vectors.

Take the current hot topic of server virtualization integrating with storage, for example.  Or storage integrating with IT management and operations frameworks.  Or storage and security.  Or storage enforcing information management policies.

The importance of tight integration between storage and adjacent IT disciplines is sometimes becoming more important than the storage itself.
Consider application integration.  Is your storage environment smart about SharePoint?  Oracle?  SAP?  Exchange?  Spring?  Can application administrators easily take advantage of specific storage features designed for their world?  EMC can have a rich, standalone discussion with Oracle DBAs, Exchange administrators, SharePoint architects, SAP consultants, mdeia workflow people, big data folks, etc. etc. and be very clear about how we've invested in making their lives easier.

Or, perhaps consider the end case of infrastructure integration -- converged environments such as a Vblock.  Or smaller environments where IT generalists have to do it all, such as with Unisphere.  In these environments, storage largely ceases to be an individual discipline and technology domain, and blends in to become "part of the whole".

The larger and more important the storage environment, the more important that it integrates tightly with the needs of other IT stakeholders: the virtualization team, the networking team, the security team, the application teams, the operations teams and so on.  Or conversely, if there's a small number of IT people trying to manage a large environment, integration and simplicity becomes just as important.

Thinking of storage as a standalone topic largely misses this "touches everything" aspect.  And the importance of integration is only going to increase over time.

Big Idea #3 -- Storage Consumption Models Are Changing
The world is going cloud, and IT is becoming "as a service".  Sometimes those services are generated internally; increasingly these services will be provided externally via service providers.
These user-visible IT services are in turn built on layered services, and storage is coming into quick focus as a candidate to be delivered as a service.

Why?  Everyone uses storage.  Here's your service catalog, here's your consumption model, what would you like?

Larger enterprise will inevitably need to adopt a storage-as-a-service mindset before too long.  And IT service providers are finding willing customers who want the convenience and flexibility of variable external services -- whether it's just storage itself, or wrapped into a higher-level service offering.
EMC storage products and methodologies are already being used to deliver storage as a service -- both for enterprise customers and the newer crop of IT service providers.  And our recent investment in growing a cadre of compatible service providers means that our customers will have even more options to do it themselves, or have someone do it on their behalf.

Don't Miss The Big Picture
I'm like a lot of people out there -- I scour the internet for news about new storage technologies, new vendors doing interesting things with storage, and the like.  As a matter of fact, a lot of people at EMC do the same thing :)

But the difference is that here at EMC we have a few fundamental beliefs in how the storage world is changing.  We've made our bets, with more on the horizon.  To be clear, EMC hasn't stopped investing around these fundamental shifts -- some the coolest stuff is yet to come.

To be successful in the storage business these days, it's clear to me that successful vendors will need more than a few products to cover the major bases.  And larger IT vendors who treat storage as merely a sideline to their other businesses will likely not fare well -- you've got to be prepared to make some pretty big bets and see them through for multiple years.

Unfortunately, the picture isn't especially pretty for smaller vendors in this regard.  I think the storage market has matured to the point where it is going to be exceedingly difficult for a new company with a hot product to make a significant dent in the storage marketplace -- unless they're acquired by a bigger company, that is ...

What does this mean for customers?
It's easy -- and it's hard.  If you step back, I think it's relatively easy to see how the storage world is changing, and how that might affect your day-to-day decisions.  Yes, it's a noisy marketplace with everyone claiming a slightly different point of view, but the big themes are there for all to see.
Doing something about it in your specific environment, well, that can be decidedly more difficult :)

By: Chuck Hollis