CloudPundit: Massive-Scale Computing
the business of Internet infrastructure, cloud computing, and data centers
Updated: 49 min 7 sec ago
Investors are asking, probably reasonably, what’s going on here. Is the overall market for cloud computing weakening? Are competitors taking more market share? Or is this largely the result of the price cuts that went into effect at the beginning of 2Q? And if it’s the price cuts, are these price cuts temporary, or are they part of a larger trend that eventually drives the price to zero?
The TL;DR version: No, cloud growth remains tremendous. No, AWS’s market share likely continues to grow despite the fact that they’re already the dominant player. Yes, this is a result of the price cuts. No, the price cuts are permanent, and yes, cuts will eventually likely drive prices down to near-cost, but this is nevertheless not a commodity market.
The deep dive follows. Note that when I use the term “market share”, I do not mean revenue; I mean revenue-generating capacity-in-use, which controls for the fact that prices vary significantly across providers.
What’s with these price drops?
I have said repeatedly in the past that the dynamics of the cloud IaaS market (which is increasingly also convergent with the high-control PaaS market) are fundamentally those of a software market. It is a market in which customers are ultimately buying IT operations management software as a service, and as they go up-stack, middleware as a service. Yes, they are also getting the underlying compute, storage, and networking resources, but the major value is the actually in all the software that automates this stuff and delivers it as an easy-to-consume, less-effort-to-manage solution. The automation reduces a customer’s labor costs, which is where the customer sees cost-savings in services over doing it themselves.
You’ll note that in other SaaS markets, the infrastructure is effectively delivered at cost. That’s extremely likely to be true of the IaaS market over time, as well. Furthermore, like other software markets, the largest vendors with the greatest breadth and depth of capabilities are the winners, and they’re made extra-sticky by their ecosystems. (Think Oracle.) Over time, the IaaS providers will make most of their margin off higher-level services rather than the raw resources; think about the difference between what a customer pays for Redshift data warehousing, versus raw EC2 compute plus EBS storage.
The magnitude of the price drops is scaring away many rival providers. So is the pace of innovation. Few competitors have the deep pockets or willpower to pour money into engineering and data centers in order to compete at this level. Most are now scurrying to stake out a niche of the market where they believe they can differentiate.
Lowering the price expands the addressable market, as well. The cheaper it is to do it in the cloud, the more difficult it is to make a business case to do an on-premises solution, especially a private cloud. Many of Gartner’s clients tell us that even if they have a financially viable case to build a private cloud right now, their costs will be essentially static over the amortization period of 3 to 5 years — versus their expectation that the major IaaS providers will drop prices 30% every year. Time-to-value for private cloud is generally 18 to 24 months, and it typically delivers a much more limited set of features, especially where developer enablement is concerned. It’s tough for internal IT to compete, especially when the major IT vendors aren’t delivering software that allows IT to create equivalent capabilities at the speed of an AWS, Microsoft, or Google.
The size of the price cut certainly had a negative impact on AWS’s revenues this past quarter. The price cut is likely larger than AWS would have done without pressure from Google. At the same time, the slight dip in revenue, versus the magnitude of the cuts, makes it clear that AWS is still growing at a staggeringly fast pace.
The Impact of Microsoft Azure
Microsoft has certainly been on an aggressive tear in the market over the last year, especially since September 2013, and Azure has been growing impressively. Microsoft has been extremely generous with both discounts and with free credits, which has helped fuel the growth of Azure usage. But interestingly, Microsoft’s proactive evangelism to their customers has helped expand the market, particularly because Microsoft has been good at convincing mainstream adopters that they’re actually late adopters, spurring companies to action. Microsoft’s comprehensive hybrid story, which spans applications and platforms as well as infrastructure, is highly attractive to many companies, drawing them towards the cloud in general.
Some customers who have been pitched Azure will actually go to Azure. But for many, this is a trigger to look at several providers. It’s not unusual for customers to then choose AWS over Azure as the primary provider, due to AWS’s better feature set, greater maturity, and better ecosystem; those customers will probably also do things in Azure, although they may be earlier-stage and smaller projects. Of course, many existing AWS customers are also adding Azure as a secondary provider. But it may well be that Microsoft’s serious entry into this market has actually helped AWS, in terms of absolute usage gains, more than it has hurt it — for the time being, of course. Over time, Microsoft’s share gains will come at AWS’s expense.
The Impact of Google
Google has been delivering technology at an impressive pace, but there is an enormous gulf between its technology capabilities and its go-to-market prowess. They also have to overcome the perception problem that people aren’t sure if Google is serious about this market, and are therefore reluctant to commit to the platform.
While everyone is super-interested in what Google is doing, at the moment they do not seem to be winning significant customers from AWS. They’re doing reasonably well in batch-computing scenarios (more as a rival to AWS spot instances), and in scenarios where all of Google Cloud Platform, including App Engine, is of interest. But they do not seem to have really gotten market momentum, yet.
However, in this market and in many other markets, a competitor does not need to win over the market leader in order to hurt them. They merely need to change customer expectations of what the price should be — and AWS has allowed Google to set the price. As long as Google seems like a credible threat, AWS will likely continue to be competitive on price, at least for those things that Google also offers. Microsoft has already publicly pledged to be price-competitive with AWS, so that pretty much guarantees that the three providers will move in lockstep on pricing.
The Impact of Smaller Providers
Digital Ocean, in particular, is making a lot of noise in the market. They’ve clearly established themselves as a VPS provider to watch. (VPS: virtual private server, a form of mass-market hosting.) However, my expectation is that Digital Ocean’s growth comes primarily at the expense of providers like Rackspace and Media Temple (owned by GoDaddy), rather than at the expense of AWS et.al. A Digital Ocean droplet (VM) is half the price of an AWS t2.micro (the cheapest thing you can get from AWS), and they’ve been generous with free-service coupons.
VPS providers have friendly control panels, services that are simple and thus easy to use, and super-low prices; they tend to have a huge customer base, but each customer is, on the average, tiny (usually just a single VM). These days, VPS is looking more and more like cloud IaaS, but the orientation of the providers tends to be quite different, both from a technology and a go-to-market perspective. There’s increasing fluidity between the VPS and cloud IaaS market, but the impact on AWS is almost certainly minimal. (I imagine that some not-insignificant percentage of AWS’s free tier would be on VPS instead if not for the free service, though.)
There’s plenty of other noise out there. IBM is aggressively pitching SoftLayer to its customer base, but the deals I’ve seen have generally been bare metal on long-term contracts, usually as a replatform of an IBM data center outsourcing deal. Rackspace’s return to its managed hosting roots is revitalizing. CenturyLink continues to have persuasive sales. VMware customers remain curious about vCHS. And so on. But none of these providers are growing the way that AWS and Microsoft Azure are, and both providers are gaining overall market share at the expense of pretty much everyone else.
The sky is not falling
With Microsoft and Google apparently now serious about this market, AWS finally has credible competitors. Having aggressive, innovative rivals are helping to push this market forward even faster, to the detriment of most other providers in the market, as well as the IT vendors selling do-it-yourself on-premises private cloud. AWS is likely to continue to dominate this market for years, but the market direction is no longer as thoroughly in its control.
Gartner’s Magic Quadrant for Cloud Infrastructure as a Service, 2014, has just been released (see the client-only interactive version, or the free reprint). If you’re a Gartner client, you can also view the related charts, which summarize the offerings, features, and data center locations in a convenient table format. (The charts are unfortunately less readable than they could be, as our publication system doesn’t allow comments in Excel spreadsheets. Sorry.)
We’re continuing to update this Magic Quadrant every nine months, since the market is moving so quickly. There have been significant changes in vendor positions since the August 2013 Magic Quadrant (the free reprint has expired, but the graphic is floating around, and Gartner clients can use the “History” tab in the online Magic Quadrant tool, which allows you to compare 2012, 2013, and 2014 interactively).
We’ve observed, over the last nine months, a major shift in Gartner’s client base — the desire to make strategic bets on cloud IaaS providers. In general, this reduces the number of significant suppliers to an organization to just one or two (whereas many organizations had as many as four), with the overwhelming bulk of the workloads going to one provider. It also means that clients are interested in knowing not just who is winning right now, but who is going to be the winner in five or even ten years. That’s really the lens that this Magic Quadrant should be viewed through: Who has what it takes to convince the customer that they can serve both current needs and will sustain market leadership over the long term?
Our clients have, since September of 2013 (which seemed to mark a change in Microsoft’s go-to-market approach for Azure), consistently viewed this as an AWS vs. Microsoft battle, with AWS continuing to win the vast majority of business but Microsoft winning significant inroads, especially with later-adopter customers. In recent weeks since the big price drops, lots of clients have been asking about the future of Google, as well, and there are a lot of curiosity questions about IBM (SoftLayer) also, although the IBM questions tend to be more outsourcing and broader-strategy in orientation. Of course, prospects consider other vendors, especially their existing incumbent vendors, as well, but AWS and Microsoft are overwhelmingly the top contenders.
What’s interesting about this year’s Visionaries is that they all have new platforms — CenturyLink with the Tier 3 acquisition, CSC with the ServiceMesh acquisition coupled with the AWS partnership, Google with Google Compute Engine, IBM with the SoftLayer acquisition, and Verizon Terremark with the still-beta Verizon Cloud. (Arguably VMware falls into this bucket as well, despite being a Niche Player this year.) These providers are in the middle of reinventing themselves, most with the idea of battling it out for the #3 spot in the market.
This is not a market for the faint of heart. (I recently asked a large vendor if they intended to compete seriously in the IaaS space, and was told, “Only an idiot takes on Amazon, Microsoft, and Google simultaneously.”) For that matter, this is not a market for the shallow of pocket. You can’t spend your way to success here, but you need engineers, intellectual property, and to be a real #3, substantial capital investment in infrastructure.
There’s also a clear convergence with the PaaS market that’s taking place here. AWS has long offered an array of services that are PaaS elements, as well as many things that sit on the spectrum between pure IaaS and pure PaaS. Microsoft and Google started as PaaS providers and then launched IaaS offerings. The distintions will blur and increasingly become less relevant, as providers fight it out on features and capabilities.
Gartner continues to separate our evaluation of related managed and professional services from the core cloud IaaS platform, because we believe that clients are increasingly choosing a platform, and then choosing consultants and managed services providers (or alternatively, turning to a trusted integrator who helps them choose the right platforms for their needs). I’ll be writing on this more in the future, but keep an eye out for the upcoming regional Magic Quadrants for Cloud-Enabled Managed Hosting for a managed services-oriented view.
When I wrote a research note called Don’t Let OpenStack Hype Distort Your Selection of a Cloud Management Platform in 2012, in September 2012, I took quite a bit of flak in public for my statements about OpenStack’s maturity. At the time, I felt that the industry was about 18 to 24 months from the point where real commercial adoption of OpenStack would begin. It now looks like I made the right call — 20 months have passed since I wrote that note, and indeed, OpenStack seems to be on the cusp of that tipping point. OpenStack is truly becoming a business. Last year’s Portland summit was a developer summit. This year’s summit has the feel of a trade show, although of course it’s still a set of working meetings as well as a user conference.
There’s much work to be done still, but things are grinding onwards in an encouraging fashion. The will to solve the common problems of installs, upgrades, and networking seems to have permeated the community sufficiently that these basic elements of usability and stability are getting into the core. The involvement of larger vendors has created a collective determination to do what it takes to make enterprise adoption of OpenStack possible, in due time.
In March of this year, I wrote a new document called An Overview of OpenStack, 2014. It contains the updated Gartner positions on OpenStack — along with practical information for users, like use cases, vendors, and how to select a distro. (No vendor has done a free reprint of the note, so it’s behind the paywall, sorry.) I have no updates to that position after this summit; it has been largely what I expected it to be. However, I did want to comment on what I see as one of the key questions now facing the OpenStack Foundation and contributing vendors.
One of the positions taken in my recent note is a re-iteration of a 2012 position — we believe that OpenStack “will eventually mature into a solid open-source core at the heart of multiple commercial products and services.” One of the key questions that seems to be at hand now is how large that core should be — a fundamental controversy for OpenStack Foundation members, each of whom has a position based on where their company adds value.
At one pole of the spectrum are the vendors who want to maximize the capabilities in OpenStack that are fully open-source — I’ll call them the “more open” camp. (End-users, of course, also all want this.) These vendors typically differentiate in some way that is not the software itself. They do consulting, they are managed services providers, they are cloud IaaS providers, or they are selling some kind of product or service that uses OpenStack under the covers but delivers some other kind of value (NFV, SaaS, and so on). They want the maximum capabilities delivered in the software, and they’re willing to contribute their own work towards this end.
At the other pole of the spectrum are the vendors who intend to sell a cloud management platform (CMP) and need to be able to differentiate — I’ll call them the “more proprietary” camp. That means that there’s the question of “how does a distro differentiate”. It has already been previously argued that installation and upgrades should be left to commercial distributions. At long last it seems to be agreed that for the good of the community at least some of these capabilities need to be decent in the core. The next controversial one seems to be an HA control plane. But it also gets into the broader question of how deep the functionality of OpenStack as a whole should go. Vendors that sell OpenStack software really fall into two broad categories — those that intend to supportively wrap what is essentially vanilla OpenStack (like the Linux vendors), and those who are building a full-fledged CMP (or CMP suite) into which OpenStack may essentially disappear near-invisibly (except for maybe an exposed API), surrounded by a rich fudgy layer of proprietary software (like HP and IBM). Most of these vendors want just enough in the OpenStack open-source to make OpenStack overall successful.
There are nuances here, of course, and many vendors fall somewhere between these two poles, but I think that summarizes the two camps pretty well. Each camp has its own beliefs about what is best for their own companies and what is best for OpenStack. These are legitimate debates about what is “just enough” functionality in OpenStack (and how that “just enough” changes over time), even amongst vendors who occupy the “more proprietary” camp — and whether that “just enough” is sufficient to satisfy the “more open” camp. Indeed, the “more open” camp may find that they cannot get their contributions accepted because the “more proprietary” camp is gatekeeping.
It is critical to note that no vendor I’ve ever spoken to thinks that OpenStack interoperability means that you should be able to easily switch between distributions or OpenStack-based service providers. Rather, the desire is to ensure that there’s enough of an interoperability construct that there can be a viable OpenStack ecosystem — it’s about the ability of ecosystem vendors to interoperate with a variety of OpenStack-based vendors, far more than it is about the user’s ability to interoperate between OpenStack-based solutions. To reiterate another point from my previous research notes: Customers should expect to be no less locked into an OpenStack-based vendor/provider than they would into any other CMP or cloud IaaS provider.