I am getting old. Back in my day, if you wanted top notch CPU performance, you had to go with a high-end x86 chip, or, if you had deeper pockets, you could get something exotic, like a PowerPC system. The industry’s dependence on x86 processors appeared to be increasing, not declining.

Ten years ago, Apple joined the x86 club, and this prompted many observers to conclude the era of non-x86 processors in the mass market was over. Just a few years later, they had to eat their words, and yet again, Apple had something to do with it. ARM servers are coming, and they could revitilise the server industry.

Rethinking Processor Design

As the paradigm shifted and mainstream users embraced smartphones and tablets, it quickly became apparent that x86 chips from Intel, AMD, and VIA, simply weren’t up to the task. While x86 was the most prolific instruction set on the planet, it wasn’t a good choice for mobile devices for a number of reasons. In fact, Intel’s instruction set still isn’t a popular choice for mobile processors, although this is starting to change thanks to Intel’s foundry tech lead. In any case, when it comes to this market segment, x86 is not as efficient as other CPU architectures out there, namely processors based on ARM’s 32-bit ARMv7 and 64-bit ARMv8 instruction sets.

Over the past decade, and especially over the past five years, ARM processors have come to dominate the smartphone and tablet landscape, and they had a lot going for them. They offered a lot of performance-per-watt, they were cheap to design, produce and deploy. Big vendors could buy the necessary building blocks and design their own processors based on ARMv7 or ARMv8, adding other components according to their needs (high-speed modems and different GPUs to name a couple).

This led some chip designers to take a somewhat different approach and design their own, custom CPU cores. Qualcomm and Apple led the way, both companies became big players on the mobile System-on-Chip (SoC) market, and their development of sophisticated, custom cores, played an instrumental role in their success. However, custom ARM cores were still used in high-end processors, while all other market segments were covered by standard ARM Cortex CPU cores, like the 32-bit Cortex-A8, A9, A7, and A15, followed up by 64-bit designs like the Cortex-A53, A57, and the new A72 core, which is about to start shipping.

The other prerequisite for ARM’s success was Microsoft’s failure.

Windows only ran on x86 processors, so if Microsoft were to gain a foothold in mobile, it would tip the scales in Intel’s favour. However, by the close of the last decade, it became apparent that Redmond had dropped the ball and ceded this lucrative market to Google and Apple. Speaking of balls, long-time Microsoft CEO Steve Ballmer departed the company a couple of years ago, admitting he and his team failed to recognize the potential of smartphone and tablets. Anyway, it’s not Ballmer’s problem anymore: he has other balls on his mind right now, basketballs to be exact.

ARM processors dominate the mobile landscape, and they’re making inroads into other product categories. Servers are a good place to start.

ARM processors dominate the mobile landscape, and they’re making inroads into other product categories. Servers are a good place to start.

However, mobile is not the first or only market segment to witness a Microsoft failure of epic proportions. The other is the server market. On the face of it, smartphones and data centres don’t have much in common, but from a technological and business perspective, they have some overlap.

Whether you’re designing a smartphone or a server, you need to emphasise similar aspects of your hardware platform, such as power efficiency, good thermals, performance per dollar, and so on. Most importantly, you don’t really need an x86-based processor for smartphones and many types of servers. Thanks to Microsoft’s failures, these market segments aren’t dominated by any flavour of Windows. They rely on UNIX-based operating systems instead: Android, iOS, and various Linux distributions.

Microsoft also attempted to leverage the potential of ARM processors, so it tried developing a version of Windows that would run on ARM hardware, which conveniently brings me to the next Microsoft fail: Windows RT. Microsoft eventually pulled the plug on Windows RT, or “Windows on ARM” as it was originally called. Microsoft’s latest Surface tablets employ x64 processors and standard Windows 10. Microsoft’s Lumia smartphone line (née Nokia Lumia) still uses ARM processors from the House of Qualcomm, but Windows Phone is all but dead as a mainstream smartphone platform.

Servers Don’t Have To Cost An ARM And A Leg

Right now, we have a couple of billion smartphone and tablets in the wild, and the vast majority are based on ARM processors. However, ARM chips aren’t making their way into other market segments. There are just a handful high-volume computing platforms based on ARM that don’t fall into the smartphone and tablet category. Google Chromebooks are probably the best-known example. However, ARM chips are used in heaps of other devices: routers, set-top boxes and smart TVs, smartwatches, some gaming devices, automotive infotainment systems and so on.

What about ARM servers?

This is where it gets tricky. I’ve been hearing talk of ARM servers since 2010, but progress has been slow and limited. ARM’s market share in the server segment remains negligible and the ecosystem remains dominated by x86 Xeon and Opteron parts from Intel and AMD respectively. Since AMD is in a world of trouble on the CPU front, Intel has managed to extend its market share lead in recent years.

ARM servers could end up significantly cheaper than their counterparts based on Intel and AMD x86 chips.

ARM servers could end up significantly cheaper than their counterparts based on Intel and AMD x86 chips.

But why did ARM servers sound like a good idea to begin with?

Money. I could try listing all the geeky points that make ARM a viable alternative to x86 in the server market, but at the end of the day it’s mostly about money, so I will attempt to explain it in a few lines.

  • Price/Performance
  • Data centre workloads are evolving and changing
  • Ability to source processors from various suppliers
  • Use of custom designed chips for various niches
  • ARM chips are more suitable to some infrastructure applications
  • It’s a good way to stick it to Intel and erode its market position (Intel is on the verge of becoming a monopoly in the server space)

We don’t need a huge and expensive Xeon processor for everything. Moreover, using obsolete x86 processors to handle undemanding workloads isn’t a good option due to their power draw. Remember, we are talking about servers, not your MacBook or desktop PC. Servers run around the clock, so every efficiency gain, including relatively small ones, tends to be important. It’s not just about getting a bigger electric bill; data centres have to be cooled and maintained, so processors with a lower Thermal Design Power (TDP) rating are much more valuable to enterprise users than individuals.

Why Use ARM Servers?

So, what sort of enterprise application are ARM processors good for?

Well, ARM expects to get the vast majority of design wins for networking infrastructure applications. Due to their flexibility, small size, efficiency, and low price, ARM processors are a great choice for infrastructure. You can use ARM processors in routers, high-performance storage solutions, and certain types of servers.

However, ARM expects the majority of enterprise growth this decade to come from servers since its other segments are already mature and it has a healthy market share in them. Server workloads are changing as well, and this trend is tied to the growth of cloud services. As a result, servers have to deal with an increasing number of smaller tasks.

Many organisations prefer to keep their options open, so they source hardware from multiple vendors. This is good news for ARM server processors because they could be marketed by a number of different companies. In addition, ARM’s licensing policies and modular approach to processor design can be utilised to design custom processors for specific applications. This is, obviously, something that’s not an option for small companies, but what might happen if big players like Amazon, Facebook or Google start asking for bespoke server processors, designed to excel at one particular application?

As for “sticking it to Intel,” I should note that I don’t mean Intel any harm, and I don’t want to see it fail or be pushed out of various market segments, but at the same time I am concerned that Intel’s domination could end up stifling growth and innovation. More competition should result in lower prices for end-users, and this is what ARM servers are all about.

Multithreading: How Many CPU Cores Is Enough?

Only a decade ago, multicore x86 processors were reserved for high-performance computers and servers, but now you can get quad-core x86 chips in $100 tablets.

In the early days of multicore computing, you still needed big CPU cores to get adequate levels of performance. Lots of software was not able to take advantage of these new processors and their extra cores, so good single-thread performance was vital. Things sure have changed; nowadays, we have octa-core smartphones, quad-core Intel tablets and phones, and 16-core x86 server processors.

There is a good reason for this. Building a multi-core processor makes perfect sense from a technological and financial perspective. It’s a lot easier to distribute the load to a few smaller, more efficient CPU cores than to develop a single, huge core capable of running at high frequencies. The multicore approach ensures superior efficiency and chip yields.

Multithreading like a boss: ARM server processors could sport a lot more physical cores than your average x86 server part.

Multithreading like a boss: ARM server processors could sport a lot more physical cores than your average x86 server part.

ARM has the potential to take the core craze to the next level. ARM CPU cores tend to be smaller than Intel’s so called “big cores” used in server and desktop parts (Intel’s “small core” Atoms are reserved for mobile, although Atom-based server parts are available too). However, this does not mean we will see 128-core or 256-core ARM processors anytime soon, although in theory, they are possible. It depends on how the new crop of ARMv8 server processors handles multithreaded loads. There are some encouraging signs, and chances are that ARM servers will be a good choice for a range of workloads that could benefit from their multicore processors.

Qualcomm’s first server processor has 24 ARMv8 CPU cores, and the chipmaker made it clear that future models will sport even more cores. Remember AMD and its server market woes? Well, the company introduced its long overdue ARM-based Opteron A1100 processor just a couple of weeks ago. Qualcomm made the announcement in October, so both these products will become available over the next few months.

Of course, Intel will not be attending this ARM party, but Qualcomm and AMD are not the only chip outfits working on ARM-based enterprise chips. Chipmakers like Broadcom, Calxeda, Cavium Networks and Huawei HiSilicon have worked on ARM-based server products as well. Nvidia and Samsung, two heavyweights in the SoC and GPU business, also experimented with ARM server parts until a couple of years ago when they decided to halt development. Texas Instruments, Xilinx and Marvell are also exploring ARM server parts.

Some of these companies worked on custom ARM cores, too, but the only non-Apple 64-bit custom ARM core available today is Nvidia’s Denver, which only got a handful of design wins.

What Are ARM Custom Cores?

I know most people can’t be bothered keeping track of all industry niches, including the CPU space, so I think now would be a good time to explain what makes ARM cores different and what custom cores actually are. I will not dissect processors and explain the difference between x86 and ARM instruction sets, but I will outline the differences from a business perspective.

You see, ARM isn’t different just because it uses a different instruction set, although that would make for a quick and geeky explanation, in my opinion, the biggest difference between Intel, AMD and ARM is not the architecture, it’s the business model. Besides, architectures change, new CPU designs are unveiled on a regular basis, but ARM’s approach to marketing and licensing its technology hasn’t changed in years.

Here is a simple example.

An Intel processor is developed by Intel, using Intel instruction sets. It is manufactured in an Intel foundry, packaged and shipped with “Intel Inside” branding. It might sound simple, but let’s not forget the billions that went into R&D over the decades, or the fact that Intel relies on its own fabs for manufacturing (and if you are in the market for a 14nm foundry, make sure you have some spare change on you, because a chip fab costs as much as a nuclear aircraft carrier).

What about ARM products? Well, ARM is not a chipmaker, it’s a chip designer, or a “fabless” chip company, so it doesn’t deal with manufacturing and does not sell own-brand chips. ARM sells something much more interesting: intellectual property. This means ARM clients can choose any of a number of different licensing plans and start making their own designs. Most of them choose ARM’s in-house designs (Cortex series CPUs, Mali series GPUs), so they pay a licensing fee for every CPU/GPU core they produce.

However, a client does not have to license these ready-to-go CPUs; it can license the architecture set instead and develop a custom core based on an ARM instruction set. This is what Apple does. It uses the ARMv8 instruction set to build big and powerful 64-bit CPU cores for its iOS devices. Nvidia’s Denver CPU is similar in this respect, and so are Qualcomm’s custom cores (32-bit Krait and 64-bit Kryo series).

Designing a custom CPU core is not easy. It’s not like you’ll find chip designers out of work and offering to design a custom processor on Craigslist, so this approach is usually reserved for big players who have the necessary technical, financial and human resources to pull it off. Therefore, most companies use off-the-shelf ARM Cortex cores instead (the 64-bit Cortex-A57 core can be employed in a server environment and it’s used by most next-generation ARM server processors).

It is important to note that virtually ARM-based chips are custom designed but the CPU cores used in most are not.

The vast majority of ARM processors rely on standard ARM CPU designs (Cortex CPUs) rather than custom CPU cores. This means chipmakers can choose any of a number of ARM CPU cores, third-party GPUs and other components, and tailor a processor to meet their needs without having to develop a custom CPU core. It’s a cheap way of making the architecture more flexible, and it has more to do with ARM’s licensing policies than engineering.

The new generation of 64-bit ARM server processors, such as AMD’s Opteron A1100, boast an impressive feature set.

The new generation of 64-bit ARM server processors, such as AMD’s Opteron A1100, boast an impressive feature set.

It is also important to note that these forthcoming ARM servers, based on the latest ARM 64-bit CPU architecture, don’t have much in common with the experimental ARM servers of years gone by. For example, one of our colleagues played around with Scaleway ARM servers, but they are based on ARMv7 processors and have a number of hardware limitations (for example, Scaleway used shared I/O controllers, and the lack of 64-bit support created another set of challenges). The new generation of ARM-based servers won’t suffer from these teething problems; they are much closer to Intel hardware in terms of features and standards.

ARM Server Pros And Cons

The problem with ARM servers is that they tend to be used for small niches, and they are not suitable for small developers who can live with any server. While some big companies find them attractive, the ARM servers that are currently available aren’t suitable for most individual developers.

However, the forthcoming server solutions are different and should appeal to more niches. This is what could make them appeal to a much wider user-base:

  • Reduced hardware costs, potentially superior efficiency (performance per dollar, performance per Watt).
  • Increasing compatibility and availability of popular ports.
  • Support for cutting-edge technology and new industry standards.
  • Ability to excel at certain types of workloads (simple but multithreaded loads).
  • Potential for more competition and product diversity than in x86 space.

I have to stress that, at this stage, some of these points are theoretical since the hardware isn’t out yet. However, while I can’t categorically claim to know what will happen over the next few quarters, I am confident the new breed of ARM servers will deliver these (and more) benefits. Why am I so confident? Well, if they didn’t have the potential to make a difference, ARM, Qualcomm, AMD, and other companies wouldn’t be wasting their time and burning money on their development.

So, what about ARM server downsides? There are quite a few, and some of them are big. Luckily, the industry is working hard to address them.

  • Hit and miss software support
  • Availability, potential deployment issues
  • ROI concerns
  • Tiny ecosystem
  • Old habits die hard

Software-related issues will, probably, be the biggest immediate concern. While a lot of popular services will run on ARM servers, software support will be a problem. It’s not enough to merely port stuff to new hardware; we have to make sure everything functions properly so there are no performance hits or failures. In other words, the ported software has to be mature. Nobody will develop and deploy a service built on buggy foundations.

The ARM server ecosystem is small, but a number of big industry names are already on board.

The ARM server ecosystem is small, but a number of big industry names are already on board.

With all the money to be made in the server market, one would expect to see fast progress, but that isn’t always the case. Adopting new hardware and tweaking all the software that runs on it is never easy, and the pace depends on market adoption. The size of the ARM server ecosystem is (very) limited, and I doubt a couple of new processors would make much of a difference in the short run. While influential companies like ARM and Qualcomm have a vested interest in seeing a pickup in demand for ARM servers, there is not much they can do about software. They have next to no influence over software developers, so they can’t force them to add ARM support to existing products.

Long story short: Take a good look at your stack and try to figure out whether everything will run properly on ARM hardware. Given enough time, developers will start adding support for ARM hardware, but this won’t be a fast process. They will have to tweak frameworks and applications to take into account a new architecture, and I suspect, many of them won’t bother until there are enough ARM servers out there (which may take years). Support for legacy software is another obvious problem.

This brings us to the next point: Market availability and potential deployment issues. There aren’t that many ARM servers out there, so choice is limited, and so is availability. A year or two down the road, we could see a number of ARM-based hosting packages on offer, but we won’t see too many. Worse, there’s a good chance these servers will be concentrated in certain parts of the world, making them less attractive to some developers. There are a lot of unknowns related to deployment, so it’s still too early to say how things will pan out.

Slow adoption could create another set of challenges. These aren’t restricted to ARM servers; they apply to most enterprise technology. A lot of organisations are bound to explore the possibility of using ARM servers, but that doesn’t necessarily mean they will actually use them. In order to ensure enough development and consumer demand, market adoption needs to grow steadily. Otherwise, risk-averse people will probably stay away, taking the wait-and-see approach. The other potential problem is economic: If developers aren’t sure the ecosystem is growing fast enough, they might conclude the potential return is simply not worth the effort.

What about these old habits? Well, since server space doesn’t evolve fast, people tend to stick to proven platforms, namely x86 hardware. The motto is simple: If it ain’t broke, don’t fix it. Industry veterans might see ARM servers as an opportunity and take a gamble on them. It would take a fair amount of courage and confidence to tie part of a complex project to what many people still perceive as an untested or immature hardware platform. I fear many people won’t be willing to take the plunge, at least not this early on.

Bright Future And A Pinch Of Hype

I’ve spent the better part of my adult life covering cutting edge silicon, and my personal take on ARM servers is that they have a lot of potential, but, they’re not for everyone. They could play a vital role in the Internet of tomorrow by providing cheap building blocks for infrastructure and handling niche server workloads.

However, at the same time, I cannot escape the feeling that ARM servers tend to be overhyped. Despite this, I don’t see them as a fad. I think they are here to stay, but vendors must carve out a few specific niches that can truly benefit from the new architecture.

In other words, we won’t see a lot of simple LAMP web hosting servers based on ARM, but we could see loads of them in more exotic niches (and some horribly boring ones). ARM processors could be a perfect fit for specific loads, especially those that can take advantage of a large number of small physical CPU cores, stuff that’s not CPU-bound. It might not sound like much, but this actually covers a lot of potential uses: data logging, large volumes of simple queries, certain types of databases, various storage services and so on.

I could go on, listing various use-cases, pros and cons of ARM servers, and potential problems, but at the end of the day, I suspect ARM server adoption will depend on good old cash. Technology aside, ARM servers will only make sense if the economic component checks out. In other words, they will have to offer a lot more bang for buck than x86 processors if they are to justify their existence.

Since this is more or less the whole point of introducing this new architecture to the server industry, I expect attractive pricing, but it will be a few months before we know for sure.

Hiring? Meet the Top 10 Freelance Back-End Developers for Hire in September 2016
Don't miss out.
Get the latest updates first.
No spam. Just great engineering and design posts.
Don't miss out.
Get the latest updates first.
Thank you for subscribing!
You can edit your subscription preferences here.

Comments

Felipe Lavratti
Are we going to see the Innovators Dilemma in action here?!
Laurent
It *is* a case of the Innovator's Dilemma. Disruptive innovations don't disrupt markets, they disrupt business models. And ARM's business model is incompatible with Intel's. It breaks the compatibility with its x86 processors and isn't as powerful - two of the things Intel typical customers are asking.
Laurent
Interesting article. I've been wondering for a few years myself if ARM processors would invade the server space. If they do, I would expect them to be first embraced by companies like Google. Because they deploy hundreds of thousands of servers, Googlers are obsessed with the ideal performance/heat ratio. Google also controls its hardware and software stack which helps embracing a new technology such as server-side ARM processors.
Nermin Hajdarbegovic
Yes, big outfits like PayPal, Facebook, Google, and Amazon are probably in the best position to leverage the efficiency of ARM servers early on. The bigger a company is, the more it's "obsessed" with getting the best possible performance/thermals/value ratio. If you have 10-20 servers and you can save 10W on each processor by using ARM, it's not a big deal... But what happens if you need 10-20 thousand units? We are already talking about megawatts and a nice heap of change at the end of the year....
Nermin Hajdarbegovic
I still don't think instruction set compatibility isn't the biggest deal here. I mean, it's an important factor, especially when it comes to smaller companies and individuals who can't control their stack the same way Google or Amazon can. However, ARM's business model doesn't have much in common with Intel's, because it's largely based on licensing. What does this mean? Well, we already saw some custom x86 chips (for example AMD parts used in gaming consoles) and the same approach could be used with other chips, both x86 and ARM. Since ARM licenses its CPU cores and instruction sets, it allows loads of companies to design custom chips, or even custom chips with custom cores. This flexibility is lacking in the x86 space. What does this mean? Well, let's say we're bigshot execs and we're forced to invest tens or hundreds of millions into our infrastructure. One of our tech guys shows up advocating the latest Intel small-core server processor, the other one thinks we should go with AMD's or Qualcomm's ARM server parts instead. Neither is a great fit for our typical loads, so we start thinking about a custom design. In some cases, we would need sixteen big A57 cores on a single chip. However, even these ARM cores are too big and powerful for what we need, so we get someone to design a custom part, with more small cores (the current A53 and A35 generation could be tweaked for this). We can even mix different cores, in theory, so instead of getting sixteen A57 cores, we can use 12, with loads of small cores on the side. The A53 is about 1/4 of the size of the A57, so we could "trade" four A57 cores for 16 smaller A53-class cores. In some extreme cases, it would be possible to use even smaller cores (A35-class) and create ARM server parts with 64, 96 or 128 physical cores. We wouldn't have to use standard nodes for them, either. So, if Intel is at 14nm FinFET with its big-core parts, we could settle on an inferior manufacturing process and use smaller, slower cores that perform well on cheaper nodes (right now the cheapest viable node is 28nm, available from TSMC, Samsung/GloFo and other foundries). Sure, 14nm is a vastly superior node, allowing higher clocks and better efficiency, but the price per transistor is much lower on 28nm (on the order of 2 to 2.5x). In some cases, the savings of sticking to an older node outweigh the potential energy efficiency boost made possible by cutting edge cores. That's why most smartphone processors are still on 28nm, despite the fact that 14nm and 20nm nodes have been available for a while: chipmakers chose the cheaper node, because it's still good enough. We could tailor chips for wildly different loads without breaking the bank, and that's the beauty of it. This is not something x86 companies can do (I mean, theoretically they could, but they won't). It might take a while, but I am sure ARM servers will be a success in certain niches, although x86 will dominate the market for years to come.
Lee
Really enjoyed this article, thanks. Off-the-shelf software may be slower to adopt, but for many web devs tasked with writing Node.js, Java or Go-based microservices and web apps, compiling down to ARM is mostly seamless. I was delighted to see I could run all of my Docker containers on a Rapsberry Pi (ARM-based) exactly the same way I could deploy to AWS for a recent project. If cloud providers can incentivise adoption by passing on costs savings to developers, I think we could see an explosion of uptake, and .arm.tar.gz files in the "downloads" section of most major vendors.
Laurent
I still see the x86 compatibility as very important for Intel. Some bigshot customers may not care, but most companies want compatibility with Windows and x86 Linux - and the software that comes with it. Volume-wise it's a huge market. Remember the Intel Itanium? It broke the x86 compatibility. We saw how that went.
Laurent
Actually I just found that article. Bloomberg believes Google is considering buying CPUs from Qualcomm: http://www.bloomberg.com/news/articles/2016-02-03/google-said-to-endorse-qualcomm-s-fledgling-server-chip-efforts-ik6ud7qg
Nermin Hajdarbegovic
Yeah, I was on the road when it came out and this post was already published, so I didn't add that bit. Bad timing. It just goes to show industry titans are taking ARM servers seriously.
Nermin Hajdarbegovic
For Intel - yes, no question about that. For AMD, which already lost much of its x86 server market share, ARM is a way of gaining more customers. Let's not even mention ARM exclusive outfits like Qualcomm, Marvell, TI etc. Ah, the Great Itanic, how could I forget :)
Laurent
That is correct. However, the initial point of this thread was about the Innovator's Dilemma and disruptive innovations. In that context, Intel is the incumbent and pretty much anybody else is jumping on the disruptive technology that is the ARM processors. What is happening right now is typical Innovator's Dilemma playbook: newcomers (or losers from the x86 war) jumped on a new technology (ARM) that the incumbent (Intel) avoids because it is disrupting its business model. The new technology survived by finding its own niche market (embedded systems then smartphones) and now that this market has grown big it is using its economies of scale to attack the incumbent's turf. First by starting from the low end (just like the first server PCs didn't do much compared to UNIX servers), then by trying to moving up. Not all disruptive innovations however manage to topple incumbents though - even in the long run. One big difference with the PC vs UNIX machines war is that Intel also enjoys large economies that UNIX manufacturers didn't. It will be interesting to see how this turns out.
Nermin Hajdarbegovic
Yep, that's true. And I don't think anyone expects ARM to "topple" Intel. However, ARM chips have the potential to disrupt certain market segments and that's a good start. I think these are the key points we need to take into account: - ARM chips are evolving at a much faster pace than x86 architectures - custom processors and custom CPU cores can provide superior performance per dollar and watt - Server workloads are changing I wonder how it will pan out, but I must stress that it will take time, a lot of time (years rather than months and quarters).
Janusz Kacalak
Here an interesting article on a new line of Intel's server CPU, originally crafted with and for Facebook needs: https://code.facebook.com/posts/1711485769063510/facebook-s-new-front-end-server-design-delivers-on-performance-without-sucking-up-power/
comments powered by Disqus