supercat
Cloud Computing

Cloud Supercomputing (part 1): What is it?

As more and more High Performance Computing (HPC) moves into the cloud, we consult a panel of 12 experts to determine the role of cloud supercomputing and where it is heading. Part one of our four-part series addresses what “cloud supercomputing” really means and where we are now.

“It depends upon the definitions of ‘cloud’ and of ‘supercomputing’. Cloud is best defined as a service delivery model, and it really doesn’t matter where that service comes from so long as the quality of service is adequate. Cloud services are usually delivered as some kind of ‘utility model’ measured by usage rather than ownership or other criterion. In that sense, many collaborative supercomputing sites could be considered part of the cloud.  Think CERN, the Large Hadron Collider, and the Higgs boson discovery. A supercomputing workflow that involved ‘High Throughput Computing’ using OpenStack to find the result.  Is that cloud supercomputing? I think so, but others may argue. 

At the end of the day, there are organisations that offer ‘service’ or ‘cloud’ based supercomputing capabilities and to me they qualify as ‘cloud supercomputing’ so long as they deliver the right quality of result.  The current state?  I think ‘cloud supercomputing’ is alive and well and growing in the way the people choose to access ‘supercomputing capabilities’, but that is not the same as thinking that you can do the exact same thing on AWS, Microsoft Azure, VMware vCloud Air, Rackspace or other services. There are some ‘high performance’ workloads that may work very well in those environments, but not all.  On the other hand, access to compute facilities of any kind is better than no access and these public cloud services enable things for people that do not have available access to other resources.” 

Peter ffoulkes 451 research


“Back in 2000, the emergence of Beowulf clusters using commodity servers lowered the cost barrier to High Performance Computing (HPC), and marked a change in accessibility of supercomputing. I worked at Apple when the System X gained press for its huge 12 terraflop from standard servers at Virgina Tech at the time between 1999 and 2008 when research labs blossomed with commodity computing to create supercomputing performance. 

Cloud computing has made those commodity HPC accessible by the hour, instead of by the truckload.  

High performance computing will solve 99% of computational problems, leaving the 1% for work in medicine, meteorology, oil and gas, particle physics and other research fields to use dedicated supercomputing from Cray or IBM. The vast majority of problems that enterprises and governments need can be solved fast enough on an HPC arrangement of servers connected with low latency.”

Giri Fox, director of customer technology services at Rackspace


“To assess the current state of cloud supercomputing, we must be sure to distinguish between high performance cloud and supercomputing. Supercomputing is very specialised, with a few hundred major users worldwide. High performance computing is the next tier down from supercomputing, built on the capability of standard cloud architectures and not on cloud ‘supercomputers’. This is a much more interesting area for cloud as it’s about fast turnaround of data analysis - which is the area of cloud that is extending into ‘ordinary’ life quite rapidly.

Cloud supercomputing is currently more of a niche player, where large cloud providers (AWS, IBM, HP) are trying to tempt supercomputer users to commit to using large volumes of high performance CPUs. But as most clouds are built from mid-range processors, there isn’t yet a great deal of provision. Additionally, cloud supercomputing is struggling with the same dynamics as the rest of the industry – how to do more with less in light of rising demand rapidly outstripping available resources.”

Dariush Marsh-Mossadeghi, director of technology strategy & architecture at DataCentred


“Cloud based supercomputing still seems to be a niche market, but one that does appeal to those with variable, but substantial computational needs.

We see little demand in our focus areas of medium sized insurance and finance, professional services and media, but do see occasional requirements from science and forecasting where general purpose cloud resources would struggle to deliver.

For those use cases I believe that paying by the hour or day for high end compute resource is a viable option, harking back to the days of mainframe timeshare.”

Edwin Wong, founder and cloud evangelist at Frontier Technology  


Is there a move to run HPC in the cloud? Yes. It’s becoming cheaper to begin using a supercomputing framework in the cloud than it is in house, largely due to the short term projects and unpredictable workloads that many end users may be facing.

Cloud supercomputing is also taking hold due to the rise in cloud providers purchasing more volume, providing their customers with better access to sophisticated GPU chips, while improving their internal networking to perform at a higher level.

However, it’s also important to remember that those running steady, long-term projects will almost certainly find it cheaper to build their own infrastructure as opposed to investing in HPC functionality; if the infrastructure needs to be relied on every day, it could provide much more value for money than HPC in the cloud. There are no silver bullets in IT!”

Daniel Beazer, senior consulting analyst at Peer 1 Hosting


“‘Cloud Supercomputing’ – the term itself delivers visions of high-performance computing (HPC) capabilities at a price point more accessible than more traditional supercomputers – but it doesn’t stop there. Trends such as DevOps and the Internet of Things demand more data processing power than ever, and many cloud providers are aware of this – many end users are beginning to demand both high performance computing but also low latency and high bandwidths to accommodate data-intensive workloads.”

Asad Malik, Product manager at MTI


Supercomputing is about the advancement of knowledge and science – amongst other things – and cloud is playing an increasing important role in that journey.

Peter ffoulkes 451 research


Read the rest of this series:

Cloud Supercomputing (part 2): Where is it heading?

Cloud Supercomputing (part 3): Factors in development?

Cloud Supercomputing (part 4): How will it help society?


PREVIOUS ARTICLE

« Cloud Supercomputing (part 2): Where is it heading?

NEXT ARTICLE

Blogging for local recipes in Kenya »