How to elevate a database to the cloud

How should you move your database structures to the cloud?

When an organisation agrees that it's time to move the company's estate of technology to the cloud, the IT manager simply lines up a sight on the migration path and presses go, fire, deploy, head for the stars… right?

Whether we are application and platform architecture specialists or not, we all know by now that the above suggestion is fanciful at best... and pragmatically impossible in reality.

There is no lift and shift when it comes to cloud migration, not for cloud databases, not for cloud applications, not for cloud components and services of any kind if they initially started off life as ‘terrestrial' (not the official term, but you get the point) pieces of technology that lived down here on Earth.

Climbing to cloud databases

As we embark on the road to cloud (surely that should be stairway) we will now encounter an increasing number of software elements born in and of the cloud. Looking specifically at databases rather than the applications they serve, the rise of so-called cloud native technologies gives us the option to use Database-as-a-Service (DBaaS) technologies that have been built as cloud native from the start.

"We are just now beginning to see people thinking about moving classic on-premises [apps and databases] to the cloud. When that happens, substantial redevelopment is necessary, especially if the application is to be cloud native. With that level of change, all the components come into question, including the database management system... and there is greater openness to changing that technology than we have seen before," said Carl Olofson, research vice president at IDC.

But, as we stand in 2020, not every subset of technology is available in cloud native form and business is likely to hang on to working ‘legacy' technologies for many years to come. It is not unreasonable to suggest that organisations will be looking at cloud migration challenges well beyond the current decade, so how should these firms move their database structures to the cloud?

Core cloud complexities

The process of putting an effective database service in cloud computing environments appears to be riddled with core command line (i.e. hard core programming) complexities.

Director of field initiatives at cloud platform database company MongoDB Dominic Wellington warns that an organisation's software developers will need an intuitive way to work with operational, transactional and analytical data to unlock the real power of cloud infrastructures at the database level. In other words, different types of data for different types of applications behaves in different ways depending on the size, shape and indeed flavour (pricing and performance structure) of the cloud service in question.

"Just shifting infrastructure to the cloud does not solve all the true challenges that developers face when working with disparate, siloed data. The need goes far beyond the database. A cloud platform that breaks down the complexities of moving between the database, data lake and/or data warehouse can help satisfy users' requirements and unlock new uses - ones that were never foreseen," said MongoDB's Wellington.

Data has its own gravity

CTO and co-founder of open source time-series database company InfluxData Paul Dix has further cautionary warnings for would-be cloud database migration projects. He says that the main thing to understand about moving databases to the cloud, or to another cloud provider for that matter, is that data has ‘gravity'.

"It takes time to move and the more data you have, the more time it's going to take. This means that the migration process isn't as simple as just provisioning it and spinning it up [to the cloud]. Because data takes time to migrate over, you have to think about what this transition is going to look like, if you're going to do a hard switchover once most (or maybe even all) of the data is migrated and if you're able to arrange some sort of downtime window," said Dix.

Of course, no single thing that exhibits gravity gets to ride for free. Dix reminds us that it costs to egress data from one zone to another and data transfer costs can be tough to estimate and manage. He notes that we will need to think about which applications are going to be accessing the database and what their workloads are and where those application servers are located.

"This makes things like selecting a cloud datacentre region important and may also impact your migration strategy as you'd want to migrate your applications over at the same time. Migrating ‘stateful' services, like databases, is significantly more difficult and requires more planning than migrating stateless application servers or pure processing workloads," added InfluxData's Dix.

Cloud database Nirvana

If we architect our cloud database correctly, then it becomes a thing of beauty that employees start to gravitate towards. You want to know what sales forecasts are for region X over the next 12-months? Sure thing, use the cloud database that now offers abstracted visualisation tools for non-techies to ask questions of it.

This is the end game, this is the goal, this is the ubiquitously connected cloud database that works with all types of data in all types of application for all types of people. It's still a distant dream for some, but it's a future certainty for all of us.