As anyone who has rented a car in the United States knows, the option for a manual transmission vehicle is known as stick-shift, or just plain and simple ‘stick’. Thankfully, automatic cars are known universally as automatics, it’s a term that doesn’t need much localisation.
On the new stretches of the information superhighway powered by cloud computing, we have a remarkably similar choice between manual and automatic management functions. As with automotive, manual control can sometimes offer more revs and stronger torque, but it may come at the price of spin-outs and skids. The question of where we now automate for control and competitive advantage is key.
Central to this question is cloud infrastructure. The cloud offers a new lane of availability, service resilience, scaling, experimentation and cost-savings, but it is how we build and manage the base layer road surface that really denotes the quality of the journey.
The mistakes that occur in manual cloud environment provisioning, set-up, deployment and management demand an additional solution - and that solution could be Infrastructure-as-Code.
What is Infrastructure-as-Code?
Infrastructure-as-Code or IaC is a means of configuring cloud infrastructure that reflects the way developers build applications. IaC allows us to ‘describe infrastructure’ in the form of source code, which can be used to automate the provisioning of that infrastructure in the datacentre, private or public cloud.
So why has IaC come about and what is its core advantage? CTO & co-founder of Coralogix Yoni Farin says that it is a reaction to the inconsistent environments, the spectre of hidden infrastructure and the ills associated with unclear architectures that emerged throughout the first age (assuming we’re in cloud 2.0 these days) of cloud engineering.
Coralogix specialises in providing a platform that empowers DevOps teams with log monitoring, metrics and security. Farin talks about the evolution of IaC and says that teams needed a mechanism to automate the creation of cloud infrastructure, roll out new environments and act as living documentation to keep track of their cloud base layer.
“Multiple environments are very useful for testing changes to code before deploying into a production environment. Despite the sophistication of modern testing tools, there really is no substitute for seeing it working. But the question organisations need to ask themselves is, are all of our environments consistent? Without knowing this, how can we know whether our development environment bears any resemblance to our production environment? This is where IaC automation comes to the fore,” said Farin.
Stay on the road to the code
Rather than manually creating each cloud environment and hoping for consistency, a business can make changes to its infrastructure IaC codebase and apply this change to each application’s live production environment. This means that environments can stay completely consistent, provided that engineers stick to the code.
Drawing upon their own experiences working with customers, the Coralogix team say that some queries have the ability to raise their head recurrently. At a certain point, someone somewhere is going to ask a simple question about the organisation’s cloud architecture - and that question is typically: ‘So, what are we actually using as our cloud foundation then?’
“This should be a simple question, but unfortunately, as your architecture scales, it becomes difficult to answer. With manual changes, users could be accessing new services and building solutions in entirely new ways without any clear and obvious visibility,” explained Farin.
IaC enables this visibility because all changes should go via the codebase. This means that IT teams can look at the code and understand how the architecture hangs together. This acts as living documentation that stays up to date by design.
Working with ‘living’ documentation
Looking at the use cases where it has seen this approach embraced, embodied and extended, Farin and the Coralogix highlight the fact that this living documentation code can also be scanned, for example using Terraform graphs to generate dot diagrams. This opens up the door for a series of diagrams, workflows and other visualisations that can mostly automate the process of documenting and tracking infrastructure decisions.
Coming back to our manual vs automated question, there’s a key learning point here for organisations looking to drive their cloud estates forward.
“When you’re clicking around on the AWS user interface, every time you wish to create a new database, you’re going to be clicking the same things over and over again. Even if you’re excellent at repetitive tasks, these actions are not the best way to fill the time of an experienced engineer. Instead, you should seek to automate these clicks,” said Farin.
IaC allows you to create code modules that will generate cloud resources, like databases, serverless infrastructure, servers, and much more. These modules are great for automating tasks that would otherwise become mundane.
If we had made any blind assumptions about cloud infrastructure before starting any of this discussion, then we may have assumed it would be a fairly statically packaged and formally engineered discipline without quite the dynamism and propensity to change we have illustrated here.
Given the movement and subsidence that can exist at this level, businesses will need to think about building consistency into their cloud infrastructure. Although it sounds like a bland enough term, consistency in cloud means knowing how much billing an organisation will incur. It also means knowing what the data classification status of any given data is, an important factor when dealing with sensitive data.
All things (and clouds) must pass
When a business builds up its cloud infrastructure, it typically takes a short-sighted view of the entire cloud lifecycle. Organisations almost never consider what they’ll need to do when they tear it all down. Lifecycle management is a complex endeavour that is usually tacked on halfway through a project, if ever, because it has been deprioritised in favour of other feature work.
“Bringing down cloud infrastructure, especially when there are multiple products in a single account, is not an easy endeavor. It takes a lot of picking through and deleting specific resources, in the correct order, to avoid errors and slowdowns. When you’re using IaC, you can typically bring down your infrastructure in a single command. For example, when you’re making use of Terraform, you can run the terraform destroy command. This command will bring down all of the relevant infrastructure for your specific module,” explained Coralogix’s Farin.
This means that, even when there are multiple projects in the same cloud account, an organisation can ‘surgically remove’ components, in the correct order, in a totally automated way.
Finally, for now, when a business runs its cloud Infrastructure-as-Code, Farin points to the need to declare your observability rules from within that code.
“This will not only enable you to describe how the system works, but also how the system is monitored. Encoding operational rules within your code is an incredibly powerful mechanism for scaling and managing your infrastructure. So in short, for these reasons and more, organisations really should be adopting an Infrastructure-as-Code approach,” concluded Farin.
Two decades into its post-millennial evolution and (successful) cloud computing is characterised by its propensity to scale, its ability to shoulder complexity and its inherent and native composability from applications, through platforms and across infrastructures. Driving this terrain on full manual control when the safety and security afforded by an automated gearbox is available amounts to dangerous driving. It’s time we kept our eyes on the road even we occasionally take our hands off the wheel.
