Why observability is a key metric for the cloud era

The abstract virtual world of cloud computing can be more directly and accurately controlled if the right observability and visibility tools are applied to as-a-Service applications and services.

Cloud computing is both abstract and virtualised. This may be the only solid truth we can pin down in the still-nascent world of as-a-Service computing, where the resources we tap into come ‘down the pipe' from a virtual backend block of IT.

Because these new forms of technology are abstracted upwards and away from the core datacentre servers that they run on, it can feel like we don't have the same granular ‘inside the engine room' feel that we're used to. Of course, the virtual world of cloud is supposed to do exactly that i.e. be virtual. But we will still need to look under the hood for a variety of reasons to know how our software code and data and working together.

These factors have given rise to a new slew of observability and visibility technologies, tools and platforms dedicated to observing… and then informing and helping to onward manage our IT resources.


Beyond systems management

Observability encompasses the areas of IT monitoring, auditing, management and systems administration, but it also goes beyond those core pillars. This is an area of deeper software log aggregation and analytics to enable us to really know how systems are working ‘in flight' as they run in live operations.

When live ‘production' IT systems are distributed (as they often will be in cloud) across multiple locations, being able to perform distributed systems tracing and receive alerts on application health is essential. Much of this data will be unstructured ‘raw' system-level data (such as log files generated by any given application action), so observability platforms will work to provide risk indicators that analyse and correlate logs, metrics and traces to extract signals.

These signals are then viewed through a visualisation layer by human eyes in order to monitor system health against Service Level Indicators (SLIs), Key Performance and Risk Indicators (KPIs, KRIs).

One specialist working at precisely this level is Sumo Logic. The company's eponymously named Sumo Logic Observability suite is powered by its own Continuous Intelligence Platform. The technology proposition here is software to provide enterprises with a unified view of real-time analytics across application and infrastructure logs, metrics, traces and metadata.

For completeness and clarification, a log is a file that contains system-generated information about events that have been executed by a software application, device, or infrastructure including data relating to processes, errors, application ‘state' and more. In contrast yet close proximity, traces are more deliberately generated views of data transactions and service progression through a wider IT system or application.


Mainstream momentum

"Observability is making the transition from being a niche concern to a mainstream approach for user experience, systems and service management in startups, SaaS and enterprise companies. Change rather than stability is the goal and there is a lot more uncertainty in systems and applications than there used to be," said James Governor, founder of technology analyst house RedMonk.

Sumo Logic is expanding its observability suite by adding distributed transaction tracing capabilities and three new suite solutions that unify application and infrastructure logs, metrics, traces and metadata and enable sophisticated analytics on both structured and unstructured data.

"Observability is the latest evolutionary step in methodology that DevOps and DevSecOps teams employ to deliver reliable digital services that, in turn, deliver best-in-class customer experience. To be reliable means always available, performant and secure," said Bruno Kurtic, founding VP of Strategy and Solutions for Sumo Logic.

Kurtic suggests that through the use of observability tools, organisations can direct their DevOps (and, specifically, their security-focused DevSecOps) teams to help secure mission critical workloads and achieve a more predictable cloud licensing model without unexpected on-demand charges or hidden costs.


Observability, but only after visibility

But if observability is a key window into the virtualised world of cloud, shouldn't we start with achieving visibility? Could the ‘view into cloud' story actually start one stage earlier?

Just as observability technologies are now being positioned as platforms in and of themselves, other firms are selling visibility solutions designed to enable software teams to know more about the clouds they are now chasing before they build and provision them for use.

Appvia is multi-cloud delivery platform designed to enable development teams to build and deploy containerised applications at scale. The company's Cost Prediction and Visibility tool is integrated within the latest version of its Kore platform.

This software gives cloud cost visibility at the planning and provisioning stage, as well as providing on-going visibility of actual costs as the cloud applications built upon the instances selection are pushed into live production.

Appvia created the tool in response to the inordinate amount of time it can take for a developer to identify the right cloud solution for their project. Software engineering teams are also able to see cost-by-performance implications across different cloud providers and reusable cloud templates (plans) to see how altering node pool sizes and instances would change these operations.


The visible observable sight-able cloud

These trends for cloud insight, foresight and oversight could be driving us towards greater confidence in cloud resources, particularly if they coalesce and integrate through industry partnerships and open technology consortia or working groups.

With a strong emphasis on enabling cloud customers to achieve smooth and secure operations, these technologies arguably go some way to tackling the whole ‘but where is the actual cloud?' question that stops some firms from taking the more virtualised route forwards.

Inevitably perhaps, the next developments in this space see an increasing amount of Machine Learning and AI applied to the toolsets on offer. Naysayers might suggest that applying more automation to technologies designed to provide more hands-on control is a counter-intuitive measure. But that's naysayers for you and automated intelligence is rarely a bad thing. Anyone can ‘see' that… right?