Containers in 2020 - what will the impact be from cloud native IT?

What can we expect from containers and software development practices in 2020?

This is a contributed article by Mark Pidgeon, Vice President Technical Services EMEA at Sumo Logic

Containers are popular for new applications, and they are only going to get more popular. At the same time, Kubernetes has become the de facto standard for container orchestration and management, particularly for companies that want to run across more than one cloud platform. In our report on customer choices around multi-cloud and containers, the use of Kubernetes scaled up with the adoption of more cloud services.

However, what will happen in 2020 based on all this adoption, and what are the potential problems that might come up?

Prediction #1 - Automating should mean more thought, not less

Kubernetes aims to make the job of running IT easier through automating more tasks and hiding all the plumbing involved. That sounds great … but it means actually thinking about what you are automating and why.

For many teams, this process is not something they have gone through from scratch. Instead, they have made what existed faster. While this might be useful, it is not effective.  So while cloud-native IT projects will become more popular in traditional enterprise IT teams, they will have to be about more than simply migrating what exists to the cloud.

Getting this right involves looking at what you are automating, what business result that automation will provide when it is done right, and working back to create the right service level objectives and indicators. Missing these steps out will lead to either replicating what already existed with less people, or to projects failing to deliver.

In 2020, there will be the first "Kubernetes fail" project stories that break through - what will be interesting to see is how much of these fails are due to implementations not getting carried out properly, and which ones are misapplications of the latest in a long line of technologies that look good on a CV but aren't suitable for the specific use case.

Prediction #2 - Kubernetes will grow up

It might seem strange to say this, but Kubernetes will have to do more of its growing up in public in order to be completely ready for the enterprise. While the automation and orchestration sides of Kubernetes work well, it is still like any child - it will need to play well with others too.

This process is already happening with open source projects for cost management like Kubecost being developed to fill gaps in what enterprises need to manage their operations. There are other data points that traditional IT teams can obtain from their IT management tools, and these will be needed for container implementations too. In 2020, these complementary projects will develop quickly as more companies adopt Kubernetes and then find the gaps in the data they have or the tasks they have to oversee. Developers in particular will lead in adopting or launching these projects - this may lead to some fragmentation in approach to start with, but there should be some consolidation around specific projects based on which tools meet developer requirements fastest.

Prediction #3 - Security for containers will get more attention

It's now more than a year since the first major Kubernetes security issue was discovered. Since then, there have been a few issues within Kubernetes itself, but these have been fixed quickly. However, containers themselves are still in their infancy when it comes to the enterprise.

In 2020, container security will move from being the preserve of specific niche players into being one of the main strands of discussion and security market consolidation. As more companies move to cloud-native IT and to containers, security teams will be scrambling to keep up with the new model and approach that containers require. Security Operations Centres will have to adapt to the ephemeral IT that these applications run on.

The challenge here is that existing security incident and event management tools don't fit with the cloud and the model for keeping containers up to date. The Cloud SIEM market is going to be fiercely competitive as older players try and get ready for cloud, and the new ones try to scale up enough to capture market share.

Next year, we should expect the first examples of security problems caused through out of date container images or poor update policies to become public. Snyk found that the ten most popular Docker images contained at least 30 vulnerabilities each this year. If we don't do something here, then the problem will only get worse in 2020.

Prediction #4 - Observability will be a battleground for businesses

Observability covers a range of different approaches and technologies around data, applications and infrastructure. From roots in cloud and application performance monitoring on one side through to developers using log management, metrics and tracing on another, observability is becoming a more strategic technology.

Why is this? The applications that are sharing log data, metrics information and being monitored are now responsible for running organisational activity, and the data can provide a proxy for how well a company itself is running. Any change - from tweaks in the application affecting performance through to bigger company decisions or launches - can have an impact, and that impact can be directly traced. Using this data used to be the preserve of developers or IT operations teams, but it is now finding its way to other teams too.

In 2020, this process will speed up, meaning more observability projects will be started and more people desperate for data will get involved. This will be great for teams that can manage this data going out to different teams effectively where they can put data in the right context; for others, there will be disappointment with projects trying and failing to meet what business teams expect around data. What will be most important is that observability projects will have to manage expectations and help business teams understand the difference between correlation and causation.

Prediction #5 - Working practices around data will have to evolve

Linked into this growth of observability will be a trend towards how company teams collaborate around data processes. In the data science sector, there's already a dearth of data scientists and machine learning engineers that can straddle the technology and the business conversations. There is a similar demand for people building up across DevOps, IT and business teams.

The use of new applications and cloud native IT based on containers is speeding this process up, helping companies implement better continuous integration and continuous deployment pipelines that get their projects out faster. In turn, this creates more data and more useful information for analysis. The challenge is that turning this raw data into insight is a harder problem to solve.

Building a continuous intelligence pipeline - taking the data from DevOps and making it useful for all the different teams involved in a business - will be a challenge for all these teams, as they have previously focused on their own needs in the software development process, rather than for the business or operations teams. This requires more business analyst skills as well as code or infrastructure experience, so expect a lot of demand for people who can translate "business speak" into both data and developer requirements.

Mark Pidgeon is Vice President Technical Services at Sumo Logic - he is responsible for managing Sumo Logic's customer success and technical services operations with customers. He has more than twenty years' experience in running pre-sales, technical support, customer success and education programmes for a range of technology companies.