Why the C-Suite needs to care about data engineering

As the digital grease-juggling engineers that make up our DevOps teams now work an explosion of data tools across complex & fragmented workflows, can database management specialists like Percona improve the data engineering team’s existence?

IDGConnect_data_engineering_C-suite_shutterstock_714159727_1200x800
Shutterstock

My door is always open. It’s a term oft-used by company directors and divisional managers seeking to convey an aura of affable approachability and amiable accessibility.

But really, even if those doors are open, who are they open to and what kind of plea will even the most broad-minded C-suite manager actually listen to?

It’s likely that most of the petitions and appeals tabled by ‘walk in’ employees would be requests like: can we have a new company wellbeing initiative, can we create more diversity in the workplace, why are our sales targets always unachievable, why doesn’t this office have any car parking… and why are there never any diet isotonic drinks in the vending machine?

The request you won’t hear

The request you or any bystander probably won’t hear is: why is our database management layer getting so tough to work at the same time as the demands on our data engineering team are increasing?

It’s just not the kind of thing people ask the company directorate is it? So if we can’t reasonably bring up these issues, what can we do?

As the digital grease-juggling engineers that make up our DevOps teams now work with an explosion of data tools across what are increasingly fragmented and more complex disjointed data-dependent workflows, can we look to database management specialists like open source centric Percona to use AI and experience to improve the data engineering team’s existence?

Percona specialises in providing specific additional management layer services for a group of popular open source databases. With a focus on MongoDB, MySQL and PostgreSQL, Percona itself operates at a completely open source level. In the company’s own terms, this means that users should be able to get value out of using the free versions of its products - although, as a business, it clearly sells enterprise subscriptions too.

“The future is connected. Nothing is getting less connected. This means integration becomes more important and we know that enterprises run as many as five, ten or twenty stacks (composed of several databases and several clouds and so on) concurrently - and what this means today is that you can’t just succeed with software alone, or managed services alone, it all needs to work together in unison,” said Donnie Berkholz, SVP for product at Percona, speaking at his company’s annual user convention this May 2022 in Austin, Texas.

In response to the reality of these complexities, Berkholz and Percona CEO Peter Zaitsev have showcased the company’s latest updates, which include the progression of the organisation’s core offering to a confirmed platform-level status. Simply named Percona Platform, this technology includes a defined Private DataBase-as-a-Service (DBaaS) capability, currently at ‘technology preview’ status level.

DevOps is nice, if you can get it

Designed to take care of developers’ data engineering requirements across both test and live production instance phases of an application’s life, this service enables developers to quickly provision and manage their database requirements on a self-service basis.

Clearly, this is a useful technology for developers, many of whom CEO Zaitsev says will have no actual data engineering team to draw upon (DevOps is a nice idea, but sometimes the Ops team never actually exists due to budget limitations) and so have to carry out all their database provisioning, analytics and management tasks themselves.

Why does this matter to the C-suite? Because there is enough database platform control in place to make the IT team happy - and that usually means happy users, happy customers and ultimately happy C-suite profit and loss meetings. It also answers the questions that could otherwise start coming through the ‘always open’ office door.

“The future is also intelligent,” said Berkholz. Thinking about the need to capture the knowledge people gain when working in organisations, he suggests that individual engineers, teams, entire business units and wider communities risk losing their earned intelligence unless know-how is codified.

“In so many cases we see that knowledge of this kind is never written down, so it needs to be encapsulated in order for us to be able to truly progress. Automation will help us [use that know-how] to track and catch configuration drift in database structures so that we can create value and work, rather than respond to [help desk] tickets,” said Berkholz.

Data diversity diagnostics

In further response to the demand described here then, Percona Platform now brings forward its Query Analytics functions. This enables developers and whichever Ops professionals do actually exist to get visibility into the working core of the database being used to support the applications they build. This technology can track and uncover performance patterns and interpret what they mean to service diagnostics and troubleshooting procedures.

Percona has also incorporated a range of what it calls ‘advisors’ to provide automated insights that ensure user databases are performing well in terms of security management, configuration and performance. Once again, it’s all about keeping the database management function working smoothly so that CIOs and CTOs can close their door when they need to.

“The future is open. Keeping open source open for real does matter and we need to make sure that it never becomes a marketing term. We need to make sure users can really get value out of truly open source free-use versions of software. Customers should never be in a position where there are ‘forced to pay’ in order to make sure any given database and wider production environment does not fall over,” said Berkholz.

The Percona Platform technology proposition is all about being able to run the databases the company supports in any environment. It provides control and visibility, simplicity and what the company calls ‘developer velocity’ overall. Percona provides developers with self-service enablement to enable faster easier use. The self-service element here also includes a collective knowledge base.

Experience on the agenda, almost?

Developer, data scientist and database administrator workplace experience is of course something that management should be thinking about. Companies have invested more heavily than ever in their digital backbones throughout the pandemic, so they need their data team’s tools to work.

“What the C-suite should be caring about (but still keeping their door open for) are issues covering data trust, data integrity and data resiliency – and database management is how you get there,” concluded Berkholz. “The management team really needs to have that base covered off so that it can focus on the velocity of the entire business operations layer (and embrace the advantaged that come with self-service, automation and portability) in order to accelerate time to market.”

Further, companies know that investment in data professionals is expensive, so they need to get the most out of them and care about how productive they are. If it’s too much to hope for ‘data engineering experience performance’ to be item #17 on the next company board meeting agenda, then perhaps we can at least hope for some wider cerebral appreciation of the concerns raised here.

Now on to item #18 - the company vending machine has no organic protein snack bars - things just got serious after all.