ThoughtSpot defines shape of the modern data stack

We are now being sold so-called ‘modern cloud’ systems, modern application platforms, modern approaches to AI and modern data and modern data analytics - so what makes data analytics modern and what ingredients go into this new mix?


Data is modern. In a sense of course, it’s not i.e. data has been around since people were carving notches into rocks and caves to create some kind of historical record or note. But when we talk about data today we are obviously talking about digitally created information assets that we use in contemporary IT systems.

It’s important to note this basic truism because the data business (as a subset trade inside the IT industry at large) is always ripe for reinvention. Just as companies are now talking about so-called ‘modern cloud’ systems (quite what old fashioned cloud was nobody seems to know), we are also hearing vendors lay claims to modern application platforms, modern approaches to AI and, perhaps most over-arching of all,  modern data and modern data analytics.

What is modern data analytics?

Modern data analytics is all about ease-of-use and ubiquity of access. This is data analytics built by data analytics engineers, but designed to be used by almost anyone in the business, often through the use of natural language search functions.

Cloud data analytics firm ThoughtSpot CEO Sudheesh Nair explains the modern age of analytics as a progression point onwards from the way we did things before the turn of the century. This was a time when enterprises started with their database (Oracle, SAP, IBM or other), then moved copies of data that they needed to analyse to a specialised analytics service (Alteryx is one example, but there are many others) and then presented that analytics to business managers in a visualisation dashboard (Tableau and Qlik are obvious examples, but again there are others).

Finally, you might get a dashboard-style business report as a result of these ‘old traditional’ processes.

Dashboards are dead, apparently

ThoughtSpot asserts that ‘dashboards are dead’, largely because its platform is designed to offer Live Analytics (CAPS intended, this is an action and a brand name) of data in motion.

“The chances of [any company] building a static business dashboard capable of delivering all the queries that a modern enterprise needs for live operations in today’s fast-moving business markets is akin to winning the national lottery,” said Sean Zinsmeister, ThoughtSpot SVP of product marketing, speaking at the company’s annual user convention this summer 2022.

What ThoughtSpot is doing is arguably quite bold; the company still refers to itself as something of a start-up, a term it may drop soon as it approaches the 1000 staff member count. The company is aiming to put its Google-search-like data query functionality into the hands of every business person so that we can all enjoy ‘self-service’ data analytics and ask questions of the company dataset. Governed by an appropriate level of policy guidelines and access privileges - we don’t want cleaning staff querying company growth stats - the data culture being promoted here will encourage everyone to start using data in their day-to-day jobs.

What does self-service analytics mean?

This term self-service is comparatively new and the IT industry has picked up on it to reflect the consumer choice power that we now all personally enjoy through everyday apps. But what does ThoughtSpot think it means in terms of data analytics?

“We need to remember an old saying: to get better answers, you need to ask better questions. When it comes to providing self-service analytics we can engender and encourage better questions if we ‘open up the aperture’ on data analytics and enable anybody in the business to ask any kind of question of the corporate data stack. Being able to interpret human language search terms and create data analytics visualisations to provide answers for working live businesspeople,” explained ThoughtSpot’s Zinsmeister.

We can also note here that in the new world of ubiquitous democratically shared data analytics, any search should also be able to build upon itself and add to the information already searched for (by the initiating user, or from others inside the company) with additional layers. As a random example, we might ask about the weather for the next month and get a temperature forecast, but a secondary search might also make sure that we include humidity stats and pollen count.

Analytics engineers need love to

Where all of this brings us to is the rise of the analytics engineer position as a formally defined job description. Given the rising complexity of data pipelines, this person will be tasked with ‘building data’ inside modern data stacks in much the same way a software programmer would build applications i.e. engaging in testing, integration, version control and so on.

What that means in real terms is the use of dedicated data modelling languages (in the context of this story ThoughtSpot does offer ThoughtSpot Modelling Language) that are both scriptable (so they work in way very close to many other language structures) and editable as analytics ‘objects’ are created, changed, developed, used and eventually retired.

As enterprises work to operationalise their data analytics and push the acceptance of a so-called data culture throughout their business, the analytics engineer is going to find their role comes into increasing focus. The provision of self-service data analytics tools such as those from ThoughtSpot need to serve end users, obviously… but they will also need to look after employees and stakeholders working on the back end, which in this case are the analytics engineers.

The modern data stack might look a lot like the traditional data stack unless we uncover some of the ingredients tabled here and start to think about how much access everyone could soon have to useable data analytics.

The scope of the modern data stack will also now widen. ThoughtSpot co-founder and executive chairman Ajeet Singh has likened a company that only looks at its own data to an autonomous car driver that only uses the rearview mirror. The point being that autonomous cars can share road or traffic information with other vehicles and build up a wider picture - so that’s how organisations should be steering themselves as well with a good 360-degree view of the world.

Will everyone start embracing data analytics and think of these functions as a natural part of business life? The answer might well be yes, after all, nobody really used email or applications at the end of the 1980s… and now we all take these technologies for granted. Things move a lot faster now, we probably don’t need to think in terms of 50-year adoption cycles either.

Whether all this modern data analytics will lead to some form of IT postmodernism is one for the philosophers and ‘self-referential epistemological relativism’ doesn’t work well for a tech trade show T-shirt either, we’re probably safe for now.