Is Visual Search Ready to Revolutionise Retail?

Using images to search through products has long been mooted but there remain challenges alongside promise

It was 15 years ago in July that advertising legend David Ogilvy died, leaving a legacy of quotes, anecdotes and advertising theories. The man who reputedly referred to WPP boss Sir Martin Sorrell as that “odious little jerk” was an advertising master, using a mix of research-based guiding principles and creativity to drive successful campaigns for major clients, including Rolls-Royce and Shell. Ogilvy is credited with discovering that the use of images in advertising massively increases the prospect of sales. Readers look at images first and then scan downwards. It sounds obvious today but it still has significant relevance to web design, content marketing and, in particular, the emergence of visual search.

The abundance of online content has made it increasingly difficult for businesses to stand out from the incessant noise of news, features, viral videos and images, Tweets and Facebook updates. The competition for web user attention is fierce and making sites sticky is increasingly difficult, so the clever and relevant use of images is crucial to grabbing but also keeping attention, isn’t it, particularly for retailers?

“Yes of course,” says Alex Straub, CEO at Wismar-based visual search and fashion industry firm Empora Group. I first met Straub in about 2006 after he had developed PIXSTA, a visual search technology company aimed at the fashion industry. It struck me then that while this was a slick and interesting idea, it was a bit ahead of its time, perhaps solving a problem that didn’t exist. PIXSTA now resides in the Empora Group of fashion technology apps and services, underpinning the notion that buyers, particularly on mobile phones need a new approach to purchasing.

In effect, it’s taking the same Ogilvy principles on images and applying them to finding relevant information and products. Visual search has taken its lead from the idea that our brains react to images more quickly and process and store information more efficiently as a result. But what does this really mean and will it really help consumers?

“Nobody wants to go through vast libraries of products to find what they are looking for,” says Straub. “This is where visual search comes in, by executing basic search tasks that are immensely helpful for consumers.”

By that, Straub means helping consumers find more relevant results based on visual rather than textual search terms. Linking images together through algorithms to enhance the search experience is unsurprisingly a goal for Google (which bought UK-based mobile visual search technology firm Plink in 2010) and more recently it is a target for Pinterest, following its acquisition of VisualGraph in January this year. Also, in July 2014 Twitter acquired visual technology firm Madbits. It’s a hot startup area too with a quick search on AngelList revealing 32 companies developing visualisation-related search technologies.

So far the most successful applications seem to be in fashion with Straub’s Empora collection (“half a billion page interactions in which visual search played a role”) and Toronto-based Slyce, although the latter has recently moved into other areas with a number of major US retailers already trialling or integrating the technology.

“Fashion is a dominant market,” says Slyce CEO Mark Elfenbein. “However, home décor, consumer product goods advertising [for example] hairstyle and makeup products, and consumer electronics are all great use cases. There are also many uses of the technology for contextual search outside of purchases. You could snap a photo of an item to instantly add it to your mobile registry or to generate a mobile coupon; take a picture of a pair of jeans and get a Levi's coupon for example.”

It’s an interesting concept. Slyce’s Shazam-like technology is aimed at retailers, brands and publishers that want to reach out to consumers more effectively, especially via the mobile.

According to Justin Speake, CEO of technology analyst Bloor Research, visual search will “add to the whole retail omni-channel experience that so many are now aiming for. Being able to point your mobile device at something, find it, and then buy it, switches the process flow.”

So the real power of visual search is its ability to knit relevant images and deliver results to users that can then interact with retailers of the products seamlessly. It’s sort of like being in a real shop?

“Yes, certainly,” Straub says. “At Empora we’ve experimented with this idea and we have plans to introduce the feature into the fashionfreax apps and Empora related product search engines like”

Murray Grigo-McMahon, design strategist at data visualisation firm Qlik, went one step further, claiming the online shopping experience is already more akin to the real shopping experience but he is concerned that with improved technology and processing power retailers may revert to old ideas.

“Let’s hope want don’t return to the 1990s ideas of 3D shopping malls that we aimlessly trudge around, no matter how cool Oculus Rift becomes,” he says.

Here, here. But we surely don’t need to do that if visual search is all it’s cracked up to be.

“It completely depends upon the generation of the audience, who it is aimed at, and the context it is used in,” says Speake at Bloor Research, “but, yes, potentially visual search could increase online sales and enhance the customer experience with a retailer.”

So how reliable and accurate is visual search?

“Visual search definitely requires better pattern recognition to improve accuracy,” says Speake, “but the biggest issue is the creation of new user experiences that take advantage of this, and, of course, taking it to market. To get a good impression of scale, you need to change how people interact with the visual experience and this requires some really simple applications that change people’s mind-sets.”

While Elfenbein at Slyce claims his technology is currently about 90% accurate, the real issue, it seems, is content. He reckons Slyce is an enabler, particularly for bricks-and-mortar retailers to compete with internet retailers. But it is reliant on the brands and retailers themselves to deliver the goods, so to speak.

“The offline experience is seeing challenges at the hands of entities like Amazon and eBay and the Slyce technology effectively allows retailers to compete. With visual search you do not need to be in the retailer’s physical or digital store to have access to the products. The entire world is a showroom now, particularly with ‘snap to buy’ functionality.”

So really, it’s all about the mobile phone?

“As an access device to retailing, yes,” says Straub. “I am 100% certain offline and online retail will be determined by what is happening on the mobile phone, but also social networks helping consumers to find the right products.”

Grigo-McMahon at Qlik agrees but with some caveats – the results are everything he says, and if the result set is too vast, the machine has to step in.

“Personally, I find browsing through a sale rack tiring. If that sale rack is on the scale of Amazon or eBay, I’m going to need help. I can process the set in front of me, but the machine will need to create smart sets for me to work with. I don’t want it to suggest the ‘answer’ as I know the subtlety and ambiguity in what I asked for… I just want it take out the cognitive overload and burden. A few people may know exactly how a Cuban heel differs from a kitten heel, but a large amount of people will not know these terms but will understand them when they see them. That's the approach we take to search at Qlik: it’s about enabling the human to solve the ambiguity in the results, as that’s something we humans are very good at.”

And that’s a key point and a potential concern for the visual search technologists. How do you recreate the human ability to analyse results? A myriad of recommendation engines have tried it but few if any have really nailed it. It’s undoubtedly still a big challenge but are we perhaps trying too hard? Should we just accept that this is a step forward and give it a go?

“Ultimately, visual search is infinitely more useful for searching against a component of an image, or frame of video, to find out information about something, such as cross-referencing against another information pool, for example, the internet, criminal records, or a customer relationship management database,” says Speake at Bloor. “This is where the market will slowly move, and that is where the value will be discovered, as it becomes more widely adopted.”

At the end of this is of course the consumer, the target, the Holy Grail. Will consumers get it and use it?

As Ogilvy once said, it’s important to remember, “The consumer isn't a moron; she is your wife.”


Marc Ambasna-Jones is a freelance writer and communications consultant that has written about technology trends and issues for over 24 years for national newspapers, consumer and business magazines. He can be found on Twitter @mambjo