Technology Planning and Analysis

AI in Art: Can a computer be an art critic?

New developments in Artificial Intelligence (AI) are now challenging the age old assumption that computers cannot be creative. Computers are composing symphonies and IBM’s famous AI machine Watson, has recently created a cookbook. In the art world, now a machine algorithm can make connections between paintings and ‘study’ art. We catch up with Ahmed Elgammal at Rutgers University in New Jersey over email to discuss his work with partner Babak Saleh on this machine algorithm to find out more.

How have you taught the machine to study art?

The interesting thing about our recent study on assessing creativity is that the machine did not know anything about art. The input to the algorithm are images of paintings, where we use off-the-shelf computer vision techniques to encode them. The algorithm then reasons about creativity based on similarities between paintings and their date of creation, using the definition of creativity that we used to design the algorithm. So we did not really teach the machine anything that is art-specific.

For earlier studies we did use annotations of paintings, such as style, genre, and their artists to teach the machine about art. However this is not used in this project.

What sort of tests have you done to see how the machine identifies “creativity”?

We proposed a validation methodology, which we call “time machine experiments”, where we change the date of an artwork to some point in the past or in the future, relative to its correct time of creation and recomputed their creativity scores.

We found that paintings from Impressionist, Post-Impressionist, Expressionist, and Cubism movements have significant gain in their creativity scores when moved back to around 1600 AD. In contrast, Neoclassicism paintings did not gain much when moved back to 1600. This makes sense, because Neoclassicism can be considered as revival to Renaissance. On the other hand, paintings from Renaissance and Baroque styles had losses in their creativity scores when moved forward to 1900 AD.

How does the algorithm decide between paintings that are creative and those that are not?

First, let’s go over the definition of creativity that we used. Historically there is an ongoing debate on how to define creativity. We can describe a person (e.g. artist, poet), a product (painting, poem), or the mental process as being creative.  In our work we focused on the creativity of products, and we used the most common definition of creativity of products, which emphasises the originality of the product and its influential value.

The algorithm is based on constructing a network between paintings and using it to infer the originality and influence of them. Think of it as a game: each painting has the same amount of creativity tokens. Then these creativity tokens are passed between paintings based on their similarity and their dates until equilibrium is reached. 

 What are some of the challenges that art historians face in analysing paintings?

Art history is a very sophisticated field that involves studying social, personal, ideological, and historical, contexts. We confess that our view of the influence and originality assessment is based on a century-old view of the problem, but there is a good reason for that. When an infant learns to walk she first crawls, then takes her first steps, later she runs. 

Artificial Intelligence is still at its infantry, especially when it tries to tackle a very sophisticated task such as creativity. So it is very natural, if we want to make the machine understand connections between paintings, to approach the problem the same way Giovanni Morelli and connoisseurial art historians tried to tackle it a century ago.

 What are some of the things your machine has spotted that art historians have missed?

In earlier works of ours, we gave examples of highly similar paintings from different art movements. In our current work, the algorithm we have for assessing creativity presents a rich tool that can be used for analysis of creativity among millions of paintings, emphasising different concepts and elements of arts, and different specific definitions of creativity. 

How fast can this machine analyse paintings compared to an art historian? Can you give examples?

I don't like to compare the machine analysis to an art historian's analysis. Art historian analysis is very sophisticated while the machine is just taking a baby steps in this domain.

Isn’t art analysis extremely subjective?

When it comes to aesthetic judgment, yes it is subjective. However when it comes to judging creativity (originality and influences) it should be objective. 

How have people responded to this type of technology in the art world? Has the response been mostly negative or positive?

We have received very diverse feedback on our work from several art historians, art critics, and artists, ranging from excitement to dismissal. Some art historians became defensive and territorial in their response, which might be because of the provocative titles sometimes used by some journalist when talking about our work.

Will this machine replace the role of art historians?

No. In doing this research our goal is not to replace art historians’ or artists’ role in judging creativity of art products. We were mainly motivated from an AI point of view. The ultimate goal of the AI research is to make machines that have perceptual, cognitive, and intellectual abilities similar to those of humans. We believe that analysis of art and judging creativity are challenging tasks that combine these three abilities. Our results offer an important breakthrough: proof that a machine can perceive, visually analyse, and reason to discover original and influential paintings throughout history.

 What are some of the machine’s limitations?

The machine is doing a very naive analysis that is mainly based on visual similarity at this point. This is even a simplified version of what connoisseurial art historians have done a century ago. The machine has a long way to go before it can make the type of sophisticated analysis that today’s art historian can do.

What’s next for the machine and are there any other projects you are working on?

We are working on several projects in our lab, with the goal of pushing the envelope of computer vision and artificial intelligence by looking at sophisticated problems that involve art analysis. This is a rich domain that combines perceptual, cognitive, and reasoning abilities, and therefore it provides several intriguing problems for AI researchers.


« Singapore wants to be a living lab for smart startups


Globetrotting Dimension Data takes new paths to services success »
Ayesha Salim

Ayesha Salim is Staff Writer at IDG Connect

  • twt
  • Mail

Recommended for You

Future-proofing the Middle East

Keri Allan looks at the latest trends and technologies

FinancialForce profits from PSA investment

Martin Veitch's inside track on today’s tech trends

Amazon Cloud looms over China: Bezos enters Alibaba home ground

Lewis Page gets down to business across global tech


Do you think your smartphone is making you a workaholic?