Statistical Data Analysis

Parallelization: A Solution to the Big Video Data Problem

During the last decade there has been a surge in the use of video for security and surveillance (both homes and public buildings such as airports, car parks, store and offices, for perimeter protection and monitoring roads and other transport systems). This was largely driven by the drop in price, and miniaturization, of the hardware devices including cameras, computer processors, and wireless communication devices. At a more sophisticated level, military and defence organisations are making greater use of a range of sensors that generate images and video, from electro-optical, infra-red, signature aperture radars (SAR) to traditional high resolution cameras. These are usually airborne or space/satellite based, but can also be ground-based, moving vehicle based or even handheld.


Vast quantities of video material can be generated at low cost and very fast - the technical challenge is how this raw information is used and analysed without the large human costs of monitoring and interpreting. The raw data is in pixel format while the useful, human-understandable information is usually in the form of scene-related features that are of a higher order of aggregation than pixels. For example, two images may have the same resolution of megapixels but different information in terms of objects of interest. Therefore, transforming autonomously and fast (in real-time) the low-level pixel-based data stream into meaningful and intelligible information in terms of objects, behaviours, story boards etc. is a difficult challenge, especially when it needs to be done on portable hardware, without direct human supervision and involvement.


Technology is now under development and being applied that can detect moving objects, detect 'novelties' (objects that may be stationary, but were not there before); segmentation of images into regions and objects (e.g. identifying roads, buildings, fields, rivers, railways etc.); track moving objects; classify objects into predefined classes (e.g. cars, motorbikes, lorries, tanks etc.); grouping objects into clusters and analysis of behaviour of objects of interest (e.g. lane change or turning by cars, speeding or acceleration/deceleration, anomaly detection, etc.).


Analysis, however, still continues to be offline - so not real-time. The reason is simple - a still image of megapixel resolution (even more with gigapixel) is represented by millions or billions of data items (each pixel of a colour image is represented by three digits itself); a video has usually 25 frames per second, making 25x3=75 times per second the millions or billions. Video footage with a length of just 12 seconds generates billions or trillions of bytes.


A possible solution could be 'parallelization' - where a huge task is broken down into smaller elements and dealt with by computers in parallel - but demands larger or more powerful (and power hungry) hardware to be used (such as the new Graphics Processing Units being developed by US firm Nvidia for example). However, when the requirement is to have a portable device rather than a rack of computers or HPC (high performance computers) and to analyse the video in real-time then the challenge is still open. It's one of the special cases of the so called "Big Data" problem. For the moment the challenges are being addressed - and have been addressed - by still making use of off line analysis, and by involving a human operator to an extent for determining user- and problem-specific parameters, analyzing the behavior and making decisions.


Current work is tackling the need for real-time analysis, on board (for airborne or moving cameras), with a human operator only needing to start, stop and supervise a whole bank of autonomous intelligent video analytic systems. Lancaster University is a world leader in the area, with original and patented technologies built on some ground-breaking results, now awaiting opportunities for commercialization. Our work has achieved a few orders of magnitude faster processing compared with original US research and patents which has allowed for processing of full High Definition video on board of portable single card computers weighing just dozens of grams. This can still be further 'parallelized'. This has been programmed, debugged and tested with hardware and laboratory environment in a field test earlier this year in May, and is due to be tested further this month [September] in a specially set up event within the MoD funded AURORA project.


The long term potential is exciting. It will be perfectly possible in the near future to have systems that can autonomously detect, analyse and communicate remotely only the most important excerpts (something like automatically extracting the highlights of football games made in Match of the Day) from long videos collected routinely from airborne, stationary and other moving cameras.


Wherever there is video capture, there is a need for intelligent analysis and autonomy to make that video of value - with a huge range of applications in cars, homes, public and commercial properties, transport networks, the media, manufacturing and engineering.



Professor Plamen Angelov is Chair in Intelligent Systems, Data Science Group Leader of the School of Computing and Communications at Lancaster University; and Director of EntelSenSys Ltd, (a spin-out company of Lancaster University)



« Top Tips: Can a Lack of WiFi Really Be Good for Business?


HIPAA Compliance and Privacy Issues »
IDG Connect

IDG Connect tackles the tech stories that matter to you

  • Mail


Do you think your smartphone is making you a workaholic?