Researchers develop 'thinking' surveillance system

US-Army funded researchers have developed a video surveillance system that aims to use artificial intelligence to predict human activity and flag anomalous behaviour.

The aim of the research is to interpret data from a video-feed and eventually predict the actions of people being observed through semantic analysis and reasoning, Carnegie Mellon researchers said in a research paper presented last week.

"The goal is to approximate human visual intelligence in making effective and consistent detections," researchers Alessandro Oltramari and Christian Lebiere said in the paper.

The research builds on a Carnegie Mellon cognitive model and software, both called 'ACT-R'. Oltramari and Lebiere developed a cognitive engine to recognise activity using parts of the ACT-R  cognitive model.

"We devised the cognitive engine to work in a human-like fashion... trying to disambiguate the scene in terms of the most reliable conceptual structures," they said.

The cognitive engine uses partial matching to, for example, recognise that an object such as a bag is more like a basket than a dog; and spreading of activation to recognise that a bag can be associated with activities such as travel.

In video footage of a person getting into a truck and driving away, the cognition engine can move from machine vision, which detects a truck, a person and both disappearing from the picture, to "a person gets into a truck and drives off".

The cognitive engine is designed to be used as a module in the DARPA-funded Mind's Eye architecture. Mind's Eye is a project to develop a smart camera that uses machine-based visual intelligence.

The researchers have a working prototype of this technology, Oltramari told SC on Monday. The research is for both commercial and military use.

"Our study, which leverages on ACT-R cognitive architecture, is open source," said Oltramari.

The researchers presented the paper, 'Using Ontologies in a Cognitive-Grounded System: Automatic Action Recognition in Video Surveillance', at the Semantic Technology for Intelligence, Defense and Security 2012 conference on Wednesday.

Research into predictive video analysis is also being carried out In Europe. The EU-funded Indect project is developing algorithms for use with existing video monitoring systems. Indect is also developing technologies for threat detection, such as child pornography, in computer networks.

This article originally appeared at scmagazineuk.com