There’s a traffic jam on your usual route home from work due to an anomalous event happening right now, so you receive a notification about the jam and a suggested alternative route. A building is on fire, and the fire brigade turns up before anyone can make a call. A terrorist attack is looming, but it is discovered and averted by the police. 

These are all scenarios that could become reality with an EU investment in the development of new, intelligent surveillance technology for large cities. 

"Our goal is to effectively manage the flow of Big Data from cities. This is one of the most important challenges of our times. It’s about how we acquire, analyse, store and visualise data from, possibly, thousands of cameras, sensors and microphones to give as much value and benefit for citizens as possible," says Alexandros Iosifidis, Associate Professor at the Department of Electrical and Computer Engineering, Aarhus University.  

The name of the project is Multimodal Extreme Scale Data Analytics for Smart Cities Environments (MARVEL) and it has a budget of EUR 5,998,066. 

Ultra-fast algorithms acquire data from all corners of the city

The goal of the researchers is to achieve real-time data acquisition and continuous analysis establishing a novel distributed and secure Edge-to-Fog-to-Cloud (E2F2C) ubiquitous computing framework in a smart city environment.  

This will make it possible to acquire and process data from many different sources much faster and much more efficiently than is possible today. 

"European cities are growing as a result of urbanisation, and we’re constantly getting new digital opportunities to bring data into use. There are cameras, microphones and umpteen sensors all over the city – not to mention all the smart phones we’re carrying around with us. Our task is to bring all this together and give back value to the city and its citizens," says Lukas Esterle, Assistant Professor at the Department of Electrical and Computer Engineering, Aarhus University.  

At the same time, the methods that researchers are developing will solve some of today’s major ethics and privacy challenges. This is because it will no longer be necessary to store sensitive personal data as data analysis takes place using ultra-fast and extremely advanced algorithms with no delay. 

"We want to create new methods that can reap the benefits of the digital transformation of cities, but this means we have to solve the security problems. How can we get most value out of the colossal amounts of data we have at our disposal without compromising the rights of citizens? This is an important part of our research," Lukas Esterle says. 

ALSO READ:Grand textile project to make Denmark circular frontrunner

Artificial intelligence spots the unusual

And what should we do about the enormous flow of data from the city floating around on the internet? How do we find connections and patterns that can transform into useful insights for the authorities, businesses, the police, hospitals and, not least, the people living in the city?

The answer is artificial intelligence. Among other things, the researchers will train their algorithms to identify unusual events; so-called anomalies in videos. 

This could be a sudden loud sound on a square in the city centre, which, combined with images of smoke and variations in people’s movement patterns, could be identified as an explosion. 

In the long term, it will also be possible to predict events such as a terrorist attack or a traffic jam by using the same methods to register deviations in data patterns. 

The researchers are now accelerating the development of technologies for deep learning to make it possible to model abstractions in data at a very high level, thereby spotting patterns and connections that we would have never discovered without computer processing power. 

"It's about connecting the entire city's massive data flow from different sources and learning what’s normal, and what’s not normal. This will eventually enable us to make reliable predictions about events and behaviour," says Alexandros. 

In the project, the researchers will demonstrate the potential of technology in three real-life European use cases.  

Three cases

In the project, the researchers will demonstrate the potential of technology in three real-life cases. 

One case will be in Trento, Italy, and it is about surveillance in public parks, and city squares for detecting anomalous events in real time. 

Read more about the experiment in Italy

The second case will be in Malta and it will look at using artificial intelligence to monitor traffic and road junctions towards safer roads.  

Read more about the experiment in Malta

The last case is an experiment at the university of Novi Sad, Serbia, in which visual and acoustic data, e.g. from drones, will be used to test the possibilities of territorial surveillance. 

Read more about the experiment in Serbia