Making sense of wicked data (Under de Radar #12)

Artificial intelligence is taking the industry by storm. But incorporating it into a high-tech system is easier said than done. At Thales, Gregor Pavlin and his team are building a deep understanding of relevant AI methods and translating them into good engineering practices.

“Surveillance systems, for example in defense or security applications, typically combine all kinds of sensors that produce vast amounts of data. This data isn’t just big, it’s also nasty: different types of data, correlated in complicated ways, ambiguous and noisy – meaning that it can be wrong,” says Gregor Pavlin. “This wicked data, as we like to call it, is, in principle, useless without proper analysis. It’s just a whole bunch of signals that in themselves don’t tell much with respect to the decision-making or control problem at hand. But hidden inside this mountain of data, there’s a wealth of actionable information. We’re developing the tools to get that information out. And because that’s an incredibly complex job, we’re using artificial intelligence to help us.”

Pavlin works at Thales Research & Technology (TRT), where he doubles as program manager and senior scientist in the AI group. “We’re a team of five engineers and researchers, including me. The role of TRT is to take exotic technology and translate it into something the business lines can use. It’s not blue skies research; it’s applied science. We follow the scientific advances in that area, identify and study the results that can be relevant to Thales and adapt them to real-world applications.”

“Take the example of an area that’s being surveilled by multiple sensors,” illustrates Pavlin. “A suspicious person is seen entering by a camera but then disappears out of sight because there’s no full coverage. A few minutes later, a human is detected elsewhere in the area. Can it be the same person? Can this person travel this distance in this amount of time? So, we have all these observations, this wicked data that’s often ambiguous and can occasionally be wrong – because sensors also make mistakes – and we’re looking to extract actionable information. AI provides the proper machinery to do that: the tools to create the models representing the problem space, the machine learning algorithms to automatically optimize the models based on the observations and the reasoning algorithms to make sense of everything.”

Deep understanding

It was at the University of Graz in Austria where Pavlin was first introduced to the challenge of making sense of wicked data. “I did my PhD there, on autonomous intelligent robots. These robots have a wide range of sensors on board and they operate in a very noisy environment. So, the conditions are far from perfect, giving rise to similar information fusion problems like those we’re researching at Thales.” In Graz, he also had the privilege to learn about AI and particularly machine learning from the top scientists in the field.

In 2004, after a short stint at an avionics company, evaluating and certifying airborne software, Pavlin came to the Netherlands. “By chance, I stumbled upon a posting for a job at the University of Amsterdam. It was a position in a project with TNO and Thales, focused on the fusion of data from different sources for crisis management purposes. I was a bit hesitant to move, but my wife pushed me to just try it for one or two years. So I did. A year or two later, I switched to Thales and I’ve never left since.”

When Pavlin joined TRT as a researcher, artificial intelligence was still an exotic technology for industrial applications. “It was long before it became the hype that it is now. In those early days, putting your money on AI was still somewhat of a gamble,” he recalls. “But because of the deep dive we had taken into the technology, it had become clear to us that this was the way forward for important applications. It’s very satisfying to see that this educated guess from so many years ago has started to pay off.”

Having this deep understanding is a crucial success factor in creating AI-based solutions, Pavlin stresses. “We thoroughly evaluate promising research avenues. We don’t invent algorithms ourselves, but we take what’s new out there, from world-class researchers, study the working principles and see how and when the methods are suitable for us – sometimes we make a derivative to fit our purposes. Next to this research, we work together with the engineers in the business lines of Thales but also with external partners, to directly apply the knowledge we’re building up.”

True AI

On top of the current AI hype, Pavlin points at a couple of trends relevant to Thales. “Especially when you have these complex problems, you very rarely find one type of algorithm that does the trick. The key is to adopt hybrid AI approaches, combining different tools like deep neural networks, probabilistic graphical models, and so on. Our focus is on understanding their properties and mixing and matching them into robust solutions. And with hybrid AI, we can build systems that require less data for training, so we’re also addressing the trend of frugal AI – training data has to be categorized by hand and by reducing the need for that data, we can cut down on the manual labeling efforts, thereby lowering the development costs.”

Another big trend is distributed AI. “Modern IoT systems combine a myriad of sensors, collecting large amounts of data. Traditionally, all the data is transferred to a central computer or cloud for processing and the results are sent back to the application in the field. Not only is central data collection often very energy consuming, but it also requires communication bandwidths that can’t always be guaranteed, and moving highly sensitive data off-premise is a security and privacy risk. To mitigate these problems, we’re seeing a shift from the cloud to the edge, crunching the bulk of the data close to the application, keeping it behind the firewall, and only transferring processing results – the information distilled from simpler data. Distributing AI in this way again requires a thorough understanding of the algorithms as some can be split more easily than others. We’ve gained a lot of experience in this area and developed a platform that allows for a fast and easy integration of distributed algorithms in the field.”

Pavlin and his colleagues are also putting a lot of energy into engineering with artificial intelligence. “Development with AI is very often mistaken for software development, but it’s a completely different, complementary discipline. It’s all about having the correct model and choosing the right reasoning and learning methods. We’re enhancing the engineering process to take this into account from inception to implementation to maintenance.”

All efforts come together in “True AI,” as Thales has coined its approach to artificial intelligence. It stands for “transparent, understandable, ethical,” meaning that users can see the data on which the conclusion is based, the results can be explained and justified, and objective standards, protocols, laws and human rights are followed in the process. Pavlin: “Our goal is to create systems that can be implemented cost-efficiently and that we can all trust. We do that by building an understanding of what AI does and translating that understanding into good engineering practices.”

Want to learn more about what it feels like to work at Thales? Looking for a job at a true high-tech company in the Netherlands? Click here for our current job openings!