Playing at the cutting edge of technology (Under the Radar #10)

Where many companies have closed their R&D playgrounds, Thales’ own Natlab is very much alive and kicking. In Hengelo, Johan de Heer is leading the applied research into brain-computer interfaces.

Brain-computer interfaces (BCIs) have a very sci-fi ring to them. Images pop up of people controlling systems with their minds. A BCI can also be used for more down-to-earth applications, however, like monitoring someone’s fatigue level. This is what Johan de Heer and his team are looking into at Thales Research & Technology (TRT) in Hengelo. “We’re measuring people’s brain activity while they’re working. When their performance starts to lag, this will activate a red light, signaling they need to take a break or someone else needs to take over.”

TRT is Thales’ incubator for new ideas. “The company as a whole comprises 6 global business units delivering solutions varying from scientific exploration technologies in space to addressing cybersecurity attacks from the deep dark web. Each of these GBUs consists of business lines, which, in turn, contain product lines. Orthogonal to this structure is a corporate research organization, TRT, with sites all over the world. In the Netherlands, we have one in Delft and one in Hengelo, which is the one I’m heading,” explains De Heer. “We run all kinds of advanced research projects, relatively independently but always focused on producing results to benefit our GBUs. Like Philips once had its famous Natuurkundig Laboratorium – you could say we’re the Natlab of Thales.”

“Although a relatively small research topic within Thales, BCIs are on the radar of our top executives,” notes De Heer. “Recently, our group CTO, Bernhard Quendt, gave a speech to one hundred of our top managers. He presented six technology areas that he thinks are highly relevant to the company, and brain-computer interfaces was one of them. At the beginning of next year, I’m organizing a BCI hackathon with students from the University of Twente, and our group CTO has agreed to come over on a Sunday, on his way from his home in Germany to our HQ in Paris, to sit on the jury.”

Serious gaming

De Heer first studied electronics at HAN University of Applied Sciences and continued with a master in cognitive science at Radboud University Nijmegen. A one-year stint at the Nijmegen Institute for Cognition and Information (now the Donders Institute) sparked his interest in applied research, which he further developed at TNO. After a PhD in cognitive science at Tilburg University, he joined the Telematics Institute in Enschede in 2001.

“They’d just decided to set up new expertise areas, and being the proactive, enterprising man that I am, I immediately proposed to do something radically different from what they’d been doing for the past five years. I called it cognitive engineering,” recalls De Heer, smilingly. “They liked the idea and made me the head of this new expertise group. With a multidisciplinary team, we started looking into how humans and systems can be better aligned so that they reinforce each other’s capabilities. One of the applications we built was a proof-of-principle of a security camera system that could autonomously zoom in on regions of interest for closer human inspection.”

In 2007, two years before the Telematics Institute was to become the now-defunct Novay, a new business opportunity presented itself – although not the one De Heer had set out for. “My plan was to start my own company in cognitive engineering and I approached an old acquaintance at Thales to land them as a launching customer. I got to pitch my idea to the then-CTO, Dick Arnold. The ten minutes he gave me turned into five hours of discussion. We clicked and soon after, as he was about to retire, he asked me to take over his director position at TRT in Enschede, later relocated to Hengelo. So, instead of my customer, Thales became my employer.”

“At the time, TRT was involved in a collaboration with the University of Twente, called T-Xchange,” says De Heer. “When I took over as director, they were focusing on virtual reality in a wide range of settings. Under my leadership, we professionalized the organization and shifted our focus to serious gaming. The partnership existed for another ten years: in the first five, we looked at gaming technology to accelerate collaborative decision-making processes, in the second five at using it for educational purposes. In 2017, the organization continued as TXchange, a wholly-owned subsidiary of Thales providing game-based learning solutions.”

Human-machine teaming

With the serious game over, De Heer and his TRT researchers in Hengelo embarked on a new adventure. “Game-based learning is all about stimulating people to acquire new knowledge and behavior. This triggered a whole train of thought with us, taking us from analyzing in-game conduct to measuring the underlying neurophysiology through bio-sensing.” As their new research topic, the group decided to look into using brain activity to assess human behavior in the interactions with (autonomous) systems.

The new program was kicked off by a feasibility study. “We initiated projects, both on a national and European level, and collaborations with bio-sensing hardware and software suppliers. We started thinking about what setups would best fit our purpose and context. One cleverly positioned sensor is clearly much more user friendly than a headset with 256 wires coming out of it. But where do we put that sensor? What do we measure? How do we interpret the measurements? How do we translate them into an assessment? And then there are the privacy aspects of gathering personal data and the ethical aspects of interfacing with the brain.”

The team’s current efforts focus on applying BCIs in human-machine teaming, as De Heer calls it. “Capturing brain signals to accurately interpret someone’s mental state, visualizing that and providing neurofeedback to reduce the stress levels – that’s basically what we’re doing. As part of a European project, for example, we’re putting people in a crisis management situation with these devices on their heads. While they’re performing their tasks, we’re measuring their mental workload in real time. Upon detection of a dip, their supervisor gets this red light, indicating that an intervention is required. These are the kind of applications we’re looking at now.”

BCIs also have an edge to them, De Heer acknowledges, alluding to the ethical aspects. “It’s also possible to feed an electrical signal back into the brain. Studies show that this can improve cognition, helping people learn better, become less tired or recover faster. The US Navy is already looking into such a bidirectional BCI. And in healthcare, deep brain stimulation has become an accepted treatment for tremors. Though still beyond our focus, it does contribute to the technology’s appeal. I would like to stress that every experiment we run has been extensively vetted by ethical committees.”

Freedom

Where many companies have closed their R&D playgrounds, Thales’ Natlab is very much alive and kicking. For De Heer, that’s one of the company’s big USPs. With sites close to the universities of Delft and Twente, TRT bridges the gap between academics and industry, between scientific research and business R&D.

“Exploring cutting-edge technologies like brain-computer interfaces, without the pressure to deliver immediate business impact – there aren’t many companies where you can do this kind of applied research anymore,” concludes De Heer. “Of course, there are limits to what we can do and yes, our impact window has become narrower, but Thales still offers the freedom to look many years into the future of technology.”

Want to learn more about what it feels like to work at Thales? Looking for a job at a true high-tech company in the Netherlands? Click here for our current job openings!