The human factors

The trouble with relying on humans is that they’re human. Their job performance depends on many factors that are out of their control and they make mistakes. We already design systems to catch errors before they cause any significant damage – but what if we had a system that allowed us to catch errors before they happened?

Imagine a tired train driver ignoring a signal or an operator struggling to cope with data overload in a metro control room or a fighter pilot being distracted and missing a critical piece of information. 

Understanding what factors cause stress and how they affect the way we can work, learn and react could have a significant impact on how we manage individual and team effectiveness. Bio-sensors could be the key to measuring not only stress, but a range of factors that can influence safety and performance. 

Signals and interference

Kees Nieuwenhuis, human factors and cognition specialist with Thales Research & Technology in the Netherlands, says there are, broadly, three main aspects that have to be covered when deploying sensors that measure humans in real time to give an integrated picture of human performance.

The first is interference – the more the sensors pick up, the greater the likelihood of confusion. 

“It’s signal versus noise,” he says. “Using several infrared-type sensors in a closed cockpit environment, for example, can cause the sensors to actually influence each other’s readings. If you have more than one source tracking the same object, you risk disrupting the signal of every one of those sources.” 

The second – and perhaps most challenging – part is understanding what a clean signal means, in terms of the parameter of mental state that you are interested in, in relation to the information being delivered by the sensors. 

“You can say, ‘I’m looking for stress’ but that means you have to know how stress actually affects a human. What are the measurable signals? And what do they reveal about someone’s stress levels or their emotional state?” says Nieuwenhuis. “And then we have to go beyond a single data source. We have to map two or more data streams simultaneously and understand what their fusion tells us.” 

And these additional data streams could be anything, from facial recognition software designed to determine what each expression means, to testing for specific elements in the blood – such as a specific hormone that acts as an indicator of agitation. 

The fusion is necessary to develop models that code for the parameters that express fatigue, stress, vigilance, attention, mental workload, cognitive control and so on. 

And then the final challenge lies in understanding how such parameters relate to improved or less than adequate performance. For example, what if being a little bit stressed in a job has a positive influence on overall alertness and responsiveness? How advantageous is a relaxed demeanour when operating a complex system? What qualifies as “good” stress? Does lower vigilance affect a pilot’s performance in all the different phases of flight? 

Finding answers to these questions will involve working with external partners – the aviation industry, or a rail transportation client, for instance – to better understand how subjects react under different conditions and how that then feeds into performance. It also means working closely with medical and psychological experts. 

“We need to identify if an operator is capable of carrying out tasks,” Nieuwenhuis says. “It’s one thing to be tired, but to be stressed as well, and then overloaded – we need to knit that all together to understand the inputs and their effects.”

For now, the focus is on developing greater understanding of both sides of the equation: human and machine. After all, any successful system will need to marry the best of both.

You can say, ‘I’m looking for stress’ but that means you have to know how stress actually affects a human. What are the measurable signals? Kees Nieuwenhuis, Human Factors & Cognition specialist