So, we wondered, what would be the effects of controlling the ambient lighting of a room using our own – or others’– mood state? What if my friends, family or coworkers could “see” how I am feeling in real-time? What if we could design a system that reacts to our mental state, in a way to enhance or balance a certain mental state? What could the mean for digital communications, exchanges and relationships?
And so, we built a such an affect-aware system, using a few Hue Lights (recently launched by Phillips) and an EEG-based emotion detection system. The principle is simple: Emotiv reads the brain activity of the user and translates it into five different emotional states: excitement, frustration, engagement, meditation and long term excitement. Our approach was to use these states as variables to control the colour emitted from the lamp (see emohue architecture). We are using Emotiv + MindyourOscs to pass the messages to a bespoke MaxMSP patch/interface which does all the processing & colour-mapping and then sends commands to the Hue lights (API).
A note here on Hue lights. They work using a wifi hub (aka ‘bridge’) which was designed with private home networks in mind. This proved problematic in our case (institutional network), so to control this entirely remotely, when necessary we are routing the API requests through as server running on a microcontroller (Intel Galileo / Rasberry Pi) and web sockets in the local network  (the red dashed line in the diagram below).
The main challenge was to create a coherent colour scheme from two (or more) variables. This was done as illustrated in the following diagram. It is an extension of Russell’s theory of affect (1980) who suggested the emotions can be mapped in a two dimensional way (the ‘circumplex’ model of affect). Russell’s idea was to map arousal along the y-axis and the valence of the emotion (negative to positive) along the x-axis. Well, attuning this scheme to the variables available from Emotiv, we used excitement as an indicator of alertness and arousal, and frustration in place of valence. Now, from experience and discussions with Emotiv we know that frustration is not a proxy for valence but, rather, a slightly different ‘dimension’ sensitive both to feelings of frustration and to active thinking. In other words it goes up both when a task gets complex and when someone is solving a maths problem. So we slightly changed the meaning of the colour scheme to reflect the Emotiv output… If you’re frustrated/thoughtful and too calm, then the room lights are set to affirm you have ‘the blues’… It may not be the finest distinction of emotional states but it will serve the purpose.
As Emotiv’s emotion readings updates at a rate of 6-7 Hz (but the raw EEG is recorded at 128Hz), and changing the room lighting several times per second would probably have the inverse effects rather than causing any feelings of empathy; most likely it would cause the ‘frustration’ levels of everyone around to go through the roof. For that reason, we are doing some averaging to arrive at a reasonable refresh/colour change rate – experimenting with various refresh rates to find a meaningful one, too often and it’s tiring, too infrequent and it’s too ambient.
Below is a screenshot of the quick interface we built to do all that, which displays the current colour, how long it has been since the last update (the “3” seconds in this screenshot) and the emotional states (black for excitement and blue for frustration) in real time and on average. It also allows to control various lights and experiment with various refresh rates.
So in the coming months, we’ll be exploring how we can use such a system in everyday life or even more experimental conditions, and what it means to mediate our emotions in real time. The next step is to link to remote locations in this way, i.e. connect our three labs and see what happens!
 thanks to CASA’s Steven Gray and Ralph Barthel for all their help and setting up the network system.