Contextual Statement:

We hope to change the users’ thought patterns to make for healthier thinking and management of stress. We want to show that stress is a tangible thing that you can see and change. We did this by creating a window into the user’s minds.

This project is a mix between interactive art and psychology all tied together in technology. In our display, we want to show people mindfulness without showing them mindfulness. We want them to discover it by interacting with our piece. We creatively brought art and psychology together to solve a very big issue in today’s society. 

“Mindfulness is a skill that allows us to be less reactive to what is happening in the moment. It is a way of relating to all experience—positive, negative and neutral—such that our overall suffering is reduced and our sense of well-being increases. To be mindful is to wake up, to recognize what is happening in the present moment. We are rarely mindful. We are usually caught up in distracting thoughts or in opinions about what is happening in the moment. This is mindlessness.” - (Germer, 2004)

I drew my ideas from different areas and stuck them together. I have some knowledge in neuroscience and psychology; so, I contributed that to the group, I also have experience in circuitry and microcontrollers, and an understanding of design and user interaction. This allowed me to be more creative by drawing my ideas from multiple fields. I did look at a lot of other interactive art displays such as Waterlight Graffiti, art galleries and on the website Pinterest (Interactive Exhibition, 2017). However, all of the other interactive projects I could find were about either directly affecting the viewers’ mental state or the user directly affecting the work. I did not find any showing the users mental state and then changing it, I know something similar exists, however, I have not found it.

Simmons (2014) book Interactive Art Therapy, showed multiple interesting interactive exercises made for psychology. This was interesting to see how they can use similar things to help patients. By reading through these different exercises it helped give me an insight into how real psychologists could use our display. “Interactive art differs from static art such as paintings because interactive art entails reciprocal response or influence between artwork and audience” - (Seevinck, 2017). Overall, from an artistic side and also a psychological side, our project shows a glimpse into the mind of the user and has the potential to change a user’s view about stress.

Conceptual Statement:

Our Brief: “Create an Immersive translation of an individual’s state of mind as an introspective experience” Which summaries to taking someone’s thoughts and emotions and making them into a physical experience, we went with stress in the final iteration. Our main idea is that by making the users’ thoughts a physical tangible thing, they can be just that. By making the stress into a storm that you can control, it indirectly shows how you can control stress in the same way.

We hope to help reduce stress in this stress-fueled world by a metaphor. We have a storm behind a window that reacts to your stress and you can put all your worries into it. By calming the storm, you are calming yourself and the window is like a window into your own mind. We want the user to realize they can separate themselves from the storm that is their stress and see stress as a tangible thing and hopefully, therefore, be able to reduce or manage it better. We want the user to type their worries into the tablet on the desk and watch them appear on the screen and then put their finger on the scanner and try to calm the storm. Thus, changing their view on stress and worries.

The final iteration of this project embodies the concept and brief by incorporating many elements. By using a real window that we are projecting onto rather than a screen with a window and a desk with photos and plants, we are adding to the realism factor making it more believable. We are using heart rate and an input terminal to get an idea of the users’ state of mind and then translating that into the physical real world that is how the projected storm reacts. The storm changes in intensity by getting louder and the rain getting faster, as well as, the addition of thunder.

The brief changed throughout the course of this project as we refined and generated new ideas. The final iteration of this project is nothing like I could have imagined at the beginning. This I think shows that the creative process has been active. To help the creative process we started the project not knowing each other apart from areas of interest. This means we could easily make creative ideas off each other and we all had a different angle to go from and different knowledge to draw off. This helped us think outside the box to best solve the problem and focus on how to make the project a reality. This lead to Storm, the final iteration of our project.

Related Blog Posts Index:

1. Forming a Team for Our Next Project.  - http://blog.epsilum.co.nz/post/164504325489
   
2. What is Movement/Motion? - http://blog.epsilum.co.nz/post/165419136404
   
3. Making a Rough Schedule and Playing with EEG - http://blog.epsilum.co.nz/post/165424187894
   
4. Emotion Idea - http://blog.epsilum.co.nz/post/165424617874
   
5. Storm - http://blog.epsilum.co.nz/post/165428915634
   
6. Brain Science! (Part 1) - http://blog.epsilum.co.nz/post/165707134634
   
7. Progress on The Storm - http://blog.epsilum.co.nz/post/165753821939
   
8. Where Storm Ended Up and My Thoughts - http://blog.epsilum.co.nz/post/167151837779
   

References:

Baer, R. (2010). Assessing Mindfulness and Acceptance Processes in Clients. New Harbinger Publications.
Germer, C. (2004). What is Mindfulness? INSIGHT JOURNAL, 24-29.
Interactive Exhibition. (2017). Retrieved from Pinterest: https://www.pinterest.nz/explore/interactive-exhibition
Seevinck, J. (2017). Emergence in Interactive Art. Springer.
Simmons, L. L. (2014). Interactive Art Therapy: “No Talent Required” Projects. Routledge.

The Finished Thing:

We ended up with a wooden window with a piece of material on the back so we could backward project a storm. This also has storm sounds through a set of speakers. We also have a heart rate monitor and a tablet on the table. The storm changes when you type in words and also changes with your heart rate.

What Went Well:

The project looks completely different to what we originally thought of and kept changing up until the last day. It has even changed since I took this picture.

The addition of a heart rate sensor was originally difficult and inaccurate but seems to be working a lot better now thanks to a mix of my case idea and Liam’s programming skills.

When we decided on backward projection we needed to find the right fabric and the branded stuff is a ridiculous price. Luckily another group was doing a similar thing and they pointed me towards a cheap fabric shop. ROAD TRIP! Kale and I went and looked through the shop for a good while and eventually found it. I also brought double what we needed because if you don’t you’ll end up needing it. I ironed It and staple gunned it to the frame tight and it looked good.

The window was originally thought to be placed in a wall we would build but this changed to a hanging design. We had a few problems here mostly the pole we brought was too small for the stands and the chain was unstable and fell in testing. I managed to fix both of these. The size of the pole I fixed with some cut up puzzle floor foam mats that I wrapped around the pole where the stand connects. I also made the chain a lot safer by turning the hooks that the window hung off, into loops by bending them around, which was a lot harder than it sounds.

After this, we moved it into position and set up the projector. Then I worked out how to change the projector so it was mirrored so everything was the right way around. Then another group decided to take up most of the studio space moving our projector so we relocated and re-setup.

The PC and projector I set up in a way that if the power was cut for some reason it would automatically turn back on and auto start all of the software; at the same time as not sleep, shutdown, or update.

The Arduino soldering also went well and we didn’t have any problems apart from length which just meant I needed to extend the wires a bit. I managed to solder around the case well and make it not look terrible

The last thing on the list is PC. I bought a spare PC in to use and tried to make the cables as tidy as possible making good use of extension cables.

What Didn’t:

I had a huge interest in using EEG for this project and did a lot of programming and research into using it for measuring stress. However, the EEG we were going to use for this project stopped functioning and we couldn't get another in time.

I also spent a large amount of time trying to work with the connect in processing, about a week. It was a nightmare. If the drivers weren't crashing, the libraries and APIs were. Eventually, I ended up just using a webcam and got the same effect but it took a matter of hours rather than days. My end goal was a program that gave a number out between 1 and 10 that showed the amount of movement of a subject. 0 being no movement and 10 being starjumps. However, this code wasn’t used in the final submission.

The stain was a fairly last minute decision and was rushed. I fell it could be better but it will do the job. Either way, it looks a bit better than plain wood. However, some stain go onto the fabric and it needed to be removed (luckily I brought double) I planned to redo it but it was done when I next saw it. It wasn’t ironed this time and wasn’t completely tight. It doesn’t look as good as it did but I hope with the projector on and lights off, no one will notice.

Something I seem to always struggle with is making time to blog. For me, it takes so much energy and I need to get into a certain mood to do it. I still have about 10 other half done related posts in my drafts folder that I didn’t get to finish. I definitely fell I could have blogged more.

The last problem that we are still dealing with is the program freezing. I blame windows and wish I installed Linux like I usually would for a project like this, but, too late now.

What I learned:

I learned a lot more about creative teams, as much as I hate working with people in a group; It is essential to the creative process or at least helps speed it up. Having other people helps trigger new ideas and also weed out bad ones. This works especially well if you haven’t worked together before like I explained in my first ctec503 post - http://blog.epsilum.co.nz/post/164504325489. 

I also learned a lot from the research I did because of this project. I built on other things that I researched in the past, such as EEG, Neurology, and Japanese concepts.

Conclusion:

This was a long project, but, it went fast! I’m glad to say its coming to an end but I definitely had some fun making this with the group, all of whom I didn’t know at all before this project started. A lot went wrong and could be better but overall I am happy with the outcome.

We have a window

We have a projector

Cool things happened

Potentially ditching wall idea 

making it seem like a real storm
eden researching

Before posting more about our project I thought I might cover some of the basics of neuroscience regarding the EEG (electroencephalogram) displayed below.

Brainwave Frequencies:
Each brainwave frequency points to a specific brain function shown in the image below. Each electrode picks up all of the brainwave frequencies. A problem is the received frequencies include frequencies from general interference and these need to be filtered out or blocked with some sort of Faraday cage, generally they are filtered out by the EEG’s software based on your country.

Parts of the brain:
Current neuroscience believes each area of the brain to control certain functions.
From Back to Front:
“The Occipital Lobe is the visual processing center of the mammalian brain containing most of the anatomical region of the visual cortex.”  - https://en.wikipedia.org/wiki/Occipital_lobe
“The parietal lobe functions in processing sensory information regarding the location of parts of the body as well as interpreting visual information and processing language and mathematics.” -http://study.com/academy/lesson/parietal-lobe-definition-functions-quiz.html 
“The temporal lobe is involved in primary auditory perception, such as hearing, and holds the primary auditory cortex. The primary auditory cortex receives sensory information from the ears and secondary areas process the information into meaningful units such as speech and words.“ - https://en.wikipedia.org/wiki/Temporal_lobe
“The frontal lobe is the part of the brain that controls important cognitive skills in humans, such as emotional expression, problem-solving, memory, language, judgment, and sexual behavior. It is, in essence, the “control panel” of our personality and our ability to communicate.” - https://www.healthline.com/human-body-maps/frontal-lobe/male
“The frontal pole is one of the three poles of the brain (along with the occipital pole and temporal pole), and corresponds to the anterior-most rounded point of the frontal lobe.“ It is the frontmost (anterior) part of the frontal lobe - https://radiopaedia.org/articles/frontal-pole
Nasal:
For more spacial resolution a nasal electrode can be used, which is an electron that is placed inside the nasal cavity. This is relatively rare from my knowledge. It is most commonly used in more accurately diagnosing epilepsy patients -  https://www.epilepsyresearch.org.uk/research_portfolio/recording-brain-activity-using-electrodes-placed-in-the-nose/
These areas are located as shown below:

How neurons work:
In certain conditions a neuron fires, this is called an action potential. An action potential is the rapid changing of charges (in the form of ions) that travel down the axon (like a cable for your body). Action potentials can be triggered by a number of things. Such as touch or any of your other senses, Chemicals in the brain, other neurons, or a direct electrical current (these are just a few).
Below is the graph of an action potential in humans, showing the voltage changes over time as the cell depolarizes then re-polarizes.

How an electrode gets a signal:
Any electrical current produces electromagnetic (EM) waves. The electrodes on the EEG are made to be extremely sensitive to these EM waves. The use of multiple electrodes can show what part of the brain signals are being sent and what kind. The more electrodes, generally the greater accuracy. Below are all the most common EEG electrode placements. The first letters represent each of the lobes explained above. 

Best EEG for us:
We decided that the Emotiv Epoc+ would be best for our project due to the locations of the electrodes, the price, and it has an API for processing which we are programming most of the project in. It is displayed below:

If you want to learn more about neuroscience yourself look at mcb80x.org

References:
http://www.measurement.sk/2002/S2/Teplan.pdf
https://link.springer.com/article/10.1007/s13534-016-0235-1

We decided emotions was a bad idea due to ethical concern’s. We are worried people may be offended or upset by us “telling them” their emotions, also emotions are also seen as private in most cases.

The main ways to tell someones emotion are behavioral and expressions, both of those you can fake, however, with EEG you are literally reading their mind. This is also described in the report I mentioned in my last post: (http://ieeexplore.ieee.org/abstract/document/5458075/?reload=true)

Our new idea is to measure stress or rather how active the brain is. So, as the user tries to relax we can change the environment. We want to do this with a storm behind a window, which is along the lines of what Eden was researching with komorebi (木漏れ日) 

There is also another common untranslatable word for a type of light in japanese knowen as kawa akari (川明かり) or light reflecting off a river.

So we basically want a window with a storm and rain projected onto it with sound effects that have different levels that change with the users brain activity read by the EEG. We would also like to use heart rate. I’m going to go and call this ame akari (雨明かり) light off rain.

I thought back to some research papers I read a while ago (Linked Below) about seeing users emotions with an EEG. This lead us to this brief: “Present An Informed environment developed off the users emotions”
I also remembered a device that uses this same idea made in the place where all the strangest things com from… Japan! It is a device worn around the waist and on the head. It is cat ears and a tail that move to the wearers emotions. They basically use the same technology that Murphy lent us, a 1 channel EEG. Because of this I am skeptical to how accurate they are. Here is the video:

Ears:

Tail:

One of the papers that lead me to think about emotions with EEG: http://ieeexplore.ieee.org/abstract/document/5458075/?reload=true

Murphy allowed us to borrow his 1 channel electroencephalogram (EEG) to play with. It measured 3 things, a blink, meditation, and attention. Not enough information to guess emotions but enough to control a few different things with concentration.

After playing with the one channel EEG we went to talk to AUT KEDRI, they have an emotiv epoc that we could use for our project, they just need to speak to their supervisor first. However, I noticed the device was corroded, the electrodes were dry and damaged, furthermore, we would need to submit a full brief and reasoning for needing the device where at this stage it is experimental and we don’t know exactly where our project will go. We do hope to get a rough working model by week 5 or 6 though as the finished project as a experience is due in week 7 of the project.

Our current brief is:
Investigate Mental and Physical Interfaces as an Interactive Experience.

Motion and Movement has different meanings in most fields. In physics, they describe how energy is transferred from potential to kinetic. In politics, they could describe a group of people that come together due to a similar set of beliefs. You can have movements of physical things and movements of non-physical things, for example, the movement of ideas. I’m interested in movements in the brain and how they affect the physical world and how the physical world affects the brain. 

Something that was mentioned in class was teams of people who don’t know eachother think more creatively; however, teams that know eachother work more efficiently. I couldn’t agree more with this statement. It is the reason why when I enter a competition with a strict deadline I will choose people I know. However, if i want to make somethong that is unique and purely creative, i need new minds, both for my sake and theirs. With people you know, you already have synced with how they think and vice versa, whereas with someone new, you don’t know where your ideas will end up. So, i formed a group with people I didn’t really know, had a slightly similar idea, and lastly had a mixed range of skills and hobbies. Thus Sheet 7 was formed.