The focus of digital technology in the last few decades has neglected human hands and bodies to a large extent. Our thoughts and feelings are strongly connected to the gestures, postures, and actions we perform. I aim to push you — as a designer — to think outside of the zone of screens.
I’d also like to ask you to start thinking critically about current technologies; touch and motor skills need to be taken into consideration when designing your very next product. Allow me to explain why.
Less Haptic Stimuli, Less Experience
According to Finnish neurophysiologist Matti Bergström, quoted in a lecture of Sofia Svanteson:
“The density of nerve endings in our fingertips is enormous. Their discrimination is almost as good as that of our eyes. If we don’t use our fingers during childhood or youth, we become “fingerblind,” this rich network of nerves is impoverished — which represents a huge loss to the brain and thwarts the individual's development as a whole. Such damage may be likened to blindness itself. Perhaps worse, while a blind person may simply not be able to find this or that object, the fingerblind cannot understand its inner meaning and value”.
Hold, Push, Swipe, Tap
If you end up as a typical white-collar worker, you’ll probably spend a significant part of your day looking at your screen, without any possibility of physically touching the things you work with. How much time do you spend on your computer at work? How much time do you spend on your phone afterwards. What about during your spare time: What do you do during those hours? Hold, push, swipe, tap.
The word “touch” is in the word “touchscreen,” but tapping and swiping a cold flat piece of matter basically neglects the sense of touch. You are capable of experiencing only a fraction of what your sense of touch allows you to during the long hours of manipulation with touchscreens.
What actions do you physically perform with your body? Perhaps you are not a very active person. What posture are you usually in? What kind of impact can sitting over the screen of a mobile phone or computer all day have on a person? Pablo Briñol, Richard E. Petty and Benjamin Wagner claim in their research article that your body posture can shape your mind.
“… We argue that any postures associated with confidence (e.g., pushing one’s chest out) should magnify the effect of anything that is currently available in people’s minds relative to postures associated with doubt (e.g., slouching forward with one’s back curved).”
As the theory of embodied cognition states, your body affects your behavior.
Tactile Feedback
Many tangible things are disappearing from our surroundings and reappearing in digital form. They are improved upon and enriched with new functions that would not be possible in the material world. A few examples are maps, calendars, notebooks and pens, printed photos, music players, calculators and compasses. However, with the loss of their material form comes also the loss of the sensations and experiences that only physical interaction with objects can give us. The “… disembodied brain could not experience the world in the same ways that we do, because our experience of the world is intimately tied to the ways in which we act in it,” writes Paul Dourish in his book Where the Action Is.
Different Activities, Different Movements
Consider some actions we perform in the physical world:
I pay for a ticket. I pull my wallet out of my bag. I open it and take out banknotes. While holding the notes in one hand, I draw some coins with my other hand. I give the money to the salesperson.
I confess love. I sit or stand opposite to the person. I look into their eyes. I blush. I say, “You know, I love you.” I am kissed.
I look for a recipe. I choose a cookbook from the shelf. I take the book. I flip a few pages, forwards, backwards. I find a recipe.
Whereas in the world of screens:
I pay for a ticket. I fill text fields. I hit a button.
I confess love. I fill a text field. I hit a button.
I look for a recipe. I fill a text field. I hit a button.
The environment surrounding us, the activities we perform and the things we come into contact with help us to perceive situations more intensely and meaningful. Phenomenologists such as Husserl, Schutz, Heidegger and Merleau-Ponty have already explored the relationship between embodied action and meaning. “For them, the source of meaning (and meaningfulness) is not a collection of abstract, idealized entities; instead, it is to be found in the world in which we act, and which acts upon us. This world is already filled with meaning. Its meaning is to be found in the way in which it reveals itself to us as being available for our actions. It is only through those actions, and the possibility for actions that the world affords us, that we can come to find the world, in both its physical and social manifestations, meaningful.” Another quote from above-mentioned book by Paul Dourish.
Because so many different activities are being carried out in the same manner in the digital world, their value is becoming less clear. I believe that haptic sense has something to do, for instance, with the perception of paying by “real” or by virtual currency — that feeling of something tangible in your hand that you are giving to someone else, compared to just tapping a flat surface to confirm that the number on the screen will be deducted from your account.
Try a simple task. Suppose you want to remember something. Write it down and see how it affects your brain. Professor Anne Mangen, who studies the impact of digital technologies on reading and writing, has shown that writing helps your brain process information and remember it much better. Physical sensorimotor activities create a stronger connection to performed tasks. That’s probably one of the reasons why paper planners are seeing a rise in sales. Sales of paper books are also rising. Giving a digital book as a gift is much less impressive than giving its paper equivalent. This points to an interesting phenomenon. Physical presents just “feel” much better. There is a trend of returning to “tangible music”, which caused an increase in vinyl sales. But are those returns to “old forms” enough? Or can we act also from the current opportunities?
Designing For Touch
How can we create more material experiences in design? What are some tangible solutions, solutions that solve problems through our senses, through our contact with the physical, material world, solutions that let us act in our surrounding as much as possible without using our smartphones or any other flat screens? There are many possible ways to get back to the physical experience.
1. Interact With Digital Technology in a More Human Way.
Make digital information tangible. Interact with it by hand gestures and movements in the material world.
One of the most famous pioneering projects with that aim was SixthSense. Back in 2009, it linked digital devices and our interactions with the physical world. This kind of wearable technology consisted of a camera, a projector hanging on the user’s neck, and color markers stuck to their fingers. The user could dial a phone number using projected keys on their palm, while the camera would record their finger movements. They could read newspapers showing live video news, or draw a circle on their wrist to check the time. The whole principle was to project visuals into the world surrounding the user. With current technology, however, that principle has transformed. The outside world is no longer altered by some projection. The only altered thing is our vision. It’s enhanced by a new layer of augmented reality (AR), by special kinds of glasses, and there is a completely new reality created in virtual reality (VR) headsets.
A more modern example is Magic Leap, a secretive project that connects virtual reality and the “real” world in a mixed reality. You can see objects in your surroundings that are not part of your reality — for example, jellyfish flying in your room. This device is exceptional because it also enables hand tracking. You are able to shoot robots falling from your ceiling, holding a real plastic gun in your hand, meanwhile controlling the interface with hand gestures. This is big progress from mostly sequential activities, which screen interfaces enable the user to do. We are getting there.
Mixed, VR and AR projects could be the future. The good thing is that these technologies are built with a huge emphasis on human behavior, psychology, physics laws and ergonomics. The experience is lived, not just observed on a screen. They are not tearing you away from the natural (or virtual) environment and sticking you in a chair to stare into a flat square. You get involved in the action, immersed in doing things and feeling emotions. All of these technologies bring you experiences. Whether they’re real or not, you will remember them as things that happened to you.
Another advantage is that they make your body move — for example, by replacing your physical screens with virtual ones. They allow you to do your work practically everywhere, possibly on the move as well. Whether you are 3D painting with a virtual brush, throwing squares (a VR game) or organizing your desktop, you are using your fingers, your hands, your wrists and whole body movements. Technology is finally adapting to you.
2. Involve More Sensory Experiences In Your Design.
If sight sensors are already occupied by some functionality, don’t add more visual stimuli. Better to include some haptics, hearing or even olfactory stimuli — thus, creating so-called multi-sensorial design. As noted in their book Product Experience, Hendrik N. J. Schifferstein and Paul Hekkert state, “By now, many different studies have suggested that the greater the number of sensory modalities that are stimulated at any one time, the richer our experiences will be.”
Let’s discuss the topic of virtual reality further. Even though it doesn’t feel like virtual could satisfy the need for material or tangible experience, VR is a perfect example of connecting several senses together, not only sight and hearing, but also touch.
There are a couple of different ways to bring touch into VR:
- The classic primitive controllers
They give you the sense of being present, just like holding a mouse, i.e. it’s one object but has a single point of interaction. Well, it actually has two controllers that are controlled by two hands. Still, the full potential of your hands and ten fingers is not being used in this case.
- Haptic gloves
These enable you to feel objects from VR in your hands. The sensors translate touch sensations into vibrations that enable you to perceive the shape of an apple or to experience rain. You can even feel the release of a virtual arrow. Obviously, all of these sensations are not the same as real ones in their fidelity. But as a whole virtual reality, they pose a question: What does it mean to be real? What makes for a real touch experience — a real touched object made of realistic, tangible material or a real feeling transmitted by neurons to your brain? Is it enough to fool your brain, without even using your hands? This is maybe the moment when we can ask, Are we just brains or whole bodies?
- Combining haptic gloves with material objects
Various games layer VR over a physical playground. One of them is The Void. As a player, you wear a vest with 22 haptic patches that vibrate and shake you at the right times. The idea is that you are playing the game in VR but all of your surroundings are tangible, so instead of seeing four empty walls, you see a large territory around you. A big stone would be perceived as a mountain, and a normal door could be transformed into a magic one. But opening the magic one would feel real because, in the end, it is. All such little gimmicks with sight, touch, hearing and even smell involve more sensory experience and make VR even more immersive.
3. When Designing For The Screen, Think About How the Task Could Be Performed In The Physical World Instead.
How would people act in their most “natural” way?
Time tracking is not always pleasant, maybe because you feel like a robot from constantly checking the time or opening and closing your time-tracking app. ZEI is a great example of tangible design. The developers found a way to get robots to do our job in the background so that we can act more like humans. This time-tracking device is an octahedron (eight sides). Each face is assigned one activity, so you can easily track time spent on different projects just by flipping it. It presents a very natural way to switch from task to task and to turn your attention from one thing to another.
When you’re designing a product, think of how users would perform without it. How do people track their work? Maybe they tend to take notes. How did people used to complete tasks in the past? Did we stand up from our chair and stretch a bit? What if every accomplished task were to be followed by a small exercise or at least standing up, to support users’ health? Many ridiculous ideas will probably appear in that kind of process, but you can get much closer to designing products for humans with such a human approach.
4. Transfer Your Digital Product To Tangible Experiences.
If you already have a product, program or app designed for the screen, think of whether there is some possibility to convert it to the physical world.
Computers made it possible to compose music by using various musical instruments that exist only in the digital world. But the dynamics of physical contact with the instrument cannot be replaced by using a computer mouse. Physically pushing keys on a piano or hitting drums with drumsticks, fast or softly, using mostly just your fingers and wrists, or blasting drums with your forearms and whole arms — these are experiences that seem to be non-transferable to computer programs.
Ableton, the well-known producer of software for music production, decided to create its own hardware, Ableton Push. The second edition of Ableton Push “puts everything you need to make music in one place — at your fingertips.” Push is basically a table with pads and controls that enable you to play drums or pitched instruments on one device. It offers a range of ways to play and manipulate samples, allowing you to capture ideas quickly. No technology stands in the way, and you can physically interact with music once again.
5. Think the Other Way Around: How Can You Upgrade Things That Already Exist With Some Digital Experience?
Classic toys, board games, paper books and notebooks, musical instruments — all of these have served us for decades and are beautiful, efficient and playful. However, many of them are disappearing because they are no longer attractive enough and are unable to compete with the digital experience. Sustain them. Upgrade them with some digital value and experience.
Playing with wooden toys is one of the best experiences for children. Their material and shape develop children’s building capacity and hand muscles. Their simplicity stimulates children’s imagination and creativity. We should not give up these benefits for a flat screen. Studio deFORM’s project KOSKI, a building block game, “connects the physical world and the digital gaming world together.” Physical, wooden toy blocks are mirrored in an iPad app and enhanced with imaginative worlds, characters and stories on the screen. The player physically alters the projected world on screen by manipulating the blocks in the real time.
While we can argue about whether this game develops a child’s imagination, I find it to be a good alternative to current tablet games.
We’re already used to old-fashioned things. There’s no need to teach users new design patterns or ways of communication with hi-tech novelties. Everyone knows how to use a paper notebook. But often when I want to write with a pen on paper, I have to think twice about it. I know that, in the end, it will have to be rewritten in some digital form so that it can be easily shared and stored. This issue was tackled by Wacom with its notebook digitizer. Its solution was the SmartPad, which converts handwriting into digital files. It also offers the possibility to combine pages of notes and to edit them.
Even existing material can take on new qualities when enriched by the digital experience. Mixing together unexpected things can create very non-traditional objects. Consider FabricKeyboard, made by MIT Media Lab’s Responsive Environments Lab. As Meg Miller explains:
"This fabric made from textile sensors allows you to play the keys like one would on a normal keyboard, or you can create the sounds by manipulating the fabric itself — by pressing, pulling, twisting and even by waving your hands above the material. The e-fabric responds to touch, pressure, stretch, proximity and electric field."
Finally, let’s consider one more reason why we should think carefully before letting traditional objects vanish away. They’ve been created from years of experience. They’ve evolved into their current form, one that fits their purpose very well. Think of how usable, convenient and pleasurable many printed books are. The rules of layout and typography from this established medium have been transferred very quickly to ebooks and web design, which are struggling to meet the standards of their physical counterparts. Think also of the non-transferable qualities: the tactile sense of progress, their collectibility, the sensuous aspects.
Some old-school materials are worth keeping, and their development should continue even in the digital era.
Tangible Future
As Andrea Resmini and Luca Rosati write in their book Pervasive Information Architecture:
"We are swinging like a pendulum. Fifty years ago we were rooted in material world. When you wanted to know something, you asked some person or read a book. Then desktop computers became our interface of choice to access information, and now we are swinging back to the real world, but we are bringing computers along. Information is becoming pervasive."
One way to bring qualities of the real world to our daily used technologies is to learn from material things. Another way is to suss out the attributes we are missing in our interaction with screens. Let your senses lead you, and think about a solution that can replace a current discomfort. The classic human-centered approach still works. However, as advanced technologies improve and extend into multiple areas of our lives, we need to think more carefully about what it means to be human. Our bodies and senses are definitely a part of it.
(cc, ra, al, yk, il)