Virtual chocolate and coffee flavoured alarms: the future of multi-sensory technology

Advance logo 63mm lockup C Screen v03

Born and raised in Adelaide, Adrian David Cheok graduated from the University of Adelaide with a First Class Honors, Bachelor of Electrical and Electronic Engineering followed by an Engineering PhD. His studies lead him to a career working on mixed reality, human-computer interfaces and ubiquitous computing amongst other things. Currently chair Professor of Pervasive Computing at City University London he is also Founder and Director of the Mixed Reality Lab in Singapore. He was formerly full Professor at Keio University and Japan Graduate School of Media Design, as well as Associate Professor at the National University of Singapore. He has previously worked in real-time systems, soft computing, and embedded computing in Mitsubishi Electric Research Labs, Japan.

Interview by Anna Groot, Advance Asia Director

What first lead you to studying electrical and electronic engineering?

As a young boy I wanted to take apart toys, games, watches etc. as I was fascinated by how they worked inside. My first computer was an Apple II and in those days Apple used to publish the full schematics (electronic circuit diagrams) of the computers. I was really fascinated by how all those electronic parts and connections could make a computer and so I studied those diagrams.

I began building electronic circuits (such as radios etc.) as a hobby when I was a teenager and when I went to university I decided to study architecture as I was also really interested in design and fashion. In those days there weren’t any courses that merged technology and the arts like digital media programs dothese days, so I decided to do architecture and do both design and technology. I found I wasn’t so interested in architecture and missed learning about electronics, so the next year I changed courses to study electrical and electronic engineering.

What are you working on at the moment?

I’m currently working on an area called multisensory internet communication, which is internet communication that involves all of our senses including touch, taste and smell. Humans will be able to experience new types of communication environments using all of the senses, where we can see virtual objects in the real environment, virtually touch someone from a distance away, and smell and taste virtual food. We will be able to communicate our feelings and emotions to each other more effectively with all of our senses. In order to do this, I’m working on new technologies that allow digital transmission of taste and smell. We have developed a digital taste interface that can produce basic tastes using electrical and heat stimulation. By varying the magnitude and frequency of the electric current and heating or cooling the tongue, the users can feel artificial taste sensations such as sour, bitter, sweet and salty on their tongue.

This is done only through electrical methods without any chemicals involved, which ultimately allows taste to be transmitted through the Internet as a stream of digital signals. I’m working on ways to digitize our sense of smell. This will lead to an interface which can produce a range of smell sensations without any chemicals, hence allowing different types of smells to be sent through the internet.

Even though humanity has never been connected so much before, we can often have a lack of understanding of real feelings or sense of presence between the sender and receiver. The metaphor of communicating through a screen or window or glass can limit the sense of immersion and limit the ability for humans to communicate effectively.

In traditional human communications, body gestures, the physical environment, and touch can often more deeply explain the intended message and provide intrinsic information, which makes for a more rich communication exchange.

Furthermore, we often communicate emotionally using all the senses simultaneously, including sight, touch, sound, but also through taste and smell, such as sharing a meal together or cooking for a partner. We thus need to create fundamentally new forms of media to connect humans in the physical world and through the virtual world, not just in the transmission of information and verbal communication, but through meaning and nonverbal communication to increase the sense of telexistence using all the senses. This will allow more opportunities for people to make meaningful exchanges using media in both the physical and virtual world.

There will be various novel research trends and standards from the study of feeling communication. At the fundamental level, we need to develop new theoretical models of communication that unleash the potential for innovation in tele-communication from the physical media through the virtual world. Human communication habits and preferences are continuously changing and evolving. A contemporary model includes the role of media and user context and provides for a model that recognizes the more complex context of the communication process and the possibilities of new media being truly extensions of man.

“recent studies show subjects using the sense of smell to determine the emotions of another person in much the same way as ants use pheromones.”

Researchers need to go beyond the traditional approach and focus on human emotions, feelings, and nonverbal language as key components in the communication process. Recent studies have helped to illustrate that human senses are more acute and versatile than expected. For example, recent studies show subjects using the sense of smell to determine the emotions of another person in much the same way as ants use pheromones. This type of research is just beginning to unfold new mysteries of human perception, which shows the potential for a new and more meaningful sense of presence with these new media technologies.

I aim to develop new technologies related to multimodal sensing and actuation to give the user more definition in their experience in the co-space environment. Visual, Auditory, Haptic, (Olfactory) Smell, and (Gustatory) Taste are the five sensors that humans use for environmental sensing, and communicating emotional feeling. In addition to traditional communication through telephone and video-conferencing, the use of smell, and taste communication will enable a new paradigm of tele-communication. It is a field, which still presents great technical challenges which can lead to early technical breakthrough results.

Based on these inventions, I believe such a multisensory telecommunication will allow new forms of collaboration and learning on a global scale. I am particularly interested in how children will grasp these technologies to make new innovation and value creation. I am thus in in the process of examining how to nurture and inspire young children to create new value propositions that will benefit their individual selves, communities and countries. In the 21st century the democratization of communication tools may allow emerging markets to make creative leaps into new business and industry. We can view young children in developing countries as creative innovators and ambassadors of new technologies, rather than passive end-user consumers. Thus in this aim, I am creating design applications and workshops with the use of new media technologies for children in local schools.

In summary, my research goals are to produce novel communication technologies and telexistence paradigms to allow a presence of all our five senses, as well as non-verbal and emotional communication through digital networks and the physical world of humans and devices/gadgets. I have great passion for making new inventions that can affect families, society, business, and the environment in the future in a positive way.

What are some of the differences you have noticed between students in Australia, Japan, Singapore and the UK?

I found Singapore university students to be generally smart as a whole but somewhat limited in creativity. I think this is because of the Singapore school system being very rigid and focusing on hard sciences. I also found that a lot of students were not really interested in their university courses, but were going through the motions to pass the exams and do well in the exams. In other words they wanted to rote learn for the exams and do well in them, but not really have an intellectual interest in the subjects. I had little contact with Singaporean students at Ph.D level in my lab at the National University of Singapore, because almost all the Ph.D students were foreigners.

In Japan I found that students were really creative and had a strong sense of curiosity. Some of the most interesting and exciting new ideas I have found come from students in Japan. They were willing to think differently. However I found in Japan students were often shy about expressing themselves or asking questions. Also I think because of the problems of the Japanese economy, quite a few university students didn’t really take their studies seriously. They were more concerned to do enough to get a job in a good company.

In UK the students are very international, because people from all around the world want to come and study here. Because of this they are competing with the world’s best and so the good students in the UK really are great, passionate and intellectual.

What are some mixed reality electronic trends you are seeing right now that really excite you?

On the commercial side I think it is very exciting that Facebook acquired Occulus Rift – this means that an influential company such as Facebook sees that Virtual Reality will be the future of the Internet, and they have a lot of capital to turn it into a reality.

On the research side, scientists are beginning to connect electrical circuits to neurons (mice, insects). I predict that with the rapid exponential increase of technology we will see a direct connection of mixed reality content to our brains, without the need for external actuators in our lifetimes.

Are there any developing trends within robotics that scare you?

Not really, I believe that in the near future robots will become part of our home life, family life, work life, just the same as Internet. Robots have an immense positive application to be friends and companions. For example there are many elderly people who have almost no contact with family or friends, leading to loneliness and depression. Imagine if we were able to create a grandma or grandpa robot to not only help them in their daily life, but to be their friends and companions.

How do you think we will be using technology in ten years that we are not doing today?

We are going to see a merging of human and machines (mentioned above). We will have direct brain connections to the virtual world and internet. Similarly more and more of our bodies will become machines (already people have implants and artificial limbs and this will only increase). Soon we will not be able to easily draw the line between human and machine / robot.

What has been your favourite project to date?

Human Pacman and Poultry Internet

If you were able to change or influence one aspect of Australia what would it be?

I think the way we have treated and continue to treat Aborigines is quite dreadful. We should have a treaty like New Zealand and America made with the native population. Also I think the way migrants are treated like a prisoner and put in faraway camps is quite terrible (we don’t really see that in the UK or Europe).

If you weren’t doing what you do now, what would you be?

I like to be on the blue sky, cutting edge so whatever the field I would like to push the boundaries be radical, and explore the unknown.

Who are some of your role models?

Nikola Tesla and Alan Turing. I was also a huge fan of Paul Keating when I was younger. I really appreciated his radical visions, eloquence with words and wit.

Which part of the world do you think is leading the way in technological innovation?

In the past I would have said Japan, but now I would say USA. After the tsunami disaster Japan had to use US military robots because the Japanese robots were not up to the task in the disaster areas. Before Docomo was the leader of mobile phone technology, now it is Apple.

Do you have any plans to return to Australia?

Yes, always. If someone offered me an interesting opportunity I would be on the next plane!

What could Australia learn from the UK and vice versa?

Even though Australia is a migrant country the UK is actually much more international. London is an incredible metropolis with all the cultures of the world. Australia should embrace much more people from around the world.

Do you have a guilty pleasure?

Dry Martini. Shaken not stirred. :)

What do you most enjoy about living in London?

It is an incredible city with so much culture, museums, history, art, theatre, entertainment. It is just incredible to be able to walk into so many world leading museums or visit famous palaces. There is also so much to do for kids in these places, so I have had an amazing time with my daughters in London. Every day and every night there is so much on in London.

Do you have a favourite spot in the world?

One of my favorite things is to be in an outdoor Japanese Onsen (hot spring) when it is snowing. It is so refreshing to be hot in the hot spring and have snow falling on your head at the same time.

Do you have a life motto?

To achieve results there is no easy way, we must work hard. There is a kind of joke, but kind of true. The first 90% of a project takes 90% of the time, and the last 10% takes 90% of the time also – i.e. we need to work hard and persevere to make success.

Do you have any advice for students about to embark on a career in electronics?

Make sure you work on something quantum step and not incremental. People will remember work that is quantum step and breaks barriers. Be radical.

– See more at: