Executive
Summary
The Leap Motion has made a technological breakthrough by providing
a new touch-free interface paradigm. The Leap Motion device is a portable USB
port that uses motion censor and cameras to control on-screen motions.
The paper examines the historical evolution of how mankind created
the first computer to the kinds of technology we have. We also explore the
current uses of the Leap Motion device and how it has benefitted those who have
chosen to integrate this device into their lives. This report aims to examine
the future considerations on the usage of the Leap Motion device and how it
could potentially aid in the development of society in different aspects such
as education, healthcare and retail just to name a few. This paper revolves
around the Leap Motion controller and how it could potentially revolutionize
our lifestyles.
Why I chose
this topic
With the development of technology and introduction of many
touch-screen devices, it is only with time that people were interested in the
technology portrayed in many blockbusters like Minority Report or Iron Man. The
Leap Motion is a technology still relatively young in its field, providing a
touch-free interface where all movements are controlled only by the movement of
your fingertips. The Leap Motion device, slightly bigger than the size of a gum
pack, is so sensitive that it tracks movements to one hundredth of a
millimeter. This performance could potentially enable many others forms of
technology to come through with the usage of such a device.
Background/Introduction
The
first ever programmable and fully automatic computer known as the Z3 Computer (Schmidhuber)
was built by German engineer Konrad Zuse in 1941. Ever since the first computer
was built, we have come a long way to developing the super machines we have
today. In the technologically advanced world we live in today, it is almost
impossible for anyone to live without a computer. The computer has drastically
transformed the way we live, from the way we perform daily tasks to finding
information off the Internet to socializing with our friends. Human-Computer
Interaction, which basically refers to how we (users) deal or utilize
computers, will only continue to evolve beyond our imaginations.
The
first few computers ever built could only be operated by trained scientists who
possessed the specific skills to manage separate manual action to perform
different arithmetic operation, which means that the computers then were not
for the general public to use. However, all that has changed. The development
of the computer hardware and software technology has made it so much simpler
such that anyone is capable of using computers. With the use of computer mouse
to help maneuver the cursors on the screen, to the development of the touchpads
on laptops to creating an easy to understand interface give us an enhanced
user-friendly experience.
In
addition, the development of gesture-control technology, which includes of
gesture recognition, has helped to improve the experience of using computers.
Gesture recognition uses a camera to interpret body movements mathematically
and then translates those gestures onto a computing device. One example of a
device that uses such technology would be Microsoft’s Kinect sensor. It is a
combination of utilizing a video camera that detects the primary colors – red,
green and blue for facial recognition and other features, depth sensor to view
the room in all lighting conditions and a multi-array microphone that
distinguishes voices from background noises (Crawford, 2010). Kinect is mostly
known for its contributions to the gaming industry that allows games to be
played without the need for controllers. However, it has also developed
programs for healthy living by coming up with exercise routines, weight-loss regimes
or simply by playing sports. Some of the most popular programs include “The Biggest Loser: Ultimate Workout”, “Kinect
Adventures” and the best-seller, “Dance Central” (Lynch).
Then,
this year in 2013, Michael Buckwald and David Holz, has managed to improve the
process of gesture motion technology to come up with the Leap Motion
Controller. The Leap Motion Controller is a small device that measures 0.5 by
1.2 by 3 inches that allows you to control movements on your computer by
tracking the gestures of your 10 fingers. The controller contains infrared
cameras that track the movements of the 10 fingers within 8 cubic feet of space
in front of the computer (Richardson, 2013). The Leap Motion controller can be
used for day-to-day activities such as browsing the Internet, or it could be
used in schools to teach particular concepts, to play video games and to
explore and possibly allow us to do much more than that.
Historical
Perspective (ie where we have come from)
From
when the first programmable computer was devised, mankind has advanced leaps
and bounds in creating the vast number of computer models and complementing
accessories we have currently. Not only that, the way in which we use these
devices are also different from what it was then. With the advent of gesture-recognition
technology, we will soon be able to realize our dreams of becoming like Tony
Stark in Iron Man or John Anderton in Minority Report where just the wave of
your hands allow you to control what you want to see in the space right in
front of you.
Before
we jump into the Leap Motion Controller, we have to go back to basics. When
computers were first used, there was hardly and graphical user interface. In
fact, in order to interact with the computer, users submitted programs by
sending in a series of punch cards where the computer would translate these
patterns using a reader and the results would be ready after several hours and
sometimes it might even take a few days, this process is also known as “batch
processing” (Reimer, 2005). Then in the midst of World War II, Vannevar Bush
working at the Massachusetts Institute of Technology (MIT) envisioned the
concept of Human Computer Interaction (HCI) by introducing Memex in an article
published in 1945. Right after WWII, the American government heavily invested
in computing innovation and their goal was to allow computers to be used by all
people, across all professions and ages. The National Aeronautic and Space
Administration, NASA, which was within the Advanced Research Projects Agency
(ARPA) then gathered the most brilliant scientists from across the country and
placed them in research institutions and among them was Douglas Engelbart. Working
independently of ARPA, he had already been researching on HCI in early 1950s at
Stanford University. Engelbart was greatly influenced by the theories of Bush
and started viewing computers as an “augmenting” tool; this means seeing
computers as a tool that empowers the user. Engelbart then headed the the
Augmentation Research Center in 1960s and the team achieved a milestone by
developing many devices and concepts of “direct manipulation” and this changed
the way we used our computers. Some examples include word processing software,
combination of computer, keyboard and video screen, the computer mouse just to
name a few.
Then
Xerox Photocopiers, gathered programmers and scientists in the Xerox Palo Alto
Research Centre (PARC), and adopted the ideologies of Englebart and created the
first personal computer prototype in around 1970s, the Xerox Alto. The
prototype encompassed the use of the computer mouse, a keyboard and a video
screen. In addition, they also came up with the first graphical user interface
known as the Xerox Star. This prototype sparked many other companies such as
Apple and Microsoft to do the same and come up with their own graphical user
interface (Zmoelnig, 2000) by coming up with Macintosh and Windows.
Even
with that kind of advancement, nothing compares to what is known as gesture
recognition technology. Gesture recognition technology is reading human motions
mathematically by using a computing device and later translating that movement
into signals that would be used as input commands. This enables human computer
interaction to become more natural because your body movements now act as
controllers (Rouse, 2011). As of now, when using computers, we are confined to
using the mouse or trackpad and keyboard to interact with the computer. We use
the computer mouse to direct where we want our cursors to be placed or where we
want certain portions clicked and we use the keyboard to type manually, letter
by letter on the keyboard. But with the advent of such technology, soon enough,
these components will not be necessary and might even become a thing of the
past. However, this will only be possible with the development of gesture
recognition technology.
An
early form of such technology would be motion capture. Basically, motion
capture refers to recording movements, which are translated to models on the
computer and that is achieved by having actors don on a bodysuit with sensors
attached to pivotal points to detect movements (Sanders). Motion capture is
most commonly known to be used in animation either in filming or creating video
games so that characters are more life-like, it is also used in scientific research,
engineering and more (“Who Uses Motion Capture?”). Motion capture, is in many
ways similar to gesture recognition technology, drawing a parallel in the way
that it allows us to capture motions on the screen simply by performing
different actions, though at the end of the day, both still serve different
primary functions. Another point to note was that early forms of gesture recognition
technology also called for people to put on some form of sensors to allow
computers to detect motion. Gesture recognition technology is able to
familiarize with certain motions that would ultimately point to a certain
command and later translate that to however we want to control our computers.
As we
can see, the technology we talked about is only at the tip of the iceberg. The
development of all sorts of technology, in one way or another, either directly
or indirectly, has contributed to what we have today and what we will have in
the future. The gesture recognition technology is fairly new and only more
explored in the 21st century with the production of certain devices
such as Mircosoft’s Kinect and the newly launched Leap Motion Controller. With
time, it is definitely possible to be able to alter the primary functions of
these devices and use them in other fields and expand the use of the gesture
recognition technology.
Even though gesture recognition technology is a relatively new piece of
technology, we have already seen some devices that have adopted the use of this
technology. Gesture recognition technology is so highly received because of the
kind of interaction experience it provides for the users. This technology is
dependent on the kind of gestures we use everyday so it makes interacting with
our electronic devices so much more natural. To break it down, gestures are
used as a form of communication. For example, a simple wave using your hand
could signify a greeting, a police officer could also use hand gestures to
direct traffic, shaking our heads means no and nodding means yes. There are
many universally known gestures that we make use of on a daily basis to convey
a message and this is what gesture recognition plays on, to integrate the use
of gestures seamlessly into our computers which later translates into
signals.
One device that was of all the rage back in 2010 when it was first
released was Microsoft’s Kinect. Kinect for Windows was able to encompass
gesture recognition technology and even voice recognition technology to give
users a more natural experience while using the device; furthermore it adds
another dimension of human understanding (“Kinect for Windows features”). Kinect
is most certainly popular for the kind of gaming experience it offers its
users. As of now, most of the games released are for Kinect for XBOX 360. There
are many games available now in the market and one of the best sellers of the
lot would be “Dance Central”. “Dance Central” is a highly popular game where
people would follow the choreography shown by the characters to the latest
songs in the game and the objective is to try to be as accurate as one can
possibly be to earn more points. Other games also include fitness programs such
as “Zumba Fitness” or “Nike + Kinect Training” where it is all about working
out and training the body, all in the name of health. There are also
educational games that can be used to aid teaching in schools. One example
would be the “National Geographic Challenge” where it teaches children general
knowledge through some games such as solving puzzles or “Body and Brain
Connection” used to teach mathematical concepts to children. Some of these
games could also be used in educating children with special needs such as
autism. These games are so successful because they actively engage their users
to think and learn at the same time.
Of
course, another new invention that has made use of such technology and created much
buzz in 2013 would be the Leap Motion Controller. Leap Motion was created to
better engage the user in using and interacting with their computers.
Eliminating the need for the computer mouse and even the keyboard. It has two
cameras and three infrared LEDs and with these components, they are able to
detect hand and finger motions in the space right in front of it. In addition,
it is very accurate that is able to sense movements of up to one hundredth of a
millimeter. As it is still relatively new in the market, there are only a handful
of uses for the Leap Motion. Though we will explore the possible potential uses
of the Leap Motion Controller and the gesture recognition technology in the
next segment. Leap Motion comes with an app store called the “Airspace”, much
like how the App store on an iPhone works. It offers a series of programs that
can be used with the Leap Motion Controller, such as games, education, music
and entertainment, creative tools and so on and so forth.
Games
One
reason why the Leap Motion is so sought after is because it enhances a gamers’
gaming experience. Gamers are could feel like they are actually part of the
game, because they are no longer bounded by the use of a controller, they are
now capable of playing the games as if they were there at the scene. “Solar
Warfare” is a game where users control their own hovering starship while
shooting down their enemies by specific hand motions. Coupled with visual
effects and specially crafted sound effects, it creates a immersive environment
for the user. Another games people might find more familiar would be “Fruit
Ninja” where you would use your finger to slice the fruits, just like how you
would on an iPhone or iPad, but this time it is done by moving your fingers
across the air.
Education
Young
children, especially those in the kindergarten level, usually enter their first
learning environment as kinesthetic learners. What this means is that children
at this age learn best through their sense of touch, by being able to feel and
move around. The Leap Motion controller would be able to target these children
because they enable users to move around and perform certain movements in order
to “lock in” their answers. This type of learning keeps the children actively
involved and engaged during the lesson because they are able to have fun while
they are learning as compared to conventional methods where students are given
pen and paper (Farwell).
One
way in which Leap Motion can aid students in their learning process is to
introduce various games that can allow students to learn and have fun
concurrently. Some games in the Airspace such as “Caterpillar Count” where it
teaches young children to count up to 100 by guiding a caterpillar to around
the screen to “collect” the subsequent numbers (“Count Your Way from Caterpillar
to Butterfly”). Furthermore, as children start to explore the world of science,
it is important for them to understand the concept of molecules and atoms, the
anatomy of animals, to know more about astrology. Some of these concepts are difficult
to understand unless we are able to visually see these models in a
3-dimensional (3D) manner. To put it simply, we can use the various learning
tools in the Airspace to learn about how molecules react with one another, the
types of forces or bonds that exist within or between molecules. There is an
app called “Molecules” where it enables the user to rotate and move the
molecules apart to have a clearer picture of how they work (“3D Molecules in
the Palm of Your Hand!”). We could even dissect frogs to understand their anatomy.
As part of an app called “Frog Dissection”, users examine the organs and learn
their functions and understand the lifecycle of frogs without having to kill a
real frog (“Forget the Formaldehyde!”). In “Solar Walk”, students “travel
through space” to learn more about the planets, its history, structure and so
much (“Solar Walk – 3D Solar System”).
These
games are enhanced because the Leap Motion controller allows users to actively
participate in their learning process. Moreover, with all the movements and
hand gestures, it could even help to retain more information as compared to
conventional teaching methods.
Music
& Entertainment and Others
Many
of the apps under this category also allow you to pick up a new skill by
learning how to play a musical instruments by offering tutorials as well as
easy to follow rhythms with varying level of difficulty. Some apps also allow
you to be your own conductor by allowing you to lead and direct an orchestra.
Other
apps include creative tools to allow users to design, edit and explore their
creative talents. They have apps that allow users to be able to paint and draw
such as PainterÒ
Freestyle without the need to use your mouse or any other expensive gadgets.
Furthermore, users would have more control and fluidity in designing their art
piece as they are able to draw more naturally with their hands. In addition, they
have picture editing software that would allow you to beautify or enhance your
photos, one such app is the “Deco Sketch” that enables users to build layers of
effects upon their original to create different types of pictures.
As can
be seen from above, these are just some of the applications of the use of the
Leap Motion controller; this is just the tip of the iceberg. There are
definitely more uses for the Leap Motion controller in the future. We will be
looking at the different prospective fields that might adopt to use the Leap
Motion controller as part of their operations. (“Eight Great Innovation Areas”,
2013)
Future Considerations (ie where we might go)
As
the Leap Motion controller continues to improve and develop over time alongside
the advances in other realms of technology, the potential uses of this
controller can only expand. The following segment will showcase the different
methods in which the Leap Motion controller can be implemented in various
existing fields and how it can be used to aid or refine certain processes. Even
though some of the functions of the Leap Motion controller are already in
place, they are not yet widely used, such as in the field of education. We will
also delve into how it might affect the medical field, online retail
experience, and security.
Education
From
the previous section, it is true that some of the apps that have been
aforementioned have already been used. However, the uses of these apps are only
minimal, at best, it only allows private users who possess the Leap Motion
controller to access such programs. Furthermore, the many apps available in
Airspace are only very minimal and more of such programs can be designed and
targeted for use in the school environment. As discussed earlier, the Leap
Motion controller is well received by students because it promises a truly
immersive and engaging session of learning for them.
Also
explained earlier, many kids in the early stages of education are kinesthetic
learners, although this sort of learning styles might change person to person
as they continue to grow. However, it is important to recognize that for any
child to learn, lessons ought to be fun and captivating. Children especially at kindergarten or at the
elementary level learn best through hands-on activities and when they are
totally drawn into the lesson. This means that they acquire information fastest
when they are fully entrenched in activities that are active; in other words,
it calls for them to be moving around.
Furthermore,
another target audience that could benefit from the use of the Leap Motion
controller would be special needs children, such as those who suffer from autism,
a developmental disability that hinders communication and social interaction
skills. One school that has adopted the use of the Leap Motion controller in
their classroom for their special needs children would be the Jackson School in
Australia. Mathieu Marunczyn, a teacher at Jackson School, reached out to Leap
Motion to request for Leap Motion controller to be set up in school for his
special needs classes. His lessons are aimed to teach his students literacy,
impart critical thinking skills and to improve cognitive skills by using the
Leap Motion controller and by downloading some of their apps. According to the
Leap Motion blog, Marunczyn mentioned that it was especially popular with
children with autism. If the games involve a social aspect, it encourages the
children to interact and communicate with one another and this is particularly
important for these children. Sometimes, they also utilize google maps to
explore different parts of the world and learn about these countries, while
engaging in conversation with their peers, supporting and cheering each other
on. This allows these children to feel empowered and make them feel like they
are apart of the society because the Leap Motion controller allows them to be
immersed into the games that their disability no longer hinders them. Another
app that stand out from the rest would be “Sortee” which requires these
children to sort out objects into different categories by using a dragging
motion. (“Students Take a #LeapInto Learning”, 2013)
Of
course, one thing to note is that this is probably one of the few schools that
have chosen to integrate the Leap Motion controller into their curriculum. The potential for the controller to be part
of the teaching arsenal is massive. There are a myriad of apps that have been
specifically created to teach children on the many wonders of the world. Some
of the apps that are released in the Airspace also come with a 3D setting such
that it allows students to see the contents on the screens “come to life”. One
such example would be the “Solar Walk” app. When students don on a pair of 3D
glasses, they are able to feel like they are actually in space. They can move
closer or further away from different planets, travel through the galaxy and
learn more about the planets at the same time.
The
future use of Leap Motion in the classroom could potentially alter the way
children are learning now. No longer would they be confined to learning of the
textbooks with pen and paper, they could learn from the real world.
Furthermore, they can learn more about the different things they do not
normally come into contact with. They are able to bring life back into the
classroom that could give each child an enhanced learning experience.
Healthcare
Another
promising use of the Leap Motion controller is in the arena of healthcare. There
are many reasons why the Leap Motion controller could integrate so well into
the daily routines of medical practitioners. Whether it is to be used in
hospitals or clinics to aid in surgery or even rehabilitation, to developing
programs to keep us fit, to even being able to access patient data are all the
possibilities where Leap Motion can be employed.
One
highlighting feature is that it allows for medical staff to operate systems
without having any physical contact with computer screens, or any other unnecessary
objects that we would touch. For instance, sometimes during surgery, medical
staff may have to utilize different systems to access information or any device
that they might require to contact other important people they need. Moreover,
staff may not be within reach of the display but still require the information
on the screen to guide them through some procedures. But for health and safety
reasons, they cannot physically touch these items and this is where Leap Motion
may be a solution to this. The “touchless” surface prevents the spreading of
diseases as it reduces the risk of transmission from one person to another, it
could also be more economical as less could be spent on sterilizing such
surfaces, it might also be a faster system which would create a more efficient working
operation and gives staff more time to focus on other areas that may be more
important. In addition, the simple hand gestures may be a faster and safer way
to manipulate data on the screen instead of using the conventional mouse or
trackpad. (“Eight Great Innovation Areas”, 2013)
Since
the Airspace is already coming up with apps that are meant for education,
entertainment and creative tools and others, they could also start developing
apps for people to keep fit in the future. They can design programs such as
workout regimes, or sports games for people to get active and exercise too,
with reference to similar apps that Microsoft’s Kinect has to offer. These apps
could also encompass a diary entry such that users can keep track of their
progress. Furthermore, one other possibility is that doctors can also make use
of the Leap Motion to keep track of the progress of their patients. After every
workout session, the Leap Motion sensor could record different data such as
heart rate, duration of workout, the different muscles used and whatnot, and
these data will be sent to the doctor for reference in the future. Though it
seems a little farfetched, it is only with time that we can achieve such use of
the Leap Motion controller. Also, this might also benefit a lot of people who
have difficulty travelling around, or those who stay far from hospitals, or
where traffic conditions are not suitable to travel in. With the Leap Motion
controller, they might be able to use the Leap Motion controller to communicate
with their doctors and have a consultation with them over the computer without
having to be physically there. Doctors can use different videos as examples
using the controller to show their patients exactly what needs to be done to
cure a certain illness or to alleviate pain.
One
other practical use of Leap Motion may be used in Medical School. It seems
almost so obvious that if Leap Motion controllers can be integrated in grade
school, then it can most certainly be operated in Medical School. And of course
it can. Leap Motion controllers coupled with 3D interactions can be used to
teach surgical procedures or concepts that may generally be more difficult to
grasp. It can be applied to demonstrate highly complex surgeries to simple procedures
like how to draw blood or any other techniques they would need to acquire. Of
course, the controller can also be used to teach patients various ways to curb
their ailments or illnesses. For example, patients that are undergoing
physiotherapy may find it easier to work from home, and the Leap Motion
controller is able to guide the person through his sessions and also keep track
of his progress using the cameras in the controllers so that doctors would be
able to alter his sessions to make it more suitable for each individual
(Norris, 2013).
These
are just some of the uses of the Leap Motion controller that can be manipulated
in the field of healthcare and this could one day be revolutionary in changing
the way the healthcare systems work. We could possibly, in the future, have a
safer, more efficient and more comprehensive system that we all stand to gain
from.
Retail
Since
the realm of online shopping began, retailers and the many brands available
have been striving to create a more immersive and interactive shopping
experience with their customers. And it is not just with reference to clothes
but it also includes furniture shopping, choosing where to go on your next
vacation, buying new musical instruments, it could give an entirely new
shopping experience. However, in this aspect, the Leap Motion controller will
not be able to stand alone but it would also require the use od 3D technology
to help customers better visualize the products, it is basically similar to or
is in some way, augmented reality.
What
this means is that whenever a user decides to do some online shopping, for
example for clothes, they are able to visually see how the clothes will look
like on their bodies without even having to put them on physically. The
pictures would then appear on the screen and users can use this tool to decide
whether or not the clothes suit them. This too can be used if an individual
decides to get new furniture, they can look for the furniture that they are
interested in online, and once they find it, they can point the controller to
the space where they would potentially like to place it and see if it matches
the rest of the décor in the house. And as many people are gaining the
opportunity to go overseas for a vacation, it is important to choose the right
destination. With the Leap Motion controller, the interested parties are able
to use the controller to travel to different parts of the world, and decide
which parts of the world they would like to go. They could zoom in on parts of
the country they are more interested in then another, where they would like
stay in and find out a lot more information before embarking on their journey. (“Eight Great Innovation Areas”, 2013)
Another
thing to note that is that this sort of technology need not be confined to just
online shopping experience. Certain shops could also make use of this advancement
and have it in their stores such that when there is high human traffic and lack
of fitting rooms, the controller could act as a alternative solution to
actually trying on the physical clothes itself. But with the help of the Leap
Motion controller, our shopping experience has only gotten a lot more exciting
and interesting with the help of the Leap Motion controller. With this, we may
be more capable of making more informed decision about our purchases instead of
being irrational about it.
Security
Soon
enough, the Leap Motion controller might even be used as a security system
because of it’s high precision. It might be even be used as an authentication
process as it scans the hands of the right individual to unlock a safe, or to
access a computer or to other information that only highly regarded personnel
is able to access (“Eight
Great Innovation Areas”, 2013). This could be done with just a wave of your hands and the
controller would be able to read and scrutinize every part of a particular
handprint, the way the lines run across your palm, if there are any unique
features, this is because no two palm prints are the same.
Even
though the controller has just been released in 2013, the potential for this
piece of device to grow and impact the world is just tremendous. Generally, the
uses of the Leap Motion controller are just endless, and there are definitely
much more that can be done with such a piece of technology. There are many
opportunities for this device to be implemented or integrated into other
systems such that it can work alongside with them and improve current systems
we have in place. It is really exciting to see how the future of such
technology is in the now and the opportunity to create a different kind of
world is right here in our hands. However, there are certain limitations to
device we would briefly look at in the following section.
