Sixth Sense Technology, What are Apple and Google Working on for future.

This post was last updated on July 23rd, 2020 at 10:43 am

With many Technology coming every day and breaking news this technology has made news for last 5 years and has been the centre of attraction for many tech experts then and now. What is so great about this technological piece that it is able to grow more and more and still opens wide opportunity for new generation to explore. The Sixth Sense technology is very much suitable name for the technology developed by Pranav Mistry, a PhD student in the Fluid Interfaces Group at the MIT Media Lab. The idea of this Technology dated way back but was finally put into shape by this young Indian Mind. Not only Google but apple and many other technical giants are running hard to hire this Indian for their design of future gadgets.

The Sixth Sense Technology augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information. You can understand it with in simple language as the way of using the digital information in the real world like using the internet browser on any flat surface or taking pictures by just your fingers in the shape of frame and similarly much more.



Let’s check out this Technology from various aspects.

What is Sixth Sense Technology?

  • ‘Sixth Sense’ is a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information.
  • Sixth Sense bridges the gap by bringing intangible, digital information out into the tangible world, and allowing us to interact with this information via natural hand.
  • The hardware components are coupled in a pendant like mobile wearable device.
  • A camera, a mirror and a projector are connected wirelessly to a bluetooth and wifi enabled smart phone that can slip comfortably into one’s pocket




  • Captures an object in view and tracks the user’s hand gestures
  • It sends the data to smart phone
  • It acts as a digital eye, connecting you to the world of digital information     


  • The projector projects visual information enabling surfaces and physical objects to be used as interfaces
  • The project itself contains a battery inside, with 3 hours of battery life.
  • A tiny LED projector displays data sent from the smart phone on any surface in view–object, wall, or person.

Mirror and Smart Phone

  • The downward-facing projector projects the output image on to the mirror
  • Mirror reflects image on to the desired surface
  • A Web-enabled smart phone in the user’s pocket processes the video data
  • Other software searches the Web and interprets the hand gestures

Color Makers

  • It is at the tip of the user’s fingers .
  • Marking the user’s fingers with red, yellow, green, and blue tape helps the webcam recognize gestures
  • The movements and arrangements of these makers are interpreted into gestures that act as interaction instructions for the projected application interfaces.


How it works 

how Sixth sense Technology works

how Sixth sense Technology works

The SixthSense prototype is comprised of a pocket projector, a mirror and a camera. The hardware components are coupled in a pendant like mobile wearable device. Both the projector and the camera are connected to the mobile computing device in the user’s pocket. The projector projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the camera recognizes and tracks user’s hand gestures and physical objects using computer-vision based techniques. The software program processes the video stream data captured by the camera and tracks the locations of the colored markers (visual tracking fiducials) at the tip of the user’s fingers using simple computer-vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction.


It has numerous application and some of them are:-

You will be shocked to learn the thrilling potential of sixth sense technology

Palm is the new dialer

advantages of technology

advantages of technology
palm telephone

This technology enables the user to call without using the dialer. The dialer will be projected on palm and the user can dial the number using other hand.

Read Books easily
Check out the ratings of the Book you are going to buy, it checks the ratings from the internet. And another amazing thing is that it reads the book for you.

Check your Flight Status

advantages of sixth sense technology

advantages of sixth sense technology

Just place the ticket in front of the projector and it checks its status from the internet.


The user just needs to make gesture of clock and the watch will be projected on the user’s hand.

Take pictures

  • If you fashion your index fingers and thumbs into a square (“framing” gesture), the system will snap a photo.
  • After taking the desired number of photos, we can project them onto a surface, and use gestures to sort through the photos, and organize and resize them.

Fingers on Brush

The user can draw anything on paint with the help of his fingers. This drawing can be 3D also. Hence, no need to use mouse.

Access Internet anywhere

The users can browse internet on any surface even on their palm. Now no more need to carry heavy laptops and palmtops to have bigger screens internet. You can actually project the Internet on any clear surface and enjoy browsing.

Video Newspapers

sci-fi making reality with sixth sense technology

newspaper technology with sixth sense

Like the video newspapers of Harry Potter this technology identifies the news headline and then projects the relevant video. Imagine a video running over your daily newspaper and you are able to see everything without making any efforts. Seems like some of the sci-fi movies are going to be no more sci-fi.

 Learn more about Sixth Sense Technology by opening this picture

Other applications

There are many possible applications with this SST project like

  1. Vedio on gesture of (action)
  2. Game on gesture
  3. 3d view of any object , building , creature , etc
  4. Video conferencing
  5. Zoom in and zoom out option
  6. Safety cheker and trouble indicator
  7. Drawing a magnifying glass symbol takes the user to the map application while an ‘@’ symbol lets the user check his mail.
  8. To make Sixth Sense work as fifith Sense for disabled person


Why this Technology is the focus of many tech companies can be understood from advantages it has for us.

SST_latest technology by pranav mistry

SST_latest technology by pranav mistry


  • Portable
  • Open source
  • Data access directly from machine in real time
  • Supports multi-touch and multi user interaction
  • Connection between real and virtual world in a flawless manner
  • Cost Effective

The device can be made in current prototype system costs approximately $350 to build. In Indian currency Rs.17000. But still the device is not available in the market because of the only reason of high internet speed on wireless devices not yet available everywhere and the second reason being the miniature model to be developed for this prototype.



  • Sixth Sense recognizes the objects around us, displaying information automatically and letting us to access it in any way we need
  • The Sixth Sense prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system
  • Portability and cost effectiveness s adds to its advantages
  • Allowing us to interact with this information via natural hand gestures

All the image credits are to the pranav mistry and team.

You can learn more about Sixth Sense technology here :- Sixth Sense Technology

Please tell us how you liked the article and what is your opinion about this technology. Would you wish to have one such technology in your pocket in the future?


Latest posts by Anurag (see all)


Blogger and Engineer. Passionate about trekking, coding and discovering new places. I love to meet new people and do sketching in my spare time.

You may also like...

2 Responses

  1. Ranveer Singh says:

    Thanks for such a nice post on sixth sense mobile. I was in search of more information on sixth sense mobile to write post on it on my blog at

    Your post give me much information on it. Very nice keep it up.

    Ranveer Singh

Leave a Reply

Your email address will not be published. Required fields are marked *