If computer could hear us


If computer could hear us

JARVIS the home computing system of Tony Stark in Iron Man movie takes cares of everything from house to his suite. JARVIS understands and converses with Stark in a natural way.

Computer hardware has always changed rapidly following the Moore law, but unfortunately input/output devices, interaction techniques, and software for human-computer interaction have not been able to match the similar growth and improvement. The GUI-based style of interaction made popular during 80 is still considered to be the only choice.

With the advancement in IOT (internet of things) which is making computing more  pervasive and ubiquitous the human computer interaction begs for newer form of interactions to meet human needs. In order to accommodate a wider range of scenarios, tasks, users, and preferences, we need to move toward interaction mediums that are natural, intuitive, and unobtrusive.

“How can we make computer invisible.”


It is not about simplifying the UI as however simplified the UI is, the interaction still requires considerable learning and actions from the user. Wouldn’t it be easier if we can ask our room to switch ON the TV or AC? If we have a questions which kids have all the time by the way, why can’t we just ask it, and get the answer rather googling it on our mobiles/laptops and then try to figure out answers from thousand of pages. The conventional interaction approach will any way fail for wearables or minituarized computers. Can you imagine reading a mail on your smart watch? It would be much intuitive though if it can read the mail for us or write the mail for us.

Sound or voice based interactions have gained lot of attraction and attention  lately reflecting the new reality. Apple Siri, google now and Microsoft cortana are some initial efforts in that direction but are mostly restricted to mobile. At our innovation lab we have tried to explore the opportunity of leveraging sound or voice as a medium for interaction for real life situations.

Supreme being the new gen genie

Man’s endless curiosity to know and gain information is as powerful as his endless desire to possess things. With this idea in mind, we created  ‘Supreme Being’. As the name suggests, the ‘Supreme Being’ can answer man’s queries and doubts and satisfy his quest for knowledge and information. This prototype is notably an example of the fact that in future, physical and digital will be blended and the difference between the physical and digital will cease to exist. Tomorrow our walls or tables or any other objects in our house or our organization can be converted into the Supreme Being which can give answers to our queries. Supreme being understands our language and understands when we are asking for answers. There is no UI, keyboard or mouse required to interact. Have a question, Just ask!! As the genie that comes out when Aladdin rubs his lamb, Supreme Being comes alive when the tile on the floor which it is programmed to is tapped or knocked. And similar to the genie, it asks you what you would like to know. From a country’s population, meanings of difficult words to stock price of a company and weather reports, Supreme Being is a talking encyclopaedia in itself. If it does not know the answer to your query it will prompt you to ask a different question in a light-hearted manner.

Powered by Wolfram Alpha, the the computational knowledge engine that answers factual queries directly by computing the answer from externally sourced “curated data” Supreme Being gives us exact answers to our queries. Functionally, it is more precise than search engines which provides you with a list of documents or web pages that might contain the information regarding the query. Apart from the wolfram computational engine, it is equipped with force sensors, speakers and microphone, which are all connected to a computer.

Supreme Being demonstrate of how can we make interaction simple, interesting and engaging using voice or sound. Supreme Being can be used in multiple scenarios to encourage learning and knowledge gathering. It is no brainer to see it useful for playful learning in schools, Information kiosk in conferences and Hospitality industries.



Music from

Shelfie the new shopping assistant

Whether we like it or not, it is a fact that shopping is a very integral part of our life today. But with online and offline shoppers vying to get customer’s attention, decision making has in fact become tougher. More often than not, we are dissatisfied with the information which is provided on the packaging of a product and have always wished for more. Hence we decided to give a new twist to the shopping experience.  What if shopping becomes interactive?

This quest to deliver experience and create intelligent environment led to the creation of ShelfHelp. ShelfHelp is a technology enabled shelf which can provide voice based information on the products placed in the shelf. Once the shopper picks up a product available in the shelf, it provides information about the product and asks him if he wants more information. It can also receive voice commands from the shopper and responds accordingly. It can even give him a sense of smell of the perfume or the smell of the coffee powder which he picks from the shelf.

Attached to the shelf are LED strips and inexpensive light sensors which capture the intensity of the light and works in tandem with the objects mapped to those sensors. A cube kept on the top layer of the shelf acts as the face which is equipped with micro-controllers which activates the LED strips and lights up while the shelf interacts with the shopper. Proximity sensors attached to the shelf will detect the proximity of the person and help the shelf to respond accordingly. High intensity speakers and microphone attached to the shelf enables the voice interaction between the shopper and the shelf.

In our quest to create intelligent environment, we used simple components which are inexpensive and easily available around us. For example, the sense of smell is powered by recoding programmable air fresheners which is easily available in the store next door.



If the future of experience is natural then there is no doubt that voice is going to control a significant part of experience in human computer interaction.