And miles to go before I sleep.
There is a traveler within every one of us. We travel not just to see and adhere to beautiful sites and sceneries but also to know about the history, culture, and traditions of the places we visit. And when visiting a historical site or a place with rich cultural heritage, we often rely either on the Internet or on the local tour guides for the information and stories attached to the place. Now, the Internet can provide you tons of Information about a particular place that you are visiting but it will most certainly present the information to you in a very dull, ineffective manner where there is no personal interaction. Where, on the other hand, a tour guide will most certainly interact with you and will also be a very good storyteller. But they might not have all the information that a digital media can provide you.
But can we, in this day and age, build a digital tour assistance which can be information-driven and be an interactive storyteller?
Our experimentations with the Tour Guide application began and we were clear about the core capabilities that the application must have.
1/ Understanding of the surrounding environment or place – in other words, the application should be able to recognize important places/objects of interest.
2/ Smart and interactive way of presenting the information.
We figured out that our application can live on a mobile phone or a device of similar kind and we might be able to use Computer Vision and Augmented Reality to deliver the intended user experience.
The Tour Guide application is an iOS app that leverages ARKit and Core ML. Using Core ML and a state-of-the-art object detection architecture that runs on top of it, the app can see and locate objects of interests or hotspots on a tourist location – in this particular use case, the Taj Mahal. A location marker/pin in dropped on each unique hotspot using Augmented Reality. To use the application, the user has to take out their mobile phones and use the camera to point towards a location that they are interested to know about. The application is smart enough to understand the sequence of stories it should play to the user. The stories are delivered via audio and while listening to the user can put the mobile phone in their pockets. The hotspots are also interactive. They can have different layers of interaction. For example, tapping the location pin on the grand archway of the Taj, the user can unlock the second level of hotspots which will be pinned on the beautiful Islamic patterns that the archways have. Tapping those pins with play the stories behind that beautiful art.