The search giant Google has jumped ahead of the innovation curve once again, and this time updating Lens, Google’s image recognition search app, now integrated into most Google, Andriod and iOS devices to allow users to ask questions with or about videos recorded on their phones.
This means Android and iOS users are now able to find information about moving objects if they can’t get search results with a picture, Lens will now allow them to take a video and then ask the question.
According to Google, the Lens photo search feature is also being updated to support asking questions using voice, just like video search. Until this update, you could only type your question into Lens after taking the photo.
With the new update, point your phone at the object, hold down the virtual shutter button on the screen, and then speak when Lens prompts you to ask your question. See how it works from Google’s Lens product update post
“Video With Voice” Lens Search Example
For example, if you are at an aquarium, and see a fish you know nothing about but are curious to learn more about, all you need is your phone and Google Lens to capture a short video of the fish and you’ll get your answer instantly.
Looking at the GIF below, the user points a phone at the event, opens the Google Lens app, and then holds down the virtual on-screen shutter button. When Lens starts recording they can ask the question “Why are they swimming together?”. Google Lens will then use the Gemini AI to respond or provide results.
GIF by Google
How To Use Google Lens Video Search
- Open your Google Lens app
- Hold down the virtual shutter button on the screen
- Google Lens will start recording video and prompt you to speak
- You can then ask your question
- Google Lens will output a response using the Gemini AI model like in the GIF above
“Image With Voice” Lens Search Example
Until the recent Google Lens update, you could only use the image search feature with text after taking the photo. With the new update, let’s say you look into the clouds and want to know what type of cloud you are seeing, just get your phone out, point to the clouds, open your Goole Lens app, press the press and hold down the on-screen shutter button and you’ll be prompted to ask your question.
Google will use Gemini AI to provide your search results. See the GIF below.
GIF By Google
How To Use Google Lens Video Search
- Point your phone at the object of interest
- Open your Google Lens app
- Hold down the virtual shutter button on the screen
- Google Lens will take a photo and prompt you to ask your question
- You can then ask your question
- Google Lens will output a search result using the Gemini AI model like in the GIF above
Conclusion
Since its announcement at Google I/O in 2017, Lens has played an important role in Google’s image recognition and augmented reality journey. With these current updates, Lens will hopefully achieve its full potential as originally intended by Google.
Bear in mind that there isn’t currently support for Lens to identify sounds in the recorded video yet, you can identify a song you are hearing in the video but this is something we are sure Google is working on, if not testing already.
Google Lens Voice for Image and Video questions are rolling out globally buy available in English only on Android and iOS, for now.
Questions People Ask
Can Google Lens identify sounds in the video?
Currently, there is no Google Lens support for identifying sounds in a video.
Where can I find out more about Google Lens?
To find out more about Google Lens, visit Google’s Lens website
