Google Photos to launch new Ask Photos feature powered by Gemini
This summer, Google Photos will introduce a new Ask Photos feature based on the Gemini artificial intelligence model. Google has shared more about this feature, explaining how it will work.
How Ask Photos works
According to Google’s research team, Ask Photos is “an example of how Gemini models can act as agents by leveraging memory features and capabilities.” Examples of queries that have been demonstrated include:
- “Show me the best photo from every national park I’ve visited.”
- “What were the themes of Lera’s birthday parties?”
Your dialog query is passed to an agent model that uses Gemini to determine the best rich search tool (RAG) to perform the task. The model first analyzes the user’s intent and then formulates a photo search using an updated vector system that extends the metadata search capabilities already built into Google Photos. The system better understands natural language concepts, such as “person smiling while riding a bicycle,” than traditional keyword search. The model then reviews the photos and videos obtained as a result of the search. Thanks to Gemini’s “long contextual window and multimodal capabilities,” it can find the most relevant information. In addition to visual content, dates, location, and other metadata are used. The model then creates a response based on the photos and videos it finds.
Additional features
You can adjust Ask Photos, and the program will remember these changes for future queries. This allows you to use the feature as an assistant, not just a search tool. Users can “view and manage memorized details at any time”. An experimental feature, possibly related to the rumored Project Ellman, will be launched in the coming months, and there are already hints of new features.
Conclusion
Ask Photos, powered by Gemini, promises to improve the search experience in Google Photos by enabling users to get accurate and relevant answers to their queries using advanced artificial intelligence technologies.
