An “Enhanced Visual Search” feature in iOS allows users to search for landmarks in the Photos app, but it requires sharing data with Apple by default. Apple occasionally makes decisions that can undermine its reputation for strong privacy practices, such as when it was found to be secretly collecting users’ Siri interactions. A recent blog post by developer Jeff Johnson raised concerns about another such decision: the “Enhanced Visual Search” toggle in the Photos app, which appears to be switched on by default, granting the device permission to send photo data to Apple.
When I checked my iPhone 15 Pro, I found the toggle turned on. You can find it in the Photos settings on your phone (via the iOS Settings app) or on a Mac (through the Photos app’s settings menu). This feature lets you search for images of landmarks or find specific photos by the names of those landmarks.
To use it, swipe up on a photo of a building and select “Look Up Landmark,” and a card will appear with the identified landmark. According to the blog, Apple then encrypts the vector embedding of the image and sends it to their servers to compare it with their database. While the technical explanation in Apple’s research papers is complex, IBM simplifies it by explaining that vector embeddings convert a data point, such as an image, into a numerical array representing its characteristics.
Although Apple claims to take privacy precautions by converting image data into a model-friendly format, the fact that this feature is on by default rather than opt-in—like the toggles for sharing analytics data, Siri recordings, and more—seems to be a missed opportunity for better user control.
