Search · AI · Cross-Surface Navigation
Before this project, "searching" on a Quest headset meant opening the Store, navigating to a search tab, typing with a virtual keyboard, and receiving results limited to apps. If you wanted to find a friend, you went to a social panel. A specific game type — back to the Store. A room you'd visited before — no search at all.
Users had learned to work around it. They'd take off the headset and search on their phone. This wasn't a search problem. It was a platform coherence problem. The absence of a unified search system was evidence of a platform that had been built feature-by-feature rather than experience-by-experience.
My brief: design a global search system for Quest — one that works across all content types, all input modalities, and that doesn't feel like a 2D web search jammed into a 3D space.
existing cross-surface search on Quest
separate places users searched for things
of users used phone to search while in headset
input modalities to design for simultaneously
Designing search for a headset means designing for three fundamentally different input methods — and users switch between them mid-session. A controller-based user hunts and pecks on a virtual keyboard. A hand-tracking user pinches keys in mid-air. A voice user speaks a query.
Each input method has different error rates, latency profiles, and preferred query lengths. Voice queries average 6–8 words and are conversational ("find me something relaxing with friends"). Keyboard queries average 2–3 words and are keyword-based ("beat saber"). Any search system had to handle both — and fail gracefully when it couldn't.
Voice: natural language
Users speak in full sentences. Queries include intent signals ("find me", "show me", "what's good for"). NLP interpretation is critical. Error tolerance must be high.
Keyboard: keyword-first
Typing in VR is effortful. Users use minimum viable queries. Autocomplete and fuzzy matching matter more than anywhere else. Patience is short.
Hand tracking: gesture-forward
Users prefer shortcut gestures over typing. "Recent searches" and predictive suggestions carry disproportionate weight. First result quality determines everything.
Users switch mid-task
Many users start with voice, then refine by keyboard. The handoff between modalities must be seamless — query text preserved, context maintained.
Early designs treated search as a feature — accessible via a search icon in the menu bar. Testing quickly showed this was wrong. Users didn't think "I need to use the search feature." They thought "I need to find something." The entry point needed to be as universal as pressing a home button on a phone.
We redesigned the entry point as a persistent "quick summon" gesture — a specific hand position that opens a floating search field, available from any context inside the headset. No navigation required. The metaphor: it's not a feature, it's a muscle memory.
"I want search to be the thing I do without thinking — like reaching for my phone."
— Research participant, concept validation study, 2023Global search — full interaction flow across voice, keyboard, and hand tracking input
When you search for "meditation" on Quest, you might mean: an app (Tripp, Nature Trek VR), a concept (YouTube meditation videos), a room (a shared relaxation space a friend invited you to), or a person (a wellness coach you follow). The results need to handle all of these without asking the user which one they meant.
We designed a ranked, categorized results layout where the most likely interpretation anchors the page and alternative categories are accessible with a single scan and select. AI-assisted ranking used query phrasing, user history, social graph activity, and time of day as signals — but always surfaced its reasoning in plain language so users could course-correct.
Worked with ML team to define a taxonomy of intent signals (navigational, exploratory, social, transactional). Each maps to a different results layout priority.
Designed a consistent "card" language for each content type (app, person, space, video) so users learn the visual vocabulary of results once and apply it everywhere.
Designed 6 "no results" and "low confidence" states that guide users toward reformulation rather than dead ends. The system never strands users.
Search history and social graph signals are used for ranking but never shown to other users. Worked with trust & safety team on the visibility model throughout.
task completion on first search attempt
reduction in users leaving headset to search on phone
of all search sessions now start with voice
user satisfaction in post-launch survey