Today, we’re thrilled to sit down with Vijay Raina, a renowned expert in enterprise SaaS technology and software design. With his deep expertise in architecture and thought leadership in the field, Vijay offers unique insights into the innovative world of multilingual semantic search and conversational assistants. Our conversation dives into the groundbreaking work behind Bosch Digital’s SmartSearch for Bosch eBike Systems, exploring the motivations for moving beyond traditional search methods, the technical intricacies of building a robust retrieval engine, the challenges of scaling for global users, and the exciting transition to chat-based assistance. Join us as we unpack the journey of creating a system that understands user intent across languages and delivers precise answers in a heartbeat.
How did the limitations of traditional keyword search inspire Bosch Digital to create something as advanced as SmartSearch for Bosch eBike Systems?
Traditional keyword search just couldn’t keep up with the real-world needs of our users. Riders, mechanics, and sales reps were typing queries like “reset Kiox 300 display” and getting buried under irrelevant results or no results at all. Synonyms, typos, and voice-to-text errors—like “Kioxx 300” or “reset chaos 300”—tripped up the system. Plus, with documentation in 27 languages, intent often got lost in translation. We realized we needed a solution that could understand meaning, not just match words, and that’s what drove us to build SmartSearch—a system focused on grasping user intent and delivering accurate answers fast.
What role did the need for multilingual support play in shaping the design and goals of SmartSearch?
Multilingual support was a cornerstone from the start. With millions of pages of manuals and specs in 27 languages, and about 5% of that content updating monthly, we had to ensure users in any language could get the same quality of results. Keyword search often failed here—different languages have unique phrasing and terminology. SmartSearch uses vector-based embeddings to bridge those gaps, capturing the semantic essence of a query regardless of language. This wasn’t just a feature; it was a necessity to provide seamless support for riders and dealers worldwide.
Can you break down the core pipeline of SmartSearch and how it transforms a user query into a precise answer?
Absolutely. SmartSearch operates on a three-step pipeline designed for speed and accuracy. First, we crawl documentation with a Rust-based tool that processes about 25 webpages per second without hitting rate limits. Second, we chunk the HTML into meaningful pieces—separating titles from content and grouping related topics—before embedding them using advanced language models to map meaning into a semantic space. Finally, we rank results with a hybrid approach, blending 70% semantic search with 30% traditional keyword methods, and refine the top hits with a MiniLM cross-encoder. This ensures answers pop up in about 750 milliseconds, even under heavy load.
What were some of the toughest technical challenges you encountered while building SmartSearch, and how did you tackle them?
One of the biggest hurdles was hitting a 10-million vector cap in our initial vector store. Beyond eight million vectors, performance slowed significantly, and storage costs ballooned with 32-bit float embeddings. Re-indexing was another pain point—every metadata update was a slog. We had to rethink storage strategies, eventually adopting quantization to shrink costs and improve speed. These constraints forced us to optimize every layer of the system, from embedding models to database choices, ensuring we could scale without breaking the bank or sacrificing response times.
How did transitioning from a search bar to a conversational assistant change the demands on your system?
Moving to a conversational assistant was a game-changer. Unlike a search bar where a few decent links might suffice, a chatbot has to nail the first response—every token passed to a language model costs money, and a wrong answer erodes user trust instantly. Chat also exploded data needs; we’re not just pulling from docs but managing conversation histories and real-time follow-ups. Our old vector store couldn’t handle this—slow re-indexing, hard limits on vectors, and no support for efficient storage tiers. We needed a solution that could scale for chat’s relentless pace while keeping costs and latency low.
What made the integration of a new vector database so critical, and how did it transform SmartSearch’s capabilities for chat?
The old vector store cracked under chat-scale demands—hard vector limits, sluggish updates, and bloated storage costs were killing us. We tested several databases with punishing workloads, and the one we chose delivered exceptional recall, kept latency under 120 milliseconds even with 400 concurrent chats, and slashed storage costs by 16x through quantization. It natively supports multi-stage retrieval and tiered storage, letting us keep active data in RAM and less-used vectors on cheaper SSDs. This transformed SmartSearch into a lightning-fast, cost-effective backbone for our assistant, perfectly suited for real-time conversations.
How do agentic workflows elevate the assistant beyond simple retrieval to handling complex user needs?
Agentic workflows take our assistant from just finding answers to solving problems. Instead of relying on a single model to handle everything, we use a team of specialized AI agents. For a query like “My Kiox 300 shows error 503, how do I fix it, and can you draft a support message if needed?” an orchestrator agent breaks it into subtasks—error lookup, troubleshooting, and ticket drafting. Each task goes to a specialist agent that consults our retrieval system, gathers facts, and builds a response. This modular approach ensures precision, transparency, and the ability to handle multi-step tasks seamlessly, from specs to real-world follow-ups.
What’s a key lesson from this project that you think could shape the future of similar systems?
One lesson stands out: clean, well-structured data is worth more than any fancy model or algorithm. We spent a week deduplicating content and stripping out boilerplate, and the jump in search quality was bigger than any tech upgrade. Also, planning for storage and compute efficiency from day one—like compression and quantization—saves massive headaches later. As systems get more complex with agentic workflows, coordinating data, cost, and latency becomes the real challenge. Build with scalability and collaboration in mind from the start, and you’ll avoid a lot of pain down the road.
What’s your forecast for the future of conversational assistants in technical support domains like this one?
I see conversational assistants becoming the standard for technical support, especially in complex domains like eBike systems. They’ll evolve beyond answering questions to acting as true partners—proactively guiding users through diagnostics, coordinating repairs, or even predicting issues based on usage data. With advancements in smaller, specialized models and smarter workflows, latency and cost barriers will keep dropping, making these tools accessible to more industries. The key will be balancing intelligence with efficiency, ensuring systems stay fast, affordable, and deeply attuned to user needs across languages and contexts.
