The search world is very big :) 5 mil records is nothing if you index logs (which is not Algolia typical use-cases) but for example this is big from an e-commerce perspective.
In the e-commerce world, the difference between 2ms and 200ms isn't that big of a deal. Search relevance, however, might be something that is important. It looks like that is something they are focusing on heavily.
Your algorithm seems OK, but what was the "traditional approach" that you compared it to, and how did you compare them? It seems like you actually gain a lot from full document search (e.g. products with multi-paragraph descriptions). Otherwise, you might as well just do a SQL query to get your results.
Ahaha, speed is huge deal for e-commerce. It has already been proved enough by big merchants tests.
Search relevance is too! The key is to have them both at the same time :)