Managing LLM Costs for High-Traffic NLWeb Searches

[Zubair-DS]

Suppose a site using NLWeb receives over 100+ k daily visitors who use only the AI-powered search (no conversions or purchases). The backend LLM is triggered for each search, potentially making token usage—and thus AI costs—quite high. What strategies or design patterns could be implemented to reduce unnecessary token usage in such heavy-traffic, low-conversion scenarios?