Searching for messages with special characters

[jhwheeler]

Problem

We're looking for guidance on searching messages with special characters, i.e. non-alphanumeric characters. We've tried several approaches, including some post-processing techniques and conditional query composition, but haven't found a consistent method.

Test Cases We've Attempted

1. Searching for a message that is solely comprised of non-alphanumeric characters, for example ???:

{ text: { $q: "???" } } // ❌ no matches

{ text: { $autocomplete: "???" } } // ❌ no matches

{ text: { $eq: "???" } } // ✅ match

$q and $autocomplete return 0 results. While $eq does return the message, if we do a partial search, e.g. ? or ??, then of course $eq won't match, and $q and $autocomplete have the same behavior of returning 0 results.

This occurs for all of the non-alphanumeric character that we have tested: ~!@#$%^&*()_+-={}|[]\:;"'<>?,./.

2. Searching for a message that contains both alphanumerics and non-alphanumerics, for example "foo bar" or "hello":

$q seems to work only if the query runs up against a word boundary:

• Searching for "foo bar":
  • "f, "fo: ❌ no matches
  • "foo: ✅ match
• Searching for "hello":
  • "h, "he, "hel, "hell: ❌ no matches
  • "hello, "hello": ✅ match

$autocomplete does not have this issue; all of the above will match.

Request

Can you provide some guidance as to how we can solve this? Using $autocomplete does solve the second issue, but not the first one (searching for messages that are only comprised of non-alphanumeric characters). This reduces usability and causes apparent bugs for our end users when they are searching for messages such as ?, !, ?!?!, and the like.

Thank you!