Skip to content

Conversation

@Aksha1812
Copy link
Collaborator

@Aksha1812 Aksha1812 commented Oct 6, 2025

Ingestion

happy and happiness stem to -> happi
personal and person stem to -> person 

→ payload: {doc:1 happy, 
            doc:2 happiness, 
            doc:3 happi,
            doc:4 personal,
            doc:5 person
            }
            
Prefix Trie path:                  ()
                                /       \
                           happ      person   
                          /     \          \
                      iness      y          al  
                  
Posting objects to be stored :

 happy : Posting {doc:1},
 happi : Posting {doc:3} ,
 happiness : Posting {doc:2}, 
 personal: Posting {doc:4}, 
 person: Posting {doc:5}, 

Stem Trie path:                    ()
                                /       \
                           happi      person   

Stem Targets to be stored :

happi -> {happy, happiness} 
person -> {personal}

(even 'happi' stems to 'happi' but mapping a word to itslef doesnt make sense)

Search

A non exact term search should return both original word and all stem parents of the stemmed root of original word

For instance for query FT.SEARCH testindex '@desc:happi'

Results should include the original word + All stem targets of word happi + Exact Stem root if exists



Other things added with this PR

  • Addressed comments onGetLanguageString() function

while (!Done() && !key_iter_.ContainsFields(field_mask_)) {

if (stemming_enabled_) {
while (stem_target_ && stem_word_iter_ != stem_target_->end()) {
Copy link
Member

@KarthikSubbarao KarthikSubbarao Oct 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For stemming, it would be good to consider a separate word iterator. This code should be re-used through the GetStemmedWordIterator and not embedded in each search operation

Also, as discussed, we only will need to use this StemmedWordIterator during the initialization of the search operations once the open PR is merged in

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants