feat: allow BPETrainer to be seeded with a set of initial tokens#1862

henrycharlesworth wants to merge 3 commits intohuggingface:mainhuggingface/tokenizers:mainfrom

henrycharlesworth:bpe_initial_tokenshenrycharlesworth/tokenizers:bpe_initial_tokensCopy head branch name to clipboard