BPE Tokenizer Simulator
Train a byte-pair encoding tokenizer one merge at a time, then use the learned vocabulary to encode and decode text. The simulator runs entirely in your browser.
Interactive demo
Open full screen ↗
Train a byte-pair encoding tokenizer one merge at a time, then use the learned vocabulary to encode and decode text. The simulator runs entirely in your browser.
Comments