resources Interfaces, Classes, Traits and Enums Tokenizer Turkish specific tokenization code. Typically, tokenizer.php either contains a stemmer for the language in question or it specifies how many characters in a char gram