tokenize English

Meaning tokenize meaning

What does tokenize mean?

tokenize verb

— (transitive, computing) To reduce to a set of tokens by lexical analysis.

— (transitive, computing) To substitute sensitive data with meaningless placeholders.

— (transitive) To treat as a token minority.