A token is the basic unit a language model handles: a fragment of a word, a word or a sign.
A token is the elementary piece of text a model processes. Depending on the language, a token can be a short word, part of a word or a punctuation mark.
Models split text into tokens before processing it, and their capacity is measured in the number of context tokens.
Billing for AI APIs is generally by the number of tokens consumed, both input and output.
Nos experts déploient ces concepts dans des solutions concrètes pour votre entreprise.