Hi All,

Just starting my online C programming class and my textbook states "In a computer language, a token is the smallest unit of the language that has a unique meaning. Thus, the reserved words, programmer-created identifiers, and all special mathematical symbols, such as + and -, are considered tokens of the C language."

Since the text previously states that the three type of C identifiers consist of keywords/reserved words, standard identifiers and programmer-created identifiers, then I'm presuming that both keywords and standard identifiers are also tokens, correct? And that punctuation marks ("()", "{}", ";" etc.), even though they represent attributes of the language, that they are considered to be part of the grammatical structure and are outside the definition of tokens, per se?

Regards,
Paul

> are outside the definition of tokens
No. The Standard defines token as either of:
keyword
identifier
constant
string-literal
punctuator
and punctuator in turn is either of
[ ] ( ) { } . ->
++ -- & * + - ~ !
/ % << >> < > <= >= == != ^ | && ||
? : ; ...
= *= /= %= += -= <<= >>= &= ^= |=
, # ##
<: :> <% %> %: %:%:

TYVM for your reply nezachem. So the text is wrong in regards to mathematical operators being considered as tokens?

Paul

Umm... they are tokens.

All operators (and thus all mathematics operators), are tokens. Your compiler help should have a list of them.

Thanks for your replies nezachem and Adak, very helpful and very much appreciated.

Paul

PS: My apologies nezachem, I totally missed the quoted part of your reply, which is why I asked for clarification.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.