Tokens, Keywords and Identifiers

How the compiler reads your code

When you write a sentence, your brain does not read letter by letter. It sees whole words. The compiler works the same way.

Before it understands anything, it chops your file into small pieces called tokens. A token is the smallest thing that still means something on its own.

Take this line:

int age = 25;

The compiler sees five tokens:

That is it. Every C++ program, no matter how big, is just a long stream of these five kinds of tokens.

If you ever get an error like "expected ';' before something", the compiler is telling you it was chopping up tokens and found one it did not expect. It is not being difficult — it genuinely cannot continue.

The five kinds of tokens

1. Keywords

Keywords are words C++ has already claimed. They have a fixed meaning and you cannot use them for anything else.

There are about 95 of them. You do not need to memorise the list. These are the ones you will actually meet early:

int      float    double   char     bool     void
if       else     switch   case     default
for      while    do       break    continue
return   const    struct   class    public   private
true     false    new      delete   sizeof

int class = 5;   // error: 'class' is a keyword

2. Identifiers

An identifier is a name you invent — for a variable, a function, a struct, anything.

The rules are strict but simple:

1. Must start with a letter or an underscore _
2. After that, letters, digits and underscores only
3. No spaces, no hyphens, no symbols
4. Cannot be a keyword
5. Case matters — age, Age and AGE are three different names

int score;        // fine
int _count;       // fine
int player2;      // fine
int total_marks;  // fine

int 2player;      // error: cannot start with a digit
int my-score;     // error: hyphen is the minus operator
int my score;     // error: space splits it into two tokens

The case-sensitivity rule causes real bugs. This compiles and then confuses you for twenty minutes:

int totalScore = 0;
totalscore = 10;   // error: 'totalscore' was not declared

One extra rule worth knowing: identifiers starting with two underscores (__value) or an underscore followed by a capital (_Value) are reserved for the compiler. They will often work, but they can silently clash with internal names. Just avoid them.

3. Literals

A literal is a value written directly in the code. What you see is exactly what you get.

42          // integer literal
3.14        // floating-point literal
'A'         // character literal — single quotes, exactly one character
"Hello"     // string literal — double quotes
true        // boolean literal

The quote rule catches almost everyone once:

char grade = 'A';     // correct
char grade = "A";     // error: "A" is a string, not a char

4. Operators

Operators do something to values.

+   -   *   /   %        // arithmetic
=                        // assignment
==  !=  <   >   <=  >=   // comparison
&&  ||  !                // logical

The single biggest beginner bug in any language lives here:

if (x = 5) { }    // assigns 5 to x, then checks if 5 is "true" — always runs!
if (x == 5) { }   // compares x with 5 — what you meant

5. Punctuators

These give your code structure. They carry no value themselves.

;    // ends a statement
{ }  // groups statements into a block
( )  // function calls, and grouping in expressions
[ ]  // array indexing
,    // separates items

int a = 5
int b = 10;   // error reported HERE, but the mistake is on the line above

Notice where the error points. The compiler kept reading past the missing semicolon, so it blames the next line. When an error looks wrong, check the line above it. This single habit will save you hours.

Whitespace and comments

Spaces, tabs and newlines are called whitespace. The compiler uses them only to separate tokens, then throws them away.

These are identical to the compiler:

int x=5;

int    x   =   5   ;

Both compile to exactly the same program. Formatting is purely for humans — but that matters, because you will reread your own code far more often than you write it.

Comments are notes the compiler skips entirely:

// Single-line comment — everything after // is ignored

/* Multi-line comment
   spanning several lines */

One trap: multi-line comments cannot be nested.

/* outer
   /* inner */
   still outer?  */   // error — the first */ already closed it

Try it

Find every mistake below. There are four.

Common mistakes

1. Using = instead of == in a condition. Assigns instead of compares, and usually runs when you did not want it to.
2. Double quotes for a single character. 'A' is a char, "A" is a string.
3. Missing semicolon. The error appears on the next line — always look up.
4. Assuming names are case-insensitive. Score and score are unrelated.
5. Nesting /* */ comments. The first */ closes it.
6. Naming something after a keyword. class, new, delete, default are all off-limits.

Summary