As we know that Software is a Program. And a Program is that which Contains set of instructions, and an Instruction contains Some Tokens. So Tokens are used for Writing the Programs the various Types of Tokens those are contained by the Programs.
As in the English language, in a paragraph all the words, punctuation mark and the blank spaces are called Tokens. Similarly in a C++ program all the C++ statements having Keywords, Identifiers, Constants, Strings, Operators and the Special Symbols are called C++ Tokens. C++ Tokens are the essential part of a C++ compiler and so are very useful in the C++ programming. A Token is an individual entity of a C++ program.
For example, some C++ Tokens used in a C++ program are:
Reserve: words long, do if, else etc.
Identifiers: Pay, salary etc.
Constant: 470.6,16,49 etc.
Strings: “Dinesh”, “2013-01” etc.
Operator: +, *, <, >=, &&,11, etc
Special symbols: 0, {}, #, @, %, etc.
Variable: A variable is used for storing a value either a number or a character and a variable also vary its value means it may change his value Variables are used for given names to locations in the Memory of Computer where the different constants are stored. these locations contain Integer ,Real or Character Constants.
For Naming a Variable There are Some Specific Rules
• A Variable name is any Combination of 1 to 8 alphabets digits or underscore.
• The First Character in the Variable name must be an alphabet
• No Commas or blanks spaces are allowed in variable name
• No Special Symbols are used in the name of the Variable.
Character Set:- A character set denotes any Alphabet or the Special Symbol that is used for representing the Information Like In C we uses Alphabets, digits and many Types of special Symbols
For Ex: in Alphabets we use A B …..Z or a b ……z
in Digits we use 1 2 3 etc
Special Symbols * ( ) _ # etc.
Data Type : Every variable has a data type which denotes the type of data which a variable will hold There are many built in data types in the c language those are following as
1) Int (integer)
2) Char (character)
3) float
4) double
5) Long
Constants: Constants, also known as literals: The term constants or literals refers to the fixed value means a Constant is that whose value is never changed at the end of the program.
for ex:
3x+2y=10
Here 3 and 2 and 10 are constants their value never be change but here x and y are the variables and may be they vary their value
There are two types of Constants in c Language.
1) Primary Constants
2) Secondary Constants
The Primary and Secondary Constants are further divided into the other Categorized
In Primary these are
Numeric Constants
Numeric constants refer to the numbers consisting of a sequence of digits (with or without decimal point) that can be either positive or negative. However, by default, numeric constants are positive. Numeric constants can be further classified as integer constants and floating-point constants, which are listed in Table
Character Constants
Character constants refer to a single character enclosed in single quotes (‘ ‘). The examples of character constants are ‘f’, ‘M’, ‘8 ‘, ‘& ‘, ‘7 ‘, etc. All character constants are internally stored as integer values.
Character constants can represent either the printable characters or the non-printable characters. The examples of printable character constants are ‘ a ‘ , ‘ 5 ‘ , ‘#’ , ‘ ; ‘ etc. However, there are few character constants that cannot be included in a program directly through a keyboard such as backspace, newline and so on. These character constants are known as non-printable constants and are included in a program using the escape sequences. An escape sequence refers to a character preceded by the backslash character (\). Some of the escape sequences used in C++ is listed in Table
String Constants
String constants refer to a sequence of any number of characters enclosed in the double quotes (” “). The examples of string constants are “hello” , “name”, “colour”, “date”, etc. Note that string constants are always terminated by the Null ( ‘\ 0 ‘ ) character.
The presence of a backs lash character in a string constant indicates an escape sequence. For example, the string constant “welcome \ “home” is displayed as welcome” home. Note that the double quote next to the backslash is an escape sequence and not a delimiter for the string constant.
In Secondary these are
1) Array
2) Pointer
3) Structure
4) Union
5) Enum or Enumeration.
For Making or Declaring a Integer Constant There are some Specific Rules like
• An Integer Constant must have at Least one Digit
• it must not have a Decimal value
• it could be either positive or Negative
• if no sign is Specified then it should be treated as Positive
• No Spaces and Commas are allowed in Name
• The Range of Integer Constant is -32768 to 32767
For Making or Declaring Real constant there are some Specific Rules like
• A Real Constant must have at Least one Digit
• it must have a Decimal value
• it could be either positive or Negative
• if no sign is Specified then it should be treated as Positive
• No Spaces and Commas are allowed in Name
In The Exponential Form of Representation the Real Constant is represented in the two Parts The part before appearing e is called mantissa whereas the part following e is called Exponent.
• In Real Constant the Mantissa and Exponent Part should be separated by letter e
• The Mantissa Part has may have either positive or Negative Sign
• Default Sign is Positive
• The Range of Real Constant is -3.4e38 to 3.4e38
For Making or Declaring a Character Constant there are some Specific Rules like
A Character is Single Alphabet a single digit or a Single Symbol that is enclosed within Single inverted commas.
• It must not have a Decimal value
• The Range of Character is -128 to 127
Identifiers The Identifiers are those which are used for giving a name to a variables, arrays, functions, classes, structures, namespaces and so on, like a b etc these are used for naming a variable. When we declare any variable then we specify a Name So that identifiers are used for naming a variable.
While defining identifiers in C++, programmers must follow the rules listed here .
• An identifier must be unique in a program .
• An identifier must contain only upper case and lower case letters, underscore character (_) ordigits 0 t0 9.
• An identifier must start with a letter or an underscore.
• An identifier in upper case is different from that in lower case.
• An identifier must be different from a keyword. In addition, identifiers that start with a double underscore ‘_’ or an underscore followed by an upper case letter must be avoided as these names are reserved by the Standard c++ Library.
• An identifier must not contain other characters such as ‘*’, ‘;’ and whitespace characters (tabs, space and newline).
Some valid andinvalid identifiers inC++ are listed here.
Po178_ddm //valid
_78hhvt4 //valid
902gt1 //invalid as it starts with a digit
Tyy;ui8 //invalid as it contains the ‘;’ character
for //invalid as it is a c++ keyword
Fg026 neo //invalid as it contains spaces
Keywords : A Keyword is that which have a special meaning those are already been explained to the c++ language like cout, cin, for, if, else , etc these are the reserve keywords Always Remember that we can t give a name to a variable as the name of a keyword we cant create new Keywords in c Language. All the keywords of C++ are listed in Table.
Operator: –An Operator is a Special Symbol, which is used for performing an Operation. So that Operator is that which have a Special Meaning and also have a Sign For Example we know that + is used for Adding two Numbers.
Depending on the number of operands and function performed, the C++ operators can be classified into various categories. These include arithmetic operators, relational operators, logical operators,’ the conditional operator, assignment operators, bitwise operators and other operators. These categories are further classified into unary operators, binary operators and ternary operators.
Statement: A Statement is that which Contains constants, variable, and also Some Operators for Example 2a+3b=146; is a Statement because in this 2, 3 and 146 are the constants and the + and = are operators and a and b are the variables. Always remember that a Statement always ends with a Semicolon. If you forgot the Semicolon at the end of the Statement then this will gives us the error.
Punctuators, also known as separators, are the tokens that serve different purposes based on the context in which they are used. Some punctuators are used as operators, some are used to demarcate a portion of the program and so on. The various punctuators defined in C ++ are asterisk ‘*’, braces ‘{ }’, brackets ‘[]’, colon ‘:’, comma ‘,’, ellipsis ‘ … ‘, equal to ‘=’, semicolon ‘;’, parentheses ‘()’ and pound (hash) ‘#’.
Dinesh Thakur holds an B.C.A, MCDBA, MCSD certifications. Dinesh authors the hugely popular