Contributor: SWAG SUPPORT TEAM
Type
RW_toKEN = Record
token_str :String[9];
token_cod :toKEN_CODE;
end;
RW_Type = Array[0..9] of RW_toKEN;
RWT_PTR = ^RW_Type;
Const
NULL = '';
Rw_2 :RW_Type = ((token_str : 'do'; token_cod : tdo),
(token_str : 'if'; token_cod : tif),
(token_str : 'in'; token_cod : tin),
(token_str : 'of'; token_cod : tof),
(token_str : 'or'; token_cod : tor),
(token_str : 'to'; token_cod : tto),
(token_str : NULL; token_cod : NO_toKEN),
(token_str : NULL; token_cod : NO_toKEN),
(token_str : NULL; token_cod : NO_toKEN),
(token_str : NULL; token_cod : NO_toKEN)
);
...the difference being the explicit declaration of the Constant
Record fields. (I'm used to Array Constants, not Record
Constants - I was unaware of the requirement)
PARSinG NUMBERS
Now we'll concentrate on parsing Integer and Real numbers.
The Pascal definition of a number begins With an UNSIGNED
Integer. An unsigned Integer consists of one or more consecutive
DIGITS. The simplest Form of a number token is an unsigned
Integer:
1 9 120 12654
A number token can also be an unsigned Integer (the whole part)
followed by a fraction part. A fraction part consists of a
decimal point followed by an unsigned Integer, such as:
123.45 0.9987564
These numbers have whole parts 123 and 0 respectively, and
fraction parts .45 and .9987564 respectively.
A number token can also be a whole part followed by an EXPONENT
part. An exponent part consists of an "E" (or "e") followed by
an unsigned Integer. An optional exponent sign, + or -, can
appear between the letter and the first exponent digit.
Examples:
134e2 2E99 123e-45 73623E+4
Finally, a number token can be a whole part followed by a
fraction part and an exponent part, in that order:
2.3498E7 0.00034e-66
I arbitrarily limit the number of digits to 20, and the exponent
value from -37 to +37 - the exact value necessary to limit this
value is dependant on how Real values are represented on the
Computer.
The "get_number" Function is likely to be the biggest Function
in your scanner, but it should be relatively straighForward to
code...in light of what has already been done With the scanner/
tokenizer module, and the definition of a number.
EXERCISE #1
Write the get_number Function to parse Integers and Real
numbers.
You will need to add the following Types and Variables to your
global data segment:
Type { add "Real"s to list... }
LITERAL_Type = (Integer_LIT, Real_LIT, String_LIT);
LITERAL_REC = Record
Case lType:LITERAL_Type of
Integer_LIT: (ivalue :Integer);
Real_LIT : (rvalue :Real );
String_LIT : (svalue :String );
end;
Var
digit_count :Word;
count_error :Boolean;
-------------- PART 2 ---------------------------------------
The rest of this post will cover two simple topics - parsing
Strings inside quotes, and parsing comments.
PARSinG COMMENTS {}
The Compiler should ignore the input between two curly braces
({}), and the curly braces themselves. My scanner is written so
the entire comment is replace by a Single blank (" "), although
you could possibly Write the scanner so that comments are
_totally_ ignored.
EXERCISE #2:
Integrate COMMENT detection into the get_Char routine, so that
when your Character fetching routine will ignore comments and
pass a blank when a comment is encountered, skipping the comment
entirely For the next fetch.
Make sure that the routine keeps reading Until the right curly
brace is detected, even past the end-of-line. if the end-of-File
is encountered beFore the right curly brace is found, an
"unexpected end" error should be generated.
PARSinG StringS (QUOTES) ''
The quote Character delimits Strings, any Character between the
Strings is ignored by the Compiler, except to stored as a String
LITERAL. if you wish a ' (quote) to be included in the literal,
and extra ' must precede it.
One possible tricky area is the {} (comment) Character. You must
be careful not to inadvertently trigger the comment routine within
the quote routine While reading a String, otherwise you will
have a BUG.
EXERCISE #3:
Add a quote routine to the get_token routine within your module,
to fetch Strings, as a LITERAL IDENTifIER when the QUOTE
Character is detected.
The following mods to your Types are required:
Eof_Char = #$7F;
Type
Char_CODE = (LETTER, DIGIT, QUOTE, SPECIAL, Eof_CODE);
{ The following code init's the Character maping table: }
Var
ch :Byte;
begin
For ch := 0 to 255 do
Char_table[ch] := SPECIAL;
For ch := ord('0') to ord('9') do
Char_table[ch] := DIGIT;
For ch := ord('A') to ord('Z') do
Char_table[ch] := LETTER;
For ch := ord('a') to ord('z') do
Char_table[ch] := LETTER;
Char_table[ord(Eof_Char)] := Eof_CODE;
Char_table[39] := QUOTE;
end;
----------------------------------------------------------------
PLEASE, please let me know what you think about these posts,
even if they're negative - I want to have some feedback on the
difficulties, and whether or not people are having trouble
following the material - I _can_ be more concise at the cost of
being more verbose - if it's needed!
if you are having problems With your source code, and want me to
do a detailed examination of your code, expecially if it's
written in a language other than Pascal, send me email via the
Internet - to avoid "carpet bombing" the conference with
undesired material.
NEXT POST:
Error codes, and putting your code to the test - our first
utility (other than the lister) : a source Program Compactor
(not cruncher).
FUTURE POSTS:
- Review and (hopefully) a status report from "students"
- Symbol table
- YA utility (cross - referencer)
- YA utility (source Program CRUNCHer)
- YA utility (source Program UNcruncher)
- Parsing simple expressions
- Utility : CALC, using infix-to-postfix conversions and stack
ops.
- Parsing statements
- Utility: Pascal syntax checker part I
- Parsing declarations (Var, Type, etc)
incl's: much improved (and much more Complex) symbol table
- Utility: Declarations analyzer.
- Syntax Checker part II
- Parsing Program, Procedure, and Function declarations
(routines).
- Syntax checker Part III
- Review and discussion?