Most parser engines, including the GOLD Parser, use a Determistic Finite Automata (DFA) to implement the tokenizer. This part of the engine scans the input and determines when and if a series of characters can be recognized as a token.
The following tags were added in Version 2.5 of the Builder.
| ##DFA-TABLE | ||
| ... | ||
| [ | ##DFA-STATES | |
| ... | ||
| [ | ##DFA-EDGES | |
| ... | ||
| ##END-DFA-EDGES | ] | |
| ... | ||
| ##END-DFA-STATES | ] | |
| ... | ||
| ##END-DFA-TABLE |
| Name | Description |
|---|---|
| %Count% | The Count variable contains the number of states in the grammar's DFA State Table. This variable is useful for array declaration or setting the global variables before storing each of the actual rules. |
| %InitialState% | This variable contains the numeric index of the initial state in the DFA State Table. |
| Name | Description |
|---|---|
| %Delimiter% | For each state in the table, this variable is set to the value set with the ##Delimiter tag. For the last item in the list, the value is set to a number of spaces. |
| %AcceptIndex% | If the state can accept a symbol, this varible will contain the symbol's index. If the state cannot accept a symbol, the value will be -1. |
| %EdgeCount% | Each state contains zero or more state transition edges. |
| %Index% | This variable contains the index of the state in the table. |
| Name | Description |
|---|---|
| %Delimiter% | For each edge in the state, this variable is set to the value set with the ##Delimiter tag. For the last item in the list, the value is set to a number of spaces. |
| %CharSetIndex% | This variable contains the index of the symbol in the grammar's Symbol Table. |
| %Target% | This variable contains the target state for the edge. |
The following displays a template that will output the content of the DFA Table using formatted text.
| ##DFA-TABLE |
| Table Count: %Count% |
| Initial State: %InitialState% |
| ##DFA-STATES |
| State %Index% |
| Accept: %AcceptIndex% |
| Edge Count: %EdgeCount% |
| Edges: |
| ##DFA-EDGES |
| %CharSetIndex% goto %Target% |
| ##END-DFA-EDGES |
| ##END-DFA-STATES |
| ##END-DFA-TABLE |
If the "Simple" example grammar is used, the program template will create the following text for DFA State #14. The states before and after #14 were excluded for brevity.
| Table Count: 65 |
| Initial State: 0 |
| . |
| . |
| . |
| State 14 |
| Accept: 27 |
| Edge Count: 2 |
| Edges: |
| 24 goto 19 |
| 10 goto 18 |