,
4 tweets,
1 min read
Read on Twitter
The oldest piece of CPython, the parser generator (pgen) has been retired and replaced with a new version! 🐍🖥️🥳
The parser generator was the first piece written for Python 29 years ago. It generates the parser rules (in the form of DFAs) from the grammar specification file.
The parser generator was the first piece written for Python 29 years ago. It generates the parser rules (in the form of DFAs) from the grammar specification file.
The new pgen, based on lib2to3/pgen2 is completely written in Python and has much better error reporting: For example, if a grammar rule with multiple productions is not LL(1) it will indicate which terminal collides in the first sets of the productions.
This also simplifies the parser code (no more #ifdef preprocessor branches only for pgen stuff all over the place) and it deduplicates the token definitions!
