[Python] Why does this grammar work using the Earley parser but not LALR(1)

I wrote this grammar to describe a simple patch procedure we use at work, operating on source code files. Engineers check in files of the form:

{ignored space}
//find_start {possible comment or directive}
{content to find}
//find_end
{ignored space}
//replace_start
{content to replace it with}
//replace_end
{ignored_space}
...


Given the following grammar, Lark, with the Earley parser, will produce an accurate parse tree of example text in the format.

start: block+

block: ignore? find_block ignore? replace_block ignore?

find_block: "//find_start" [comment] content "//find_end"
replace_block: "//replace_start" content "//replace_end"

_NL: /\n/+
LINE.-10: /.+/
COMMENT: /\s.+/
content: (LINE _NL)+
ignore: (LINE _NL)+
comment: (COMMENT _NL)

%import common.NEWLINE
%import common.WS
%ignore WS
%ignore NEWLINE


However, when attempting to use the LALR(1) parser, I get the following exception:

lark.exceptions.UnexpectedToken: Unexpected token Token('LINE', 'a') at line 3, column 1.
Expected one of:
* _NL
Previous tokens: [Token('LINE', 'asdf')]



I've tried several iterations of this grammar, and had alternating success in LALR(1) and Earley. This version is the first to produce the output I'm looking for. After spending some time looking online, I understand that the grammar I've cobbled together isn't the best, and I'm open to any suggestions for improvements.

Example text I've been using to test the grammar:

//find_start asdf
a
//find_end

//replace_start
A
//replace_end

//find_start
b
//find_end
this should be ignored
//replace_start
B
//replace_end

//find_start
c
//find_end

//replace_start
C
C
//replace_end


For those curious why we use such an admittedly hacky patch process, it's somewhat by necessity. These patches operate on Verilog RTL code. Due to IP restrictions, the engineers writing the original behavioral code are not allowed to know about library specifics. Physical design engineers like myself take this behavioral code and convert parts of it into structural, gate-level code for optimization. This results in patches that can never be checked back into upstream. Part of why I'm writing this grammar is to eventually remove this process entirely.

Continue reading...