Well, not as clear-cut as I thought.
First problem is on the listing. The nesting indicator is only two digits, so greater than 99 levels of nesting and it prints as "**".
However, more than 99 levels of nesting work.
However, nowhere near as many as I thought.
It seems like 16380 is either the number of levels of nesting you can have, or 32760 is the maximum number of statements you can include in the nest. I'm not going to look further to find out which
Beyond that point the compiler goes to pot, with a number of weird errors.
So, yes, there is an undocumented limit to the number of levels of nesting, at least for an IF and presumably for anything else that can be nested.
However, except for a small possibility for generated code, the limit is so far from approachable as to not matter. Even the 99 levels of nesting would be enormously difficult to code, understand or change.
At 16380 levels the program does still work
At 16381, it just doesn't compile, with multiple messages entirely undrelated to the nested code.
Out of interest, here's how I generated the program, using DFSORT.
//SYSIN DD *
OPTION COPY
OUTFIL FNAMES=OUT1,INCLUDE=(1,1,CH,EQ,C'A'),REMOVECC,
HEADER1=(8:'ID DIVISION.',/,
8:'PROGRAM-ID. NESTING.',/,
8:'DATA DIVISION.',/,
8:'WORKING-STORAGE SECTION.',/,
8:'01 A PIC X VALUE "A".',/,
8:'01 B PIC X VALUE "B".',/,
8:'01 C PIC X(8)BX(8) VALUE "NEW".',/,
8:'PROCEDURE DIVISION.')
OUTFIL FNAMES=OUT2,INCLUDE=(1,1,CH,EQ,C'A'),REMOVECC,
TRAILER1=(12:'MOVE B TO A',/,
12:'.',/,
12:'MOVE WHEN-COMPILED TO C',/,
12:'DISPLAY C',/,
12:'DISPLAY ">" A "<"',/,
12:'GOBACK',/,
12:'.')
OUTFIL REPEAT=16380
//SORTIN DD *
IF A EQUAL TO B CONTINUE ELSE
I then concatenated the three output datasets (OUT1, SORTOUT, OUT2) as input to the COBOL compiler.
On executing the program, the value "B" is displayed, which is the expected result. I turned the OPTimizer off, of course. Mmmm.... perhaps I should try with it on as well, to see what happens...