PDFBOX-6200: Link the document state to substructures without recursion

[sharanggupta]

Fixes PDFBOX-6200 (and the same problem reported as PDFBOX-6039).

COSUpdateState.setOriginDocumentState() links the document state to the managed object and to all its substructures by calling itself for each entry of a dictionary or an array and for the target of a dereferenced COSObject. A deeply nested structure therefore overflows the stack, which is what the DSS users signing large AutoCAD documents with an incremental save hit in PDFBOX-6200.

As Michael Klink suggested on PDFBOX-6039, the recursion is replaced by a loop over an explicit stack (ArrayDeque) rather than a depth limit, so no structure is left without its document state. The traversal visits the same objects with the same rules: a state that already has an origin is skipped (which also stops at cycles), update() is called unless dereferencing, and the substructures are those of dictionaries, arrays and dereferenced indirect objects. Only the visiting order differs, which does not matter as each state is updated independently.

Test: TestCOSUpdateInfo.testSetOriginDocumentStateOfDeeplyNestedStructure links a document state to a chain of 50,000 nested dictionaries, arrays and indirect objects and checks that the deepest one is linked and marked as updated. On trunk it fails with a StackOverflowError thrown by the recursive setOriginDocumentState() calls; with the change it passes. The cos, pdfwriter and PDDocument tests pass, and the full pdfbox module build passes locally.

[THausherr]

Why not just do it the classic way by using a "visited" Set? Like in PageTree and Splitter?

[sharanggupta]

The recursive version already has the equivalent of a visited set: setOriginDocumentState() returns at once when the state already has an origin document state, so each object is visited only once and cycles terminate, as with the set in PageTree and Splitter. The overflow is caused by depth, not by cycles. The method goes depth-first through every entry of every dictionary and array and every dereferenced object, so the stack depth is the length of the longest path through objects that have not been visited yet. In a document with many interlinked objects, like the AutoCAD file in PDFBOX-6200, that path can run to tens of thousands of objects even though none is visited twice.

The test shows this: its chain of 50,000 nested objects has no cycle, and trunk still throws a StackOverflowError despite the visited check. A visited set doesn't bound the depth. Only an explicit stack does, or a depth limit as in the PDFBOX-6039 patch, which leaves objects without a state. The loop keeps the same visit-once check and only keeps the pending objects on the heap instead of the call stack.

[THausherr]

Ah, sorry, now I get it. The point is to eliminate recursion. (Which you mentioned in the code 🤦‍♂️)

[THausherr]

Thank you for your contribution, I have committed it; the PR should close automatically with the next commit (for some reason the github mirror is often 1 commit behind)