I am curious what the constraints are to make this work and in which environments it does? Does it work in PDF viewers outside the browser? Is there documentation what is available in which environment? What is enabled by default, can be switched on or off?
It might work in Foxit as I believe it supports some scripting. Most of the other native PDF renderers are more static, as far as I know. In either case, I was most interested in the browser-native engines, as I always thought of them as more "static"/limited.
As for documentation on specific features: to be honest, I just looked at the implementations of PDF.js and PDFium. Both only support a subset of the "standard" API, likely for security reasons. But PDF.js for example allows changing a field's background color (colored pixels!), and PDFium allows modifying their position/bounding box (I tried a high res color display by moving a row vertically as if it's a scanline, but things become quite laggy).
Hidden in some paragraph it does say
> Instead, PDF.js runs under the origin resource://pdf.js. This prevents access to local files, but it is slightly more privileged in other aspects.
Seems like it's not an XSS letting you take over the website origin, but it lets you run JS under this resource://pdf.js origin. Could be an interesting vector when combined with other weaknesses, but not an instant knock out as I expected when I read the title and saw the points :)
You are right for the case where Firefox's PDF.js is used (local or remote file in a tab or iframe). The XSS problem however is with web-applications that themselves use PDF.js. In that case, it does not run in a separate or special origin; that is a Firefox thing.
You are also right that the PDF format supports JavaScript, but that is something unrelated to this, and indeed highly sandboxed in all cases.
Great work! We had the same idea at the same time, here's my version of PDF Doom:
Source: https://github.com/thomasRinsma/pdfdoom
Playable here: https://th0mas.nl/downloads/doom.pdf
Yours is neater in many ways though!