Adds RemoveHidden bool field to ReadabilityOptions that evaluates JavaScript on the live page to remove all elements with computed display: none before readability extraction
Defines an internal pageEvaluator interface so the concrete Playwright-backed document supports JS evaluation without changing the Document interface
Adds PageEvaluate method to the document struct
Includes tests for the happy path, error propagation from JS evaluation, and the unsupported-Document case
Some websites embed hidden display: none elements as anti-AI-scraping honeypots containing prompt injection attacks. These are invisible to users but get picked up by readability extraction. CSS selectors can't target computed styles, so JavaScript evaluation with getComputedStyle is needed.
Test plan
go test ./... passes
Manual test against a page with known hidden honeypot elements (e.g. Together AI blog post mentioned in #62)
## Summary
- Adds `RemoveHidden bool` field to `ReadabilityOptions` that evaluates JavaScript on the live page to remove all elements with computed `display: none` before readability extraction
- Defines an internal `pageEvaluator` interface so the concrete Playwright-backed document supports JS evaluation without changing the `Document` interface
- Adds `PageEvaluate` method to the `document` struct
- Includes tests for the happy path, error propagation from JS evaluation, and the unsupported-Document case
Closes #62
## Context
Some websites embed hidden `display: none` elements as anti-AI-scraping honeypots containing prompt injection attacks. These are invisible to users but get picked up by readability extraction. CSS selectors can't target computed styles, so JavaScript evaluation with `getComputedStyle` is needed.
## Test plan
- [x] `go test ./...` passes
- [ ] Manual test against a page with known hidden honeypot elements (e.g. Together AI blog post mentioned in #62)
When RemoveHidden is true, JavaScript is evaluated on the live page to
remove all elements with computed display:none before readability
extraction. This defends against anti-scraping honeypots that embed
prompt injections in hidden DOM elements.
The implementation uses an optional pageEvaluator interface so that the
concrete document (backed by Playwright) supports it while the Document
interface remains unchanged.
Closes#62
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Summary
RemoveHidden boolfield toReadabilityOptionsthat evaluates JavaScript on the live page to remove all elements with computeddisplay: nonebefore readability extractionpageEvaluatorinterface so the concrete Playwright-backed document supports JS evaluation without changing theDocumentinterfacePageEvaluatemethod to thedocumentstructCloses #62
Context
Some websites embed hidden
display: noneelements as anti-AI-scraping honeypots containing prompt injection attacks. These are invisible to users but get picked up by readability extraction. CSS selectors can't target computed styles, so JavaScript evaluation withgetComputedStyleis needed.Test plan
go test ./...passes