Commit Graph

4 Commits

Author SHA1 Message Date
65cf6b027f feat: add RemoveHidden option to strip display:none elements before extraction
All checks were successful
CI / vet (pull_request) Successful in 34s
CI / test (pull_request) Successful in 1m1s
CI / build (pull_request) Successful in 1m5s
When RemoveHidden is true, JavaScript is evaluated on the live page to
remove all elements with computed display:none before readability
extraction. This defends against anti-scraping honeypots that embed
prompt injections in hidden DOM elements.

The implementation uses an optional pageEvaluator interface so that the
concrete document (backed by Playwright) supports it while the Document
interface remains unchanged.

Closes #62

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 14:06:17 +00:00
c1a5814732 feat: add ReadabilityWithOptions for DOM cleanup before extraction
All checks were successful
CI / build (pull_request) Successful in 46s
CI / test (pull_request) Successful in 48s
CI / vet (pull_request) Successful in 1m50s
Sites with infinite scroll (e.g. The Verge) load additional articles
into the DOM, which get included in readability extraction. Add
ReadabilityOptions.RemoveSelectors to strip elements by CSS selector
before parsing, avoiding the need to reimplement the readability
pipeline downstream.

Closes #60

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 01:09:28 +00:00
567a9f9212 added archive, megamillions, and powerball site logic 2024-12-23 03:18:50 -05:00
cbd6682257 initial commit 2024-12-07 03:53:46 -05:00