Commit Graph

64 Commits

Author SHA1 Message Date
132817144e refactor: deduplicate numericOnly and DuckDuckGo result extraction
All checks were successful
CI / build (pull_request) Successful in 29s
CI / vet (pull_request) Successful in 1m1s
CI / test (pull_request) Successful in 1m4s
- Extract identical numericOnly inline functions from powerball and
  megamillions into shared sites/internal/parse.NumericOnly with tests
- Extract duplicated DuckDuckGo result parsing from Search() and
  GetResults() into shared extractResults() helper

Closes #13, #14

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:26:54 +00:00
384566e016 Merge pull request 'Consolidate user-agent strings (#17)' (#39) from fix/user-agent-consistency into main
All checks were successful
CI / build (push) Successful in 1m34s
CI / vet (push) Successful in 1m34s
CI / test (push) Successful in 1m34s
2026-02-15 16:25:03 +00:00
097b2e12c7 fix: consolidate user-agent strings into DefaultUserAgent constant
All checks were successful
CI / build (pull_request) Successful in 44s
CI / test (pull_request) Successful in 46s
CI / vet (pull_request) Successful in 1m28s
Define DefaultUserAgent (Firefox/147.0) in playwright.go and reference
it from NewBrowser, NewInteractiveBrowser, and CLI flags. Previously
three different UA strings existed (two at 142.0, one outdated at 133.0).

Closes #17

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:24:44 +00:00
0df639abea Merge pull request 'Fix ShowBrowser merge + consistent browser defaults (#15, #16)' (#38) from fix/merge-options-and-browser-defaults into main
All checks were successful
CI / build (push) Successful in 32s
CI / vet (push) Successful in 1m50s
CI / test (push) Successful in 1m51s
2026-02-15 16:23:07 +00:00
328455de32 fix: ShowBrowser merge behavior and consistent browser defaults
All checks were successful
CI / vet (pull_request) Successful in 1m49s
CI / build (pull_request) Successful in 1m51s
CI / test (pull_request) Successful in 1m52s
- Change ShowBrowser from bool to *bool so nil means "don't override"
  in mergeOptions(), fixing the bug where it always overwrote the base
- Add Bool() helper for convenient *bool construction
- Align NewInteractiveBrowser default from Chromium to Firefox to match
  NewBrowser
- Update README example and CLI flags for the *bool change

Closes #15, #16

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:22:49 +00:00
85e4632ea9 Merge pull request 'Fix updateCookies error + context-aware sleep (#7, #18)' (#37) from fix/cookies-error-and-context-sleep into main
All checks were successful
CI / test (push) Successful in 1m33s
CI / build (push) Successful in 1m35s
CI / vet (push) Successful in 1m34s
2026-02-15 16:20:06 +00:00
769b870a17 fix: check Cookies() error and use context-aware sleep
All checks were successful
CI / build (pull_request) Successful in 46s
CI / vet (pull_request) Successful in 47s
CI / test (pull_request) Successful in 1m22s
- playwright.go: check error from page.Context().Cookies() before
  iterating over results, preventing silent failures
- archive.go: replace time.Sleep(5s) with context-aware select using
  time.After, allowing the operation to be cancelled promptly

Closes #7, #18

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:19:49 +00:00
8b136b9dda Merge pull request 'Fix cmd flags and defer-before-error-check (#8, #19)' (#36) from fix/cmd-flags-and-defer-ordering into main
All checks were successful
CI / vet (push) Successful in 37s
CI / test (push) Successful in 51s
CI / build (push) Successful in 52s
2026-02-15 16:18:54 +00:00
fca50a47c3 Merge pull request 'Fix DuckDuckGo error handling (#5, #6)' (#35) from fix/duckduckgo-error-handling into main
Some checks failed
CI / build (push) Has been cancelled
CI / test (push) Has been cancelled
CI / vet (push) Has been cancelled
2026-02-15 16:18:50 +00:00
991c43d020 Merge pull request 'Fix archive cmd panic on short content (#9)' (#34) from fix/archive-cmd-short-content into main
Some checks failed
CI / test (push) Has been cancelled
CI / build (push) Has been cancelled
CI / vet (push) Has been cancelled
2026-02-15 16:18:46 +00:00
2aa565d3a0 Merge pull request 'Fix nil-pointer panics (#10, #11)' (#33) from fix/nil-pointer-panics into main
Some checks failed
CI / build (push) Has been cancelled
CI / test (push) Has been cancelled
CI / vet (push) Has been cancelled
2026-02-15 16:18:41 +00:00
2af4cbcdce Merge pull request 'Fix XSS vulnerability in SetAttribute (#12)' (#32) from fix/escape-javascript-xss into main
Some checks failed
CI / build (push) Has been cancelled
CI / vet (push) Has been cancelled
CI / test (push) Has been cancelled
2026-02-15 16:18:36 +00:00
e5e0db85e8 fix: use merged flags in archive cmd and move defer after error checks
All checks were successful
CI / vet (pull_request) Successful in 29s
CI / build (pull_request) Successful in 32s
CI / test (pull_request) Successful in 57s
- Fix archive cmd passing only archive-specific Flags instead of the
  merged flags variable that includes browser flags (#8)
- Move defer DeferClose() after error checks in 6 locations to prevent
  calling Close on nil values (#19):
  - sites/duckduckgo/cmd/duckduckgo/main.go
  - sites/duckduckgo/duckduckgo.go
  - sites/google/cmd/google/main.go
  - sites/wegmans/cmd/wegmans/main.go
  - sites/wegmans/wegmans.go
  - sites/aislegopher/aislegopher.go

Closes #8, #19

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:17:38 +00:00
a12c9f7cb6 fix: propagate errors from DuckDuckGo search and GetResults
Some checks failed
CI / test (pull_request) Failing after 6m12s
CI / vet (pull_request) Failing after 6m12s
CI / build (pull_request) Failing after 6m15s
- Change SearchPage.GetResults() to return ([]Result, error) so ForEach
  errors are no longer silently discarded
- Fix Search() to return the ForEach error instead of nil
- Update cmd caller to check GetResults() errors

Closes #5, #6

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:16:04 +00:00
b4e462a6b4 fix: prevent panic on short article content in archive cmd
All checks were successful
CI / vet (pull_request) Successful in 1m6s
CI / build (pull_request) Successful in 1m7s
CI / test (pull_request) Successful in 1m8s
Add length check before slicing article.Content[:32], matching the
safe truncation pattern already used in cmd/browser/main.go.

Closes #9

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:14:32 +00:00
6c68062e56 fix: add nil guards to prevent nil-pointer panics
All checks were successful
CI / test (pull_request) Successful in 46s
CI / build (pull_request) Successful in 47s
CI / vet (pull_request) Successful in 59s
- document.go: check if resp is nil before calling resp.Status() in
  Refresh(), since Playwright's Reload() can return a nil response
- archive.go: check SelectFirst() results for nil before calling
  Type() and Click(), preventing panics when DOM elements are missing

Closes #10, #11

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:13:43 +00:00
6e94bfe10f fix: eliminate XSS vulnerability in SetAttribute by using Playwright arg passing
All checks were successful
CI / build (pull_request) Successful in 47s
CI / test (pull_request) Successful in 48s
CI / vet (pull_request) Successful in 1m1s
Replace string interpolation in SetAttribute with Playwright's Evaluate
argument passing mechanism. This structurally eliminates the injection
surface — arbitrary name/value strings are safely passed as JavaScript
arguments rather than interpolated into the expression string.

The vulnerable escapeJavaScript helper (which only escaped \ and ') is
removed since it is no longer needed.

Closes #12

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:12:46 +00:00
49f294e884 docs: add README.md and CLAUDE.md
All checks were successful
CI / test (push) Successful in 32s
CI / vet (push) Successful in 45s
CI / build (push) Successful in 46s
Add project documentation:
- README.md with installation, usage examples, API reference, and project structure
- CLAUDE.md with developer guide, architecture overview, conventions, and issue label docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 11:10:28 -05:00
05ca15b165 fix: add go.sum to repository for CI builds
All checks were successful
CI / build (push) Successful in 34s
CI / vet (push) Successful in 57s
CI / test (push) Successful in 1m0s
The go.sum file was not tracked, causing CI to fail with
"missing go.sum entry" errors during build/test/vet.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 14:15:43 -05:00
294097c3b6 fix: use setup-go@v3 (latest available on gitea.com mirror)
Some checks failed
CI / vet (push) Failing after 26s
CI / build (push) Failing after 32s
CI / test (push) Failing after 36s
The gitea.com/actions/setup-go mirror only has tags up to v3.
v3 supports go-version-file which is all we need.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 14:13:46 -05:00
022e002f98 ci: use gitea.com action mirrors instead of github.com
Some checks failed
CI / test (push) Failing after 15s
CI / vet (push) Failing after 15s
CI / build (push) Failing after 18s
GitHub is returning 500 errors for actions/checkout and actions/setup-go.
Switch to Gitea's own mirrors at gitea.com/actions/ to avoid the dependency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 14:09:46 -05:00
51ce639994 ci: re-trigger workflow after transient GitHub 500 error
Some checks failed
CI / test (push) Failing after 10s
CI / build (push) Failing after 2m2s
CI / vet (push) Failing after 2m2s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 14:06:39 -05:00
cb2ed10cfd refactor: restructure API, deduplicate code, expand test coverage
Some checks failed
CI / build (push) Failing after 2m4s
CI / test (push) Failing after 2m6s
CI / vet (push) Failing after 2m19s
- Extract shared DeferClose helper, removing 14 duplicate copies
- Rename PlayWright-prefixed types to cleaner names (BrowserOptions,
  BrowserSelection, NewBrowser, etc.)
- Rename fields: ServerAddress, RequireServer (was DontLaunchOnConnectFailure)
- Extract shared initBrowser/mergeOptions into browser_init.go,
  deduplicating ~120 lines between NewBrowser and NewInteractiveBrowser
- Remove unused locator field from document struct
- Add tests for all previously untested packages (archive, aislegopher,
  wegmans, useragents, powerball) and expand existing test suites
- Add MIGRATION.md documenting all breaking API changes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 13:59:47 -05:00
e7b7e78796 fix: bug fixes, test coverage, and CI workflow
Some checks failed
CI / vet (push) Failing after 15s
CI / build (push) Failing after 30s
CI / test (push) Failing after 36s
- Fix Nodes.First() panic on empty slice (return nil)
- Fix ticker leak in archive.go (create once, defer Stop)
- Fix cookie path matching for empty and root paths
- Fix lost query params in google.go (u.Query().Set was discarded)
- Fix type assertion panic in useragents.go
- Fix dropped date parse error in powerball.go
- Remove unreachable dead code in megamillions.go and powerball.go
- Simplify document.go WaitForNetworkIdle, remove unused root field
- Remove debug fmt.Println calls across codebase
- Replace panic(err) with stderr+exit in all cmd/ programs
- Fix duckduckgo cmd: remove useless defer, return error on bad safesearch
- Fix archive cmd: ToConfig returns error instead of panicking
- Add 39+ unit tests across 6 new test files
- Add Gitea Actions CI workflow (build, test, vet in parallel)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 11:14:19 -05:00
steve
e807dbb2ff feat: add KeyboardInsertText to InteractiveBrowser
Exposes Playwright's Keyboard.InsertText() which dispatches only an
input event (no keydown/keyup). This is essential for pasting text
into password fields and custom input components that don't handle
rapid-fire synthetic key events from Type().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 15:16:15 +00:00
52a9cb585d feat: add InteractiveBrowser API for remote browser control
Exposes low-level mouse, keyboard, screenshot, navigation, and cookie
extraction APIs via a new InteractiveBrowser interface. Designed for
interactive browser proxy sessions where direct page control is needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 02:58:00 -05:00
868acfae40 Add context support to Playwright browser initialization
Refactored Playwright initialization to ensure context propagation. Updated `NewPlayWrightBrowser` and related methods to accept `context.Context` for better cancellation and timeout handling. Improved error resilience and concurrency during browser setup.
2025-10-28 00:24:19 -04:00
82fce5a200 Handle unit suffix in price parsing and add logging
Refined price parsing logic to strip trailing periods from units (e.g., "lb." -> "lb") for better handling. Added logging for debugging extracted response data.
2025-10-20 22:36:20 -04:00
5fe7313fa4 Refine status check logic when handling document requests in Playwright 2025-10-12 20:17:04 -04:00
39c2c7d37a Add UseLocalOnly flag to connection options in Playwright logic 2025-10-12 00:17:11 -04:00
e32a6fa791 Add UseLocalOnly option to Playwright connection logic
Introduced the `UseLocalOnly` option to prevent connections to a remote Playwright server and enforce usage of the local server. Updated relevant connection logic to respect this new option.
2025-10-12 00:10:58 -04:00
afa0238758 Restrict Price assignment to unit price with "lb" only 2025-10-11 23:48:09 -04:00
9ae8619f93 Enhance price parsing to handle non-zero unit price
Updated price extraction logic to set `Price` from `UnitPrice` when it is non-zero, ensuring more accurate parsing.
2025-10-11 23:34:41 -04:00
f4caef22b0 Add timeout option to Playwright server connection
Introduced a 30-second timeout for connecting to the Playwright server. Added logging for connection attempts to improve debugging and enhance connection reliability.
2025-10-10 20:25:27 -04:00
9947cae947 Refine selectors and enhance price parsing with logging
Adjusted HTML selectors for improved compatibility and updated price parsing logic to handle additional formats. Added logging to provide better debugging insights during price extraction.
2025-10-10 14:42:06 -04:00
Steve Dudenhoeffer
dc43d1626a Parse drawing date from Powerball numbers page 2025-09-16 11:17:04 -04:00
Steve Dudenhoeffer
2d60940001 Refactored jackpot handling and updated dependencies
Replaced `currency.Amount` with `int` for jackpot values to simplify representation. Adjusted parsing logic accordingly. Updated Go version to 1.24.0 and refreshed dependencies in go.mod for compatibility.
2025-09-16 10:52:49 -04:00
d0fffb0411 Add warning log for Playwright server connection failure
Introduce a warning log to notify when failing to connect to the Playwright server, followed by a fallback to launching a local browser.
2025-08-29 01:39:40 -04:00
8b4e43c40f Add DontLaunchOnConnectFailure and refine server connection logic
Introduced the `DontLaunchOnConnectFailure` option to prevent browser launch on connection failure when connecting to a Playwright server. Enhanced environment variable handling for server address based on browser type, improving flexibility and reliability in connection scenarios.
2025-08-29 01:38:00 -04:00
6f4ca22b6a Update Firefox user agent version in PlayWrightBrowserOptions
Adjusted the default `UserAgent` in `PlayWrightBrowserOptions` to reflect the latest Firefox version (142.0) for improved compatibility and accuracy.
2025-08-29 00:25:45 -04:00
8aee8f0502 Add support for connecting to a remote Playwright server
Introduced `PlayWrightServerAddress` to `PlayWrightBrowserOptions`, allowing the browser to connect to a remote Playwright server if specified. Defaults to the `PLAYWRIGHT_SERVER_ADDRESS` environment variable. Updated initialization logic to handle both local launches and remote connections seamlessly.
2025-08-29 00:25:23 -04:00
Steve Dudenhoeffer
203b97d957 Update default UserAgent string in PlayWrightBrowser
Changed the UserAgent to represent a macOS system using Firefox 137.0. This ensures the browser identification aligns with updated standards and improves compatibility.
2025-05-27 01:46:06 -04:00
39453288ce Add OpenSearch and SearchPage functionality for DuckDuckGo
Introduced the `OpenSearch` method and `SearchPage` interface to streamline search operations and allow for loading more results dynamically. Updated dependencies and modified the DuckDuckGo CLI to utilize these enhancements.
2025-03-18 02:42:50 -04:00
7c0e44a22f Add viewport dimensions and dark mode support
This commit introduces optional viewport dimensions and dark mode support to the PlayWrightBrowserOptions struct and its usage. It ensures more control over browser display settings and improves flexibility when configuring browser contexts. Additionally, visibility checking logic in SetHidden was refined to avoid redundant operations.
2025-03-15 00:46:02 -04:00
0f9f6c776d Rename SetVisible to SetHidden for clearer semantic meaning
The method and its implementation now align with setting an element's "hidden" property instead of "visible." This change improves code clarity and consistency with expected behavior.
2025-03-03 23:39:37 -05:00
62cb6958fa Add SetVisible and SetAttribute methods to Node interface
This commit introduces two new methods, SetVisible and SetAttribute, to the Node interface. These methods allow toggling element visibility and setting attributes dynamically. Additionally, a helper function, escapeJavaScript, was added to ensure proper escaping of JavaScript strings.
2025-03-03 23:31:51 -05:00
964a98a5a8 Handle commands without automatic reaction responses
Introduce `ErrCommandNoReactions` to allow commands to opt out of success reactions. Adjust bot behavior to respect this error and prevent reactions when applicable, ensuring cleaner and more controlled responses. Add error handling and safeguard workers against panics.
2025-01-22 21:06:07 -05:00
81ea656332 Add unit price and unit parsing for items
This update enhances the `Item` structure to include `UnitPrice` and `Unit` fields. Additional logic is implemented to extract and parse unit pricing details from the HTML, improving data accuracy and granularity.
2025-01-21 19:42:25 -05:00
6de455b1bd Add price extraction and validate URL structure in parsers
Added price field to Item struct in AisleGopher and implemented logic to extract price data. Updated Wegmans parser to validate URL structure by ensuring the second segment is "product". These changes improve data accuracy and error handling.
2025-01-20 13:00:59 -05:00
f37e60dddc Add Wegmans module to fetch item details and prices
Introduce functionality to retrieve item details, including name and price, from Wegmans using a browser-based scraper. This includes a CLI tool to execute searches and robust error handling for URL validation and browser interactions.
2025-01-20 12:28:29 -05:00