[build] Add weekly CI check for Plausible pageview usage - #17900
Open
titusfortner wants to merge 1 commit into
Open
titusfortner wants to merge 1 commit into
titusfortner wants to merge 1 commit into
Conversation
Contributor
PR Summary by QodoAdd weekly GitHub Actions check for Plausible pageview overage + Slack alert
AI Description
Diagram
High-Level Assessment
Files changed (2)
|
Contributor
Code Review by Qodo
1. No tests for Plausible script
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
Comment on lines
+60
to
+63
| pageviews="$(jq -r '.results.pageviews.value // empty' <<<"$body" 2>/dev/null || true)" | ||
|
|
||
| if [[ ! "$pageviews" =~ ^[0-9]+$ ]]; then | ||
| echo "::error::No pageviews value in the Plausible response" |
Contributor
There was a problem hiding this comment.
1. No tests for plausible script 📘 Rule violation ☼ Reliability
This PR adds a new weekly CI check with non-trivial parsing and budget/threshold calculations, but introduces no test coverage to validate behavior across success/error responses. Lacking tests makes the workflow brittle and increases the chance of silent regressions in monitoring/alerting logic.
Agent Prompt
## Issue description
The new Plausible usage check script adds behavior (API response parsing and threshold/budget calculations) without any accompanying tests.
## Issue Context
Compliance requires new behavior to be covered by tests where practical, preferring small/unit tests. For this script, you can make the core logic testable without hitting the real Plausible API by isolating calculation/parsing into functions and feeding representative sample JSON inputs.
## Fix Focus Areas
- scripts/github-actions/check-plausible-usage.sh[60-101]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+39
to
+42
| if ! response="$(curl --silent --show-error --write-out '\n%{http_code}' \ | ||
| --get "https://plausible.io/api/v1/stats/aggregate" \ | ||
| --header "Authorization: Bearer ${PLAUSIBLE_STATS_KEY}" \ | ||
| --data-urlencode "site_id=${SITE_ID}" \ |
Contributor
There was a problem hiding this comment.
2. Curl request can hang 🐞 Bug ☼ Reliability
check-plausible-usage.sh calls Plausible via curl without any connection/overall timeout, so a stalled network request can block the scheduled workflow until GitHub’s job timeout and delay/prevent the Slack alert.
Agent Prompt
### Issue description
The Plausible API request uses `curl` without `--connect-timeout` and `--max-time` (and optionally retries). A hung TCP/TLS connection or slow upstream can stall the whole workflow run.
### Issue Context
This script is executed by a weekly scheduled workflow; the job only notifies Slack on failure, so a hung request delays the alert.
### Fix Focus Areas
- scripts/github-actions/check-plausible-usage.sh[39-45]
- .github/workflows/plausible-usage.yml[1-6]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+14
to
+16
| SITE_ID="${PLAUSIBLE_SITE_ID:-manager.selenium.dev}" | ||
| THRESHOLD="${PLAUSIBLE_THRESHOLD:-17000000}" | ||
| MONTHLY_LIMIT="${PLAUSIBLE_MONTHLY_LIMIT:-75000000}" |
Contributor
There was a problem hiding this comment.
3. Unvalidated numeric overrides 🐞 Bug ≡ Correctness
PLAUSIBLE_THRESHOLD and PLAUSIBLE_MONTHLY_LIMIT are documented as overridable but never validated as numeric; invalid values can be coerced (e.g., to 0) or trigger arithmetic/awk errors, causing incorrect over/under decisions or confusing failures.
Agent Prompt
### Issue description
`PLAUSIBLE_THRESHOLD` / `PLAUSIBLE_MONTHLY_LIMIT` are treated as numbers but are not validated before use. In bash arithmetic and awk, malformed values may be coerced or error out, leading to false alerts or hard-to-debug failures.
### Issue Context
The script already validates `pageviews` is numeric, but not the user-supplied numeric configuration.
### Fix Focus Areas
- scripts/github-actions/check-plausible-usage.sh[14-16]
- scripts/github-actions/check-plausible-usage.sh[62-76]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔗 Related Issues
Related to #17881
💥 What does this PR do?
selenium-tlcfor us to investigate for suspicious profiles.🔧 Implementation Notes
period=7d, which folds in today's partial data and makes the number depend on what time the job runs.🤖 AI assistance
💡 Additional Considerations
🔄 Types of changes