feat/r-script-commands-triggers - #438
Abhishek84313 wants to merge 2 commits into
Conversation
fdelbrayelle
left a comment
There was a problem hiding this comment.
Kestra Plugin Code Review
Business Requirements — met
No linked issue is referenced in the PR body. The stated goal (bring plugin-script-r trigger support to parity with the Shell/Node modules by adding ScriptTrigger/CommandsTrigger) is self-consistent and fully implemented, with tests and QA examples matching the description.
Kestra Guidelines — 4 findings
- 🟠 TODO placeholders left in production code (both new files) — forbidden by the Code Comments guideline.
- 🟠
lastMatched(AtomicBoolean) not excluded from Lombok@ToString/@EqualsAndHashCode(both files) — correctness bug, see inline comment. - 🟠 Plugin how-to doc
plugin-script-r/src/main/resources/doc/io.kestra.plugin.scripts.r.mdnot updated to mention the two new triggers — mandatory per the 'Plugin How-To Doc' section whenever a trigger is added to an existing plugin. Node's equivalent doc has a one-line mention of itsCommandsTrigger/ScriptTrigger; the R doc still only listsScript/Commands. - 🟡 Required
exitConditionis rendered with.orElse("")(both files) instead of failing loudly on an unrenderable value — see inline comment.
Security (OWASP Top 10:2025 + KPS) — 0 blocking issues
No secret fields, no new HTTP calls, no unsafe deserialization, no shell string concatenation beyond what the existing Commands/Script tasks already do (out of scope of this diff). The regex-timeout resource-leak noted under Performance has a DoS angle (A10-adjacent) but is tracked there since its primary consequence is thread-pool exhaustion, not an authorization/injection gap.
Performance — 1 finding
- 🟠 ReDoS guard in
matchesConditionabandons the backtracking regex thread on the sharedForkJoinPool.commonPool()instead of cancelling it — see inline comments on both files.
Additional non-blocking notes
- 🟡 Edge-mode tests (
edgeMode_preventsConsecutiveEmitinCommandsTriggerTest, and the threeedgeMode_*tests inScriptTriggerTest) re-implement!lastMatched.getAndSet(x) && xinline rather than callingtrigger.evaluate()twice on the same instance. They verifyAtomicBooleanarithmetic, not the trigger's actual edge-mode wiring — a regression inevaluate()'s edge logic wouldn't be caught by these tests. Given edge mode is this PR's headline feature, at least one@KestraTestshould callevaluate()twice on the same trigger instance and assert the second call returnsOptional.empty(). - 🟡 DRY:
CommandsTrigger/ScriptTriggerduplicate ~90% of their logic (evaluate,matchesCondition,extractFailure,Output). This mirrors pre-existing duplication already merged for Shell/Node/Ruby, so it's not net-new debt introduced by this PR, and the TODO (flagged above for violating the no-TODO-comment rule) at least names the follow-up. Non-blocking, but worth tracking as a real issue rather than a comment. - 🟢 Nit: trigger/task ids in tests are fixed strings (
"commands-trigger", …) rather than randomized, per the Flaky Test Prevention guideline — mirrors existing Node/Shell test precedent, so not introduced by this PR.
Verdict: REQUEST CHANGES
| } | ||
| ) | ||
| // TODO: extract shared trigger logic (evaluate, matchesCondition, extractFailure, Output) | ||
| // into an AbstractScriptTrigger in plugin-script to reduce duplication across Shell, Node, Ruby, R, etc. |
There was a problem hiding this comment.
🟠 [Guidelines] TODO placeholder forbidden
Problem: The Code Comments guideline states: "Never add section banners, dividers, or TODO placeholders unless explicitly requested." This TODO documents real follow-up debt but violates that rule as written.
Fix: Move this note into the PR description (already done there) or a tracked follow-up issue instead of a source comment.
| // so edge mode may re-fire once after a scheduler restart. | ||
| @Builder.Default | ||
| @Getter(AccessLevel.NONE) | ||
| private final AtomicBoolean lastMatched = new AtomicBoolean(false); |
There was a problem hiding this comment.
🟠 [Guidelines/Correctness] Mutable AtomicBoolean not excluded from Lombok equals/hashCode/toString
Problem: java.util.concurrent.atomic.AtomicBoolean does not override equals()/hashCode() (identity-based). Because the class-level @EqualsAndHashCode includes this field by default, two structurally identical CommandsTrigger instances (e.g. the same trigger deserialized twice from the same flow YAML, as happens on every flow parse/revision check) will always compare unequal, since each gets its own AtomicBoolean(false) instance. This is exactly the case the guidelines call out under Task Lifecycle Hooks: 'any non-config field carrying runtime state ... must be annotated @EqualsAndHashCode.Exclude and @ToString.Exclude.'
Fix: Annotate lastMatched with @EqualsAndHashCode.Exclude and @ToString.Exclude.
| var future = CompletableFuture.supplyAsync( | ||
| () -> pattern.matcher(haystack).find() | ||
| ); | ||
| return future.get(5, TimeUnit.SECONDS); |
There was a problem hiding this comment.
🟠 [Performance] Abandoned future leaks a thread on the shared common pool
Problem: On TimeoutException the CompletableFuture is never cancelled. java.util.regex.Matcher#find() is not interruptible, so the ForkJoinPool.commonPool() worker running the catastrophic-backtracking regex keeps burning CPU indefinitely in the background — it isn't freed when this method returns. Repeated pathological exitCondition values across polls (interval can be as low as a few seconds) progressively pin common-pool worker threads, starving every other CompletableFuture/parallel-stream consumer in the same worker JVM, including unrelated plugins.
Fix: Run the match on a dedicated, bounded single-thread daemon executor created once per class (not the shared common pool) so a stuck matcher only ever wastes one dedicated thread instead of contending with the JVM-wide common pool; consider also caching/rejecting patterns that have already timed out once to avoid repeatedly re-spawning stuck threads for the same condition.
|
|
||
| String renderedCondition = runContext.render(this.exitCondition) | ||
| .as(String.class) | ||
| .orElse(""); |
There was a problem hiding this comment.
🟡 [Guidelines] Required property silently defaults to empty string on render failure
Problem: exitCondition is @NotNull/mandatory, but .orElse("") swallows a render failure (e.g. an undefined variable in a templated condition) instead of surfacing it. The guideline requires mandatory properties to be 'checked during the rendering' — here a bad render just makes matchesCondition permanently return false (cond.isEmpty() short-circuits), so the trigger silently never fires with no error anywhere in the logs.
Fix: Use .orElseThrow(() -> new IllegalArgumentException("exitCondition could not be rendered — check the expression")) since there is no sensible default for a required field.
| ) | ||
| } | ||
| ) | ||
| // TODO: extract shared trigger logic (evaluate, matchesCondition, extractFailure, Output) |
There was a problem hiding this comment.
🟠 [Guidelines] TODO placeholder forbidden
Problem: Same as CommandsTrigger.java — 'Never add section banners, dividers, or TODO placeholders unless explicitly requested.'
Fix: Track the AbstractScriptTrigger extraction as a follow-up issue instead of a source comment.
| // so edge mode may re-fire once after a scheduler restart. | ||
| @Getter(AccessLevel.NONE) | ||
| @Builder.Default | ||
| private final AtomicBoolean lastMatched = new AtomicBoolean(false); |
There was a problem hiding this comment.
🟠 [Guidelines/Correctness] Mutable AtomicBoolean not excluded from Lombok equals/hashCode/toString
Problem: Same issue as CommandsTrigger.java#132 — AtomicBoolean has identity-based equals(), so this field breaks structural equality between otherwise-identical ScriptTrigger instances (e.g. across flow revision comparisons).
Fix: Annotate lastMatched with @EqualsAndHashCode.Exclude and @ToString.Exclude.
| var future = CompletableFuture.supplyAsync( | ||
| () -> pattern.matcher(haystack).find() | ||
| ); | ||
| return future.get(5, TimeUnit.SECONDS); |
There was a problem hiding this comment.
🟠 [Performance] Abandoned future leaks a thread on the shared common pool
Problem: Same issue as CommandsTrigger.java#215 — the future is never cancelled on timeout, and Matcher#find() isn't interruptible, so a stuck regex keeps a ForkJoinPool.commonPool() worker pinned indefinitely.
Fix: Use a dedicated bounded executor for this matching call instead of the shared common pool.
|
|
||
| String renderedCondition = runContext.render(this.exitCondition) | ||
| .as(String.class) | ||
| .orElse(""); |
There was a problem hiding this comment.
🟡 [Guidelines] Required property silently defaults to empty string on render failure
Problem: Same issue as CommandsTrigger.java#172 — a mandatory exitCondition that fails to render is silently coerced to "", making the trigger permanently non-matching with no diagnostic.
Fix: Use .orElseThrow(...) with a message naming exitCondition and the fix.
The R ScriptTrigger/CommandsTrigger in this branch are byte-for-byte identical to PR kestra-io#438, which is dedicated to the R module. Remove them here so kestra-io#446 is scoped to the Perl triggers and kestra-io#438 owns the R work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat/r-script-commands-triggers * feat/perl-script-commands-triggers * chore(r): drop R triggers, keep them in #438 The R ScriptTrigger/CommandsTrigger in this branch are byte-for-byte identical to PR #438, which is dedicated to the R module. Remove them here so #446 is scoped to the Perl triggers and #438 owns the R work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(perl): guard trigger exitCondition regex against ReDoS Run the user-supplied exitCondition regex with a 5s timeout and fall back to substring matching, as the Ruby/Shell/Node/Bun triggers do, so a catastrophic-backtracking pattern can no longer hang the scheduler poll thread. Document the in-memory edge-state limitation and add condition tests for pathological and invalid regexes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(perl): make ReDoS test hit the timeout and fix CommandsTrigger example (a+)+$ is memoized by the JDK 25 regex engine and fails instantly, so the test never reached the 5s guard; use (.*a){20}$ and assert the elapsed time. The CommandsTrigger example ran `perl missing.pl`, which exits 2 and never matched `exit 1`; use `perl -e 'exit 1'` instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(perl): persist trigger edge state in the namespace KV store The in-memory lastMatched flag was rebuilt with the trigger on every poll, so edge mode fired on every matching poll. Keep the previous result in the namespace KV store instead, as the Bun/.NET/PowerShell triggers do. Replaces the tautological AtomicBoolean edge tests with EdgeStateTest and an evaluate-level test that polls through a serialized copy. Refs #449 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: jymaire <jmaire@kestra.io> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: jymaire <jymaire@users.noreply.github.com>
feat(r): add ScriptTrigger and CommandsTrigger to plugin-script-r
What changes are being made and why?
The R module only shipped
ScriptandCommandstasks, so an R script could be run by a flow but could not start one. Teams that watch an external system with R (a CRAN job, a data-quality check, a health probe) had to wrap it in aScheduleplus a conditional task, or poll from outside Kestra.This PR adds two polling triggers to
plugin-script-r, bringing it in line with the Shell and Node modules:io.kestra.plugin.scripts.r.ScriptTrigger— polls by running an inline R script in a container and starts the flow when the result matches a condition.io.kestra.plugin.scripts.r.CommandsTrigger— same, driven by a list of R commands instead of an inline script.Behaviour shared by both triggers:
containerImageProperty<String>r-basescript/commandsProperty<String>/Property<List<String>>exitConditionProperty<String>intervalDurationPT60SedgeProperty<Boolean>truenot matching → matchingtransition.Condition matching (
matchesCondition) supports two forms:exit N— case-insensitive, compares against the process exit code. Anullexit code never matches.vars; if the pattern is invalid or takes longer than 5s to evaluate, it falls back to a plain substring check. The 5s cap exists so a user-supplied pattern cannot hang the scheduler thread through catastrophic backtracking.Edge mode (
edge: true, the default) is anti-spam: a condition that stays true across polls emits once, not on every tick. Setedge: falseto emit on every matching poll.Failure handling: a
RunnableTaskExceptionis unwrapped down the cause chain to find aTaskExceptionand recover its exit code, so a failing script still produces a usableexitCodeoutput instead of propagating. Any other exception during evaluation is logged atWARNand returnsOptional.empty()— a broken poll never blocks the scheduler.Outputs exposed to the flow (
{{ trigger.* }}):timestamp,condition(the renderedexitCondition),exitCode, andvars.Known limitation: the
edgestate (lastMatched) is an in-memoryAtomicBoolean. It resets when the trigger is rehydrated, so edge mode may re-fire once after a scheduler restart. Both classes carry aTODOto extract the duplicatedevaluate/matchesCondition/extractFailure/Outputlogic into a sharedAbstractScriptTriggerinplugin-script— the duplication across Shell, Node, Ruby and R is deliberate for now and should be consolidated in a follow-up rather than inside this PR.Files changed
plugin-script-r/src/main/java/io/kestra/plugin/scripts/r/ScriptTrigger.javaplugin-script-r/src/main/java/io/kestra/plugin/scripts/r/CommandsTrigger.javaplugin-script-r/src/test/java/io/kestra/plugin/scripts/r/ScriptTriggerTest.javaplugin-script-r/src/test/java/io/kestra/plugin/scripts/r/ScriptTriggerConditionTest.javaplugin-script-r/src/test/java/io/kestra/plugin/scripts/r/CommandsTriggerTest.java@KestraTestintegration tests.plugin-script-r/src/test/java/io/kestra/plugin/scripts/r/CommandsTriggerConditionTest.javaplugin-script-r/build.gradleschedulerandworkertest dependencies needed by@KestraTest.AGENTS.mdplugin-script-r.How the changes have been QAed?
CommandsTrigger— fires when a command exits non-zero:ScriptTrigger— inline R script, edge mode on:ScriptTrigger— match on structured outputs rather than exit code:Automated coverage
CommandsTriggerTestis a@KestraTestthat runs the trigger end to end against a realr-basecontainer and asserts on the generated execution:1againstexitCondition: "exit 1"emits, withexitCode == 1,condition == "exit 1"and a non-nulltimestamp;::{"outputs":{"listing":"toto"}}::againstexitCondition: "toto"emits, withexitCode == 0and non-nullvars;0againstexitCondition: "exit 1"does not emit.ScriptTriggerTest,ScriptTriggerConditionTestandCommandsTriggerConditionTestcovermatchesConditionand edge-mode transitions as pure unit tests, driving theOutputmodel directly. They deliberately avoid needing an R runtime, since it is not present on every CI machine — the runtime-backed coverage lives inCommandsTriggerTest.Condition cases asserted:
exit 0/exit 1/exit 42matching and mismatching, uppercaseEXIT 1(case-insensitivity), null exit code, substring match in vars, absent substring, regex match in vars, empty haystack, empty condition, and null condition.Run them with:
Setup Instructions
r-base.r-baseimage is pulled from Docker Hub on first use; no credentials needed.containerImageat an image that already includes them or install them inside the script/commands../gradlew :plugin-script-r:testneeds Docker running forCommandsTriggerTest; the three unit test classes run without it.Contributor Checklist ✅