Skip to content

feat/r-script-commands-triggers - #438

Open
Abhishek84313 wants to merge 2 commits into
kestra-io:mainfrom
Abhishek84313:feat/r-script-commands-triggers
Open

Abhishek84313 wants to merge 2 commits into
kestra-io:mainfrom
Abhishek84313:feat/r-script-commands-triggers

Conversation

@Abhishek84313

Copy link
Copy Markdown
Contributor

feat(r): add ScriptTrigger and CommandsTrigger to plugin-script-r

What changes are being made and why?

The R module only shipped Script and Commands tasks, so an R script could be run by a flow but could not start one. Teams that watch an external system with R (a CRAN job, a data-quality check, a health probe) had to wrap it in a Schedule plus a conditional task, or poll from outside Kestra.

This PR adds two polling triggers to plugin-script-r, bringing it in line with the Shell and Node modules:

  • io.kestra.plugin.scripts.r.ScriptTrigger — polls by running an inline R script in a container and starts the flow when the result matches a condition.
  • io.kestra.plugin.scripts.r.CommandsTrigger — same, driven by a list of R commands instead of an inline script.

Behaviour shared by both triggers:

Property Type Default Purpose
containerImage Property<String> r-base Image used by the underlying task.
script / commands Property<String> / Property<List<String>> — (required) What is executed on each poll.
exitCondition Property<String> — (required) Condition evaluated after each run.
interval Duration PT60S Time between polls.
edge Property<Boolean> true Emit only on a not matching → matching transition.

Condition matching (matchesCondition) supports two forms:

  1. exit N — case-insensitive, compares against the process exit code. A null exit code never matches.
  2. Anything else — compiled as a regex and matched against the task's emitted vars; if the pattern is invalid or takes longer than 5s to evaluate, it falls back to a plain substring check. The 5s cap exists so a user-supplied pattern cannot hang the scheduler thread through catastrophic backtracking.

Edge mode (edge: true, the default) is anti-spam: a condition that stays true across polls emits once, not on every tick. Set edge: false to emit on every matching poll.

Failure handling: a RunnableTaskException is unwrapped down the cause chain to find a TaskException and recover its exit code, so a failing script still produces a usable exitCode output instead of propagating. Any other exception during evaluation is logged at WARN and returns Optional.empty() — a broken poll never blocks the scheduler.

Outputs exposed to the flow ({{ trigger.* }}): timestamp, condition (the rendered exitCondition), exitCode, and vars.

Known limitation: the edge state (lastMatched) is an in-memory AtomicBoolean. It resets when the trigger is rehydrated, so edge mode may re-fire once after a scheduler restart. Both classes carry a TODO to extract the duplicated evaluate / matchesCondition / extractFailure / Output logic into a shared AbstractScriptTrigger in plugin-script — the duplication across Shell, Node, Ruby and R is deliberate for now and should be consolidated in a follow-up rather than inside this PR.

Files changed

File Change
plugin-script-r/src/main/java/io/kestra/plugin/scripts/r/ScriptTrigger.java New — 294 lines.
plugin-script-r/src/main/java/io/kestra/plugin/scripts/r/CommandsTrigger.java New — 290 lines.
plugin-script-r/src/test/java/io/kestra/plugin/scripts/r/ScriptTriggerTest.java New — edge mode + condition matching.
plugin-script-r/src/test/java/io/kestra/plugin/scripts/r/ScriptTriggerConditionTest.java New — parameterized condition matrix.
plugin-script-r/src/test/java/io/kestra/plugin/scripts/r/CommandsTriggerTest.java New — @KestraTest integration tests.
plugin-script-r/src/test/java/io/kestra/plugin/scripts/r/CommandsTriggerConditionTest.java New — parameterized condition matrix.
plugin-script-r/build.gradle Adds scheduler and worker test dependencies needed by @KestraTest.
AGENTS.md Lists the two new classes under plugin-script-r.

How the changes have been QAed?

CommandsTrigger — fires when a command exits non-zero:

id: r_commands_trigger
namespace: company.team

triggers:
  - id: on_fail
    type: io.kestra.plugin.scripts.r.CommandsTrigger
    interval: PT5S
    exitCondition: "exit 1"
    commands:
      - Rscript -e 'stop("boom")'

tasks:
  - id: log
    type: io.kestra.plugin.core.log.Log
    message: "Triggered with exitCode={{ trigger.exitCode }} (condition={{ trigger.condition }})"

ScriptTrigger — inline R script, edge mode on:

id: r_script_trigger
namespace: company.team

triggers:
  - id: script_failure
    type: io.kestra.plugin.scripts.r.ScriptTrigger
    interval: PT10S
    exitCondition: "exit 1"
    edge: true
    script: |
      stop("boom")

tasks:
  - id: log
    type: io.kestra.plugin.core.log.Log
    message: "Triggered with exitCode={{ trigger.exitCode }} (condition={{ trigger.condition }})"

ScriptTrigger — match on structured outputs rather than exit code:

id: r_script_trigger_vars
namespace: company.team

triggers:
  - id: on_ready
    type: io.kestra.plugin.scripts.r.ScriptTrigger
    interval: PT30S
    exitCondition: "status=\\w+"
    edge: true
    script: |
      cat('::{"outputs":{"status":"status=ready"}}::\n')

tasks:
  - id: log
    type: io.kestra.plugin.core.log.Log
    message: "vars={{ trigger.vars }}"

Automated coverage

CommandsTriggerTest is a @KestraTest that runs the trigger end to end against a real r-base container and asserts on the generated execution:

  • a command exiting 1 against exitCondition: "exit 1" emits, with exitCode == 1, condition == "exit 1" and a non-null timestamp;
  • a command emitting ::{"outputs":{"listing":"toto"}}:: against exitCondition: "toto" emits, with exitCode == 0 and non-null vars;
  • a command exiting 0 against exitCondition: "exit 1" does not emit.

ScriptTriggerTest, ScriptTriggerConditionTest and CommandsTriggerConditionTest cover matchesCondition and edge-mode transitions as pure unit tests, driving the Output model directly. They deliberately avoid needing an R runtime, since it is not present on every CI machine — the runtime-backed coverage lives in CommandsTriggerTest.

Condition cases asserted: exit 0/exit 1/exit 42 matching and mismatching, uppercase EXIT 1 (case-insensitivity), null exit code, substring match in vars, absent substring, regex match in vars, empty haystack, empty condition, and null condition.

Run them with:

./gradlew :plugin-script-r:test

Setup Instructions

  • Docker must be available to the Kestra worker — both triggers execute through a container, defaulting to r-base.
  • The r-base image is pulled from Docker Hub on first use; no credentials needed.
  • For scripts that need CRAN packages, either point containerImage at an image that already includes them or install them inside the script/commands.
  • ./gradlew :plugin-script-r:test needs Docker running for CommandsTriggerTest; the three unit test classes run without it.

Contributor Checklist ✅

@github-project-automation github-project-automation Bot moved this to To review in Pull Requests Sep 16, 2026
@MilosPaunovic MilosPaunovic added kind/external Pull requests raised by community contributors area/plugin Plugin-related issue or feature request labels Sep 18, 2026
@MilosPaunovic
MilosPaunovic requested review from a team and fdelbrayelle September 18, 2026 05:38

@fdelbrayelle fdelbrayelle left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kestra Plugin Code Review

Business Requirements — met

No linked issue is referenced in the PR body. The stated goal (bring plugin-script-r trigger support to parity with the Shell/Node modules by adding ScriptTrigger/CommandsTrigger) is self-consistent and fully implemented, with tests and QA examples matching the description.

Kestra Guidelines — 4 findings

  • 🟠 TODO placeholders left in production code (both new files) — forbidden by the Code Comments guideline.
  • 🟠 lastMatched (AtomicBoolean) not excluded from Lombok @ToString/@EqualsAndHashCode (both files) — correctness bug, see inline comment.
  • 🟠 Plugin how-to doc plugin-script-r/src/main/resources/doc/io.kestra.plugin.scripts.r.md not updated to mention the two new triggers — mandatory per the 'Plugin How-To Doc' section whenever a trigger is added to an existing plugin. Node's equivalent doc has a one-line mention of its CommandsTrigger/ScriptTrigger; the R doc still only lists Script/Commands.
  • 🟡 Required exitCondition is rendered with .orElse("") (both files) instead of failing loudly on an unrenderable value — see inline comment.

Security (OWASP Top 10:2025 + KPS) — 0 blocking issues

No secret fields, no new HTTP calls, no unsafe deserialization, no shell string concatenation beyond what the existing Commands/Script tasks already do (out of scope of this diff). The regex-timeout resource-leak noted under Performance has a DoS angle (A10-adjacent) but is tracked there since its primary consequence is thread-pool exhaustion, not an authorization/injection gap.

Performance — 1 finding

  • 🟠 ReDoS guard in matchesCondition abandons the backtracking regex thread on the shared ForkJoinPool.commonPool() instead of cancelling it — see inline comments on both files.

Additional non-blocking notes

  • 🟡 Edge-mode tests (edgeMode_preventsConsecutiveEmit in CommandsTriggerTest, and the three edgeMode_* tests in ScriptTriggerTest) re-implement !lastMatched.getAndSet(x) && x inline rather than calling trigger.evaluate() twice on the same instance. They verify AtomicBoolean arithmetic, not the trigger's actual edge-mode wiring — a regression in evaluate()'s edge logic wouldn't be caught by these tests. Given edge mode is this PR's headline feature, at least one @KestraTest should call evaluate() twice on the same trigger instance and assert the second call returns Optional.empty().
  • 🟡 DRY: CommandsTrigger/ScriptTrigger duplicate ~90% of their logic (evaluate, matchesCondition, extractFailure, Output). This mirrors pre-existing duplication already merged for Shell/Node/Ruby, so it's not net-new debt introduced by this PR, and the TODO (flagged above for violating the no-TODO-comment rule) at least names the follow-up. Non-blocking, but worth tracking as a real issue rather than a comment.
  • 🟢 Nit: trigger/task ids in tests are fixed strings ("commands-trigger", …) rather than randomized, per the Flaky Test Prevention guideline — mirrors existing Node/Shell test precedent, so not introduced by this PR.

Verdict: REQUEST CHANGES

}
)
// TODO: extract shared trigger logic (evaluate, matchesCondition, extractFailure, Output)
// into an AbstractScriptTrigger in plugin-script to reduce duplication across Shell, Node, Ruby, R, etc.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 [Guidelines] TODO placeholder forbidden
Problem: The Code Comments guideline states: "Never add section banners, dividers, or TODO placeholders unless explicitly requested." This TODO documents real follow-up debt but violates that rule as written.
Fix: Move this note into the PR description (already done there) or a tracked follow-up issue instead of a source comment.

// so edge mode may re-fire once after a scheduler restart.
@Builder.Default
@Getter(AccessLevel.NONE)
private final AtomicBoolean lastMatched = new AtomicBoolean(false);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 [Guidelines/Correctness] Mutable AtomicBoolean not excluded from Lombok equals/hashCode/toString
Problem: java.util.concurrent.atomic.AtomicBoolean does not override equals()/hashCode() (identity-based). Because the class-level @EqualsAndHashCode includes this field by default, two structurally identical CommandsTrigger instances (e.g. the same trigger deserialized twice from the same flow YAML, as happens on every flow parse/revision check) will always compare unequal, since each gets its own AtomicBoolean(false) instance. This is exactly the case the guidelines call out under Task Lifecycle Hooks: 'any non-config field carrying runtime state ... must be annotated @EqualsAndHashCode.Exclude and @ToString.Exclude.'
Fix: Annotate lastMatched with @EqualsAndHashCode.Exclude and @ToString.Exclude.

var future = CompletableFuture.supplyAsync(
() -> pattern.matcher(haystack).find()
);
return future.get(5, TimeUnit.SECONDS);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 [Performance] Abandoned future leaks a thread on the shared common pool
Problem: On TimeoutException the CompletableFuture is never cancelled. java.util.regex.Matcher#find() is not interruptible, so the ForkJoinPool.commonPool() worker running the catastrophic-backtracking regex keeps burning CPU indefinitely in the background — it isn't freed when this method returns. Repeated pathological exitCondition values across polls (interval can be as low as a few seconds) progressively pin common-pool worker threads, starving every other CompletableFuture/parallel-stream consumer in the same worker JVM, including unrelated plugins.
Fix: Run the match on a dedicated, bounded single-thread daemon executor created once per class (not the shared common pool) so a stuck matcher only ever wastes one dedicated thread instead of contending with the JVM-wide common pool; consider also caching/rejecting patterns that have already timed out once to avoid repeatedly re-spawning stuck threads for the same condition.


String renderedCondition = runContext.render(this.exitCondition)
.as(String.class)
.orElse("");

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 [Guidelines] Required property silently defaults to empty string on render failure
Problem: exitCondition is @NotNull/mandatory, but .orElse("") swallows a render failure (e.g. an undefined variable in a templated condition) instead of surfacing it. The guideline requires mandatory properties to be 'checked during the rendering' — here a bad render just makes matchesCondition permanently return false (cond.isEmpty() short-circuits), so the trigger silently never fires with no error anywhere in the logs.
Fix: Use .orElseThrow(() -> new IllegalArgumentException("exitCondition could not be rendered — check the expression")) since there is no sensible default for a required field.

)
}
)
// TODO: extract shared trigger logic (evaluate, matchesCondition, extractFailure, Output)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 [Guidelines] TODO placeholder forbidden
Problem: Same as CommandsTrigger.java — 'Never add section banners, dividers, or TODO placeholders unless explicitly requested.'
Fix: Track the AbstractScriptTrigger extraction as a follow-up issue instead of a source comment.

// so edge mode may re-fire once after a scheduler restart.
@Getter(AccessLevel.NONE)
@Builder.Default
private final AtomicBoolean lastMatched = new AtomicBoolean(false);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 [Guidelines/Correctness] Mutable AtomicBoolean not excluded from Lombok equals/hashCode/toString
Problem: Same issue as CommandsTrigger.java#132 — AtomicBoolean has identity-based equals(), so this field breaks structural equality between otherwise-identical ScriptTrigger instances (e.g. across flow revision comparisons).
Fix: Annotate lastMatched with @EqualsAndHashCode.Exclude and @ToString.Exclude.

var future = CompletableFuture.supplyAsync(
() -> pattern.matcher(haystack).find()
);
return future.get(5, TimeUnit.SECONDS);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 [Performance] Abandoned future leaks a thread on the shared common pool
Problem: Same issue as CommandsTrigger.java#215 — the future is never cancelled on timeout, and Matcher#find() isn't interruptible, so a stuck regex keeps a ForkJoinPool.commonPool() worker pinned indefinitely.
Fix: Use a dedicated bounded executor for this matching call instead of the shared common pool.


String renderedCondition = runContext.render(this.exitCondition)
.as(String.class)
.orElse("");

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 [Guidelines] Required property silently defaults to empty string on render failure
Problem: Same issue as CommandsTrigger.java#172 — a mandatory exitCondition that fails to render is silently coerced to "", making the trigger permanently non-matching with no diagnostic.
Fix: Use .orElseThrow(...) with a message naming exitCondition and the fix.

jymaire added a commit to Abhishek84313/plugin-scripts that referenced this pull request Sep 22, 2026
The R ScriptTrigger/CommandsTrigger in this branch are byte-for-byte
identical to PR kestra-io#438, which is dedicated to the R module. Remove them
here so kestra-io#446 is scoped to the Perl triggers and kestra-io#438 owns the R work.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
jymaire added a commit that referenced this pull request Sep 23, 2026
* feat/r-script-commands-triggers

* feat/perl-script-commands-triggers

* chore(r): drop R triggers, keep them in #438

The R ScriptTrigger/CommandsTrigger in this branch are byte-for-byte
identical to PR #438, which is dedicated to the R module. Remove them
here so #446 is scoped to the Perl triggers and #438 owns the R work.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(perl): guard trigger exitCondition regex against ReDoS

Run the user-supplied exitCondition regex with a 5s timeout and fall
back to substring matching, as the Ruby/Shell/Node/Bun triggers do, so a
catastrophic-backtracking pattern can no longer hang the scheduler poll
thread. Document the in-memory edge-state limitation and add condition
tests for pathological and invalid regexes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(perl): make ReDoS test hit the timeout and fix CommandsTrigger example

(a+)+$ is memoized by the JDK 25 regex engine and fails instantly, so the
test never reached the 5s guard; use (.*a){20}$ and assert the elapsed
time. The CommandsTrigger example ran `perl missing.pl`, which exits 2 and
never matched `exit 1`; use `perl -e 'exit 1'` instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(perl): persist trigger edge state in the namespace KV store

The in-memory lastMatched flag was rebuilt with the trigger on every poll,
so edge mode fired on every matching poll. Keep the previous result in the
namespace KV store instead, as the Bun/.NET/PowerShell triggers do.

Replaces the tautological AtomicBoolean edge tests with EdgeStateTest and
an evaluate-level test that polls through a serialized copy.

Refs #449

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: jymaire <jmaire@kestra.io>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: jymaire <jymaire@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/plugin Plugin-related issue or feature request kind/external Pull requests raised by community contributors

Projects

Status: To review

Development

Successfully merging this pull request may close these issues.

3 participants