feat: manual context compression - #9795
C10H14N2O5 wants to merge 18 commits into
Conversation
| f"Compress completed." | ||
| f" {prev_tokens} -> {tokens_after_summary} tokens," | ||
| f" compression rate: {compress_rate:.2f}%.", |
| ) | ||
| else: | ||
| logger.info( | ||
| f"Compress completed. {prev_tokens} -> {tokens_after_summary} tokens." |
There was a problem hiding this comment.
Hey - I've found 1 issue
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="astrbot/builtin_stars/builtin_commands/commands/conversation.py" line_range="311" />
<code_context>
+ "❌ Context compression requires admin permission in a shared "
+ "group conversation."
+ )
+ return
+
+ if not provider_settings.get("enable", True):
</code_context>
<issue_to_address>
**issue (bug_risk):** When the compact event is stopped before or after the provider call, the command returns immediately after sending the transient progress message and never sets a terminal result. WebChat therefore retains or clears the progress state without receiving the documented cancellation message, leaving the user without an explicit outcome.
**Triggers:** When `/stop` marks the compact event itself as stopped.
**Suggested fix:** Call `reply(cancelled)` before returning from both `message.is_stopped()` branches, unless the surrounding transport explicitly guarantees a terminal response for stopped events.
</issue_to_address>Sourcery assessment
Needs a human reviewer. 1 finding to address first, and /compact replaces persisted conversation history with an LLM-generated summary, permanently discarding older exact messages; reverting the code would not restore histories already compressed. The impact is limited to opted-in conversations and requires an explicit command, but an incorrect summary or authorization check could still cause unrecoverable context loss.
Blocking findings: astrbot/builtin_stars/builtin_commands/commands/conversation.py:311
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
Hi @Soulter, when you have time, could you please take a quick look at whether the feature direction of this PR fits AstrBot’s current roadmap? I’m mainly looking for direction-level feedback before further iteration; no rush on the detailed review. Thanks! |
|
I'll resolve the existing merge conflicts and adapt this PR to the latest Agent Runner configuration changes in #9821, then mark it ready for review again. |
78915fc to
77cdaa1
Compare
There was a problem hiding this comment.
Hey - I've reviewed your changes and they look great!
Sourcery assessment
Needs a human reviewer. The command permanently overwrites a conversation's persisted history with an LLM summary, so omitted details cannot be recovered by reverting the code and may require manual reconstruction. The impact is bounded to conversations where an administrator or user explicitly enables and invokes the feature, rather than affecting all conversations by default.
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
I'll resolve the current merge conflicts and update this PR for the latest v4.28.1 changes, then mark it ready for review again. |
6e2bf2b to
11c668b
Compare
There was a problem hiding this comment.
Hey - I've found 2 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="astrbot/core/agent/context/token_counter.py" line_range="15" />
<code_context>
)
async def process(
- self, messages: list[Message], trusted_token_usage: int = 0
+ self,
+ messages: list[Message],
</code_context>
<issue_to_address>
**issue (bug_risk):** Renaming the public keyword from `trusted_token_usage` to `reported_token_usage` causes existing custom `TokenCounter` implementations and callers that pass `trusted_token_usage=` to raise `TypeError`, despite the PR claiming to preserve the trusted-token-usage interface.
**Triggers:** When a plugin or downstream caller invokes `TokenCounter.count_tokens(..., trusted_token_usage=...)` by keyword.
**Suggested fix:** Accept both keyword names during the compatibility window, or retain `trusted_token_usage` in the public protocol and use `reported_token_usage` only for the internal call path.
</issue_to_address>
### Comment 2
<location path="astrbot/core/agent/context/manager.py" line_range="127-139" />
<code_context>
- # calculate compress rate
- compress_rate = (tokens_after_summary / self.config.max_context_tokens) * 100
- logger.info(
- f"Compress completed."
- f" {prev_tokens} -> {tokens_after_summary} tokens,"
- f" compression rate: {compress_rate:.2f}%.",
- )
+ if self.config.max_context_tokens > 0:
+ compress_rate = (
</code_context>
<issue_to_address>
**🚨 nitpick (security):** Automatic compression still logs the exact estimated token transition (`prev_tokens -> tokens_after_summary`), contradicting the stated privacy change to avoid logging context token counts and exposing conversation-size metadata in application logs.
**Triggers:** When automatic or forced compression completes successfully and INFO logging is enabled.
**Suggested fix:** Log only a fixed completion message, or move token metrics to a deliberately privacy-reviewed telemetry channel rather than the general application log.
```suggestion
logger.info("Compress completed.")
```
</issue_to_address>Sourcery assessment
Needs a human reviewer. 1 finding to address first, and when /compact is invoked, the conversation record is replaced with an LLM-generated summary and older messages are discarded, so a wrong or incomplete summary can permanently lose conversation details. Reverting the code stops future compression but does not restore histories already rewritten; the impact is bounded to conversations where the feature is enabled and invoked.
Blocking findings: astrbot/core/agent/context/token_counter.py:15
|
I investigated the two CodeQL clear-text logging alerts. They appear to be false positives involving |
|
After syncing with upstream, CI picked up SQLModel 0.0.46 and hit the breaking datetime changes introduced in 0.0.45, causing 11 existing tests to fail. This also reproduces on upstream master and is tracked in #10205. As a temporary compatibility measure, both dependency manifests now constrain SQLModel to >=0.0.24,<0.0.45; the 11 affected tests pass locally with 0.0.44. |
TL;DR
/compactcommand for the local Agent Runner so users can explicitly request LLM-based context compression before the automatic threshold is reached.local + llm_compressconfigurations.11c668b2.Background
Fixes #9281
Related to #8348 and #9252, but intentionally does not implement configurable automatic compression thresholds.
AstrBot already supports automatic context management through maximum-turn enforcement and a token-threshold safeguard. When a request approaches the model context-window limit,
ContextManagercan summarize the history with an LLM or truncate it by conversation turns. However, token-triggered compression does not run until its threshold is reached. With models offering 512K, 1M, or larger context windows, users may encounter practical degradation far below the current 82% threshold:This PR adds the manual
/compactworkflow available in other agent harnesses. It lets users request compression at an appropriate point without changing AstrBot's automatic threshold or existing automatic behavior.The first version uses a command instead of a dedicated ChatUI button. AstrBot's command system already provides registration, autocomplete, enable/disable controls, renaming, and permission management. Reusing it keeps the patch reviewable while supporting WebChat and other messaging platforms without a new API or UI component.
Because an LLM-generated summary may omit role state, narrative facts, or task details, this capability is marked experimental and remains disabled by default. Users must explicitly opt in before the command can run.
Modifications / 改动点
1.
/compactuser commandRegister
/compactin the built-in command plugin and reuse the existing command-management behavior:llm_compresscontext strategy.The live user-visible flow remains intentionally concise:
The generated summary is never printed into the chat. WebChat additionally receives an
agent_statsevent so the context ring immediately reflects the estimated size of the compressed history.WebChat marks the progress status as ephemeral. The live client replaces it with the terminal result, while persisted history after a successful run contains only:
2. Opt-in experimental setting
Add the following configuration value:
{ "agent_runner": { "runner_type": "local", "config": { "compression": { "enable_manual_context_compression": false } } } }Configuration behavior:
false, so upgrading does not silently enable the feature for existing users.agent_runner.runner_type=localandagent_runner.config.compression.overflow_strategy=llm_compress.false, matching the embedded-runner semantics introduced by refactor: embed agent runner configuration in profiles #9821.provider_settings.enable_manual_context_compressionvalues migrate into the embedded local compression config through the existing migration flow. Upstream config version 4, its one-time step-limit upgrade, and future config versions are preserved.3. Reuse the existing compression flow
Add an optional
force_compressargument toContextManager.process(). Its default remainsfalse, preserving all existing automatic callers.With
force_compress=True, the manual command:_run_compression(), the existingTokenCounter, and the LLM summary compressor; andAutomatic compression retains its existing half-truncation protection. Only the manual path disables that fallback so a failed
/compactoperation cannot destructively truncate the original history.4. Preserve the latest complete turn
Add an optional
preserve_latest_roundbehavior to the LLM summary compressor. It defaults to disabled and therefore does not alter automatic compression output.When enabled by manual compression:
agent_runner.config.compression.keep_recent_ratio, without splitting logical turns.This reduces the risk of changing the user's most recent requirements, tool state, or editable response.
5. Checkpoint-aware history persistence
The manual command reads the persisted database history and uses the existing checkpoint utilities during conversion:
bind_checkpoint_messages()associates persisted checkpoints with their messages.dump_messages_with_checkpoints()serializes the compressed result.6. Token-benefit validation and non-destructive failure
Estimate tokens both before and after compression:
On success, the compressed history and
token_usage=0are saved in the same database update. The next normal model response refreshes token state using the provider's actual usage information.7. Session locking, concurrency revalidation, and stop handling
/compactacquires the same unified-message-origin (UMO) session lock used by normal local Agent requests:/new, or modifies the same history through the Dashboard/API, the result is discarded./stoprequest prevents the compressed result from being written and explicitly reports that the original history was preserved.Dashboard and API history updates do not acquire this session lock. The additional conversation-ID and history comparisons therefore protect against external changes that occur while the LLM summary is being generated.
8. Safe verification after database exceptions
If
update_conversation()raises an exception, the command does not immediately assume that persistence failed. It safely reloads the database state once:This covers cases where a database update commits successfully but the client still receives an exception.
9. WebChat statistics, transient progress, and log privacy
webchat_ephemeralchain type and remains visible to connected clients.agent_statsevent is sent on a best-effort basis after persistence and after leaving the session lock.10. Reuse provider resolution
Refactor the existing context-compression provider resolver so both normal Agent construction and
/compactcan call it without changing its selection semantics:Existing automatic compression provider selection remains unchanged.
Scope and Compatibility
This PR intentionally keeps the following boundaries:
The implementation is based on upstream
masteratb06550567b25d7ef4404bf526e448c869cc5d4ecand retains the embedded Agent Runner configuration introduced by refactor: embed agent runner configuration in profiles #9821.Upstream attachment-upload limits, resumable chunked uploads, Local computer permission validation, and config-version 4 migration behavior are preserved.
Only AstrBot's built-in local Agent Runner is supported.
Dify, Coze, Alibaba Cloud Bailian, DeerFlow, and other remote runners own their context remotely; this PR does not attempt to rewrite remote history.
No dedicated Compress now ChatUI button is added.
No backend API or OpenAPI schema is added.
The automatic 82% compression threshold is unchanged.
The configurable automatic token thresholds proposed in [Feature]Token-Threshold Context Compression #8348 and [Feature] 上下文压缩阈值可配置化 #9252 are out of scope.
The generated summary is not exposed in chat.
No dependency or lockfile is changed.
No database schema is changed.
Existing legacy manual-compression values are migrated into the embedded local runner config; missing values remain disabled.
Default behavior for all existing automatic compression callers remains unchanged.
This is NOT a breaking change. / 这不是一个破坏性变更。
Changed Files and Size
This PR changes 26 tracked files relative to
b06550567b25d7ef4404bf526e448c869cc5d4ec:astrbot/builtin_stars/builtin_commands/commands/conversation.py/compactgates, locking, compression, persistence, and feedbackastrbot/builtin_stars/builtin_commands/main.py/compactastrbot/core/agent/context/compressor.pyastrbot/core/agent/context/config.pyastrbot/core/agent/context/manager.pyastrbot/core/astr_main_agent.pyastrbot/core/config/agent_runner.pyastrbot/core/utils/migra_helper.pyastrbot/core/config/default.pyconfig-metadata.jsonfilesPatch size:
Most added lines are regression tests. The implementation adds no remote-runner compatibility layer, UI component, or dependency.
Screenshots and Test Results
Current Tested State
3.12.139.1.10.15.22b06550567b25d7ef4404bf526e448c869cc5d4eced8101b492a9b83fa1101c04d5e5533099ed92d211c668b2287c0267437950ef52ec5d301f8df103Feature Regression passed on
ed8101b49. The Focused Conflict and Upload Regression and Embedded Agent Runner Configuration Regression results below were recorded on11c668b2. Test runs used isolatedASTRBOT_ROOTdirectories under the system temporary directory and disabled pytest's cache provider. The suites overlap; their counts are not additive.Focused Conflict and Upload Regression
Result: 200 passed, 3 warnings in 18.82s.
Coverage includes transient progress and durable terminal messages over SSE and WebSocket, profile runtime/permission validation, config metadata, legacy migration, manual command behavior, resumable uploads, upload limits, filenames, and image formats.
The warnings concern deprecated
audioop, the deprecated DashScope Assistants API, and an unawaited cleanup coroutine in the unchanged upstream chunked-upload code.Feature Regression
Result on
ed8101b49: 364 passed, 1 warning in 17.31s.This includes automatic compression behavior, forced compression without half truncation, latest-round and checkpoint preservation, command gates, provider failures, token-benefit checks, stop handling, session locks, concurrent history changes, database verification states, log privacy, transient progress, durable terminal results, context statistics, and streaming-disconnect persistence.
Compression metrics use independent estimates from the actual messages; the forced manual path uses local estimates throughout.
Embedded Agent Runner Configuration Regression
Result: 228 passed, 2 warnings in 16.34s.
Coverage includes defaults, normalization, save/load and profile round trips, runner switching, and metadata keys for all four locales. Existing migration tests now also verify:
Formatting and Static Checks
The inaccessible local
.pytest_cachedirectory was excluded from Ruff's traversal. Ruff lint also passed with every Python file in the PR explicitly supplied, including the tests. All four locale JSON files parsed successfully and contained the manual-compression description and hint.Repository-wide Ruff format and lint checks, explicit lint of the affected Python files, and
git diff --checkpassed.GitHub CI
See the GitHub checks attached to the latest PR commit for current CI results.
Reviewer Verification Steps
llm_compress, and enable Manual Context Compression (Experimental)./compactbelow the automatic 82% threshold.Dashboard Build and Manual E2E Verification
11c668b2287c0267437950ef52ec5d301f8df103: 5653 modules transformed; exit code 0. Dependencies were installed withpnpm@10.28.2 install --frozen-lockfile;npm run buildran the font-subset script,vue-tsc --noEmit, andvite build.11c668b2287c0267437950ef52ec5d301f8df103using the freshly built Dashboard, and the PR author confirmed that everything worked normally.ja-JP) alongside Chinese, English, and Russian.Earlier Full Repository and CI Results
The following records are from the previous PR head,
6e2bf2b559f75afa665e5d71346b2062c84bca85, based onfe3d77568b88ea3be83b2190d515da0a039da399. They are separate from the current-head regression, build, and manual E2E results above.Manual Behavior and Screenshots
WebChat compression and Context Ring change
QQ Official Bot
Failure preserves the original history
The compression prompt was temporarily changed to request expansion, verifying that output without a token reduction is never persisted:
The following failure paths were also verified:
None of these tested paths reduced the context ring, half-truncated history, or damaged subsequent recall.
Original history and Context Ring remain unchanged after failure
Session isolation and concurrency
/compacttwice in the same session did not restore stale history or drop the latest turn./compactwaited for it and then compressed history containing the completed response./newor modifying history through another entry point during compression prevented persistence:Stop behavior
A WebChat stop request cannot immediately cancel every in-flight provider request, but it prevents the result from being persisted. After the provider returned, the command reported:
History and the context ring remained unchanged. This protects persisted state, although a slow provider request may still run to completion and incur cost, as noted under Known Limitations.
Configuration and i18n
agent_runner.config.compression.enable_manual_context_compression.手动上下文压缩(实验性),Manual Context Compression (Experimental),Ручное сжатие контекста (экспериментальная функция), and手動コンテキスト圧縮(実験的).Observed configuration:
Simplified Chinese (
zh-CN)English (
en-US)Russian (
ru-RU)Japanese (
ja-JP)Known Limitations
ConversationManager.update_conversation(), the final update is not a database-level compare-and-swap. This path mitigates concurrent changes by re-reading and comparing the active conversation and persisted history immediately before the update.SQLModel compatibility note
After syncing with upstream, CI picked up SQLModel 0.0.46 and hit the breaking datetime changes introduced in 0.0.45, causing 11 existing tests to fail. This also reproduces on upstream master and is tracked in #10205. As a temporary compatibility measure, both dependency manifests now constrain SQLModel to
>=0.0.24,<0.0.45; the 11 affected tests pass locally with 0.0.44.Checklist / 检查清单
😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
/ 如果 PR 中有新加入的功能,已经通过 Issue / 邮件等方式和作者讨论过。
👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
/ 我的更改经过了良好的测试,并已在上方提供了“验证步骤”和“运行截图”。
🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in
requirements.txtandpyproject.toml./ 我确保没有引入新依赖库,或者引入了新依赖库的同时将其添加到
requirements.txt和pyproject.toml文件相应位置。😮 My changes do not introduce malicious code.
/ 我的更改没有引入恶意代码。
Summary by Sourcery
Enable users to manually request safe LLM-based context compression for local Agent Runner conversations.
New Features:
/compactcommand for manually compressing local Agent Runner conversation context with LLM summaries.Bug Fixes:
Enhancements:
Build:
Tests: