Skip to content

[Bug]: Preset-overridden skill hashes are not synchronized in integration manifest during integration upgrade #4696

Description

@jiezhengj

Bug Description

When an enabled preset overrides a core command that is rendered as an agent skill (such as for codex in .agents/skills/<name>/SKILL.md), running specify integration upgrade <agent> --force succeeds, but the integration manifest .specify/integrations/<agent>.manifest.json retains the hash of the base/default template.

Because command_upgrade.py invokes new_manifest.save() before calling _register_presets_for_agent(), the preset's re-registration overwrites the skill file on disk with the preset content without updating the manifest.

As a result:

  1. specify integration status immediately reports the skill file as modified with finding manifest-file-modified.
  2. Any subsequent specify integration upgrade <agent> (without --force) is blocked by check_modified().

Steps to Reproduce

  1. Initialize a new project with an agent integration that uses skills (e.g., codex):

    specify init --here --integration codex
    specify integration status
    # Status: OK (0 modified, 0 missing)
  2. Add a local preset that wraps or overrides a core command (e.g. speckit.tasks or speckit.specify):
    Create test-preset/preset.yml:

    schema_version: "1.0"
    preset:
      id: "test-task-preset"
      name: "Test Task Preset"
      version: "1.0.0"
      description: "Test preset overriding speckit.tasks"
    requires:
      speckit_version: ">=0.1.0"
    provides:
      templates:
        - type: "command"
          name: "speckit.tasks"
          file: "commands/speckit.tasks.md"
          strategy: "wrap"

    Install the preset:

    specify preset add --dev ./test-preset
  3. Upgrade the integration with --force:

    specify integration upgrade codex --force

    Output reports: ✓ Integration 'Codex' upgraded successfully.

  4. Check integration status:

    specify integration status

Expected Behavior

specify integration upgrade codex --force should re-baseline or synchronize the hashes of skills written by active presets into .specify/integrations/codex.manifest.json.
After upgrade completes, specify integration status should report OK with 0 modified managed files.

Actual Behavior

specify integration status reports Modified managed files: 1 and status WARNING:

  • Finding: manifest-file-modified (codex): .agents/skills/speckit-tasks/SKILL.md was modified after installation.

Subsequent specify integration upgrade codex without --force fails with:
1 file(s) have been modified since installation... Use --force to overwrite modified files.

Specify CLI Version

1.0.10

AI Agent

Codex CLI

Operating System

macOS (Darwin arm64)

Python Version

Python 3.12

Error Logs

$ specify integration status
Integration status: WARNING
Default integration: codex
Installed integrations: codex
Multi-install safe: yes
Shared templates target alignment: none
Modified managed files: 1
Missing managed files: 0
Invalid manifest paths: 0
Unchecked manifests: 0

Findings:
- warning manifest-file-modified (codex): .agents/skills/speckit-tasks/SKILL.md was modified after installation.
  Suggestion: Run `specify integration upgrade codex` or reinstall the integration.

Additional Context

Root Cause Analysis

In src/specify_cli/integrations/command_upgrade.py:

    # Phase 1: Setup writes default skills and records their hashes
    integration.setup(
        project_root,
        new_manifest,
        ...
    )
    ...
    # Manifest is saved to disk with base skill hashes
    new_manifest.save()
    _write_integration_json(project_root, installed_key, installed_keys, settings)

    ...

    # Phase 2: Presets are re-registered AFTER manifest was saved!
    if key == installed_key:
        _register_extensions_for_agent(...)
        _register_presets_for_agent(
            project_root,
            key,
            continuing="The integration was upgraded, but installed presets may need re-registration.",
        )

_register_presets_for_agent() calls PresetManager.register_enabled_presets_for_agent(agent_key).
This regenerates and writes the preset-overridden content to .agents/skills/<name>/SKILL.md (with frontmatter source: "preset:<preset_id>").
However, new_manifest is never updated with these newly written preset file hashes.

Suggested Fix Shape

After _register_presets_for_agent() completes, if any preset was registered, inspect skills tracked in both the preset registry and new_manifest. When a skill file on disk has matching provenance (e.g. source == f"preset:{preset_id}"), update new_manifest.record_existing(relative) and save new_manifest.

This ensures that legitimate preset overrides are recognized as the clean baseline, while preserving tamper detection for manual edits.

AI Disclosure

AI assistance (Antigravity) supervised by human maintainer was used to trace the execution path in command_upgrade.py and draft this bug report. All code paths and CLI behaviors were verified against a clean reproduction environment.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    triage-nice-to-haveVerdict: evidence-backed fix or greenlit feature — land after review

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions