Skip to content

Fix stale, hanging, and unrun ggsql-jupyter protocol tests - #557

Open
samclark2015 wants to merge 2 commits into
mainfrom
sclark/posit-dev-positron-15512
Open

samclark2015 wants to merge 2 commits into
mainfrom
sclark/posit-dev-positron-15512

Conversation

@samclark2015

@samclark2015 samclark2015 commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes posit-dev/positron#15512: the Python protocol tests under ggsql-jupyter/tests/ had been stale and unrun (not wired into CI) since ~January 2026. This PR fixes the suite and wires it into CI.

  • test_compliance.py: fixed a stale implementation string mismatch, a pager assertion that could never pass, and a kernelspec install that could clobber a developer's real ggsql kernelspec (now isolated to a scratch JUPYTER_DATA_DIR under a ggsql-test name). Also fixed two independent hangs — test_shutdown was killing the class-shared kernel before test_status_messages ran (alphabetical test ordering), and test_kernel_info waited forever on a reply it explicitly filters out via get_non_kernel_info_reply — plus a third bug where test_shutdown read its reply from the wrong ZMQ channel (control, not shell).
  • The class inherited from jupyter_kernel_test.KernelTests without defining most of the sample attributes that drive its built-in tests, so they silently skipped. Checked kernel.rs's message dispatch to see what's real: complete_request, inspect_request, history_request and pager support don't exist, and display_data is never emitted, so those tests are now explicitly overridden with a comment explaining why. code_generate_error, code_execute_result, complete_code_samples and incomplete_code_samples are backed by real behavior, so they're now defined and those tests actually run.
  • test_integration.py: replaced a blind sleep(3) with a real kernel-readiness check. kernel_manager/console_kernel_manager are now session-scoped rather than relaunching a kernel per test; TestShutdown gets its own private kernel (via a new _launch_kernel context manager) instead of killing the shared one, and its shutdown reply is now read from the control channel, where it actually arrives, rather than shell.
  • Both files built the kernel independently; that's now conftest.py's build_kernel_binary(), shared by both, so it happens once rather than being duplicated.
  • requirements.txt: dropped jupyterlab/ipykernel, unused by either test file.
  • Added a jupyter-protocol-tests job to .github/workflows/build.yaml, which builds the kernel once before running pytest and sets GGSQL_SKIP_GENERATE=1 for that step since the grammar was already generated by the build step just before it. Runs with continue-on-error: true for now — see below.
  • Updated ggsql-jupyter/tests/README.md and ggsql-jupyter/CLAUDE.md to match.

A separate, pre-existing bug found along the way

While verifying these fixes, kernel startup intermittently panicked (crashed the whole kernel process) from a bug in the vendored zeromq crate's RepSocket, used for ggsql-jupyter's heartbeat socket — unrelated to anything in this PR. Filed as posit-dev/ggsql#556. It hits a meaningful fraction of local runs, so the new CI job has continue-on-error: true until #556 is fixed, to avoid a flaky-red job on every PR.

Test plan

  • test_integration.py: all tests passing locally, including the now session-scoped kernel fixtures and the isolated TestShutdown kernel
  • test_compliance.py: passing locally — including the newly-activated test_error, test_execute_result, and test_is_complete — when kernel startup doesn't hit ggsql-jupyter's heartbeat socket can panic the whole kernel (zeromq RepSocket, not yet implemented) #556's panic; confirmed the real ggsql kernelspec is never touched across multiple runs
  • Confirmed via ggsql-jupyter/src/kernel.rs that shutdown_reply is sent on the control channel, matching the test_shutdown fixes in both files
  • Full combined suite (test_integration.py + test_compliance.py) run clean end-to-end: 32 passed, 1 skipped, 4 subtests passed, 0 failures
  • New jupyter-protocol-tests CI job goes green (or at least runs, given continue-on-error) on this PR

🤖 Generated with Claude Code

samclark2015 and others added 2 commits September 21, 2026 15:59
test_compliance.py and test_integration.py under ggsql-jupyter/tests/ had
gone unrun and untested since ~January 2026 (posit-dev/positron#15512):
a stale implementation string, a pager assertion that could never pass,
a kernelspec install that could clobber a developer's real "ggsql" kernel,
and two independent hangs (test_shutdown killing the class-shared kernel
before test_status_messages runs alphabetically, and test_kernel_info
waiting forever on a reply it explicitly filters out) meant the suite
never ran to completion. test_shutdown also read its reply from the wrong
ZMQ channel. test_integration.py's kernel_manager fixture used a blind
sleep(3) instead of a real readiness check.

Fixes all of the above, isolates the kernelspec install to a scratch
JUPYTER_DATA_DIR, drops unused test dependencies, and wires both files
into a new CI job. That job runs with continue-on-error for now: kernel
startup intermittently panics on a heartbeat-socket bug in the vendored
zeromq crate (#556), unrelated to this suite, which will
make the job flaky until that's fixed separately.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… build the kernel once

test_integration.py launched a fresh kernel per test; kernel_manager and
console_kernel_manager are now session-scoped, and TestShutdown gets its own
private kernel (via a new _launch_kernel context manager) instead of killing
the shared one, since that's the one thing every other test in the file
still needs. Its shutdown reply is also now read from the control channel,
where it actually arrives, rather than shell.

test_compliance.py inherited from jupyter_kernel_test.KernelTests without
defining any of the sample attributes that drive its built-in tests, so
most of them silently skipped. Checked kernel.rs's message dispatch to see
what's real: complete_request, inspect_request, history_request and pager
support don't exist, and display_data is never emitted, so those tests are
now explicitly overridden with a comment explaining why. code_generate_error,
code_execute_result, complete_code_samples and incomplete_code_samples are
backed by real behavior, so they're now defined and those tests actually run.

Both files built the kernel independently; that's now conftest.py's
build_kernel_binary(), shared by both. build.yaml's protocol-test step sets
GGSQL_SKIP_GENERATE=1 since the grammar was already generated by the build
step just before it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@samclark2015
samclark2015 marked this pull request as ready for review September 23, 2026 20:23

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python protocol tests for ggsql-jupyter are stale, mostly skipped, and not run in CI

1 participant