Skip to content

Make ELF loading respect program header virtual addresses for non-PIE binaries - #1530

Open
cshung wants to merge 8 commits into
hyperlight-dev:mainfrom
cshung:cshung/non-pie-elf-loading
Open

cshung wants to merge 8 commits into
hyperlight-dev:mainfrom
cshung:cshung/non-pie-elf-loading

Conversation

@cshung

@cshung cshung commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Summary

For non-PIE (ET_EXEC) ELF binaries, the guest page table now maps the code region at the ELF's declared virtual address rather than identity-mapping it at the GPA. This allows statically-linked binaries with a fixed load address (e.g., --image-base=0x200000) to execute correctly.

Problem

Previously, Hyperlight assumed code GVA == code GPA (identity mapping). Non-PIE binaries that declare a non-zero base virtual address (via program header p_vaddr) would triple-fault because the guest CPU jumped to the ELF's declared entrypoint VA, which wasn't mapped in the page tables.

Solution

  • Compute code_virt_base from the ELF's lowest LOAD segment p_vaddr
  • For non-PIE (base_va > 0): map code at the declared VA in the guest page tables
  • For PIE (base_va == 0): preserve existing identity mapping behavior (with assertion to guard the invariant)
  • Compute entrypoint as code_virt_base + (entrypoint_va - base_va)

The fix leverages the existing Mapping struct's support for phys_base != virt_base — no changes to the page table code itself.

Testing

  • New non_pie_guest_hello_world integration test exercises full guest lifecycle (init, COW, function call, return value) with a non-PIE simpleguest built at --image-base=0x200000
  • All existing PIE guest tests continue to pass (identity mapping preserved)
  • Tested on Windows/WHP and Linux/KVM

Build infrastructure

  • Added build-rust-guests-non-pie Justfile targets
  • Non-PIE build runs first in guests recipe to avoid clobbering normal guest binaries
  • Added simple_guest_non_pie_as_string() test helper

Contributes to: #1408

Copilot AI review requested due to automatic review settings June 14, 2026 22:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR adds support for running and testing non-PIE Rust guest binaries, including updating snapshot virtual-address mapping so non-PIE guests execute at their declared ELF virtual addresses.

Changes:

  • Add a helper in hyperlight_testing to locate the non-PIE simpleguest binary.
  • Update snapshot mapping/entrypoint calculation to support non-identity VA mappings for non-PIE code regions.
  • Add a new integration test and build automation (Justfile) to produce and run a non-PIE guest.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
src/hyperlight_testing/src/lib.rs Adds path helper(s) for locating non-PIE Rust guest binaries.
src/hyperlight_host/tests/integration_test.rs Adds an integration test that boots and calls into a non-PIE guest.
src/hyperlight_host/src/sandbox/snapshot/mod.rs Adjusts snapshot mappings and entrypoint VA calculation to support non-PIE guests.
Justfile Adds tasks to build and stage non-PIE Rust guest artifacts.

Comment thread Justfile Outdated
Comment thread src/hyperlight_host/src/sandbox/snapshot/mod.rs Outdated
Comment thread src/hyperlight_host/src/sandbox/snapshot/mod.rs Outdated
Comment thread src/hyperlight_testing/src/lib.rs
@cshung
cshung force-pushed the cshung/non-pie-elf-loading branch from 7468e06 to 40c43be Compare June 14, 2026 23:10
@cshung

cshung commented Jun 14, 2026

Copy link
Copy Markdown
Contributor Author

Addressed Copilot review feedback:

  1. Justfile (RUSTFLAGS on Linux) — Fixed. \RUSTFLAGS\ now prefixes \cargo\ directly after \cd &&\ so the env var applies to the build command.

  2. snapshot/mod.rs (virt_base mapping) — No change needed. The code region is a single contiguous block where
    gn.guest_region.start == load_addr\ — there is no offset to lose. The \�ssert_eq!\ is deliberate per codebase convention.

  3. snapshot/mod.rs (entrypoint underflow) — Fixed. Now uses \checked_sub\ with a proper error for malformed ELFs.

  4. lib.rs (doc says 'elf binary') — No change needed. Hyperlight guests are always ELF regardless of host OS.

@ludfjig ludfjig added the kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. label Jun 15, 2026
@cshung
cshung force-pushed the cshung/non-pie-elf-loading branch 2 times, most recently from 3fa110a to 35fef7b Compare June 15, 2026 19:43

@ludfjig ludfjig left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. @syntactically could you have a look too

@syntactically syntactically left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this! It looks like it is moving in a good direction. I have a couple of minor suggestions/comments, as well as a bit of an alternative design that you could take (but totally up to you on that one!)

Comment thread Justfile Outdated
Comment thread src/hyperlight_host/src/sandbox/snapshot/mod.rs Outdated
Comment thread src/hyperlight_host/src/sandbox/snapshot/mod.rs Outdated
Comment thread src/hyperlight_host/src/sandbox/snapshot/mod.rs Outdated
@cshung cshung added the ready-for-review PR is ready for (re-)review label Jun 23, 2026
Comment thread src/hyperlight_host/tests/integration_test.rs Outdated

@ludfjig ludfjig left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but will once again defer to @syntactically for final review

syntactically
syntactically previously approved these changes Jul 15, 2026

@syntactically syntactically left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me, thank you and sorry for the delay reviewing!

I do think that there is one minor move that would make the code a little bit cleaner / better fitting with existing conventions, which I've commented on inline. Feel free to push back with a reason why this code belongs here, though!

Comment thread src/hyperlight_host/src/sandbox/snapshot/mod.rs Outdated
@cshung
cshung force-pushed the cshung/non-pie-elf-loading branch 2 times, most recently from 52a8d4f to b53a7c5 Compare July 18, 2026 13:54
@cshung

cshung commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

All review comments have been addressed. The suggestion to move conflict-check logic into \layout.rs\ (from the Jul 15 review) is implemented in the follow-up PR #1655 via the \code_virt_base()\ method which handles both VA selection and overlap validation.

@syntactically — ready for re-review when you get a chance!

syntactically
syntactically previously approved these changes Jul 20, 2026

@syntactically syntactically left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you've already done that, I think it would be slightly nice to get it cherry-picked here without the ASLR changes, since I'm not totally sure we are decided on doing ASLR---it has pretty limited benefits for the modal hyperlight use case because one can't (generally) re-slide an image after a snapshot is taken, and in practice almost all images of a given binary are expected to descend from one snapshot.

As I said before I don't feel incredibly strongly though---I will leave the decision up to you.

@cshung
cshung force-pushed the cshung/non-pie-elf-loading branch 6 times, most recently from 46c86bc to 93e5b1d Compare August 12, 2026 14:55
@cshung
cshung force-pushed the cshung/non-pie-elf-loading branch from 93e5b1d to c87a4cd Compare August 19, 2026 15:15

@syntactically syntactically left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is used by the GDB debug path (\GetCodeSectionOffset\ in \x86_64.rs) to report where the code section lives in guest virtual space. Without it, GDB cannot resolve symbols after snapshot restore. It is also needed for
ead_guest_memory_by_gva\ (tracing feature) to translate GVAs back to physical offsets.

Ah, of course. I think the SandboxMemoryLayout is already accessible in the relevant places and a lot of the information about the layout is already centralised there; could we just move this into that?

Silently relocating other regions would make behavior hard to reason about, and those regions have fixed GPAs that the PEB and guest runtime depend on. I think erroring is the right call here

Well, in the non-PIE mode, those gpas are not independent of the executable, because the code segment is near the bottom of the initial setup snapshot region. So, it's kind of being consistent to move them. However, I don't feel strongly.

Comment thread src/hyperlight_host/src/mem/layout.rs
Comment thread src/hyperlight_host/src/mem/layout.rs
Comment thread src/hyperlight_host/src/mem/memory_region.rs
Comment thread src/hyperlight_host/src/mem/memory_region.rs Outdated
Comment thread src/hyperlight_host/src/sandbox/snapshot/mod.rs Outdated
Comment thread src/hyperlight_host/src/sandbox/snapshot/file/mod.rs Outdated
Comment thread src/hyperlight_host/src/sandbox/snapshot/mod.rs
@cshung
cshung force-pushed the cshung/non-pie-elf-loading branch 5 times, most recently from c3b1897 to 93dc8f1 Compare August 26, 2026 17:23
@cshung
cshung force-pushed the cshung/non-pie-elf-loading branch from 93dc8f1 to 57cd8d3 Compare September 2, 2026 22:47
@dblnz

dblnz commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Have you by any chance tried debugging a guest using gdb/lldb to ensure PIE binaries behave correctly when debugged? Also, the same question applies to crashdump.

@cshung

cshung commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

Yes. The branch has end-to-end tests using the PIE simpleguest binary.

  • test_gdb_end_to_end verifies source breakpoint and symbol resolution.
  • test_gdb_from_snapshot verifies the same after snapshot restore.
  • test_crashdump_gdb_symbols verifies that GDB resolves the saved PC using the core file's AT_ENTRY load bias.
  • test_crashdump_gdb_symbols_from_snapshot verifies that path after snapshot restore.

The latest x64 KVM run executed and passed all four tests. The GDB tests also passed on x64 Hyper-V. These were real GDB sessions, not compile-only checks or skipped tests: https://github.com/hyperlight-dev/hyperlight/actions/runs/35659787035

I have not separately exercised LLDB. The GDB and crashdump tests cover the PIE load-address behavior this PR changes.

Comment thread src/hyperlight_host/src/sandbox/snapshot/mod.rs Outdated
Comment thread src/hyperlight_host/src/sandbox/snapshot/file/config.rs Outdated
cshung and others added 8 commits September 23, 2026 10:25
Add support for running non-PIE (ET_EXEC) guest binaries by mapping
code at the ELF's declared virtual address rather than assuming
identity mapping (physical == virtual).

Changes:
- Add is_pie() and base_va() methods to ExeInfo/ElfInfo to detect
  ET_DYN vs ET_EXEC binaries and extract the base virtual address
- Add SandboxMemoryLayout::code_virt_base() to compute the correct
  virtual base for the code region and validate it doesn't conflict
  with other memory regions
- Update snapshot creation to use non-identity virtual mapping for
  non-PIE code regions
- Add non-PIE guest build step to CI (cargo hyperlight with
  -C relocation-model=static -C link-args=--no-pie)
- Add integration test verifying non-PIE guest execution
- Add test helper for locating non-PIE guest binaries

Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 6f20a05d-6bee-4e2e-b320-12f8d9759bbc
Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>
Change GuestMemoryRegion::HostBaseType from () to usize so that
GuestMemoryRegion becomes a proper mapping: host_region carries
guest physical addresses (GPA) and guest_region carries guest
virtual addresses (GVA). For identity-mapped regions both are
the same. For non-PIE code the Code region's guest_region is
overridden to the ELF-declared virtual address.

Remove the guest_virt_addr field from MemoryRegion_ since its
role is now served by the guest_region/host_region split in
GuestMemoryRegion.

Use checked_add for the code VA overlap check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 6f20a05d-6bee-4e2e-b320-12f8d9759bbc
Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>
Rename get_memory_regions_ to get_memory_regions and remove the
generic type parameter. All callers use GuestMemoryRegion, so the
generic is unnecessary. The host_base argument is now always
BASE_ADDRESS, supplied internally.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 6f20a05d-6bee-4e2e-b320-12f8d9759bbc
Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 6f20a05d-6bee-4e2e-b320-12f8d9759bbc
Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 6f20a05d-6bee-4e2e-b320-12f8d9759bbc
Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>
@cshung
cshung force-pushed the cshung/non-pie-elf-loading branch from 3aec388 to 5568099 Compare September 23, 2026 17:59
@jsturtevant
jsturtevant enabled auto-merge (squash) September 23, 2026 21:30

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. ready-for-review PR is ready for (re-)review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants