Skip to content

[python] Preserve nested union container values - #25005

Closed
markstuart-oai wants to merge 1 commit into
OpenAPITools:masterfrom
markstuart-oai:codex/fix-python-nested-union-roundtrips
Closed

markstuart-oai wants to merge 1 commit into
OpenAPITools:masterfrom
markstuart-oai:codex/fix-python-nested-union-roundtrips

Conversation

@markstuart-oai

@markstuart-oai markstuart-oai commented Sep 23, 2026

Copy link
Copy Markdown

Why

A Python union containing an array/map of another generated union can silently lose valid data. For example, with Message.content = oneOf(string, array<Part>) and Part = oneOf(Text, Image), Message.from_dict({"content": [{"text": "x"}]}) serializes the content as [null]. The outer union lets Pydantic construct the inner wrapper without selecting its actual_instance. to_json() also exposes wrapper fields for explicitly constructed nested values.

Reproduced on 7.25.0 and current master 6f77724ce5d4a9d61ccb91fde60d80f1c38b0bed, using Python 3.12.13 and Pydantic 2.12.5/2.13.5. The reduction needs no OpenAI extensions, seed, nullable fields, or OpenAPI 3.1. Related: #22261 and OpenAI's serialization report; this is an independent reproduction, not an identification of that reporter's unknown toolchain.

Change

  • Decode composed-model leaves inside union arrays/maps through their generated from_dict, then retain the existing Pydantic container validation and union matching rules.
  • Leave ordinary models, primitives and enums on their existing parsing path. Routing ordinary models through from_dict would change omitted nullable fields into explicit nulls; regression coverage prevents that.
  • Project nested union values recursively for to_dict/to_json, without depending on ApiClient.
  • Keep constructors, class names, actual_instance, and legacy public/wire dictionary behavior. The intended observable correction is that modern to_dict returns nested data instead of wrapper objects. This is not a general rewrite of Pydantic model validation and does not fix the separate python-pydantic-v1 generator.

Four implementation files and two fixture/test files are authored changes. The remaining files are regenerated samples, including the two Pydantic-v1 samples that share the fixture.

Validation

  • Native PythonClientCodegenTest: 90 passed; CLI build passed.
  • Existing Python model/deserialization/serialization tests plus new regressions: 120 passed, 2 existing skips. The exact new regression suite is 14 failed / 14 passed on unmodified master, 28 passed with the fix.
  • Default sample mypy: 153 files clean. Legacy dictionary sample: 11 tests passed, 17 files mypy-clean.
  • Regenerated all 12 affected sample configurations with bin/generate-samples.sh.
  • Independent model-only generation verifies nested unions without supporting API files, enum/primitive handling, ambiguity rejection, typed construction, and omission/null preservation on both Pydantic versions.
  • Actual Chat-operation schema slice: 17 cases. Text/image/audio content arrays are repaired; all other captured request bodies are unchanged. Exact signed-64 endpoints survive and adjacent invalid values remain rejected. No network request was sent.

Native command: ./mvnw -pl modules/openapi-generator-cli -am package -DskipTests=false -Dtest=PythonClientCodegenTest -Dsurefire.failIfNoSpecifiedTests=false -Dmaven.javadoc.skip=true (optional remote build cache and build-scan upload disabled locally). No full live-Petstore integration claim.

PR checklist

  • Read the contribution guidelines.
  • Build the affected generator, run its tests and regenerate affected samples. No generator option/docs change.
  • Request Python generator review: @wing328.

Summary by cubic

Fixes Python union models losing data when they contain arrays or maps of other generated unions: from_dict no longer produces [null] for nested values, and to_dict/to_json return the nested data instead of wrapper objects.

  • Decodes composed-model leaves inside union containers through their generated from_dict, preserving Pydantic container validation and union matching rules.
  • Leaves ordinary models, primitives, and enums on their existing parsing path so omitted nullable fields stay omitted.
  • Recursively projects nested union values for to_dict/to_json without depending on ApiClient; constructors, class names, and actual_instance behavior are unchanged.
  • Adds regression fixtures and regenerated samples across affected configurations; the python-pydantic-v1 generator is not affected.

Written for commit 2b275f9. Summary will update on new commits.

Review in cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant