Skip to content

[WIP] handle diverging protocols - #22025

Closed
ilevkivskyi wants to merge 13 commits into
python:masterfrom
ilevkivskyi:limit-proto-stack
Closed

ilevkivskyi wants to merge 13 commits into
python:masterfrom
ilevkivskyi:limit-proto-stack

Conversation

@ilevkivskyi

Copy link
Copy Markdown
Member

No description provided.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@ilevkivskyi

Copy link
Copy Markdown
Member Author

@JukkaL I am still investigating things here, but there is something interesting I noticed: pandas requires insanely large protocol stack (it still crashes with stack of size 20). I am not sure how exactly, but this is caused by the cycle we have is_subtype() -> unify_generic_callable() -> infer_constraints() -> make_simplified_union() -> is_subtype(). That make_simplified_union() is technically needed to make type inference stable w.r.t. equivalent types.

But, the fun part is removing union simplification there makes pandas (and various other libraries that depend on it) ~2x faster (yes, you read that right, twice faster, not 2% faster). This however makes couple tests fail and causes a bit of primer fallout (mostly because of redundant unions in some asyncio stubs).

For now I will probably simply set some high protocol stack size. This should fix at least three crashes. But I also think we should consider cutting some corners in type inference (in a separate PR), since 2x speed-up looks very tempting.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

@ilevkivskyi

Copy link
Copy Markdown
Member Author

OK, I think I have enough data to make a real PR, I will close this one an open a new clean PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant