[red-knot] AnnAssign with no RHS is not a Definition

[carljm]

My plan for handling declared types is to introduce a Declaration in addition to Definition. A Declaration is an annotation of a name with a type; a Definition is an actual runtime assignment of a value to a name. A few things (an annotated function parameter, an annotated-assignment with an RHS) are both a Definition and a Declaration.

This more cleanly separates type inference (only cares about Definition) from declared types (only impacted by a Declaration), and I think it will work out better than trying to squeeze everything into Definition. One of the tests in this PR (annotation_only_assignment_transparent_to_local_inference) demonstrates one reason why. The statement x: int should have no effect on local inference of the type of x; whatever the locally inferred type of x was before x: int should still be the inferred type after x: int. This is actually quite hard to do if x: int is considered a Definition, because a core assumption of the use-def map is that a Definition replaces the previous value. To achieve this would require some hackery to effectively treat x: int sort of as if it were x: int = x, but it's not really even equivalent to that, so this approach gets quite ugly.

As a first step in this plan, this PR stops treating AnnAssign with no RHS as a Definition, which fixes behavior in a couple added tests.

This actually makes things temporarily worse for the ellipsis-type test, since it is defined in typeshed only using annotated assignments with no RHS. This will be fixed properly by the upcoming addition of declarations, which should also treat a declared type as sufficient to import a name, at least from a stub.

[github-actions]

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

[MichaReiser]

My plan for handling declared types is to introduce a Declaration in addition to Definition. A Declaration is an annotation of a name with a type; a Definition is an actual runtime assignment of a value to a name. A few things (an annotated function parameter, an annotated-assignment with an RHS) are both a Definition and a Declaration.

I'm a bit concerned about creating two ingredients because of the performance implications. We sofar tried to come up to reduce the ingredients but this approach would increase the number of ingredients. Unfortunately, I don't feel like i have a good enough understanding of what you're trying to do or what motivates to have two ingredients to make a concrete suggestion.

Following the outlined logic, a fully annotated class or function should then also create a Definition and a Declaration because they are both. Arguably, even a partially annotated function is both (and so are its parameters).

[AlexWaygood]

I think this is correct in terms of the semantics when it comes to runtime files, but I'd like to hear more about how you plan to handle the semantics of stub files. It seems like there are two possible ways of handling stubs:

1. Say that annotations do actually create definitions in stub files as well as declarations
2. Say that annotations never create definitions, only declarations, but that a declaration has a different meaning in the context of a stub file

It sounds like you've chosen option (2) here but intuitively to me that feels like the more complicated option

[MichaReiser]

There's no reason to block this PR. I think the code here looks good but I'm interested to learn more about how you plan to introduce a new Declaration ingredient.

[AlexWaygood]

I guess a question I have here is: so what's the point of this annotation, if our more precise inference takes priority over the declared type? Should we emit a diagnostic warning the user that the declaration will have no effect?

Hmm, and what about something like this? Our precise inference will probably cause the type to be inferred as list[Literal[1, 2, 3]], but that's clearly not what the user wants. And if we infer the precise type, we'll emit a spurious error on the takes_list_of_ints(x) call due to list invariance:

x: list[int] = [1, 2, 3]

def takes_list_of_ints(arg: list[int]) -> None:
    pass

takes_list_of_ints(x)

[carljm]

This is something I can try to clarify more in the design doc, but I'll try to answer here briefly.

The x: int annotation in this particular code sample has no effect, because there is no assignment to x anywhere following it. But in general, the effect that it would have would be that any assignment to x after x: int must assign something that is assignable to int, or else there will be an invalid-assignment diagnostic. Without the x: int, any subsequent assignment to x would be allowed and we'd just happily allow the shadowing to a new inferred type (or infer the union if it's conditional.)

Regarding generics, let me get into that more in the design document.

[carljm]

It looks like we need more of a design document on declared types implementation; let's move the broader discussion on my PR description there instead, since it's only tangentially related to this PR. This PR makes sense as an initial step regardless, because the added tests are needed and fix current bugs, and the implementation changes will be easy enough to adjust to whatever we do for declared types.