TypeVar in route handler parameters recognized as strings not their constraint or bound types.

[dmajere]

Describe the bug
TypeVar in route handler parameters recognized as strings not their constraint or bound types.

To Reproduce

• create generic typevars
• use typevars as typehints for route handler parameters

from typing import TypeVar
from starlette.responses import PlainTextResponse

str_T = TypeVar('str_T', bound=str)
bool_T = TypeVar('bool_T', bound=bool)
int_T = TypeVar('int_T', bound=int)
const_T = TypeVar('const_T', int, float)

@router.get('/str_T/{elem}')
async def generic_str(elem: str_T):
    return PlainTextResponse(str(type(elem)))

@router.get('/bool_T/{elem}')
async def generic_bool(elem: bool_T):
    return PlainTextResponse(str(type(elem)))

@router.get('/int_T/{elem}')
async def generic_int(elem: int_T):
    return PlainTextResponse(str(type(elem)))

@router.get('/const_T/{elem}')
async def generic_const(elem: const_T):
    return PlainTextResponse(str(type(elem)))

curl http://localhost:8000/str_T/str
<class 'str'>%                                                                                                                                                                                               

curl http://localhost:8000/int_T/1
<class 'str'>%

curl http://localhost:8000/bool_T/True 
<class 'str'>%

curl http://localhost:8000/const_T/4
<class 'str'>%

Expected behavior
When for generics defined with constraints type casting is probably would be impossible, since
we have a list of possible types, for bounded typevars fastapi should type cast to bounded type.

[dmontagu]

Is there a desired use case for this? I'm having a hard time imagining a concrete scenario where supporting bounded typevars is more appropriate than just placing the bound itself in the annotation. (Not to say a reasonable application doesn't exist, just that I can't think of one.)

The typing API is not especially clean, and as a result I think supporting this feature might have the potential to add serious backwards-compatibility pain over time. As a result, I think it would be better for fastapi to not support this feature unless it enabled a capability that is currently missing or difficult to achieve cleanly.

[dmajere]

No, I would say that there is no strong desire.
It's more for documentation-like purposes, usually It gives you more information on what code is doing when you see domain-specific types.
However, yes, this is more of a nice-to-have than a necessity.

[dmontagu]

@dmajere if you want domain-specific types, you might try typing.NewType (which is supported by fastapi currently). Based on your description it sounds like this may actually be more appropriate than a TypeVar anyway.

[dmajere]

@dmontagu
This works perfectly, thanks.

[dmontagu]

Glad to hear it! I use typing.NewType extensively in my own projects.

Especially for object IDs -- this makes it much easier to prevent silly mistakes where you accidentally call a function expecting an ID of one type of object with an ID of another type. If everything is an integer or a UUID, it can be very painful to debug the problem, but using typing.NewType means mypy will catch it for you without any effort.

[euri10]

I'm a mypy newb and didn;t get the subtlety of this, care giving a little snippet @dmontagu ?

…

On Wed, Sep 18, 2019 at 6:30 AM dmontagu ***@***.***> wrote: Glad to hear it! I use typing.NewType extensively in my own projects. Especially for object IDs -- this makes it much easier to prevent silly mistakes where you accidentally call a function expecting an ID of one type of object with an ID of another type. If everything is an integer or a UUID, it can be very painful to debug the problem, but using typing.NewType means mypy will catch it for you without any effort. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#533>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAINSPT6N2OPP6GITZULKRLQKGVFDANCNFSM4IW4EZSA> .

-- benoit barthelet http://pgp.mit.edu/pks/lookup?op=get&search=0xF150E01A72F6D2EE

[dmontagu]

Here's an example that demonstrates the value:

Imagine you have a database of products, and have an API for marking a product as "saved" for a particular user. Without NewType, some of the functions/classes involved might look like:

import uuid

from pydantic import BaseModel

class Product(BaseModel):
    product_id: uuid.UUID
    title: str

class SavedProductRecord(BaseModel):
    saved_product_id: uuid.UUID
    product_id: uuid.UUID
    user_id: uuid.UUID

# "CRUD" functions
def read_saved_product(saved_product_id: uuid.UUID) -> SavedProductRecord:
    pass

def read_product(product_id: uuid.UUID) -> Product:
    pass

Now pretend you get a SavedProductRecord instance from somewhere (e.g., after loading all of the records for a particular user), and then want to load the associated product. You go to call read_product, but your mind slips for a second and you pass the function the saved_product_id, rather than the product_id:

record: SavedProductRecord = ...  # pretend this came from somewhere
read_product(record.saved_product_id)

You won't be able to find the matching product because you gave the wrong id (should have been product_id, not saved_product_id), but there's nothing in the code here that could be detected as a bug.

Since both product_id and saved_product_id have the same type and similar names, this error is hard to detect either algorithmically or as a human. But in an intuitive sense, the product_id and saved_product_id are a totally different "types" of things -- they should never be used interchangeably.

typing.NewType gives a way to formalize this idea in a way that works with type checkers.

Here's what the above code would look like with NewType:

from typing import NewType
import uuid

from pydantic import BaseModel

SavedProductID = NewType("SavedProductID", uuid.UUID)
ProductID = NewType("ProductID", uuid.UUID)
UserID = NewType("UserID", uuid.UUID)

class Product(BaseModel):
    product_id: ProductID
    title: str


class SavedProductRecord(BaseModel):
    saved_product_id: SavedProductID
    product_id: ProductID
    user_id: UserID


def read_saved_product(saved_product_id: SavedProductID) -> SavedProductRecord:
    pass


def read_product(product_id: ProductID) -> Product:
    pass


record: SavedProductRecord = ...  # pretend this came from somewhere
read_product(record.saved_product_id)

If you call mypy on this now, you'll get:

error: Argument 1 to "read_product" has incompatible type "SavedProductID"; expected "ProductID"

Also, PyCharm can detect the error:

With the pydantic mypy and/or pycharm plugins, you can also get ID type checking in BaseModel init signatures (very useful when converting one model to another).

---

This same idea is useful well beyond just ID types; any time you can imagine a domain-specific "type", but are representing the data as a bare str, int, etc., you can use NewType to get type checking for the data. An example of detecting this type of error might look like:

Bad:

class UserRegistrationRequest(BaseModel):
    username: str
    password: str

def create_user(password: str, username: str):
    ...

@app.post("/register")
def register(registration_request: UserRegistrationRequest) -> None:
    # whoops, wrong order
    create_user(registration_request.username, registration_request.password)  

Good:

RawPassword = NewType("RawPassword", str)
class UserRegistrationRequest(BaseModel):
    username: str
    password: RawPassword

def create_user(password: RawPassword, username: str):
    ...

@app.post("/register")
def register(registration_request: UserRegistrationRequest) -> None:
    # the error gets detected by mypy since registration_request.username is not a RawPassword
    create_user(registration_request.username, registration_request.password)

---

If you need to convert an instance of a type to a NewType-version, you just do:

RawPassword = NewType("RawPassword", str)
my_string = "helloworld"  # mypy thinks this is a string
my_password = RawPassword(my_string)  # mypy thinks this is a RawPassword

NewTypes are treated as subclasses of their "origin" type, so you could pass a RawPassword anywhere a str would be accepted. At runtime, calling RawPassword(my_string) actually does nothing -- at runtime RawPassword is essentially the same as lambda x: x (this ensures minimal performance impact). But mypy treats it differently.

[euri10]

ok I already told you @dmontagu but that's an amazing way of explaining clearly a feature a lot of people would not even notice it exists, just because you explains very well the problem it solves at first, it doesn't come out of the blue, it's pragmatic.

I just read again the mypy docs about NewType and I was debating if I found the git man pages more or less convoluted...

edit: that should go to pydantic docs !

[tiangolo]

Thanks for the help here everyone! 👏 🙇

Thanks for reporting back and closing the issue @dmajere 👍

[emilhdiaz]

One use case where it would be helpful to have support for this is when using class based router views from the fastapi_utils package: https://fastapi-utils.davidmontague.xyz/user-guide/class-based-views/

With a generic base class for all my routers:

ModelReadType = TypeVar("ModelReadType")
ModelCreateType = TypeVar("ModelCreateType")

class AppRouter(Generic[ModelReadType, ModelCreateType]):
    session: Session = Depends(GetSession)

    @router.post('/', response_model=ModelReadType)
    async def _create(self, user: ModelCreateType = Body(embed=False)):
        ....

    ... remaining CRUD operations

I could quickly inherit all of this CRUD functionality in my actual routes and even override route methods if I wanted to:

class UserRouter(AppRouter[UserReadModel, UserCreateModel]):
    ....

    @router.read('/', response_model=ModelReadType)
    async def _get(self, id: str):
        ....

Currently, this allows me to use the endpoints, but without any type inference or validation on those input models.

[github-actions]

Assuming the original need was handled, this will be automatically closed now. But feel free to add more comments or create new issues or PRs.