Testing custom error handlers

[gonzacastro]

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

• I commit to help with one of those options 👆

Example Code

class BaseError(Exception):
    def __init__(self, message, status_code) -> None:
        self.message = message
        self.status_code = status_code

async def error_handler(request: Request, exc: BaseError):
    return JSONResponse(content={"message": exc.message}, status_code=exc.status_code)

Description

Given the example code, which is only a custom exception class and its handler, what I was trying while making the tests was to cover the handler lines but it keeps being uncovered, even when I force the exception to be raised. By the time, it's not that relevant because it's only one line but it'd get a pain if one handler is larger...
I tried several things, from raising the exception myself to adding the handler to the test client, so please, if anyone know about this I'd really appreciate the help.

Operating System

Windows

Operating System Details

No response

FastAPI Version

0.78.0

Python Version

3.9.13

Additional Context

No response

[odiseo0]

What tool are you using for testing the application? Also, can you show how are you testing it?

[odiseo0]

Btw, this code is similar to yours

from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse


class BaseError(Exception):
    def __init__(self, message: str, status_code: int) -> None:
        self.message = message
        self.status_code = status_code


app = FastAPI()

@app.exception_handler(BaseError)
async def error_handler(request: Request, exc: BaseError):
    print("Hello")
    return JSONResponse(content={"message": exc.message}, status_code=exc.status_code)


@app.get("/")
def index():
    raise BaseError(message="Error", status_code=404)

The output I get when visiting / is the following:

Hello
INFO:     127.0.0.1:63867 - "GET / HTTP/1.1" 404 Not Found

Maybe you're missing the exception_handler decorator

[gonzacastro]

Thanks for replying! I think I'm not missing the decorator, actually the example code is just a snipper, the actual code is divided into different files, and the decorator is for the app. When I run the app it works and handles the exception properly but my problem is when I want to write the unit tests. Please tell me if I was clear.

[odiseo0]

Understood. How are you writing the test and what is the behavior that you expect from the exception_handler?

[gonzacastro]

I'm running the tests with pytest and I'm patching some random app function with a BaseError side effect so it triggers the handler. What I expect is for the test to execute the handler code lines. FYI I also tried to call the handler method directly but it returns me a coroutine object instead of the JSONResponse.

[JarroVGIT]

I'm patching some random app function with a BaseError side effect so it triggers the handler.

I don’t think this works the way you think it works. Especially if you say that when testing the handler directly, and you didn’t expect a coroutine to be returned by an asynchronous function🤔 your (async) test client needs to await the function for it to return a value.

Provide the testcase that you tried, including any fixtures and mocking, maybe that clarifies.

[gonzacastro]

After several tries, I could achieve to cover the file with tests.

@pytest.mark.anyio
async def test_error_handler():
    response = await error_handler(None, error)
    assert response.body == {'message': 'test'}

Thank you guys for replying!

[Kludex]

The best way to test custom exception handlers would be to test an endpoint that triggers that exception, and verify that the response matches what the custom exception handler is doing.

[blenzi]

Hello, would you have a full working example, please? I would like to catch any exception, that will normally turn into HTTPException with status_code 500. But during testing, the original exception is raised and not handled.

@app.exception_handler(500)
async def server_error(request: Request, exc: Exception):
    ...
    return JSONResponse(
        status_code=500,
        content={"message": f"Problem with request, issue opened: {issue_iid}"},
    )

Test file:

import pytest

@pytest.fixture
def mock_error_500():
    """Add route to raise ValueError"""

    @app.get("/raise-500")
    async def raise_value_error():
        raise ValueError("This is a test")

def test_server_error(fixture, mock_error_500, get_headers):  # noqa F811
    with get_client() as client:
        response = client.get("/raise-500")  # raises ValueError !
        assert response.status_code == 500

[tianjing-li]

@Kludex have there been any updates on this issue? I also was not able to directly test the behaviour of the exception handler, it works when running the app and an endpoint raises an unexpected error, but testing it just raises the initial error without going through the handler.

My code is similar to @blenzi 's