The memory usage piles up over the time and leads to OOM

[prav2019]

First check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.
• After submitting this, I commit to one of:
  • Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
  • I already hit the "watch" button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
  • Implement a Pull Request for a confirmed bug.

Example

Here's a self-contained, minimal, reproducible, example with my use case:

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def read_root():
    return {"Hello": "World"}

Description

• Open the browser and call the endpoint /.
• It returns a JSON with {"Hello": "World"}.
• But I expected it to return {"Hello": "Sara"}.

Environment

• OS: [e.g. Linux / Windows / macOS]:
• FastAPI Version [e.g. 0.3.0]:

To know the FastAPI version use:

python -c "import fastapi; print(fastapi.__version__)"

• Python version:

To know the Python version use:

python --version

Additional context

Tracemalloc gave insight on the lines , that are top consumers of memory: (top one seems to be below line in uvicorn)
/usr/local/lib/python3.6/site-packages/uvicorn/main.py:305:
Line:
loop.run_until_complete(self.serve(sockets=sockets))

[prav2019]

same issue i faced @prav2019

Any solution to overcome that @Riki-1mg

[Riki-1mg]

No @prav2019,
are you using aiohttp as http clients in your service ?
#1623

[prav2019]

No @Riki-1mg

[teymour-aldridge]

@app.get("/")
def read_root():
  return {"Hello": "World"}

Surely the expected behaviour here is to return {"Hello": "World"}?

If you want this function to return {"Hello": "Sara"} you'd probably need to do something like:

@app.get("/")
def read_root():
  return {"Hello": "Sara"}

Further, I can't reproduce your error on my "machine" (it's sitting on a cloud somewhere). You can see the full details here but everything looks to be working fine.

I suspect that this is specific to your operating system setup, etc. Would you please provide some more info needed/useful to know in context of how to reproduce the error?

• How much RAM does your machine have?
• How much memory is each function using (ideally include all the debugging output).
  This wil look something like the below (example from the Python documentation)

[ Top 10 ]
<frozen importlib._bootstrap>:716: size=4855 KiB, count=39328, average=126 B
<frozen importlib._bootstrap>:284: size=521 KiB, count=3199, average=167 B
/usr/lib/python3.4/collections/__init__.py:368: size=244 KiB, count=2315, average=108 B
/usr/lib/python3.4/unittest/case.py:381: size=185 KiB, count=779, average=243 B
/usr/lib/python3.4/unittest/case.py:402: size=154 KiB, count=378, average=416 B
/usr/lib/python3.4/abc.py:133: size=88.7 KiB, count=347, average=262 B
<frozen importlib._bootstrap>:1446: size=70.4 KiB, count=911, average=79 B
<frozen importlib._bootstrap>:1454: size=52.0 KiB, count=25, average=2131 B
<string>:5: size=49.7 KiB, count=148, average=344 B
/usr/lib/python3.4/sysconfig.py:411: size=48.0 KiB, count=1, average=48.0 KiB

• Does your out-of-memory error include a traceback? Please include that if that's the case.

[prav2019]

@teymour-aldridge : here is the debug statistics:
Memory Statistics
Top 10 Files
/usr/local/lib/python3.6/site-packages/uvicorn/main.py:305: size=1652 KiB (+1499 KiB), count=4597 (+4172), average=368 B KB

/usr/local/lib/python3.6/site-packages/starlette/applications.py:136: size=1288 KiB (+1173 KiB), count=2290 (+2086), average=576 B KB

/usr/local/lib/python3.6/threading.py:347: size=943 KiB (+854 KiB), count=1836 (+1657), average=526 B KB

/usr/local/lib/python3.6/queue.py:145: size=919 KiB (+835 KiB), count=1783 (+1619), average=528 B KB

/usr/local/lib/python3.6/asyncio/locks.py:233: size=885 KiB (+807 KiB), count=9633 (+8771), average=94 B KB

/usr/local/lib/python3.6/site-packages/uvicorn/protocols/http/httptools_impl.py:82: size=788 KiB (+717 KiB), count=6876 (+6264), average=117 B KB

/usr/local/lib/python3.6/site-packages/uvicorn/protocols/http/httptools_impl.py:77: size=751 KiB (+684 KiB), count=2289 (+2086), average=336 B KB

/usr/local/lib/python3.6/site-packages/uvicorn/protocols/http/httptools_impl.py:146: size=725 KiB (+662 KiB), count=15984 (+14611), average=46 B KB

/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/result.py:376: size=657 KiB (+590 KiB), count=10490 (+9426), average=64 B KB

/usr/local/lib/python3.6/site-packages/uvicorn/protocols/http/httptools_impl.py:285: size=609 KiB (+555 KiB), count=4589 (+4183), average=136 B KB

Scanned Lines that consumes more memory
uvicorn/main.py

loop.run_until_complete(self.serve(sockets=sockets))
starlette/applications.py

scope["app"] = self
python3.6/threading.py

waiters_to_notify = _deque(_islice(all_waiters, n))
python3.6/queue.py

self.not_empty.notify()
asyncio/locks.py

self._waiters = collections.deque()
http/httptools_impl.py

self.parser = httptools.HttpRequestParser(self)
http/httptools_impl.py

self.config = config
http/httptools_impl.py

self.parser.feed_data(data)
engine/result.py

for obj_elem in elem[4]
http/httptools_impl.py

self.timeout_keep_alive, self.timeout_keep_alive_handler

[prav2019]

@teymour-aldridge the above grows and cause OOM after a while

[teymour-aldridge]

@prav2019 I can't reproduce the bug on either my machine or on a cloud-hosted linux container; this leads me to believe that the problem is in the way that your machine/environment setup.

In the issue template, it asks for the following fields – would you mind filling them in?

OS: [e.g. Linux / Windows / macOS]:
FastAPI Version [e.g. 0.3.0]:

Also, there's a "checklist" at the top of the issue which you should fill out!

[prav2019]

@teymour-aldridge , this usually happens when there are some traffic over a period of time. Usually this is happening in our prod environment.
OS: using docker image: python 3.6 [so debian ] **Linux
FastAPI version: fastapi[all]==0.20.0

[prav2019]

@teymour-aldridge , current status:
Top 10 Files
/usr/local/lib/python3.6/site-packages/uvicorn/main.py:305: size=3650 KiB (+1922 KiB), count=10158 (+5349), average=368 B KB

/usr/local/lib/python3.6/site-packages/starlette/applications.py:136: size=2851 KiB (+1504 KiB), count=5069 (+2673), average=576 B KB

/usr/local/lib/python3.6/threading.py:347: size=2104 KiB (+1110 KiB), count=4083 (+2155), average=528 B KB

/usr/local/lib/python3.6/queue.py:145: size=2049 KiB (+1087 KiB), count=3974 (+2109), average=528 B KB

/usr/local/lib/python3.6/asyncio/locks.py:233: size=1948 KiB (+1017 KiB), count=21295 (+11208), average=94 B KB

/usr/local/lib/python3.6/site-packages/uvicorn/protocols/http/httptools_impl.py:82: size=1744 KiB (+920 KiB), count=15226 (+8037), average=117 B KB

/usr/local/lib/python3.6/site-packages/uvicorn/protocols/http/httptools_impl.py:77: size=1663 KiB (+877 KiB), count=5068 (+2673), average=336 B KB

/usr/local/lib/python3.6/site-packages/uvicorn/protocols/http/httptools_impl.py:146: size=1598 KiB (+839 KiB), count=35181 (+18488), average=47 B KB

/usr/local/lib/python3.6/site-packages/uvicorn/protocols/http/httptools_impl.py:285: size=1349 KiB (+712 KiB), count=10163 (+5364), average=136 B KB

/usr/local/lib/python3.6/site-packages/uvicorn/protocols/http/httptools_impl.py:216: size=1288 KiB (+676 KiB), count=30112 (+15818), average=44 B KB

[teymour-aldridge]

0.20.0 is a somewhat old version of FastAPI, what happens if you use the latest release instead?

[prav2019]

@teymour-aldridge haven't tried with latest version ? but i like to give it a shot, if there were any memory fix done with the versions after 0.20.0

[prav2019]

@teymour-aldridge updated to latest version, will report back, once I see the result, Thanks

[prav2019]

@teymour-aldridge I checked on our staging, after updating to latest versions [fast api and uvicorn], the memory issue still exists

[prav2019]

@teymour-aldridge can this one work, i saw this in uvicorn documentation:
--limit-max-requests - Maximum number of requests to service before terminating the process. Useful when running together with a process manager, for preventing memory leaks from impacting long-running processes.

[teymour-aldridge]

@prav2019 I don't know, it might do. How many requests are you handling and how many machines do you have?

[rakshit-agrawal]

We have recently experienced similar issues in our application, and even though there are several other differences, it is highly likely that the cause of increasing memory utilization is similar to the issue here. Is there any update or insight on this that might be helpful?

[dncastilho]

At the moment I configured gunicorn to restart workers after 500 requests and the problem is now gone. But I don't consider this a solution, its a workaround - though an effective one.

[OscarGarciaF]

Hi @dncastilho
Could you elaborate on your workaround?
I used --limit-max-requests but this will only kill workers but never restart them, after all workers are shutdown the application stops.
How did you managed to restart the workers?

[dncastilho]

Hi @dncastilho Could you elaborate on your workaround? I used --limit-max-requests but this will only kill workers but never restart them, after all workers are shutdown the application stops. How did you managed to restart the workers?

I use a gunicorn_config.py file, and the parameter I include there is

max_requests = 500

And that does restart the application server. Reference: docs

[dncastilho]

Here's my full config file, by the way (but it is pretty unremarkeable. I built it based on a sample, left the comments there so you understand what is what):

# Sample Gunicorn configuration file.

#
# Server socket
#
#   bind - The socket to bind.
#
#       A string of the form: 'HOST', 'HOST:PORT', 'unix:PATH'.
#       An IP is a valid HOST.
#
#   backlog - The number of pending connections. This refers
#       to the number of clients that can be waiting to be
#       served. Exceeding this number results in the client
#       getting an error when attempting to connect. It should
#       only affect servers under significant load.
#
#       Must be a positive integer. Generally set in the 64-2048
#       range.
#

bind = 'unix:/home/deploy/gunicorn.sock'
backlog = 2048

#
# Worker processes
#
#   workers - The number of worker processes that this server
#       should keep alive for handling requests.
#
#       A positive integer generally in the 2-4 x $(NUM_CORES)
#       range. You'll want to vary this a bit to find the best
#       for your particular application's work load.
#
#   worker_class - The type of workers to use. The default
#       sync class should handle most 'normal' types of work
#       loads. You'll want to read
#       http://docs.gunicorn.org/en/latest/design.html#choosing-a-worker-type
#       for information on when you might want to choose one
#       of the other worker classes.
#
#       A string referring to a Python path to a subclass of
#       gunicorn.workers.base.Worker. The default provided values
#       can be seen at
#       http://docs.gunicorn.org/en/latest/settings.html#worker-class
#
#   worker_connections - For the eventlet and gevent worker classes
#       this limits the maximum number of simultaneous clients that
#       a single process can handle.
#
#       A positive integer generally set to around 1000.
#
#   timeout - If a worker does not notify the master process in this
#       number of seconds it is killed and a new worker is spawned
#       to replace it.
#
#       Generally set to thirty seconds. Only set this noticeably
#       higher if you're sure of the repercussions for sync workers.
#       For the non sync workers it just means that the worker
#       process is still communicating and is not tied to the length
#       of time required to handle a single request.
#
#   keepalive - The number of seconds to wait for the next request
#       on a Keep-Alive HTTP connection.
#
#       A positive integer. Generally set in the 1-5 seconds range.
#

workers = 3
worker_class = 'uvicorn.workers.UvicornWorker'
worker_connections = 1000
timeout = 90
keepalive = 2
max_requests = 500

#
#   spew - Install a trace function that spews every line of Python
#       that is executed when running the server. This is the
#       nuclear option.
#
#       True or False
#

spew = False

#
# Server mechanics
#
#   daemon - Detach the main Gunicorn process from the controlling
#       terminal with a standard fork/fork sequence.
#
#       True or False
#
#   raw_env - Pass environment variables to the execution environment.
#
#   pidfile - The path to a pid file to write
#
#       A path string or None to not write a pid file.
#
#   user - Switch worker processes to run as this user.
#
#       A valid user id (as an integer) or the name of a user that
#       can be retrieved with a call to pwd.getpwnam(value) or None
#       to not change the worker process user.
#
#   group - Switch worker process to run as this group.
#
#       A valid group id (as an integer) or the name of a user that
#       can be retrieved with a call to pwd.getgrnam(value) or None
#       to change the worker processes group.
#
#   umask - A mask for file permissions written by Gunicorn. Note that
#       this affects unix socket permissions.
#
#       A valid value for the os.umask(mode) call or a string
#       compatible with int(value, 0) (0 means Python guesses
#       the base, so values like "0", "0xFF", "0022" are valid
#       for decimal, hex, and octal representations)
#
#   tmp_upload_dir - A directory to store temporary request data when
#       requests are read. This will most likely be disappearing soon.
#
#       A path to a directory where the process owner can write. Or
#       None to signal that Python should choose one on its own.
#

daemon = False
raw_env = [
    'ENV=PROD'
]
pidfile = None
umask = 0
user = None
group = None
tmp_upload_dir = None

#
#   Logging
#
#   logfile - The path to a log file to write to.
#
#       A path string. "-" means log to stdout.
#
#   loglevel - The granularity of log output
#
#       A string of "debug", "info", "warning", "error", "critical"
#

errorlog = '-'
loglevel = 'debug'
accesslog = '-'
access_log_format = '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"'

#
# Process naming
#
#   proc_name - A base to use with setproctitle to change the way
#       that Gunicorn processes are reported in the system process
#       table. This affects things like 'ps' and 'top'. If you're
#       going to be running more than one instance of Gunicorn you'll
#       probably want to set a name to tell them apart. This requires
#       that you install the setproctitle module.
#
#       A string or None to choose a default of something like 'gunicorn'.
#

proc_name = None

#
# Server hooks
#
#   post_fork - Called just after a worker has been forked.
#
#       A callable that takes a server and worker instance
#       as arguments.
#
#   pre_fork - Called just prior to forking the worker subprocess.
#
#       A callable that accepts the same arguments as after_fork
#
#   pre_exec - Called just prior to forking off a secondary
#       master process during things like config reloading.
#
#       A callable that takes a server instance as the sole argument.
#

def post_fork(server, worker):
    server.log.info("Worker spawned (pid: %s)", worker.pid)

def pre_fork(server, worker):
    pass

def pre_exec(server):
    server.log.info("Forked child, re-executing.")

def when_ready(server):
    server.log.info("Server is ready. Spawning workers")

def worker_int(worker):
    worker.log.info("worker received INT or QUIT signal")

    ## get traceback info
    import threading, sys, traceback
    id2name = {th.ident: th.name for th in threading.enumerate()}
    code = []
    for threadId, stack in sys._current_frames().items():
        code.append("\n# Thread: %s(%d)" % (id2name.get(threadId,""),
            threadId))
        for filename, lineno, name, line in traceback.extract_stack(stack):
            code.append('File: "%s", line %d, in %s' % (filename,
                lineno, name))
            if line:
                code.append("  %s" % (line.strip()))
    worker.log.debug("\n".join(code))

def worker_abort(worker):
    worker.log.info("worker received SIGABRT signal")

[OscarGarciaF]

Thanks a lot @dncastilho, unfortunately the restarts kill my application due to memory usage errors despite the dashboard showing the usage not going over 60%, very strange.

[pdd-vn]

@lorosanu I have tested your simple-api using gunicorn instead of uvicorn. The memory usage over time is pretty stable. Maybe it was a uvicorn problem?

EDIT: command I used:

mprof run gunicorn simple_api.app:app \
         --worker-class uvicorn.workers.UvicornWorker \
         --workers=4 \
         --bind=0.0.0.0:8080

[anuriq]

Just posting this here, maybe someone will find it useful.
In our case after a lot of testing we found that source of memory leak was in the code itself, but fastapi + starlette + uvicorn setup amplified that leak and it started troubling us. After checking different things, it was pinpointed to anyio's thread pool for running non-async code. By default, it creates 40 threads in this pool. And you could say that it amplifies leak by x40.

from anyio import CapacityLimiter
from anyio.lowlevel import RunVar
from fastapi import FastAPI

app = FastAPI(title="Some app")

@app.on_event("startup")
def startup() -> None:
    RunVar("_default_thread_limiter").set(CapacityLimiter(1))

After using this code leak became so weak, that we could ignore it. NOTE: this code basically makes non-async part of your API single-threaded, which is bad of course. But if mem leak is bigger problem, it will help.

[geril2207]

I also faced problem with memory leaking, every request increase docker container memory usage around 0.5mb for request (simple request) and for me solving the problem was disable debug in fastapi setup

[jinluyang]

In my case, debug is already false but the oom still exists. I have uvicorn==0.20.0, fastAPI==0.83.0. My app is not using async, so it has a internal fastAPI threadpool. I tried add async to my function and the memory seems still increasing. My function is like:

I used pmap -x to see the memory and can see the difference after running for a while. On the left is pmap before and on the right is after:

it seems the difference is about the anonymous space. I don't know if this information can help guys to find out the problem.

[AdityaSoni19031997]

I am not sure but debug is False, by default, right? Just out of curiosity, If we run the same API on flask for an e.g., Do we still see the same memory degradation over time?

Thanks

[kee0624]

Hi, I would know if is any update for this issue?

[swapkh91]

seeing this on GKE, the dips are when I redeployed the service

fastapi==0.88.0
uvicorn==0.20.0

[suniladityajatni]

We solve our problem using hypercorn instead of uvicorn.

[Firexd2]

hypercorn can't solve this issue at all :(

[suniladityajatni]

We solve our problem using hypercorn instead of uvicorn.

[suniladityajatni]

here is a minimal reproducible example pls correct me if I made any mistake.

[Kludex]

Where?

[suniladityajatni]

sorry forgot to share the link here it is https://github.com/suniladityajatni/MemoryLeak

[curtiscook]

Old thread, but this is also how I solved my problem and it also gave me HTTP/2 support

[mntolia]

@suniladityajatni Thanks for the update. Why didn't you use gunicorn?

[evaldask]

It's been only a couple of days, but since upgrading to python 3.12 and latest library versions for fastapi, uvicorn, gunicorn seems like issue disappeared.

[danieltanfh95]

We got the issue for one of our internal services that has a few thousand requests per seconds.
Can confirm that moving to hypercorn solves this issue. We already use async def.

[edgarhuichen]

Sadly moving to hypercorn did not solve the same issue for me. The container memory usage still goes up slowly. I run the background task that executes some data processing each hour.

[Samdaso-o]

Hello, Is there any other updates on this issue?

[jgould22]

Honestly you are unlikely to see an update on this discussion, if it was an issue it would/should have been closed because it has sorta become a dumping ground for people who may or may not have the same problem.

There are tones of people in this thread all with

• Different python versions
• Different fastapi versions (some years old)
• Different pydantic versions

etc.

There appears to be one MRE from over a year ago which kludex couldn't reproduce and no one else seemed to try or comment on it.

Absent a solid MRE for the developers these things are very hard to debug and you are unlikely to see any change.

[Kludex]

Thanks for the consideration of not pinging me, I really appreciate it! ❤️

[curtiscook]

@tiangolo maybe you please close this due to age and in favor of a new MRE? Personally, the memory leak I experienced was resolved years ago.. after moving off of uvicorn

[Kludex]

Currently, there's no leak in uvicorn.

[Kludex]

I'll lock this.

Please create a new MRE on a new discussion (feel free to link this discussion on it). This is too old, and a lot has changed since it.