Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix hanging upload of large files #3489

Merged
merged 6 commits into from
Sep 11, 2024
Merged

Fix hanging upload of large files #3489

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
a34d50b
make method async
noamblitz Sep 9, 2024
e094a76
Merge branch 'main' into fix/hanging-bytes
dekkers Sep 9, 2024
d4b771f
Run sync code in threadpool and add integration test for big file
dekkers Sep 9, 2024
1ef26e1
Run poetry
dekkers Sep 9, 2024
f6a8675
Merge branch 'main' into fix/hanging-bytes
dekkers Sep 11, 2024
d5e3edb
Merge branch 'main' into fix/hanging-bytes
underdarknl Sep 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions bytes/bytes/api/router.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
from uuid import UUID

import structlog
from asgiref.sync import async_to_sync
from cachetools import TTLCache, cached
from fastapi import APIRouter, Depends, HTTPException, Query, Request
from fastapi.responses import Response
Expand Down Expand Up @@ -147,7 +146,7 @@ def get_normalizer_meta(


@router.post("/raw", tags=[RAW_TAG])
def create_raw(
async def create_raw(
request: Request,
boefje_meta_id: UUID,
mime_types: list[str] | None = Query(None),
Expand All @@ -162,10 +161,8 @@ def create_raw(
if meta_repository.has_raw(meta, parsed_mime_types):
return RawResponse(status="success", message="Raw data already present")

# FastAPI/starlette only has async versions of the Request methods, but
# all our code is sync, so we wrap it in async_to_sync.
data = async_to_sync(request.body)()

# Await the request body directly
data = await request.body()
raw_data = RawData(value=data, boefje_meta=meta, mime_types=parsed_mime_types)
with meta_repository:
raw_id = meta_repository.save_raw(raw_data)
Expand Down