Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty blob access error using Java SDK #2458

Open
cnaj opened this issue Aug 30, 2024 · 7 comments · May be fixed by #2461 or #2462
Open

Empty blob access error using Java SDK #2458

cnaj opened this issue Aug 30, 2024 · 7 comments · May be fixed by #2461 or #2462
Assignees
Labels
alignment Alignment between Azurite with Azure Storage production blob-storage bug Something isn't working

Comments

@cnaj
Copy link

cnaj commented Aug 30, 2024

Which service(blob, file, queue, table) does this issue concern?

blob

Which version of the Azurite was used?

3.32.0

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

Docker

What's the Node.js version?

n/a

What problem was encountered?

Downloading an empty (0 bytes) blob from Azurite gives a 416 error response, while Azure works as expected. With version 3.31.0, both Azurite and Azure work as expected without an error.

This seems to contradict #2410, which reported the opposite behavior for an earlier Azurite version. As a consequence, #2417 was implemented.

Steps to reproduce the issue?

See https://github.com/cnaj/azurite-empty-blob-issue-demo for a reproduction of the issue.

Have you found a mitigation/solution?

no

@blueww
Copy link
Member

blueww commented Sep 2, 2024

@cnaj
For the download request, do you add header "x-ms-rage" with a range end bigger than the real blob size 0?

  • If so, this is expected to fail, and should also fail on server. (Please share the Azure server request trace if not fail on public Azure server)
  • If not, pleas share the Azurite debug log, and we need look into the debug log to continue the investigation.

@blueww blueww self-assigned this Sep 2, 2024
@blueww blueww added blob-storage bug Something isn't working labels Sep 2, 2024
@cnaj
Copy link
Author

cnaj commented Sep 2, 2024

Indeed, the Azure server responds with 416, too, but it was hidden by the Java Azure SDK. However, there seems to be a difference in the 416 response: The Azure server adds a "Content-Range: bytes */0" header in the response, which Azurite doesn't. Apparently, this triggers the Java SDK to retry the download with the correct range.

Logs for the public Azure server:

[Test worker] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP request","method":"GET","url":"https://xxxx.blob.core.windows.net/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","tryCount":1,"Date":"Mon, 02 Sep 2024 09:52:34 GMT","x-ms-version":"2024-08-04","x-ms-range":"bytes=0-4194303","x-ms-client-request-id":"b22d6b28-5c86-417e-8696-e3fdf9d931b9","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","redactedHeaders":"Authorization","content-length":0}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP response","statusCode":416,"url":"https://xxxx.blob.core.windows.net/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","durationMs":20,"content-length":249,"Date":"Mon, 02 Sep 2024 09:52:34 GMT","Server":"Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0","x-ms-version":"2024-08-04","Content-Range":"bytes */0","x-ms-error-code":"InvalidRange","Content-Type":"application/xml","x-ms-request-id":"4fedb6ba-101e-0000-291d-fdff97000000","x-ms-client-request-id":"b22d6b28-5c86-417e-8696-e3fdf9d931b9","content-length":249}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP request","method":"GET","url":"https://xxxx.blob.core.windows.net/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","tryCount":1,"Date":"Mon, 02 Sep 2024 09:52:34 GMT","x-ms-version":"2024-08-04","x-ms-range":"bytes=0--1","x-ms-client-request-id":"8ef21149-1b2b-46cc-a0e8-868a2ae5d5c3","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","redactedHeaders":"Authorization","content-length":0}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP response","statusCode":200,"url":"https://xxxx.blob.core.windows.net/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","durationMs":71,"content-length":0,"Date":"Mon, 02 Sep 2024 09:52:34 GMT","Server":"Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0","x-ms-lease-status":"unlocked","x-ms-version":"2024-08-04","x-ms-lease-state":"available","x-ms-blob-type":"BlockBlob","x-ms-server-encrypted":"true","Last-Modified":"Mon, 02 Sep 2024 09:52:34 GMT","Content-MD5":"1B2M2Y8AsgTpgAmY7PhCfg==","x-ms-creation-time":"Mon, 02 Sep 2024 09:52:34 GMT","ETag":"0x8DCCB34F85C7630","Content-Type":"application/octet-stream","Accept-Ranges":"bytes","x-ms-request-id":"c925b91e-b01e-0019-441d-fd7f2c000000","x-ms-client-request-id":"8ef21149-1b2b-46cc-a0e8-868a2ae5d5c3","content-length":0}

Logs for Azurite:

[Test worker] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP request","method":"GET","url":"http://127.0.0.1:10000/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","tryCount":1,"Date":"Mon, 02 Sep 2024 09:53:35 GMT","x-ms-version":"2024-08-04","x-ms-range":"bytes=0-4194303","x-ms-client-request-id":"9abe0988-16f7-45e9-b6a3-cee54657deea","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","redactedHeaders":"Authorization","content-length":0}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP response","statusCode":416,"url":"http://127.0.0.1:10000/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","durationMs":3,"Date":"Mon, 02 Sep 2024 09:53:35 GMT","Server":"Azurite-Blob/3.32.0","Transfer-Encoding":"chunked","Connection":"keep-alive","content-type":"application/xml","x-ms-error-code":"InvalidRange","x-ms-request-id":"387ca0c3-1fc3-46a9-91b8-8b48cc2441c5","redactedHeaders":"Keep-Alive"}

No further GET is being performed for Azurite by the SDK.

Logs were acquired by setting the environment variable AZURE_HTTP_LOG_DETAIL_LEVEL=HEADERS, as documented at https://learn.microsoft.com/en-us/azure/developer/java/sdk/logging-overview

@blueww blueww added the alignment Alignment between Azurite with Azure Storage production label Sep 3, 2024
@blueww
Copy link
Member

blueww commented Sep 3, 2024

@cnaj

A Fix PR is already raised, please check if it works for you.
#2461

@cnaj
Copy link
Author

cnaj commented Sep 3, 2024

Thanks for the PR! I've tested it with my Java reproduction test, but now the followup request generates a 500 error.

Test logs (as before):

[Test worker] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP request","method":"GET","url":"http://127.0.0.1:10000/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","tryCount":1,"Date":"Tue, 03 Sep 2024 10:46:02 GMT","x-ms-version":"2024-08-04","x-ms-range":"bytes=0-4194303","x-ms-client-request-id":"71c01aca-2873-4884-98d1-51e9b148fb5f","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","redactedHeaders":"Authorization","content-length":0}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP response","statusCode":416,"url":"http://127.0.0.1:10000/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","durationMs":12,"Date":"Tue, 03 Sep 2024 10:46:02 GMT","Server":"Azurite-Blob/3.32.0","Transfer-Encoding":"chunked","Content-Range":"bytes */0","Connection":"keep-alive","content-type":"application/xml","x-ms-error-code":"InvalidRange","x-ms-request-id":"0f138ca1-55d0-428f-b8f6-66783c4805a8","redactedHeaders":"Keep-Alive"}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP request","method":"GET","url":"http://127.0.0.1:10000/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","tryCount":1,"Date":"Tue, 03 Sep 2024 10:46:02 GMT","x-ms-version":"2024-08-04","x-ms-range":"bytes=0--1","x-ms-client-request-id":"feca516f-681d-4eed-acda-c8cd80cdab5f","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","redactedHeaders":"Authorization","content-length":0}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP response","statusCode":500,"url":"http://127.0.0.1:10000/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","durationMs":3,"content-length":0,"Date":"Tue, 03 Sep 2024 10:46:02 GMT","Server":"Azurite-Blob/3.32.0","Connection":"keep-alive","redactedHeaders":"Keep-Alive","content-length":0}

Azurite debug log:

2024-09-03T10:46:02.481Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: BlobStorageContextMiddleware: RequestMethod=GET RequestURL=http://127.0.0.1/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt RequestHeaders:{"host":"127.0.0.1:10000","date":"Tue, 03 Sep 2024 10:46:02 GMT","authorization":"SharedKey devstoreaccount1:F4noZ3BQJwqCiercZUEdxB4Pwui33GtaDnzlrdTI6/o=","x-ms-version":"2024-08-04","x-ms-range":"bytes=0-4194303","x-ms-client-request-id":"71c01aca-2873-4884-98d1-51e9b148fb5f","accept":"application/xml","user-agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","content-length":"0"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2024-09-03T10:46:02.482Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=test-42b8bbde-5e85-4359-9130-64822da82035 Blob=empty-file.txt
2024-09-03T10:46:02.482Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 verbose: DispatchMiddleware: Dispatching request...
2024-09-03T10:46:02.483Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: DispatchMiddleware: Operation=Blob_Download
2024-09-03T10:46:02.483Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2024-09-03T10:46:02.483Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: PublicAccessAuthenticator:validate() Start validation against public access.
2024-09-03T10:46:02.483Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 debug: PublicAccessAuthenticator:validate() Getting account properties...
2024-09-03T10:46:02.483Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: test-42b8bbde-5e85-4359-9130-64822da82035, blob: empty-file.txt
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container test-42b8bbde-5e85-4359-9130-64822da82035
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"GET\n\n\n\n\n\nTue, 03 Sep 2024 10:46:02 GMT\n\n\n\n\n\nx-ms-client-request-id:71c01aca-2873-4884-98d1-51e9b148fb5f\nx-ms-range:bytes=0-4194303\nx-ms-version:2024-08-04\n/devstoreaccount1/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt"
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:F4noZ3BQJwqCiercZUEdxB4Pwui33GtaDnzlrdTI6/o=
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 verbose: DeserializerMiddleware: Start deserializing...
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: HandlerMiddleware: DeserializedParameters={"options":{"range":"bytes=0-4194303","requestId":"71c01aca-2873-4884-98d1-51e9b148fb5f","leaseAccessConditions":{},"cpkInfo":{},"modifiedAccessConditions":{}},"version":"2024-08-04"}
2024-09-03T10:46:02.486Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: ErrorName=StorageError ErrorMessage=The range specified is invalid for the current size of the resource.  ErrorHTTPStatusCode=416 ErrorHTTPStatusMessage=The range specified is invalid for the current size of the resource. ErrorHTTPHeaders={"x-ms-error-code":"InvalidRange","x-ms-request-id":"0f138ca1-55d0-428f-b8f6-66783c4805a8","Content-Range":"bytes */0"} ErrorHTTPBody="<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>InvalidRange</Code>\n  <Message>The range specified is invalid for the current size of the resource.\nRequestId:0f138ca1-55d0-428f-b8f6-66783c4805a8\nTime:2024-09-03T10:46:02.486Z</Message>\n</Error>" ErrorStack="StorageError: The range specified is invalid for the current size of the resource.\n    at StorageErrorFactory.getInvalidPageRange2 (/Users/cernaj/github/Azure/Azurite/dist/src/blob/errors/StorageErrorFactory.js:79:27)\n    at BlobHandler.downloadBlockBlobOrAppendBlob (/Users/cernaj/github/Azure/Azurite/dist/src/blob/handlers/BlobHandler.js:649:53)\n    at BlobHandler.download (/Users/cernaj/github/Azure/Azurite/dist/src/blob/handlers/BlobHandler.js:45:25)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set HTTP code: 416
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set HTTP status message: The range specified is invalid for the current size of the resource.
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set HTTP Header: x-ms-error-code=InvalidRange
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set HTTP Header: x-ms-request-id=0f138ca1-55d0-428f-b8f6-66783c4805a8
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set HTTP Header: Content-Range=bytes */0
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set content type: application/xml
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set HTTP body: "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>InvalidRange</Code>\n  <Message>The range specified is invalid for the current size of the resource.\nRequestId:0f138ca1-55d0-428f-b8f6-66783c4805a8\nTime:2024-09-03T10:46:02.486Z</Message>\n</Error>"
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: EndMiddleware: End response. TotalTimeInMS=6 StatusCode=416 StatusMessage=The range specified is invalid for the current size of the resource. Headers={"server":"Azurite-Blob/3.32.0","x-ms-error-code":"InvalidRange","x-ms-request-id":"0f138ca1-55d0-428f-b8f6-66783c4805a8","content-range":"bytes */0","content-type":"application/xml"}
2024-09-03T10:46:02.492Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: BlobStorageContextMiddleware: RequestMethod=GET RequestURL=http://127.0.0.1/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt RequestHeaders:{"host":"127.0.0.1:10000","date":"Tue, 03 Sep 2024 10:46:02 GMT","authorization":"SharedKey devstoreaccount1:PWXMvUmglG/3azeGiPoTLvQIuiXesM4WNIYPFhNVo3o=","x-ms-version":"2024-08-04","x-ms-range":"bytes=0--1","x-ms-client-request-id":"feca516f-681d-4eed-acda-c8cd80cdab5f","accept":"application/xml","user-agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","content-length":"0"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2024-09-03T10:46:02.492Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=test-42b8bbde-5e85-4359-9130-64822da82035 Blob=empty-file.txt
2024-09-03T10:46:02.492Z dbf3ca32-86b7-4a6b-8228-21bab486b27c verbose: DispatchMiddleware: Dispatching request...
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: DispatchMiddleware: Operation=Blob_Download
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: PublicAccessAuthenticator:validate() Start validation against public access.
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c debug: PublicAccessAuthenticator:validate() Getting account properties...
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: test-42b8bbde-5e85-4359-9130-64822da82035, blob: empty-file.txt
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container test-42b8bbde-5e85-4359-9130-64822da82035
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"GET\n\n\n\n\n\nTue, 03 Sep 2024 10:46:02 GMT\n\n\n\n\n\nx-ms-client-request-id:feca516f-681d-4eed-acda-c8cd80cdab5f\nx-ms-range:bytes=0--1\nx-ms-version:2024-08-04\n/devstoreaccount1/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt"
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:PWXMvUmglG/3azeGiPoTLvQIuiXesM4WNIYPFhNVo3o=
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c verbose: DeserializerMiddleware: Start deserializing...
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: HandlerMiddleware: DeserializedParameters={"options":{"range":"bytes=0--1","requestId":"feca516f-681d-4eed-acda-c8cd80cdab5f","leaseAccessConditions":{},"cpkInfo":{},"modifiedAccessConditions":{}},"version":"2024-08-04"}
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c error: ErrorMiddleware: Received an error, fill error information to HTTP response
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c error: ErrorMiddleware: ErrorName=RangeError ErrorMessage=deserializeRangeHeader: raw range value bytes=0--1 is wrong. ErrorStack="RangeError: deserializeRangeHeader: raw range value bytes=0--1 is wrong.\n    at deserializeRangeHeader (/Users/cernaj/github/Azure/Azurite/dist/src/blob/utils/utils.js:46:15)\n    at BlobHandler.downloadBlockBlobOrAppendBlob (/Users/cernaj/github/Azure/Azurite/dist/src/blob/handlers/BlobHandler.js:638:64)\n    at BlobHandler.download (/Users/cernaj/github/Azure/Azurite/dist/src/blob/handlers/BlobHandler.js:45:25)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c error: ErrorMiddleware: Set HTTP code: 500
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: EndMiddleware: End response. TotalTimeInMS=1 StatusCode=500 StatusMessage=undefined Headers={"server":"Azurite-Blob/3.32.0"}

Looks like the Azure SDK sends x-ms-range: bytes=0--1, which I would interpret as "from 0 to -1", i.e. until the end. Azure responds with 200 (see previous my comment).

@blueww
Copy link
Member

blueww commented Sep 4, 2024

@cnaj

I tried to search x-ms-range: bytes=0--1 expected behavior from rest API doc, but not find anything.
https://learn.microsoft.com/en-us/rest/api/storageservices/get-blob?tabs=microsoft-entra-id
https://learn.microsoft.com/en-us/rest/api/storageservices/specifying-the-range-header-for-blob-service-operations

I also tried to send it with storage .net SDK, and get error "Specified argument was out of the range of valid values." from SDK side.

x-ms-range: bytes=0--1 looks is a none documented behavior.
Azurite is based on rest API doc/swagger, we will try to get aligned behavior as Azure server, but cover all the corner cases is not our first priority.

Azurite welcome contribution!
It would be great if you could raise a PR to fix this issue!

The change should be in the function:

export function deserializeRangeHeader(

Please make sure change log is updated, and enough test coverage in the PR.

@cnaj
Copy link
Author

cnaj commented Sep 4, 2024

So it's actually the Java SDK's fault because it sends an invalid header.

Nevertheless, it looks like Azure chooses to ignore the Range header (see https://httpwg.org/specs/rfc9110.html#field.range), either because of invalid range or because the content is empty.

From the RFC:

A server that supports range requests MAY ignore or reject a Range header field that contains an invalid ranges-specifier (...)

and

A server that supports range requests MAY ignore a Range header field when the selected representation has no content (i.e., the selected representation's data is of zero length).

We can see this from the fact that Azure reponds with 200 OK, and not with 206 Partial Content.

In this case, I think that Azurite should implement the same behavior, and not accept the invalid range request, which would also fix my use case (and be usable with the Java SDK). I'd be glad to help (time permitting), but TypeScript is not my main strength...

@cnaj cnaj changed the title Empty blob access error Empty blob access error using Java SDK Sep 4, 2024
@blueww
Copy link
Member

blueww commented Sep 4, 2024

So it's actually the Java SDK's fault because it sends an invalid header.

Nevertheless, it looks like Azure chooses to ignore the Range header (see https://httpwg.org/specs/rfc9110.html#field.range), either because of invalid range or because the content is empty.

From the RFC:

A server that supports range requests MAY ignore or reject a Range header field that contains an invalid ranges-specifier (...)

and

A server that supports range requests MAY ignore a Range header field when the selected representation has no content (i.e., the selected representation's data is of zero length).

We can see this from the fact that Azure reponds with 200 OK, and not with 206 Partial Content.

In this case, I think that Azurite should implement the same behavior, and not accept the invalid range request, which would also fix my use case (and be usable with the Java SDK). I'd be glad to help (time permitting), but TypeScript is not my main strength...

@cnaj

Thanks for the reply!
As you indicate,
A server that supports range requests MAY ignore or reject a Range header field that contains an invalid ranges-specifier (...)

Current Azurite reject the invalid range, this is also aligned with RFC. However, Azure server choose ignore it, which is also aligned with RFC.
As this is a corner case, we won't take it as our recent priority.
Looking forward for your fixing PR.

@cnaj cnaj linked a pull request Sep 4, 2024 that will close this issue
cb-robot pushed a commit to couchbase/asterixdb that referenced this issue Nov 30, 2024
- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Opening an input stream on empty blobs in azurite throws a null
  pointer exception in versions > 3.31.0.
  Azure/Azurite#2458
- Use a less conflicting port for gcs tests

Change-Id: Ia6dff5172b91f0d0e8a26c0e69aceff3b74896c2
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19147
Integration-Tests: Jenkins <[email protected]>
Tested-by: Jenkins <[email protected]>
Reviewed-by: Murtadha Hubail <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alignment Alignment between Azurite with Azure Storage production blob-storage bug Something isn't working
Projects
None yet
2 participants