Panic while attempting to authenticate with Azure Bearer token

[Rob-DL]

I'm trying to access a delta lake that's on an Azure Blob store. For security reasons the only authentication method I can use is the bearer token. I'm trying to read the table using the following python code.

dt = DeltaTable(
    "az://mycontainer/path_to_table",
    storage_options={
        "AZURE_STORAGE_ACCOUNT_NAME": "account-name",
        "ACCOUNT_NAME": "account-name",
        "BEARER_TOKEN": "eyJ0eXA...",
    },
)

But when I run this I'm greeted with the following panic from the underlaying rust code:

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: InvalidByte(132, 46)', C:\Users\robdl\.cargo\registry\src\github.com-1ecc6299db9ec823\object_store-0.5.3\src\azure\credential.rs:156:48
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last):
  File "C:\Users\rolaat\test\deltars.py", line 16, in <module>
    dt = DeltaTable(
  File "C:\Users\rolaat\.venv\tmpvenv\lib\site-packages\deltalake\table.py", line 121, in __init__
    self._table = RawDeltaTable(
pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: InvalidByte(132, 46)

The problem seems to be with that the token isn't valid base64, which is technically true as these Azure generated tokens aren't padded correctly. But even after correcting for this I keep getting the same error. I generated several tokens trough the azure cli using the command below, but they all result in the same error.

az account get-access-token --resource https://myproject.blob.core.windows

Is this a bug, or am I misunderstanding the BEARER_TOKEN authentication option of delta-rs? I couldn't find anything in the documentation. Perhaps somebody could add more examples on how to properly use this?

[roeap]

Hi @Rob-DL,

Just checked it out in the underlying object store crate. Turns out there is a bug, and it tries to use the token as an access key to create a signature, which is not the intended usage.

Good news is that's a straight forward fix and I hope to contribute some other things. there soon as well and will also take care of that fix then.

Do you mind opening a issue in this repo here to keep track of it?

[Rob-DL]

Thans for checking, i've opened the following issue: #1117