Skip to content

Commit

Permalink
#111 Created a bridge between the Pathlike and the Bucket API (#112)
Browse files Browse the repository at this point in the history
* #111 Created a bridge between the Pathlike and SaaS File API

* #111 Mentioned the bridge between the Pathlike and SaaS File API in the changelog.

* #111 Addressing linting errors.

* #111 Fixed import path in unit tests.

* #111 Made the implementation of the Pathlike generic

* #111 Made the implementation of the Pathlike generic
  • Loading branch information
ahsimb authored Apr 17, 2024
1 parent 31deeba commit 631ed3f
Show file tree
Hide file tree
Showing 6 changed files with 737 additions and 148 deletions.
6 changes: 4 additions & 2 deletions doc/changes/unreleased.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,11 @@
logging.basicConfig(level=logging.INFO)
```

- Support for viewing BucketFS as a directory
- Support for viewing BucketFS as a directory

Added the Pathlike protocol as described in the [design document](../design/bucketpath.rst).
Added the PathLike protocol as described in the [design document](../design/bucketpath.rst).
Extracted bucket interface into BucketLike protocol.
Implemented PathLike for buckets based on BucketLike protocol.


## Internal
Expand Down
82 changes: 82 additions & 0 deletions exasol/bucketfs/_buckets.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
ByteString,
Iterable,
Iterator,
Protocol,
)

import requests
Expand All @@ -20,7 +21,88 @@
)


class BucketLike(Protocol):
"""
Definition of the Bucket interface.
It is compatible with both on-premises an SaaS BucketFS systems.
"""

@property
def files(self) -> Iterable[str]:
"""
Returns an iterator over the bucket files.
A usage example:
print(list(bucket_api.files))
output:
[dir1/subdir1/file1.dat, dir1/subdir2/file2.txt, ....]
Note that the paths will look like in the example above, i.e. POSIX style,
no backslash at the start or at the end.
"""

def delete(self, path: str) -> None:
"""
Deletes a file in the bucket.
:param path: Path of the file to be deleted.
Q. What happens if the path doesn't exist?
A. It does nothing, no error.
Q. What happens if the path points to a directory?
A. Same. There are no directories as such in the BucketFS, hence
a directory path is just a non-existent file.
"""

def upload(self, path: str, data: ByteString | BinaryIO) -> None:
"""
Uploads a file to the bucket.
:param path: Path in the bucket where the file should be uploaded.
:param data: Either a binary array or a binary stream, e.g. a file opened in the binary mode.
Q. What happens if the parent is missing?
A. The bucket doesn't care about the structure of the file's path. Looking from the prospective
of a file system, the bucket will create the missing parent, but in reality it will just
store the data indexed by the provided path.
Q. What happens if the path points to an existing file?
A. That's fine, the file will be updated.
Q. What happens if the path points to an existing directory?
A. The bucket doesn't care about the structure of the file's path. Looking from the prospective
of a file system, there will exist a file and directory with the same name.
Q. How should the path look like?
A. It should look like a POSIX path, but it should not contain any of the NTFS invalid characters.
It can have the leading and/or ending backslashes, which will be subsequently removed.
If the path doesn't conform to this format an BucketFsError will be raised.
"""

def download(self, path: str, chunk_size: int = 8192) -> Iterable[ByteString]:
"""
Downloads a file from the bucket. The content of the file will be provided
in chunks of the specified size. The full content of the file can be constructed using
code similar to the line below.
content = b''.join(api.download_file(path))
:param path: Path of the file in the bucket that should be downloaded.
:param chunk_size: Size of the chunks the file content will be delivered in.
Q. What happens if the file specified by the path doesn't exist.
A. BucketFsError will be raised.
Q. What happens if the path points to a directory.
A. Same, since a "directory" in the BucketFS is just a non-existent file.
"""


class Bucket:
"""
Implementation of the On-Premises bucket.
"""

def __init__(
self,
name: str,
Expand Down
Loading

0 comments on commit 631ed3f

Please sign in to comment.