Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.4.0 #16

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
f571796
Implement save stream IO (no-firefox)
elliotsayes Feb 16, 2023
6f43f7b
Add support for aborting on failed verification
elliotsayes Feb 16, 2023
dedfa0e
Release writer lock
elliotsayes Feb 16, 2023
72957fc
Remove close on writer
elliotsayes Feb 16, 2023
52a213c
Implement download using StreamSaver.js
elliotsayes Feb 23, 2023
19e6be3
Fix improper closing
elliotsayes Feb 23, 2023
6b49c83
Wait for readyFuture again
elliotsayes Feb 23, 2023
5ce5a5a
Switch to DartIOFileSaver for saveStream:
elliotsayes Feb 23, 2023
870cdae
Add code to find empty files
elliotsayes Feb 24, 2023
d844bfc
ArDriveDownloader uses same download filename...
elliotsayes Feb 24, 2023
cf46631
Better handling of no content-type
elliotsayes Feb 24, 2023
b3a491b
Fix basename bug (oops)
elliotsayes Feb 24, 2023
41e1ad7
Fix extension ..
elliotsayes Feb 24, 2023
9a8d652
Fix wrong arg order (oops)
elliotsayes Feb 24, 2023
21f5a45
Change saveFileStream API to completer
elliotsayes Feb 27, 2023
03482c6
Better abort logic
elliotsayes Feb 27, 2023
3698836
Implement save status stream
elliotsayes Feb 28, 2023
2ae9de2
Rename flag to saveResult
elliotsayes Feb 28, 2023
8a82370
Remove commented code
elliotsayes Mar 22, 2023
d2f71fa
More descriptive exceptions
elliotsayes Mar 22, 2023
2366575
Move empty file finder to path_utils
elliotsayes Mar 22, 2023
9de8d78
Make "test" var name more explicit
elliotsayes Mar 22, 2023
229147e
Clearer name: emptyFile => nonexistentFile
elliotsayes Apr 5, 2023
2631a98
variable name `extension` => `fileExtension`
elliotsayes Apr 5, 2023
dde4cd0
More clear exit condition on while loop
elliotsayes Apr 5, 2023
37294df
Add TODO to catch near-infinite loops
elliotsayes Apr 5, 2023
46f2c11
Work around weird mime behavior?
elliotsayes Apr 5, 2023
5df5b19
Fixed comparison to use lowercase
elliotsayes Apr 6, 2023
abd6e86
feat(save files on app dir)
thiagocarvalhodev Aug 23, 2023
295b3ae
Update fvm_config.json
thiagocarvalhodev Aug 23, 2023
a07b81c
Update pubspec.yaml
thiagocarvalhodev Aug 23, 2023
49ef040
Update pubspec.yaml
thiagocarvalhodev Aug 23, 2023
136d27a
Update web_io.dart
thiagocarvalhodev Aug 23, 2023
3a60e46
feat(IOFolder)
thiagocarvalhodev Sep 26, 2023
080f74c
feat(uploader)
thiagocarvalhodev Oct 3, 2023
3ca5ffd
Update io_folder_test.dart
thiagocarvalhodev Oct 3, 2023
9f0c97d
feat(iofolder)
thiagocarvalhodev Oct 4, 2023
2359255
upgrade deps
thiagocarvalhodev Oct 4, 2023
daef5c6
remove folder tree generation of files
thiagocarvalhodev Oct 4, 2023
7bd6e71
fix lint rules
thiagocarvalhodev Oct 4, 2023
599780a
Merge pull request #10 from elliotsayes/write-file-stream
thiagocarvalhodev Oct 6, 2023
3ce83fd
Merge branch 'dev' into PE-3699-uploads-large-files-for-public-drives
thiagocarvalhodev Oct 6, 2023
5998dfd
add stream saver
thiagocarvalhodev Oct 6, 2023
b42e3dc
Update web_io.dart
thiagocarvalhodev Oct 25, 2023
8c6ac43
Update web_io.dart
thiagocarvalhodev Oct 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .fvm/fvm_config.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"flutterSdkVersion": "3.10.0",
"flutterSdkVersion": "3.10.2",
"flavors": {}
}
1 change: 1 addition & 0 deletions lib/ardrive_io.dart
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ export 'src/io_entity.dart';
export 'src/io_exception.dart';
export 'src/io_file.dart';
export 'src/io_folder.dart';
export 'src/mobile/mobile_io.dart';
export 'src/utils/mime_type_utils.dart';
export 'src/utils/path_utils.dart';
export 'src/utils/permissions.dart';
20 changes: 12 additions & 8 deletions lib/src/ardrive_downloader.dart
Original file line number Diff line number Diff line change
Expand Up @@ -15,20 +15,24 @@ import 'package:flutter_downloader/flutter_downloader.dart';
/// running at the time, it will only cancel the current one.
///
// TODO: Add an interface for this class and implement a Web and mobile specific downloader
class ArDriveDownloader {
class ArDriveMobileDownloader {
late String _currentTaskId;

String get currentTaskId => _currentTaskId;

Stream<int> downloadFile(String downloadUrl, String fileName) async* {
Stream<int> downloadFile(
String downloadUrl, String fileName, String? contentType) async* {
await requestPermissions();
await verifyPermissions();

final downloadDir = await getDefaultMobileDownloadDir();
final saveFileName =
await nonexistentFileName(downloadDir, fileName, contentType);

final taskId = await FlutterDownloader.enqueue(
url: downloadUrl,
savedDir: downloadDir,
fileName: fileName,
fileName: saveFileName,
saveInPublicStorage: true,
showNotification: true,
openFileFromNotification: false,
Expand All @@ -46,24 +50,24 @@ class ArDriveDownloader {
}

Stream<int> _handleProgress(String taskId) {
ReceivePort _port = ReceivePort();
ReceivePort port = ReceivePort();
StreamController<int> controller = StreamController<int>();

/// Remove previous port and track only one download
IsolateNameServer.removePortNameMapping('downloader_send_port');

IsolateNameServer.registerPortWithName(
_port.sendPort, 'downloader_send_port');
port.sendPort, 'downloader_send_port');

_port.listen((dynamic data) {
DownloadTaskStatus status = DownloadTaskStatus(data[1]);
port.listen((dynamic data) {
DownloadTaskStatus status = DownloadTaskStatus.values[data[1]];
int progress = data[2];

/// only track the progress of current task id
if (status == DownloadTaskStatus.enqueued) {
controller.sink.add(0);
} else if (status == DownloadTaskStatus.running) {
debugPrint('Download progress: ' + progress.toString());
debugPrint('Download progress: $progress');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we get rid of this log?

controller.sink.add(progress);
} else if (status == DownloadTaskStatus.complete) {
controller.sink.add(100);
Expand Down
14 changes: 14 additions & 0 deletions lib/src/ardrive_io.dart
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,18 @@ import 'mobile/mobile_io.dart';
import 'web/stub_web_io.dart' // Stub implementation
if (dart.library.html) 'web/web_io.dart';

class SaveStatus {
final int bytesSaved;
final int totalBytes;
final bool? saveResult;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the null value have a specific meaning here? Is it different than defaulting to false?

Edit - Having it be null means the download hasn't yet finished.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can rename it to hasFinished or isResultSaved, or similar.


SaveStatus({
required this.bytesSaved,
required this.totalBytes,
this.saveResult,
});
}

/// API for I/O operations
///
/// Opens the platform specific file picker to pick files and folders, and save files using
Expand Down Expand Up @@ -36,4 +48,6 @@ abstract class ArDriveIO {
Future<IOFolder> pickFolder();

Future<void> saveFile(IOFile file);

Stream<SaveStatus> saveFileStream(IOFile file, Completer<bool> finalize);
}
10 changes: 5 additions & 5 deletions lib/src/cache_storage.dart
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,19 @@ class IOCacheStorage {

debugPrint('saving file on local storage');

final _file = File('${cacheDir.path}/${entity.name}');
final file = File('${cacheDir.path}/${entity.name}');

final readStream = File(entity.path).openRead();

final writeStream = await _file.open(mode: FileMode.write);
final writeStream = await file.open(mode: FileMode.write);

await for (List<int> chunk in readStream) {
await writeStream.writeFrom(chunk);
}

await writeStream.close();

return _file.path;
return file.path;
}

Future<IOFile> getFileFromStorage(String fileName) async {
Expand All @@ -32,9 +32,9 @@ class IOCacheStorage {

final cacheDir = await _getCacheDir();

final _file = File('${cacheDir.path}/$fileName');
final file = File('${cacheDir.path}/$fileName');

return adapter.fromFile(_file);
return adapter.fromFile(file);
}

Future<void> freeLocalStorage() async {
Expand Down
3 changes: 2 additions & 1 deletion lib/src/file_provider.dart
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ abstract class FolderProvider {
}

/// Provides multiple `IOFiles` or an `IOFolder`
abstract class MultiFileProvider extends FileProvider with FolderProvider {
abstract class MultiFileProvider extends FileProvider
implements FolderProvider {
Future<List<IOFile>> pickMultipleFiles({
List<String>? allowedExtensions,
required FileSource fileSource,
Expand Down
2 changes: 2 additions & 0 deletions lib/src/io_exception.dart
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ class FileSystemPermissionDeniedException extends IOException {
List<Permission> permissionsDenied;
}

class FileReadWritePermissionDeniedException extends IOException {}

class EntityPathException extends IOException {}

class UnsupportedPlatformException extends IOException {
Expand Down
6 changes: 3 additions & 3 deletions lib/src/io_folder.dart
Original file line number Diff line number Diff line change
Expand Up @@ -70,13 +70,13 @@ class _FileSystemFolder extends IOFolder {
/// `_mountFolderChildren` mounts recursiverly the folder hierarchy. It gets only
/// the current level entities loading only `IOFile` and `IOFolder`
Future<List<IOEntity>> _mountFolderStructure() async {
List<IOEntity> _children = [];
List<IOEntity> children = [];

for (var fs in _folderContent) {
_children.add(await _addFolderNode(fs));
children.add(await _addFolderNode(fs));
}

return _children;
return children;
}

Future<IOEntity> _addFolderNode(FileSystemEntity fsEntity) async {
Expand Down
126 changes: 119 additions & 7 deletions lib/src/mobile/mobile_io.dart
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
import 'dart:async';
import 'dart:io';

import 'package:ardrive_io/ardrive_io.dart';
import 'package:ardrive_io/src/utils/completer.dart';
import 'package:file_saver/file_saver.dart' as file_saver;
import 'package:flutter/foundation.dart';
import 'package:mime/mime.dart' as mime;
import 'package:path/path.dart' as p;

Expand Down Expand Up @@ -64,9 +67,19 @@ class MobileIO implements ArDriveIO {
}

@override
Future<void> saveFile(IOFile file) async {
Future<void> saveFile(IOFile file, [bool saveOnAppDirectory = false]) async {
try {
await _fileSaver.save(file);
await _fileSaver.save(file, saveOnAppDirectory: saveOnAppDirectory);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can get rid of this try..catch because it's rethrowing and nothing else.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment for the method below.

} catch (e) {
rethrow;
}
}

@override
Stream<SaveStatus> saveFileStream(
IOFile file, Completer<bool> finalize) async* {
try {
yield* _fileSaver.saveStream(file, finalize);
} catch (e) {
rethrow;
}
Expand All @@ -78,12 +91,24 @@ class MobileIO implements ArDriveIO {
/// This implementation uses the `file_saver` package.
///
/// Throws an `FileSystemPermissionDeniedException` when user deny access to storage
///
/// `saveOnAppDirectory` is not supported on this implementation
class MobileSelectableFolderFileSaver implements FileSaver {
final DartIOFileSaver _dartIOFileSaver;

MobileSelectableFolderFileSaver({DartIOFileSaver? dartIOFileSaver})
: _dartIOFileSaver = dartIOFileSaver ?? DartIOFileSaver();

@override
Future<void> save(IOFile file) async {
Future<void> save(IOFile file, {bool saveOnAppDirectory = false}) async {
await requestPermissions();
await verifyPermissions();

if (saveOnAppDirectory) {
await _dartIOFileSaver.save(file, saveOnAppDirectory: saveOnAppDirectory);
return;
}

await file_saver.FileSaver.instance.saveAs(
name: file.name,
bytes: await file.readAsBytes(),
Expand All @@ -92,13 +117,19 @@ class MobileSelectableFolderFileSaver implements FileSaver {

return;
}

@override
Stream<SaveStatus> saveStream(IOFile file, Completer<bool> finalize) {
// file_saver doesn't seem to support support saving streams
throw UnimplementedError();
}
}

/// Saves a file using the `dart:io` library.
/// It will save on `getDefaultMobileDownloadDir()`
class DartIOFileSaver implements FileSaver {
@override
Future<void> save(IOFile file) async {
Future<void> save(IOFile file, {bool saveOnAppDirectory = false}) async {
await requestPermissions();
await verifyPermissions();

Expand All @@ -111,24 +142,105 @@ class DartIOFileSaver implements FileSaver {
fileName += '.$fileExtension';
}

if (saveOnAppDirectory) {
await _saveOnAppDir(file, fileName);
return;
}

await _saveOnDownloadsDir(file, fileName);
}

Future<void> _saveOnDownloadsDir(IOFile file, String fileName) async {
/// platform_specific_path/Downloads/
final defaultDownloadDir = await getDefaultMobileDownloadDir();

final newFile = File(defaultDownloadDir + fileName);
final newFile = await nonexistentFile(defaultDownloadDir, file);

await newFile.writeAsBytes(await file.readAsBytes());
}

Future<void> _saveOnAppDir(IOFile file, String fileName) async {
final appDir = await getDefaultAppDir();

final newFile = File(appDir + fileName);

await newFile.writeAsBytes(await file.readAsBytes());
}

@override
Stream<SaveStatus> saveStream(IOFile file, Completer<bool> finalize) async* {
var bytesSaved = 0;
final totalBytes = await file.length;
yield SaveStatus(
bytesSaved: bytesSaved,
totalBytes: totalBytes,
);

try {
await requestPermissions();
await verifyPermissions();

/// platform_specific_path/Downloads/
final defaultDownloadDir = await getDefaultMobileDownloadDir();

final newFile = await nonexistentFile(defaultDownloadDir, file);

final sink = newFile.openWrite();

// NOTE: This is an alternative to `addStream` with lower level control
const flushThresholdBytes = 50 * 1024 * 1024; // 50 MiB
var unflushedDataBytes = 0;
await for (final chunk in file.openReadStream()) {
if (await completerMaybe(finalize) == false) break;

sink.add(chunk);
unflushedDataBytes += chunk.length;
if (unflushedDataBytes > flushThresholdBytes) {
await sink.flush();
unflushedDataBytes = 0;
}

bytesSaved += chunk.length;
yield SaveStatus(
bytesSaved: bytesSaved,
totalBytes: totalBytes,
);
}
await sink.flush();
await sink.close();

final finalizeResult = await finalize.future;
if (!finalizeResult) {
debugPrint('Cancelling saveStream...');
await newFile.delete();
}

yield SaveStatus(
bytesSaved: bytesSaved,
totalBytes: totalBytes,
saveResult: finalizeResult,
);
} catch (e) {
yield SaveStatus(
bytesSaved: bytesSaved,
totalBytes: totalBytes,
saveResult: false,
);
}
}
}

/// Defines the API for saving `IOFile` on Storage
abstract class FileSaver {
factory FileSaver() {
if (Platform.isAndroid || Platform.isIOS) {
return MobileSelectableFolderFileSaver();
return DartIOFileSaver();
}
throw UnsupportedPlatformException(
'The ${Platform.operatingSystem} platform is not supported');
}

Future<void> save(IOFile file);
Future<void> save(IOFile file, {bool saveOnAppDirectory = false});

Stream<SaveStatus> saveStream(IOFile file, Completer<bool> finalize);
}
6 changes: 6 additions & 0 deletions lib/src/utils/completer.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
import 'dart:async';

Future<T?> completerMaybe<T>(Completer<T> completer) =>
completer.isCompleted
? completer.future
: Future.value(null);
Loading