My application has a function that allows to upload videos and other large files to Amazon S3 buckets via Amplify storage. The below function works as expected and uploads a file to my S3 bucket:
Future<String> uploadFile({File? file, String path = "", String fileName = "", String fileExtension = ""}) async {
try {
final key = path+fileName+'.'+fileExtension;
final result = await Amplify.Storage.uploadFile(
local: file!,
key: key,
);
return result.key;
} catch (e) {
throw e;
} }
However, given the size of some of these files, the time required to upload can be large and I want to avoid that my users have to wait until the file is completely uploaded. Therefore I want to run the above function in a dart Isolate, so that the file can continue uploading while my user Navigates to different screens of the app.
I tried to achieve this with the below function:
static uploadFileIsolate(List<dynamic> args) async {
try {
SendPort responsePort = args[0];
File file = args[1];
String path = args[2];
String fileName = args[3];
String fileExtension = args[4];
final key = path+fileName+'.'+fileExtension;
final result = await Amplify.Storage.uploadFile(
local: file,
key: key,
);
Isolate.exit(responsePort, result);
} catch (e) {
throw e;
} }
Which I call like this from my main function:
final p = ReceivePort();
Isolate.spawn(uploadFileIsolate, [p.sendPort, file, path, fileName, fileExtension]);
This is not working and throws the below error:
[VERBOSE-2:dart_isolate.cc(1111)] Unhandled exception:
RangeError (index): Invalid value: Valid value range is empty: 0
#0 DatabaseService.uploadFileIsolate (package:mastory/services/database_service.dart:202:7)
#1 _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:300:17)
#2 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)
After reading this issue it makes me think that the error comes from the fact that Amplify is not configured in the 'isolated' environment.
So my question is: How can I make Amplify's uploadFile function work in an Isolate?
Or, if that is not possible, how can I make sure that the uploadFile function continues until completion while allowing my user to Navigate to other screens in my app?
Related
I'm running a web server instance in my Flutter app in order to run an image conversion tool, written in JavaScript. The JS code sends a POST command along with a body which contains the raw image data.
Using httpServer and VirtualDirectory, I'm serving all of the required files if JS calls GET, and now when it calls the POST command, I need to convert the POST request to raw data and save it to a file, as it contains the image data I need.
The current web server logic is written in Python. It uses rfile.read in order to write the data from the request into a file.
contentLength = int(self.headers['Content-Length'])
open("output_files/output.%03d.jpg"%frame,"wb").write(self.rfile.read(contentLength))
This is what I'm trying to recreate in Flutter. Here's my code so far.
_startServer({required String basePath}) async {
var server = await HttpServer.bind(InternetAddress.loopbackIPv4, 8080);
virDir = VirtualDirectory('$tempDir/converter/')
..allowDirectoryListing = true;
debugPrint(
"Server running on IP : ${server.address} On Port : ${server.port}");
await for (var request in server) {
switch (request.method) {
case 'GET':
String path = request.uri.toString();
if (!path.contains(basePath)) {
path = basePath + request.uri.toString();
}
debugPrint('request uri: $path');
final File file = File(path);
if (await file.exists()) {
request.response.statusCode = HttpStatus.ok;
virDir?.serveFile(file, request);
} else {
debugPrint('Could not find: $file');
request.response.statusCode = HttpStatus.notFound;
}
break;
case 'POST':
debugPrint('Content Length: ${request.headers.contentLength}');
debugPrint('Content Type: ${request.headers.contentType}');
final File image =
File('$basePath/videos/frame.${intFixed(frame, 3)}.jpg');
if (!image.existsSync()) {
request.response.statusCode = HttpStatus.ok;
request.response.write('Finished');
request.response.headers.contentType = ContentType.text;
request.response.headers.contentLength = 'Finished'.length;
await request.response.close();
return;
}
final File newImage =
File('$basePath/output_files/output.${intFixed(frame, 3)}.jpg');
ByteData data = ByteData(request.headers.contentLength);
final buffer = data.buffer;
await newImage.writeAsBytes(
buffer.asUint8List(0, request.headers.contentLength));
frame++;
debugPrint('$frame');
request.response.statusCode = HttpStatus.ok;
request.response.headers.contentType = ContentType.text;
request.response.headers.contentLength = "Success".length;
request.response.write("Success");
await request.response.close();
}
}
}
Specifically, this part:
ByteData data = ByteData(request.headers.contentLength);
final buffer = data.buffer;
await newImage.writeAsBytes(
buffer.asUint8List(0, request.headers.contentLength));
When I set a breakpoint and check data, there's no data per-se. Just a list of zeros.
How do I convert the POST request to raw in order to save it to a file? The content length and the content type is correct (image/jpeg), but getting it to data is really stumping me.
After a lot of trial and error, the solution is to use await request.single which outputs a Uint8List, then write that to file as it's a stream of the HTTPRequest object itself.
https://api.flutter.dev/flutter/dart-async/Stream/single.html
final data = await request.single;
final file = File('$basePath/output_files/output.${intFixed(frame, 3)}.jpg');
file.writeAsBytesSync(data);
I am trying to upload an image to Firebase Storage using Flutter Web. I have followed many tutorials online on how to go about this, but I end up with the same error.
This is my code for upload
pickImageFromGallery() async {
imageFile = await ImagePickerWeb.getImageInfo;
setState(() {
_image = imageFile!.fileName!;
});
}
Future<void> uploadFile(
MediaInfo mediaInfo, String ref, String fileName) async {
try {
String mimeType = mime(path.basename(mediaInfo.fileName!))!;
final String extension = extensionFromMime(mimeType)!;
var metadata = fb.UploadMetadata(
contentType: mimeType,
);
fb.StorageReference storageReference =
fb.storage().ref('images').child('$fileName.$extension');
fb.UploadTaskSnapshot uploadTaskSnapshot =
await storageReference.put(mediaInfo.data, metadata).future;
Uri imageUri = await uploadTaskSnapshot.ref.getDownloadURL();
setState(() {
_imageURL = imageUri.toString();
});
} catch (e) {
setState(() {
_imageURL = e.toString();
});
}
}
This is my Storage rules for public access
rules_version = '2';
service firebase.storage {
match /b/{bucket}/o {
match /{allPaths=**} {
allow read, write;
}
}
}
This is the error I get
FirebaseError: Firebase Storage: An unknown error occurred, please check the error payload for server response.
{
"error": {
"code": 404,
"message": "Not Found."
}
} (storage/unknown)
I have tried everything online that seems to be the solution, but there is no exact solution for this problem. Please help me figure out what may be the problem.
I am using Flutter Web, and the following packages:
firebase,
path,
image_picker_web,
mime_type
EDIT: Working Code
The problem seemed to be with the StorageReference. It worked once edited to the following:
fb.StorageReference storageReference = fb.storage().refFromURL('gs://storageBucket/').child('$fileName.$extension');
replace 'storageBucket' with your storageBucket name shown in Firebase Storage
I also experienced this, because i did not update the firebase storage rules to allow me to write to the database
I got the same error.
It happens when the path can't reach the bucket, which I found out means "404 not found". Check the bucket name in storageBucket in firebaseConfig. In my case, I was using Dotenv and left a space after the double quotes.
Dotenv was converting it to "" double quotation marks.
You can check the bucket name in the Firebase storage console. You don't need the "gs://" at the beginning or the "/" at the end. Just check the name. If you are using Dotenv, watch out for spaces.
I got this error because when I copied the REACT_APP_FIREBASE_STORAGE_BUCKET variables from the .env file used in my local environment to Netlify which is my hosting service, I included the quotation marks (that is, I entered 'my.secret.url' instead of my.secret.url.
I'm using the file_picker package for flutter https://pub.dev/packages/file_picker
I have read many times that because you can’t access paths on web browsers, you need to use the bytes property, e.g.
FilePickerResult result = await FilePicker.platform.pickFiles();
if(result != null) {
var path = print(result.files.single.path); // this will return null
var bytes = print(result.files.singe.bytes); // this will return a Uint8List of bytes
} else {
// User canceled the picker
}
But I have to upload the images my users select from their devices via the web (so for all types of devices) to my Stripe Connect API in order for them to have a validated identity_document when they register. The bytes Uint8List will throw an error from firebase, here is my code:
export const uploadIdentityFront = async (uid: any, identityFront: any) => {
const fp = fs.readFileSync(identityFront);
const frontIdentity = await stripe.files.create({
file: {
data: fp,
name: 'identityFront.jpg',
type: 'application/octet-stream',
},
purpose: 'identity_document',
});
await updateId(uid, { frontIdentityFileId: frontIdentity.id })
return frontIdentity;
}
The error thrown:
[firebase_functions/unknown] TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string or an instance of Buffer or URL. Received an instance of Array
I will need to send stripe an image document via the file system's readFileSync property in order to do this, but with Flutter Web not being able to print the path for the image chosen by the user, I am stuck on how to resolve this issue
I use this code to send bytes to my server, which uses stream to send. You can use http package to send streams.
var request = http.MultipartRequest(
'POST',
Uri.parse('_url'),
);
request.files.add(
http.MultipartFile.fromBytes(
'identityFront', //name of field which you receive in api
bytes, // bytes
filename: 'identityFront.jpg', // optional name
//contentType: content, optional media Type
));
request.fields.addEntries([
MapEntry('uid', 'uid_value_in_String_Type'),
]);
await request.send();
I finally solved it. For anyone trying to upload a file to Stripe via flutter web, don't create a fs.readFileSync in your backend server side code. Instead, remove it and upload a file like this:
export const uploadIdentityFront = async (uid: any, identityFront: any) => {
const frontIdentity = await stripe.files.create({
file: {
data: identityFront,
name: 'identityFront.jpg',
type: 'image/jpg',
},
purpose: 'identity_document',
});
await updateId(uid, { frontIdentityFileId: frontIdentity.id })
return frontIdentity;
}
This way, you can upload the file via the file_picker package and uploading it as a picker.file.first.bytes. But don't wrap it in a string - send it just like this as a callable function in firebase functions:
await uploadFrontPassport.call(
<dynamic, dynamic>{'identityFront':picked.files.first.bytes}
);
Or How to upload an image to s3 with public access by Flutter Amplify?
In my current flutter project, I can't pass ACL:public-read property while uploading files to S3 using amplify.
And because of this, whenever I'm uploading a new file to s3, I need to make it public manually.
So I just want to upload a new file with public read access for everyone.
I found some solutions for the Javascript project but not in the Flutter project.
Below is a method, I'm using to upload.
Future<String> uploadFile(String fileName, File local) async {
try {
Map<String, String> metadata = <String, String>{};
metadata['name'] = 'filename';
metadata['desc'] = 'A file';
S3UploadFileOptions options = S3UploadFileOptions(accessLevel: StorageAccessLevel.guest, metadata: metadata);
UploadFileResult result = await Amplify.Storage.uploadFile(key: fileName, local: local, options: options);
return result.key;
} catch (e) {
print('UploadFile Err: ' + e.toString());
}
return null;
}
I think you should be using Dio for declaring the client object that will be used for posting the request
You can find an example code in the following answer
So far Flutter Amplify is not giving any option to upload images with public access.
It always uploads with private read access.
So I updated a few things in my project as described below.
Before Amplify integration I was uploading images to S3 and storing that URL to my server, and wherever I have to display, I'm just fetching URL from my server and loading images.
But now I'm storing key(that is used to upload images to S3 by Amplify) to my server.
And to display the image I'm getting the image URL from Amplify using that key(which is stored in my server).
Amplify adds a token to the image URL with a default validity of 7 days
Future<String> getUrl(String key) async {
try {
S3GetUrlOptions options = S3GetUrlOptions(accessLevel: StorageAccessLevel.guest, expires: 10000);
GetUrlResult result = await Amplify.Storage.getUrl(key: key, options: options);
String url = result.url;
return url;
} catch (e) {
print('GetUrl Err: ' + e.toString());
}
return null;
}
So it can be displayed by ImageView.
I was looking for a way where I could listen to changes performed to a file. For example, there is a file say 'x.json' where I will read and write on this file.
I will write data like, write['key']['c'] = 3 //just for representation purposes.
{
"key":{
"a":1,
"b":2,
"c":3
}
}
since 'c':3 has been added to 'key', I want that to reflect in my app. ie: read
My Code:
final directory = await getApplicationDocumentsDirectory();
File xfile = File('${directory.path}/x.json');
_fileSubscription = xfile.watch().listen((FileSystemEvent event) {
if (mounted) {
setState(() {
currentXFileContent= jsonDecode(xfile.readAsStringSync());
print('Within stream, currentXFileContent: $currentXFileContent');
});
}
});
I get this message
Unhandled Exception: FileSystemException: File system watching is not supported on this platform, path = ''
How can I solve this?