Downloading progress in dart:http - flutter

I'm trying to make a progress bar indicator for a downloading file, but if I add a listener to the StreamedResponse, the piping works, but does to not finish its future.
final client = new http.Client();
http.StreamedResponse response = await client.send(http.Request("GET", Uri.parse('someurl')));
var received = 0;
var length = response.contentLength;
//if I remove this listener, the await below gets completed
var listen = response.stream.listen((List<int> bytes) {
received += bytes.length;
print("${(received / length) * 100} %");
});
var sink = downloadFile.openWrite();
await response.stream.pipe(sink);
listen.cancel();
sink.close();
On github they already advised someone that it was supposed to work, but on StreamedResponse docs stays that This should always be a single-subscription stream.. So, adding a listener to calculate the percentage seems to bugs StreamedResponse pipe in someway. Any idea on how to get this to work?

#pskink comment let me to this solution that works for every type of sink you are using.
var length = response.contentLength;
var received = 0;
var sink = downloadFile.openWrite();
await response.stream.map((s) {
received += s.length;
print("${(received / length) * 100} %");
return s;
}).pipe(sink);
another way to accomplish this, if you are writing into a file, is to watch file length
var length = response.contentLength;
var sink = downloadFile.openWrite();
Future.doWhile(() async {
var received = await downloadFile.length();
print("${(received / length) * 100} %");
return received != length;
});
await response.stream.pipe(sink);

Related

How to send multiple(as a list) files in restApi in flutter with single key from form-data

I want to send multiple Files(Images) in post api from flutter with single key from form-data.
Just like in the picture for "images[]" key i have multiple images to send.
var uri = Uri.parse("${baseUrl}/user/addPost");
https.MultipartRequest request = new https.MultipartRequest('POST', uri);
request.fields['user_id'] = "54";
request.fields['title'] = postAdModel.title!;
request.fields['type'] = postAdModel.type!;
request.fields['category_id'] = postAdModel.category_id.toString();
////////////////////////// Thumbnail Image Adding /////////////////
final stream = https.ByteStream(postAdModel.thumbnail!.openRead());
stream.cast();
final length = await postAdModel.thumbnail!.length();
var multiport = https.MultipartFile(
'thumbnail', // key
stream,
length,
filename: postAdModel.thumbnail!.path,
);
request.files.add(multiport);
////////////////////////* Gallery Images Adding */////////////////////
List<https.MultipartFile> galleryImages = <https.MultipartFile>[];
for (int i = 0; i < postAdModel.images!.length; i++) {
File imageFile = postAdModel.images![i];
var stream = new https.ByteStream(imageFile.openRead());
stream.cast();
var length = await imageFile.length();
var multipartFile = https.MultipartFile(
"images[]",
stream,
length,
filename: imageFile.path);
galleryImages.add(multipartFile);
}
request.files.addAll(galleryImages);
var response = await request.send();

How can I get multiple messages from dart isolate?

How can I get multiple messages from dart isolate?
I'm trying to create an excel file and want to do some operation on that file in an isolate. Before doing an operation on that file, I want to return an message to main isolate, that excel file is created.
Here is function goes in isolate :
foo(String filePath){
// create excel file
var bytes = File(filePath).readAsBytesSync();
var excel = Excel.decodeBytes(bytes);
//HERE I WANT TO SEND THE MESSAGE THAT CREATING EXCEL FILE IS DONE
// some operatoin on excel file
var result = doSomeOperation(excel);
return result;
}
Main isolate code :
var result = await compute(foo, filePath);
What should I do to get creating file message before the actual result comes?
For excel, I'm using excel: ^2.0.0-null-safety-3 package.
Compute only returns one result. If you want to pass multiple 'events' back to the main isolate then you need to use the full Isolate logic (with sendPort and receivePort).
For example, the following code runs in an isolate, and downloads a file while emitting float values to represent progress, potentially a String to indicate log messages and then a bool to indicate success or failure upon completion.
Future<void> isolateDownload(
DownloadRequest request) async {
final sendPort = request.sendPort;
if (sendPort != null) {
var success = false;
var errorMessage = '';
var url = Uri.parse('a_url_based_on_request');
IOSink? out;
try {
http.StreamedResponse response =
await http.Client().send(http.Request('GET', url));
if (response.statusCode == 200) {
var filePath =
join(request.destinationDirPath, '${request.fileName}.ZIP');
var contentLength = response.contentLength;
var bytesLoadedUpdateInterval = (contentLength ?? 0) / 50;
var bytesLoaded = 0;
var bytesLoadedAtLastUpdate = 0;
out = File(filePath).openWrite();
await response.stream.forEach((chunk) {
out?.add(chunk);
bytesLoaded += chunk.length;
// update if enough bytes have passed since last update
if (contentLength != null &&
bytesLoaded - bytesLoadedAtLastUpdate >
bytesLoadedUpdateInterval) {
sendPort.send(bytesLoaded / contentLength);
bytesLoadedAtLastUpdate = bytesLoaded;
}
});
success = true;
if (contentLength != null) {
sendPort.send(1.0); // send 100% downloaded message
}
} else {
errorMessage =
'Download of ${request.fileName} '
'received response ${response.statusCode} - ${response.reasonPhrase}';
}
} catch (e) {
errorMessage = 'Download of ${request.chartType}:${request.chartName} '
'received error $e';
} finally {
await out?.flush();
await out?.close();
if (errorMessage.isNotEmpty) {
sendPort.send(errorMessage);
}
sendPort.send(success);
}
}
}
The code that spawns the isolate then simply checks for the type of the message passed to it to determine the action.
Future<bool> _downloadInBackground(
DownloadRequest request) async {
var receivePort = ReceivePort();
request.sendPort = receivePort.sendPort;
var isDone = Completer();
var success = false;
receivePort.listen((message) {
if (message is double) {
showUpdate(message);
}
if (message is String) {
log.fine(message); // log error messages
}
if (message is bool) {
success = message; // end with success or failure
receivePort.close();
}
}, onDone: () => isDone.complete()); // wraps up
await Isolate.spawn(isolateDownload, request);
await isDone.future;
return success;
}

Download a sub stream with youtube_explode_dart

I'm developing a Flutter app, using the package youtube_explode_dart.
Currently I download all the audio stream, then crop it using ffmpeg. Quickly this looks like :
var id = "aboZctrHfK8";
var file = File("mySong.mp4");
var start = "2000ms";
var end = "5000ms";
// Downloading audio only
var manifest = await yt.videos.streamsClient.getManifest(id);
var audio = manifest.audioOnly.first;
var audioStream = yt.videos.streamsClient.get(audio);
var output = file.openWrite(mode: FileMode.writeOnlyAppend);
for (final data in audioStream) {
output.add(data);
}
// Croping audio
FlutterFFmpeg _flutterFFmpeg = new FlutterFFmpeg();
await _flutterFFmpeg.executeWithArguments([
"-v", "error",
"-ss", start,
"-to", end,
"-i", file.path,
"-acodec", "copy", "myCroppedSong.mp4"]);
Now I'm facing another issue: Some videos are really heavy and take a while to download. This is not acceptable for my end users, especially since I only want part of the original audio.
Is there a way to download only a subset of the audio stream?
Something like:
for (final data in audioStream.subset(start, end)) {
output.add(data);
}
It would be anwsome!
import 'package:youtube_explode_dart/youtube_explode_dart.dart';
// Initialize the YoutubeExplode instance.
final yt = YoutubeExplode();
Future<void> ExplodeDown() async {
stdout.writeln('Type the video id or url: ');
var url = stdin.readLineSync()!.trim();
// Save the video to the download directory.
Directory('downloads').createSync();
// Download the video.
await download(url);
yt.close();
exit(0);
}
Future<void> download(String id) async {
// Get video metadata.
var video = await yt.videos.get(id);
// Get the video manifest.
var manifest = await yt.videos.streamsClient.getManifest(id);
var streams = manifest.videoOnly;
// Get the audio track with the highest bitrate.
var audio = streams.first;
var audioStream = yt.videos.streamsClient.get(audio);
// Compose the file name removing the unallowed characters in windows.
var fileName = '${video.title}.${audio.container.name}'
.replaceAll(r'\', '')
.replaceAll('/', '')
.replaceAll('*', '')
.replaceAll('?', '')
.replaceAll('"', '')
.replaceAll('<', '')
.replaceAll('>', '')
.replaceAll('|', '');
var file = File('downloads/$fileName');
// Delete the file if exists.
if (file.existsSync()) {
file.deleteSync();
}
// Open the file in writeAppend.
var output = file.openWrite(mode: FileMode.writeOnlyAppend);
// Track the file download status.
var len = audio.size.totalBytes;
var count = 0;
// Create the message and set the cursor position.
var msg = 'Downloading ${video.title}.${audio.container.name}';
stdout.writeln(msg);
// Listen for data received.
// var progressBar = ProgressBar();
await for (final data in audioStream) {
// Keep track of the current downloaded data.
count += data.length;
// Calculate the current progress.
var progress = ((count / len) * 100).ceil();
print (progress);
// Update the progressbar.
// progressBar.update(progress);
// Write to file.
output.add(data);
}
await output.close();
}
**How to download load youtube video & audio & play stream audio **
String youTubeLink = "https://www.youtube.com/watch?v=Ja-85lFDSEM";
Future<void> _downloadVideo(youTubeLink) async{
final yt = YoutubeExplode();
final video = await yt.videos.get(youTubeLink);
// Get the video manifest.
final manifest = await yt.videos.streamsClient.getManifest(youTubeLink);
final streams = manifest.muxed;
final audio = streams.first;
final audioStream = yt.videos.streamsClient.get(audio);
final fileName = '${video.title}.${audio.container.name.toString()}'
.replaceAll(r'\', '')
.replaceAll('/', '')
.replaceAll('*', '')
.replaceAll('?', '')
.replaceAll('"', '')
.replaceAll('<', '')
.replaceAll('>', '')
.replaceAll('|', '');
final dir = await getApplicationDocumentsDirectory();
final path = dir.path;
final directory = Directory('$path/video/');
await directory.create(recursive: true);
final file = File('$path/video/$fileName');
final output = file.openWrite(mode: FileMode.writeOnlyAppend);
var len = audio.size.totalBytes;
var count = 0;
var msg = 'Downloading ${video.title}.${audio.container.name}';
stdout.writeln(msg);
await for (final data in audioStream){
count += data.length;
var progress = ((count / len) * 100).ceil();
print(progress);
output.add(data);
}
await output.flush();
await output.close();
}

Prefix text to ASP.NET Core response body

I'm trying to prepend the string )]}',\n to any response body that's JSON. I thought that an IAsyncResultFilter would be what I needed to use, but I'm not having luck. If I use the below code, it appends the text to the response since calling await next() writes to the response pipe. If I try and look at the context before that though, I can't tell what the response will actually be to know if it's JSON.
public class JsonPrefixFilter : IAsyncResultFilter
{
public async Task OnResultExecutionAsync(ResultExecutingContext context, ResultExecutionDelegate next)
{
var executed = await next();
var response = executed.HttpContext.Response;
if (response.ContentType == null || !response.ContentType.StartsWith("application/json"))
return;
var prefix = Encoding.UTF8.GetBytes(")]}',\\n");
var bytes = new ReadOnlyMemory<byte>(prefix);
await response.BodyWriter.WriteAsync(bytes);
}
}
Thanks to timur's post I was able to come up with this working solution.
public class JsonPrefixFilter : IAsyncResultFilter
{
public async Task OnResultExecutionAsync(ResultExecutingContext context, ResultExecutionDelegate next)
{
var response = context.HttpContext.Response;
// ASP.NET Core will always send the contents of the original Body stream back to the client.
var originalBody = response.Body;
// We want to write into a memory stream instead of the actual response body for now.
var ms = new MemoryStream();
response.Body = ms;
// After this call the body is written into the memory stream and the properties
// of the response object are populated.
await next();
if (response.ContentType != null && response.ContentType.StartsWith("application/json")) {
var prefix = Encoding.UTF8.GetBytes(")]}',\\n");
var prefixMemoryStream = new MemoryStream();
await prefixMemoryStream.WriteAsync(prefix);
await prefixMemoryStream.WriteAsync(ms.ToArray());
prefixMemoryStream.Seek(0, SeekOrigin.Begin);
// Now put the stream back that .NET wants to use and copy the memory stream to it.
response.Body = originalBody;
await prefixMemoryStream.CopyToAsync(response.Body);
} else {
// If it's not JSON, don't muck with the stream, so just put things back.
response.Body = originalBody;
ms.Seek(0, SeekOrigin.Begin);
await ms.CopyToAsync(response.Body);
}
}
}
Update:
I never liked the above, so I switched to this solution. Instead of calling AddJsonOptions, I took inspiration from ASP.NET's formatter to use this instead:
public class XssJsonOutputFormatter : TextOutputFormatter
{
private static readonly byte[] XssPrefix = Encoding.UTF8.GetBytes(")]}',\n");
public JsonSerializerOptions SerializerOptions { get; }
public XssJsonOutputFormatter()
{
SerializerOptions = new() {
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
ReferenceHandler = ReferenceHandler.IgnoreCycles
};
SupportedEncodings.Add(Encoding.UTF8);
SupportedMediaTypes.Add(MediaTypeHeaderValue.Parse("application/json"));
}
public override sealed async Task WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
{
ArgumentNullException.ThrowIfNull(context, nameof(context));
ArgumentNullException.ThrowIfNull(selectedEncoding, nameof(selectedEncoding));
var httpContext = context.HttpContext;
var objectType = context.Object?.GetType() ?? context.ObjectType ?? typeof(object);
var responseStream = httpContext.Response.Body;
try {
await responseStream.WriteAsync(XssPrefix);
await JsonSerializer.SerializeAsync(responseStream, context.Object, objectType, SerializerOptions, httpContext.RequestAborted);
await responseStream.FlushAsync(httpContext.RequestAborted);
} catch (OperationCanceledException) when (context.HttpContext.RequestAborted.IsCancellationRequested) {
}
}
}
Now, when you call .AddControllers() you just set that as the first output formatter:
services.AddControllers(options => {
options.Filters.Add(new ProducesAttribute("application/json"));
options.OutputFormatters.Insert(0, new XssJsonOutputFormatter());
});
Obviously you could improve this to take serialization options in the constructor, but all my project would work exactly like the above so I just hardcoded it right in.
You could've used Seek on a steam to rewind it. Issue is, you can only keep adding onto default HttpResponseStream, it does not support seeking.
So you can employ the technique from this SO answer and temporarily replace it with MemoryStream:
private Stream ReplaceBody(HttpResponse response)
{
var originBody = response.Body;
response.Body = new MemoryStream();
return originBody;
}
private async Task ReturnBodyAsync(HttpResponse response, Stream originalBody)
{
response.Body.Seek(0, SeekOrigin.Begin);
await response.Body.CopyToAsync(originalBody);
response.Body = originalBody;
}
public async Task OnResultExecutionAsync(ResultExecutingContext context, ResultExecutionDelegate next)
{
var originalBody = ReplaceBody(context.HttpContext.Response); // replace the default stream with MemoryStream
await next(); // we probably dont care about the return of this call. it's all in the context
var response = context.HttpContext.Response;
if (response.ContentType == null || !response.ContentType.StartsWith("application/json"))
return;
var prefix = Encoding.UTF8.GetBytes(")]}',\\n");
var bytes = new ReadOnlyMemory<byte>(prefix);
response.Body.Seek(0, SeekOrigin.Begin); // now you can seek. but you will notice that it overwrites the response so you might need to make extra space in the buffer
await response.BodyWriter.WriteAsync(bytes);
await ReturnBodyAsync(context.HttpContext.Response, originalBody); // revert the reference, copy data into default stream and return it
}
this is further complicated by the fact that you need to restore reference to original stream, so you have to careful around that.
This SO answer has a bit more context.

How to send big text data from tcp socket to Flutter app?

I have a 3000 rows and 20 column data most cell has 50 or 80 character long text. I test using aqueduct and Dart Server to see which one is better intermediate option in between server data and Flutter communication.
I use 2 option for the test. All I can test is that for (var data in _socket) runs about 10 or 12 times to receive whole data. On the other hand _socket.liste lost much of the data.
In both I even use await Future.delayed(Duration(seconds: 7))...
And result was same. In short data they're both very fast. but for big data I am having problem to get correct result.
How to send big text data from tcp socket to Flutter app?
// OPTION 1
_socket.listen((List<int> data) {
String _reply = new String.fromCharCodes(data).trim();
_reply = "$_reply$_tmpValue";
});
// OPTION 2
await for (var data in _socket) {
String _reply = new String.fromCharCodes(data).trim();
_reply = "$_reply$_tmpValue";
}
UPDATE:
final Socket client = await Socket.connect('192.168.22.120', 3000);
client.add(utf8.encode('$_requestedData\r\n'));
client.listen(
(var data) {
var ints = new Uint8List.fromList(data).buffer.asInt64List();
ints.forEach((i) => print('Got $i'));
},
onDone: () { print('Done'); client.close(); },
onError: (e) { print('Got error $e'); client.close(); });
print('main done');
RESULT
main done
_reply:
Got 4206144564389427761
Got 6861850004482897203
Got 2368148215793798714
UPDATE (Working Solution):
_socket = await Socket.connect('127.0.0.1’, 3000);
_socket.write(_myQuery);
await for (var data in _socket) {
_reply = new String.fromCharCodes(data);
_rawValue = _rawValue + _reply;
int dataLen;
dataLen = _reply.length;
int carriageReturnPosition;
carriageReturnPosition = dataLen - 2;
int newLinePosition;
newLinePosition = dataLen - 1;
// In my scenario it loops 32 times and it takes about 145 seconds to get big data
if (_reply.substring(carriageReturnPosition, newLinePosition) == '\r' &&
_reply.substring(newLinePosition, dataLen) == '\n') {
await _socket.close();
_returnData = _reply;
}
}