I am using dart:io's HTTP client to download large files the following way:
final url = Uri.parse(fileUrl);
final request = await httpClient.getUrl(url);
final response = await request.close();
Then I use the response of type HttpClientResponse which implements Stream<List<int>>, where each emitted List<int> represents a chunk of the file being downloaded.
My aim is to "transform" that stream into another one and to do so, I have recently learned of await for + yield which allows to do just that:
Stream<Event> processResponse(HttpClientResponse response) async* {
await for (final List<int> chunk in response) {
yield Event.FileChunk(chunk: chunk);
}
yield const Event.Completed();
}
Initially, and before learning about await for, I intended to use the method Stream<T>.listen which takes 4 parameters:
the onData callback for each emitted value in the stream,
the onError callback to notify errors while emitting values,
the onDone callback to notify the stream is closing and won't be emitting any new values and finally
the cancelOnError boolean parameter that is self-explaining.
By using await for () {}, I believe I cover #1 and #3 since I'll get all emitted values and when done, the code will leave the await for's scope which should mean it's done.
But what about errors? Since these streams will be lengthy and heavy IO streams, surely there can be an IO error at any moment. How are they reported? How can I access their data in order to report those and debug if needed?
Just use good ol' try catch, see this reduced example:
Stream<int> inee()async*{
try{
for (int i = 0; i <= 5; i++) {
if(i==4)throw Exception('Hated number 4');
yield i;
}
}catch(e,st){
print('Catch exception $e');
yield 33;
} finally{
//cleanup of incomplete files
}
}
void main() {
inee().listen(print);
}
The output would be the following:
0
1
2
3
Catch exception Exception: Hated number 4
3
inside the catch you can still yield, log your exceptions . You can also add the logic to delete incomplete files to the finally block.
Related
I'm working on a flutter application which requires using local storage. As the document directory path in Android will be given in Future<Directory>, I have to check whether the future finishes or not, each time I want to use the path.
The code may be similar to below
class DataStructure {
late Future<Directory> _dir = getApplicationDocumentsDirectory();
Future<void> read(String fileName) async {
Directory dir = await _dir;
// Do something using dir
}
}
This might be stupid, but I write C++ code in most of the time, so I want to reduce number of times pushing a function to the event queue for a better performance (I guess the compiler cuts async functions with await into "functions" to be pushed to the event queue). So I wrote experiment code to check whether await on a finished future will cut the control flow or not. (I mean, in the newest version of Dart, async functions will execute until the first 'await' keyword arises. I'm wondering if this is still the case if the future has already been finished.)
Future<void> voidFuture(String name) async {
print('$name: voidFuture');
}
Future<void> printSingle(String name, int index) async {
print('$name: -- index = $index');
}
// This emit function will print one line each time
// it gets chance to be executed.
// Because the 'await' keyword will cut the control flow,
// it can only print once each time it's executed,
// and must wait for the next chance to print again.
// This feature makes it appropriate for testing
// when and where the control flow is cut,
// as each cut will lead to one line of output.
Future<void> emit(String name, int count) async {
for (int index = 0; index != count; ++index) {
await printSingle(name, index);
}
}
Future<void> task_0() async {
const String name = 'task_0';
Future<void> emitFinish = emit(name, 3);
await voidFuture(name);
print('$name: output after await');
await emitFinish;
}
Running task_0 in my environment (Dart SDK version: 2.18.5 (stable) on "windows_x64") gives output as below:
task_0: -- index = 0
task_0: voidFuture
task_0: -- index = 1
task_0: output after await
task_0: -- index = 2
Which is the same as what I expected. And the weird thing comes when I change the emit() function:
Future<void> emit(String name, int count) async {
for (int index = 0; index != count; ++index) {
// Before:
// await printSingle(name, index);
// After:
await Future(() {
print('$name: -- index = $index');
});
}
}
Then the output becomes
task_0: voidFuture
task_0: output after await
task_0: -- index = 0
task_0: -- index = 1
task_0: -- index = 2
and it makes no sense for me that the third line, -- index = 0 comes after output after await. It seems that a function is more privileged than a future from constructor?
And my main question is "Will 'await' waits for a finished future?", so I wrote code below:
Future<String> stringFuture() async {
return '.';
}
Future<void> task_3() async {
const String name = 'task_3';
Future<void> emitFinish = emit(name, 4);
Future<String> futureString = stringFuture();
print('$name: before await of futureString');
await futureString;
print('$name: 1st await of futureString over');
await 1;
print('$name: 1st await of constant over');
await 2;
print('$name: 2nd await of constant over');
await emitFinish;
}
With the first version of emit(), the output is
task_3: -- index = 0
task_3: before await of futureString
task_3: -- index = 1
task_3: 1st await of futureString over
task_3: -- index = 2
task_3: 1st await of constant over
task_3: -- index = 3
task_3: 2nd await of constant over
Which means even await for a constant integral will push the lines after it to the event queue.
(Of course, with the second version of emit() all its output comes after the last print() in task_3() , and I don't know why)
I know there are manys work-arounds, one of them will be using a T? value to be assigned after the first time the Future<T> finishes, and check whether value == null each time using it.
But the questions I want to ask are:
What does the keyword await do internally? Please come with details that are enough to explain phenomena above.
Is there any way of overriding the default await behavior? E.g., by overriding a method?
What's the preferred way of using a Future value for many times?
(Unrelated to above) How to stop at the welcome page in Flutter to wait for these async functions, e.g.,
getApplicationDocumentsDirectory()
to finish before building all the widgets?
Most results I got from Google were introduction on async and await keywords for beginners, and I couldn't find much material explaining the behavior of await in Dart API Reference Documentation.
Thank you for saving a heart broken by await >_<
await is syntactic sugar for registering callbacks through the Future API. For example:
Future<int> foo() async {
var x = await someIntFuture();
return otherStuff(x);
}
is basically transformed into:
Future<int> foo() {
return someIntFuture.then((x) {
return otherStuff(x);
});
}
await registers a Future.then callback and returns that new Future. That it returns a Future means that await always yields (even in cases such as await null). This is also why when you invoke an async function, its body is executed synchronously until it reaches its first await.
it makes no sense for me that the third line, -- index = 0 comes after output after await. It seems that a function is more privileged than a future from constructor?
From the documentation for the Future constructor:
Creates a future containing the result of calling computation asynchronously with Timer.run.
The callback you supply to the Future constructor is invoked asynchronously; it is scheduled. This is different from calling an async function, which as stated before executes synchronously as much as possible first.
And my main question is "Will 'await' waits for a finished future?"
It doesn't matter if the Future is already completed or not. await always yields.
Is there any way of overriding the default await behavior? E.g., by overriding a method?
As mentioned, await is syntactic sugar. What you could do is to create a class that implements the Future API and handles .then differently (which is what Flutter's SynchronousFuture class does), but I wouldn't recommend it (for the same reasons why the SynchronousFuture documentation discourages its use).
What's the preferred way of using a Future value for many times?
Depends on the situation. In general, try to await the Future once and store the result somewhere (such as in a local variable). Otherwise I'd just await the Future multiple times and not worry about it until there's evidence that it's performance-critical.
(Unrelated to above) How to stop at the welcome page in Flutter to wait for these async functions
Depends. For some things, you can simply make your main function async and await whatever asynchronous initialization you want to do before calling runApp. In other cases (particularly for long-running ones), you should use a FutureBuilder.
(Also, in the future, separate questions should be asked separately.)
I had to do some work around combining streams where the returned streams would either be stream list of query snapshot, or stream lists of document snapshot. In any case I had to combine the streams and output the result in the UI. I chose not to use rxdart and i wasn't creating any special classes for the particular objects I wanted.
I did run into some major issues with async generator methods.
question 1:
The async generator will only await for one function and ignore any other awaits inside the generator function e.g.
when creating a function e.g.
Stream<List> testFunc3(Stream<List> stream, Stream<List> stream2, List<dynamic> anArray,
List<dynamic> anArray2) async* {
print('testfunc 3');
List streamList = [];
List streamList2 = [];
await for (List<dynamic> a in stream) {
streamList.clear();
a.forEach((element) {
if (anArray.contains(element['name'])) {
if (anArray2.contains(element['name'])) {
print('test 1');
if (!streamList.contains(element)) streamList.add(element);
}
}
});
yield streamList;
}
await for (List<dynamic> a in stream2) {
streamList2.clear();
a.forEach((element) {
if (anArray.contains(element['name'])) {
if (anArray2.contains(element['name'])) {
print('test 2');
}
}
});
yield streamList2;
}
}
is there a way to get around this behaviour?
question 2:
If I declare multiple variables that are functions which return streams, only the methods that are used in say StreamGroup.merge() are executed. streams not included in the merge never run.
e.g. z never returns a result because testfunc3 is never executed. unless i add z to the StreamGroup.merge().
keep in mind this isn't the whole function, just the relevant sections and i'm aware this method needs to yield a result
Stream<List> getStreamPlaces() async* {
Stream<List> x = this.testFunc2(stream, anArray);
Stream<List> y = this.testFunc2(textStream, anArray2);
Stream<List> z = this.testFunc3(radius);
Stream<List> g = StreamGroup.merge([x, y]);
// yields something
}
because of these issues i had to do some long winded approaches to filtering and getting the stream output I wanted. Is this a limitation or bug in dart or am i missing some fundamental understanding? Is there a keyword I could have used to force the execution of these methods?
any help would be appreciated.
for question 2. i do remember codewithAndrea briefly mentioned that an optimization of dart or flutter prevents streams from being executed unless they are used in the code but I couldn't find it when I went searching through his vids.
I have a problem where i want to read some data from database and i want my function to wait for the data before proceeding executing the rest of my code. i am using stream with await and async but doesnt look like it is working for me.
here is my code
void updateIncome() async {
Stream<List<IncomeData>> _currentEntries;
_currentEntries = database.watchIncomeForUpdate(this.income);
await _currentEntries.forEach((List<IncomeData> x) {
x.forEach((element) {
print('AWAIT');
}
);
});
print('FINISH');
}
here is the procedure that call my database and get data
Stream<List<IncomeData>> watchIncomeForUpdate(IncomeData entry) {
return (select(income)..where((t) =>t.id.isBiggerOrEqualValue(entry.id) & t.groupId.equals(entry.groupId))
..orderBy([(t) => OrderingTerm(expression: t.dateReceived)])).watch();
}
when i run the function updateIncome(), it prints FINISH first which make me believe that the await/async is not working by waiting for the foreach to loop through all elements in the list.
i tried to move the await keyword in the function call
_currentEntries = await database.watchIncomeForUpdate(this.income);
i get a warning message: await applied to Stream<List> which i not a Future
can someone help me? what i am doing wrong?
i want to wait for database to get the data, loop and print AWAIT then when finish, it should proceed with rest of code and print FINISH. the function that call the database return 8 rows. so when i loop using foreach, it should print AWAIT 8 times follow by FINISH.
how can i fix my code so that the function calls the database ,loop through the elements and wait until the loop finish before proceeding with the rest of the code outside of the loop?
Since watchIncomeForUpdate is not a Future function, you can't wait for a non-future function.
void updateIncome() async {
await for(var x in database.watchIncomeForUpdate(this.income)){
x.forEach((element) {
print('AWAIT');
}
);
});
print('FINISH');
}
Ref: https://dart.dev/tutorials/language/streams
Thanks for all replies. i figured it out.
i changed function from this
Stream<List<IncomeData>> watchIncomeForUpdate(IncomeData entry) {
return (select(income)..where((t) =>t.id.isBiggerOrEqualValue(entry.id) & t.groupId.equals(entry.groupId))
..orderBy([(t) => OrderingTerm(expression: t.dateReceived)])).watch();
}
to this
Future<List<IncomeData>> watchIncomeForUpdate(IncomeData entry) async {
return (select(income)..where((t) =>t.id.isBiggerOrEqualValue(entry.id) & t.groupId.equals(entry.groupId))
..orderBy([(t) => OrderingTerm(expression: t.dateReceived)])).get();
}
then call the procedure as
data = await database.watchIncomeForUpdate(this.income);
Please help me understand, why this code not working!
I try to get data from a Stream (Firestore), and take this data to a list. I want to wait until the list is ready, and with this list do something. But .then or .whenComplete fires before the list is ready...
This is the function to make the list and return it:
Future<List<EventDistance>> getEventsDistanceList(String eventId) async{
Stream<FS.QuerySnapshot> qs = EventDistanceDataRepository().getStreamByEventId(eventId: eventId);
List<EventDistance> dList = [];
EventDistance eventDistance;
qs.forEach((document) {
document.forEach((docs) {
eventDistance = eventDistanceFromJson(docs.data());
dList.add(eventDistance);
print(eventDistance.Name); //(3.) only for testing, to see if docs is not empty
}
);
});
print('return'); //(1.) only for testing, to see when return is fired
return dList;
}
(return also fires before)
i use this code so:
Future<List<EventDistance>> dList = getEventsDistanceList(filteredList[index].id );
dList.then((value) {
print('value: $value'); //(2.) only for testing,to see if the returned list is empty or not (empty :-( )
doSomething;
});
When i run, i recive first 'return' (1.), then 'value: null' (2.) (and an empty list) and then the elements of the list (Name1, Name2 ...) (3.).
What do i wrong? How to wait to receive the list first?
Thanks for the answeres!
To become more confident with async operations read the perfect
article by Didier Boelens
Let check what is going on in your code
Your getEventsDistanceList() routine is pure synchronous - all of it's content runs synchronously step by step
synchronously subscribe to a Stream in qs.forEach and set callback listener (document) { ... } which will be fired on each stream item somewhere in future
synchronous call print('return') is fired
finally getEventsDistanceList() returns
you listen to this Future returned from getEventsDistanceList() until it complete and then then() is fired with call to print('value: $value')
first stream item is received and callback fired with print(eventDistance.Name)
5th step will repeat with new items until stream completes or ended with error (see Stream.forEach implementation)
I supposed you need only first Stream item (if not, do not hesistate reach me in comments)
If so rewrite your code
EventDistanceDataRepository()
.getStreamByEventId(eventId: eventId)
.first
.then((document) => document.map((docs) => eventDistanceFromJson(docs.data())).toList())
.then((value) { doSomething;});
I prefer more readable await notation
final FS.QuerySnapshot document = await EventDistanceDataRepository()
.getStreamByEventId(eventId: eventId)
.first;
final List<EventDistance> listOfEvents = document.docs.map((e) => eventDistanceFromJson(e.data())).toList();
doSomething with this list
You need to use await in asynchronous functions. I'm guessing
Stream<FS.QuerySnapshot> qs =
EventDistanceDataRepository().getStreamByEventId(eventId: eventId);
Should be
Stream<FS.QuerySnapshot> qs = await
EventDistanceDataRepository().getStreamByEventId(eventId: eventId);
Where ever the operation that takes a long time happens gets the await keyword.
Try the code labs to get better with async await
works fine! the final code is:
final List<EventDistance> listOfEvents = document.docs.map((e) => eventDistanceFromJson(e.data())).toList();
My goal with this function is to return an integer representing all of the habits completed by the user. My databases structure is a collection of habits, each with a sub-collection containing its history. The function below utilizes a list of habits, and using the habit's ID it gets a stream of the history for that habit. My problem is that when I try to loop through the history for each habit it stalls at the await for. Any advice would be appreciated, I am still trying to fully understand streams/asynchronous functions.
Future<int> getCompleteHabits(User user, List<Habit> habits) async {
int completed = 0;
if(habits.isNotEmpty) {
for(Habit habit in habits) {
Stream<List<HabitHistory>> streamHistory = db.streamHabitHistory(user, habit.id);
await for(var h in streamHistory){
print(habit.title);
print(h);
}
print("test");
}
}
return completed;
}
This function will add to the variable completed, however, since I cannot access all the history I have yet to do so.
This function prints the following to the console, it doesn't get to the print("test").
flutter: Get in bed by 11:30 pm
flutter: [Instance of 'HabitHistory', Instance of 'HabitHistory', Instance of 'HabitHistory', Instance of 'HabitHistory', Instance of 'HabitHistory']
From the documentaion on Streams:
Streams are done when there are no more events in them, and the code receiving the events is notified of this just as it is notified that a new event arrives. When reading events using an await for loop, the loops stops when the stream is done.
The reason your loop is hanging is that the stream hasn't yet been closed. It will continue to hang until the stream gets closed or until you execute a break or return statement within the loop.
As Abion47 stated the stream has not been closed so the await for is never complete. It is waiting for new events.
Knowing this I can use the await streamHabitHistory(user, habit.id).first to get the list (so it is not waiting for events). The code is below.
Future<int> getCompleteHabits(User user, List<Habit> habits) async {
int completed = 0;
if(habits.isNotEmpty) {
for(Habit habit in habits) {
List<HabitHistory> history = await db.streamHabitHistory(user, habit.id).first;
for (HabitHistory h in history) {
completed = completed + h.completed.length;
}
}
}
return completed;
}