When I add some elements in HashMap in flutter like this:
void loadingMoreChannel(RefreshController _refreshController) async {
articleRequest.pageNum = articleRequest.pageNum + 1;
List<Channel> channels = await Repo.getChannels(articleRequest);
channels.addAll(channels);
_refreshController.loadComplete();
}
first step fetched some channel information from the server side, then add the channel info into List, shows error:
Concurrent modification during iteration: Instance(length:15) of '_GrowableList'.<…>
what should I do to add the channel elements into list concurrent?
Seems you need to rename the variable. You can just do
void loadingMoreChannel(RefreshController _refreshController) async {
articleRequest.pageNum = articleRequest.pageNum + 1;
List<Channel> result = await Repo.getChannels(articleRequest);
channels.addAll(result); //it was having same name
_refreshController.loadComplete();
}
Related
So I am using this function that utilizes the dart_ping.dart package
and I would like to only extract the ip, ttl and time from the result this function is giving.
void ping(String site) async {
// Create ping object with desired args
final ping = Ping(site, count: 5);
print('Running command: ${ping.command}');
// Begin ping process and listen for output
ping.stream.listen((PingData event) {
setState(() {
ping1 = event;
});
pingsArray.add(event);
print(event.summary);
});
}
you have to extract them from the PingData so in below code I will show you how to get the values. the ip, ttl, time from the response are assigned to the varialbles ip, ttl, time. You can probably start from there. Since I do not exactly know what you want to achieve, this is probably all I can show you.
void _ping(String site) async {
// Create ping object with desired args
final ping = Ping(site, count: 5);
print('Running command: ${ping.command}');
// Begin ping process and listen for output
ping.stream.listen((PingData event) {
final res = event.response;
if (res == null) return;
final ip = res.ip;
final ttl = res.ttl;
final time = res.time;
});
}
Thank you.
I am writing a test program to explore the use of Isolates in Dart/Flutter. One type of isolate that I have is started and stopped using a switch on a Flutter UI. This sends a message (START and STOP) to the isolate and I use a ValueNotifier to detect these commands and respond to them. When I initially wrote this, the Isolate ran continuously and didn’t respond the to the STOP command, which I understand is because the event queue would never be empty to process it.
Based on the first answer to this thread...
How to terminate a long running isolate #2
… and the suggested approach on this blog page:
https://hackernoon.com/executing-heavy-tasks-without-blocking-the-main-thread-on-flutter-6mx31lh
… I have tried to break my code up using Futures. I have split my run block into runWorker() and runChunk() functions, with runChunk called repeatedly as long as the Isolate should be running (run == true). I am doing something wrong because the Isolate still runs away and does not process the STOP command. I get slightly different results depending on whether I call runChunk directly or using Future.delayed (as per the hackernoon blog page), but neither approach works.
I believe that the STOP code works because if I remove the processing loop then everything triggers as expected, and I had an earlier version of this that worked when I included a 'Future.delayed' of 1 microsecond between each counter loop. So assume I am just using Futures incorrectly and not freeing up the event queue between 'runChunk' calls.
Can anyone tell me what I am doing wrong here? Here is the code for my isolate...
import 'dart:async';
import 'dart:isolate';
import 'package:flutter/material.dart';
class ContinuousIsolator {
ContinuousIsolator(
{required int channel, required void Function(double) setCounter}) {
print('Isolator initialisation');
_channel = channel;
_setCounter = setCounter;
spawn();
}
late int _channel; // The id of this isolate
late void Function(double) _setCounter;
late SendPort _port;
late Isolate _isolate;
ReceivePort receivePort = ReceivePort();
// Spawn a new isolate to complete the countdown (or up)
// channel = the number of the isolate
// counter = the value to count down from (or up to)
void spawn() {
print('Isolator establishing receiver');
receivePort.listen((msg) {
// print('Isolator message received');
// Unpack the map from the returned string
Map<int, dynamic> map = Map<int, dynamic>.from(msg);
// There should be only one key:value pair
for (var key in map.keys) {
msg = map[key]; // Extract the message
}
// print('Channel $_channel received "$msg" of type ${msg.runtimeType}');
// If we have received a Sendport, then add it to the port map
if (msg is SendPort) {
_port = msg;
} else {
// Otherwise process the message
// If it contains 'END' then we need to terminate the isolate
switch (msg) {
case 'END':
_isolate.kill();
// Isolate has completed, then close this receiver port
receivePort.close();
break;
default:
_setCounter(msg); // Send message to display
break;
}
}
});
// Start the isolate then let's get working on the countdown timer
Isolate.spawn(worker, {_channel: receivePort.sendPort}).then((isolate) {
_isolate = isolate; // Capture isolate so we can kill it later
});
}
void run() {
print('Sending START to worker');
_port.send('START');
}
void stop() {
print('Sending STOP to worker');
_port.send('STOP');
}
void end() {
_port.send('END'); // Send counter value to start countdown
}
}
void worker(Map<int, dynamic> args) {
int? id; // Number for this channel
ReceivePort receivePort = ReceivePort(); // Receive port for
SendPort? sendPort;
ValueNotifier<String> message = ValueNotifier('');
const double start = 10000000;
double counter = start;
const int chunkSize = 1000;
bool down = true;
bool run = true;
// Unpack the args to get the id and sendPort.
// There should be only one key:value pair
dynamic msg = '';
Map<int, dynamic> map = Map<int, dynamic>.from(args);
for (var key in map.keys) {
id = key; // Extract the isolate id
msg = map[key]; // Extract the message
}
// First message should contain the receivePort for the main isolate
if (msg is SendPort) {
sendPort = msg;
// print('args: $args port: $sendPort');
print('worker $id sending send port');
sendPort.send({id: receivePort.sendPort});
}
double getCounter() {
return counter;
}
void setCounter(double value) {
counter = value;
}
bool getDown() {
return down;
}
void setDown(bool value) {
down = value;
}
Future runChunk(
int chunkSize,
bool Function() getDown,
void Function(bool) setDown,
double Function() getCounter,
void Function(double) setCounter) {
const double start = 10000000;
print('Running chunk, counter is ${getCounter()}');
for (int i = 0; i < chunkSize; i++) {
// print('Worker $id in the while loop');
if (getDown() == true) {
setCounter(getCounter() - 1);
if (getCounter() < 0) setDown(!getDown());
} else {
setCounter(getCounter() + 1);
if (getCounter() > start) setDown(!getDown());
}
if ((getCounter() ~/ 1000) == getCounter() / 1000) {
// print('Continuous Counter is ${getCounter()}');
sendPort!.send({id: getCounter()});
} // Send the receive port
}
return Future.value();
}
void changeMessage() {
print('Message has changed to ${message.value}');
if (message.value == 'START') {
run = true;
} else {
run = false;
}
}
void runWorker() async {
message.addListener(changeMessage);
print('Worker running counter down from $counter');
while (run == true) {
// This line appears to run the isolate, but there is no output/feedback to the GUI
// The STOP command does not interrupt operation.
Future.delayed(const Duration(microseconds: 0),
() => runChunk(chunkSize, getDown, setDown, getCounter, setCounter));
// This line runs the isolate with feedback to the GUI
// The STOP command does not interrupt operation.
runChunk(chunkSize, getDown, setDown, getCounter, setCounter);
}
message.removeListener(changeMessage);
}
// Establish listener for messages from the controller
print('worker $id establishing listener');
receivePort.listen((msg) {
print('worker $id has received $msg');
switch (msg) {
case 'START':
print('Worker $id starting run');
message.value = msg;
runWorker();
break;
case 'STOP':
print('Worker $id stopping run');
message.value = msg;
break;
case 'END':
message.removeListener(changeMessage);
receivePort.close;
break;
default:
break;
}
});
}
Ok, so it turns out that the problem was not with the Futures, but with this line of code in the runWorker() method:
while (run == true) {
... which (I think) was blocking the event queue.
I also think I was over-thinking things using a ValueNotifier (although I don't think there is any reason why this wouldn’t work).
I have simplified my code so that the main computation method (runChunk) checks if it should still be running after each chunk, and if the answer is yes then it just calls itself again to run the next chunk.
The run flag is set to true and runChunk called when START is received from the parent and the run flag is set to false when STOP is received. This removes the need for the ValueNotifier.
Everything works as expected now and the isolate computation (‘runChunk’) can be started and stopped on demand.
Here is the revised code (the print lines are debug lines that I have left in):
import 'dart:async';
import 'dart:isolate';
class ContinuousIsolator {
ContinuousIsolator(
{required int channel, required void Function(double) setCounter}) {
print('Isolator initialisation');
_channel = channel;
_setCounter = setCounter;
spawn();
}
late int _channel; // The id of this isolate
late void Function(double)
_setCounter; // Callback function to set on-screen counter
late SendPort _port; // SendPort of child isolate to communicate with
late Isolate _isolate; // Pointer to child isolate
ReceivePort receivePort = ReceivePort(); // RecevierPort for this class
// Spawn a new isolate to complete the countdown (or up)
// channel = the number of the isolate
// counter = the value to count down from (or up to)
void spawn() {
print('Isolator establishing receiver');
// Establish a listener for messages from the child
receivePort.listen((msg) {
// Unpack the map from the returned string (child sends a single map
// contained key: isolate id and value: message)
Map<int, dynamic> map = Map<int, dynamic>.from(msg);
// There should be only one key:value pair received
for (var key in map.keys) {
msg = map[key]; // Extract the message
}
// If we have received a Sendport, then capture it to communicate with
if (msg is SendPort) {
_port = msg;
} else {
// Otherwise process the message
// If it contains 'END' then we need to terminate the isolate
switch (msg) {
case 'END':
_isolate.kill();
// Isolate has completed, then close this receiver port
receivePort.close();
break;
default:
_setCounter(msg); // Send message to display
break;
}
}
});
// Start the child isolate
Isolate.spawn(worker, {_channel: receivePort.sendPort}).then((isolate) {
_isolate = isolate; // Capture isolate so we can kill it later
});
}
// Class method to start the child isolate doing work (countdown timer)
void run() {
print('Sending START to worker');
_port.send('START');
}
// Class method to stop the child isolate doing work (countdown timer)
void stop() {
print('Sending STOP to worker');
_port.send('STOP');
}
// Class method to tell the child isolate to self-terminate
void end() {
_port.send('END'); // Send counter value to start countdown
}
}
// Child isolate function that is spawned by the parent class ContinuousIsolator
// Called initially with single map of key: 'unique channel id' and value:
// receiver port from the parent
void worker(Map<int, dynamic> args) {
int? id; // Unique id number for this channel
ReceivePort receivePort = ReceivePort(); // Receive port for this isolate
SendPort? sendPort; // Send port to communicate with the parent
const double start = 10000000; // Starting counter value
double counter = start; // The counter
const int chunkSize =
100; // The number of counter decrements/increments to process per 'chunk'
bool down = true; // Flag to show is counting 'down' (true) or 'up' (false)
bool run = false; // Flag to show if the isolate is running the computation
// Unpack the initial args to get the id and sendPort.
// There should be only one key:value pair
dynamic msg = '';
Map<int, dynamic> map = Map<int, dynamic>.from(args);
for (var key in map.keys) {
id = key; // Extract the isolate id
msg = map[key]; // Extract the message
}
// The first message should contain the receivePort for the main isolate
if (msg is SendPort) {
sendPort = msg; // Capture sendport to communicate with the parent
print('worker $id sending send port');
// Send the receiver port for this isolate to the parent
sendPort.send({id: receivePort.sendPort});
}
// Method to get the current counter value
double getCounter() {
return counter;
}
// Method to set the current counter value
void setCounter(double value) {
counter = value;
}
// Method to get the down flag value
bool getDown() {
return down;
}
// Method to set the down flag value
void setDown(bool value) {
down = value;
}
// This function does the main work of the isolate, ie the computation
Future<void> runChunk(
int chunkSize, // The number of loops to process for a given 'chunk'
bool Function() getDown, // Callback to get bool down value
void Function(bool) setDown, // Callback to set bool down value
double Function() getCounter, // Call back to get current counter value
void Function(double) setCounter) // Callback to set counter value
async {
const double start = 10000000; // Starting value for the counter
// Count down (or up) the counter for chunkSize iterations
for (int i = 0; i < chunkSize; i++) {
// Counting down...
if (getDown() == true) {
setCounter(getCounter() - 1);
// If reached zero, flip the counting up
if (getCounter() < 0) setDown(!getDown());
} else {
// Counting up...
setCounter(getCounter() + 1);
// If reached start (max), flip the counting down
if (getCounter() > start) setDown(!getDown());
}
// Update the display every 1000 points
if ((getCounter() ~/ 1000) == getCounter() / 1000) {
sendPort!.send({id: getCounter()}); // Notify parent of the new value
}
}
// If the isolate is still running (parent hasn't sent 'STOP') then
// call this function again to iterate another chunk. This gives the event
// queue a chance to process the 'STOP'command from the parent
if (run == true) {
// I assume Future.delayed adds call back onto the event queue
Future.delayed(const Duration(microseconds: 0), () {
runChunk(chunkSize, getDown, setDown, getCounter, setCounter);
});
}
}
// Establish listener for messages from the controller
print('worker $id establishing listener');
receivePort.listen((msg) {
print('worker $id has received $msg');
switch (msg) {
case 'START':
// Start the worker function running and set run = true
print('Worker $id starting run');
run = true;
runChunk(chunkSize, getDown, setDown, getCounter, setCounter);
break;
case 'STOP':
// Set run = false to stop the worker function
print('Worker $id stopping run');
run = false;
break;
case 'END':
// Inform parent that isolate is shutting down
sendPort?.send({id: msg});
receivePort.close; // Close the receiver port
break;
default:
break;
}
});
}
Following Firestore documentation and other posts, I arrived at the following function:
Future<void> downloadProductImages() async {
//TODO: modify on suppliers admin that images are saved with the correct name.
final storageRef = FirebaseStorage.instance.ref();
int count = 1;
for (final ref in product.imgsMap.keys.toList()) {
print(ref);
try {
final childRef = storageRef.child(ref);
const maxSize = 10 * 1024 * 1024;
final Uint8List data = (await childRef.getData(maxSize))!;
//check if file is jpeg
late String ext;
if (data[0] == 0xFF &&
data[1] == 0xD8 &&
data[data.length - 2] == 0xFF &&
data[data.length - 1] == 0xD9) {
ext = 'jpeg';
} else {
ext = 'png';
}
final content = base64Encode(data!.toList());
AnchorElement(href: "${product.imgsMap[ref]}")
..setAttribute("download", " ${product.name}_$count.$ext")
..setAttribute("type", "image/$ext")
..click();
count++;
} on FirebaseException catch (e) {
//TODO: HANDLE EXCEPTIONS
print(e);
} on Exception catch (e) {
// Handle any errors.
print(e);
}
}
// File file = // generated somewhere
// final rawData = file.readAsBytesSync();
}
I am able to download multiples files, however the images are not recognized as such on Chrome Mobile's downloads, which is my main target.
I am guessing that I am not building correctly the Anchor Element.
How can I fix this method or is there a different way to download multiple images on Flutter Web?
I am developing a Flutter Web application that, after clicking a download button, I need to download multiple images from Firebase Storage.
How can I don this on Flutter Web?
UPDATE:
After following Frank's suggestion on the comments, and other posts. I wrote the following function:
Future<void> downloadProductImages() async {
//TODO: modify on suppliers admin that images are saved with the correct name.
final storageRef = FirebaseStorage.instance.ref();
int count = 1;
for (final ref in product.imgsMap.keys.toList()) {
print(ref);
try {
final childRef = storageRef.child(ref);
const maxSize = 10 * 1024 * 1024;
final Uint8List data = (await childRef.getData(maxSize))!;
//check if file is jpeg
late String ext;
if (data[0] == 0xFF &&
data[1] == 0xD8 &&
data[data.length - 2] == 0xFF &&
data[data.length - 1] == 0xD9) {
ext = 'jpeg';
} else {
ext = 'png';
}
// Data for "images/island.jpg" is returned, use this as needed.
final content = base64Encode(data!.toList());
AnchorElement(
href:
"data:application/octet-stream;charset=utf-16le;base64,$content")
//href: "image/$ext;charset=utf-16le;base64,$content")
..setAttribute("download", "${product.name}_$count.$ext")
..click();
count++;
} on FirebaseException catch (e) {
//TODO: HANDLE EXCEPTIONS
print(e);
} on Exception catch (e) {
// Handle any errors.
print(e);
}
}
// File file = // generated somewhere
// final rawData = file.readAsBytesSync();
}
On chrome mobile, the files are downloaded but are not recognized as pictures. It seems that the AnchorElment doesn't have the correct href.
Any ideas?
I am stuck with the following problem:
We are writing a library app which contains a bunch of internal documents (roughly 900).
The total size of all docs is around 2,5+ GB.
I have to find a way how to initialise the app with 2,5GB of Docs on first start.
My idea was to create a zip file of the initial load, download it from server and unpack it during setup on first start. However I am not finding a way how to unzip such a big file, as all solutions read the zip to memory first (completely) before writing it to storage.
I also do not want to make 900+ calls to our web-server to download the Documents on first start.
Deployment target is iOS and Android, possibly win+mac later on.
Any ideas?
i tested it on flutter linux where Uri.base points to the root of the project (the folder with pubspec.yaml, README.md etc) - if you run it on android / iOS check where Uri.base points to and change baseUri if it is not good location:
final debug = true;
final Set<Uri> foldersCreated = {};
final baseUri = Uri.base;
final baseUriLength = baseUri.toString().length;
final zipFileUri = baseUri.resolve('inputFolder/backup.zip');
final outputFolderUri = baseUri.resolve('outputFolder/');
print('0. files will be stored in [$outputFolderUri]');
final list = FileList(zipFileUri, debug: debug);
print('1. reading ZipDirectory...');
final directory = ZipDirectory.read(InputStream(list));
print('2. iterating over ZipDirectory file headers...');
for (final zfh in directory.fileHeaders) {
final zf = zfh.file;
final content = zf.content;
// writing file
final uri = outputFolderUri.resolve(zf.filename);
final folderUri = uri.resolve('.');
if (foldersCreated.add(folderUri)) {
if (debug) print(' #### creating folder [${folderUri.toString().substring(baseUriLength)}] #### ');
Directory.fromUri(folderUri).createSync(recursive: true);
}
File.fromUri(uri).writeAsBytesSync(content);
print("file: [${zf.filename}], compressed: ${zf.compressedSize}, uncompressed: ${zf.uncompressedSize}, length: ${content.length}");
}
list.close();
print('3. all done!');
and here is a List backed by a LruMap that reads data in chunks from your huge zip file:
class FileList with ListMixin<int> {
RandomAccessFile _file;
LruMap<int, List<int>> _cache;
final int maximumPages;
final int pageSize;
final bool debug;
FileList(Uri uri, {
this.pageSize = 1024, // 1024 is just for tests: make it bigger (1024 * 1024 for example) for normal use
this.maximumPages = 4, // maybe even 2 is good enough?
this.debug = false,
}) {
_file = File.fromUri(uri).openSync();
length = _file.lengthSync();
_cache = LruMap(maximumSize: maximumPages);
}
void close() => _file.closeSync();
#override
int length;
int minIndex = -1;
int maxIndex = -1;
List<int> page;
#override
int operator [](int index) {
// print(index);
// 1st cache level
if (index >= minIndex && index < maxIndex) {
return page[index - minIndex];
}
// 2nd cache level
int key = index ~/ pageSize;
final pagePosition = key * pageSize;
page = _cache.putIfAbsent(key, () {
if (debug) print(' #### reading page #$key (position $pagePosition) #### ');
_file.setPositionSync(pagePosition);
return _file.readSync(pageSize);
});
minIndex = pagePosition;
maxIndex = pagePosition + pageSize;
return page[index - pagePosition];
}
#override
void operator []=(int index, int value) => null;
}
you can play with pageSize and maximumPages to find the optimal solution - i think you can start with pageSize: 1024 * 1024 and maximumPages: 4 but you have to check it by yourself
of course all of that code should be run in some Isolate since it takes a lot of time to unzip couple of GB and then your UI will freeze, but first run it as it is and see the logs
EDIT
it seems that ZipFile.content has some memory leaks so the alternative could be a "tar file" based solution, it uses tar package and since it reads a Stream as an input you can use compressed *.tar.gz files (your Documents.tar had 17408 bytes while Documents.tar.gz has 993 bytes), notice that you can even read your data directly from the socket's stream so no need for any intermediate .tar.gz file:
final baseUri = Uri.base;
final tarFileUri = baseUri.resolve('inputFolder/Documents.tar.gz');
final outputFolderUri = baseUri.resolve('outputFolder/');
print('0. files will be stored in [$outputFolderUri]');
final stream = File.fromUri(tarFileUri)
.openRead()
.transform(gzip.decoder);
final reader = TarReader(stream);
print('1. iterating over tar stream...');
while (await reader.moveNext()) {
final entry = reader.current;
if (entry.type == TypeFlag.dir) {
print("dir: [${entry.name}]");
final folderUri = outputFolderUri.resolve(entry.name);
await Directory.fromUri(folderUri).create(recursive: true);
}
if (entry.type == TypeFlag.reg) {
print("file: [${entry.name}], size: ${entry.size}");
final uri = outputFolderUri.resolve(entry.name);
await entry.contents.pipe(File.fromUri(uri).openWrite());
}
}
print('2. all done!');