I want to break the for loop in GWT callback's execute method response.
For Example,
for (int idx = 0; idx < recordList.getLength(); idx++) { //Starting ABC FOR LOOP
ABCDMI.addData(recordList.get(idx),
new DSCallback() {
public void execute(DSResponse response, Object rawData, DSRequest request) {
if(response.getAttribute("UnSuccess") != null && !response.getAttribute("UnSuccess").equalsIgnoreCase("")) {
break; //I want to break ABC FOR LOOP here.
}
}
}
Can anybody help me in this?
When you call an asynchronous method, you dont know how long it will take. In your examples all of these calls will be sent in almost the same instant, but the response would come in any time in the future, so the order is not guaranteed.
Of-course you cannot break a loop inside your callback, but you can handle the loop inside your callback calling the async method from it each time one call finishes.
This example should work in your case, and all callbacks would be executed sequentially.
DSCallback myCallBack = new DSCallback() {
int idx = 0;
int length = recordList.getLength();
public void execute(DSResponse response, Object rawData, DSRequest request) {
if (++idx < length
&& (response.getAttribute("UnSuccess") == null
|| !response.getAttribute("UnSuccess").equalsIgnoreCase(""))) {
ABCDMI.addData(recordList.get(idx), this);
}
}
};
ABCDMI.addData(recordList.get(0), myCallBack);
Related
I want to batch many futures into a single request that triggers either when a maximum batch size is reached, or a maximum time since the earliest future was received is reached.
Motivation
In flutter, I have many UI elements which need to display the result of a future, dependent on the data in the UI element.
For instance, I have a widget for a place, and a sub-widget which displays how long it will take to walk to a place. To compute the how long it will take to walk, I issue a request to Google Maps API to get the travel time to the place.
It is more efficient and cost-effective to batch all these API requests into a batch API request. So if there are 100 requests made instantaneously by the widgets, then the futures could be proxied through a single provider, which batches the futures into a single request to Google, and unpacks the result from Google into all the individual requests.
The provider needs to know when to stop waiting for more futures and when to actually issue the request, which should be controllable by the maximum "batch" size (i.e., # of travel time requests), or the maximum amount of time you are willing to wait for batching to take place.
The desired API would be something like:
// Client gives this to tell provider how to compute batch result.
abstract class BatchComputer<K,V> {
Future<List<V>> compute(List<K> batchedInputs);
}
// Batching library returns an object with this interface
// so that client can submit inputs to completed by the Batch provider.
abstract class BatchingFutureProvider<K,V> {
Future<V> submit(K inputValue);
}
// How do you implement this in dart???
BatchingFutureProvider<K,V> create<K,V>(
BatchComputer<K,V> computer,
int maxBatchSize,
Duration maxWaitDuration,
);
Does Dart (or a pub package) already provide this batching functionality, and if not, how would you implement the create function above?
This sounds perfectly reasonable, but also very specialized.
You need a way to represent a query, to combine these queries into a single super-query, and to split the super-result into individual results afterwards, which is what your BatchComputer does. Then you need a queue which you can flush through that under some conditions.
One thing that is clear is that you will need to use Completers for the results because you always need that when you want to return a future before you have the value or future to complete it with.
The approach I would choose would be:
import "dart:async";
/// A batch of requests to be handled together.
///
/// Collects [Request]s until the pending requests are flushed.
/// Requests can be flushed by calling [flush] or by configuring
/// the batch to automatically flush when reaching certain
/// tresholds.
class BatchRequest<Request, Response> {
final int _maxRequests;
final Duration _maxDelay;
final Future<List<Response>> Function(List<Request>) _compute;
Timer _timeout;
List<Request> _pendingRequests;
List<Completer<Response>> _responseCompleters;
/// Creates a batcher of [Request]s.
///
/// Batches requests until calling [flush]. At that pont, the
/// [batchCompute] function gets the list of pending requests,
/// and it should respond with a list of [Response]s.
/// The response to the a request in the argument list
/// should be at the same index in the response list,
/// and as such, the response list must have the same number
/// of responses as there were requests.
///
/// If [maxRequestsPerBatch] is supplied, requests are automatically
/// flushed whenever there are that many requests pending.
///
/// If [maxDelay] is supplied, requests are automatically flushed
/// when the oldest request has been pending for that long.
/// As such, The [maxDelay] is not the maximal time before a request
/// is answered, just how long sending the request may be delayed.
BatchRequest(Future<List<Response>> Function(List<Request>) batchCompute,
{int maxRequestsPerBatch, Duration maxDelay})
: _compute = batchCompute,
_maxRequests = maxRequestsPerBatch,
_maxDelay = maxDelay;
/// Add a request to the batch.
///
/// The request is stored until the requests are flushed,
/// then the returned future is completed with the result (or error)
/// received from handling the requests.
Future<Response> addRequest(Request request) {
var completer = Completer<Response>();
(_pendingRequests ??= []).add(request);
(_responseCompleters ??= []).add(completer);
if (_pendingRequests.length == _maxRequests) {
_flush();
} else if (_timeout == null && _maxDelay != null) {
_timeout = Timer(_maxDelay, _flush);
}
return completer.future;
}
/// Flush any pending requests immediately.
void flush() {
_flush();
}
void _flush() {
if (_pendingRequests == null) {
assert(_timeout == null);
assert(_responseCompleters == null);
return;
}
if (_timeout != null) {
_timeout.cancel();
_timeout = null;
}
var requests = _pendingRequests;
var completers = _responseCompleters;
_pendingRequests = null;
_responseCompleters = null;
_compute(requests).then((List<Response> results) {
if (results.length != completers.length) {
throw StateError("Wrong number of results. "
"Expected ${completers.length}, got ${results.length}");
}
for (int i = 0; i < results.length; i++) {
completers[i].complete(results[i]);
}
}).catchError((error, stack) {
for (var completer in completers) {
completer.completeError(error, stack);
}
});
}
}
You can use that as, for example:
void main() async {
var b = BatchRequest<int, int>(_compute,
maxRequestsPerBatch: 5, maxDelay: Duration(seconds: 1));
var sw = Stopwatch()..start();
for (int i = 0; i < 8; i++) {
b.addRequest(i).then((r) {
print("${sw.elapsedMilliseconds.toString().padLeft(4)}: $i -> $r");
});
}
}
Future<List<int>> _compute(List<int> args) =>
Future.value([for (var x in args) x + 1]);
See https://pub.dev/packages/batching_future/versions/0.0.2
I have almost exactly the same answer as #lrn, but have put some effort to make the main-line synchronous, and added some documentation.
/// Exposes [createBatcher] which batches computation requests until either
/// a max batch size or max wait duration is reached.
///
import 'dart:async';
import 'dart:collection';
import 'package:quiver/iterables.dart';
import 'package:synchronized/synchronized.dart';
/// Converts input type [K] to output type [V] for every item in
/// [batchedInputs]. There must be exactly one item in output list for every
/// item in input list, and assumes that input[i] => output[i].
abstract class BatchComputer<K, V> {
const BatchComputer();
Future<List<V>> compute(List<K> batchedInputs);
}
/// Interface to submit (possible) batched computation requests.
abstract class BatchingFutureProvider<K, V> {
Future<V> submit(K inputValue);
}
/// Returns a batcher which computes transformations in batch using [computer].
/// The batcher will wait to compute until [maxWaitDuration] is reached since
/// the first item in the current batch is received, or [maxBatchSize] items
/// are in the current batch, whatever happens first.
/// If [maxBatchSize] or [maxWaitDuration] is null, then the triggering
/// condition is ignored, but at least one condition must be supplied.
///
/// Warning: If [maxWaitDuration] is not supplied, then it is possible that
/// a partial batch will never finish computing.
BatchingFutureProvider<K, V> createBatcher<K, V>(BatchComputer<K, V> computer,
{int maxBatchSize, Duration maxWaitDuration}) {
if (!((maxBatchSize != null || maxWaitDuration != null) &&
(maxWaitDuration == null || maxWaitDuration.inMilliseconds > 0) &&
(maxBatchSize == null || maxBatchSize > 0))) {
throw ArgumentError(
"At least one of {maxBatchSize, maxWaitDuration} must be specified and be positive values");
}
return _Impl(computer, maxBatchSize, maxWaitDuration);
}
// Holds the input value and the future to complete it.
class _Payload<K, V> {
final K k;
final Completer<V> completer;
_Payload(this.k, this.completer);
}
enum _ExecuteCommand { EXECUTE }
/// Implements [createBatcher].
class _Impl<K, V> implements BatchingFutureProvider<K, V> {
/// Queues computation requests.
final controller = StreamController<dynamic>();
/// Queues the input values with their futures to complete.
final queue = Queue<_Payload>();
/// Locks access to [listen] to make queue-processing single-threaded.
final lock = Lock();
/// [maxWaitDuration] timer, as a stored reference to cancel early if needed.
Timer timer;
/// Performs the input->output batch transformation.
final BatchComputer computer;
/// See [createBatcher].
final int maxBatchSize;
/// See [createBatcher].
final Duration maxWaitDuration;
_Impl(this.computer, this.maxBatchSize, this.maxWaitDuration) {
controller.stream.listen(listen);
}
void dispose() {
controller.close();
}
#override
Future<V> submit(K inputValue) {
final completer = Completer<V>();
controller.add(_Payload(inputValue, completer));
return completer.future;
}
// Synchronous event-processing logic.
void listen(dynamic event) async {
await lock.synchronized(() {
if (event.runtimeType == _ExecuteCommand) {
if (timer?.isActive ?? true) {
// The timer got reset, so ignore this old request.
// The current timer needs to inactive and non-null
// for the execution to be legitimate.
return;
}
execute();
} else {
addPayload(event as _Payload);
}
return;
});
}
void addPayload(_Payload _payload) {
if (queue.isEmpty && maxWaitDuration != null) {
// This is the first item of the batch.
// Trigger the timer so we are guaranteed to start computing
// this batch before [maxWaitDuration].
timer = Timer(maxWaitDuration, triggerTimer);
}
queue.add(_payload);
if (maxBatchSize != null && queue.length >= maxBatchSize) {
execute();
return;
}
}
void execute() async {
timer?.cancel();
if (queue.isEmpty) {
return;
}
final results = await computer.compute(List<K>.of(queue.map((p) => p.k)));
for (var pair in zip<Object>([queue, results])) {
(pair[0] as _Payload).completer.complete(pair[1] as V);
}
queue.clear();
}
void triggerTimer() {
listen(_ExecuteCommand.EXECUTE);
}
}
How do I block the current thread until the OnComplete handler of my observer has finished, without the use of threading primitives?
Here is my code. I want that the Console.WriteLine("Press... statement should be executed only after the OnComplete handler, namely ResetCount has finished executing.
class Program
{
private static long totalItemCount = 0;
private static long listCount = 0;
static void Main()
{
Console.WriteLine($"Starting Main on Thread {Thread.CurrentThread.ManagedThreadId}\n");
var o = Observable.Timer(TimeSpan.FromSeconds(3), TimeSpan.FromSeconds(1))
.Take(20)
.Concat(Observable.Interval(TimeSpan.FromSeconds(0.01)).Take(200))
.Buffer(TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(5));
o.Subscribe(Print, onCompleted: ResetCount);
// How I make sure this line appears only after the OnComplete has fired?
// Do I have to use traditional threading primitives such as wait handles?
// Or just cause the main thread to sleep long enough? That doesn't seem right.
Console.WriteLine("\nPress any key to exit...");
Console.ReadKey();
}
private static void ResetCount()
{
if (listCount > 0)
{
Console.WriteLine($"{totalItemCount} items processed in {listCount} lists.");
}
else
{
Console.WriteLine($"{totalItemCount} items processed.");
}
Interlocked.Exchange(ref totalItemCount, 0);
Interlocked.Exchange(ref listCount, 0);
}
static void Print<T>(T value)
{
var threadType = Thread.CurrentThread.IsBackground ? "Background" : "Foreground";
if (value is IList)
{
var list = value as IList;
Console.WriteLine($"{list.Count} items in list #{Interlocked.Increment(ref listCount)}:");
foreach (var item in list)
{
Console.WriteLine($"{item.ToString()}, ({threadType} #{Thread.CurrentThread.ManagedThreadId}), Item #{Interlocked.Increment(ref totalItemCount)}");
}
Console.WriteLine();
}
else
{
Console.WriteLine($"{value.ToString()}, ({threadType} #{Thread.CurrentThread.ManagedThreadId}), Item #{Interlocked.Increment(ref totalItemCount)}");
}
}
}
On Rx we have specific schedulers to handle threading, synchronization and related.
You can read more about that here:
http://www.introtorx.com/content/v1.0.10621.0/15_SchedulingAndThreading.html
But basically what you're looking for is changing this line:
.Buffer(TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(5), Scheduler.CurrentThread);
They're several ways to test/validate a Rx query. Keep in mind that wouldn't be the answer for all the problems.
My application needs only fixed no of records to be read
& processed. How to limit this if I am using a flatfileItemReader ?
In DB based Item Reader, I am returning null/empty list when max_limit is reached.
How to achieve the same if I am using a org.springframework.batch.item.file.FlatFileItemReader ?
For the FlatFileItemReader as well as any other ItemReader that extends AbstractItemCountingItemStreamItemReader, there is a maxItemCount property. By configuring this property, the ItemReader will continue to read until either one of the following conditions has been met:
The input has been exhausted.
The number of items read equals the maxItemCount.
In either of the two above conditions, null will be returned by the reader, indicating to Spring Batch that the input is complete.
If you have any custom ItemReader implementations that need to satisfy this requirement, I'd recommend extending the AbstractItemCountingItemStreamItemReader and going from there.
The best approch is to write a delegate which is responsible to track down number of read records and stop after a fixed count; the components should take care of execution context to allow restartability
class CountMaxReader<T> implements ItemReader<T>,ItemStream
{
private int count = 0;
private int max = 0;
private ItemReader<T> delegate;
T read() {
T next = null;
if(count < max) {
next = delegate.read();
++count;
}
return next;
}
void open(ExecutionContext executionContext) {
((ItemStream)delegate).open(executionContext);
count = executionContext.getInt('count', 0);
}
void close() {
((ItemStream)delegate).close(executionContext);
}
void update(ExecutionContext executionContext) {
((ItemStream)delegate).update(executionContext);
executionContext.putInt('count', count);
}
}
This works with any reader.
public class CountMaxFlatFileItemReader extends FlatFileItemReader {
private int counter;
private int maxCount;
public void setMaxCount(int maxCount) {
this.maxCount = maxCount;
}
#Override
public Object read() throws Exception {
counter++;
if (counter >= maxCount) {
return null; // this will stop reading
}
return super.read();
}
}
Something like this should work. The reader stops reading, as soon as null is returned.
I'll create an observable (through a variety of means) and return it to interested parties, but when they're done listening, I'd like to tear down the observable so it doesn't continue consuming resources. Another way to think of it as creating topics in a pub sub system. When no one is subscribed to a topic any more, you don't want to hold the topic and its filtering around anymore.
Rx already has an operator to suit your needs - well two actually - Publish & RefCount.
Here's how to use them:
IObservable xs = ...
var rxs = xs.Publish().RefCount();
var sub1 = rxs.Subscribe(x => { });
var sub2 = rxs.Subscribe(x => { });
//later
sub1.Dispose();
//later
sub2.Dispose();
//The underlying subscription to `xs` is now disposed of.
Simple.
If I have understood your question you want to create the observable such that when all subscribers have disposed their subscription i.e there is no more subscriber, then you want to execute a clean up function which will stop the observable from production further values.
If this is what you want then you can do something like below:
//Wrap a disposable
public class WrapDisposable : IDisposable
{
IDisposable disp;
Action act;
public WrapDisposable(IDisposable _disp, Action _act)
{
disp = _disp;
act = _act;
}
void IDisposable.Dispose()
{
act();
disp.Dispose();
}
}
//Observable that we want to clean up after all subs are done
public static IObservable<long> GenerateObs(out Action cleanup)
{
cleanup = () =>
{
Console.WriteLine("All subscribers are done. Do clean up");
};
return Observable.Interval(TimeSpan.FromSeconds(1));
}
//Wrap the observable
public static IObservable<T> WrapToClean<T>(IObservable<T> obs, Action onAllDone)
{
int count = 0;
return Observable.CreateWithDisposable<T>(ob =>
{
var disp = obs.Subscribe(ob);
Interlocked.Increment(ref count);
return new WrapDisposable(disp,() =>
{
if (Interlocked.Decrement(ref count) == 0)
{
onAllDone();
}
});
});
}
//Usage example:
Action cleanup;
var obs = GenerateObs(out cleanup);
var newObs = WrapToClean(obs, cleanup);
newObs.Take(6).Subscribe(Console.WriteLine);
newObs.Take(5).Subscribe(Console.WriteLine);
Here is a sample code to retrieve data from a database using the yield keyword that I found in a few place while googling around :
public IEnumerable<object> ExecuteSelect(string commandText)
{
using (IDbConnection connection = CreateConnection())
{
using (IDbCommand cmd = CreateCommand(commandText, connection))
{
connection.Open();
using (IDbDataReader reader = cmd.ExecuteReader())
{
while(reader.Read())
{
yield return reader["SomeField"];
}
}
connection.Close();
}
}
}
Am I correct in thinking that in this sample code, the connection would not be closed if we do not iterate over the whole datareader ?
Here is an example that would not close the connection, if I understand yield correctly..
foreach(object obj in ExecuteSelect(commandText))
{
break;
}
For a db connection that might not be catastrophic, I suppose the GC would clean it up eventually, but what if instead of a connection it was a more critical resource?
The Iterator that the compiler synthesises implements IDisposable, which foreach calls when the foreach loop is exited.
The Iterator's Dispose() method will clean up the using statements on early exit.
As long as you use the iterator in a foreach loop, using() block, or call the Dispose() method in some other way, the cleanup of the Iterator will happen.
Connection will be closed automatically since you're using it inside "using" block.
From the simple test I have tried, aku is right, dispose is called as soon as the foreach block exit.
#David : However call stack is kept between call, so the connection would not be closed because on the next call we would return to the next instruction after the yield, which is the while block.
My understanding is that when the iterator is disposed, the connection would also be disposed with it. I also think that the Connection.Close would not be needed because it would be taken care of when the object is disposed because of the using clause.
Here is a simple program I tried to test the behavior...
class Program
{
static void Main(string[] args)
{
foreach (int v in getValues())
{
Console.WriteLine(v);
}
Console.ReadKey();
foreach (int v in getValues())
{
Console.WriteLine(v);
break;
}
Console.ReadKey();
}
public static IEnumerable<int> getValues()
{
using (TestDisposable t = new TestDisposable())
{
for(int i = 0; i<10; i++)
yield return t.GetValue();
}
}
}
public class TestDisposable : IDisposable
{
private int value;
public void Dispose()
{
Console.WriteLine("Disposed");
}
public int GetValue()
{
value += 1;
return value;
}
}
Judging from this technical explanation, your code will not work as expected, but abort on the second item, because the connection was already closed when returning the first item.
#Joel Gauvreau : Yes, I should have read on. Part 3 of this series explains that the compiler adds special handling for finally blocks to trigger only at the real end.