RXjava complicated async task - rx-java2

Question

I am trying to use async rxjava flow to handle a complicated task.
I get list of metrics, and list of cities and get their values (list of date and value). Each metric is stored in a different db, so lots of HTTP calls are expected.
The result order means nothing so I want to:
run over all the metrics and asynchronously.
For every metric I want to run over all the cities asynchronously.
For every city and metric I want to create a HTTP call to fetch the data.
The end response should look like this:
"result": {
"metric1": [
{
"cityId": 8,
"values": [
{
"date": "2017-09-26T10:49:49",
"value": 445
},
{
"date": "2017-09-27T10:49:49",
"value": 341
}
]
},
{
"cityId": 9,
"values": [
{
"date": "2017-09-26T10:49:49",
"value": 445
},
{
"date": "2017-09-27T10:49:49",
"value": 341
}
]
}
],
"metric2": [
{
"cityId": 8,
"values": [
{
"date": "2017-09-26T10:49:49",
"value": 445
},
{
"date": "2017-09-27T10:49:49",
"value": 341
}
]
},
{
"cityId": 9,
"values": [
{
"date": "2017-09-26T10:49:49",
"value": 445
},
{
"date": "2017-09-27T10:49:49",
"value": 341
}
]
}
]
}
}
}
How can I achieve this? Got lost with all the observables. Which should be blocking, which should return a real value?
EDIT: added code sample
Here is what I have so far:
1. for each metric (parallel I guess):
Observable.from(request.getMetrics())
.subscribe(metric -> {
List metricDataList = getMetricData(metric, citiesList);
result.put(metric.getName(), metricDataList);
});
return ImmutableMap.of("result", result);
2. Get metric data (according to cities):
final List<Map<String,Object>> result = new ArrayList<>();
Observable.from(citiesList).subscribe(city ->
result.add(ImmutableMap.of(
"id", city.getId(),
"name", city.getName(),
"value", getMetricValues(metricType, city.getId())
.toBlocking()
.firstOrDefault(new ArrayList<>()))));
return result;
3. According to metric I decide which service to invoke:
private Observable<List<AggregatedObject>> getMetricValues(AggregationMetricType metric, Integer cityId) {
switch (metric) {
case METRIC_1: return fetchMetric1(city);
case METRIC_2: return fetchMetric2(city);
}
return Observable.empty();
}
4. Invoke the service:
public Observable<List<AggregatedObject>> fetchMetric1(Integer city) {
return Observable.just(httpClient.getData(city)
.map(this::transformCountResult)
.onErrorResumeNext(err -> Observable.from(new ArrayList<>()));
}
5. Transform the received data:
protected List<AggregatedObject> transformCountResult(JsonNode jsonNode) {
...
}
It is working, though I am not sure that I have implemented it correctly regarding blocking and concurrency.
Help please.
Regards,
Ido

Answer 1

In the step you call "2) get metric data", you could refactor as
final List<Map<String,Object>> result = new ArrayList<>();
Observable.from(citiesList)
.flatMap(city -> getMetricValues(metricType, city.getId())
.firstOrDefault(new ArrayList<>())
.map( metric -> ImmutableMap.of(
"id", city.getId(),
"name", city.getName(),
"value", metric) ) )
.toList()
.toBlocking()
.subscribe( resultList -> result = resultList );
Moving the toBlocking() outside of the processing means that individual requests will occur in a more parallel fashion.