Firebase Realtime DB query returns both a list and a map when using startAfter - flutter

I have the following piece of code, that is simplified for the purpose of illustration:
List<Name> _nameList = [];
_getNameList() {
String dbPath = “names/“;
var startAfter = _nameList.length - 1;
final db = FirebaseDatabase.instance
.ref()
.child(dbPath)
.orderByKey()
.startAfter(startAfter.toString())
.limitToFirst(4);
final nameFuture = db.once();
nameFuture.then((event) {
List<Name> nameList = [];
print(event.snapshot.value);
if (event.snapshot.value != null) {
for (var item in (event.snapshot.value as List<Object?>)) {
if (item != null) {
Name d = Name.fromRTDB(item);
nameList.add(d);
}
}
}
if (nameList.isNotEmpty) {
setState(() {
_nameList.addAll(nameList);
});
}
});
}
It initially reads the first 4 Name objects from Firebase Realtime DB. And whenever it is called, I expect it to read the very next 4 Name objects and attach them to the _nameList as stated in the code.
IMPORTANT to note that keys to each Name object in the Realtime DB are as 0, 1, 2, 3, 4, and so on. If I am not totally wrong, this actually should make the life easier when using startAfter.
As you may recognize, I print out the values coming from DB, as print(event.snapshot.value);.
The first read works perfectly fine, where startAfter is set to "-1". Example output from print(event.snapshot.value);:
[{name: "John"}, {name: "Johanna"}, {name: "Maria"}, {name: "Steve"}]
Problem 1
When I run the function again, where the startAfter is now 3, I see the following output:
[null, null, null, null, {name: "Michael"}, {name: "Maradona"}, {name: "Pelle"}, {name: "Messi"}]
I expect the list to have the size of 4 but it includes 8 items. And the first 4 are null. Why do I have 8 items in the returned list, and why are the first 4 null?
Problem 2
When I run the function for the 3rd time, where the startAfter is now 7, I get the following output this time:
{10: {name: "Jordan"}, 8: {name: "James"}, 9: {name: "Rambo"}, 11: {name: "Brian"}}
While the first 2 calls returned lists, the 3rd call returned a map.
Why does it all of a sudden return a map in the 3rd call?
And I make the call with orderByKey() but the returned keys in the map are not in order; why?

Problem 1: When I run the function again, where the startAfter is now 3, I see the following output:
[null, null, null, null, {name: "Michael"}, {name: "Maradona"}, {name: "Pelle"}, {name: "Messi"}]
This is the expected behavior for the Realtime Database API. When it sees a bunch of (sufficiently) sequential numeric keys, it assumed that data was originally an array and returns the data as an array. Since your array doesn't have items at indexes [0-3] those indexes show us as null.
This so-called array coercion depends very much on the data, but typically stops happening once there are too many (I don't recall the exact value for that) missing values at the start of the array, which I think is what may be causing the other problem.
The solution is to not use sequential numeric keys in your Realtime Database data, but either use the push() function to generate keys (recommended) or to prefix the numbers with a short string, e.g. "key_3", "key_4", etc.
Also see: Best Practices: Arrays in Firebase.

Related

Flutter Odoo : how to read all fields

iam making a flutter app that depending on odoo
and i want to get all fields in a module
so iam using read method
and iam depending on this library
http://oogbox.com/page/odoo-api-flutter.html
https://pub.dartlang.org/packages/odoo_api/versions/1.0.1
the problem is that i tried everything to get all ids
i changed the List ids to []
and to null
and nothing working
and this is the code
final ids = [1, 2, 3, 4, 5];
final fields = ["id", "name", "email"];
client.read("res.partner", ids, fields).then((OdooResponse result) {
if (!result.hasError()) {
List records = result.getResult();
} else {
print (result.getError());
}
});
From the documentation this read method doesn't allow you to bring all records, you should use searchRead() and pass to the domain param an empty list to do that.

Dataloader did not return an array of the same length?

I Am building an express JS application with graphql, and mongodb (mongoose). I am using facebooks Dataloader to batch and cache requests.
Its working perfectly fine except for this use case.
I have a database filled with users posts. Each post contains the users ID for reference. When i make a call to return all the posts in the database. The posts are returned fine but if i try to get the user in each post. Users with multiple posts will only return a single user because the key for the second user is cached. So 2 posts(keys) from user "x" will only return 1 user object "x".
However Dataloader has to return the same amount of promises as keys that it recieves.
It has a option to specify cache as false so each key will make a request. But this doesnt seem to work for my use case.
Sorry if i havn't explained this very well.
this is my graphql request
query {
getAllPosts {
_id // This is returned fine
user {
_id
}
}
}
Returned error:
DataLoader must be constructed with a function which accepts Array<key> and returns Promise<Array<value>>, but the function did not return a Promise of an Array of the same length as the Array of keys.
are you trying to batch post keys [1, 2, 3] and expecting to get user results [{ user1 }, {user2}, { user1 }]?
or are you trying to batch user keys [1, 2] and expecting to get post results [{ post1}, {post3}] and [{ post2 }]?
seems like only in the second case will you run into a situation where you have length of keys differing from length of results array.
to solve the second, you could do something like this in sql:
const loader = new Dataloader(userIDs => {
const promises = userIDs.map(id => {
return db('user_posts')
.where('user_id', id);
});
return Promise.all(promises);
})
loader.load(1)
loader.load(2)
so you return [[{ post1}, {post3}], [{ post2 }]] which dataloader can unwrap.
if you had done this instead:
const loader = new Dataloader(userIDs => {
return db('user_posts')
.where('user_id', [userIDs]);
})
loader.load(1)
loader.load(2)
you will instead get [{ post1}, {post3}, { post2 }] and hence the error: the function did not return a Promise of an Array of the same length as the Array of keys
not sure if the above is relevant / helpful. i can revise if you can provide a snippet of your batch load function
You need to map the data returned from the database to the Array of keys.
Dataloader: The Array of values must be the same length as the Array of keys
This issue is well explained in this YouTube Video - Dataloader - Highly recommended

Store contents with rest proxy giving incorrect count

ExtJS 5.1.x, with several stores using rest proxy.
Here is an example:
Ext.define('cardioCatalogQT.store.TestResults', {
extend: 'Ext.data.Store',
alias: 'store.TestResults',
config:{
fields: [
{name: 'attribute', type: 'string'},
{name: 'sid', type: 'string'},
{name: 'value_s', type: 'string'},
{name: 'value_d', type: 'string'}
],
model: 'cardioCatalogQT.model.TestResult',
storeId: 'TestResults',
autoLoad: true,
pageSize: undefined,
proxy: {
url: 'http://127.0.0.1:5000/remote_results_get',
type: 'rest',
reader: {
type: 'json',
rootProperty: 'results'
}
}
}
});
This store gets populated when certain things happen in the API. After the store is populated, I need to do some basic things, like count the number of distinct instances of an attribute, say sid, which I do as follows:
test_store = Ext.getStore('TestResults');
n = test_store.collect('sid').length);
The problem is that I have to refresh the browser to get the correct value of 'n,' otherwise, the count is not right. I am doing a test_store.load() and indeed, the request is being sent to the server after the .load() is issued.
I am directly querying the backend database to see what data are there in the table and to get a count to compare to the value given by test_store.collect('sid').length);. The strange thing is that I am also printing out the store object in the debugger, and the expected records (when compared to the content in the database table) are displayed under data.items array, but the value given by test_store.collect('sid').length is not right.
This is all done sequentially in a success callback. I am wondering if there is some sort of asynchronous behavior giving me the inconsistent results between what is is the store and the count on the content of the store?
I tested this with another store that uses the rest proxy and it has the same behavior. On the other hand, using the localStorage proxy gives the correct count consistent with the store records/model instances.
Here is the relevant code in question, an Ajax request fires off and does its thing correctly, and hit this success callback. There really isn't very much interesting going on... the problem section is after the console.log('TEST STORE HERE'); where I get the store, print the contents of the store, load/sync then print the store (which works just fine) and then finally print the length of uniquely grouped items by the sid attribute (which is what is not working):
success: function(response) {
json = Ext.decode(response.responseText);
if(json !== null && typeof (json) !== 'undefined'){
for (i = 0, max = json.items.length; i < max; i += 1) {
if (print_all) {
records.push({
sid: json.items[i].sid,
attribute: json.items[i].attribute,
string: json.items[i].value_s,
number: json.items[i].value_d
});
}
else {
records.push({
sid: json.items[i].sid
})
}
}
//update store with data
store.add(records);
store.sync();
// only add to store if adding to search grid
if (!print_all) {
source.add({
key: payload.key,
type: payload.type,
description: payload.description,
criteria: payload.criteria,
atom: payload.atom,
n: store.collect('sid').length // get length of array for unique sids
});
source.sync();
}
console.log('TEST STORE HERE');
test_store = Ext.getStore('TestResults');
test_store.load();
test_store.sync();
console.log(test_store);
console.log(test_store.collect('sid').length)
}
// update grid store content
Ext.StoreMgr.get('Payload').load();
Ext.ComponentQuery.query('#searchGrid')[0].getStore().load();
}
For completeness, here is the data.items array output items:Array[2886]
which is equivalent count of unique items grouped by the attribute sid and finally the output of console.log(test_store.collect('sid').length), which gives the value from the PREVIOUS run of this: 3114...

Mongodb random sample selection but unique and different every time I ask for a new sample

I have Mongo Database, and I want to select a random sample of users, which I found a solution for in different places (including here).
Now, the hard part: How to get a unique and random set of users every time I make a request to mongo that doesn't correlate to the random sample I received in the (at least) last request.
Thanks!
One idea is to store the used up samples in an array and do your random query with the addition of the not-in-array-elements condition $nin. Below is sample code for demonstrating $nin, which you can edit and play around with on my Saturn Fiddle.
// Welcome to SaturnAPI!
// Start collaborating with MongoDB fiddles and accomplish more.
// Start your code below these comments.
// Create a new collection
var Posts = new Mongo.Collection(null);
//Insert some data
Posts.insert({
number: 1,
author: "Saturn Sam",
message: "Hello!"
});
Posts.insert({
number: 2,
author: "Saturn Sam2",
message: "Hello!"
});
Posts.insert({
number: 3,
author: "Saturn Sam3",
message: "Hello!"
});
// Returns all records
// Posts.find({}).fetch()
// Returns all records with `number` not equal to `2` or `1`
Posts.find({number: {$nin: [2, 1]}}).fetch()

Building a dynamic mongo query for meteor

I'm building an app that has clickable 'filters'; I'm creating a list of objects(?) that I want to pass to a mongo 'find', so that I can pull out listings if selected attributes match a certain score.
My data is structured like this (a snippet):
name: 'Entry One',
location: {
type: 'Point',
coordinates: [-5.654182,50.045414]
},
dogs: {
score: '1',
when: 'seasonal',
desc: 'Dogs allowed from October to April'
},
lifeguard: {
score: '1',
when: 'seasonal',
desc: 'A lifeguard hut is manned between April and October',
times: ''
},
cafe: {
score: '1',
name:'Lovely cafe',
open:'seasonal'
}, ...
My search variable is a list of objects (I think?) that I assign to a session variable. If I output this session var ('searchString') via JSON.stringify, it looks like this:
{"cafe":{"score":"1"},"dogs":{"score":"1"}}
I'd like to pass this to my mongo find so that it only lists entries that match these scores on these attributes, but it's returning zero results. Do I need to somehow make this an $and query?
Currently it looks like this:
Beaches.find(searchString);
Unfortunately as soon as I drop searchString into the find, I get zero results even if it's empty {}. (When it's just a find() the entries list fine, so the data itself is ok)
What am I doing wrong? I'm relatively new to mongo/meteor, so I apologise in advance if it's something stupidly obvious!
Don't stringify the query. Flatten the object instead. Example:
Beaches.find({
"cafe.score": 1,
"dogs.score": 1,
});