My question will seem quite simple, but I'm having a hard time understanding how certain queries work with ObjectBox and Flutter.
I have a List object. I would like to retrieve them all, but display only the first 5, sorted by ascending name.
I find many examples with filter queries in the documentation, but the getAll() does not allow to easily integrate the limit and I find very few examples. I think I'm doing it wrong. What is the best way to do this query ?
final boxListCoins = objectBoxDatabase.store.box<CoinsList>();
// final query = boxListCoins.getAll(); ?
final query = (boxListCoins.query()..order(CoinsList_.coinId, flags: Order.descending)).build();
query.limit = 5;
final coinsList = query.find();
I see that you want to get all your results but you want to display the first 5. So what I would do in your case is:
(1) get all your results(based on your query)
(2) then set another property equal to the top 5:)
** now you have a list of all your entries and you have your top 5 that you can display.
heres how I would code it up!
final boxListCoins = objectBoxDatabase.store.box<CoinsList>();
// final query = boxListCoins.getAll(); ?
final query = (boxListCoins.query()..order(CoinsList_.coinId, flags: Order.descending)).build();
// this is your list of all of the coins!
final coinsList = query.find();
// This is you list of the Top 5
var topFive = [];
if (coinsList.length >= 5) {
// take your top5 here :) also feel free to turn this iterable into what ever you want I just choose to keep it a list :)
topFive = coinsList.take(5).toList();
} else {
// if you length is not greater than 5 then you have your top automatically.
topFive = coinsList;
}
Hope this helps:)
Happy Coding, also I'm a developer #Pieces and if you need a place to store helpful code snippets, like this one, check it out here https://code.pieces.app/
I'm trying to test out Firebase to allow users to post comments using push. I want to display the data I retrieve with the following;
fbl.child('sell').limit(20).on("value", function(fbdata) {
// handle data display here
}
The problem is the data is returned in order of oldest to newest - I want it in reversed order. Can Firebase do this?
Since this answer was written, Firebase has added a feature that allows ordering by any child or by value. So there are now four ways to order data: by key, by value, by priority, or by the value of any named child. See this blog post that introduces the new ordering capabilities.
The basic approaches remain the same though:
1. Add a child property with the inverted timestamp and then order on that.
2. Read the children in ascending order and then invert them on the client.
Firebase supports retrieving child nodes of a collection in two ways:
by name
by priority
What you're getting now is by name, which happens to be chronological. That's no coincidence btw: when you push an item into a collection, the name is generated to ensure the children are ordered in this way. To quote the Firebase documentation for push:
The unique name generated by push() is prefixed with a client-generated timestamp so that the resulting list will be chronologically-sorted.
The Firebase guide on ordered data has this to say on the topic:
How Data is Ordered
By default, children at a Firebase node are sorted lexicographically by name. Using push() can generate child names that naturally sort chronologically, but many applications require their data to be sorted in other ways. Firebase lets developers specify the ordering of items in a list by specifying a custom priority for each item.
The simplest way to get the behavior you want is to also specify an always-decreasing priority when you add the item:
var ref = new Firebase('https://your.firebaseio.com/sell');
var item = ref.push();
item.setWithPriority(yourObject, 0 - Date.now());
Update
You'll also have to retrieve the children differently:
fbl.child('sell').startAt().limitToLast(20).on('child_added', function(fbdata) {
console.log(fbdata.exportVal());
})
In my test using on('child_added' ensures that the last few children added are returned in reverse chronological order. Using on('value' on the other hand, returns them in the order of their name.
Be sure to read the section "Reading ordered data", which explains the usage of the child_* events to retrieve (ordered) children.
A bin to demonstrate this: http://jsbin.com/nonawe/3/watch?js,console
Since firebase 2.0.x you can use limitLast() to achieve that:
fbl.child('sell').orderByValue().limitLast(20).on("value", function(fbdataSnapshot) {
// fbdataSnapshot is returned in the ascending order
// you will still need to order these 20 items in
// in a descending order
}
Here's a link to the announcement: More querying capabilities in Firebase
To augment Frank's answer, it's also possible to grab the most recent records--even if you haven't bothered to order them using priorities--by simply using endAt().limit(x) like this demo:
var fb = new Firebase(URL);
// listen for all changes and update
fb.endAt().limit(100).on('value', update);
// print the output of our array
function update(snap) {
var list = [];
snap.forEach(function(ss) {
var data = ss.val();
data['.priority'] = ss.getPriority();
data['.name'] = ss.name();
list.unshift(data);
});
// print/process the results...
}
Note that this is quite performant even up to perhaps a thousand records (assuming the payloads are small). For more robust usages, Frank's answer is authoritative and much more scalable.
This brute force can also be optimized to work with bigger data or more records by doing things like monitoring child_added/child_removed/child_moved events in lieu of value, and using a debounce to apply DOM updates in bulk instead of individually.
DOM updates, naturally, are a stinker regardless of the approach, once you get into the hundreds of elements, so the debounce approach (or a React.js solution, which is essentially an uber debounce) is a great tool to have.
There is really no way but seems we have the recyclerview we can have this
query=mCommentsReference.orderByChild("date_added");
query.keepSynced(true);
// Initialize Views
mRecyclerView = (RecyclerView) view.findViewById(R.id.recyclerView);
mManager = new LinearLayoutManager(getContext());
// mManager.setReverseLayout(false);
mManager.setReverseLayout(true);
mManager.setStackFromEnd(true);
mRecyclerView.setHasFixedSize(true);
mRecyclerView.setLayoutManager(mManager);
I have a date variable (long) and wanted to keep the newest items on top of the list. So what I did was:
Add a new long field 'dateInverse'
Add a new method called 'getDateInverse', which just returns: Long.MAX_VALUE - date;
Create my query with: .orderByChild("dateInverse")
Presto! :p
You are searching limitTolast(Int x) .This will give you the last "x" higher elements of your database (they are in ascending order) but they are the "x" higher elements
if you got in your database {10,300,150,240,2,24,220}
this method:
myFirebaseRef.orderByChild("highScore").limitToLast(4)
will retrive you : {150,220,240,300}
In Android there is a way to actually reverse the data in an Arraylist of objects through the Adapter. In my case I could not use the LayoutManager to reverse the results in descending order since I was using a horizontal Recyclerview to display the data. Setting the following parameters to the recyclerview messed up my UI experience:
llManager.setReverseLayout(true);
llManager.setStackFromEnd(true);
The only working way I found around this was through the BindViewHolder method of the RecyclerView adapter:
#Override
public void onBindViewHolder(final RecyclerView.ViewHolder holder, int position) {
final SuperPost superPost = superList.get(getItemCount() - position - 1);
}
Hope this answer will help all the devs out there who are struggling with this issue in Firebase.
Firebase: How to display a thread of items in reverse order with a limit for each request and an indicator for a "load more" button.
This will get the last 10 items of the list
FBRef.child("childName")
.limitToLast(loadMoreLimit) // loadMoreLimit = 10 for example
This will get the last 10 items. Grab the id of the last record in the list and save for the load more functionality. Next, convert the collection of objects into and an array and do a list.reverse().
LOAD MORE Functionality: The next call will do two things, it will get the next sequence of list items based on the reference id from the first request and give you an indicator if you need to display the "load more" button.
this.FBRef
.child("childName")
.endAt(null, lastThreadId) // Get this from the previous step
.limitToLast(loadMoreLimit+2)
You will need to strip the first and last item of this object collection. The first item is the reference to get this list. The last item is an indicator for the show more button.
I have a bunch of other logic that will keep everything clean. You will need to add this code only for the load more functionality.
list = snapObjectAsArray; // The list is an array from snapObject
lastItemId = key; // get the first key of the list
if (list.length < loadMoreLimit+1) {
lastItemId = false;
}
if (list.length > loadMoreLimit+1) {
list.pop();
}
if (list.length > loadMoreLimit) {
list.shift();
}
// Return the list.reverse() and lastItemId
// If lastItemId is an ID, it will be used for the next reference and a flag to show the "load more" button.
}
I'm using ReactFire for easy Firebase integration.
Basically, it helps me storing the datas into the component state, as an array. Then, all I have to use is the reverse() function (read more)
Here is how I achieve this :
import React, { Component, PropTypes } from 'react';
import ReactMixin from 'react-mixin';
import ReactFireMixin from 'reactfire';
import Firebase from '../../../utils/firebaseUtils'; // Firebase.initializeApp(config);
#ReactMixin.decorate(ReactFireMixin)
export default class Add extends Component {
constructor(args) {
super(args);
this.state = {
articles: []
};
}
componentWillMount() {
let ref = Firebase.database().ref('articles').orderByChild('insertDate').limitToLast(10);
this.bindAsArray(ref, 'articles'); // bind retrieved data to this.state.articles
}
render() {
return (
<div>
{
this.state.articles.reverse().map(function(article) {
return <div>{article.title}</div>
})
}
</div>
);
}
}
There is a better way. You should order by negative server timestamp. How to get negative server timestamp even offline? There is an hidden field which helps. Related snippet from documentation:
var offsetRef = new Firebase("https://<YOUR-FIREBASE-APP>.firebaseio.com/.info/serverTimeOffset");
offsetRef.on("value", function(snap) {
var offset = snap.val();
var estimatedServerTimeMs = new Date().getTime() + offset;
});
To add to Dave Vávra's answer, I use a negative timestamp as my sort_key like so
Setting
const timestamp = new Date().getTime();
const data = {
name: 'John Doe',
city: 'New York',
sort_key: timestamp * -1 // Gets the negative value of the timestamp
}
Getting
const ref = firebase.database().ref('business-images').child(id);
const query = ref.orderByChild('sort_key');
return $firebaseArray(query); // AngularFire function
This fetches all objects from newest to oldest. You can also $indexOn the sortKey to make it run even faster
I had this problem too, I found a very simple solution to this that doesn't involved manipulating the data in anyway. If you are rending the result to the DOM, in a list of some sort. You can use flexbox and setup a class to reverse the elements in their container.
.reverse {
display: flex;
flex-direction: column-reverse;
}
myarray.reverse(); or this.myitems = items.map(item => item).reverse();
I did this by prepend.
query.orderByChild('sell').limitToLast(4).on("value", function(snapshot){
snapshot.forEach(function (childSnapshot) {
// PREPEND
});
});
Someone has pointed out that there are 2 ways to do this:
Manipulate the data client-side
Make a query that will order the data
The easiest way that I have found to do this is to use option 1, but through a LinkedList. I just append each of the objects to the front of the stack. It is flexible enough to still allow the list to be used in a ListView or RecyclerView. This way even though they come in order oldest to newest, you can still view, or retrieve, newest to oldest.
You can add a column named orderColumn where you save time as
Long refrenceTime = "large future time";
Long currentTime = "currentTime";
Long order = refrenceTime - currentTime;
now save Long order in column named orderColumn and when you retrieve data
as orderBy(orderColumn) you will get what you need.
just use reverse() on the array , suppose if you are storing the values to an array items[] then do a this.items.reverse()
ref.subscribe(snapshots => {
this.loading.dismiss();
this.items = [];
snapshots.forEach(snapshot => {
this.items.push(snapshot);
});
**this.items.reverse();**
},
For me it was limitToLast that worked. I also found out that limitLast is NOT a function:)
const query = messagesRef.orderBy('createdAt', 'asc').limitToLast(25);
The above is what worked for me.
PRINT in reverse order
Let's think outside the box... If your information will be printed directly into user's screen (without any content that needs to be modified in a consecutive order, like a sum or something), simply print from bottom to top.
So, instead of inserting each new block of content to the end of the print space (A += B), add that block to the beginning (A = B+A).
If you'll include the elements as a consecutive ordered list, the DOM can put the numbers for you if you insert each element as a List Item (<li>) inside an Ordered Lists (<ol>).
This way you save space from your database, avoiding unnecesary reversed data.
I am new to jayData and am trying to filter on an entity set. The filter needs to perform an complex evaluation beyond what I saw in the samples.
Here is a working sample of what I am trying to accomplish (the listView line isn't and is just there to show what I plan to do with the data):
function () {
var weekday = moment().isoWeekday()-1;
console.log(weekday);
var de = leagueDB.DailyEvents.toArray(function (events) {
console.log(events);
var filtered = [];
for (var e = 0; e < events.length;e++) {
console.log(events[e]);
console.log(events[e].RecurrenceRule);
var rule = RRule.fromString(events[e].RecurrenceRule);
var ruleOptions = rule.options.byweekday;
var isDay = ruleOptions.indexOf(weekday);
console.log(ruleOptions, isDay);
if(isDay =! -1)
{
filtered.push(events[e]);
}
}
$("#listView").kendoListView({dataSource:filtered});
});
Basically it is just evaluating a recurring rule string to see if the current day meets that criteria, if so add that event to the list for viewing.
But it blows up when I try to do this:
eventListLocal:leagueDB.DailyEvents.filter(function(e){
console.log("The Weekday is:"+viewModel.weekday);
console.log(e);
console.log("The recurrence rule is:"+e.RecurrenceRule);
var rruleOptions = viewModel.rruleOptions(e.RecurrenceRule);
if (rruleOptions !== -1) {
return true;
}
}).asKendoDataSource()
The error that is generating is:
Exception: Unable to resolve type:undefined
The thing is it seems to be occurring on "e" and the console logs like the event is not being passed in. However, I am not seeing a list either. In short I am really confused as to what is going on.
Any help would be appreciated.
Thanks,
You can't write filter expressions such as this.
When you write .filter(...), jaydata will parse your expression and then it will generate filter for underlying provider, for example where for webSql and $filter for oDataProvider.
Both JayData expression parser and the data provider itself should understand your filter.
Your filter is not suitable for this approach, because most of your codes are not familiar for jaydata expression parser and the underlying data provider, for example your console.log etc.
You can simplify your filter, or you should load all your data into an array, and then you can use filter method of array itself, there, you can write any filter you like, and your filter will work. Of course this has performance issue in some scenarios when your data set is large.
Read more on http://jaydata.org/tutorials/entityexpressions-the-heart-of-jaydata
I'm profiling my application locally (using the Dev server) to get more information about how GAE works. My tests are comparing the common full Entity query and the Projection Query. In my tests both queries do the same query, but the Projection is specified with 2 properties. The test kind has 100 properties, all with the same value for each Entity, with a total of 10 Entities. An image with the Datastore viewer and the Appstats generated data is shown bellow. In the Appstats image, Request 4 is a memcache flush, Request 3 is the test database creation (it was already created, so no costs here), Request 2 is the full Entity query and Request 1 is the projection query.
I'm surprised that both queries resulted in the same amount of reads. My guess is that small and read operations and being reported the same by Appstats. If this is the case, I want to separate them in the reports. That's the queries related functions:
// Full Entity Query
public ReturnCodes doQuery() {
DatastoreService dataStore = DatastoreServiceFactory.getDatastoreService();
for(int i = 0; i < numIters; ++i) {
Filter filter = new FilterPredicate(DBCreation.PROPERTY_NAME_PREFIX + i,
FilterOperator.NOT_EQUAL, i);
Query query = new Query(DBCreation.ENTITY_NAME).setFilter(filter);
PreparedQuery prepQuery = dataStore.prepare(query);
Iterable<Entity> results = prepQuery.asIterable();
for(Entity result : results) {
log.info(result.toString());
}
}
return ReturnCodes.SUCCESS;
}
// Projection Query
public ReturnCodes doQuery() {
DatastoreService dataStore = DatastoreServiceFactory.getDatastoreService();
for(int i = 0; i < numIters; ++i) {
String projectionPropName = DBCreation.PROPERTY_NAME_PREFIX + i;
Filter filter = new FilterPredicate(DBCreation.PROPERTY_NAME_PREFIX + i,
FilterOperator.NOT_EQUAL, i);
Query query = new Query(DBCreation.ENTITY_NAME).setFilter(filter);
query.addProjection(new PropertyProjection(DBCreation.PROPERTY_NAME_PREFIX + 0, Integer.class));
query.addProjection(new PropertyProjection(DBCreation.PROPERTY_NAME_PREFIX + 1, Integer.class));
PreparedQuery prepQuery = dataStore.prepare(query);
Iterable<Entity> results = prepQuery.asIterable();
for(Entity result : results) {
log.info(result.toString());
}
}
return ReturnCodes.SUCCESS;
}
Any ideas?
EDIT: To get a better overview of the problem I have created another test, which do the same query but uses the keys only query instead. For this case, Appstats is correctly showing DATASTORE_SMALL operations in the report. I'm still pretty confused about the behavior of the projection query which should also be reporting DATASTORE_SMALL operations. Please help!
[I wrote the go port of appstats, so this is based on my experience and recollection.]
My guess is this is a bug in appstats, which is a relatively unmaintained program. Projection queries are new, so appstats may not be aware of them, and treats them as normal read queries.
For some background, calculating costs is difficult. For write ops, the cost are returned with the results, as they must be, since the app has no way of knowing what changed (which is where the write costs happen). For reads and small ops, however, there is a formula to calculate the cost. Each appstats implementation (python, java, go) must implement this calculation, including reflection or whatever is needed over the request object to determine what's going on. The APIs for doing this are not entirely obvious, and there's lots of little things, so it's easy to get it wrong, and annoying to get it right.
I have a sorted mutable array of a class called Topic. The topics represent a an array of Publications. I present the topics in a table, and periodically fetch new publications from a web service. When a new publication arrives, I'd like to add to the table with an animation.
What's bothering me is the computational work I need to do to add into this array, and answer the correct index path. Can someone suggest a more direct way than this:
// add a publication to the topic model. if the publication has a new topic, answer
// the index path of the new topic
- (NSIndexPath *)addPublication:(Publication *)pub {
// first a search to fit into an existing topic
NSNumber *topicId = [pub valueForKey:#"topic_id"];
for (Topic *topic in self.topics) {
if ([topicId isEqualToNumber:[topic valueForKey:"id"]]) {
// this publication is part of an existing topic, no new index path
[topic addPublication:pub];
return nil;
}
}
// the publication must have a new topic, add a new topic (and therefore a new row)
Topic *topic = [[Topic alloc] initWithPublication:publication];
[self.topics addObject:topic];
// sort it into position
[self.topics sortUsingSelector:#selector(compareToTopic:)];
// oh no, we want to return an index path, but where did it sort to?
// yikes, another search!
NSInteger row = [self.topics indexOfObject:topic];
return [NSIndexPath indexPathForRow:row inSection:0];
}
// call this in a loop for all the publications I fetch from the server,
// collect the index paths for table animations
// so much computation, poor user's phone is going to melt!
There's no getting around the first search, I guess. But is there some more efficient way to add a new thing to an array, maintaining a sort and remembering where it got placed?
It's pretty straightforward to insert a value into a sorted list. Think about how you would insert the number "3" into the list "1, 2, 7, 9", for instance. You want to do exactly the same thing.
Loop through the array by index, using a for loop.
For each object, use compareToTopic: to compare it to the object you want to insert.
When you find the appropriate index to insert at, use -[NSArray insertObject:atIndex:] to insert it.
Then return an NSIndexPath with that index.
Edit: and, as the other answers point out, a binary search would be faster -- but definitely trickier to get right.
This is almost certainly not an issue; NSArrays are actually hashes, and search is a lot faster than it would be for a true array. How many topics can you possibly have anyways?
Still, if you measure the performance and find it poor, you could look into using a B-tree; Kurt Revis commented below with a link to a similar structure (a binary heap) in Core Foundation: CFBinaryHeap.
Another option (which would also need to be measured) might be to do the comparison as you walk the array the first time; you can mark the spot and do the insertion directly:
NSUInteger insertIndex = 0;
NSComparisonResult prevOrder = NSOrderedDescending;
for (Topic *topic in self.topics) {
NSComparisonResult order = [topicId compareToTopic:topic];
if (NSOrderedSame == order) {
// this publication is part of an existing topic, no new index path
[topic addPublication:pub];
return nil;
}
else if( prevOrder == NSOrderedDescending &&
order == NSOrderedAscending )
{
break;
}
insertIndex++;
prevOrder = order;
}
Please note that I haven't tested this, sorry.
I'm not sure this is actually better or faster than the way you've written it, though.
Don't worry about the work the computer is doing unless it's demonstrably doing it too slowly.
What you have done is correct I guess. There's another way. You can write your own binary search implementation method. (Which has only few lines of code). And you can retrieve the index where the new object should fit in. And add the new object to the required index using insertObject:atIndex: method.