CS193P Smashtag Popularity Extra Task #3 - Get only new tweets using "IN" keyword - swift

I'm working on Stanford CS193p's SmashTag Popularity Mentions assignment (asst. #5) and I've got everything working well. I'm working on Extra Task #3:
Loading up a lot of data by querying for an existing instance in the database, then inserting if not found over and over again, one by one (like we did in lecture), can be pretty poor performing. Enhance your application to make this more efficient by checking for the existence of a pile of things you want to be in the database in one query (then only creating the ones that don’t exist). The predicate operator IN might be of value here.
I've managed to do something like this, but not using the IN predicate operator! This is inside the perform block of my updateDatabase(with newTweets:) method: (I've called my core data entity CDTweet
if let databaseTweets = try? globalContext!.fetch(NSFetchRequest<CDTweet>(entityName: "CDTweet")) {
let databaseIDs = databaseTweets.map { $0.id! }
for twitterInfo in newTweets where !databaseIDs.contains(twitterInfo.id) {
_ = CDTweet.createCDTweet(with: twitterInfo, into: globalContext!)
}
}
As you can see, I get all the tweets in the database, get their IDs, and then only create a new tweet for internet-fetched-Twitter.Tweets whose IDs are not in the array of database tweets.
This appears to function properly (i.e., create only the new tweets), but I am very curious how the instructor imagined it would work with the IN predicate operator. Has anyone done this and can lend some insight?
Note: A number of solutions I've seen have a core data entity for the Search Term (usually called Query). I don't have this, only entities for Tweet and Mention, and I have everything working fine.

You need something like this (i assume searchIDs is an array of values you are looking for):
var searchIDs = // ... an array of IDs you are searching for
var fetchRequest = NSFetchRequest<CDTweet>(entityName: "CDTweet")
let predicate = NSPredicate(format: "id IN %#", searchIDs)
fetchRequest.predicate = predicate
databaseTweets = try? globalContext!.fetch(fetchRequest) {
// here you should only get all entries with IDs in the newTweets array
}
Details about predicates can be found here, about predicate syntax especially here

Related

Swift/FireStore - Appending/updating elements in an array

Ok, so this feature is semi-working:
API.FIRESTORE_DOCUMENT_USERID(userID: userID).updateData(["user_rating":FieldValue.arrayUnion([averageRating])])
As you can tell, I'm trying to update the user_rating field and pass in the average rating. However, since the average rating can be anywhere from 1-5, including decimals/doubles, there is a likelihood that the average rating will already exist i.e. 1.5, 2.5, 2.7 etc. However, I cannot seem too append the same value twice. If I change the rating, then sure I can append. As far as I'm aware, each entry in an array has it's own ID? Any way I can allow for duplicates?
Thank you.
Found the solution, you must first read the existing data in the array, then append whatever data you wish then once again update the data, like so:
[YOUR_REFERENCE].getDocument { (querySnapshot, error) in
if let error = error {
print(error.localizedDescription)
} else {
var array:[THE_DATA_TYPE] = querySnapshot?.get("[FIELD_NAME]") as [THE_DATA_TYPE]
array.append([DATA_YOU_WISH_TO_APPEND])
querySnapshot?.reference.updateData(["[FIELD_NAME]" : array])
}
}
I've made it generic in the event anyone ever just wishes to copy and paste.

Getting ElasticSearch document fields inside of loaded records in searchkick

Is it possible to get ElasticSearch document fields inside of loaded AR records?
Here is a gist that illustrates what I mean: https://gist.github.com/allomov/39c30905e94c646fb11637b45f43445d
In this case I want to avoid additional computation of total_price after getting response from ES. The solution that I currently see is to include the relationship and run total_price computation for each record, which is not so optimal way to perform this operation, as I see it.
result = Product.search("test", includes: :product_components).response
products_with_total_prices = result.map do |product|
{
product: product
total_price: product.product_components.map(&:price).compact.sum
}
end
Could you please tell if it is possible to mix ES document fields into AR loaded record?
As far as I'm aware it isn't possible to get a response that merges the document fields into the loaded record.
Usually I prefer to completely rely on the data in the indexed document where possible (using load: false as a search option), and only load the AR record(s) as a second step if necessary. For example:
result = Product.search("test", load: false).response
# If you also need AR records, could do something like:
product_ids = result.map(&:id)
products_by_id = {}
Product.where(id: product_ids).find_each do |ar_product|
products_by_id[ar_product.id] = ar_product
end
merged_result = result.map do |es_product|
es_product[:ar_product] = products_by_id[es_product.id]}
end
Additionally, it may be helpful to retrieve the document stored in the ES index for a specific record, which I would normally do by defining the following method in your Product class:
def es_document
return nil unless doc = Product.search_index.retrieve(self).presence
Hashie::Mash.new doc
end
You can use select: true and the with_hit method to get the record and the search document together. For your example:
result = Product.search("test", select: true)
products_with_total_prices =
result.with_hit.map do |product, hit|
{
product: product,
total_price: hit["_source"]["total_price"]
}
end

Query latest entries using Firebase in Swift

I'm new using Firebase and I can't find how to do what sounds really simple to do : list all the latest entries of my database.
Here is a screenshot of what my database looks like :
So, I'm trying to list the latest entries like that :
// picturesRef = FIRDatabase.database().reference().child("pictures")
let _ = self.picturesRef.queryOrdered(byChild: "createdTime").queryLimited(toLast: 50).observe(.value, with: { snapshot in
// Stocking the result into picturesArr array
for elem in picturesArr {
print(elem.createdTime)
}
})
And right now, when I'm displaying the createdTime value of each item, I have something like :
1484738582.0
1484000086.0
1484738279.0
1484734358.0
1484625525.0
1484728677.0
Which doesn't seem to be ordered from the oldest entry to the newest one...
Also, when I replace "createdTime" in the query by "fieldThatDoesntExist", I have the exact same result.
Anyone would know where did I do something wrong in the code ? Thanks in advance !
The query returns the items in the correct order. But most likely (the relevant code seems to be missing from your question) you're losing that order when you convert the snapshot to a dictionary (which is unordered by definition).
To keep the items in the correct order, iterate over snapshot.children:
let picturesRef = FIRDatabase.database().reference().child("pictures")
let _ = picturesRef
.queryOrdered(byChild: "createdTime")
.queryLimited(toLast: 50)
.observe(.value, with: { snapshot in
for child in snapshot.children {
print(child.key)
print(child.child("createdTime").value
}
})
Also see:
Firebase snapshot.key not returning actual key?
Firebase access keys in queryOrderBy
Retrieving Data using Firebase Swift
Iterate over snapshot children in Swift (Firebase) (I usually try to avoid allObjects, but it's fine too)
Actualy, after sorting the array returned by the query with this :
picturesArr.sort(by: {$0.createdTime > $1.createdTime})
I've figured out that the query returns the entries that I'm looking for but not sorted.
It looks a bit wierd to me, maybe someone knows why or even better, how to get the result already sorted ?

Querying in Firebase by child of child

I have a structure of objects in Firebase looking like this:
-KBP27k4iOTT2m873xSE
categories
Geography: true
Oceania: true
correctanswer: "Yaren (de facto)"
languages: "English"
question: "Nauru"
questiontype: "Text"
wronganswer1: "Majuro"
wronganswer2: "Mata-Utu"
wronganswer3: "Suva"
I'm trying to find objects by categories, so for instance I want all objects which has the category set to "Oceania".
I'm using Swift and I can't really seem to grasp the concept of how to query the data.
My query right now looks like this:
ref.queryEqualToValue("", childKey: "categories").queryOrderedByChild("Oceania")
Where ref is the reference to Firebase in that specific path.
However whatever I've tried I keep getting ALL data returned instead of the objects with category Oceania only.
My data is structured like this: baseurl/questions/
As you can see in the object example one question can have multiple categories added, so from what I've understood it's best to have a reference to the categories inside your objects.
I could change my structure to baseurl/questions/oceania/uniqueids/, but then I would get multiple entries covering the same data, but with different uniqueid, because the question would be present under both the categories oceania and geography.
By using the structure baseurl/questions/oceania/ and baseurl/questions/geography I could also just add unique ids under oceania and geography that points to a specific unique id inside baseurl/questions/uniqueids instead, but that would mean I'd have to keep track of a lot of references. Making a relations table so to speak.
I wonder if that's the way to go or? Should I restructure my data? The app isn't in production yet, so it's possible to restructure the data completely with no bigger consequences, other than I'd have to rewrite my code, that pushes data to Firebase.
Let me know, if all of this doesn't make sense and sorry for the wall of text :-)
Adding some additional code to Tim's answer for future reference.
Just use a deep query. The parent object key is not what is queried so it's 'ignored'. It doesn't matter whether it's a key generated by autoId or a dinosaur name - the query is on the child objects and the parent (key) is returned in snapshot.key.
Based on your Firebase structure, this will retrieve each child nodes where Oceania is true, one at a time:
let questionsRef = Firebase(url:"https://baseurl/questions")
questionsRef.queryOrderedByChild("categories/Oceania").queryEqualToValue(true)
.observeEventType(.ChildAdded, withBlock: { snapshot in
print(snapshot)
})
Edit: A question came up about loading all of the values at once (.value) instead of one at at time (.childAdded)
let questionsRef = Firebase(url:"https://baseurl/questions")
questionsRef.queryOrderedByChild("categories/Oceania").queryEqualToValue(true)
.observeSingleEventOfType(.Value, withBlock: { snapshot in
print(snapshot)
})
Results in (my Firebase structure is a little different but you get the idea) uid_1 did not have Oceania = true so it was omitted from the query
results.
Snap (users) {
"uid_0" = {
categories = {
Oceania = 1;
};
email = "dude#thing.com";
"first_name" = Bill;
};
"uid_2" = {
categories = {
Oceania = 1;
};
"first_name" = Peter;
};
}
I think this should work:
ref.queryOrderedByChild("categories/Oceania").queryEqualToValue(true)

Fetch only tasks with a certain value

I'm making a to do app that has tasks for specific teams using CloudKit as my backend. I've set it up so that the first screen you see is the teams list. When you select the team, I want to just see all the tasks related to that specific team. I've gotten the app to save and fetch tasks but all of the tasks appear under every team.
I am very new to CloudKit (I used Parse before) and also new to predicates. Currently I am passing the team recordID as a string using the prepareForSegue function to my Tasks View Controller. In Parse, that was all I needed to get just the tasks for that team. I can see that the saving is working correctly when I look at CloudKit Dashboard (every task has the correct team recordID in its record).
I know the problem is somewhere in the next bit of code but I am unsure of how to go about doing it.
func loadTasks() {
tasks = [CKRecord]()
let query = CKQuery(recordType: "Tasks", predicate: NSPredicate(format: "TRUEPREDICATE", argumentArray: nil))
publicData.performQuery(query, inZoneWithID: nil) { (results:[CKRecord]?, error:NSError?) -> Void in
if let tasks = results {
self.tasks = tasks
dispatch_async(dispatch_get_main_queue(), { () -> Void in
self.tableView.reloadData()
})
}
}
self.refreshControl?.endRefreshing()
}
So I think I'm asking how to use predicates to filter the data to only show the tasks for the selected team. I only have what I have currently because I followed a tutorial.
Instead of passing the current predicate, use something along the lines of:
let predicate = NSPredicate(format: "team == %#", team)
let query = CKQuery(recordType: "Tasks", predicate: predicate)
Where team is the relevant team, which is likely to be a CKReference.
Michael,
A predicate is like a regular expression. Are you familiar with regular expressions? If you are, then you find predicates a piece of cake.
All your saying here is I want to you to return all the records you have in the "Tasks" database in which the team field equals the value in the team variable.
Your name sake Michael answered the question; in honestly he deserves some kudos here.
Hope this short explanation helps!
I figured it out.
I don't quite understand it still but it works. If someone is willing to explain this, I would be grateful.
let query = CKQuery(recordType: "Tasks", predicate: NSPredicate(format: "team==%#", self.team))