I've never really gotten the nuances of async operations so time and again, I get stymied. And I just can't figure it out.
I'm trying to do some very simple web scraping.
My local volleyball association has a page (verbose HTML, not responsive, not mobile-friendly, yaddah, yaddah, yaddah) which shows the refs assigned to each game of the season. I'm trying to write a silly little app which will scrape that page (no API, no direct access to db, etc.) and display the data in a grouped table. The first group will show today's matches (time, home team, away team). The second group will show tomorrow's matches. Third group shows the entire season's matches.
Using code I found elsewhere, my viewDidLoad loads the page, scrapes the data and parses it into an array. Once I've parsed the data, I have three arrays: today, tomorrow, and matches, all are [Match].
override func viewDidLoad() {
super.viewDidLoad()
let url = URL(string: urlString)!
let request = NSMutableURLRequest(url: url)
let task = URLSession.shared.dataTask(with: request as URLRequest) {
data, response, error in
if let error = error {
print (error)
} else {
if let unwrappedData = data {
// scrape, scrape, parse, parse
matchRow = ...
self.matches.append(matchRow)
if matchRow.date == todaysDate {
self.today.append(matchRow)
} else if matchRow.date == tomorrowsDate {
self.tomorrow.append(matchRow)
}
}
}
}
task.resume()
}
As I'm sure is no surprise to anyone who understands async operations, my table is empty. I've checked and I see the the data is there and properly parsed, etc. But I can't for the life of me figure out how get the data in my table. The way I have it now, the data is not ready when numberOfSections or numberOfRowsInSection is called.
I've found the Ray Wenderlich tutorial on URLSession and I also have a Udemy course (Rob Percival) that builds an app to get the weather using web scraping, but in both those instances, the app starts and waits for user input before going out to the web to get the data. I want my app to get the data immediately upon launch, without user interaction. But I just can't figure out what changes I need to make so that those examples work with my program.
Help, please.
You can simply reload the tableviews once the data arrays are getting populated from the URLSession completion block. Have you tried that. Sample snippet may be like the one follows.
let task = URLSession.shared.dataTask(with: request as URLRequest) {
data, response, error in
if let error = error {
print (error)
} else {
if let unwrappedData = data {
// scrape, scrape, parse, parse
matchRow = ...
self.matches.append(matchRow)
if matchRow.date == todaysDate {
self.today.append(matchRow)
} else if matchRow.date == tomorrowsDate {
self.tomorrow.append(matchRow)
}
}
DispatchQueue.main.async { [weak self] in
self?.todayTableView.reloadData()
self?.tomorrowTableView.reloadData()
}
}
}
Related
I'm trying to create a feed that should include posts made by people a user follows, and I'm using following function to get posts for user feed:
func observePosts(for userID: String, completion: #escaping ([UserPost], String) -> Void) {
let databaseKey = "Posts/\(userID)/"
databaseRef.child(databaseKey).queryOrderedByKey().queryLimited(toLast: 12).observe(.value) { snapshot in
var retrievedPosts = [UserPost]()
var lastSeenPostKey = ""
let dispatchGroup = DispatchGroup()
for snapshotChild in snapshot.children.reversed() {
guard let postSnapshot = snapshotChild as? DataSnapshot,
let postDictionary = postSnapshot.value as? [String: Any]
else {
print("Couldn't cast snapshot as DataSnapshot")
return
}
dispatchGroup.enter()
lastSeenPostKey = postSnapshot.key
do {
let jsonData = try JSONSerialization.data(withJSONObject: postDictionary as Any)
let decodedPost = try JSONDecoder().decode(UserPost.self, from: jsonData)
UserService.shared.observeUser(for: decodedPost.userID) { postAuthor in
decodedPost.author = postAuthor
retrievedPosts.append(decodedPost)
dispatchGroup.leave()
}
} catch {
print("Couldn't create UserPost from postDictionary")
}
}
dispatchGroup.notify(queue: .main) {
completion(retrievedPosts, lastSeenPostKey)
}
}
}
First step I'm getting user feed posts with .observe function and after that I'm iterating through every received post calling a UserService.shared.observeUser method that observes user data for every post to set post author's profile picture and username.
The problem with my method is that I'm using DispatchGroup's .enter and .leave methods to notify once every observe function got user data so that I could call completion block in which I'm reloading tableView.
Sure it works fine first time but since it's observe function it's called every time there's a change to user so once user updates his data (profile pic, usernama etc.) it crashes on dispatchGroup.leave() line since it's not balanced with dispatchGroup.enter()
What would be a correct approach to solve this issue?
I've been trying to solve this problem for 3 days searching all the internet, and only advice I found is to fetch it one time using observeSingleValue() instead of observing, but I really need to observe user profile data to keep profile pics and other necessary data up to date.
ok so i've been trying to get this problem figured out for 2 days now, hoping someone can help.
quick background, i’m making an api request for data. using a function that calls a service function i made. now everything works good on the first load, collectionview loads fine. at some point i run another call for more data. accept now i get a URL Error.
This doesn’t work on my iphone, but does work perfectly on simulator, so not sure what it could be.
heres the service function that makes the api request:
func fetchYoutubeData(interest: String, maxResult: Int, pageToken: String, completion: #escaping(Result<Youtube, WHError>) -> Void) {
let urlString = baseYoutubeURL+interest+youtubeAPIKey+"&maxResults=\(maxResult)&pageToken=\(pageToken)"
guard let url = URL(string: urlString) else {
completion(.failure(.URLError))
return
}
let session = URLSession.shared
let task = session.dataTask(with: url) { (data, _, error) in
if let _ = error {
completion(.failure(.DataError))
return
}
guard let data = data else { return }
do {
let result = try JSONDecoder().decode(Youtube.self, from: data)
print(result)
completion(.success(result))
} catch {
completion(.failure(.JSONError))
}
}
task.resume()
}
heres my controller function that calls service and handles the data on completion:
func fetchNewData(maxResult: Int, pageToken: String) {
guard let interest = self.interest.text else { return }
print(pageToken)
NetworkServices.shared.fetchYoutubeData(interest: interest, maxResult: maxResult, pageToken: pageToken) { [unowned self] (result) in
switch result {
case .success(let youtubeGroup):
let items = youtubeGroup.items
self.youtubeData.items.append(contentsOf: items)
DispatchQueue.main.async {
self.horizontalCollectionView.reloadData()
}
case .failure(let error):
print("DOES GET ERROR")
print(error)
}
}
}
again it works perfectly on simulator, but not on my device, i can get first call to work, but after that, once i use pageToken to get more data, i get a URL Error.
any help would be seriously appreciated
so of course like alot of things , it's something small I missed. I'll keep the question here, incase someone finds themself in a similar situation.
the problem was in the first request I reformat the "interest" to remove spaces from the string and replace them with +. you need to do this for the Youtube API query string that you provide.
i did this in the first request, but for my query for additional data, I forgot to reformat the request in the separate call.
easy fix thankfully
I added "App Transport Security Settings" to the Info.plist file then selected "Allow Arbitrary Loads" and set that value to "YES" and that worked. My assumption is that there was an issue with the security cert on the client development environment that was preventing the app from making calls over HTTPS.
And never fear, I only allow arbitrary loads in the dev and qa environments.
I am new to Realm DataBase and I need a way to read data from realmCloud, but from two different app projects. The way I have tried to implement this is by using query-synced realm. At the moment I'm using a singe realm user to write the data in one app, and the same realm user to read data from another app. The problem is that making a query from the second app(the one used for reading) doesn't return any realm objects ( I have also noticed that user identifier is different from the first one, and also the user permissions are nil.
I have tried setting permissions directly from RealmStudio since documentation is not precise on how to set them from code
func openRealm() {
do {
realm = try Realm(configuration: SyncUser.current!.configuration())
let queryResults = realm.objects(*className*.self)
let syncSubscription = queryResults.subscribe()
let notificationToken = queryResults.observe() { [weak self] (changes) in
switch (changes) {
case .initial: print(queryResults)
case .error(let error): print(error)
default: print("default")
}
}
for token in queryResults {
print(token.tokenString)
}
syncSubscription.unsubscribe()
notificationToken.invalidate()
} catch {
print(error)
}
}
This function prints the data in one app project, but used in another app project with the same user logged in, and the same classFile referenced in the project, it does not. (note that SyncUser.current.identifier is different also
There are a couple of issues.
Some of these calls are asynchronous and the code in your question is going out of scope before the data is sync'd (retreived). The bottom line is code is faster than the internet and you need to design the flow of the app around async calls; don't try to work with the data until it's available.
For example
let notificationToken = queryResults.observe() { [weak self] (changes) in
//here is where results are fully populated
}
// this code may run before results are populated //
for token in queryResults {
print(token.tokenString)
}
Also, let notificationToken is a local var and goes out of scope before the results are populated as well.
These issues are super easy to fix. First is to keep the notification token alive while waiting for results to be populated and the second is to work with the results inside the closure, as that's when they are valid.
var notificationToken: NotificationToken? = nil //a class var
func openRealm() {
do {
let config = SyncUser.current?.configuration()
let realm = try Realm(configuration: config!)
let queryResults = realm.objects(Project.self)
let syncSubscription = queryResults.subscribe(named: "my-projects")
self.notificationToken = queryResults.observe() { changes in
switch changes {
case .initial:
print("notification: initial results are populated")
queryResults.forEach { print($0) }
case .update(_, let deletions, let insertions, let modifications):
print("notification: results, inserted, deleteed or modified")
insertions.forEach { print($0) } //or mods or dels
case .error(let error):
fatalError("\(error)")
}
}
} catch {
print(error)
}
}
deinit {
self.notificationToken?.invalidate()
}
The other advantage of keeping that token (and its corresponding code) alive is when there are further changes, your app will be notified. So if another project is added for example, the code in the 'changes' section will run and display that change.
I'm testing out the Kentico Cloud Swift SDK to return some 'article' content types (I have created two of them and they are published).
I am using the Boilerplate code as described here:
The result I get is : [Kentico Cloud] Getting items action has succeeded. Received nil items.
My code:
let client = DeliveryClient.init(projectId: <project id>, previewApiKey: <preview key>, secureApiKey: <secure key>, enableDebugLogging: true)
func getArticles(){
// Note: Using "items" as custom query returns all content items,
// but to map them to a single model, a filter is needed.
let customQuery = "items?system.type=article"
// More about strongly-typed models https://github.com/Kentico/cloud-sdk-swift#using-strongly-typed-models
client.getItems(modelType: Article.self, customQuery: customQuery) { (isSuccess, itemsResponse, error) in
if isSuccess {
// We get here and itemsResponse != nil but items == nil
if let articles = itemsResponse?.items {
for article in articles {
}
}
} else {
if let error = error {
print(error)
}
}
}
}
I believe this error message would appear before ObjectMapper is triggered to convert the JSON into Article objects. I could be wrong though.
Anyone have any ideas?
UPDATE
Interestingly, if I request a single article object like so ...
client.getItem(modelType: Article.self, itemName: <codename>) { (isSuccess, itemResponse, error) in
if isSuccess {
if let article = itemResponse?.item {
// Use your item here
}
} else {
if let error = error {
print(error)
}
}
}
... then it works. I get the Article object. It's just asking for all of the articles that fails.
I'm going to investigate the issue later today, however, from your description, it might be caused by the Delivery API item readiness delay - the project was not fully synced with the delivery API yet. After the publishing/unpublishing item or creating/generating the project, there might be a small delay in processing messages by Delivery API which could cause unavailability of item. This delay might be variable - from my experience, it may vary from a couple of seconds to 2-3 minutes. Nevertheless, I'm going to check it just to be sure. I'll keep you updated.
Edit: I'm pretty sure the project was not synced and processed on the Delivery API at the time you were requested the items. The API returned 200, which caused isSuccess in the callback to be true, however, there might have been none or just a subset of items available - I've reproduced this behavior (screenshot below), although it's by design (the content/messages in Event Hub must be processed asynchronously).
I've also suggested the improvement for Kentico Cloud's documentation to mention/explain the possible delay caused by processing events queue messages from Event Hubs.
Just to be sure - could you try it again with your getArticles custom query?
Edit2: Back to your question about the ObjectMapper. This is not an error just a debug message, however, there shouldn't be probably nil but 0 (zero) in the debug message. This message came from:
private func sendGetItemsRequest<T>(url: String, completionHandler: #escaping (Bool, ItemsResponse<T>?, Error?) -> ()) where T: Mappable {
sessionManager.request(url, headers: self.headers).responseObject { (response: DataResponse<ItemsResponse<T>>) in
switch response.result {
case .success:
if let value = response.result.value {
let deliveryItems = value
if self.isDebugLoggingEnabled {
print("[Kentico Cloud] Getting items action has succeeded. Received \(String(describing: deliveryItems.items?.count)) items.")
}
completionHandler(true, deliveryItems, nil)
}
case .failure(let error):
if self.isDebugLoggingEnabled {
print("[Kentico Cloud] Getting items action has failed. Check requested URL: \(url)")
}
completionHandler(false, nil, error)
}
}
}
Ok. This is very weird. After checking the API by requesting an individual item (see the update in the post above), and getting a result (woot). It now seems the original code (unchanged) now works.
I'm wondering if it takes a while for the data to propagate and be available in the API?
Who knows. Weird.
Sorry, newbie here and Ive read extensively about completion handlers, dispatch queues and groups but I just can't get my head around this.
My app loads an array of Google Place IDs and then wants to query Google to get full details on each place. The problem is, due to async processing the Google Lookup Place returns immediately and the callback happens much further down the line so whats the "proper way" to know when the last bit of data has come in for my inquiries because the function ends almost immedately ?
Code is attached. Thanks in advance.
func testFunc() {
let googlePlaceIDs = ["ChIJ5fTXDP8MK4cRjIKzek6L6NM", "ChIJ9Wd6mGYGK4cRiWd0_bkohHg", "ChIJaeXT08ASK4cRkCGpGgzYpu8", "ChIJkRkS4BapK4cRXCT8-SJxNDI", "ChIJ3wDV_2zX5IkRtd0hg2i1LhE", "ChIJb4wUsI5w44kRnERe7ywQaJA"]
let placesClient = GMSPlacesClient()
for placeID in googlePlaceIDs {
placesClient.lookUpPlaceID(placeID, callback: { (place, error) in
if let error = error {
print("lookup place id query error: \(error.localizedDescription)")
return
}
guard let place = place else {
print("No place details for \(placeID)")
return
}
print("Place Name = \(place.name)")
})
}
print("Done")
}