We are having java application which consumes salesforce partner.wsdl. We login to salesforce instance, then we get metadata for all the objects and we cache it. As salesforce objects become more we are seeing more time in getting metadata and cache it for first call.
What is the best way we can reduce this time, even if more objects are introduced in salesforce?
Is there any soap api call I can make to get metadata only for the object and its dependencies?
do we need to use only describeSobject to get these information.?
Cache the SF responses, flush the cache once a day, not with every request?
Look into REST API, either as complete replacement or just to take advantage of the "if-modified-since". This header works also per object.
Experiment with queries on EntityDefinition table to learn names of only the objects you're interested with (you probably don't care about apex classes, custom settings, *share and *history tables..). For example https://stackoverflow.com/a/64276053/313628
Then yes - describe just them, using REST or SOAP's describeSobject. If you have many objects - the network roundtrips might be an issue, you'd need to debug the app to see where it spends most time. Combat it by requesting up to 100 objects at a time, maybe issuing multiple requests (async processing? threads?) and combining results later.
Does it have to be partner WSDL? You could "preload" objects in your app using enterprise wsdl and combine some techniques listed above.
Is it possible to get stream related information in data load editor in Qlik Sense App? I cannot create new apps. I can access other information like app id, app path etc.
The stream that an application is published in, to the best if my knowledge, is not available by a System function within the load script. You can certainly pull that information from the Repository API inline. The obvious candidate for doing so would be the monitor_apps_REST_app data connection which should be present on your system. You should be able to pass the combination of DocumentName() as a variablized WHERE clause into the REST connection pull and then store the stream name or ID.
What is the ultimate use case?
I am using a web application for doing data entry which has a mechanism for storing the data entry form (which is an html form) in the browser cache IndexDB.
I am able to see the form in the browser dev tool like so :
I want to know for how long the Index DB will be able to store the form in the browser? Is it possible that it is months since the browser cache was same? Will closing the browser clear the keys? or is this persistent enough storage to last for a few months?
Is it possible to find out when(the exact date or time) the cache entry was made in the IndexDB?
I am asking this because I suspect some discperancy in the form for some of our users as the data being sent is a little different than expected.
Any help would is appreciated.
Thanks
DHIS2, the application you are referring to, has an application you and other users can use to clear any cached data. This app is named "Browser Cache Cleaner", and gives you a list of different things to clear. I would try this app and see if your users still have these issues.
Databases don't expose the timestamp of when the database record was last modified. That's something the developer needs make the application to store in the database records. For example, one could have created_at and modified_at columns to track when the record was created and when was it last modified.
IndexedDB is a persistent client storage API, so yes, data will stay permanently unless the user clears the browser's cache.
If there is some discrepancy in the form being sent, I would look at the caching strategy. Offline data caching is a pretty broad topic (also I don't know much about your application), but Google's Offline Cookbook is a good place to start digging in this topic, as long as caching strategies for your use.
UPDATE - Not sure how to update this to say pate's comment made me realise what I was doing wrong? Either way this is resolved for me now thanks
I return JSON from the server that contains a list of objects, each representing a digital post it note (text, position on page, database id etc.) I cache this in the service worker during fetch as normal.
Currently as users update/add post its I update a local copy of the JSON in the javascript as well as sending the info to the server.
What I want to do is as they update/add items the client JS will also save the new JSON to the application cache, then on page load use a cache-while-revalidate pattern so they only need to refresh if another user makes changes to their data. Otherwise they will get the cached JSON that will already contain their most recent changes.
As the application cache is versioned and the version number is stored in the sw.js file I'm currently sending a message (using MessageChannel) from the client to the SW to get the version number so the client can then put the JSON into the right cache. The only other options I can think of are to either make the application cache version a global variable somewhere other then the SW.js or just send the entire JSON in the message to the SW and let it put the update JSON into the cache.
Either way these all seem like workarounds/anti-patterns and I can't seem to find a better way of the client updating the application cache.
The other reason I want to do this is so that I can eventually move to an offline mode of working using the background sync api to handle add/updates etc. so want the cached JSON to be as up to date as possible.
Am I missing a trick somewhere?
I am trying to write a Core Data application for the iPhone that uses an external data source. I'm not really using Core Data to persist my objects but rather for the object life-cycle management. I have a pretty good idea on how to use Core Data for local data, but have run into a few issues with remote data. I'll just use Flickr's API as an example.
The first thing is that if I need say, a list of the recent photos, I need to grab them from an external data source. After I've retrieved the list, it seems like I should iterate and create managed objects for each photo. At this point, I can continue in my code and use the standard Core Data API to set up a fetch request and retrieve a subset of photos about, say, dogs.
But what if I then want to continue and retrieve a list of the user's photos? Since there's a possibility that these two data sets might intersect, do I have to perform a fetch request on the existing data, update what's already there, and then insert the new objects?
--
In the older pattern, I would simply have separate data structures for each of these data sets and access them appropriately. A recentPhotos set and and a usersPhotos set. But since the general pattern of Core Data seems to be to use one managed object context, it seems (I could be wrong) that I have to merge my data with the main pool of data. But that seems like a lot of overhead just to grab a list of photos. Should I create a separate managed object context for the different set? Should Core Data even be used here?
I think that what I find appealing about Core Data is that before (for a web service) I would make a request for certain data and either filter it in the request or filter it in code and produce a list I would use. With Core Data, I can just get list of objects, add them to my pool (updating old objects as necessary), and then query against it. One problem, I can see with this approach, however, is that if objects are externally deleted, I can't know, since I'm keeping my old data.
Am I way off base here? Are there any patterns people follow for dealing with remote data and Core Data? :) I've found a few posts of people saying they've done it, and that it works for them, but little in the way of examples. Thanks.
You might try a combination of two things. This strategy will give you an interface where you get the results of a NSFetchRequest twice: Once synchronously, and once again when data has been loaded from the network.
Create your own subclass of
NSFetchRequest that takes an additional block property to
execute when the fetch is finished.
This is for your asynchronous
request to the network. Let's call
it FLRFetchRequest
Create a class to which you pass
this request. Let's call it
FLRPhotoManager. FLRPhotoManager has a method executeFetchRequest: which takes an
instance of the FLRFetchRequest and...
Queues your network request based on the fetch request and passes along the retained fetch request to be processed again when the network request is finished.
Executes the fetch request against your CoreData cache and immediately returns the results.
Now when the network request finishes, update your core data cache with the network data, run the fetch request again against the cache, and this time, pull the block from the FLRFetchRequest and pass the results of this fetch request into the block, completing the second phase.
This is the best pattern I have come up with, but like you, I'm interested in other's opinions.
It seems to me that your first instincts are right: you should use fetchrequests to update your existing store. The approach I used for an importer was the following: get a list of all the files that are eligible for importing and store it somewhere. I'm assuming here that getting that list is fast and lightweight (just a name and an url or unique id), but that really importing something will take a bit more time and effort and the user may quit the program or want to do something else before all the importing is done.
Then, on a separate background thread (this is not as hard as it sounds thanks to NSRunLoop and NSTimer, google on "Core Data: Efficiently Importing Data"), get the first item of that list, get the object from Flickr or wherever and search for it in the Core Data database (carefully read Apple's Predicate Programming Guide on setting up efficient, cached NSFetchRequests). If the remote object already lives in Core Data, update the information as necessary, if not insert. When that is done, remove the item from the to-be-imported list and move on to the next one.
As for the problem of objects that have been deleted in the remote store, there are two solutions: periodic syncing or lazy, on-demand syncing. Does importing a photo from Flickr mean importing the original thing and all its metadata (I don't know what the policy is regarding ownership etc) or do you just want to import a thumbnail and some info?
If you store everything locally, you could just run a check every few days or weeks to see if everything in your local store is present remotely as well: if not, the user may decide to keep the photo anyway or delete it.
If you only store thumbnails or previews, then you will need to connect to Flickr each time the user wants to see the full picture. If it has been deleted, you can then inform the user and delete it locally as well, or mark it as not being accessible any more.
For a situation like this you could use Cocoa's archiving facilities to save the photo objects (and an index) to disk between sessions, and just overwrite it all every time the app calls home to Flickr.
But since you're already using Core Data, and like the features it provides, why not modify your data model to include a "source" or "callType" attribute? At the moment you're implicitly creating a bunch of objects with source "Flickr API", but you can just as easily treat the different API calls as unique sources and then store that explicitly.
To handle deletion, the simplest way would be to clear the data store each time it's refreshed. Otherwise you'd need to iterate over everything and only delete the photo objects with filenames that weren't included in the new results.
I'm planning to do something similar to this myself so I hope this helps.
PS: If you're not storing the photo objects between sessions at all, you could just use two different contexts and query them separately. As long as they're never saved, and the central store doesn't have anything in it already, it would work just like you describe.