How to request data from Firestore defensively with Flutter - flutter

Since Firestore is a NoSQL database and has no strict type rules and defined document structures, I think about handling corrupt data in my Flutter app.
In case you wonder why I want to request defensively, even when it is no third-party API -> I can think of three reasons why my app crashes because of corrupt data:
I add wrong data via the Firebase console like using type string for a field that should have been type number (happened repeatedly).
A bug in my app adds corrupt data to Firestore.
A user has an old version of my app installed which can not handle the new Firestore data structure.
My requirements: The app should not crash when corrupt data from Firestore is requested, but the corrupt data should be reported so that the data can be fixed in Firestore a.s.a.p.

What do you think about the following approach?
Assume we have a model Movie.
Movie {
final String title;
final int releaseYear;
Movie({required this.title, required this.releaseYear});
Movie.from(Map<String, dynamic> data)
: title = data['title'],
releaseYear = data['release_year'];
}
The named constructor from parses the document data from DocumentSnapshot.data() and returns our model. This works fine as long as the data has a field title of type String and a field release_year of type int (number in Firestore).
Let's assume the field release_year is missing in the actual data. This will let the request crash. Thus, the current user can't do anything with the respective movie and I as developer won't notice, because it happened silently on the device of the user.
To fix the first issue, we can use defensive parsing with fallback data like this: data['release_year'] ?? -1. No crash happens, but I as developer still don't notice and can't fix the data.
To fix also this issue we could use Firebase Crashlytics. The only problem is that if we use defensive parsing to prevent crashing no log will be sent to Firebase. That's why I came up with this solution:
final snapshot = await FirebaseFirestore.instance.collection('movies').doc('123').get();
try {
return Movie.from(snapshot.data()!);
} catch (e) {
await FirebaseCrashlytics.instance
.recordError(e, e.stackTrace(), reason: 'data of movie ${snapshot.id} is corrupt');
return Movie.fromCorrupt(snapshot.data()!);
}
First, the app tries to parse the document data without any fallback mechanism. If the data is corrupt an exception is thrown and catched. In the catch block, the error is send to Firebase and then the defensive parsing constructor fromCorrupt is called to let the user continue in the app with the remaining data. In fromCorrupt each field is checked on null and type, before it is used to create the model Movie. If a value is null or of a wrong type, a fallback value is used.
What do you think of my approach? Am I overengineering? 😅

Related

Flutter cubit doesn't change state

I'm studying bloc in Flutter now and faced a problem.
In my test app I fetch weather data and when trying to display it I see no changes and no errors.
The trouble is: I do get the data and can see with print, but when it comes to displaying with BlocBuilder, it seems to not change the state.
This is the link to my repo - https://github.com/PhilippBekher/flutter_weather
The logics for displaying data are in main.dart
The weather data is accessible via the link - https://api.weatherapi.com/v1/forecast.json?key=%20f148ee51b7cd4e2390e110556221408&q=London&days=1
When trying to print the state inside the file for the cubit, I can't see the emitted state (WeatherFetchLoaded):
The screen below:
I ran your code. Here is the outcome
It came in is catch block line.Here you are emitting no value. So only Its still in loading state.
Future<void> fetchWeatherApi() async {
emit(WeatherFetchLoading());
try {
final Weather weather = await apiRepository.getWeather();
print('hello');
emit(WeatherFetchLoaded(weather: weather));
} on Failure catch (err) {
emit(WeatherFetchError(failure: err));
} catch (err) {
print("Error 1: $err"); **// Here**
}
}
It shows
'double' is not a subtype of type 'int'
This is a parsing issue while converting to the values to double.
Your approach of parsing json is not a recommended way. You can read here more about json parsing.
If you need to continue to debug ,it may be time consuming to find exact field. You can try looking on each field . You need to make sure you are parsing correct data type. Model class and json should have same data type.

DataStoreError: The operation couldn’t be completed. (SQLite.Result error 0.)

I am using AWS Appsync, AWS datastore, Aws Cognito, Aws API. When I am trying to save data on AWS Datastore it gives me this error
DataStoreError: The operation couldn’t be completed. (SQLite.Result error 0.).
let msg = Message.init(....)
//where Message is genrated from amplify codegen models
print(msg) // Output Message()
Amplify.DataStore.save(msg) { result in
print(msg)// Output Message()
switch result {
case .success:
print("Post saved successfully!")
case .failure(let error):
print("Error saving post \(error)")
}
}
I also get this response in my console.
[SQLiteStorageEngineAdapter] insert into Message (.....)
values (NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL)
I have created another sample project using the same model and it works fine there.
After spending 8 - 9 days found this. Target < Project Name < Build Settings < Reflection Metadata level. Make sure you select "All" in this.
This setting controls the level of reflection metadata the Swift compiler emits.
All: Type information about stored properties of Swift structs and classes, Swift enum cases, and their names, are emitted into the binary for reflection and analysis in the Memory Graph Debugger.
Without Names: Only type information about stored properties and cases are emitted into the binary, with their names omitted. -disable-reflection-names
None: No reflection metadata is emitted into the binary. Accuracy of detecting memory issues involving Swift types in the Memory Graph Debugger will be degraded and reflection in Swift code may not be able to discover children of types, such as properties and enum cases. -disable-reflection-metadata.
In my case that was in None. Please make sure you select "All".
This error was returned under similar circumstances in my case: saving an item. It was only throwing errors when I tried saving a model that I had recently created in Amplify. I was able to solve it by quitting the emulator (ios) and rerunning the application. It seems that resetting the environment is a solution to 50% of the problems with Amplify
#Mandhata Singh solution did not work for me.
I deleted the app and installed it again and it worked fine.
Note: deleting the app will delete the database with it. so the data will be lost.

How to save/Manage Game Progress in Mobile devices (Unity)?

Currently I am saving game progress in a file in JSON format. Each time player completes the level a new entry will be added to JSON file as follow "level_N":{"score":234,"points":22},
this file will be saved at the end of Level complete. Consider scenario in which player reaches level 2345 or so, In that case saving to the file on level complete takes considerably longer time in some mobile devices. how to manage saving such a large amount of data ? do I have to use some other formats ? or do i have to save each level detail in separate file ?
PlayerPrefs is the easiest way to save data. It’s designed to handle basic data types (int, string, float) and works like a dictionary, so you can simply store JSON string as key-value pairs. And there is no size limit on iOS or Android (in webplayer it's limited to 1MB).
// read
PlayerPrefs.GetString(string key, string value);
// write
PlayerPrefs.SetString(string key, string value);
// load and update
const saveKey = "level_N";
Private void SaveProgress()
{
string saveValue = "your JSON string";
string loadValue = PlayerPrefs.GetString(saveKey);
if (!saveValue.Equals(loadValue))
{
PlayerPrefs.SetString(saveKey, saveValue);
PlayerPrefs.Save();
}
}
User data will be automatically written to disk during OnApplicationQuit(), but you may want to use PlayerPrefs.Save() in case your game crashes.
Having said that, saving a large amount of data to PlayerPrefs on mobile devices might be slow. So if you want to improve your game performance or need more space, you can use Application.persistentDataPath to save data to a public directory on the device. Similar to PlayerPrefs, data is not cleared when app is updated.
FileStream file = File.Open(Application.persistentDataPath + "/gameInfo.dat", FileMode.Open);
You can use PlayerPrefs class to save to save the local data.
In your case, you can trun your json to string then call PlayerPrefs.SetString to save/PlayerPrefs.GetString to get.
PlayerPrefs is always a good choice for local data managing but if you planning to add online features to your game, I recommend you to take a look at Firebase Realtime Database for sync json data with offline support. And let the firebase manage performance issues on mobile devices. Here you can see how to integrate Firebase Database to your Unity project easily.

Standard practice error handling for partial data uploads in Firestore?

Would like to know what (if any) the standard / best-practice way is for handling failed / partial data uploads to firestore DB.
For example, suppose a user needs to upload to two different collections via something like
let res = undefined
let docID = firebase.auth().currentUser.uid
try {
res = await firebase.firestore().collection('someCollection')
.doc(docID)
.set(someData)
} catch (error) { console.error(`Some message ${res}`) }
try {
res = await firebase.firestore().collection('someOtherCollection')
.doc(docID)
.set(someOtherData)
} catch (error) { console.error(`Some message ${res}`) }
and that for whatever reason, the program successfully writes to someCollection, but not to someOtherCollection. How should this be handled? Should we attempt to roll back what has been written (via the catch blocks) or wait until the end of upload attempts and add / upload an additional flag to the document and only treat data as valid in the future if it has this flag? Something else?
** Realize that this is close to some Questions to Avoid Asking, but in this case am asking specifically for Firestore API (eg. any functions / objects within the firestore() API that is specifically intended to handel these cases) and best practices (if there are any in the wider world (pretty new to using Firebase / Firestore)).
You should use a Firestore transaction to ensure that all the documents have updated atomically. This will ensure that all of the documents are updated, or none of them.
From the referenced docs:
Cloud Firestore supports atomic operations for reading and writing data. In a set of atomic operations, either all of the operations succeed, or none of them are applied.

CKAsset in server record contains no fileURL, cannot even check for nil

I am testing a sync conflict when I save a record that contains a CKAsset (simply a JPG image) using CKModifyRecordsOperation with a save policy of .IfServerRecordUnchanged. I am getting the error CKErrorCode.ServerRecordChanged. This CKError returns me useful information for conflict resolution, including the CKRecord I tried to save, and the current server version of the record. The first is in error.userInfo[CKRecordChangedErrorClientRecordKey] the second is in error.userInfo[CKRecordChangedErrorServerRecordKey].
My problem is I am trying to access the server record's CKAsset with this code:
if let photoAsset = rec["myPhoto"] as? CKAsset {
print("PhotoAsset.fileURL: \(photoAsset.fileURL)") // BAD_ACCESS ERROR HERE
self.myPartner.photo = NSData(contentsOfURL: photoAsset.fileURL)
}
I don't get how this is possible. But after further investigating, I print out the client and server CKRecords and the server one is missing the 'path' property.
client CKAsset...myPhoto (modified) -> CKAsset: 0x7b960d90; path=~/tmp/BF185B2C-7A39-4730-9530-9797E843243Aphoto, size=373959, uploadRank=0, uploadReceipt=A92Eg1qoyPG7yrg3, UUID=3C2D5DC8-4FF5-4A81-853B-395FC1C59862, referenceSignature=<012fd149 200fc600 617e3907 88763e3e 5002abbf 5b>, flags=uploaded, wrappedEncryptionKey=, signature=<0134a297 38d52f5f 9275bfba fce5b1a8 3d6b9692 d3>
server CKAsset...myPhoto = CKAsset: 0x7be700d0; referenceSignature=<015337bd 84409893 7c014f46 36248d27 ce911dc3 7a>, size=373959, uploadRank=0, UUID=DF5D2EB4-033C-49A2-AF52-6055B5A44106, wrappedEncryptionKey=<767e7cfd d1e62110 32119ee9 f6f026b3 5bcf0cc3 8053a4de>, signature=<0134a297 38d52f5f 9275bfba fce5b1a8 3d6b9692 d3>
Notice how path=~/tmp/C706423B-A3E8-4051-A9B3-483C718BFBF5photo is missing from the server one? Can anyone explain this? To fix it I try to avoid touching the CKAsset from the server record. I would like to at least be able to check for nil. I wanted to put this out there in case it helps anyone else.
Due to the crash on accessing fileURL, this is most likely a framework bug. Probably an oversight on account of the CKRecord being buried in a dictionary. I just follow it up with a regular fetch(withRecordID:).
I'm experiencing this issue as well on iOS 11.2.1 when accessing CKAsset from serverRecord property in CKRecord. It's a little bit frustrating. A workaround is fetching the object once again via func fetch(withRecordID... and then access fileURL.
This seems like the correct behavior, not a bug.
CloudKit informed you that your write operation failed because you weren't working from the latest CKRecord and gave you a non-hydrated version of the server's current CKRecord so you could determine which fields were different from your starting point. The rest is up to you.
If CloudKit returned the fully hydrated server record in the error response for a write operation, it would potentially waste enormous amounts of bandwidth/resources.
That is why CKAssets exist: to separate the size-constrained key-value fields associated with a CKRecord from the unlimited-size binary assets that can be attached to them.