My scenary is something like this
//a lot of database dependant processing
db.SaveChangesAsync(); //asynchronouly saving database changes
//a lot of ANOTHER long term database INDEPENDANT stuff
since await db.SaveChangesAsync() waits until all changes are made to database (could take some time) and I don't want to wait, but in paralell I want to make the other log term stuff below WHILE SaveChanges is saving changes, why should I use await? If I dont use await I receive the worning in the image below.
This warning is quite clear - you're starting an asynchronous task, but you have no mechanism for being notified when it's complete. What if you do need to know when it's done, later? Before returning from your method, or at a point where you need to be sure that the data is synchronized with the DB?
awaiting the operation is one way to ensure that, but as you say, you don't want to stop the independent processing while the data is saved. await is for asynchrony, not for parallelism.
Additionally, awaiting the Save task ensures that you catch any exceptions that are thrown by the DB. When you simply release the task, any exceptions are lost - at best, you can handle TaskScheduler.UnobservedTaskException to handle unobserved tasks that throw exceptions, but a much simpler way is to simply observe the task.
One way to observe the task is await, as you know. Another is to simply save the Task object that SaveChangesAsync returns in a variable. This will allow you to await (or Wait()) on that Task later, catch its exceptions (either by try/catching the await call or by attaching a ContinueWith to the Task) and will let the compiler rest easy knowing you're not abandoning the task to its fate:
//a lot of database dependant processing
var dbSaveTask = db.SaveChangesAsync(); //asynchronouly saving database changes
//a lot of ANOTHER long term database INDEPENDANT stuff
// now, before assuming the DB is up to date, ensure the TAsk is complete.
await dbSaveTask;
// now do something Db-dependent!
Related
I am writing for learning purposes a cross-platform to-do app with Flutter and Firestore. Currently, I have the following design, and I would like to know if there are better alternatives.
One of the main screens of the app shows a list of all tasks. It does this by subscribing to the corresponding Firestore collection, which we'll say is /tasks for simplicity.
FirebaseFirestore.instance.collection("tasks").snapshots()
Each tile in the ListView of tasks can be clicked. Clicking a tile opens a new screen (with Navigator.push) showing details about that specific task.
Importantly, this screen also needs to update in real-time, so it is not enough to just pass it the (local, immutable) task object from the main screen. Instead, this screen subscribes to the individual Firestore document corresponding to that task.
FirebaseFirestore.instance.collection("tasks").doc(taskId).snapshots()
This makes sense to me logically: the details page only needs to know about that specific document, so it only subscribes to it to avoid receiving unnecessary updates.
The problem is since the collection-wide subscription for the main screen is still alive while the details screen is open, if the document /tasks/{taskId} gets updated, both listeners will trigger. According to the answers in this, this and this question, this means I will get charged for two (duplicate) reads for any single update to that document.
Furthermore, each task can have subtasks. This is reflected in Firestore as a tasks subcollection for each task. For example, a nested task could have the path: /tasks/abc123/tasks/efg875/tasks/aay789. The main page could show all tasks regardless of nesting by using a collection group query on "tasks". The aforementioned details page also shows the tasks' subtasks by listening to the subcollection. This allows to make complex queries on subtasks (filtering, ordering, etc.), but again the disadvantage is getting duplicate reads for every update to a subtask.
The alternative designs that occur to me are:
Only keep a single app-wide subscription to the entire set of tasks (be it a flat collection or a collection group query) and do any and all selection, filtering, etc. on the client. For example, the details page of a task would use the same collection-wide subscription and select the appropriate task out of the set every time. Any filtering and ordering of tasks/subtasks would be done on the client.
Advantages: no duplicate reads, minimizes the Firestore cost.
Disadvantages: might be more battery intensive for the client, and code would become more complex as I'd have to select the appropriate data out of the entire set of tasks in every situation.
Cancel the collection-wide subscription when opening the details page and re-start it when going back to the main screen. This means when the details page is open, only updates to that specific task will be received, and without being duplicated as two reads.
Advantages: no duplicate reads.
Disadvantages: re-starting the subscription when going back to the main screen means reading all of the documents in the first snapshot, i.e. one read per task, which might actually make the problem worse. Also, it could be quite complicated to code.
Do any of these designs seem the best? Is there another better alternative I'm missing?
Create a TaskService or something similar in your app that handles listening to the FirebaseFirestore.instance.collection("tasks").snapshots() call, then in your app, subscribe to updates to that service rather than Firebase itself (you can create two Stream objects, one for global updates, one for specific updates).
Then, you've only one read going on in your Firebase collection. Everything is handled app side.
Pseudo-code:
class TaskService {
final List<Task> _tasks = [];
final StreamController<List<Task>> _signalOnTasks = StreamController.broadcast();
final StreamController<Task> _signalOnTask = StreamController.broadcast();
get List<Task> allTasks => _tasks;
Stream<List<Task>> get onTasks => _signalOnTasks.stream;
Stream<List<Task>> get onTask => _signalOnTask.stream;
void init() {
FirebaseFirestore.instance.collection("tasks").snapshots().listen(_onData);
}
void _onData(snapshot) {
/// get/update our tasks (maybe check for duplicates or whatever)
_tasks.addAll(snapshot.documents);
/// dispatch our signal streams
_signalOnTasks.add(snapshot.documents);
for(final task in snapshot.documents) {
_signalOnTask.add(task);
}
}
}
You can make TaskService and InheritedWidget to get access to it wherever (or use the provider package), the add your listeners to whatever stream you're interested in. You'll need just to check in your listener to onTask that it's the correct task before doing anything with it.
In my tests, I've got some database actions that aren't exposed as Futures at the test level. Sometimes, my tests run fast enough that close() in my cleanup happens before those database actions complete, and then I get ugly errors. Is there a way to detect how many statements are in-flight or otherwise hold off close()?
When you execute a query you get Future[A] where A is the result of the query.
You can compose all your queries using Future.sequence() to get a single future composedFuture which will be completed when all your queries have returned result.
Now you can use composedFuture.map(_ => close()) to make sure that all queries have finished execution and then you close the resource.
Best option is to expose the actions as future and then compose them.
Otherwise you can put Thread.sleep(someSensibleTime) and hope your future completes within someSensibleTime, but this will make your tests slow and errorprone.
I think it may be database-dependant rather than slick-dependant.
For example, mysql technologies allow you to see currently running queries with the query "show processlist", and act accordingly.
If that's not an option, I suppose that you could poll the db to observe a selected side effect, and close() afterwards ?
In my extension I have a set of operations that are generated by user activities. Each operation consists of several steps.
To handle those operations I implemented a scheduler task (extension "scheduler" 6.2.0). Now the point is: steps of each operation must be done one after the other, not parallel. That means: at start the scheduler task should find next "free" operation, lock it and handle it.
For locking purposes database table with operations has an integer column "isLocked". So I wanted to use following SQL statement to lock an operation:
$lockID = time();
'UPDATE operations SET isLocked = '.$lockID.' WHERE isLocked = 0 AND uid = '.$freeOperationFound->getUid().';'
After this SQL command I wanted to check if lock was set:
$repository->findOneByIsLocked($lockID);
If locking was successful operation step handling can start.
If meanwhile another instance of scheduler task locks this operation, the SQL statement above does nothing because of condition: WHERE isLocked = 0.
The problem is: Extbase ignores SQL UPDATE-statements.
If I just update the free operation object via repository the lock of another task instance can be overwritten. I need some kind of "conditional" update.
I think I got it: $GLOBALS['TYPO3_DB']->exec_UPDATEquery is the answer.
The only question remaining is, if this method is also depricated in FLOW, like $query->statement of Repository.
While the exec_UPDATEquery function from the DatabaseConnection class certainly gets the job done, here is the solution via extbase. It might make more sense if you need to work with the Operation object after you lock it.
$persistenceManager = GeneralUtilities::makeInstance('TYPO3\CMS\extbase\Persistence\PersistenceManager');
$freeOperation = $repository->findOneByIsLocked(0);
$freeOperation->setIsLocked(time());
$repository->update($freeOperation);
$persistenceManager->persistAll();
$freeOperation->myOperation();
$freeOperation->myOtherOperation();
$freeOperation->setIsLocked(0);
$repository->update($freeOperation);
$persistenceManager->persistAll();
The reason why you need to persist manually is, that your task is not within the context of a ActionController Action. And even if you were, it wouldn't automatically persist your changes until the end of the Action. Doing it through extbase might be the safer option because you can be sure to actually work on the exact same operation that you have just locked.
In our webapp, we have lots of queries running. Most of them reading data but some update queries with high priority might come. Since, we'd like to cancel read queries but when using KILL, I'd like the read query to return certain dataset result or execution result upon receiving cancel.
My intention is to mimic the behavior of signal in C programs for which a signal handler is invoked upon receiving a kill signal.
Is there any method to define an asynchrnous KILL signal handler for SPs?
This is not a fully tested answer. But it is a bit more than just a comment.
One is to have dirty read (with nolock).
This part is tested I do this all time.
Build a large scalable app you need to resort to this and manage it.
A dirty read will not block an update.
You can get that - a dirty read.
A lot of people think a dirty read may get corrupt data.
If you are updating smith to johnson the select is not going to get smison.
The select is going to get smith and it will be immediately stale.
But how is that worse then taking a read lock?
The read get smith and blocks the update.
Once the read locks are cleared it is updated.
I would contend that blocking an update is also stale data.
If you are using reader I think you could pass the same cancellation token to each select and then just cancel the one token.
But if may not process the CancellationToken until it read the row so it may not cancel a query a long running query that has not yet returned any rows.
DbDataReader.ReadAsync Method (CancellationToken)
Or if you are not using reader look at
SqlCommand.Cancel
As far as getting cancel to return alternate data. I doubt SQL is going to do that.
I have a play model called "JobStatus" and it's just got one property, an enum with a JobState, (Running/notRunning).
The class extends model and is implemented as a singleton. You call it's getInstance() method to get the only record in the underlying table.
I have a job that runs every month and in the job I will toggle the state of the JobStatus object back and forth at various times and call .save().
I've noticed it isn't actually saving.
When the job starts off, it's first line of code is
JobStatus thisJobStatus = jobStatus.getInstance();
...// exit if already running
thisJobStatus.JobState = JobState.Running;
thisJobStatus.save()
then when the job is done it will change the status back to NotRunning and save again.
The issue is that when I look in the MySql database the actual record value is never changed.
This causes a catastrophic failure because when other nodes try to run the job they check the state and since they're seeing it as NotRunning, they all try to run the job also.
So my clever scheme for managing job state is failing because the actual value isn't getting commited to the DB.
How do I force Play to write to the DB right away when I call .save() on a model?
Thanks
Josh
try adding this to your JobStatus and call it after save.
public static void commit(){
JobStatus.em().getTransaction().commit();
JobStatus.em().getTransaction().begin();
JobStatus.em().flush();
JobStatus.em().clear();
}
I suppose you want to mark your job as "running" pretty much as the first thing when the job starts? In that case, you shouldn't have any other ongoing database statements yet...
To commit your changes in the database immediately (instead of after the job has ended), add the following commands after the thisJobStatus.save(); method call:
JPA.em().flush();
JPA.em().getTransaction().commit();
Additionally, since you're using MySQL, you might want to lock the row immediately upon retriveval using the SELECT ... FOR UPDATE clause. (See MySQL Reference Manual for more information.) Of course, you wouldn't want to have that in your getInstance() method, otherwise every fetch operation would lock the record.