On our form.io instance, we need to delete all user submissions older than n days (for data protection reasons).
What is the best way to achieve that? Is there an (undocumented) batch DELETE method in the api? Something like
DELETE {{projectUrl}}/{{formPath}}/submission?data.created_lt={{date}}
Am i missing any other option?
https://apidocs.form.io/#1f207caa-9d04-3e81-2973-e4bf82ee5190
It seems like you can query against submissions in a form, so if there is no undocumented batch delete, you could still find all of the submissions that need to be deleted and delete them one by one.
To query for specific submissions
In order to find specific submissions within the index, you can also perform queries for any poperty within a submission. [...] So, lets say you wish to find all people in the submissions who have the last name of Smith, you would run the following query:
{{appUrl}}/register/submission?data.lastName=Smith
Related
I'm planning on using rxdb + hasura/postgresql in the backend. I'm reading this rxdb page for example, which off the bat requires sync-able entities to have a deleted flag.
Q1 (main question)
Is there ANY point at which I can finally hard-delete these entities? What conditions would have to be met - eg could I simply use "older than X months" and then force my app to only ever displays data for less than X months?
Is such a hard-delete, if possible, best carried out directly in the central db, since it will be the source of truth? Would there be any repercussions client-side that I'm not foreseeing/understanding?
I foresee the number of deleted's growing rapidly in my app and i don't want to have to store all this extra data forever.
Q2 (bonus / just curious)
What is the (algorithmic) basis for needing a 'deleted' flag? Is it that it's just faster to check a flag rather than to check for the omission of an object from, say, a very large list. I apologize if it's kind of a stupid question :(
Ultimately it comes down to a decision that's informed by your particular business/product with regards to how long you want to keep deleted entities in your system. For some applications it's important to always keep a history of deleted things or even individual revisions to records stored as a kind of ledger or history. You'll have to make a judgement call as to how long you want to keep your deleted entities.
I'd recommend that you also add a deleted_at column if you haven't already and then you could easily leverage something like Hasura's new Scheduled Triggers functionality to run a recurring job that fully deletes records older than whatever your threshold is.
You could also leverage Hasura's permissions system to ensure that rows that have been deleted aren't returned to the client. There is documentation and examples for ways to work with soft deletes and Hasura
For your second question it is definitely much faster to check for the deleted flag on records than to have to try and diff the entire dataset looking for things that are now missing.
I'm trying to implement a read confirmation in a number of wiki pages.
I'm trying to use trackers.
General Description:
Employees in our company are assigned to read a number of official procedures.
I'm trying to implement a process where:
Each employee is assigned procedures he needs to read according to his department (Group).
Each procedure is a wiki page
At the end of each procedure there will be a confirmation form in the following format.
Users that don't need to read this procedure won't see this form.
Users that confirmed reading the document will see a message like:
You've confirmed reading this procedure.
Administrators will be able to monitor who read what procedure.
Questions:
How do I hide the tracker plugin from users who don't belong to the department (Group)
How do I display a different message once the user confirmed the read
Thanks
There are different way to achieve this and it require a bit of thinking (trade-off from a method to another), but this is what I’ll do.
Have 2 groups (before approving - after approving)
Display the procedures using a plugin listExecute and having at the end the approving checkbox to have some actions (notification, group changes, etc).
Enclosing everything in a tracker and turn it into a multipage forms can also be the way.
Your case remind me other use case I worked on including Official Procedures reading including quick test (to check if the procedures were understood) and approving mechanism. Look at https://doc.tiki.org/PluginExercise ;)
Good luck
Bernard
https://www.facebook.com/bsfez
Another, possibly simpler (? ;) way to do that would be to use just plugins group and list, maybe like this:
{GROUP(groups="This Department")}
{LIST()}
{filter type="trackeritem"}
{filter field="tracker_id" content="42"}
{filter field="tracker_field_procedurePage" content="{{page}}"}
{filter field="tracker_field_userLogin" content="{{user}}"}
{OUTPUT()}~tc~Item found, so already done~/tc~You already did this bit{OUTPUT}
{ALTERNATE()}~tc~Nothing found, show the form~/tc~{tracker trackerId=42 etc...}{ALTERNATE}
{LIST}
{ELSE}
You don't need to fill in the form
{GROUP}
This is totally untested i'm afraid, and i'm not 100% sure you can use a plugin in the ALTERNATE section, but give it a go? If it doesn't work, try using {display format="wiki plugin" etc...} which might do the trick - good luck!
Im using a Page (type Folder) to show all records with this pid. Is it possible group these records somehow? Theres a field in my DB called "vid", which contains the uid of some other records. I want the records in my folder to be grouped by this uid. Any suggestions? (Using Typo3 4.6.3)
Ok, then the simple answer is No. Grouping is not possible with the default backend list view module. You can sort, and search/filter there, but not more. You may write a custom backend module that does the trick for you.
What I could also imagine is to use the export function in the list module (there is a button somewhere) and then do the grouping with your favorite spreadsheet tool (like excel). Depending on how often you need this feature that may be a simple workaround that does not require and additional coding.
I developed an application where the user can create tasks (like a an agenda, ERP or CRM)
So, the user creates a task and that task has an expiration date. The idea is to erase the entry when the task expires.
I've been thinking solutions, like having a timer and so, but:
There is any method or way to create data that expires?? I mean, does MySQL, MSSQL (or any DB Manager) support something like this natively?
I would be great to have something like this:
CREATE TABLE [MyTASK] (expires on mydate action = DELETE){
mydate,
mysomething,
myagain
}
And then, the FakeSQL erases the data when the field "mydate" expires.
Data won't expire itself. You'll have to run something to find and remove it.
MS SQL Server supports the concept of scheduled jobs. So you can specify a stored proc that erases anything over (say) a week old, and configure that job to run every night.
mySQL seems to have Event Schedulers since 5.1.
If storage space is not an issue, it's worth a thought about putting in a bit field to mark a record as deleted so that none of your data is lost in case you want to add a part to your application that can look at all previous tasks. Like some administrative tools. Let's say employees are making tasks, and a task has expired and you don't want normal users to be able to modify or access expired tasks, so you "Delete" them when they expire. But one day the big boss comes along and says he wants a list of all tasks in the last year. This bit field could come in handy. It's just a thought and there may be better suggestions out there
I keep seeing questions floating through that make reference to a column in a database table named something like DateLastUpdated. I don't get it.
The only companion field I've ever seen is LastUpdateUserId or such. There's never an indicator about why the update took place; or even what the update was.
On top of that, this field is sometimes written from within a trigger, where even less context is available.
It certainly doesn't even come close to being an audit trail; so that can't be the justification. And if there is and audit trail somewhere in a log or whatever, this field would be redundant.
What am I missing? Why is this pattern so popular?
Such a field can be used to detect whether there are conflicting edits made by different processes. When you retrieve a record from the database, you get the previous DateLastUpdated field. After making changes to other fields, you submit the record back to the database layer. The database layer checks that the DateLastUpdated you submit matches the one still in the database. If it matches, then the update is performed (and DateLastUpdated is updated to the current time). However, if it does not match, then some other process has changed the record in the meantime and the current update can be aborted.
It depends on the exact circumstance, but a timestamp like that can be very useful for autogenerated data - you can figure out if something needs to be recalculated if a depedency has changed later on (this is how build systems calculate which files need to be recompiled).
Also, many websites will have data marking "Last changed" on a page, particularly news sites that may edit content. The exact reason isn't necessary (and there likely exist backups in case an audit trail is really necessary), but this data needs to be visible to the end user.
These sorts of things are typically used for business applications where user action is required to initiate the update. Typically, there will be some kind of business app (eg a CRM desktop application) and for most updates there tends to be only one way of making the update.
If you're looking at address data, that was done through the "Maintain Address" screen, etc.
Such database auditing is there to augment business-level auditing, not to replace it. Call centres will sometimes (or always in the case of financial services providers in Australia, as one example) record phone calls. That's part of the audit trail too but doesn't tend to be part of the IT solution as far as the desktop application (and related infrastructure) goes, although that is by no means a hard and fast rule.
Call centre staff will also typically have some sort of "Notes" or "Log" functionality where they can type freeform text as to why the customer called and what action was taken so the next operator can pick up where they left off when the customer rings back.
Triggers will often be used to record exactly what was changed (eg writing the old record to an audit table). The purpose of all this is that with all the information (the notes, recorded call, database audit trail and logs) the previous state of the data can be reconstructed as can the resulting action. This may be to find/resolve bugs in the system or simply as a conflict resolution process with the customer.
It is certainly popular - rails for example has a shorthand for it, as well as a creation timestamp (:timestamps).
At the application level it's very useful, as the same pattern is very common in views - look at the questions here for example (answered 56 secs ago, etc).
It can also be used retrospectively in reporting to generate stats (e.g. what is the growth curve of the number of records in the DB).
there are a couple of scenarios
Let's say you have an address table for your customers
you have your CRM app, the customer calls that his address has changed a month ago, with the LastUpdate column you can see that this row for this customer hasn't been touched in 4 months
usually you use triggers to populate a history table so that you can see all the other history, if you see that the creationdate and updated date are the same there is no point hitting the history table since you won't find anything
you calculate indexes (stock market), you can easily see that it was recalculated just by looking at this column
there are 2 DB servers, by comparing the date column you can find out if all the changes have been replicated or not etc etc ect
This is also very useful if you have to send feeds out to clients that are delta feeds, that is only the records that have been changed or inserted since the data of the last feed are sent.