I have a bulkified trigger that uses maps in order to avoid running into SOQL governor limits. This trigger also uses static class variables for recursion and for limiting queries.
What I'm doing is when a bulk operation such as an insert or update is started on the object, Contact object in this case, then the trigger will build the maps of related Accounts on the first trigger, and will use those maps for the rest of the trigger firings.
Here is an example of the operation that is working great, but only for After Update & After Insert trigger operations:
Check that a static class variable is not true.
If variable is not true build maps.
Set the static class variable to true.
Perform the trigger operations.
For the insert/update triggers, the session state is maintained and the static class variable is not reset until after the end of the bulk operation.
However for the before delete trigger, it seems there is no session state, and that the session is reset each time a record is deleted. The session is reset, but the governor limits are cumulitive for a bulk delete of records. So with a before delete trigger, even if using maps, the soql query count keeps counting running into the infamous 'Too Many Sql Queries' limit for the deletion of more than 100 records.
Any thoughts on how to prevent running into the SOQL limit would be much much appreciated. I wasn't able to find anything on this anywhere.
One option you could take is to use the trigger to schedule a batch apex class for execution. For which ever object is the one which kicks off the cascading delete, use the trigger to create an instance of the batch, passing to it a list of source IDs.
Then in the execute method of the batch class you can then build up the maps etc. for each batch and perform the deletes in there. Batch apex has considerably higher governor limits at the sacrifice of synchronous execution, that said the process will generally kick off within a couple of seconds of your operation.
Other than this it may simply be a case of optimising your code such that cascading deletes always work on lists as large as possible (up to the 200 limit of course), or maybe you could use Master Detail relationships to take care of some of the delete operations for you?
Related
I have a CICS program, which will read a DB2 table to obtain the rules based of the field name. Let's say my record type is AA and this type will have at least 20 rules that I need to do loop in DB2 tables. Like wise I have few record types and many more rules tied to each type.
I get data from MQ and for each record type I call separate CICS program. So when I have to process high load, DB2 rules table is getting held by so many program and this causing performance issue.
I want to get away from DB2 and load this rules in CICS Container and maintain periodically. But I'm not sure if this will work. I don't want to use or create VSAM's. I'm looking for some kind of storage I could use and maintain in CICS.
My question is. If I create a pipeline and container will I able to access them by multiple program at a same time and will data stored rules stay in Container after successful get?
Before reading further, please understand that DB2 solves all the sharing and locking problems very efficiently. I've never encountered a problem with too many transactions trying to read a DB2 table concurrently. Updating, yes; a mix of updates and reads, yes; just reading, no.
So, in order to implement your own caching of a DB2 table inside CICS you need a data store. As #BruceMartin indicates, a TS queue is an option, I would say that given your other constraints it is your only option.
In order to automate this you must create a trigger on your DB2 table that fires after INSERT, UPDATE, or DELETE. The trigger must cause the TS queue to be repopulated. The repopulation mechanism could be EXCI or MQ, as the code performing the repopulation must execute within CICS.
During the repopulation, all transactions reading the TS queue must wait for the repopulation to complete. This can be done with the CICS ENQ API, with a caveat. In order to prevent all these transactions from single-threading through their TS queue read due to always ENQing, I suggest using two TS queues, one holds the DB2 data and the other is a "trigger" TS queue. The contents of the trigger TS queue are not significant, you can store a timestamp, or "Hello, World", or "ABC" it doesn't matter.
A normal transaction attempts a read of the trigger TS queue. If the read is unsuccessful the transaction simply reads the TS queue with the DB2 data. But if the read is successful then repopulation is in progress and the transaction ENQs on a resource (call it XYZ). On return from the ENQ, DEQ and read the TS queue with the DB2 data.
During repopulation, a program executed by the trigger on the DB2 table executes in CICS. First ENQing on resource XYZ, then creating the trigger TS queue, then deleting the TS queue with the DB2 data, then creating the TS queue and populating it with the new DB2 data, deleting the trigger TS queue, finally DEQing resource XYZ. I would strongly suggest using a multi-row SELECT to obtain the DB2 data as it is significantly more efficient than the traditional OPEN CURSOR, FETCH, CLOSE CURSOR method.
For illustration, say I'm updating a table ProductOffers and their prices. Mutations to this table are of the form: add new ProductOffer, change price of existing ProductOffer.
Based on the above changes, I'd like to update a Product-table which holds pricing info per product aggregated over all offers.
It seems logical to implement this using a row-based update/insert trigger, where the trigger runs a procedure creating/updating a Product row.
I'd like to properly implement concurrent updates (and thus triggers). I.e.: updating productOffers of the same Product concurrently, would potentially lead to wrong aggregate values (because multiple triggered procedures would concurrently attempt to insert/update the same Product-row)
It seems I cannot use row-based locking on the product-table (i.e.: select .. for update) because it's not guaranteed that a particular product-row already exists. Instead the first time around a Product row must be created (instead of updated) once a ProductOffer triggers the procedure. Afaik, row-locking can't work with new rows to be inserted, which totally makes sense.
So where does that leave me? Would I need to roll my own optimistic locking scheme? This would need to include:
check row not exists => create new row fail if already exists. (which is possible if 2 triggers concurrently try to create a row). Try again afterwards, with an update.
check row exists and has version=x => update row but fail if row.version !=x. Try again afterwards
Would the above work, or any better / more out-of-the-box solutions?
EDIT:
For future ref: found official example which exactly illustrates what I want to accomplish: Example 39-6. A PL/pgSQL Trigger Procedure For Maintaining A Summary Table
Things are much simpler than you think they are, thanks to the I an ACID.
The trigger you envision will run in the same transaction as the data modification that triggered it, and each modification to the aggregate table will first lock the row that it wants to update with an EXCLUSIVE lock.
So if two concurrent transactions cause an UPDATE on the same row in the aggregate table, the first transaction will get the lock and proceed, while the second transaction will have to wait until the first transaction commits (or rolls back) before it can get the lock on the row and modify it.
So data modifications that update the same row in the aggregate table will effectively be serialized, which may hurt performance, but guarantees exact results.
As title say, it is possible to issue a query on psql with a "begin", query, and "commit".
What I want to know is what happens if I don't use a "begin" command?
Some database engine will allow you to execute modifications (INSERT, UPDATE, DELETE) without an open transaction. It's basically assumed that you have an instant BEGIN / COMMIT around each of your instructions, which is a bad practice in case something goes wrong in a batch of many instructions.
You can still make a SELECT, but no INSERT, UPDATE, DELETE without a BEGIN to enforces the good practice. That way, if something goes wrong, a ROLLBACK is instantly executed, canceling all your modifications as if they never existed.
Using a transaction around a batch of various SELECT will guarantee that the data you get for each SELECT matches the same version of the database at the instant you open the transaction depending on your ISOLATION level.
Please read this for more information :
http://www.postgresql.org/docs/9.5/static/sql-start-transaction.html
and
http://www.postgresql.org/docs/9.5/static/tutorial-transactions.html
If you don't use BEGIN/COMMIT, it's the same as wrapping each individual query in a BEGIN/COMMIT block. You can use BEGIN/COMMIT to group multiple queries into a single transaction. A few reasons you might want to do so include
Updating multiple tables at the same time. For instance, usually when you delete a record you also want to delete other rows that reference it. If you do this in the same transaction, nothing will ever be able to reference a row that's already been deleted.
You want to be able to revert some changes if something goes wrong later. Suppose you're writing some user inputted data to multiple tables. At some point you realize that some of it isn't formatted properly. You probably wouldn't want to insert any of it, so you should wrap the entire operation in a transaction.
If you want to ensure the data you're updating hasn't been updated while you're writing to it. Suppose I'm adding $10 to a bank account from two separate connections. I want to add $20 in total - I don't want one of the UPDATEs to clobber the other.
Postgres gives you the first two of these by default. The last one would require a higher transaction isolation level, and makes your query run the risk of raising a serialization error. Transaction isolation levels are a fairly complicated topic, so if you want more info on them the best place to go is the documentation.
I need to modify a Trigger (which use a particular FUNCTION) already defined and it is being in use. If i modify it using CREATE OR REPLACE FUNCTION, what is the behaviour of Postgres? will it "pause" the old trigger while it is updating the function?. As far as i know, Postgres should execute all the REPLACE FUNCTION in one transaction (so the tables are locked and so the triggers being modify while it is updating, then next transactions locked will use the new FUNCTION not the old one. is it correct?
Yes. According to the documentation:
http://www.postgresql.org/docs/9.0/static/explicit-locking.html
Also, most PostgreSQL commands automatically acquire locks of appropriate modes to ensure that referenced tables are not dropped or modified in incompatible ways while the command executes. (For example, ALTER TABLE cannot safely be executed concurrently with other operations on the same table, so it obtains an exclusive lock on the table to enforce that.)
will it "pause" the old trigger while it is updating the function?
It should continue executing the old trigger functions when calls are in progress (depending on the isolation level, subsequent calls in the same transaction should use the old definition too; I'm not 100% sure the default level would do so, however), block new transactions that try to call the function while it's being updated, and execute the new function once it's replaced.
As far as i know, Postgres should execute all the REPLACE FUNCTION in one transaction (so the tables are locked and so the triggers being modify while it is updating, then next transactions locked will use the new FUNCTION not the old one. is it correct?
Best I'm aware the function associated to the trigger doesn't lock the table when it's updated.
Please take this with a grain of salt, though: the two above statements amount to what I'd intuitively expect mvcc to do, rather than knowing this area of Postgres' source code off the top of my head. (A few core contributors periodically come to SO, and might eventually chime in with a more precise answer.)
Note that this is relatively straightforward to test, that being said: open two psql sessions, open two transactions, and see what happens...
What is the difference between FOR and AFTER in trigger definition. Any benefits of using one vs another?
If I issue an update statement which updates 5 rows, does the trigger (with FOR UPDATE) fires 5 times? If it is so, is there any way to make trigger fire only once for the entire UPDATE statment (even though it updates multiple rows)
Is there any chance/situation of having more than one row in "inserted" or "deleted" table at any time in a trigger life cycle. If it so, can I have a very quick sample on that?
thanks
Trigger fire once for each batch and should always be designed with that in mind. Yes if you do a multi-row update insert or delte, allthe rows will be in the inserted or deleted tables. For instance the command
Delete table1 where state = 'CA'
would have all the rows in the table that have a state of CA in them even if it was 10,000,000 of them. That is why trigger testing is critical and why the trigger must be designed to handle multi-row actions. A trigger that works well for one row may bring the deatabase toa screeching halt for hours if poorly designed to handle mulitple rows or could cause data integrity issues if not designed correctly to handle mulitple rows. Triggers should not rely on either cursors or loops for the most part but on set-based operations. If you are setting the contents of inserted or delted to a variable, you are almost certainly expecting one row and yor trigger will not work properly when someone does a set-based operation on it.
SQL Server has two basic kinds of DML triggers, after triggers which happen after the record has been placed in the table. These are typically used to update some other table as well. Before triggers take the place of the insert/update/delete, they are used for special processing onthe table inserted usually. It is important to know that a before trigger will not perform the action that was sent to the table and if you still want to delete/update or insert as part of the trigger you must write that into the trigger.