I have a cloud function that write in a large number of documents in firestore. I get the following error: "Exception from a finished function: Error: 4 DEADLINE_EXCEEDED: Deadline exceeded".
I read that maybe it could be cause by the 500 commit limit. How could I manage to pass this limit?
Code: https://pastebin.com/nqsGuMm6
Log: Function terminated with status: ok (in 4/5 seconds)
And then: Exception from a finished function: Error: 4 DEADLINE_EXCEEDED: Deadline exceeded
(I have thousands of documents inside "Schedine", "in corso")
Related
I am getting below error while running my dataflow. This dataflow was running fine till yesterday. From today onwards I am getting error like this
Operation on target LoadAccount failed:
[plugins.adfprod.AutoResolveIntegrationRuntime.45 WorkspaceType: CCID:<1a11d7e0-b019-4845-ab29-641100c79f04>] The job has surpassed the max number of seconds it can be in ResourceAcquisition state [1000s], so ending the job.
Error Message - The job has surpassed the max number of seconds it can
be in ResourceAcquisition state [1000s], so ending the job.
In a lot of cases of Data Factory the MAX limitations are only soft restrictions that can easily be lifted via a support ticket.
There is no such thing as a limitless cloud platform.
Refer this article by MRPAULANDREW
According to this response you can set a time limit for a query operation via find() parameter or a collection method:
cursor = db.collection.find(max_time_ms=1)
or
cursor = db.collection.find().max_time_ms(1)
The doc says:
max_time_ms (optional): Specifies a time limit for a query operation. If the specified time is exceeded, the operation will be aborted and ExecutionTimeout is raised. Pass this as an alternative to calling max_time_ms() on the cursor.
We're currently experiencing a problem that a query runs for ~30 minutes before it eats all the RAM and the server dies. I hope this parameter gives a hard limit on the query and after the given time the server gives up.
Since our app is full of finds and cursors: is there a way how to set this parameter directly in the MongoClient constructor?
The doc says:
socketTimeoutMS: (integer or None) Controls how long (in milliseconds) the driver will wait for a response after sending an ordinary (non-monitoring) database operation before concluding that a network error has occurred. Defaults to None (no timeout).
connectTimeoutMS: (integer or None) Controls how long (in milliseconds) the driver will wait during server monitoring when connecting a new socket to a server before concluding the server is unavailable. Defaults to 20000 (20 seconds).
serverSelectionTimeoutMS: (integer) Controls how long (in milliseconds) the driver will wait to find an available, appropriate server to carry out a database operation; while it is waiting, multiple server monitoring operations may be carried out, each controlled by connectTimeoutMS. Defaults to 30000 (30 seconds).
...couldn't find another timeout and none of these seem to be the equivalent of max_time_ms. Am I missing something?
I am running a create table query in Hiveql and obtain the following error when it is run:
Status: Failed
Counters limit exceeded: Too many counters: 2001 max=2000
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Counters limit exceeded: Too many counters: 2001 max=2000
I have attempted to set the counters to to a greater number, i.e.
set tez.counters.max=16000;
However, it still falls over with the same error.
My query incorporates 13 left joins but the data sets are relatively small (1,000's rows). The query did work when there were roughly 10 joins but since I've added additional joins it has started to fail.
Any suggestions on how I can configure this to work would be greatly appreciated!
You need to find real initial error log from failed container. The error you have shown here is not initial error. 2001 containers (including their restart attempts) have failed because of some other error (which you really need to fix), then all job was terminated, all other containers were killed because of Failed Counters limit. Go to Job tracker and find some failed (not killed) container and read it's log. The real problem is not in limit and changing the Failed Counters limit will not help.
Divide your query into multiple step and then run it.
As you said your query works with 10 joins,So first create the table which has data with first 10 joins and then with the new table,create other table which has data from first table and three other tables.
I faced the same issue as I was applying union all statement on 100 tables.But when I started to run only 10 tables at a time it works.
Hope This Helps!!!!
I have a python script that executes multiple sql scripts (one after another) in Redshift. Some of the tables in these sql scripts can be queried multiple times. For ex. Table t1 can be SELECTed in one script and can be dropped/recreated in another script. This whole process is running in one transaction. Now, sometimes, I am getting deadlock detected error and the whole transaction is rolled back. If there is a deadlock on a table, I would like to wait for the table to be released and then retry the sql execution. For other types of errors, I would like to rollback the transaction. From the documentation, it looks like the table lock isn't released until end of transaction. I would like to achieve all or no data changes (which is accomplished by using transaction) but also would like to handle deadlocks. Any suggestion on how this can be accomplished?
I would execute all of the SQL you are referring to in one transaction with a retry loop. Below is the logic I use to handle concurrency issues and retry (pseudocode for brevity). I do not have the system wait indefinitely for the lock to be released. Instead I handle it in the application by retrying over time.
begin transaction
while not successful and count < 5
try
execute sql
commit
except
if error code is '40P01' or '55P03'
# Deadlock or lock not available
sleep a random time (200 ms to 1 sec) * number of retries
else if error code is '40001' or '25P02'
# "In failed sql transaction" or serialized transaction failure
rollback
sleep a random time (200 ms to 1 sec) * number of retries
begin transaction
else if error message is 'There is no active transaction'
sleep a random time (200 ms to 1 sec) * number of retries
begin transaction
increment count
The key components are catching every type of error, knowing which cases require a rollback, and having an exponential backoff for retries.
while update 20000 record by the orientdb java api.got following warning message and start new another procces and update records from beginning .even though previous updating process is run, after updated 12000 record.
warning: connection re-acquired transparently after xxx ms and y retries : no errors will be thrown at application level
I tried to insert 20000 record by increasing time out period. but it doesn't work.
would please help me to stop, start new process.