Azure Runbook for SQL procedure execution :Facing Timeout Issues - powershell

I have created a run-book in azure to automate my sproc execution on a scheduled interval, i see a timeout exception while the sproc gets executed.
I have tried changing the connection timeout value in my connection string to "0" to make it indefinite but I still see the same issue reoccurring. When I execute this stored procedure it hardly takes 3 minutes but the run-book fails with timeout exception.Can someone help me if I am missing anything.
Runbook type : PowerShell Workflow Runbook
Thanks,
Manoj.

Setting your connection string timeout to 0 will only affect the connection timeout, in other words you are giving your process infinite time to establish a connection with the database. Without seeing your code, I suspect the issue is with your command timeout - your stored proc is likely taking longer than the default 30s to complete. Before executing the stored proc you should set an appropriate timeout on the SqlCommand instance:
# Create command with a 10 minute timeout
$DatabaseCommand = New-Object System.Data.SqlClient.SqlCommand
$DatabaseCommand.Connection = $DatabaseConnection
$DatabaseCommand.CommandTimeout = 600
$DatabaseCommand.CommandText = "..."
# Execute non query
$NonQueryResult = $DatabaseCommand.ExecuteNonQuery()

Related

Error with SQL query: The query failed to parse. Exception from HResult: 0x80040E14

I am using SSIS to create a data flow task to a postgresql server database.
I get the Error with SQL query:
The query failed to parse. Exception from HResult: 0x80040E14
See below screenshot:
https://ibb.co/7KcBnMG
https://ibb.co/zR093SQ
The query being tried is:
INSERT INTO public.controlflow_example(rollnumber) VALUES (1)
The connection itself is fine. The schema is public, the table is indeed spelled controlflow_example, and the column is integer type named rollnumber
Even using
SELECT *
FROM public.controlflow_example
as an even simpler query gives the same error.
If i try to run the package it fails with the following error:
SSIS package "C:\Users\AJ\Documents\Visual Studio 2017\Projects\control_flow_example\control_flow_example\Package.dtsx" starting.
Error: 0xC002F210 at Execute SQL Task, Execute SQL Task: Executing the query "INSERT INTO public.controlflow_example(rollnumber)..." failed with the following error: "Exception from HRESULT: 0x80040E14". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Task failed: Execute SQL Task
Warning: 0x80019002 at Package: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "C:\Users\AJ\Documents\Visual Studio 2017\Projects\control_flow_example\control_flow_example\Package.dtsx" finished: Failure.
The program '[15288] DtsDebugHost.exe: DTS' has exited with code 0 (0x0).
Any advice please?
I have already searched for similar questions here and elsewhere but could not find a resolution.
Thank you.
Maybe you should change the function name and use private it works properly.
INSERT INTO public.controlflow_example column name VALUES ('1');
Are you trying to store the result set into an SSIS object? The result error means you did not set up the result properly. If you are not expecting any results back, then set result set to be nothing. If you are, then check to make sure you’re returning the results correctly.
Check out how to return results https://www.google.com/amp/s/www.red-gate.com/simple-talk/sql/ssis/ssis-basics-using-the-execute-sql-task-to-generate-result-sets/amp/
Good Luck

SQL Timeout when running an Entity Framework Migration

I am currently experiencing a SQL Timeout when running a SQL() command inside of an EF Migration.
Situation:
I am replacing a bunch (>50) tables with one table, and need to convert the data from those tables I'm about to drop into the new table. I've organized the Migration the following way:
1.Create the new table.
In the same Migration, use the SQL() function to run a sql script that migrates the data.
3.Drop all the old tables.
Currently, the migration gives the following error:
System.Data.SqlClient.SqlException (0x80131904): Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
The statement has been terminated. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
The error is happening in an environment where I give them an installer, and they run it without me involved, so I can't manually run individual migrations, and pause in the middle to run the SQL script.
Is there any way to change the timeout for a connection, or get around this issue?
Environment:
EF 6.0 Code First
SQL Server 2012
See this answer.
Use Configuration.cs file to set custom time out:
internal sealed class Configuration :
DbMigrationsConfiguration<ApplicationDbContext>
{
public Configuration()
{
AutomaticMigrationsEnabled = false;
ContextKey = "YourDbContext";
// New timeout in seconds
this.CommandTimeout = 60 * 5;
}
}
With this method, you can change the timeout only for migration and not everyone using your default connection string.
You should be able to set the connection timeout in the connection string, something like:
Connection Timeout=180;

Powercenter SQL1224N error connecting DB2

Im running a workflow in powercenter that is constatnly getting an SQL1224N error.
This process execute a query against one table (POLIZA) with 800k rows, it retrieves the first 10k rows and then it start to execute to another table with 75M rows, at ths moment in DB2 an idle thread error appear but the PWC process still running retrieving the 75M rows, when it is completed (after 20 minutes) the errros comes up related with the first table:
[IBM][CLI Driver] SQL1224N A database agent could not be started to service a request, or was terminated as a result of a database system shutdown or a force command. SQLSTATE=55032
sqlstate = 40003
[IBM][CLI Driver] SQL1224N A database agent could not be started to service a request, or was terminated as a result of a database system shutdown or a force command. SQLSTATE=55032
sqlstate = 40003
Database driver error...
Function Name : Fetch
SQL Stmt : SELECT POLIZA.BSPOL_BSCODCIA, POLIZA.BSPOL_BSRAMOCO
FROM POLIZA
WHERE
EXA01.POLIZA.BSPOL_IDEMPR='0015' for read only with ur
Native error code = -1224
DB2 Fatal Error].
I have a similar process runing against the same 2 tables and it is woking fine where the only difference I can see is that the DB2 user is different.
Any idea how can i fix this?
Regards
The common causes for -1224 are:
Your instance or database has crashed, or
Something/somebody is forcing off your application (FORCE APPLICATION or equivalent)
As for the crash, I think you would know by know. This typically requires a database or instance restart. At any rate, can you please have a look into your DIAGPATH to check for any FODC* directories whose timestamp would match the timestamp of the -1224 errors?
As for the FORCE case, you should find some evidence of the -1224 in db2diag.log. Try searching for the decimal -1224, but also for its hex representation (0xFFFFFB38).

OBDC and ExecuteReader issues

I'm trying to do a little powerShell script, which establishes a OBDC connection using SQL Server Native Client 11.0 as Driver and retrieve some data which, in my case, is just getting the list to the database tables.
##### Start the database connection and set up environment
$DbString="Driver=SQL Server Native Client 11.0;Server=[server_name];Database=[db_name];Uid=[user];Pwd=admin#[pwd];Encrypt=yes;Connection Timeout=30;"
$DBConnection=New-Object System.Data.Odbc.OdbcConnection
$DBCommand=New-Object System.Data.Odbc.OdbcCommand
$DBConnection.ConnectionString=$DbString
$DBConnection.Open()
$DBCommand.Connection=$DBConnection
$sqlQuery = "show tables;"
$DBCommand.CommandText = $sqlQuery
$DBCommand.ExecuterReader() //Crashes
The problem is when I run it, I get this problem
ERROR [42000] [Microsoft][SQL Server Native Client 11.0][SQL Server] Could not find stored procedure 'show'
It seems like the problem comes from ExecuteReader but I can't find a way out, please help !

"Lost connection to MySQL server during query" in Google Cloud SQL

I am having a weird, recurring but not constant, error where I get "2013, 'Lost connection to MySQL server during query'". These are the premises:
a Python app runs around 15-20minutes every hour and then stops (hourly scheduled by cron)
the app is on a GCE n1-highcpu-2 instance, the db is on a D1 with a per package pricing plan and the following mysql flags
max_allowed_packet 1073741824
slow_query_log on
log_output TABLE
log_queries_not_using_indexes on
the database is accessed only by this app and this app only so the usage is the same, around 20 consecutive minutes per hour and then nothing at all for the other 40 minutes
the first query it does is
SELECT users.user_id, users.access_token, users.access_token_secret, users.screen_name, metadata.last_id
FROM users
LEFT OUTER JOIN metadata ON users.user_id = metadata.user_id
WHERE users.enabled = 1
the above query joins two tables that are each around 700 lines longs and do not have indexes
after this query (which takes 0.2 seconds when it runs without problems) the app starts without any issues
Looking at the logs I see that each time this error presents itself the interval between the start of the query and the error is 15 minutes.
I've also enabled the slow query log and those query are registered like this:
start_time: 2014-10-27 13:19:04
query_time: 00:00:00
lock_time: 00:00:00
rows_sent: 760
rows_examined: 1514
db: foobar
last_insert_id: 0
insert_id: 0
server_id: 1234567
sql_text: ...
Any ideas?
If your connection is idle for the 15 minute gap the you are probably seeing GCE disconnect your idle TCP connection, as described at https://cloud.google.com/compute/docs/troubleshooting#communicatewithinternet. Try the workaround that page suggests:
sudo /sbin/sysctl -w net.ipv4.tcp_keepalive_time=60 net.ipv4.tcp_keepalive_intvl=60 net.ipv4.tcp_keepalive_probes=5
(You may need to put this configuration into /etc/sysctl.conf to make it permanent)