I'm using SQL Server Express 2008 and I'm doing a bulk insert of data. I'd like to have more verbose error messages, ideally printing the data that failed to be inserted. Is that possible?
It is possible, but it can require a lot of effort to to do this--I recall working on a subsystem for a few days before I got it to do everything it needed to do. I believe this is one of the (few but still too many) places where, upon hitting an error, SQL will return two (2) error messages back-to-back, the second message is vague and indistinct, and all the error handling functions can only access info pertaining to that second lame message, and not the first one where the real info is. I don't have the code in front of me, but the logic was something like:
Use the "errorfile" option on BULK INSERT to generate an error file IF the bulk insert fails
TRY/CATCH the bulk insert call, and carefully check the returned error number
If the error is the appropriate type, open and read the contents of the file to determine what went wrong where, and build your error message around that
Awkward as anything, but ultimately it worked out pretty well. So long as the drive+path+filename you were inserting from didn't exceed 128 characters (in SQL 2005, and I just bet they didn't fix that in 2008.) I do not count Bulk Insert as one of my favorite commands.
Related
Is it possible to create a warning message in a trigger in Firebird 2.5?
I know I can create an exception message which will stop the user from saving the record changes, but in this instance I don't mind if the user continues.
Could I call a procedure that generates the message?
There is no mechanism in Firebird to produce warnings in PSQL code, you can only raise exceptions, which in triggers will result in the effect of the executed statement that fired the trigger to be undone.
In short, this is not possible.
There are workarounds possible, but those would require 'external' protocols, like, for example, inserting the warning message into a global temporary table, requiring the calling code to explicitly select from that temporary table after execution.
SQL model does provide putting query on pause and then waiting for extra input from client to either unfreeze it or fail it. SQL is not user-interactive service and there is no confirmation dialogs. You have to rethink your application design.
One possible avenue, nominally staying withing 2-tier client-server framework, would be creating temporary tabless for all the data you want to save (for example transaction-scope GTTs), and then have TWO stored procedures. One SP would be sanity-checking and returning list of warnings, if any. Another SP then would dump the data from GTTs to main, persistent tables without doing those checks.
Your client app would select warnings from the check-SP first, if it returns any then show them to the user, then either call save-SP and commit, or rollback without calling save-SP.
This is abusing C/S idea, so there would be dragons. First of all, you would have to have several GTTs and two SPs for E-V-E-R-Y pausable data saving in your app. And that can be a lot.
Also, notice, that database data may change after you called check-SP and before you called save-SP. Becuse some OTHER application running elsewhere could be changing and committing data during that pause. Especially if you transaction was of READ COMMMITTED kind. But with SNAPSHOT tx too.
Better approach would be to drop C/S scheme and go to 3-tier model, AKA multi-tier, AKA "Application Server". That way your client app sends the "briefcase" of data to the app-server, it would be app-server (not SQL triggers) doing all the data validation, and then it would be saving it to data storage backend, SQL or any other.
There, of course, still would be that problem, that data could had been changed by other users, why you paused one user and waited him to read and decide. But you would have more flexibility in app-server on data reconcilation, than you would have with plain SQL.
I realize this is a bad question, but I don't know where else to turn.
can someone point me to where I can find the list of reports failure codes for IBM? I've tried searching for it in the IBM documentation, and in general google search, but this particular error is unique and I've never seen it before.
I'm trying to find out what code 262148 means.
Background:
I built a datastage job that has:
ORACLE CONNECTOR --> TRANSFORMER -> HIERARCHICAL DATA
The intent is to pull data from a ORACLE table, and output the response of the select statement into a JSON file. I'm using the HIERARCHICAL stage to set it. When tested in the stage, no problems, I see the JSON output.
However, when I run the job, it squawks:
reports failure code 262148
then the job aborts. There are no warnings, no signs, no errors prior to this line.
Until I know what it is, I can't troubleshoot.
If someone can point me to where the list of failure codes are, i can proceed.
Thanks!
can someone point me to where I can find the list of reports failure codes for IBM?
Here you go:
https://www.ibm.com/support/knowledgecenter/en/ssw_ibm_i_73/rzahb/rzahbsrclist.htm
While this list does not list your specific error code, it does categorize many other codes, and explains how the code breakdown works. While this list is not specifically for DataStage, in my experience IBM standards are generally compatible across different products. In this list every code that starts with a 2 is a disk failure, so maybe run a disk checker. That's the best I've got as far as error codes.
Without knowledge of the inner workings of the product, there is not much more you can do beyond checking system health in general (especially disk, network and permissions in this case). Personally, I prefer to go get internal knowledge whenever exterior knowledge proves insufficient. I would start with a network capture, as I'm sure there's a socket involved in the connection between the layers. Compare the network capture from when the select statement is run from the hierarchical from one captured when run from the job. There may be clues in there, like reset or refused connections.
I see how to debug queries stored as Functions in the database. But my problem is with an external QGIS plugin that connects to my Postgres 10.4 via network and does a complex query and calculations, and stores the results back into PostGIS tables:
FOR r IN c LOOP
SELECT
(1 - ST_LineLocatePoint(path.geom, ST_Intersection(r.geom, path.geom))) * ST_Length(path.geom)
INTO
station
(continues ...)
When it errors, it just returns that line number as the failing location, but no clue where it was in the loop through hundreds of features. (And any features it has processed are not stored to the output tables when it fails.) I totally don't know enough about the plugin and about SQL to hack the external query, and I suspect if it was a reasonable task the plugin author would have included more revealing debug messages.
So is there some way I could use pgAdmin4 (or anything) from the server side to watch the query process? Even being able to see if it fails the first time through the loop or later would help immensely. Knowing the loop count at failure would point me to the exact problem feature. Being able to see "station" or "r.geom" would make it even easier.
Perfectly fine if the process is miserably slow or interferes with other queries, I'm the only user on this server.
This is not actually a way to watch the RiverGIS query in action, but it is the best I have found. It extracts the failing ST_Intersects() call from the RiverGIS code and runs it under your control, where you can display any clues you want.
When you're totally mystified where the RiverGIS problem might be, run this SQL query:
SELECT
xs."XsecID" AS "XsecID",
xs."ReachID" AS "ReachID",
xs."Station" AS "Station",
xs."RiverCode" AS "RiverCode",
xs."ReachCode" AS "ReachCode",
ST_Intersection(xs.geom, riv.geom) AS "Fraction"
FROM
"<your project name>"."StreamCenterlines" AS riv,
"<your project name>"."XSCutLines" AS xs
WHERE
ST_Intersects(xs.geom, riv.geom)
ORDER BY xs."ReachID" ASC, xs."Station" DESC
Obviously replace <your project name> with the QGIS project name.
Also works for the BankLines step if you replace "StreamCenterlines" with "BankLines". Probably could be adapted to other situations where ST_Intersects() fails without a clue.
You'll get a listing with shorter geometry strings for good cross sections and double-length strings for bad ones. Probably need to widen your display column a lot to see this.
Works for me in pgAdmn4, or in QGIS3 -> Database -> DB Manager -> (click the wrench icon). You could select only bad lines, but I find the background info helpful.
I'm running jobs through Datastage with the DELETE then INSERT connector. I'm having several jobs failing for this error:
DB2_Connector: DB2 reported: SQLSTATE = 02000 Native Error Code = 100, Msg = IBM[CLIDriver][DB2/NT64] SQL01000W No row was found for FETCH, UPDATE, or DELETE
When I run the delete statement in Data Studio directly in DB2, it gives this same error so I know it's a DB2 error, not a Datastage error.
Is there anyway to supress the message in Datastage or when I run the statement in DB2 is there anyway I can keep that message from coming up? It's stopping my DS jobs now with a Fatal error and not continuing to load.
There has got to be a way to turn off the message. I know in SQL Server if no rows are found it does not give this error, it just says zero or doesn't return records but in DB2 this error is coming up and I'm not sure if there is a way to turn it off.
First of all, you seem to be confused about precisely what an error is, and what a message is.
An error is when something goes wrong.
A message is when some piece of software is kind enough to let you know that something went wrong.
From this it follows that suppressing a message has no bearing whatsoever on the actual error. Your software is not failing because of the message, your software is failing because something is going wrong. Receiving a message about it is actually a good thing: the alternative would be your software failing without you being given any clue whatsoever as to what is going wrong.
Suppressing or otherwise ignoring errors is like hiding your head in the sand: you are still going to end up as meal.
So, what you need to make go away is the error, not the message.
Which means that you have to figure out what you did wrong.
Luckily, you have the message giving you a hint as to what you did wrong, though you have to keep in mind that messages are sometimes misleading.
SQLState 02000 is not an error, it is a warning. (And note that DB2_Connector is not saying ERROR!!!1!:, it is saying DB2 reported:.) Luckily JDBC issues warnings when it detects situations that might be indicative of errors; there is a lot of software out there that ignores JDBC warnings, (essentially hiding your head in the sand for you, how nice,) but luckily DB2_Connector reports them.
What this means is that one of two things is going wrong:
Either your assumption that it is okay if no rows are found is wrong, and the fact that no rows were found is the cause of your problem, which means that you have to somehow make sure that some rows are found, or
Your assumption that it is okay if no rows are found is correct, in which case the warning reported has absolutely nothing to do with the problem at hand, so it can safely be ignored, and you have to look at the problem elsewhere.
TSQL here. Specifically Server 2008(literally just upgraded)
Concerning stored procedures: Try/Catch
I was trying to make a list of cases when a Select Statement will throw an exception. The ones I can think of are syntax related(includes null variables) and divide by zero. I'm only guessing there are just a whole boat load of them for Insert/Alter and Create/Truncate.
If you happen to know of a good source link, that would be great.
This question came up when I was reading this exhaustive blog post about error handling for SQL server. It's titled for SQL Server 2000, but I think most of it still applies.
edit
Sorry, I meant to link this earlier. . .
http://msdn.microsoft.com/en-us/library/aa175920(v=sql.80).aspx
Outside for compile ("didnt' run") errors, you have at least these runtime errors
arithmetic errors
These change based on various SET statement
Example: get sql server to warn about truncation / rounding
overflow errors
example: one of the rows overflows smallint in some calculation
CAST errors
eg you try ISNUMERIC in a WHERE or CASE and try to cast 'bob^' or 1.23 to int
See Why use Select Top 100 Percent?
However, you'd always want to use TRY/CATCH though, surely...?
Adding to gbn's post, you can also get locking errors like lock wait timeouts and deadlocks.
If you are referencing #Temp tables, you can get "Invalid object name '#Temp'" errors, because these are unbound until the statement executes.
If you are in READ UNCOMMITTED or WITH (NOLOCK), you can get error: 601 - "Could not continue scan with NOLOCK due to data movement."
If your code runs .NET code, you would probably get exceptions from there.
If your code selects from a remote server, you could a whole different set of errors about connections.