I have to execute a query on Pervasive but some rows have wrong data, and I'm getting "Error row"-msg, if I exclude the rows with errors the query works well, but I have some cases when the number of fields is so huge and I can't know exactly which row has the errors.
I checked the log but doesn't say nothing interesting
I want to know if exist some statement for check that or any tool that can helps.
Thanks in advance
Related
I use datagrip as client to connect redshift and encounter a stranger issue which exhaust my whole day.
When I run my query sql the datagrip complains
[XX000] ERROR: invalid string enlargement request size 1073741823
It seemed that there dont exist a place that I can check more detail error log. And I google this error it also have very little similar question and it seemed maybe due to my field is too long which exceed the max length that redshift can accept. But actually, the story is not such for me I dont have long field, then I comment all my sql statement and re-add them incrementally to locate this issue statement.
Finally, I find the error-msg-triggered statement as below:
(
case when trunc(request_date_skip_weekend_tmp) = to_date('2022-03-21', 'YYYY-MM-DD')
then dateadd(day, 1, trunc(request_date_skip_weekend_tmp))
else request_date_skip_weekend_tmp end
)
request_date_skip_weekend,
After I change it with:
dateadd(day, 1, trunc(request_date_skip_weekend_tmp)) request_date_skip_weekend,
the error complain disappear, it is very hard for me to accept the relationship error message and the sql change, I dont know why my the former statement will trigger error complain.
I will appreciate if you can spot why the former expression error or share some knowledge about where can I fetch more detail error message to know what happened.
Your code snippet is dates and timestamps but the error is for strings. So it is likely you have identified a "trigger" and not root cause. Also since you report that the SQL is very long you could be dealing with compiler optimization changes, moving the failure. Removing a CASE can cause the compiler/optimizer to choose different structures for the query.
One experiment to try is to change the to_date() to a cast to timestamp so there are no implicit casts ('2022-03-21'::timestamp). This is unlikely the cause but it may help.
I expect you will need to post the query to get more help. How large is it? This error could be related to building a large string in the query OR could be related to the text of the query OR creating the output. This isn't a standard "string too long" message so this is something more implicit. You could post to a google doc or some other file sharing service. Just link in the question.
I get an odd error when trying to query too many dates from a date-partitioned historical database:
q)eod: h"select from eod where date within 2018.01.01 2018.04.22"
'/tablepath/2018.04.04/eod/somecolumn: invalid host
q)eod: h"select from eod where date within 2018.01.17 2018.04.20"
'/tablepath/2018.04.20/eod/othercolumn: invalid host
q)eod: h"select from eod where date within 2018.01.18 2018.04.20"
q)
Note that both dates mentioned in the error messages are within the date range that we manage to extract in the end, and that it fails on a different column each time. This seems to indicate it's something to do with the size of the table being pulled, but when we check the size of the largest table we managed to get:
q)(-22!eod) % 1024 * 1024
646.9043
q)count eod
2872546
we find that it's not particularly large by either memory size nor by number of rows.
Googling for "invalid host" errors doesn't seem to turn up anything relevant, and I'm not seeing anything in the kdb docs about size limits that would be relevant. Anyone got any ideas?
Edit:
When loading the table in a session and making the queries directly, we get what appears to be the same error, but with a different message. For instance:
q)jj: select from eod where date within 2018.01.01 2018.04.22
Too many compressed files open
k){0!(?).#[x;0;p1[;y;z]]}
'./2018.04.04/eod/settlecab: No such file or directory
.
?
(+`exch`date`class..
q.Q))
Note that the file ./2018.04.04/eod/settlecab does in fact exist, and contains data:
I have no problem loading the data for just the date mentioned in the error, and the column mentioned has meaningful values:
q)jj: select from eod where date=2018.04.04
q)select count i by settlecab from jj
settlecab| x
---------| -----
0 | 41573
1 | 2269
The key point seems to be the Too many compressed files open message, but what can I do about this?
Edit for Summary/Solutions:
The table in question had many columns, all stored in a compressed format. When issuing a query against too many dates at once, kdb would try to mmap all of those columns at once, running into a limit on how many compressed files could be open at once.
Once I understood the problem, several solutions were available:
I could pull only certain columns from the database, reducing the number of files that kdb needed to keep open,
I could force kdb to pull all the data into memory by adding a dummy where clause to the query, such as (null column) | not null column (hacky, but it works),
I could have upgraded the kdb version and lifted OS limits (not practical in my case).
I still have no idea why this resulted in an invalid host error when querying the database remotely.
First off, can we just clarify the database structure you're working with. It seems from the filepaths returned in your errors that you've got a date-partitioned database. Did you mean non-segmented database when you said non-partitioned in your original query?
In terms of a fix for your issue, have you tried loading your database into a session, and making those queries directly? If so do you get the same issues?
If that seems to be working alright, the problem might lie with how you're defining your database handle. How is h defined in your original example?
It might also be worth trying to select individual dates from your database, to try and isolate the problem, and to determine if it lies with your on-disk data. Try specifically querying the dates that are mentioned in your errors.
You could also try performing your original queries with a subset a columns, again to try and pinpoint where your issue is coming from.
Let us know if you get any further with this.
Joseph
date;intvalue
"2016-12-27";1
"2016-12-27";2
"2016-12-27";3
"2016-12-27";4
"2016-12-28";1
"2016-12-28";2
"2016-12-28";3
"2016-12-28";4
In this resultset, who i could in a where condition remove the last's three lines?
If I put (date!='2016-12-28' and intvalue>1) then all rows are removed.. only the first remains..
Should have a way which the where the arguments of 'where clause' can be processed by row and not by column..
Can anyone help me and give me a tip in this problem?
thanks and sorry my english.
It's not clear what is the algorithm. If you need the get just the first 5 rows from the resultset, "limit 5" is good, as #Vao Tsun suggested. But if the resultset can change and you need the get records according to a where clause, depending on your needs you can play with logical operands, e.g.
where not (myDate>='2016-12-28' and myInt>1)
where (myDate<'2016-12-28' or myInt=1)
where (myDate<='2016-12-27' or (myDate='2016-12-28' and myInt=1))
etc.
I get the following error when running an sqr report on DB2:
SQL0100W - No row was found for FETCH, UPDATE or DELETE; or the result of a query is an empty table. SQLSTATE=02000
The sql in question runs correctly when I paste it into RapidSQL, replacing the parameters. The sql in question is an insert-select. No rows are returned by the select, and this is fine... I expect the report to be blank for my parameters.
Any idea how I can get around this?
DB2 returns always an SQL0100 warning (this is a warning, not an error - errors would have negative values) when no rows are returned. That's the way it is.
I don't know peoplesoft at all - so I can't give you any pointers with that. Back when I was programming for DB2 we ignored those SQL0100 warnings.
If SQR can't gracefully handle a NOT_FOUND SQL0100 return, then code a preliminary query to return a count of the number of rows that satisfy the conditions of the actual query. Check the result of the count in an if-then block in SQR to run the actual query if and only if the row count returned by the preceding query was not zero.
Turns out to be an environment setup issue. Got resolved with no change from me after a couple of builds....
Strange :-/
if you delete delete more than one record using logic operation like delete from tabname where columnnmae=deleterecord and columnnmae=deleterecord then they show this type error.machine an
If you've got a very complicated SELECT statement and some records aren't included because of a join, what is the easiest way to debug this and find the reasons why?
Change the JOINS / INNER JOINS to OUTER JOINS and look for NULLs where they shouldn't be.
You could include your ON logic in the WHERE clause, phrase it like this:
WHERE 1=1
AND...
AND...
and just comment out as many of the terms until you isolate the unexpected behaviour.
I don't know if this will help you, but when I find myself with a complex Select that I'm having a hard time maintaining, or debugging, I'll break it up into separate common table expressions (CTE's). I've found this makes many of my queries much easier to understand and maintain.
Binary Search stylee:
If you have 10 joins, comment out the last 5.
Still have the problem? comment out the last 2/3 joins that are still uncommented
Still have the problem? comment out the last 1/2 joins that are still uncommented
Do this until you get down to it working, then the problem will lie in the last joins you commented out.
Yes you could do them one at a time but this is usually quicker.
Obviously you will have to comment out all the columns not used in the select statement, but I normally just put /* */ around all the columns, then put a * instead.
Just look at the number of results returned.