I am getting the following error:
Conversion failed when converting the varchar value '2010-01-10' to data type int.
While running the following command in a query window in Management Studio:
SELECT * FROM (
SELECT CAST(group_id AS INT) AS qqq from view_jct_snapshot_group_items
) AS X
WHERE qqq = CAST(14290 AS INT)
However, when I run just this portion of the query, I get results and no errors:
SELECT CAST(group_id AS INT) AS qqq from view_jct_snapshot_group_items
Similarly with the following:
SELECT * FROM (
SELECT CAST(group_id AS INT) AS qqq from view_jct_snapshot_group_items
) AS X
What is going on, and how can I use my where clause without getting an error?
The table that is the source data for the view most likely has the value '2010-01-10' in the group_id column and this is excluded by a filter in the View itself.
Views are inline constructs expanded out by the query optimiser and it looks like your query is pushing the cast down to before the filter occurs.
Try
SELECT *
FROM (SELECT CAST(CASE
WHEN group_id NOT LIKE '%[^0-9]%' THEN group_id
END AS INT) AS qqq
FROM view_jct_snapshot_group_items) AS X
WHERE qqq = 14290
SELECT CAST(group_id AS INT) AS qqq from view_jct_snapshot_group_items
This will need to be able to cast all values in the view to int. Clearly at least one of the values is not an int. Try putting your WHERE clause in the derived table not outside of it.
In general it is bad idea to store data that is not the same datatype in one field. This cause many problems in writing queries and requires you to do a lot of what should be unnecessary work (at a performance cost system wide) to cast the data to the correct type. Personally I would revisit your desing to see why integers and dates are stored in the same field.
Related
I have next JSON:
{"promptnum":4,"corpuscode":"B0014","prompttype":"video","skipped":false,"transcription":"1","deviceinfo":{"DEVICE_ID":"exynos980","DEVICE_MANUFACTURER":"samsung","DEVICE_SERIAL":"unknown","DEVICE_DESIGN":"a51x","DEVICE_MODEL":"SM-A5160","DEVICE_OS":"android","DEVICE_OS_VERSION":"10","DEVICE_CARRIER":"","DEVICE_BATTERY_LEVEL":"70.00%","DEVICE_BATTERY_STATE":"unplugged","Current App Version":"1.1.0","Current App Build":"6"}}
I want to get values from 1-st level and 2-nd level.
1-st level: "promptnum":4,"corpuscode":"B0014","prompttype":"video","skipped":false,"transcription":"1","deviceinfo":...
2-nd level:
"deviceinfo":{"DEVICE_ID":"exynos980","DEVICE_MANUFACTURER":"samsung","DEVICE_SERIAL":"unknown","DEVICE_DESIGN":"a51x","DEVICE_MODEL":"SM-A5160","DEVICE_OS":"android","DEVICE_OS_VERSION":"10","DEVICE_CARRIER":"","DEVICE_BATTERY_LEVEL":"70.00%","DEVICE_BATTERY_STATE":"unplugged","Current App Version":"1.1.0","Current App Build":"6"}
When I parse 1-st level with
SELECT d.*
FROM (
SELECT c.json_parse, c.json_parse.deviceinfo AS device_info
FROM (
SELECT JSON_PARSE(file_attr)
FROM public.dc_ac_files
) AS c) AS d
it's work well.
But when I try to get values from 2-nd level with
SELECT d.*, l.DEVICE_ID
FROM (
SELECT c.json_parse, c.json_parse.deviceinfo AS device_info
FROM (
SELECT JSON_PARSE(file_attr)
FROM public.dc_ac_files
) AS c) AS d, d.device_info AS l
it doesn't work - no errors and no data.
If I know, it's right way to parse nested json, but it doesn't work for me.
Can you help me?
Viktor you have a couple of issues. First the notation "AS d, d.device_info AS l" is used to unnest arrays in your super data. You don't have any arrays to unnest so this is returning zero rows.
Second Redshift defaults to lower case for all column names so DEVICE_ID is being seen as device_id. You can enable case sensitive column names by setting the enable_case_sensitive_identifier connection variable to true and quoting all column names that require upper characters. "SET enable_case_sensitive_identifier TO true;" and changing l.DEVICE_ID to l."DEVICE_ID".
You also have unneeded layers in your query.
Putting all these together you can run:
SELECT l, l.deviceinfo, l.deviceinfo."DEVICE_ID"
FROM (
SELECT JSON_PARSE(file_attr) AS l
FROM public.dc_ac_files
) AS c
You also don't need SUPER data type to perform this. This can be done with json string parsing functions.
SELECT file_attr, json_extract_path_text(file_attr, 'deviceinfo') as deviceinfo, json_extract_path_text(file_attr, 'deviceinfo','DEVICE_ID') as device_id
FROM public.dc_ac_files
I'm trying to sum values from a jsonb type column in a table in an Aurora/Postgres database but it doesn't seem to work.
select (payload->>'loanAmount')::int from rfqs limit 1;
Gives a results of 10000 (int4).
select sum((payload->>'loanAmount')::int) from rfqs limit 1;
Gives a result of: ERROR: invalid input syntax for integer: "2000.5"
It seems like this is something to do with the way the ->> operator converts the json to a string, but it's like something is wrong with that string which prevents it from being correctly typecast to an int.
As a test I did select SUM(('10000'::int)); which worked fine and returned 10000 as expected.
Any ideas?
This will allow you to understand (you will see what the problem is with "::int")
select sum(payload->>'loanAmount') from rfqs
which is the same as:
select sum(payload->>'loanAmount') from rfqs limit 1
(An aggregate without group by returns only on row, so "limit 1" is a bit superfluous)
Try
SELECT sum(to_number((payload->>'loanAmount'),'999999999D9999')) from rfqs
see http://www.sqlfiddle.com/#!17/9c30a/8
Some of your "loanAmount" properties does not have integer value. First record does though.
To find offenting records:
SELECT payload FROM rfqs WHERE (payload->>'loanAmount') <> trunc(payload->>'loanAmount')
Is there a way to use the results of a function call in the order by clause?
My current attempt (I've also tried some slight variations).
SELECT it.item_type_id, it.asset_tag, split_part(it.asset_tag, 'ASSET', 2)::INT as tag_num
FROM serials.item_types it
WHERE it.asset_tag LIKE 'ASSET%'
ORDER BY split_part(it.asset_tag, 'ASSET', 2)::INT;
While my general assumption is that this can't be done, I wanted to know if there was a way to accomplish this that I wasn't thinking of.
EDIT: The query above gives the following error [22P02] ERROR: invalid input syntax for integer: "******"
Your query is generally OK, the problem is that for some row the result of split_part(it.asset_tag, 'ASSET', 2) is the string ******. And that string cannot be cast to an integer.
You may want to remove the order by and the cast in the select list and add a where split_part(it.asset_tag, 'ASSET', 2) = '******', for instance, to narrow down that data issue.
Once that is resolved, having such a function in the order by list is perfectly fine. The quoted section of the documentation in the comments on the question is referring to applying an order by clause to the results of UNION, INTERSECTION, etc. queries. In other words, the order by found in this query:
(select column1 as result_column1 from table1
union
select column2 from table 2)
order by result_column1
can only refer to the accumulated result columns, not to expressions on individual rows.
I thought I understood how I can do a SELECT from the results of another SELECT statement, but there seems to be some sort of blurring of scope that I don't understand. I am using SQL Server 2008R2.
It is easiest to explain with an example.
Create a table with a single nvarchar column - load the table with a single text value and a couple of numbers:
CREATE TABLE #temptable( a nvarchar(30) );
INSERT INTO #temptable( a )
VALUES('apple');
INSERT INTO #temptable( a )
VALUES(1);
INSERT INTO #temptable( a )
VALUES(2);
select * from #temptable;
This will return: apple, 1, 2
Use IsNumeric to get only the rows of the table that can be cast to numeric - this will leave the text value apple behind. This works fine.
select cast(a as int) as NumA
from #temptable
where IsNumeric(a) = 1 ;
This returns: 1, 2
However, if I use that exact same query as an inner select, and try to do a numeric WHERE clause, it fails saying cannot convert nvarchar value 'apple' to data type int. How has it got the value 'apple' back??
select
x.NumA
from
(
select cast(a as int) as NumA
from #temptable
where IsNumeric(a) = 1
) x
where x.NumA > 1
;
Note that the failing query works just fine without the WHERE clause:
select
x.NumA
from
(
select cast(a as int) as NumA
from #temptable
where IsNumeric(a) = 1
) x
;
I find this very surprising. What am I not getting? TIA
If you take a look at the estimated execution plan you'll find that it has optimized the inner query into the outer and combined the WHERE clauses.
Using a CTE to isolate the operations works (in SQL Server 2008 R2):
declare #temptable as table ( a nvarchar(30) );
INSERT INTO #temptable( a )
VALUES ('apple'), ('1'), ('2');
with Numbers as (
select cast(a as int) as NumA
from #temptable
where IsNumeric(a) = 1
)
select * from Numbers
The reason you are getting this is fair and simple. When a query is executed there are some steps that are being followed. This is a parse, algebrize, optimize and compile.
The algebrize part in this case will get all the objects you need for this query. The optimize will use these objects to create a best query plan which will be compiled and executed...
So, when you look into that part you will see it will do a table scan on #temptable. And #temptable is defined as the way you created your table. That you will do some compute on it is a different thing..... The column still has the nvarchar datatype..
To know how this works you have to know how to read a query. First all the objects are retrieved (from table, inner join table), then the predicates (where, on), then the grouping and such, then the select of the columns (with the cast) and then the orderby.
So with that in mind, when you have a combination of selects, the optimizer will still process it that way.. since your select is subordinate to the from and join parts of your query, it will be a reason for getting this error.
I hope i made it a little clear?
The optimizer is free to move expressions in the query plan in order to produce the most cost efficient plan for retrieving the data (the evaluation order of the predicates is not guaranteed). I think using the case expression like bellow produces a NULL in absence of the ELSE clause and thus takes the APPLE out
select a from #temptable where case when isnumeric(a) = 1 then a end > 1
I'm modifying an existing query for a client, and I've encountered a somewhat baffling issue.
Our client uses SQL Server 2008 R2 and the database in question provides the user the ability to specify custom fields for one of its tables by making use of an EAV structure. All of the values stored in this structure are varchar(255), and several of the fields are intended to store dates. The query in question is being modified to use two of these fields and compare them (one is a start, the other is an end) against the current date to determine which row is "current".
The issue I'm having is that part of the query does a CONVERT(DateTime, eav.Value) in order to turn the varchar into a DateTime. The conversions themselves all succedd and I can include the value as part of the SELECT clause, but part of the question is giving me a conversion error:
Conversion failed when converting date and/or time from character string.
The real kicker is this: if I define the base for this query (getting a list of entities with the two custom field values flattened into a single row) as a view and select against the view and filter the view by getdate(), then it works correctly, but it fails if I add a join to a second table using one of the (non-date) fields from the view. I realize that this might be somewhat hard to follow, so I can post an example query if desired, but this question is already getting a little long.
I've tried recreating the basic structure in another database and including sample data, but the new database behaves as expected, so I'm at a loss here.
EDIT In case it's useful, here's the statement for the view:
create view Festival as
select
e.EntityId as FestivalId,
e.LookupAs as FestivalName,
convert(Date, nvs.Value) as ActivityStart,
convert(Date, nve.Value) as ActivityEnd
from tblEntity e
left join CustomControl ccs on ccs.ShortName = 'Activity Start Date'
left join CustomControl cce on cce.ShortName = 'Activity End Date'
left join tblEntityNameValue nvs on nvs.CustomControlId = ccs.IdCustomControl and nvs.EntityId = e.EntityId
left join tblEntityNameValue nve on nve.CustomControlId = cce.IdCustomControl and nve.EntityId = e.EntityId
where e.EntityType = 'Festival'
The failing query is this:
select *
from Festival f
join FestivalAttendeeAll fa on fa.FestivalId = f.FestivalId
where getdate() between f.ActivityStart and f.ActivityEnd
Yet this works:
select *
from Festival f
where getdate() between f.ActivityStart and f.ActivityEnd
(EntityId/FestivalId are int columns)
I've encountered this type of error before, it's due to the "order of operations" performed by the execution plan.
You are getting that error message because the execution plan for your statement (generated by the optimizer) is performing the CONVERT() operation on rows that contain string values that can't be converted to DATETIME.
Basically, you do not have control over which rows the optimizer performs that conversion on. You know that you only need that conversion done on certain rows, and you have predicates (WHERE or ON clauses) that exclude those rows (limit the rows to those that need the conversion), but your execution plan is performing the CONVERT() operation on rows BEFORE those rows are excluded.
(For example, the optimizer may be electing to a do a table scan, and performing that conversion on every row, before any predicate is being applied.)
I can't give a specific answer, without a specific question and specific SQL that is generating the error.
One simple approach to addressing the problem would be to use the ISDATE() function to test whether the string value can be converted to a date.
That is, replace:
CONVERT(DATETIME,eav.Value)
with:
CASE WHEN ISDATE(eav.Value) > 0 THEN CONVERT(DATETIME, eav.Value) ELSE NULL END
or:
CONVERT(DATETIME, CASE WHEN ISDATE(eav.Value) > 0 THEN eav.Value ELSE NULL END)
Note that the ISDATE() function is subject to some significant limitations, such as being affected by the DATEFORMAT and LANGUAGE settings of the session.
If there is some other indication on the eav row, you could use some other test, to conditionally perform the conversion.
CASE WHEN eav.ValueIsDateTime=1 THEN CONVERT(DATETIME, eav.Value) ELSE NULL END
The other approach I've used is to try to gain some modicum of control over the order of operations of the optimizer, using inline views or Common Table Expressions, with operations that force the optimizer to materialize them and apply predicates, so that happens BEFORE any conversion in the outer query.