Can you tell me why this is throwing an error in Redshift?
WITH Testing_PADDING AS (SELECT '12345678' AS column1)
SELECT LPAD(column1, 9,'0') FROM Testing_PADDING;
Here is the error I receive:
"Invalid operation: failed to find conversion function from "unknown" to text;"
Redshift can't determine data type from the context, so you need to explicitly set it
WITH Testing_PADDING AS (SELECT '12345678'::text AS column1)
SELECT
LPAD(column1, 9, '0')
FROM Testing_PADDING;
I suspect that one of your strings isn't being seen as text - likely the column1 text. (Sorry don't have a cluster up not to test)
Try:
WITH Testing_PADDING AS (SELECT '12345678'::text AS column1)
SELECT LPAD(column1, 9,'0'::text) FROM Testing_PADDING;
I am failing to achieve simple SQL query in Spark.
I would like to write the below query in Scala Spark:
select * from emp where emp_id in (select distinct manager_id from emp ;
Below is what I tried:
empdf.where(col("emp_id").isin(empdf.select("manager_id").collect().map(_(0)).toList)).show()
I got the below error:
java.lang.RuntimeException: Unsupported literal type class scala.collection.immutable.$colon$colon List(null, 68319, 68319, 68319, 65646, 65646, 69062, 66928, 66928, 66928, 66928, 67858, 66928, 67832)
It's better to do a semi join to avoid collecting as list:
empdf.alias("t1").join(empdf.alias("t2"), expr("t1.emp_id = t2.manager_id"), "left_semi")
If you want to use isin, you can expand the List using : _* (see this post):
empdf.where(col("emp_id").isin(empdf.select("manager_id").collect().map(_(0)).toList: _*)).show()
Or use isInCollection:
empdf.where(col("emp_id").isInCollection(empdf.select("manager_id").collect().map(_(0)).toList)).show()
You can also write as a SQL,
empdf.createorreplaceview("emp")
spark.sql("select * from emp where emp_id in (select distinct manager_id from emp ")
I want to do a count(*) of the number of a rows fom a DB2 database.
The basic query is the following:
select
SUBSTR("Request_Detail",LOCATE('/',"Request_Detail")+1,LOCATE('/',"Request_Detail",LOCATE('/',"Request_Detail")+1)-LOCATE('/',"Request_Detail"))
from "Request_Analisys"
WHERE
"Sample_Date_and_Time">=1200323230000000 and "Sample_Date_and_Time"<1200332300000000
and "Request_Detail" <> '[Summary]'
and "Request_Detail" not like 'WS:%'
Now I'd like to do a count(*) of the resulting rows, but if I do a query like this:
select
count(*),
SUBSTR("Request_Detail",LOCATE('/',"Request_Detail")+1,LOCATE('/',"Request_Detail",LOCATE('/',"Request_Detail")+1)-LOCATE('/',"Request_Detail"))
from "Request_Analisys"
WHERE
"Sample_Date_and_Time">=1200323230000000 and "Sample_Date_and_Time"<1200332300000000
and "Request_Detail" <> '[Summary]'
and "Request_Detail" not like 'WS:%'
It gives the error:
18:51:58 FAILED [SELECT - 0 rows, 0.032 secs] 1) [Code: -119, SQL State: 42803] An expression starting with "Request_Detail" specified in a SELECT clause, HAVING clause, or ORDER BY clause is not specified in the GROUP BY clause or it is in a SELECT clause, HAVING clause, or ORDER BY clause with a column function and no GROUP BY clause is specified.. SQLCODE=-119, SQLSTATE=42803, DRIVER=4.22.29
2) [Code: -727, SQL State: 56098] An error occurred during implicit system action type "2". Information returned for the error includes SQLCODE "-119", SQLSTATE "42803" and message tokens "Request_Detail".. SQLCODE=-727, SQLSTATE=56098, DRIVER=4.22.29
How could I do to get the count of the rows?
Which Request_Detail line's substr would you think it shows after the count?
If you count the lines, the result set will be a single line, and using any columns in it makes no sense.
If you want multiple lines, with a count for each found substr, you need to GROUP BY this substr.
This may work...
select
count(
SUBSTR("Request_Detail"
,LOCATE('/',"Request_Detail")+1
,LOCATE('/',"Request_Detail",LOCATE('/',"Request_Detail")+1)
-LOCATE('/',"Request_Detail")))
)
from "Request_Analisys"
WHERE
"Sample_Date_and_Time">=1200323230000000 and "Sample_Date_and_Time"<1200332300000000
and "Request_Detail" <> '[Summary]'
and "Request_Detail" not like 'WS:%'
But if not this should..
with cte as (
select
SUBSTR("Request_Detail"
,LOCATE('/',"Request_Detail")+1
,LOCATE('/',"Request_Detail",LOCATE('/',"Request_Detail")+1)
-LOCATE('/',"Request_Detail"))) as mydetail
from "Request_Analisys"
WHERE
"Sample_Date_and_Time">=1200323230000000 and "Sample_Date_and_Time"<1200332300000000
and "Request_Detail" <> '[Summary]'
and "Request_Detail" not like 'WS:%'
)
select count(*) from cte
I suggest you use REGEXP_EXTRACT to pick what you want out of your "Request_Detail" column. This is more flexable than using SUBSTR and LOCATE, and will avoid the statement was not executed because a numeric argument of a scalar function is out of range.. error
e.g
select
REGEXP_EXTRACT("Request_Detail",'.*/(.+/)',1,1,'',1)
, SUBSTR("Request_Detail",LOCATE('/',"Request_Detail")+1,LOCATE('/',"Request_Detail",LOCATE('/',"Request_Detail")+1)-LOCATE('/',"Request_Detail"))
FROM TABLE(VALUES('aaaa/bbbb/ccc')) AS T("Request_Detail")
returns
1 |2
------|-----
bbbb/ |bbbb/
so, you could then do this
SELECT
COUNT(*)
, REGEXP_EXTRACT("Request_Detail",'.*/(.+/)',1,1,'',1)
FROM
"Request_Analisys"
GROUP BY
REGEXP_EXTRACT("Request_Detail",'.*/(.+/)',1,1,'',1)
for example
Im trying to make a query that will check if there is any row which has a salary that is 10000 higher than the salary for that department but when I try to run it I get this Error:
FAILED: ParseException line 3:39 mismatched input 'SELECT' expecting ) near '''' in expression specification
this is the query Im using
set AVERAGES ='SELECT ROLE, AVG(AnnualSalary) From Salaries GROUP BY ROLE';
SELECT ROLE, AVG(AnnualSalary) FROM Salaries
GROUP BY ROLE, AnnualSalary HAVING AnnualSalary > ('${hiveconf:AVERAGES}' + 10000);
Currently Hive does not support storing the query result into variable.
You can use window function to achieve this.
select * from
( select *,
avg(AnnualSalary) over(partition by ROLE) role_avg
from
Salaries
) a
where
AnnualSalary > role_avg+10000
Is there any way to skip rows that are null or empty? I could use some help with sorting the output of a subtable. My having statement is returning an error I can’t fix.
It returns the error: ERROR: invalid input syntax for integer: " "
This seems to be because some rows in my table will have either null values or be empty and the having statement is getting hung up there.
Here is the full query
SELECT
count(job),
year,
zipcode
FROM
(
SELECT
substring (cast(dobjobs.prefilingdate AS varchar), '^\d\d\d\d') AS year,
dobjobs.job,
dobjobs.bbl,
pluto_17v1.zipcode
FROM
dobjobs
JOIN pluto_17v1 ON dobjobs.bbl = pluto_17v1.bbl
GROUP BY
dobjobs.prefilingdate,
dobjobs.bbl,
pluto_17v1.zipcode,
dobjobs.job
ORDER BY
year
) AS sub
GROUP BY
year, zipcode
HAVING
CAST( zipcode AS int ) IN (10039, 10039, 10026, 10030, 10037, 10027, 10032, 10033, 10040, 10034, 10031)
ORDER BY
year;