i have a query where in i am passing date range inputs
#Query(value = "SELECT ord.purchaseOrderNumber,ord.salesOrderNumber,ord.quoteNumber"
+ " FROM Order ord WHERE ord.purchaseOrderNumber :poNumber and ord.receiveDate between to_date(:dateFrom,'MM/DD/YYYY') AND to_date(:dateTo,'MM/DD/YYYY')
so i am passing valid number to the :poNumber and passing null values to the date placeholders and the query is not returning any values where it should return 1 row based on the :poNumber , so how do i handle this NULL input values to the dates
I suppose you need the OR operator between the ord.purchaseOrderNumber and ord.receiveDate
#Query(value = "SELECT ord.purchaseOrderNumber,ord.salesOrderNumber,ord.quoteNumber"
+ " FROM Order ord WHERE ord.purchaseOrderNumber :poNumber OR (ord.receiveDate between to_date(:dateFrom,'MM/DD/YYYY') AND to_date(:dateTo,'MM/DD/YYYY'))
Related
Source data
I am working on an ELT project to load data from CSV files into PostgreSQL where I will transform it. The CSV files have many columns that are consistent across files, but also contain activity columns that are inconsistent with names like Date (05/19/2020), Type (05/19/2020), etc.
In the loading script I am merging all of the columns with dates in the column name into one jsonb column so I don't have to constantly add new columns to the raw data table.
The resulting jsonb column in the raw data table looks like this:
id
activity
12345678
{"Date (05/19/2020)": null, "Type (05/19/2020)": null, "Date (06/03/2020)": "06/01/2020", "Type (06/03/2020)": "E"}
98765432
{"Date (05/19/2020)": "05/18/2020", "Type (05/19/2020)": "B", "Date (10/23/2020)": "10/26/2020", "Type (10/23/2020)": "T"}
JSON to columns
Using the amazing create_jsonb_flat_view function from this post I can convert the jsonb to columns like this:
id
Date (05/19/2020)
Type (05/19/2020)
Date (06/03/2020)
Type (06/03/2020)
Type (10/23/2020
Date (10/23/2020)
Type (10/23/2020)
10629465
null
null
06/01/2020
E
98765432
05/18/2020
B
10/26/2020
T
Need to move part of column name to row
Now, this is where I'm stuck. I need to remove the portion of the column name that is the Activity Date (e.g. (05/19/2020)) and create a row for each id and ActivityDate with additional columns for Date and Type like this:
id
ActivityDate
Date
Type
12345678
05/19/2020
null
null
12345678
06/03/2020
06/01/2020
E
98765432
05/19/2020
05/18/2020
B
98765432
10/23/2020
10/26/2020
T
I followed your link to the create_jsonb_flat_view article yesterday and then forgot this question. While I thank you for pointing me there, I think that mentioning it worked against you.
A more conventional approach using regexp_replace() works here. I left the date values as strings, but you can convert them with to_date() if needed:
with parse as (
select id, e.k, e.v,
regexp_replace(e.k, '\s+\([0-9/]{10}\)', '') as k_no_date,
regexp_replace(e.k, '^.+([0-9/]{10}).+', '\1') as k_date_only
from rawinput
cross join lateral jsonb_each_text(activity) as e(k, v)
)
select id,
k_date_only as activity_date,
min(v) filter (where k_no_date = 'Date') as date,
min(v) filter (where k_no_date = 'Type') as type
from parse
group by id, k_date_only;
db<>fiddle here
#Mike-Organek's Answer works beautifully!
However, I was curious if the regexp_replace() calls might be slowing the query down a bit and it seemed I could get the same results using a simpler function.
Since Mike gave me a great example to start with I modified it to split on the space between Date and (05/19/2020).
For 20,000 rows, it went from taking an avg of 7 sec on my local machine to an avg of .9 sec.
Here is the resulting query:
with parse as (
select id, e.k, e.v,
split_part(e.k, ' ', 1) as k_no_date,
trim(split_part(e.k, ' ', 2),'()') as k_date_only
from rawinput
cross join lateral jsonb_each_text(activity) as e(k, v)
)
select id,
k_date_only as activity_date,
min(v) filter (where k_no_date = 'Date') as date,
min(v) filter (where k_no_date = 'Type') as type
from parse
group by id, k_date_only;
I have a list of ints:
var ids = new List { 10, 20 };
And I need to find Users with that ids:
context.Users.FromSqlInterpolated($#"
select Users.*
where Users.Id in ({String.Join(',', ids)})"
But I get the following error:
'Conversion failed when converting the nvarchar value '10, 20' to data type int.'
How can I use such a parameter?
Using Interpolated method is not appropriate here, because {String.Join(',', ids)} defines single string placeholder, hence EF Core binds single nvarchar parameter with value '10,20', so the actual SQL is like this
select Users.*
where Users.Id in ('10,20')
which is invalid, hence the exception.
You should use Raw method instead. Either
var query = context.Users.FromSqlRaw($#"
select Users.*
where Users.Id in ({String.Join(',', ids)})");
which will embed literal values
select Users.*
where Users.Id in (10,20)
or if you want to parameterize it, generate parameter placeholders like {0}, {1} etc. inside the SQL and pass values separately:
var placeholders = string.Join(",", Enumerable.Range(0, ids.Count)
.Select(i => "{" + i + "}"));
var values = ids.Cast<object>().ToArray();
var query = context.Users.FromSqlRaw($#"
select Users.*
where Users.Id in ({placeholders})", values);
which would generate SQL like this
select Users.*
where Users.Id in (#p0,#p1)
If you need to combine .FromSql() and SQL WHERE x IN () then perform a second operation on the output IQueryable<T> of the .FromSql() call.
var ids = new List { 10, 20 };
var query = context.Users.FromSqlInterpolated(
$#"select *
from Users");
if (ids?.Count > 0)
{
query = query.Where(user => ids.Contains(user.Id));
}
Resultant SQL approximates to:
select * from (
select *
from Users
) u
where u.Id in (10,20)
No invalid SQL if ids is empty
No empty result if ids is empty string (I was working with a string data type)
This query selects random client_id value from the table. And i need that value in another query.
String s= "SELECT client_id FROM clients OFFSET random()*(select count(*) from clients) LIMIT 1";
d=conn.createStatement();
ResultSet rs1 = d.executeQuery(s);
UPDATE: I tried to cast like this
int val = ((Number) rs1).intValue();
sql error shows like this
rg.postgresql.jdbc3g.Jdbc3gResultSet cannot be cast to java.lang.Number
I want to use the value of result set in my another query
insert into invheader(invdate,client_id,amount,tax,total,closed,ship_via,note)
values(
to_date('"+indate+"', 'DD-mon-YYYY'),
'"+rs1+"', //<<---- i need to pass the value of result set in integer format
'"+amont+"',
'"+tx+"',
'"+tot+"',
'"+closd+"',
'"+shipvia+"',
'"+not+"')";
UPDATE: I am passing value from servlet to postgresdb
Avoid the round trip and do it all in SQL
with client_id as (
select client_id
from clients
offset random() * (select count(*) from clients)
limit 1
)
insert into invheader (
invdate, client_id, amount, tax, total, closed, ship_via, note
) values (
to_date('"+indate+"', 'DD-mon-YYYY'),
(select client_id from client_id), //<<---- i need to pass the value of result set in integer format
'"+amont+"',
'"+tx+"',
'"+tot+"',
'"+closd+"',
'"+shipvia+"',
'"+not+"')";
And be very aware of SQL injection:
https://www.google.com/search?q=sql+injection&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a&channel=sb
I have a Firebird 2.5 database .As an example I have stored procedure with a name QRESULT which expected return is:
Parameter - DATATYPE
a - date
b - numeric(18,0)
c - integer
d - varchar(50)
and so on....
I use PHP - PDO to query the firebird database using the procedure QRESULT like this:
SELECT a,b,d from QRESULT() where a = "some value"
I need to run some query before QRESULT procedure and i need it to return the datatype of all the columns that QRESULT would return if it was ran. So i can help user to type proper value for my "where" clause.I know i can set that manually in the user interface, but in the real project there are lots of procedures and if there is a way i can make my filter interface generate dynamically i would be happy about that.If this is not possible for a stored procedure i can make it with select statements.I just need some lead.
The information you want is in the RDB$PROCEDURE_PARAMETERS table, basically what you need is query
SELECT r.RDB$PARAMETER_NAME ParName, F.RDB$FIELD_TYPE ParType
FROM RDB$PROCEDURE_PARAMETERS r
JOIN RDB$FIELDS F ON(F.RDB$FIELD_NAME = R.RDB$FIELD_SOURCE)
WHERE r.RDB$PROCEDURE_NAME = 'QRESULT'
AND r.RDB$PARAMETER_TYPE = 1
ORDER BY r.RDB$PARAMETER_TYPE, r.RDB$PARAMETER_NUMBER
Note that the SP name should be in upper case as this is how it is stored into system tables (unless you use quoted identifiers). If you want to get both input and output parameters the delete the r.RDB$PARAMETER_TYPE = 1 predicate from the WHERE (type 0 is input parameters and 1 is output).
The type returned by this query is integer id for the type, quick googling found this:
14,"TEXT "
7,"SHORT "
8,"LONG "
9,"QUAD "
10,"FLOAT "
27,"DOUBLE "
35,"TIMESTAMP "
37,"VARYING "
261,"BLOB "
40,"CSTRING "
45,"BLOB_ID "
12,"DATE "
13,"TIME "
16,"INT64 "
but if you want to have more precise type then see this SO post.
My query looks like this:
SELECT mthreport.*
FROM crosstab
('SELECT
to_char(ipstimestamp, ''mon DD HH24h'') As row_name,
varid::text || log.varid || ''_'' || ips.objectname::text As bucket,
COUNT(*)::integer As bucketvalue
FROM loggingdb_ips_boolean As log
INNER JOIN IpsObjects As ips
ON log.Varid=ips.ObjectId
WHERE ((log.varid = 37551)
OR (log.varid = 27087)
OR (log.varid = 50876)
OR (log.varid = 45096)
OR (log.varid = 54708)
OR (log.varid = 47475)
OR (log.varid = 54606)
OR (log.varid = 25528)
OR (log.varid = 54729))
GROUP BY to_char(ipstimestamp, ''yyyy MM DD HH24h''), row_name, objectid, bucket
ORDER BY to_char(ipstimestamp, ''yyyy MM DD HH24h''), row_name, objectid, bucket' )
As mthreport(item_name text, varid_37551 integer,
varid_27087 integer ,
varid_50876 integer ,
varid_45096 integer ,
varid_54708 integer ,
varid_47475 integer ,
varid_54606 integer ,
varid_25528 integer ,
varid_54729 integer ,
varid_29469 integer)
the query can be tested against a test table with this connection string:
"host=bellariastrasse.com port=5432 dbname=IpsLogging user=guest password=guest"
The query is syntactically correct and runs fine. My problem is that it the COUNT(*) values are always filling the leftmost column. however, in many instances the left columns should have a zero, or a NULL, and only the 2nd (or n-th) column should be filled. My brain is melting and I cannot figure out what is wrong!
The solution for your problem is to use the crosstab() variant with two parameters.
The second parameter (another query string) produces the list of output columns, so that NULL values in the data query (the first parameter) are assigned correctly.
Check the manual for the tablefunc extension, and in particular crosstab(text, text):
The main limitation of the single-parameter form of crosstab is that
it treats all values in a group alike, inserting each value into the
first available column. If you want the value columns to correspond to
specific categories of data, and some groups might not have data for
some of the categories, that doesn't work well. The two-parameter form
of crosstab handles this case by providing an explicit list of the
categories corresponding to the output columns.
Emphasis mine. I posted a couple of related answers recently here or here or here.