How to implement raw sql query in Tastypie - tastypie

I am trying to do this simple query, but it does not work. Thanks.
SELECT * FROM TSimple where (start_date < '2012-04-20' and end_date is null) or
(end_date > '2012-04-20' and start_date < '2012-04-20')
class TSimple (models.Model):
start_date = models.DateTimeField()
end_date = models.DateTimeField(blank=True, null=True)
...
class TSimpleResource(ModelResource):
def dehydrate(self, bundle):
request_method = bundle.request.META['REQUEST_METHOD']
if request_method=='GET':
new_date = bundle.request.GET.get('new_date', '')
qs = TSimple.objects.raw(
'SELECT * FROM TSimple where (start_date<=\'' +
new_date + '\' and end_date>=\'' +
new_date + '\') or (start_date<=\'' + new_date +
'\' and end_date is null)')
ret_list = [row for row in qs]
// NOT WORK. Not able to get correct json data in javascript.
// It needs return bundle. HOW to replace bundle?
// Is this correct way to do it?
return ret_list
else:
// This is ok.
return bundle
I have following questions:
1) (raw sql method) If implementing in dehydrate method is correct way to do it? If it is, above does not work. It should return bundle object. How to construct new bundle?
If above method is ok, I noticed that bundle already constructed .data field with default query(?), which will be thrown away with new query. That raise the questions if this is right way to do it.
2) If there are other raw sql method to do it? Where to execute the sql?
3) How to do it in filter?
4) I know sql and not familiar with complex filter. That's why I am trying to use raw sql method to do quick prototype. What are the draw back? I noticed that using Tastypie has many unnecessary queries which I don't know how to get rid of it. Example, query on table with foreign key trigger query to another table's data, which I don't want to get.

I figure out the filter and it seems worked. But I am still interested in raw sql.
def apply_filters(self, request, applicable_filters):
base_filter = super(TSimpleResource, self).apply_filters(request,
applicable_filters)
new_date = request.GET.get('new_date', None)
if new_date:
qset = (
(
Q(start_date__lte=new_date) &
Q(end_date__gte=new_date)
) |
(
Q(start_date__lte=new_date) &
Q(end_date__isnull=True)
)
)
base_filter = base_filter.filter(qset)
return base_filter

Related

Springboot JPA Cast numeric substring to string

I have a JPA/Springboot application backed by a Postgres database. I need to get a records that is equal to a substring passed back to the server.
For example:
Select * from dp1_attachments where TRIM(RIGHT(dp1_submit_date_dp1_number::text, 5)) ='00007'
This query works in PgAdmin, but not in the JPA #Query statement.
#Query("SELECT a.attachmentsFolder as attachmentsFolder, a.attachmentNumber as attachmentNumber, a.attachmentName as attachmentName, a.dp1SubmitDateDp1Number as dp1SubmitDateDp1Number,a.attachmentType as attachmentType, a.attachmentDate as attachmentDate, a.attachmentBy as attachmentBy "
+ "FROM DP1Attachments a WHERE TRIM(SUBSTRING(a.dp1SubmitDateDp1Number::text, 5 )) = :dp1Number")
I've also tried CASTing the parameter like this:
#Query("SELECT a.attachmentsFolder as attachmentsFolder, a.attachmentNumber as attachmentNumber, a.attachmentName as attachmentName, a.dp1SubmitDateDp1Number as dp1SubmitDateDp1Number,a.attachmentType as attachmentType, a.attachmentDate as attachmentDate, a.attachmentBy as attachmentBy "
+ "FROM DP1Attachments a WHERE TRIM(SUBSTRING(CAST(a.dp1SubmitDateDp1Number as string, 5 ))) = :dp1Number")
but the application won't even run, and returns an error that the query isn't valid.
If I make no attempt to cast it, I get an error that function pg_catalog.substring(numeric, integer) does not exist
UPDATE
I've also tried creating a native query instead but that also doesn't seem to work.
List<DP1AttachmentsProjection> results = em.createNativeQuery("Select * FROM dp1_attachments WHERE TRIM(RIGHT(CAST(dp1_submit_date_dp1_number as varchar),5)) =" + dp1Number).getResultList();
In place of varchar I have also tried string and text.
Errors come back similar to ERROR: operator does not exist: text = integer. Its like the CAST is being ignored and I'm not sure why.
I also tried the following as a native query:
em.createNativeQuery("Select * FROM dp1_attachments WHERE TRIM(RIGHT(dp1_submit_date_dp1_number::varchar),5)) =" + dp1Number).getResultList();
and get ERROR: syntax error at or near ":"
FINAL SOLUTION
Thanks to #Nenad J I altered the query to get the final working solution:
#Query(value = "SELECT a.attachments_Folder as attachmentsFolder, a.attachment_Number as attachmentNumber, a.attachment_Name as attachmentName, a.dp1_Submit_Date_Dp1_Number as dp1SubmitDateDp1Number,a.attachment_Type as attachmentType, a.attachment_Date as attachmentDate, a.attachment_By as attachmentBy FROM DP1_Attachments a WHERE TRIM(RIGHT(CAST(a.dp1_Submit_Date_Dp1_Number as varchar ), 5 )) = :dp1Number", nativeQuery = true)"
Default substring returns a string, so substring(integer data,5) returns a string. Thus no need for the cast.
#Query("SELECT * FROM DP1Attachments a WHERE TRIM(SUBSTRING(a.dp1SubmitDateDp1Number, 5)) = :dp1Number")
But I recommend in this case use native query like this:
Put this code in your attachment repository.
#Query(value="SELECT * FROM DP1Attachments AS a WHERE TRIM(SUBSTRING(a.dp1SubmitDateDp1Number, 5 )) = :dp1Number", nativeQuery=true)
Be careful with the column's name.

SELECT with substring in ON clause?

I have the following select statement in ABAP:
SELECT munic~mandt VREFER BIS AB ZZELECDATE ZZCERTDATE CONSYEAR ZDIMO ZZONE_M ZZONE_T USAGE_M USAGE_T M2MC M2MT M2RET EXEMPTMCMT EXEMPRET CHARGEMCMT
INTO corresponding fields of table GT_INSTMUNIC_F
FROM ZCI00_INSTMUNIC AS MUNIC
INNER JOIN EVER AS EV on
MUNIC~POD = EV~VREFER(9).
"where EV~BSTATUS = '14' or EV~BSTATUS = '32'.
My problem with the above statement is that does not recognize the substring/offset operation on the 'ON' clause. If i remove the '(9) then
it recognizes the field, otherwise it gives error:
Field ev~refer is unknown. It is neither in one of the specified tables
nor defined by a "DATA" statement. I have also tried doing something similar in the 'Where' clause, receiving a similar error:
LOOP AT gt_instmunic.
clear wa_gt_instmunic_f.
wa_gt_instmunic_f-mandt = gt_instmunic-mandt.
wa_gt_instmunic_f-bis = gt_instmunic-bis.
wa_gt_instmunic_f-ab = gt_instmunic-ab.
wa_gt_instmunic_f-zzelecdate = gt_instmunic-zzelecdate.
wa_gt_instmunic_f-ZZCERTDATE = gt_instmunic-ZZCERTDATE.
wa_gt_instmunic_f-CONSYEAR = gt_instmunic-CONSYEAR.
wa_gt_instmunic_f-ZDIMO = gt_instmunic-ZDIMO.
wa_gt_instmunic_f-ZZONE_M = gt_instmunic-ZZONE_M.
wa_gt_instmunic_f-ZZONE_T = gt_instmunic-ZZONE_T.
wa_gt_instmunic_f-USAGE_M = gt_instmunic-USAGE_M.
wa_gt_instmunic_f-USAGE_T = gt_instmunic-USAGE_T.
temp_pod = gt_instmunic-pod.
SELECT vrefer
FROM ever
INTO wa_gt_instmunic_f-vrefer
WHERE ( vrefer(9) LIKE temp_pod ). " PROBLEM WITH SUBSTRING
"AND ( BSTATUS = '14' OR BSTATUS = '32' ).
ENDSELECT.
WRITE: / sy-dbcnt.
WRITE: / 'wa is: ', wa_gt_instmunic_f.
WRITE: / 'wa-ever is: ', wa_gt_instmunic_f-vrefer.
APPEND wa_gt_instmunic_f TO gt_instmunic_f.
WRITE: / wa_gt_instmunic_f-vrefer.
ENDLOOP.
itab_size = lines( gt_instmunic_f ).
WRITE: / 'Internal table populated with', itab_size, ' lines'.
The basic task i want to implement is to modify a specific field on one table,
pulling values from another. They have a common field ( pod = vrefer(9) ). Thanks in advance for your time.
If you are on a late enough NetWeaver version, it works on 7.51, you can use the OpenSQL function LEFT or SUBSTRING. Your query would look something like:
SELECT munic~mandt VREFER BIS AB ZZELECDATE ZZCERTDATE CONSYEAR ZDIMO ZZONE_M ZZONE_T USAGE_M USAGE_T M2MC M2MT M2RET EXEMPTMCMT EXEMPRET CHARGEMCMT
FROM ZCI00_INSTMUNIC AS MUNIC
INNER JOIN ever AS ev
ON MUNIC~POD EQ LEFT( EV~VREFER, 9 )
INTO corresponding fields of table GT_INSTMUNIC_F.
Note that the INTO clause needs to move to the end of the command as well.
field(9) is a subset operation that is processed by the ABAP environment and can not be translated into a database-level SQL statement (at least not at the moment, but I'd be surprised if it ever will be). Your best bet is either to select the datasets separately and merge them manually (if both are approximately equally large) or pre-select one and use a FAE/IN clause.
They have a common field ( pod = vrefer(9) )
This is a wrong assumption, because they both are not fields, but a field an other thing.
If you really need to do that task through SQL, I'll suggest you to check native SQL sentences like SUBSTRING and check if you can manage to use them within an EXEC_SQL or (better) the CL_SQL* classes.

Hibernate: StoredProcedure with recursive depthsearch: Mapping/Output Problems

I searching for help. I have to map my Postgres 9.4 Database (DB) with Hibernate 5.2, of course it's an study task. The biggest Problem is, that I'm no brain in Hibernate, Java and coding itself XD
It's an SozialNetwork DB. To map the DB with Hibernate doing fine.
Now I should map a stored produce. This Produce should find the shortest friendship path between two persons. In Postgres the produce working fine.
That are the relevant DB-Tables:
For Person:
CREATE TABLE Person (
PID bigint NOT NULL,
firstName varchar(50) DEFAULT NULL,
lastName varchar(50) DEFAULT NULL,
(some more...)
PRIMARY KEY (PID)
);
And for the Relationship between to Persons:
CREATE TABLE Person_knows_Person (
ApID bigint NOT NULL,
BpID bigint REFERENCES Person (PID) (..)
knowsCreationDate timestamp,
PRIMARY KEY (ApID,BpID));
And that is the Stored Produce in short:
CREATE OR REPLACE FUNCTION ShortFriendshipPath(pid bigint, pid2 bigint)
RETURNS TABLE (a_pid bigint, b_pid bigint, depth integer, path2 bigint[], cycle2 boolean)
AS $$
BEGIN
RETURN QUERY
SELECT * FROM (
WITH RECURSIVE FriendshipPath(apid, bpid, depth, path, cycle) AS(
SELECT pkp.apid, pkp.bpid,1,
ARRAY[pkp.apid], false
FROM person_knows_person pkp
WHERE apid=$1 --OR bpid=$1
UNION ALL
SELECT pkp.apid, pkp.bpid, fp.depth+1, path || pkp.apid,
pkp.apid = ANY(path)
FROM person_knows_person pkp, FriendshipPath fp
WHERE pkp.apid = fp.bpid AND NOT cycle)
SELECT *
FROM FriendshipPath WHERE bpid=$2) AS OKOK
UNION
SELECT * FROM (
WITH RECURSIVE FriendshipPath(apid, bpid, depth, path, cycle) AS(
SELECT pkp.apid, pkp.bpid,1,
ARRAY[pkp.apid], false
FROM person_knows_person pkp
WHERE apid=$2 --OR bpid=$1
UNION ALL
SELECT pkp.apid, pkp.bpid, fp.depth+1, path || pkp.apid,
pkp.apid = ANY(path)
FROM person_knows_person pkp, FriendshipPath fp
WHERE pkp.apid = fp.bpid AND NOT cycle)
SELECT *
FROM FriendshipPath WHERE bpid=$1) AS YOLO
ORDER BY depth ASC LIMIT 1;
END;
$$ LANGUAGE 'plpgsql' ;
(Sorry for so much code, but it's for both directions, and before I post some copy+reduce misttakes^^)
The Call in Postgre for example:
SELECT * FROM ShortFriendshipPath(10995116277764, 94);
gives me this Output:
enter image description here
I use the internet for help and find 3 solutions for calling:
direct SQL call
call with NamedQuery and
map via XML
(fav found here)
I faild with all of them XD
I favorite the 1. solution with this call in session:
Session session = HibernateUtility.getSessionfactory().openSession();
Transaction tx = null;
try {
tx = session.beginTransaction();
System.out.println("Please insert a second PID:");
Scanner scanner = new Scanner(System.in);
long pid2 = Long.parseLong(scanner.nextLine());
// **Insert of second ID*/
Query query2 = session.createQuery("FROM " + Person.class.getName() + " WHERE pid = :pid ");
query2.setParameter("pid", pid2);
List<Person> listB = ((org.hibernate.Query) query2).list();
int cnt1 = 0;
while (cnt1 < listB.size()) {
Person pers1 = listB.get(cnt1++);
pid2 = pers1.getPid();
}
// Query call directly:
Query querySP = session.createSQLQuery("SELECT a_pid,path2 FROM ShortFriendshipPath(" + pid + "," + pid2 + ")");
List <Object[]> list = ((org.hibernate.Query) querySP).list();
for (int i=0; i<list.size();i++){
Personknowsperson friendship = (Personknowsperson)result.get(i);
}
} catch (Exception e) { (bla..)}
} finally { (bla....) }
Than I get following Error:
javax.persistence.PersistenceException:
org.hibernate.MappingException: No Dialect mapping for JDBC type: 2003
(..blabla...)
I understand why. Because my output is not of type Personknowsperson. I found an answer: that I have to say Hibernate what is the correct formate. And should use 'UserType'. So I try to find some explanations for how I create my UserType. But I found nothing, that I understand. Second Problem: I'm not sure what I should use for the bigint[] (path2). You see I'm expert -.-
Than I got the idea to try the 3.solution. But the first problem I had was where should I write the xml stuff. Because my Output is no table. So I try in the .cfg.xml but than Hibernate say that
Caused by: java.lang.IllegalArgumentException: org.hibernate.internal.util.config.ConfigurationException: Unable to perform unmarshalling at line number -1 and column -1 in RESOURCE hibernate.cfg.xml. Message: cvc-complex-type.2.4.a: Ungültiger Content wurde beginnend mit Element 'sql-query' gefunden. '{some links}' wird erwartet.
translation:
invalid content found starts with 'sql-query'
Now I'm a nervous wreck. And ask you.
Could someone explain what I have to do and what I did wrong (for dummies please). If more code need (java classes or something else) please tell me. Critic for coding also welcome, cause I want improve =)
Ok, I'm not an expert in postgressql, not hibernate, nor java. (I'm working with C#, SQL Server, NHibernate so ...) I still try to give you some hints.
You probably can set the types of the columns using addXyz methods:
Query querySP = session
.createSQLQuery("SELECT * FROM ShortFriendshipPath(...)")
.addScalar("a_pid", LongType.INSTANCE)
...
// add user type?
You need to create a user type for the array. I don't know how and if you can add it to the query. See this answer here.
You can also add the whole entity:
Query querySP = session
.createSQLQuery("SELECT * FROM ShortFriendshipPath(...)")
.addEntity(Personknowsperson.class)
...;
I hope it takes the mapping definition of the corresponding mapping file, where you can specify the user type.
Usually it's much easier to get a flat list of values, I mean a separate row for each different value in the array. Like this:
Instead of
1 | 2 | (3, 4, 5) | false
You would get:
1 | 2 | 3 | false
1 | 2 | 4 | false
1 | 2 | 5 | false
Which seems denormalized, but is actually the way how you build relational data.
In general: use parameters when passing stuff like ids to queries.
Query querySP = session
.createSQLQuery("SELECT * FROM ShortFriendshipPath(:pid1, :pid2)")
.setParameter("pid1", pid1)
.setParameter("pid2", pid2)
...

Metadata about a column in SQL Server 2008 R2?

I'm trying to figure out a way to store metadata about a column without repeating myself.
I'm currently working on a generic dimension loading SSIS package that will handle all my dimensions. It currently does :
Create a temporary table identical to the given table name in parameters (this is a generic stored procedure that receive the table name as parameter, and then do : select top 0 * into ##[INSERT ORIGINAL TABLE NAME HERE] from [INSERT ORIGINAL TABLE NAME HERE]).
==> Here we insert custom code for this particular dimension that will first query the data from a datasource and get my delta, then transform the data and finally loads it into my temporary table.
Merge the temporary table into my original table with a T-SQL MERGE, taking care of type1 and type2 fields accordingly.
My problem right now is that I have to maintain a table with all the fields in it to store a metadata to tell my scripts if this particular field is type1 or type2... this is nonsense, I can get the same data (minus type1/type2) from sys.columns/sys.types.
I was ultimately thinking about renaming my fields to include their type in it, such as :
FirstName_T2, LastName_T2, Sex_T1 (well, I know this can be type2, let's not fall into that debate here).
What do you guyz would do with that? My solution (using a table with that metadata) is currently in place and working, but it's obvious that repeating myself from the systables to a custom table is nonsense, just for a simple type1/type2 info.
UPDATE: I also thought about creating user defined types like varchar => t1_varchar, t2_varchar, etc. This sounds like something a bit sluggy too...
Everything you need should already be in INFORMATION_SCHEMA.COLUMNS
I can't follow your thinking of not using provided tables/views...
Edit: As scarpacci mentioned, this somewhat portable if needed.
I know this is bad, but I will post an answer to my own question... Thanks to GBN for the help tho!
I am now storing "flags" in the "description" field of my columns. I, for example, can store a flag this way : "TYPE_2_DATA".
Then, I use this query to get the flag back for each and every column :
select columns.name as [column_name]
,types.name as [type_name]
,extended_properties.value as [column_flags]
from sys.columns
inner join sys.types
on columns.system_type_id = types.system_type_id
left join sys.extended_properties
on extended_properties.major_id = columns.object_id
and extended_properties.minor_id = columns.column_id
and extended_properties.name = 'MS_Description'
where object_id = ( select id from sys.sysobjects where name = 'DimDivision' )
and is_identity = 0
order by column_id
Now I can store metadata about columns without having to create a separate table. I use what's already in place and I don't repeat myself. I'm not sure this is the best possible solution yet, but it works and is far better than duplicating information.
In the future, I will be able to use this field to store more metadata, where as : "TYPE_2_DATA|ANOTHER_FLAG|ETC|OH BOY!".
UPDATE :
I now store the information in separate extended properties. You can manage extended properties using sp_addextendedproperty and sp_updateextendedproperty stored procedures. I have created a simple store procedure that help me to update those values regardless if they currently exist or not :
create procedure [dbo].[UpdateSCDType]
#tablename nvarchar(50),
#fieldname nvarchar(50),
#scdtype char(1),
#dbschema nvarchar(25) = 'dbo'
as
begin
declare #already_exists int;
if ( #scdtype = '1' or #scdtype = '2' )
begin
select #already_exists = count(1)
from sys.columns
inner join sys.extended_properties
on extended_properties.major_id = columns.object_id
and extended_properties.minor_id = columns.column_id
and extended_properties.name = 'ScdType'
where object_id = (select sysobjects.id from sys.sysobjects where sysobjects.name = #tablename)
and columns.name = #fieldname
if ( #already_exists = 0 )
begin
exec sys.sp_addextendedproperty
#name = N'Scd_Type',
#value = #scdtype,
#level0type = N'SCHEMA',
#level0name = #dbschema,
#level1type = N'TABLE',
#level1name = #tablename,
#level2type = N'COLUMN',
#level2name = #fieldname
end
else
begin
exec sys.sp_updateextendedproperty
#name = N'Scd_Type',
#value = #scdtype,
#level0type = N'SCHEMA',
#level0name = #dbschema,
#level1type = N'TABLE',
#level1name = #tablename,
#level2type = N'COLUMN',
#level2name = #fieldname
end
end
end
Thanks again

ADO.NET: How to have N parameters?

i need to mark a batch of rows in the database as "processed".
In the olden days, when things were easier, i would create some SQL that said:
UPDATE Readings SET IsProcessed = 1 WHERE ReadingDateTime IN (
"2010-10-07 22:02:13.327",
"2010-10-07 22:02:14.213",
"2010-10-07 22:02:15.595",
...
"2010-10-07 23:03:36.981")
by looping through a list of dates:
sql = "UPDATE Readings SET IsProcessed = 1 WHERE ReadingDateTime IN (";
foreach (DateTime datetime in dates)
{
sql = sql + CRLF+ DateTimeToSql(datetime)+",";
}
sql = sql+")";
And issue the SQL:
ExecuteNonQuery(connection, sql);
And all was good.
Now i want to try to do things the hard way; i want to try to use parameters:
sql = ???;
command.CommandText = sql;
DbCommand command = connection.CreateCommand();
foreach (DateTime datetime in readings)
{
command.Parameters.Add(new SqlParameter(???, SqlDbType.DateTime) {Value = datetime});
}
using (DbTransaction transaction = connection.BeginTransaction())
{
command.Transaction = transaction;
command.ExecuteNonQuery();
transaction.Commit();
}
The only trick is what to put in the sql string, and what to put in each iteration of the parameters loop. They need some name.
What's the recommended way to name arbitrary number of parameters?
I think what you could do is something like this:-
sql = "UPDATE Readings SET IsProcessed = 1 WHERE ReadingDateTime IN (";
for (int count=0; count<dates.Length; count++)
{
sql = sql + CRLF + ":param" + count;
}
sql = sql+")";
for (int count=0; count<dates.Length; count++)
{
command.Parameters.Add(new SqlParameter(":param" + count, SqlDbType.DateTime) {Value = datetime});
}
However, i think that having parameters specifically in this scenario is kind of unnecessary.
Considering that your dynamic values are dates and not strings, you can instead validate the dates directly using a TryParse to ensure that they are the correct datatype before you append them in your original solution!!
I dont know if you are achieving anything extra over that by using parameters in this case.
You can go as minimal as you like: the provider will accept parameters named #0, #1, etc. But if you want to be more descriptive, just use a base name of #ReadingDateTime, with an integer suffix from 0..n. Easy peasy.