As we have seen that after you get a data source. We need to configure SQL dialects based on the database we use. After we select a particular dialect, How would that be used to make SQL queries specific to DB. Do frameworks like hibernate and JOOQ construct SQL queries in string based on the selected dialect ? If so which would be the most optimal way to support this in a framework of our own ?
Do frameworks like hibernate and JOOQ construct SQL queries in string based on the selected dialect
Yes. In jOOQ, there's an internal StringBuilder that collects SQL fragments from your expression tree, which are generated for your target SQL dialect specifically. You can see how that works in action on this website: https://www.jooq.org/translate. Try translating for example this input: SELECT * FROM t LIMIT 1 (which could correspond to your jOOQ API usage ctx.selectFrom(T).limit(1). It translates to:
-- Oracle 12c and more
SELECT * FROM t FETCH NEXT 1 ROWS ONLY
-- Oracle 11g and less
SELECT *
FROM (
SELECT x.*, rownum rn
FROM (SELECT * FROM t) x
WHERE rownum <= 1
)
WHERE rn > 0
If so which would be the most optimal way to support this in a framework of our own ?
You need:
An expression tree representation of your SQL query.
Optionally, you can parse a string to build this expression tree, like jOOQ's parser if you want to support actual SQL, or you can have your own language abstraction like Hibernate did with HQL / JPQL
Traverse that expression tree using something like a visitor to collect the SQL strings and bind variables.
But!
Do not build your own when you have off the shelf products like jOOQ or to some lesser extent Hibernate that can do the same. Building such a generic SQL abstraction is really difficult, and unless you want to actually sell such a product (you probably don't given your question), investing this time into building this product is not worth it at all.
The above LIMIT emulation is one of the more simple examples from jOOQ. Here's a lot more to help you decide against rolling your own, and that answer is still just scratching the surface of what jOOQ does behind the scenes.
Related
I have a custom query along these lines. I get the list of orderIds from outside. I have the entire order object list with me, so I can change the query in any way, if needed.
#Query("SELECT p FROM Person p INNER JOIN p.orders o WHERE o.orderId in :orderIds)")
public List<Person> findByOrderIds(#Param("orderIds") List<String> orderIds);
This query works fine, but sometimes it may have anywhere between 50-1000 entries in the orderIds list sent from outside function. So it becomes very slow, taking as much as 5-6 seconds which is not fast enough. My question is, is there a better, faster way to do this? When I googled, and on this site, I see we can use ANY, EXISTS: Postgresql: alternative to WHERE IN respective WHERE NOT IN or create a temporary table: https://dba.stackexchange.com/questions/12607/ways-to-speed-up-in-queries-under-postgresql or join this to VALUES clause: Alternative when IN clause is inputed A LOT of values (postgreSQL). All these answers are tailored towards direct SQL calls, nothing based on JPA. ANY keyword is not supported by spring-data. Not sure about creating temporary tables in custom queries. I think I can do it with native queries, but have not tried it. I am using spring-data + OpenJPA + PostgresSQL.
Can you please suggest a solution or give pointers? I apologize if I missed anything.
thanks,
Alice
You can use WHERE EXISTS instead of IN Clause in a native SQL Query as well as in HQL in JPA which results in a lot of performance benefits. Please see sample below
Sample JPA Query:
SELECT emp FROM Employee emp JOIN emp.projects p where NOT EXISTS (SELECT project from Project project where p = project AND project.status <> 'Active')
I've written a SQL query that basically selects from a number of tables to determine which ones have rows that were created since a particular date. My SQL looks something like this:
SELECT widget_type FROM(
SELECT 'A' as widget_type
FROM widget_a
WHERE creation_timestamp > :cutoff
UNION
SELECT 'B' as widget_type
FROM widget_b
WHERE creation_timestamp > :cutoff
) types
GROUP BY widget_type
HAVING count(*)>0
That works well in SQL but I recently found that, while JPA may use unions to perform "table per class" polymorphic queries, JPQL does not support unions in queries. So that leaves me wondering whether JPA has an alternative I could use to accomplish the same thing.
In reality, I would be querying against a dozen tables, not just two, so I would like to avoid doing separate queries. I would also like to avoid doing a native SQL query for portability reasons.
In the question I linked to above, it was asked whether the entities that map to widget_a and widget_b are part of the same inheritance tree. Yes, they are. However, if I selected from their base class, I don't believe I would have a way of specifying different string constants for the different child entities, would I? If I could select an entity's class name instead of a string I provide, that might serve my purpose too. But I don't know if that's possible either. Thoughts?
I did a little more searching and found a (seemingly obscure) feature of JPA that serves my purpose perfectly. What I found is that JPA 2 has a type keyword that allows you to limit polymorphic queries to a particular subclass, like so:
SELECT widget
FROM BaseWidget widget
WHERE TYPE(widget) in (WidgetB, WidgetC)
I've found that JPA (or at least Hibernate as a JPA implementation) allows you to use type not only in constraints but also in select lists. This is approximately what my query ended up looking like:
SELECT DISTINCT TYPE(widget)
FROM BaseWidget widget
WHERE widget.creationTimestamp > :cutoff
That query returns a list of Class objects. My original query was selecting string literals because that's closest to what I might have done in SQL. Selecting Class is actually preferable in my case. But if I did prefer to select a constant based on an entity's type, that is the exact scenario that Oracle's documentation uses to illustrate case statements:
SELECT p.name
CASE TYPE(p)
WHEN Student THEN 'kid'
WHEN Guardian THEN 'adult'
WHEN Staff THEN 'adult'
ELSE 'unknown'
END
FROM Person p
Some JPA providers do support UNION,
http://wiki.eclipse.org/EclipseLink/UserGuide/JPA/Basic_JPA_Development/Querying/JPQL#UNION
but your query seems very complex, and non object-oriented, so using a native SQL query would probably be best.
I know how to use "DISTINCT" in Doctrine 2, but I really need to use "DISTINCT ON (field)" and I don't know how to do this with the QueryBuilder.
My SQL query looks like:
SELECT DISTINCT ON (currency) currency, amount FROM payments ORDER BY currency
And this query works perfect, but I can't use it with the QueryBuilder. Maybe I could write this query on some other way?
I would suggest that the SELECT DISTINCT ON (..) construct that PostgreSQL supports is outside the Object Relational Model (ORM) that is central to Doctrine. Or, perhaps put another way, because SELECT DISTINCT ON (..) is rare in SQL implementations Doctrine haven't coded for it.
Regardless of the actual logic for it not working, I would suggest you try Doctrine's "Native SQL". You need to map the results of your query to the ORM.
With NativeQuery you can execute native SELECT SQL statements and map
the results to Doctrine entities or any other result format supported
by Doctrine.
In order to make this mapping possible, you need to describe to
Doctrine what columns in the result map to which entity property. This
description is represented by a ResultSetMapping object.
With this feature you can map arbitrary SQL code to objects, such as
highly vendor-optimized SQL or stored-procedures.
SELECT DISTINCT ON (..) falls into vendor-optimized SQL I think, so using NativeQuery should allow you to access it.
Doctrine QueryBuilder has some limitations. Even if I didn't check if it's was possible with query builder, I do not hesitate to use DQL when I do not know how to write the query with query builder.
Check theses examples at
http://doctrine-orm.readthedocs.org/en/latest/reference/dql-doctrine-query-language.html#dql-select-examples
Hope this help.
INDEX BY can be used in DQL, allowing first result rows indexed by the defined string/int field to be overwritten by following ones with the same index:
SELECT
p.currency,
p.amount
FROM Namespace\To\Payments p INDEX BY p.currency
ORDER BY p.currency ASC
DQL - EBNF - INDEX BY
I have a generic code that is used to retrieve DDL information from a Firebird database (FB2.1). It generates SQL code like
SELECT * FROM MyTable where 'c' <> 'c'
I cannot change this code. Actually, if that matters, it is inside Report Builder 10.
The fact is that some tables from my database are becoming a litle too populated (>1M records) and that query is starting to take too long to execute.
If I try to execute
SELECT * FROM MyTable where SomeIndexedField = SomeImpossibleValue
it will obviously use that index and run very quickly.
Well, it wouldn´t be that hard to the database find out that that is an impossible matcher and make some sort of optimization and avoid testing it against each row.
Is there any way to make my firebird database to optimize that search?
As the filter condition is a negative proposition (and also doesn't refer a column to search, but only a value to compare to another value), Firebird need to do a full table scan (without use any index) to confirm that aren't any record that meet your criteria.
If you can't change you need to wait for the upcoming 3.0 version, that will implement the Boolean data type, and therefore should start to evaluate "constant" fake comparisons in advance (maybe the client library will do this evaluation before send the statement to the server?).
I'm using Zend Framework for my website and I'd like to retrieve some data from my PostgreSQL database.
I have a request like :
SELECT DISTINCT ON(e.id) e.*, f.*, g.* FROM e, f, g
WHERE e.id = f.id_e AND f.id = g.id_f
This request works well but I don't know how to convert the DISTINCT ON(e.id) with Zend.
It seems that I can get DISTINCT rows but no distinct columns.
$select->distinct()->from("e")->join("f", "e.id = f.id_e")
->join("g", "f.id = g.id_f");
Any idea on how to make a select with distinct column ?
Thanks for help
You probably can't do this with Zend Framework since distinct on is not part of the SQL standard (end of page in Postgres documentation). Although Postgres supports it, I would assume its not part of Zend Framework because you could in theory configure another database connection which does not offer support.
If you know in advance that you're developing for a specific database (Postgres in this case), you could use manually written statements instead. You'll gain more flexibility within the queries and better performance at the cost of no longer being able to switch databases.
You would then instantiate a Zend_Db_Apdapter for Postgres. There a various methods available to get results for SQL queries which are described in the frameworks documentation starting at section Reading Query Results. If you choose to go this route I'd recommend to create an own subclass of the Zend_Db_Adapter_Pgsql class. This is to be able to convert data types and throw exceptions in case of errors instead of returning ambiguous null values and hiding error causes.