I am currently using this sqlite query in my application. Two tables are used in this query.....
UPDATE table1 set visited = (SELECT COUNT(DISTINCT table1.itemId) from 'table2' WHERE table2.itemId = table1.itemId AND table2.sessionId ='eyoge2avao');
It is working correct.... My problem is it is taking around 10 seconds to execute this query and retrieve the result..... Don't know what to do... Almost all other process are in right way.. So it seems the problem is with this query formation...
Plz someone help with how to optimize this query....
Regards,
Brian
Make sure you have indexes on the following (combinations of) fields:
table1.itemId
(This will speed up the DISTINCT clause, since the itemId will already be in the correct order).
table2.itemId, table2.sessionId
This will speed up the WHERE clause of your SELECT statement.
How many rows are there in these tables?
Aso try doing an EXPLAIN on your SELECT command to see if it gives you any helpful advice.
Related
code:
SELECT * FROM `detail` WHERE country='Malaysia' or state='' or region='' ORDER BY rand() LIMIT 4
In this query I want to find record which is related to malaysia. Only one record that I have in my table which is related to country='malaysia' but it show other 4 records. I don't have any idea why its happening?. So, How can I solve this issue? Please help me.
Thank You
You are also including records which have empty string for the state or region. Maybe you should just be checking the country field:
SELECT *
FROM detail
WHERE country = 'Malaysia'
ORDER BY rand()
LIMIT 4;
You can also remove
ORDER BY rand()
because sql doesn't sort the result by default, so this statement is superfluous.
I'm trying to extract the data by joining the two table, in pyspark. My join Query looks like:
SELECT COUNT(DISTINCT m.ticker),to_date(m.date) FROM extractalpha_cam2 m LEFT OUTER JOIN TOP1000 u ON u.date = to_date(m.date) GROUP BY m.date ORDER BY m.date
It is throwing the error:
Error:Py4JJavaError: An error occurred while calling
z:org.apache.zeppelin.spark.ZeppelinContext.showDF
But when, i tried extracting the data from each table, it's working fine. My queries from single table are like
SELECT to_date(date) FROM extractalpha_cam2
SELECT date from TOP1000
These two queries working fine. Can anyone help me in extracting the data from both table by joining.
It would be really helpful if anyone can share any such link, which can guide me in writing the efficient queries in pyspark.
I checked and found that, this error comes when, the job you are running took more time than the time you set for timeout. In my case it was 300 seconds.
Let me know if anyone has more valuable answer than this. Thanks
I have a custom query along these lines. I get the list of orderIds from outside. I have the entire order object list with me, so I can change the query in any way, if needed.
#Query("SELECT p FROM Person p INNER JOIN p.orders o WHERE o.orderId in :orderIds)")
public List<Person> findByOrderIds(#Param("orderIds") List<String> orderIds);
This query works fine, but sometimes it may have anywhere between 50-1000 entries in the orderIds list sent from outside function. So it becomes very slow, taking as much as 5-6 seconds which is not fast enough. My question is, is there a better, faster way to do this? When I googled, and on this site, I see we can use ANY, EXISTS: Postgresql: alternative to WHERE IN respective WHERE NOT IN or create a temporary table: https://dba.stackexchange.com/questions/12607/ways-to-speed-up-in-queries-under-postgresql or join this to VALUES clause: Alternative when IN clause is inputed A LOT of values (postgreSQL). All these answers are tailored towards direct SQL calls, nothing based on JPA. ANY keyword is not supported by spring-data. Not sure about creating temporary tables in custom queries. I think I can do it with native queries, but have not tried it. I am using spring-data + OpenJPA + PostgresSQL.
Can you please suggest a solution or give pointers? I apologize if I missed anything.
thanks,
Alice
You can use WHERE EXISTS instead of IN Clause in a native SQL Query as well as in HQL in JPA which results in a lot of performance benefits. Please see sample below
Sample JPA Query:
SELECT emp FROM Employee emp JOIN emp.projects p where NOT EXISTS (SELECT project from Project project where p = project AND project.status <> 'Active')
I usually use ORM instead of SQL and I am slightly out of touch on the different JOINs...
SELECT `order_invoice`.*
, `client`.*
, `order_product`.*
, SUM(product.cost) as net
FROM `order_invoice`
LEFT JOIN `client`
ON order_invoice.client_id = client.client_id
LEFT JOIN `order_product`
ON order_invoice.invoice_id = order_product.invoice_id
LEFT JOIN `product`
ON order_product.product_id = product.product_id
WHERE (order_invoice.date_created >= '2009-01-01')
AND (order_invoice.date_created <= '2009-02-01')
GROUP BY `order_invoice`.`invoice_id`
The tables/ columns are logically names... it's an shop type application... the query works... it's just very very slow...
I use the Zend Framework and would usually use Zend_Db_Table_Row::find(Parent|Dependent)Row(set)('TableClass') but I have to make lots of joins and I thought it'll improve performance by doing it all in one query instead of hundreds...
Can I improve the above query by using more appropriate JOINs or a different implementation? Many thanks.
The query is wrong, the GROUP BY is wrong. All columns in the SELECT-part that are not in an aggregate function, have to be in the GROUP BY. You mention only one column.
Change the SQL Mode, set it to ONLY_FULL_GROUP_BY.
When this is done and you have a correct query, use EXPLAIN to find out how the query is executed and what indexes are used. Then start optimizing.
some time ago I happend to resolve a PG related problem with this SO question of mine.
Basically it's about using row_number over a partition in 8.4.
Sadly now I have to create the same thing for 8.2 since one of my customers is on
8.2 and needs it desperatly.
What I do know (on 8.4) is the following:
SELECT custId, custName, 'xyz-' || row_number() OVER (PARTITION by custId)
AS custCode
Basically counting the occurances of custId and assigning custCodes from that.
(just an example, to show what I to; of course the query is way more complex)
I figured the solutions provided to the question mentioned above, but did'nt get them
working since there's one more hurdle to take. I don't run SQL directly I have to
embed it into a xml based config file which creates a certain xml format from the query
results. So creating temp stuff or procedures is not really an option.
So here's the question, does anyone of you guys have an idea how to port that solution of
mine to PG 8.2?
TIA
K
use depesz solution http://www.depesz.com/index.php/2007/08/17/rownum-anyone-cumulative-sum-in-one-query/