I asked this question previously regarding SQL Server: Complicated SQL Query--finding items matching multiple different foreign keys
Basically, I need to be able to find products that match multiple criteria. I have a scenario where I need to find products that match each of multiple categories and are found in multiple invoices.
The solution was a rather complex set of unions, which amounts to counting the number times a product matched the criteria and filtering for items whose count matched the count of criteria.
; with data (ID, Count) as (
select pc.ProductID, count(*) from ProductCategories pc (nolock)
inner join #categoryIDs /*table valued param*/ c on c.ID = pc.CategoryID
union all
select ip.ProductID, count(*) from InvoiceProducts ip (nolock)
inner join #invoiceIDs i on i.ID = ip.InvoiceID
)
select d.ID from data d
group by d.ID
having sum(d.Count) = #matchcount
But now, I am considering a NoSQL provider. So my question is, how would I create an index function to match this kind of query in RavenDB (or some other NoSQL project)?
A mental shift is required to properly set this up with RavenDB (or any other document DB). The problem is with the hacks we all used to make when working with structured data against an SQL server.
Therefore, the question here is how your data is modeled. To be more exact - how are you going to use it most often; based on that there are certain guidelines on which entities to define and how to link them together.
For a simple Product object, with String[] of categories, you can query the DB like this:
// Query on nested collections - will return any product with category "C#"
products = from p in session.Query<Product>()
where p.Categories.Any(cat => cat == "C#")
select c;
You can add as many Where clauses as you want. An index will be automatically created for you - but it is recommended to use static indexes when you've settled on a Model.
More on this topic:
http://ayende.com/blog/4801/leaving-the-relational-mindset-ravendbs-trees
https://github.com/ravendb/docs
Related
Okay. So the question that I got asked by the teacher was this:
(5 marks) Construct a SQL query on the dvdrental database that uses a natural join of two or more tables and an additional where condition. (E.g. find the titles of films rented by a particular customer.) Note the hints on the course news page if your query returns nothing.
Here is the layout of the database im working with:
http://www.postgresqltutorial.com/wp-content/uploads/2013/05/PostgreSQL-Sample-Database.png
The hint to us was this:
PostgreSQL hint:
If a natural join doesn't produce any results in the dvdrental DB, it is because many tables have the last update: timestamp field, and thus the natural join tries to join on that field as well as the intended field.
e.g.
select *
from film natural join inventory;
does not work because of this - it produces an empty table (no results).
Instead, use
select *
from film, inventory
where film.film_id = inventory.film_id;
This is what I did:
select *
from film, customer
where film.film_id = customer.customer_id;
The problem is I cannot get a particular customer.
I tried doing customer_id = 2; but it returns a error.
Really need help!
Well, it seems that you would like to join two tables that have no direct relation with each other, there's your issue:
where film.film_id = customer.customer_id
To find which films are rented by which customer you would have to join customer table with rental, then with inventory and finally with film.
The task description states
Construct a SQL query on the dvdrental database that uses a natural join of two or more tables and an additional where condition.quote
I need the JPQL in this format:
SELECT *
FROM Person p
LEFT JOIN Address a ON p.id = a.id AND a.flat = 100;
But when i run the code the query being slightly modified henceforth the output of data changed..
SELECT * FROM Person p LEFT JOIN Address a ON p.id = a.id where a.flat = 100;
I suppose you want to format the sql output.
So for hibernate, you can set hibernate.format_sql=true to well format the sql.
If you are using spring boot, you can config it via spring.jpa.properties.hibernate.format_sql=true simply.
JPQL does not allow joining to arbitrary other root objects, so you can't do that in JPQL; only allowing you to join along relations. Individual vendors may offer a vendor extension to allow joining across arbitrary roots but you then lose portability.
To provide more than that you have to post the actual JPA entities, and you haven't.
I have a many to many relationship between two tables which is represented with an intermediary table. I am using the ZF1 table relationships methodology to model the database in my application and this works fine. One thing I am struggling with is to pull data from the intermediary table when performing a many to many lookup. For exmaple:
productsTable
product_id,
product_name
customerTable
customer_id,
customer_name
salesTable
customer_id,
product_id,
date_of_sale
In this case where the sales table is the intermediary table and the many to many relationship is between customers and products. I add the referenceMap to the sales table model for products and customers and the dependent table "sales" to the product table model and the customer table model.
I can then successfully use the following code to get all the products for a given customer (or vice-versa).
$productTable = new productsTable();
$product = $productTable->find(1)->current();
$customers = $product->findManyToManyRowset('customerTable','salesTable');
But it does not include the "date_of_sale" value in the rowset returned. Is there a way of including the values from the intermediary table without doint a separate database lookup. Ican't see anything in the zf docs.
Any help would be cool.
I hope to eventually replace the zend_table with a datamapper implementation as it seems highly inefficient in terms of the number of db queries it executes which could be hugely reduced with slightly more complex SQL join queries rather than multiple simple selects but for now I'm stuck with this.
Thanks.
You can use JOIN queries in your code to make it in one call. From what I understand, you want to build this query
SELECT p.*, c.*, s.date_of_sale FROM sales AS s
INNER JOIN products AS p ON p.product_id = s.product_id
INNER JOIN customers AS c ON c.customer_id = s.customer_id
WHERE p.product_id = '1';
To achieve this, you can refer to Complex SQL query into Zend to see how I translate it to a Zend_Db Query or just use something like
$select = $this->select()->setIntegrityCheck(false);
$select->from(array('s'=>'sales'), array('date_of_sale'));
$select->join(array('p'=>'products'), 'p.product_id = s.product_id', '*');
$select->join(array('c'=>'customers'), 'c.customer_id = s.customer_id', '*');
$select->where('p.product_id = '.$productId);
Obviously, it would be better to quoteInto the where clause but I'm just too lazy.
Hope this helps !
I have a table with posts, which are categorized by:
type
tag
language
All of those "categories" are stored in next tables (posts_types) and connected via next tables (posts_types_assignment).
COUNTing in PostgreSQL is really slow (i have more than 500k records in that table) and i need to get the number of posts categorized by any combination of type/tag/lang.
If i would solve it through triggers, it would be full of many multi-level loops, which really doesn't look like nice and is hard to maintenance.
Is there any other solution how to effectively get actual number of posts categorized in any type/tag/language?
Let me get this straight.
You have a table posts. You have a table posts_types. The two have a many to many join on posts_types_assignment. And you have some query like this that is slow:
SELECT count(*)
FROM posts p
JOIN posts_types_assigment pta1
ON p.id = pta1.post_id
JOIN posts_types pt1
ON pt1.id = pta1.post_type_id
AND pt1.type = 'language'
AND pt1.name = 'English'
JOIN posts_types_assigment pta2
ON p.id = pta2.post_id
JOIN posts_types pt2
ON pt2.id = pta2.post_type_id
AND pt2.type = 'tag'
AND pt2.name = 'awesome'
And you would like to know why it is painfully slow.
My first note is that PostgreSQL would have to do a lot less work if you had the identifiers in the posts table rather than in the joins. But that is a moot issue, the decision has been made.
My more useful note is that I believe that PostgreSQL has a similar query optimizer to Oracle. In which case to limit the combinatorial explosion of possible query plans that it has to consider, it only considers plans that start with some table, and then repeatedly joins on one more data set at a time. However no such query plan will work here. You can start with pt1, get 1 record, then go to pta1, get a bunch of records, join p, wind up with the same number of records, then join pta2, and now you get a huge number of records, then join to pt2, get just a few records. Joining to pta2 is the slow step, because the database has no idea which records you want, and therefore has to create a temporary result set for every combination of a post and a piece of metadata (type, language or tag) on it.
If this is indeed your problem, then the right plan looks like this. Join pt1 to pta1, put an index on it. Join pt2 to pta2, then join to the result of the first query, then join to p. Then count. This means that we don't get huge result sets.
If this the case, there is no way to tell the query optimizer that this once you want it to think up a new type of execution plan. But there is a way to force it.
CREATE TEMPORARY TABLE t1
AS
SELECT pta*
FROM posts_types pt
JOIN posts_types_assignment pta
ON pt.id = pta.post_type_id
WHERE pt.type = 'language'
AND pt.name = 'English';
CREATE INDEX idx1 ON t1 (post_id);
CREATE TEMPORARY TABLE t2
AS
SELECT pta*
FROM posts_types pt
JOIN posts_types_assignment pta
ON pt.id = pta.post_type_id
JOIN t1
ON t1.post_id = pta.post_id
WHERE pt.type = 'language'
AND pt.name = 'English';
SELECT COUNT(*)
FROM posts p
JOIN t1
ON p.id = t1.post_id;
Barring random typos, etc, this is likely to perform somewhat better. If it doesn't, double check the indexes on your tables.
As btilly notes, and if he has correctly guessed the schema, the table design does not help - it seems (at first sight, at least) that, for example, to have three tables posts_tag(post_id,tag) post_lang(post_id,lang) post_type(post_id,type) would be more natural and much more efficient.
Apart from that (or in addition to that), one could think of a table or materialized view that summarizes all the possible countings, with columns (lang,type,tag,nposts). Of course, to compute this in full would be VERY slow, but (apart from the first time) it can be done either in full "in background", at some intervals (if the data does not vary much, and if you don't require exact counts), or eagerly with triggers.
See for example here
(I don't think I have titled this question correctly - but I don't know how to describe it)
Here is what I am trying to do:
Let's say I have a Person table that has a PersonID field. And let's say that a Person can belong to many Groups. So there is a Group table with a GroupID field and a GroupMembership table that is a many-to-many join between the two tables and the GroupMembership table has a PersonID field and a GroupID field. So far, it is a simple many to many join.
Given a list of GroupIDs I would like to be able to write a query that returns all of the people that are in ALL of those groups (not any one of those groups). And the query should be able to handle any number of GroupIDs. I would like to avoid dynamic SQL.
Is there some simple way of doing this that I am missing?
Thanks,
Corey
select person_id, count(*) from groupmembership
where group_id in ([your list of group ids])
group by person_id
having count(*) = [size of your list of group ids]
Edited: thank you dotjoe!
Basically you are looking for Persons for whom there is no group he is not a member of, so
select *
from Person p
where not exists (
select 1
from Group g
where not exists (
select 1
from GroupMembership gm
where gm.PersonID = p.ID
and gm.GroupID = g.ID
)
)
You're basically not going to avoid "dynamic" SQL in the sense of dynamically generating the query at query time. There's no way to hand a list around in SQL (well, there is, table variables, but getting them into the system from C# is either impossible (2005 & below) or else annoying (2008)).
One way that you could do it with multiple queries is to insert your list into a work table (probably a process-keyed table) and join against that table. The only other option would be to use a dynamic query such as the ones specified by Jonathan and hongliang.