I have two huge (few hundred thousands of records) collections Col1 and Col2, and I need to fetch joined data from both of them. There is a join criteria that lets me dramatically decrease number of records returned to few hundreds, so in SQL language I would run something like
SELECT ... FROM Col1 INNER JOIN Col2 ON Col1.field1 = Col2.field2
and it would run pretty fast as Col1.field1 and Col2.field2 are indexed fields. Is there any direct way or workaround to do the same thing fast in MongoDb with indexes usage, not to scan all the items?
Note: I cannot redesign collections to merge them into one.
MongoDB has no JOIN so there is not a fast equivalent. It is most likely a schema design issue but you said you can't change that. You can't query multiple collections in one query.
You can either do the join client-side in 2 queries or you can do it in non-live style by doing a map-reduce and generating a 3rd collection.
Reference this other question for details on how to do a map-reduce
In order to join in MongoDb 4.2 you can use aggregation and $lookup like this query:
db.collection.aggregate([
{ $lookup: { from: "...", ... } }
])
It is usefull for me
More information: https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/
the join in MongoDB is so expensive. 2 solutions:
Redesign merge them into one
limit, match before you join
Related
I have 2 collections companies and products where partnerId in common. I need to find the names of the company companies.name which are not available in products collection matching the partnerId. How can I do that? I have tried in both the aggregate and SQL as follows
db.companies.aggregate([{$lookup:
{
from: "products",
localField: "partner",
foreignField: "_id",
as: "products"
}
}])
and
mb.runSQLQuery(`
SELECT name
FROM companies
WHERE NOT EXISTS (
SELECT *
FROM products
WHERE partnerId=companies.partnerId
`);
I am using nosqlbooster. The first query is returning all the documents of companies and the second query is throwing an error. Thanks in advance
I don't recommend using a 3rd party to create your MongoDB queries from SQL. SQL and MongoDB query language are very different and you will not be able to effectively or efficiently utilize this language by doing so.
As a rule, in document DBs the data should be stored denormalized, unlike in transactional tabular databases where the data should normalized. So a better design would be to add the data as sub-documents i.e. a list inside the document or a dictionary/object within the document.
With the data design you have implemented, a simple query will not help - each query only runs on a single collection. However, you can use the aggregation pipeline's $loopup operator to enrich the results from one collection with results from another..
I have some complicated queries, but don't know how to code for fetching records based on my query, which has relation with one collection to other collection
I am working one use case poc, where I want to query multiple documents present in the same collection in MongoDB based on some condition? an example would be enough.
Looking for a query similar like SQL select query on two tables(following query) is it possible in MongoDB?
select t1.X, t1.Y_DT,t1.Z,t1.adj,t1.bjc,t1.jbc,t1.mnk,t2.adj1,t2.bjc1,t2.jbc1,t2.mnk1 from test250 t1, test350 t2 where t1.X = t2.X AND t1.Y_DT=t2.Y_DT AND t1.Z = t2.Z;
if possible, any example query greatly helps me out.
I need the most viable way to search docs with the following structure.
{ _id:"",
var1: number,
var2: number,
var3: number,
}
In the sql way the resultset i need would be UNION of these three (n records sorted by var1,n records sorted by var2,n records sorted by var3)
with UNION I expect the duplicates to be removed.
Being new to mongodb i am not able to find the right way to write a query for such an operation, i believe it must be possible in mongodb.
If in case its not possible, could you please suggest an alternate nosql solution.
The closest MongoDB operator to what you are looking for is an $or, but that isn't quite the same as an SQL UNION which combines two separate queries into a single result. MongoDB queries are always against a single collection, but $or allows you to have multiple query clauses.
For example:
db.collection.find(
// Find documents matching any of these values
{$or:[
{var1: 123},
{var2: 456},
{var3: 789}
]}
).sort(
// Sort in ascending order
{var1:1, var2:1, var3:1}
)
Since you are limited to querying a single collection, results will already be de-duplicated at the document level and all results will share the same sort order if one is specified.
If you want to simulate a UNION (or other operation working with multiple collections/queries) in MongoDB, you will have to write multiple queries and merge the result sets in your application code.
These are two collections
lb_order_details
lb_orders
Can anyone suggest to me how to join -- like MySQL join -- and print these two MongoDB data collections using PHP?
There is no such thing as join in mongodb but you can create relation by keeping id field common in every collection for specific user. You have to manage your database wisely to overcome join issue.
There are no joins in MongoDB. You need to retrieve data with two distinct MongoDB queries and deal with data on PHP side.