Query to find connected components in mongodb graph collection? - mongodb

Question

I want to group the connected component in the mongodb collection.
Example:
{ '_id': 1, 'data': '...', 'similar_id': [2,3,4] }
{ '_id': 2, 'data': '...', 'similar_id': [1] }
{ '_id': 3, 'data': '...', 'similar_id': [1,4] }
{ '_id': 4, 'data': '...', 'similar_id': [1,3] }
{ '_id': 7, 'data': '...', 'similar_id': [2,3,4] }
{ '_id': 5, 'data': '...', 'similar_id': [6] }
{ '_id': 6, 'data': '...', 'similar_id': [5] }
Diagram for above network.
So I want a query which can find connected components.
{ '_id': ..., 'groups': {[1,2,3,4], [5,6], [7]} }
The result may not need to look like above but only in some form such that they are separated in different groups.

Answer 1

It ain't pretty but this is what I got, a brief description of my strategy was initially creating two groups of nodes. one contains node that are "connected" (i.e both x=>y and y=>x edges exist). and the other are potential single nodes. meaning they had one or zero of the x=>y or y=>x edges.
Once achieving this all we have to do is reducing the array by connecting connected nodes.
Mind you I fully believe this is not the "best" way to achieve the result you want as I just focused on getting it done without over thinking about performance or redundancy. with that said I'm define myself as a Mongo enthusiast and I would definitely say I struggled with this a little. For me this is usually a red flag that says my schema or db solution is wrong (maybe use a graph db?). Again these are just my opinions and it's entirely possible I just tangled myself with this pipeline.
It's worth mentioning I considered an approach using $graphLookup however on a fully connected or nearly fully connected graph this has a required depth usage of n where n=number of node, eventually I decided against it although thi approach might viable if you have any prior knowledge that can limit the depth to a certain constant.
db.collection.aggregate([
{
$unwind: {
path: "$similar_id",
preserveNullAndEmptyArrays: true
}
},
{
$addFields: {
similar_id: {
$ifNull: [
"$similar_id",
"$_id"
]
}
}
},
{
$sort: {
_id: 1,
similar_id: -1
}
},
{
$addFields: {
tmpId: {
$cond: [
{
$gt: [
"$similar_id",
"$_id"
]
},
[
"$_id",
"$similar_id"
],
[
"$similar_id",
"$_id"
]
]
}
}
},
{
$group: {
_id: "$tmpId",
sum: {
$sum: 1
}
}
},
{
$facet: {
single: [
{
$match: {
sum: 1
}
},
{
$unwind: "$_id"
},
{
$group: {
_id: null,
potentionals: {
$addToSet: "$_id"
}
}
}
],
clusters: [
{
$match: {
sum: 2
}
},
{
$group: {
_id: null,
edges: {
$addToSet: "$_id"
},
}
},
{
$project: {
all: {
$reduce: {
input: "$edges",
initialValue: [],
in: {
$setUnion: [
"$$this",
"$$value"
]
}
}
},
groups: {
$reduce: {
input: "$edges",
initialValue: [],
in: {
$cond: [
{
$gt: [
{
$size: {
$filter: {
input: "$$value",
as: "subgroup",
cond: {
$gt: [
{
$size: {
$setIntersection: [
"$$subgroup",
"$$this"
]
}
},
0
]
}
}
}
},
0
]
},
{
$map: {
input: "$$value",
as: "subgroup",
in: {
$cond: [
{
$gt: [
{
$size: {
$setIntersection: [
"$$subgroup",
"$$this"
]
}
},
0
]
},
{
"$setUnion": [
"$$this",
"$$subgroup"
]
},
"$$subgroup"
]
}
}
},
{
$concatArrays: [
"$$value",
[
"$$this"
]
]
}
]
}
}
}
}
}
]
}
},
{
$unwind: {
path: "$single",
preserveNullAndEmptyArrays: true
}
},
{
$unwind: {
path: "$clusters",
preserveNullAndEmptyArrays: true
}
},
{
$project: {
groups: {
$concatArrays: [
"$clusters.groups",
{
$map: {
input: {
$filter: {
input: "$single.potentionals",
as: "pot",
cond: {
$eq: [
{
$size: {
$setIntersection: [
[
"$$pot"
],
"$clusters.all"
]
}
},
0
]
}
}
},
as: "single",
in: [
"$$single"
]
}
}
]
}
}
}
])
MongoPlayground

Answer 2

Sorry for such late reply, but maybe other will find this useful.
You can try using NetworkX library in Python.
1st unwind similar_id to have documents with pairs {'_id':1,'similar_id':2}
import networkx as nx
unwind={'$unwind':'$similar_id'}
pipeline=[unwind]
cursor=db.collection.aggregate(pipeline)
G=nx.Graph()
for c in cursor:
G.add_edge(c['_id'],c['similar_id'])
all_clusters=list(nx.connected_components(G)) # a list of all connected components
len(all_clusters) # number of connected components