Find distinct field in multiple same collections mongodb - mongodb

Question

I have 7 collections in one database that the collections are the same and I have to find a unique field in those collections and count it or show it in a new collection named result.
I tried lookup but it it did not work. Can anyone guide me how to do the job?
These are 7 collections: 20200309-20200310-20200311-20200312-20200313-20200314-20200315 and result is the collection for store the result of query.
col : 20200309
{
"_id" : ObjectId("5e6f7c7c0371c86b8737628b"),
"sid" : 13328,
"trans-id" : "PROV_158374123364907198165",
"status" : "1",
"base-price-point" : "6000",
"msisdn" : "989115506327",
"keyword" : "",
"validity" : 0,
"next_renewal_date" : "",
"shortcode" : "",
"billed-price-point" : "",
"trans-status" : 0,
"chargeCode" : "AVMREWCAVMAW6000",
"datetime" : "2020-03-09 11:37:13.649",
"event-type" : "1.5",
"channel" : "system"
}
{
"_id" : ObjectId("5e6f7c7c0371c86b8737628c"),
"sid" : 13328,
"trans-id" : "PROV_158374123384007267165",
"status" : "1",
"base-price-point" : "6000",
"msisdn" : "989107351827",
"keyword" : "",
"validity" : 0,
"next_renewal_date" : "",
"shortcode" : "",
"billed-price-point" : "",
"trans-status" : 0,
"chargeCode" : "AVMREWCAVMAW6000",
"datetime" : "2020-03-09 11:37:13.840",
"event-type" : "1.5",
"channel" : "system"
}
col : 20200310
{
"_id" : ObjectId("5e6f7d140371c86b873e6bce"),
"sid" : 13328,
"trans-id" : "PROV_158383144246275616515",
"status" : "1",
"base-price-point" : "6000",
"msisdn" : "989909789746",
"keyword" : "",
"validity" : 0,
"next_renewal_date" : "",
"shortcode" : "",
"billed-price-point" : "",
"trans-status" : 0,
"chargeCode" : "AVMREWCAVMAW6000",
"datetime" : "2020-03-10 12:40:42.462",
"event-type" : "1.5",
"channel" : "system"
}
{
"_id" : ObjectId("5e6f7d140371c86b873e6bcf"),
"sid" : 13328,
"trans-id" : "PROV_158382430015338271227",
"status" : "1",
"base-price-point" : "6000",
"msisdn" : "989901812412",
"keyword" : "",
"validity" : 0,
"next_renewal_date" : "",
"shortcode" : "",
"billed-price-point" : "",
"trans-status" : 0,
"chargeCode" : "AVMREWCAVMAW6000",
"datetime" : "2020-03-10 10:41:40.153",
"event-type" : "1.5",
"channel" : "system"
}
and there is 5 collections like above

Answer 1

The following script can be used to get distinct msisdn values in an array and the count. Run the script from the mongo shell.
For this to run efficiently, there needs to be an index on the two fields msisdn and status as: { status: 1, msisdn: 1}. The index needs to be created on all the collections. Note the indexes are not required in case the collections have just few thousand documents.
collections = [ "20200309", "20200310", // ... ]
combined = [ ]
for (let coll of collections) {
combined = combined.concat( db.getCollection(coll).distinct( "msisdn", { status: "1" } ) )
}
combined = [...new Set(combined) ]
print('Combined count:', combined.length)
The combined = [...new Set(combined) ] statement removes duplicates from the combined distinct values from the seven collections.