De-merge document based on a field's array data - mongodb

I have my MongoDb documents like this:
{
... OTHER FIELDS ...
"Name": "3m india ltd.",
"BSECode": "523395",
"NSECode": "3MINDIA",
"News": {
"Bites": [
{
"href": "https://www.thehindubusinessline.com/companies/3m-india-develops-cleaning-solution-covid-relevant-signage-for-global-application/article34462888.ece",
"headline": "3M India develops cleaning solution, Covid-relevant signage for global application",
"timestamp": "1 day ago",
"publisher": "The Hindu Businessline"
},
{
"href": "https://www.business-standard.com/article/news-cm/apar-industries-ltd-leads-losers-in-a-group-121042701094_1.html",
"headline": "Apar Industries Ltd leads losers in ‘A’ group",
"timestamp": "6 days ago",
"publisher": "Business Standard"
}
]
}
... OTHER FIELDS ...
}
I need to break each of these documents wrt News.Bites array field data like the following example:
{
"Name": "3m india ltd.",
"BSECode": "523395",
"NSECode": "3MINDIA",
"News": {
"href": "https://www.thehindubusinessline.com/companies/3m-india-develops-cleaning-solution-covid-relevant-signage-for-global-application/article34462888.ece",
"headline": "3M India develops cleaning solution, Covid-relevant signage for global application",
"timestamp": "1 day ago",
"publisher": "The Hindu Businessline"
}
},
{
"Name": "3m india ltd.",
"BSECode": "523395",
"NSECode": "3MINDIA",
"News": {
"href": "https://www.business-standard.com/article/news-cm/apar-industries-ltd-leads-losers-in-a-group-121042701094_1.html",
"headline": "Apar Industries Ltd leads losers in ‘A’ group",
"timestamp": "6 days ago",
"publisher": "Business Standard"
}
}
This is the projection query I have come up with yet:
coll.find({
'News': {
'$exists': true
}
}, {
'_id': 0,
'News.Bites': 1,
'NSECode': 1,
'BSECode': 1,
'Name': 1
});
Can anyone please help with this?

Related

When creating a template in whatsapp cloud api it is always rejected

I am trying to create a template from postman by sending this object:
{
"name": "listado_productos",
"language": "es",
"category": "TRANSACTIONAL",
"components": [
{
"type" : "HEADER",
"format": "document",
"example": {
"header_handle": [
"https://drive.google.com/file/d/1CcTpDZL3p0ltMFhIKU9Vhz1LWG0bQFpN/view?usp=share_link"
]}
},
{
"type": "BODY",
"text": "Buenas tardes, acá le enviamos el nuevo listado de precios. Muchas gracias"
},
{
"type": "FOOTER",
"text": "ABS."
}
]
}
If I try an example from the official business-management-api guide, it also rejects it for non-compliance with terms and conditions.
"name": "promotional_message",
"language": "en_US",
"category": "TRANSACTIONAL",
"components": [{
"type":"BODY",
"text":"Hi {{1}}, get an extra 10% off every order above $300.",
"example":{"body_text":[["Sonia"]]}
},
{
"type":"HEADER",
"format":"TEXT",
"text": "Bonus Tuesday starts now!"
},
{
"type":"FOOTER",
"text":"Not interested? Tap Stop promotions"
},
{
"type":"BUTTONS",
"buttons":[{"type":"QUICK_REPLY", "text": "Shop now"},
{"type":"QUICK_REPLY", "text": "Stop promotions"}]
}]
The response is that it is rejected for being duplicate or not complying with WhatsApp Busiess Api terms and services but it is not true, I read the terms and conditions and the file / message format complies. Does anyone identify the problem?

How to use embedsMany in laravel to get all attributes? Only foreign key is returning

Hello Good Developers,
I am trying to implement embedsMany relationship of jenssegers/laravel-mongodb
I have two collections:
ProfileSection - {
"_id": "5c865ea4257db43fe4007331",
"general_name": "MY_PROFILE",
"type": "public",
"points": 100,
"status": 1,
"translated": [
{
"con_lang": "US-EN",
"country_code": "US",
"language": "EN",
"text": "My Profile",
"description": "My Profile"
},
...
],
"updated_at": "2019-03-11T13:12:04.000Z",
"created_at": "2019-03-11T13:12:04.000Z"
}
Profile Questions - {
"_id": "5c865ea3257db43fe40072b2",
"id": "STANDARD_EDUCATION",
"general_name": "STANDARD_EDUCATION",
"country_code": "US",
"order": 1,
"profile_section_id": "5c865ea4257db43fe4007331",
"profile_section": "My Profile",
"translated": [
{
"con_lang": "US-EN",
"text": "What is the highest level of education you have completed?",
"hint": null,
"mapping": {},
"answers": [
{
"precode": "1",
"text": "3rd Grade or less",
"mapping": {}
}
]
},
{...}
],
"updated_at": "2019-03-11T13:12:03.000Z",
"created_at": "2019-03-11T13:12:03.000Z"
}
In ProfileSection I have added
public function questions()
{
return $this->embedsMany(ProfilerQuestions::class, '_id', 'profile_section_id');
}
If I execute ProfileSection::find('5c865ea4257db43fe4007331')->questions
It returns me Profile Questions Object with only one attribute: 5c865ea4257db43fe4007331 i.e ObjectId of Profile Section
I tried using ->with('questions) before accessing questions object
like this
but it's not working
I don't understand what's the issue will definitely need some help.

Cloudant Search index

I have a table with data as below:
[
{
"payment_id": 1,
"provider_id": "ABC123 ",
"status": "pending",
"item_service": [
{
"code": "NY100",
"provider_type":"Medical_Center",
"description": "Initial Consultation - History, examination and treatment",
"rate": "20"
},
{
"code": "NY101",
"provider_type":"Medical_Center",
"description": "Brief Consultation - Selective review, examination and treatment",
"rate": "25"
},
{
"code": "NY102",
"provider_type":"Medical_Center",
"description": "Standard Consultation - History, examination and treatment",
"rate": "30"
}
]
}
]
and the Search index function
the returned results are:
Please give me your thought how to split data and display with the key name on each value in results. eg:
"code": "PY102",
"provider_type":"Medical_Center",
"description": "Standard Consultation - History, examination and treatment",
"rate": "30"
If your make your index like:
function (doc) {
if (doc.item_service){
for (var m in doc.item_service){
for (var n in doc.item_service[m]){
index(n, doc.item_service[m][n], {"store":true});
}
}
}
}
than your fields will be:
"fields": {
"rate": [
"30",
"25",
"20"
],
"description": [
"Standard Consultation - History, examination and treatment",
"Brief Consultation - Selective review, examination and treatment",
"Initial Consultation - History, examination and treatment"
],
"code": [
"NY102",
"NY101",
"NY100"
],
"provider_type": [
"Medical_Center",
"Medical_Center",
"Medical_Center"
]
}
Is this the result you intended to get?

Country dimension in Analytics Reporting API v4

I want to get a report from Google Analytics v4 for pageviews by country. Since my website is targeting a specific region, I'm filtering by countries I'm most interested in using ga:countryIsoCode dimension so I don't have to type complete country name in the filter.
Now I'd be also interested in getting pageviews for all other countries combined into 'other' entry. What would be the easiest way to achieve that?
I've tried dynamic segments so far adding ga:segment to dimensions and a NOT filter in the segment with the same rule I used for filtering counties (except ga:countryIsoCode isn't allowed in segments, so I used ga:country). This produced 3 rows of data with each of three countries appearing along with 'other' every time.
I'd appreciate an advice on this as it seems like a common use case to me. Thanks in advance!
I was able to achieve that by using segments. This approach doesn't allow to batch requests though unless all of them have the same segment definition. Also, there're limitations on the number of segments to include in the report request. Very surprised this functionality is so hard to achieve. For interested, there's an example request:
{
"reportRequests": [{
"viewId": "ga:XXXXXXXXX",
"dateRanges": [{
"startDate": "YYYY-MM-DD",
"endDate": "YYYY-MM-DD"
}],
"metrics": [{
"expression": "ga:pageviews"
}, {
"expression": "ga:sessions"
}],
"dimensions": [{
"name": "ga:segment"
}],
"segments": [{
"dynamicSegment": {
"name": "Other",
"sessionSegment": {
"segmentFilters": [{
"not": "True",
"simpleSegment": {
"orFiltersForSegment": [{
"segmentFilterClauses": [{
"dimensionFilter": {
"dimensionName": "ga:country",
"operator": "IN_LIST",
"expressions": ["United States", "United Kingdom"]
}
}]
}]
}
}]
}
}
},
{
"dynamicSegment": {
"name": "United States",
"sessionSegment": {
"segmentFilters": [{
"simpleSegment": {
"orFiltersForSegment": [{
"segmentFilterClauses": [{
"dimensionFilter": {
"dimensionName": "ga:country",
"operator": "EXACT",
"expressions": "United States"
}
}]
}]
}
}]
}
}
},
{
"dynamicSegment": {
"name": "United Kingdom",
"sessionSegment": {
"segmentFilters": [{
"simpleSegment": {
"orFiltersForSegment": [{
"segmentFilterClauses": [{
"dimensionFilter": {
"dimensionName": "ga:country",
"operator": "EXACT",
"expressions": "United Kingdom"
}
}]
}]
}
}]
}
}
}],
'hideTotals': 'true',
'hideValueRanges': 'true'
}]
}

MongoDB Database Structure and Best Practices Help

I'm in the process of developing Route Tracking/Optimization software for my refuse collection company and would like some feedback on my current data structure/situation.
Here is a simplified version of my MongoDB structure:
Database: data
Collections:
“customers” - data collection containing all customer data.
[
{
"cust_id": "1001",
"name": "Customer 1",
"address": "123 Fake St",
"city": "Boston"
},
{
"cust_id": "1002",
"name": "Customer 2",
"address": "123 Real St",
"city": "Boston"
},
{
"cust_id": "1003",
"name": "Customer 3",
"address": "12 Elm St",
"city": "Boston"
},
{
"cust_id": "1004",
"name": "Customer 4",
"address": "16 Union St",
"city": "Boston"
},
{
"cust_id": "1005",
"name": "Customer 5",
"address": "13 Massachusetts Ave",
"city": "Boston"
}, { ... }, { ... }, ...
]
“trucks” - data collection containing all truck data.
[
{
"truckid": "21",
"type": "Refuse",
"year": "2011",
"make": "Mack",
"model": "TerraPro Cabover",
"body": "Mcneilus Rear Loader XC",
"capacity": "25 cubic yards"
},
{
"truckid": "22",
"type": "Refuse",
"year": "2009",
"make": "Mack",
"model": "TerraPro Cabover",
"body": "Mcneilus Rear Loader XC",
"capacity": "25 cubic yards"
},
{
"truckid": "12",
"type": "Dump",
"year": "2006",
"make": "Chevrolet",
"model": "C3500 HD",
"body": "Rugby Hydraulic Dump",
"capacity": "15 cubic yards"
}
]
“drivers” - data collection containing all driver data.
[
{
"driverid": "1234",
"name": "John Doe"
},
{
"driverid": "4321",
"name": "Jack Smith"
},
{
"driverid": "3421",
"name": "Don Johnson"
}
]
“route-lists” - data collection containing all predetermined route lists.
[
{
"route_name": "monday_1",
"day": "monday",
"truck": "21",
"stops": [
{
"cust_id": "1001"
},
{
"cust_id": "1010"
},
{
"cust_id": "1002"
}
]
},
{
"route_name": "friday_1",
"day": "friday",
"truck": "12",
"stops": [
{
"cust_id": "1003"
},
{
"cust_id": "1004"
},
{
"cust_id": "1012"
}
]
}
]
"routes" - data collections containing data for all active and completed routes.
[
{
"routeid": "1",
"route_name": "monday1",
"start_time": "04:31 AM",
"status": "active",
"stops": [
{
"customerid": "1001",
"status": "complete",
"start_time": "04:45 AM",
"finish_time": "04:48 AM",
"elapsed_time": "3"
},
{
"customerid": "1010",
"status": "complete",
"start_time": "04:50 AM",
"finish_time": "04:52 AM",
"elapsed_time": "2"
},
{
"customerid": "1002",
"status": "incomplete",
"start_time": "",
"finish_time": "",
"elapsed_time": ""
},
{
"customerid": "1005",
"status": "incomplete",
"start_time": "",
"finish_time": "",
"elapsed_time": ""
}
]
}
]
Here is the process thus far:
Each day drivers begin by Starting a New Route. Before starting a new route drivers must first input data:
driverid
date
truck
Once all data is entered correctly the Start a New Route will begin:
Create new object in collection “routes”
Query collection “route-lists” for “day” + “truck” match and return "stops"
Insert “route-lists” data into “routes” collection
As driver proceeds with his daily stops/tasks the “routes” collection will update accordingly.
On completion of all tasks the driver will then have the ability to Complete the Route Process by simply changing “status” field to “active” from “complete” in the "routes" collection.
That about sums it up. Any feedback, opinions, comments, links, optimization tactics are greatly appreciated.
Thanks in advance for your time.
You database schema looks like for me as 'classic' relational database schema. Mongodb good fit for data denormaliztion. I guess when you display routes you loading all related customers, driver, truck.
If you want make your system really fast you may embedd everything in route collection.
So i suggest following modifications of your schema:
customers - as-is
trucks - as-is
drivers - as-is
route-list:
Embedd data about customers inside stops instead of reference. Also embedd truck. In this case schema will be:
{
"route_name": "monday_1",
"day": "monday",
"truck": {
_id = 1,
// here will be all truck data
},
"stops": [{
"customer": {
_id = 1,
//here will be all customer data
}
}, {
"customer": {
_id = 2,
//here will be all customer data
}
}]
}
routes:
When driver starting new route copy route from route-list and in addition embedd driver information:
{
//copy all route-list data (just make new id for the current route and leave reference to routes-list. In this case you will able to sync route with route-list.)
"_id": "1",
route_list_id: 1,
"start_time": "04:31 AM",
"status": "active",
driver: {
//embedd all driver data here
},
"stops": [{
"customer": {
//all customer data
},
"status": "complete",
"start_time": "04:45 AM",
"finish_time": "04:48 AM",
"elapsed_time": "3"
}]
}
I guess you asking yourself what do if driver, customer or other denormalized data changed in main collection. Yeah, you need update all denormalized data within other collections. You will probably need update billions of documents (depends on your system size) and it's okay. You can do it async if it will take much time.
What benfits in above data structure?
Each document contains all data that you may need to display in your application. So, for instance, you no need load related customers, driver, truck when you need display routes.
You can make any difficult queries to your database. For example in your schema you can build query that will return all routes thats contains stops in stop of customer with name = "Bill" (you need load customer by name first, get id, and look by customer id in your current schema).
Probably you asking yourself that your data can be unsynchronized in some cases, but to solve this you just need build a few unit test to ensure that you update your denormolized data correctly.
Hope above will help you to see the world from not relational side, from document database point of view.