I am using Facebook Attribution API to collect attribution data using the Facebook graph API. I am able to make GET requests and get responses from the API but I have some trouble to build my request parameters.
If you are familiar with Facebook, there is a page available at https://business.facebook.com/attribution where you get informations about all of your different sources channels. In my case I have around 10 different sources channels listed bellow:
What I am trying to achieve is to get the net_media_cost for each of those channels. However, the official documentation of the Attribution API does not say anything about this. Maybe it's not possible, but I guess the Attribution API should be able to do it.
If you check the bottom of the documentation, there is a source channel but it's a generic channel type and not accurate enough for what I am looking for (google generic vs google paid for example)
This is the request I am doing when using the source_channel from the documentation. The same from the Conversion Metrics by Attribution Model on the Attribution API documentation:
https://graph.facebook.com/v9.0/?
ids=<BUSINESS_UNIT_ID>
&fields=conversion_events
.filter_by({"ids":["<CONVERSION_EVENT_ID>"]})
.metric_scope({
"filters":{
"click_lookback_window":"2419200",
"view_lookback_window":"86400",
"visit_configuration":"include_paid_organic"
},
"time_period":"last_thirty_days"
})
{id,name,cost_per_1k_impressions,cost_per_click,cost_per_visit,net_media_cost,report_click_through_rate,report_clicks,report_impressions,report_visits,last_touch_convs,last_touch_convs_per_1k_impress,last_touch_convs_per_click,last_touch_convs_per_visit,last_touch_cpa,last_touch_revenue,last_touch_roas,total_conversions,
metrics_breakdown
.dimensions(["source_channel"])
{source_channel,cost_per_1k_impressions,cost_per_click,cost_per_visit,net_media_cost,report_click_through_rate,report_clicks,report_impressions,report_visits,last_touch_convs,last_touch_convs_per_1k_impress,last_touch_convs_per_click,last_touch_convs_per_visit,last_touch_cpa,last_touch_revenue,last_touch_roas}
}
&access_token=<TOKEN>
This is the result:
{
"<business_unit_id>: {
"conversion_events": {
"data": [
{
"id": "<custom_conversion_id>",
"name": "Total Device",
"cost_per_1k_impressions": 298883073.80651,
"cost_per_click": 7611028.5055993,
"cost_per_visit": 2292374.9356985,
"net_media_cost": 10227213000000,
"report_click_through_rate": 0.039269735172667,
"report_clicks": 1343736,
"report_impressions": 34218107,
"report_visits": 4461405,
"last_touch_convs": 16041,
"last_touch_convs_per_1k_impress": 0.46878689110417,
"last_touch_convs_per_click": 0.011937612745361,
"last_touch_convs_per_visit": 0.0035955041068901,
"last_touch_cpa": 637567046.94221,
"last_touch_revenue": 5752967.9242927,
"last_touch_roas": 5.6251570435588e-7,
"total_conversions": 23156,
"metrics_breakdown": {
"data": [
{
"source_channel": 3,
"cost_per_click": 0,
"cost_per_visit": 0,
"net_media_cost": 0,
"report_clicks": 1154015,
"report_impressions": 0,
"report_visits": 1071516,
"last_touch_convs": 10801,
"last_touch_convs_per_click": 0.0093594970602635,
"last_touch_convs_per_visit": 0.010080110796292,
"last_touch_cpa": 0,
"last_touch_revenue": 3243197.5881846
},
{
"source_channel": 1,
"cost_per_visit": 0,
"net_media_cost": 0,
"report_clicks": 0,
"report_impressions": 0,
"report_visits": 920001,
"last_touch_convs": 4434,
"last_touch_convs_per_visit": 0.0048195599787392,
"last_touch_cpa": 0,
"last_touch_revenue": 2165112.2343256
},
{
"source_channel": 2,
"cost_per_1k_impressions": 298883073.80651,
"cost_per_click": 53906594.420228,
"cost_per_visit": 95867239.714663,
"net_media_cost": 10227213000000,
"report_click_through_rate": 0.0055444621761221,
"report_clicks": 189721,
"report_impressions": 34218107,
"report_visits": 106681,
"last_touch_convs": 806,
"last_touch_convs_per_1k_impress": 0.023554780514305,
"last_touch_convs_per_click": 0.0042483436203689,
"last_touch_convs_per_visit": 0.0075552347653284,
"last_touch_cpa": 12688849875.931,
"last_touch_revenue": 344658.10178253,
"last_touch_roas": 3.3700100093988e-8
},
{
"source_channel": 0,
"cost_per_visit": 0,
"net_media_cost": 0,
"report_clicks": 0,
"report_impressions": 0,
"report_visits": 2363207,
"last_touch_convs": 0,
"last_touch_convs_per_visit": 0,
"last_touch_revenue": 0
}
]
}
}
]
},
"id": "318429215527453"
}
}
The result of that request has everything I am looking for expect that the source_channel is the one from the official documentation and I would need to have it spliced based on my own attribution channels.
Related
I am trying to integrate this add stock feature into the app,but i am kind of stuck how do pass those incremented items data to http package,i have made view and shows data just like this
please checkout entire code for this process
https://github.com/Abhijithsp/flutter-image_upload/blob/master/Add%20Stocks.dart
Process
How do i pass the data just like this
{
"id" : "", //NOT NULL IF EDIT
"station_id": 2,
"name" : "Test Stocks",
"unit" : "1", (DROPDOWN) ==>1 UNITS
"branches" : [{
"branch_id": 87,
"kg" : 10,
"gm" : 5,
"ltr" : 0,
"ml" : 0,
"counter" : 0
},{
"branch_id": 88,
"kg" : 10,
"gm" : 8,
"ltr" : 0,
"ml" : 0,
"counter" : 0
},
]
}
You can use list_branch_details along with its StockBranchDetailsModel store inside it for every rows. and whenever any change in data the you can save to particular position of that list named list_branch_details and while sending data
var data ={};
var branches=[];
var branch ={};
data["name"] = "Test Stocks";
data["unit"] = "1";
for(int i =0;i< list_branch_details.length; i++ ) {
branch = {};
branch["branch_id"] = list_branch_details.get(i).getBranchId();
branch["kg"] = list_branch_details.get(i).getKg();
branches.add(branch);
}
data["branches"] = branches;
and just encode it.
jsonEncode(data);
I have a very large collection (millions of documents) with data which looks like:
u'timestamp': 1481454871423.0,
u'_id': ObjectId('584d351772c4d8106cc43116'),
u'data': {
...
},
u'geocode': [{u'providerId': 2, u'placeId': 97459515},
{u'providerId': 3, u'placeId': 237},
{u'providerId': 3, u'placeId': 3}]
I want a query which targets a providerId and placeId pair, and returns 10 records only, within a timestamp range.
To that end I perform queries like:
'geocode.providerId': 3,
'geocode.placeId': 3
'timestamp': { '$gte': 1481454868360L,
'$lt': 1481454954839L }
And I provide a hint, to target the index which looks like:
[('geocode.providerId', 1), ('geocode.placeId', 1), ('timestamp', 1)]
where 1 is ascending. Before iterating over the returned cursor, it is limited to 10 records and sorted ascending on timestamp (which should be it's default state due to the index).
A pymongo query looks like:
collection.find(findDic).hint(hint).sort([('timestamp', pymongo.ASCENDING)]).skip(0).limit(10)
The query explains come back looking like:
{
u'executionStats': {
u'executionTimeMillis': 1270,
u'nReturned': 10,
u'totalKeysExamined': 568686,
u'allPlansExecution': [],
u'executionSuccess': True,
u'executionStages': {
u'needYield': 0,
u'saveState': 4442,
u'memUsage': 54359,
u'restoreState': 4442,
u'memLimit': 33554432,
u'isEOF': 1,
u'inputStage': {
u'needYield': 0,
u'saveState': 4442,
u'restoreState': 4442,
u'isEOF': 1,
u'inputStage': {
u'needYield': 0,
u'docsExamined': 284964,
u'saveState': 4442,
u'restoreState': 4442,
u'isEOF': 1,
u'inputStage': {
u'saveState': 4442,
u'isEOF': 1,
u'seenInvalidated': 0,
u'keysExamined': 568686,
u'nReturned': 284964,
u'invalidates': 0,
u'keyPattern': {u'geocode.providerId': 1,
u'timestamp': 1,
u'geocode.placeId': 1},
u'isUnique': False,
u'needTime': 283722,
u'isMultiKey': True,
u'executionTimeMillisEstimate': 360,
u'dupsTested': 568684,
u'restoreState': 4442,
u'direction': u'forward',
u'indexName': u'geocode.providerId_1_geocode.placeId_1_timestamp_1',
u'isSparse': False,
u'advanced': 284964,
u'stage': u'IXSCAN',
u'dupsDropped': 283720,
u'needYield': 0,
u'isPartial': False,
u'indexBounds': {u'geocode.providerId': [u'[3, 3]'
],
u'timestamp': [u'[-inf.0, 1481455513378)'
],
u'geocode.placeId': [u'[MinKey, MaxKey]'
]},
u'works': 568687,
u'indexVersion': 1,
},
u'nReturned': 252823,
u'needTime': 315863,
u'filter': {u'$and': [{u'geocode.placeId': {u'$eq': 3}},
{u'timestamp': {u'$gte': 1481405886510L}}]},
u'executionTimeMillisEstimate': 970,
u'alreadyHasObj': 0,
u'invalidates': 0,
u'works': 568687,
u'advanced': 252823,
u'stage': u'FETCH',
},
u'nReturned': 0,
u'needTime': 315864,
u'executionTimeMillisEstimate': 1150,
u'invalidates': 0,
u'works': 568688,
u'advanced': 0,
u'stage': u'SORT_KEY_GENERATOR',
},
u'nReturned': 10,
u'needTime': 568688,
u'sortPattern': {u'timestamp': 1},
u'executionTimeMillisEstimate': 1200,
u'limitAmount': 10,
u'invalidates': 0,
u'works': 568699,
u'advanced': 10,
u'stage': u'SORT',
},
u'totalDocsExamined': 284964,
},
u'queryPlanner': {
u'parsedQuery': {u'$and': [{u'geocode.placeId': {u'$eq': 3}},
{u'geocode.providerId': {u'$eq': 3}},
{u'timestamp': {u'$lt': 1481455513378L}},
{u'timestamp': {u'$gte': 1481405886510L}}]},
u'rejectedPlans': [],
u'namespace': u'mxp957.tweet_244de17a-aa75-4da9-a6d5-97b9281a3b55',
u'winningPlan': {
u'sortPattern': {u'timestamp': 1},
u'inputStage': {u'inputStage': {u'filter': {u'$and': [{u'geocode.placeId': {u'$eq': 3}},
{u'timestamp': {u'$gte': 1481405886510L}}]},
u'inputStage': {
u'direction': u'forward',
u'indexName': u'geocode.providerId_1_geocode.placeId_1_timestamp_1',
u'isUnique': False,
u'isSparse': False,
u'isPartial': False,
u'indexBounds': {u'geocode.providerId': [u'[3, 3]'],
u'timestamp': [u'[-inf.0, 1481455513378)'
],
u'geocode.placeId': [u'[MinKey, MaxKey]'
]},
u'isMultiKey': True,
u'stage': u'IXSCAN',
u'indexVersion': 1,
u'keyPattern': {u'geocode.providerId': 1,
u'timestamp': 1,
u'geocode.placeId': 1},
}, u'stage': u'FETCH'},
u'stage': u'SORT_KEY_GENERATOR'},
u'limitAmount': 10,
u'stage': u'SORT',
},
u'indexFilterSet': False,
u'plannerVersion': 1,
},
u'ok': 1.0,
u'serverInfo': {
u'host': u'rabbit',
u'version': u'3.2.11',
u'port': 27017,
u'gitVersion': u'009580ad490190ba33d1c6253ebd8d91808923e4',
},
}
I don't understand why all of these documents need to be examined. In the case above, the size of the collection is only 284587 which means that every record was looked at twice! I want totalKeysExamined to only be 10, but am struggling to see how to achieve this.
I am using MongoDB version 3.2.11 and pymongo.
As Astro mentioned, the issue is that MongoDB was not using the index effectively.
MongoDB team say that this is resolved in later versions:
https://jira.mongodb.org/browse/SERVER-27386
Also an option is to remove providerId from the index. In my use case, providerId is one of two values, and most of the time will always be the same value. It represents which API was used to geocode; my system only supports two, and only has one enabled at any one time.
See the commit that would resolve this:
https://github.com/watfordxp/GeoTweetSearch/commit/420536e4a138fb22e0dd0e61ef9c83c23a9263c1
Firebase docs covers Filtering by a specified child key. But it assumes that the child value is know. It does not cover a situation where the child is a dynamic value (eg : Student Roll Number)
Specified Child Name Example Data
{
"Student1": {
"semister": {
"english" : 80,
"maths" : 60
}
},
"Student2": {
"semister": {
"english" : 40,
"maths" : 45
}
}
}
Get students who have scored marks higher than 50 in maths
[GET] https://<baseurl>/students.json?orderBy="semister/maths"&startAt=50
Dynamic Child Name Example Data
{
"Student1": {
"unit1": {
"english" : 80,
"maths" : 60
},
"unit2": {
"english" : 80,
"maths" : 60
}
},
"Student2": {
"semister": {
"english" : 40,
"maths" : 50
},
"unit2": {
"english" : 80,
"maths" : 60
}
}
}
I get error by using below
Get students who have scored marks higher than 50 in maths in any unit
[GET] https://<baseurl>/students.json?orderBy="$unit/maths"&startAt=50
Any help / suggestion is appreciated
The problem is you don't have a firebase location for all your units. If you want to have more powerful queries, you need to make your data more relational.
`
Students: {
student1: {
units: {
unit1:true,
unit2:true
}
},
student2: {
units:{
unit3:true,
unit4:true
}
}
}
Units: {
unit1: {
student: student1
english: 80,
math: 40,
},
unit2: {
student: student1
english: 60,
math: 75,
}
unit3: {
student: student2
english: 63,
math: 29,
}
unit4: {
student: student2
english: 20,
math: 90,
}
}
`
This organization would allow you to perform the query you want, but more importantly, other objects can reference the same instance of unit1, so now you have one source of truth for the units information so you won't have conflicting data.
This is a small code I am using to do a simple search:
import com.sksamuel.elastic4s.{ElasticsearchClientUri, ElasticClient}
import com.sksamuel.elastic4s.ElasticDsl._
import org.elasticsearch.common.settings.ImmutableSettings
object Main3 extends App {
val uri = ElasticsearchClientUri("elasticsearch://localhost:9300")
val settings = ImmutableSettings.settingsBuilder().put("cluster.name", "elasticsearch").build()
val client = ElasticClient.remote(settings, uri)
if (client.exists("bands").await.isExists()) {
println("Index already exists!")
val num = readLine("Want to delete the index? ")
if (num == "y") {
client.execute {deleteIndex("bands")}.await
} else {
println("Leaving this here ...")
}
} else {
println("Creating the index!")
client.execute(create index "bands").await
client.execute(index into "bands/artists" fields "name"->"coldplay").await
val resp = client.execute(search in "bands/artists" query "coldplay").await
println(resp)
}
client.close()
}
This is result that I get:
Connected to the target VM, address: '127.0.0.1:51872', transport: 'socket'
log4j:WARN No appenders could be found for logger (org.elasticsearch.plugins).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Creating the index!
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" : [ ]
}
}
Disconnected from the target VM, address: '127.0.0.1:51872', transport: 'socket'
Process finished with exit code 0
Creation of an index and adding document to this index is running fine but a simple search query is giving no result. I even checked this on Sense.
GET bands/artists/_search
{
"query": {
"match": {
"name": "coldplay"
}
}
}
gives
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.30685282,
"hits": [
{
"_index": "bands",
"_type": "artists",
"_id": "AU21OYO9w-qZq8hmdTOl",
"_score": 0.30685282,
"_source": {
"name": "coldplay"
}
}
]
}
}
How to solve this issue?
I suspect what is happening is that you are doing the search straight after the index operation in your code. However in elasticsearch documents are not ready for search immediately. See refresh interval setting here. (So when you use the rest client, you are waiting a few seconds by virtue of the fact you have to manually flick between tabs, etc).
You could test this quickly by putting a Thread.sleep(3000) after the index. If that confirms it then works, then you need to think about how you want to write your program.
Normally you just index, and when the data is available, then its available. This is called eventual consistency. In the meantime (seconds) users might not have it available to search. That's usually not a problem.
If it IS a problem, then you will have to do some tricks like we do in the unit tests of elastic4s where you keep 'count'ing until you get back the right number of documents.
Finally, you can also manually 'refresh' the index to speed things up, by calling
client.execute {
refresh index "indexname"
}
But that's usually only used when you turn off the automatic refreshing for bulk inserts.
I have about 1000000 documents in a collections (random generated).
Sample document:
{
"loc": {
"lat": 39.828475,
"lon": 116.273542
},
"phone": "",
"keywords": [
"big",
"small",
"biggest",
"smallest"
],
"prices": [
{
"category": "uRgpiamOVTEQ",
"list": [
{
"price": 29,
"name": "ehLYoPpntlil"
}
]
},
{
"category": "ozQNmdwpwhXPnabZ",
"list": [
{
"price": 96,
"name": "ITTaLHf"
},
{
"price": 88,
"name": "MXVgJFBgtwLYk"
}
]
},
{
"category": "EDkfKGZSou",
"list": [
{
"price": 86,
"name": "JluoCLnenOqDllaEX"
},
{
"price": 35,
"name": "HbmgMDfxCOk"
},
{
"price": 164,
"name": "BlrUD"
},
{
"price": 106,
"name": "LOUcsMDeaqVm"
},
{
"price": 14,
"name": "rDkwhN"
}
]
}
],
}
Search without indexes
> db.test1.find({"prices.list.price": { $gt: 190 } }).explain()
{
"cursor" : "BasicCursor",
"isMultiKey" : false,
"n" : 541098,
"nscannedObjects" : 1005584,
"nscanned" : 1005584,
"nscannedObjectsAllPlans" : 1005584,
"nscannedAllPlans" : 1005584,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 8115,
"nChunkSkips" : 0,
**"millis" : 13803,**
"server" : "localhost:27017",
"filterSet" : false
}
With indexes:
> db.test1.ensureIndex({"prices.list.price":1,"menu.list.name":1})
{
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 1,
"numIndexesAfter" : 2,
"ok" : 1
}
> db.test1.find({"prices.list.price": { $gt: 190 } }).explain()
{
"cursor" : "BtreeCursor prices.list.price_1_prices.list.name_1",
"isMultiKey" : true,
"n" : 541098,
"nscannedObjects" : 541098,
"nscanned" : 868547,
"nscannedObjectsAllPlans" : 541098,
"nscannedAllPlans" : 868547,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 16852,
"nChunkSkips" : 0,
**"millis" : 66227,**
"indexBounds" : {
"menu.list.price" : [
[
190,
Infinity
]
],
"menu.list.name" : [
[
{
"$minElement" : 1
},
{
"$maxElement" : 1
}
]
]
},
"server" : "localhost:27017",
"filterSet" : false
}
Have any idea why indexed search slower than without index ?
Also i will use:
db.test1.find( { loc : { $near : [39.876045, 32.862245]}}) (need 2d indexes)
db.test1.find({ keywords:{$in: [ "small", "another" ] }}) (use index for keywords)
db.test1.find({"prices.list.name":/.s./ }) (no need to index because i will use regex)
An index allows faster access to location of the document that satisfies the query.
In your example, your query selects half of all the documents in the collection. So even though the index scan provides faster access to know which documents will match the query predicate, it actually creates a lot more work overall.
In collection scan, the query is scanning all of the documents, and checking the field that you are querying by to see if it matches. Half the time it ends up selecting the document.
In index scan, the query is traversing half of all the index entries and then jumping from them directly to the documents that satisfy the query predicate. That's more operations in your case.
In addition, while doing this, the operations are yielding the read mutex when they need to wait for the document they have to read to be brought into RAM, or when there is a write that is waiting to go, and the index scan is showing double the number of yields as the collection scan. If you don't have enough RAM for your working set, then adding an index will put more pressure on the existing resources and make things slower, rather than faster.
Try the same query with price compared to a much larger number, like 500 (or whatever would be a lot more selective in your data set). If the query is still slower with an index, then you are likely seeing a lot of page faulting on the system. But if there is enough RAM for the index, then the indexed query will be a lot faster while the unindexed query will be just as slow.
First, as a suggestion you will get more faster while querying arrays with elemMatch.
http://docs.mongodb.org/manual/reference/operator/query/elemMatch/
In your case
db.test1.find({"prices.list.price":{ $elemMatch: { $gte: 190 }} })
Second thing is
To index a field that holds an array value, MongoDB adds index items
for each item in the array. These multikey indexes allow MongoDB to
return documents from queries using the value of an array. MongoDB
automatically determines whether to create a multikey index if the
indexed field contains an array value; you do not need to explicitly
specify the multikey type.
Consider the following illustration of a multikey index:
Diagram of a multikey index on the addr.zip field. The addr field contains an array of
address documents. The address documents contain the zip field.
Multikey indexes support all operations supported by other MongoDB
indexes; however, applications may use multikey indexes to select
documents based on ranges of values for the value of an array.
Multikey indexes support arrays that hold both values (e.g. strings,
numbers) and nested documents.
from http://docs.mongodb.org/manual/core/index-multikey/