Can't get Service Alerts Protobuff to include header_text or description_text using Python gtfs_realtime_pb2 module - service

We are having difficulty adding header_text and description_text to a Service Alerts protobuff file. We are attempting to match the example shown on this page here.
https://developers.google.com/transit/gtfs-realtime/examples/alerts
Our data starts in the following dictionary:
alerts_dict = {
"header": {
"gtfs_realtime_version": "1",
"timestamp": "1543318671",
"incrementality": "FULL_DATASET"
},
"entity": [{
"497": {
"active_period": [{
"start": 1525320000,
"end": 1546315200
}],
"url": "http://www.capmetro.org/planner",
"effect": 4,
"header_text": "South 183: Airport",
"informed_entity": [{
"route_type": "3",
"route_id": "17",
"trip": "",
"stop_id": "3304"
}, {
"route_type": "3",
"route_id": "350",
"trip": "",
"stop_id": "3304"
}],
"description_text": "Stop closed temporarily",
"cause": 2
},
"460": {
"active_period": [{
"start": 1519876800,
"end": 1546315200
}],
"url": "http://www.capmetro.org/planner",
"effect": 4,
"header_text": "Ave F / Duval Detour",
"informed_entity": [{
"route_type": "3",
"route_id": "7",
"trip": "",
"stop_id": "1167"
}, {
"route_type": "3",
"route_id": "7",
"trip": "",
"stop_id": "1268"
}],
"description_text": "Stop closed temporarily",
"cause": 2
}
}]
}
Our Python code is as follows:
newfeed = gtfs_realtime_pb2.FeedMessage()
newfeedheader = newfeed.header
newfeedheader.gtfs_realtime_version = '2.0'
for alert_id, alert_dict in alerts_dict["entity"][0].iteritems():
print(alert_id)
print(alert_dict)
newentity = newfeed.entity.add()
newalert = newentity.alert
newentity.id = str(alert_id)
newtimerange = newalert.active_period.add()
newtimerange.end = alert_dict['active_period'][0]['end']
newtimerange.start = alert_dict['active_period'][0]['start']
for informed in alert_dict['informed_entity']:
newentityselector = newalert.informed_entity.add()
newentityselector.route_id = informed['route_id']
newentityselector.route_type = int(informed['route_type'])
newentityselector.stop_id = informed['stop_id']
print(alert_dict['description_text'])
newdescription = newalert.header_text
newdescription = alert_dict['description_text']
newalert.cause = alert_dict['cause']
newalert.effect = alert_dict['effect']
pb_feed = newfeed.SerializeToString()
with open("servicealerts.pb", 'wb') as fout:
fout.write(pb_feed)
The frustrating part is that we don't receive any sort of error message. Everything appears to run properly but the resulting pb file doesn't contain the new header_text or description_text items.
We are able to read the pb file using the following code:
feed = gtfs_realtime_pb2.FeedMessage()
response = open("servicealerts.pb")
feed.ParseFromString(response.read())
print(feed)
We truly appreciate any help that anyone can offer in pointing us in the right direction of figuring this out.

I was able to find the answer. This Python Notebook showed that by properly formatting the dictionary the PB could be generated with a few of lines of code.
from google.transit import gtfs_realtime_pb2
from google.protobuf.json_format import MessageToDict
newfeed = gtfs_realtime_pb2.FeedMessage()
ParseDict(alerts_dict, newfeed)
pb_feed = newfeed.SerializeToString()
with open("servicealerts.pb", 'wb') as fout:
fout.write(pb_feed)
All I had to do was format by dictionary properly.
if ALERT_GROUP_ID not in entity_dict.keys():
entity_dict[ALERT_GROUP_ID] = {"id": ALERT_GROUP_ID,
"alert":{
"active_period": [{
"start": int(START_TIME),
"end": int(END_TIME)
}],
"cause": cause_dict.get(CAUSE, ""),
"effect": effect_dict.get(EFFECT),
"url": {
"translation": [{
"text": URL,
"language": "en"
}]
},
"header_text": {
"translation": [{
"text": HEADER_TEXT,
"language": "en"
}]
},
"informed_entity": [{
'route_id': ROUTE_ID,
'route_type': ROUTE_TYPE,
'trip': TRIP,
'stop_id': STOP_ID
}],
"description_text": {
"translation": [{
"text": "Stop closed temporarily",
"language": "en"
}]
},
},
}
# print(entity_dict[ALERT_GROUP_ID]["alert"]['informed_entity'])
else:
entity_dict[ALERT_GROUP_ID]["alert"]['informed_entity'].append({
'route_id': ROUTE_ID,
'route_type': ROUTE_TYPE,
'trip': TRIP,
'stop_id': STOP_ID
})

Related

Updating Mongo DB collection field from object to array of objects

I had to change one of the fields of my collection in mongoDB from an object to array of objects containing a lot of data. New documents get inserted without any problem, but when attempted to get old data, it never maps to the original DTO correctly and runs into errors.
subject is the field that was changed in Students collection.
I was wondering is there any way to update all the records so they all have the same data type, without losing any data.
The old version of Student:
{
"_id": "5fb2ae251373a76ae58945df",
"isActive": true,
"details": {
"picture": "http://placehold.it/32x32",
"age": 17,
"eyeColor": "green",
"name": "Vasquez Sparks",
"gender": "male",
"email": "vasquezsparks#orbalix.com",
"phone": "+1 (962) 512-3196",
"address": "619 Emerald Street, Nutrioso, Georgia, 6576"
},
"subject":
{
"id": 0,
"name": "math",
"module": {
"name": "Advanced",
"semester": "second"
}
}
}
This needs to be updated to the new version like this:
{
"_id": "5fb2ae251373a76ae58945df",
"isActive": true,
"details": {
"picture": "http://placehold.it/32x32",
"age": 17,
"eyeColor": "green",
"name": "Vasquez Sparks",
"gender": "male",
"email": "vasquezsparks#orbalix.com",
"phone": "+1 (962) 512-3196",
"address": "619 Emerald Street, Nutrioso, Georgia, 6576"
},
"subject": [
{
"id": 0,
"name": "math",
"module": {
"name": "Advanced",
"semester": "second"
}
},
{
"id": 1,
"name": "history",
"module": {
"name": "Basic",
"semester": "first"
}
},
{
"id": 2,
"name": "English",
"module": {
"name": "Basic",
"semester": "second"
}
}
]
}
I understand there might be a way to rename old collection, create new and insert data based on old one in to new one. I was wondering for some direct way.
The goal is to turn subject into an array of 1 if it is not already an array, otherwise leave it alone. This will do the trick:
update args are (predicate, actions, options).
db.foo.update(
// Match only those docs where subject is an object (i.e. not turned into array):
{$expr: {$eq:[{$type:"$subject"},"object"]}},
// Actions: set subject to be an array containing $subject. You MUST use the pipeline version
// of the update actions to correctly substitute $subject in the expression!
[ {$set: {subject: ["$subject"] }} ],
// Do this for ALL matches, not just first:
{multi:true});
You can run this converter over and over because it will ignore converted docs.
If the goal is to convert and add some new subjects, preserving the first one, then we can set up the additional subjects and concatenate them into one array as follows:
var mmm = [ {id:8, name:"CORN"}, {id:9, name:"DOG"} ];
rc = db.foo.update({$expr: {$eq:[{$type:"$subject"},"object"]}},
[ {$set: {subject: {$concatArrays: [["$subject"], mmm]} }} ],
{multi:true});

Microsoft bot framework Probelms with WebChat and Messenger

I deployed my bot to azure ,and to messenger channel .
The bot works fine on emulator , while , video card do not work on web-chat and messenger .
Also , I created an adaptive form card so the user can send data , it works fine on emulator , but
web-chat and messenger it show up but when trying to submit it show code error .
Here is the code for my video card :
public static VideoCard GetVideoCard()
{
var videoCard = new VideoCard
{
Title = "Big Buck Bunny",
Subtitle = "by the Blender Institute",
Text = "Big Buck Bunny (code-named Peach) is a short computer-animated comedy film by the Blender Institute",
//Image = new ThumbnailUrl
//{
// Url = "https://upload.wikimedia.org/wikipedia/commons/thumb/c/c5/Big_buck_bunny_poster_big.jpg/220px-Big_buck_bunny_poster_big.jpg",
//},
Media = new List<MediaUrl>
{
new MediaUrl()
{
Url = "http://techslides.com/demos/sample-videos/small.mp4",
},
},
Buttons = new List<CardAction>
{
new CardAction()
{
Title = "Learn More",
Type = ActionTypes.OpenUrl,
Value = "https://peach.blender.org/",
},
},
};
return videoCard;
} .
the form card (u can see it by copy past this code to https://adaptivecards.io/designer/) :
{
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"type": "AdaptiveCard",
"version": "1.0",
"body": [
{
"type": "ColumnSet",
"columns": [
{
"type": "Column",
"width": 2,
"items": [
{
"type": "TextBlock",
"text": "Get In Touch",
"weight": "Bolder",
"size": "Medium"
},
{
"type": "TextBlock",
"text": "We are interested in advenced technology, we are interested in your business, we are interested in you.\n",
"isSubtle": true,
"wrap": true
},
{
"type": "TextBlock",
"text": "Don't worry, we'll never share or sell your information.",
"isSubtle": true,
"wrap": true,
"size": "Small"
},
{
"type": "TextBlock",
"text": "Your name",
"wrap": true
},
{
"type": "Input.Text",
"placeholder": "Full name",
"id": "name",
"spacing": "Medium"
},
{
"type": "TextBlock",
"text": "Your email",
"wrap": true
},
{
"type": "Input.Text",
"id": "email",
"placeholder": "youremail#example.com",
"style": "Email"
},
{
"type": "TextBlock",
"text": "Subject"
}
]
}
]
},
{
"type": "Input.Text",
"placeholder": "message",
"id": "message",
"isMultiline": true
}
],
"actions": [
{
"type": "Action.Submit",
"title": "Submit",
"id": "submitid"
}
]
}
For the Video Card , I have fixed it , the only one probelm is facebook is not rendering the card as it suppose to be :
sample on FB :
Sample on webchat

How to change HERE maps type in SAP Visual Business GeoMap?

According to this Map Provider Configuration Changes, I use this configuration to add HERE maps in GeoMap:
var oMapConfig = {
"MapProvider": [{
"name": "HEREMAPS",
"type": "HERETerrainMap",
"description": "",
"tileX": "256",
"tileY": "256",
"maxLOD": "20",
"copyright": "Tiles Courtesy of HERE Maps",
"Source": [{
"id": "s1",
"url": "https://1.base.maps.cit.api.here.com/maptile/2.1/maptile/newest/reduced.day/{LOD}/{X}/{Y}/256/png8?app_id=MY_ID&app_code=MY_CODE"
}, {
"id": "s2",
"url": "https://2.base.maps.cit.api.here.com/maptile/2.1/maptile/newest/reduced.day/{LOD}/{X}/{Y}/256/png8?app_id=MY_ID&app_code=MY_CODE"
}
]
}],
"MapLayerStacks": [{
"name": "DEFAULT",
"MapLayer": {
"name": "layer1",
"refMapProvider": "HEREMAPS",
"opacity": "1.0",
"colBkgnd": "RGB(255,255,255)"
}
}]
};
this.oMap.setMapConfiguration(oMapConfig);
this.oMap.setRefMapLayerStack("DEFAULT");
But my map is in black and white style:
What I want is standard map:
In Configuring HERE (formerly Nokia, NAVTEQ) maps, new server URL is provided, I've tried this, but not working.
{
"id": "s1",
"url": http://1.maps.nlp.nokia.com/maptile/2.1/maptile/newest/normal.day/{LOD}/{X}/{Y}/256/png?app_id=YOUR_APP_ID&app_code=YOUR_APP_CODE"
}, {
"id": "s2",
"url": "http://2.maps.nlp.nokia.com/maptile/2.1/maptile/newest/normal.day/{LOD}/{X}/{Y}/256/png?app_id=MY_APP_ID&app_code=MY_APP_CODE"
}
And failed to find MapProvider configuration documentation in setMapConfiguration of GeoMap
Just change reduced.day to normal.day in your map URL, and you'll get colored map:)
edit:
Please refer to https://developer.here.com/documentation/map-tile/topics/examples.html for detailed APIs

In MongoDb, with a Projection, remove array with one empty object [{}]

In MongoDb, with a Projection, I want to remove an array that contains one empty object [{}] resulting from a $unwind preserveNullAndEmptyArrays & group.
[{
"title": "Papaye",
"childrens": [{}],
"parents": [{
"title": "Arbres fruitiers",
"url": "/documents/plantes/arboriculture/arbres-fruitiers"
}
],
"url": "/documents/plantes/arboriculture/arbres-fruitiers/papaye"
},
{
"title": "Arbres fruitiers",
"childrens": [{
"title": "Tavelure",
"url": "/documents/maladies/tavelure"
},
{
"title": "Longane",
"url": "/documents/plantes/arboriculture/arbres-fruitiers/longane"
}],
"parents": [{
"title": "Arboriculture",
"url": "/documents/plantes/arboriculture"
}],
"url": "/documents/plantes/arboriculture/arbres-fruitiers"
}
Pipeline like :
var pipeline = [];
pipeline.push({$match:{url:/^\//}});
(...)
var proj = {};
proj.title = true;
proj.parents = true;
proj.url = true;
proj.parents = ???
proj.childrens = ???
pipeline.push({$project:proj});
db.getCollection('Pages').aggregate(pipeline)
Thanks in advance
Ok, I found the solution. Just test if url is null for the first array entry.
proj.childrens = {
"$cond", [
"$eq", [
"$arrayElemAt",["$childrens.url",0],
null
],
[],
"$childrens"
]
};

HTTP requests are tampered by other HTTP requests, with golang and standard library

cat main.go:
```
package main
import (
"encoding/json"
"log"
"net"
"net/http"
"net/http/fcgi"
"os"
)
func main() {
//setup the config
configFile := "config.json"
fd, err := os.Open(configFile)
if err != nil {
log.Fatalf("Can't open config file: %v", configFile)
}
CFG := config{}
err = json.NewDecoder(fd).Decode(&CFG)
if err != nil {
log.Fatalf("parse config error: %v", err)
}
//init DB connection
db.InitConnectionInfo(CFG.Database.Host, CFG.Database.Port, CFG.Database.Database, CFG.Database.Username, CFG.Database.Password)
//register HTTP handler
sessionHandler := &handlers.SessionHandler{}
http.Handle("/sessions", sessionHandler)
http.Handle("/sessions/", sessionHandler)
userHandler := &handlers.UserHandler{
Facebook: &oa.OAuth{AppId: CFG.Facebook.Key, Secret: CFG.Facebook.Secret},
Sina: &oa.OAuth{AppId: CFG.Sina.Key, Secret: CFG.Sina.Secret},
Google: &oa.OAuth{AppId: CFG.Google.Key, Secret: CFG.Google.Secret},
Tencent: &oa.OAuth{AppId: CFG.Tencent.Key, Secret: CFG.Tencent.Secret},
Mixpanel: &hu.Share{Token: CFG.Mixpanel.Token},
FacebookShare: &hu.Share{Token: CFG.Facebook.Token},
SinaShare: &hu.Share{Token: CFG.Sina.Token},
GoogleShare: &hu.Share{Token: CFG.Google.Token},
TencentShare: &hu.Share{Token: CFG.Tencent.Token},
}
http.Handle("/users", userHandler)
http.Handle("/users/", userHandler)
//and so on ...
//run server
log.Println("start listen: ", CFG.FcgiAddr)
l, _ := net.Listen("tcp", CFG.FcgiAddr)
log.Fatalf("server error is %v", fcgi.Serve(l, nil))
//##select {}
log.Println("end listen")
}
```
build it and deploy behind nginx.
then client query /users/1234567/places, /users/1234567, and so on...
get the response is {blank data}, {normal user(1234567) data} or {normal user(1234567) data}, {normal user(1234567) data}.
same prefix of query, and same handler, looks like it is overload the response by subsequent HTTP request.
How can I do it?
May need me to give an example of a response:
correct response is :
{
"meta": {
"code": 200,
"text": "OK"
},
"data": {
"count": 21,
"place-tag-maps": [{
"id": "95842310160384",
"place-id": "95551731663150",
"tag-id": "95551579750669",
"ct": "2014-07-01T09:07:28Z"
}, {
"id": "95842310160385",
"place-id": "95551731663150",
"tag-id": "95551579750694",
"ct": "2015-01-15T17:41:23Z"
}, {
"id": "96262389694470",
"place-id": "95551731663150",
"tag-id": "95910120456455",
"ct": "2016-07-18T13:11:39Z"
}, ...],
"places": [{
"id": "95551731663150",
"name": "Kam Fung Restaurant",
"address": "G/F, 41 Spring Garden Ln",
"coordinate": {
"latitude": 22.275576,
"longitude": 114.172582
},
"telephone": "+852 2572 0526",
"city-id": "95530516807703",
"city": "Hong Kong",
"country": "Hong Kong",
"type": "4sq",
"ref-id": "4b1613f8f964a520cdb623e3",
"ct": "2016-02-23T07:42:43.565489Z",
"mt": "2017-02-22T09:35:48.302929Z",
"rating": 7.5,
"stats": {
"foursquare": {
"count": 111,
"value": 7.5
},
"spottly": {
"save-count": 12
}
},
"permanent-close": false,
"price": ""
}, ...],
"posts": [{
"collection-id": "95551746474003",
"coordinate": {
"latitude": 22.275576,
"longitude": 114.172582
},
"ct": "2017-02-22T09:35:47Z",
"facebook-tag-users": [],
"id": "97501586849795",
"medias": [],
"message": "",
"mt": "2017-02-22T09:35:47Z",
"owner-id": "96527264645120",
"place-id": "95551731663150",
"share-to": [],
"star": 5,
"status": "Done",
"tags": []
}, ...],
"users": [{
"id": "95551581323446",
"uid": "hk_epicurus",
"uid-ignore-case": "hk_epicurus",
"name": "Hk Epicurus",
"head": "https://d278wa0j9nq2mp.cloudfront.net/uploader/54aa335ddf4e63450002919a.jpeg",
"site": "www.hkepicurus.com",
"location": "Hong Kong",
"description": "Hong Kong Food \u0026 Travel Bear.\nGrew up in Aust, Malaysia, Tokyo \u0026 HK. \nInstagram: EpicurusHongKong\nFacebook, Spottly \u0026 Twitter: HK Epicurus \n微博: 香港美食-伊比\nFacebook Fans Page: http://on.fb.me/1qDyiIk",
"ct": "2013-10-13T19:12:41Z",
"mt": "2015-11-08T15:22:45Z"
}, ...]
}
}
and
{
"meta": {
"code": 200,
"text": "OK"
},
"data": {
"friends": [{
"id": "97331335725056",
"from-id": "97273770803200",
"to-id": "96527264645120",
"ct": "2017-01-23T07:58:41Z"
}],
"user": {
"ct": "2013-04-30T23:30:05Z",
"description": "Founder and Chief Everything Officer of Spottly. Loves to eat. Have a really bad memory. Wants to remember the best places and make travel research better",
"followers": {
"count": 41291
},
"followings": {
"count": 322
},
"head": "https://d278wa0j9nq2mp.cloudfront.net/uploader/525ffac8df4e6347870145ef.jpeg",
"id": "96527264645120",
"location": "Vancouver | Hong Kong | Beijing ",
"mt": "2015-10-27T09:47:12Z",
"name": "Edwyn Chan",
"site": "http://spottly.com/edwyn",
"uid": "edwyn",
"uid-ignore-case": "edwyn"
}
}
}
but mistake response is
{
"meta": {
"code": 200,
"text": "OK"
}
}
and
{
"meta": {
"code": 200,
"text": "OK"
},
"data": {
"friends": [{
"id": "97331335725056",
"from-id": "97273770803200",
"to-id": "96527264645120",
"ct": "2017-01-23T07:58:41Z"
}],
"user": {
"ct": "2013-04-30T23:30:05Z",
"description": "Founder and Chief Everything Officer of Spottly. Loves to eat. Have a really bad memory. Wants to remember the best places and make travel research better",
"followers": {
"count": 41291
},
"followings": {
"count": 322
},
"head": "https://d278wa0j9nq2mp.cloudfront.net/uploader/525ffac8df4e6347870145ef.jpeg",
"id": "96527264645120",
"location": "Vancouver | Hong Kong | Beijing ",
"mt": "2015-10-27T09:47:12Z",
"name": "Edwyn Chan",
"site": "http://spottly.com/edwyn",
"uid": "edwyn",
"uid-ignore-case": "edwyn"
}
}
}
the first response is error. or it's same as the second response.
sequence request is correct response, parallel request is incorrect response.
problem resolved。
the basic reason is this case:
var x = &{...} //init value
fillX(..., x) //fill the fields of x pointer
//here the fields of x pointer is not same to inner of fillX func
so, change to
var x = &{...}
x = fullX(..., x)
the problem resolved.
but why go pointer is the behavior?
Finally, this problem resolved.
Because the http.Handle register path-pattern and handler pair, the handler is construct by register time only-once, not for any request of path-pattern.
if record info within handler, It will been change by after request.