Building a simple Overpass Turbo search query for amenity - openstreetmap

I've read though the documentation but still a little confused - I would like to build a simple query for example all fuel stations within an area and just output to a csv the name, brand and lat / long only.
Eg I would like something like this:
XYZ Service Station, Shell, -33.8042874, 18.872144
SO far I am using this but not really getting the output I would like.
[out:csv(::id,name,::lat,::lon; false; ",")]
[bbox:{{bbox}}];
nwr[amenity=fuel];
out center;

Related

How can I obtain all of the street names under a polygon in Overpass?

I am new to overpass (after only discovering it last night). I have a polygon I drew on QGIS and I plan to obtain its coordinates (long, lat). I'd then like to use these coordinates in overpass to obtain all of the road names in that area. I found a query online that obtains all road names in a city:
[out:csv ("name")][timeout:2500];
{{geocodeArea:Roma}}->.searchArea;
(
way["highway"]["name"](area.searchArea);
);
for (t["name"])
{
make street name=_.val;
out;
}
How can I adjust the following query so that I can specify a polygon function instead of city/area name?. I'm mindful of the syntax:
(poly:"latitude_1 longitude_1 latitude_2 longitude_2 latitude_3 longitude_3 …");
I'm just not sure where it would go in the query. I tried a few times but I was receiving errors or just blank results. Hopefully if I see an example I should be able to carry out my task effectively.

After doing some research I found the answer. This would give me a list of road names:
[out:csv ("name")][timeout:2500];
way["highway"]["name"]
(poly:"51.5566 0.0763 51.5734 0.0724 51.5203 0.0293");
out tags;
And this would give me a map view:
way["highway"]["name"]
(poly:"51.5566 0.0763 51.5734 0.0724 51.5203 0.0293");
out geom;

Grafana dashboard to display a metric for a key in JSON Loki record

I'm having trouble understanding how to create a dashboard time series plot to display a single key/value from a Loki log which is in JSON format.
eg:
here is my query in the Explorer:
{job="railsdevlogs"}|json
which returns log lines such as:
{"date":"2022-01-05T21:27:21.895Z","pool":{"Pool Size":50,"Current":5,"Active":1,"Idle":4,"Dead":0,"Timeout":"5 sec"},"puma":{"Started At":"2022-01-05T20:35:26Z","Max Threads":16,"Pool Capacity":16,"Running":1,"Backlog":0,"IO Handles":15,"File Handles":2,"Socket Handles":4,"Server Log Size":46750072},"process":[{"Name":"ruby.exe","Process ID":656,"Threads":11,"Working Set":150728704,"Virtual Size":288079872},{"Name":"mysqld.exe","Process ID":4836,"Threads":3,"Working Set":360448,"Virtual Size":4445065216},{"Name":"mysqld.exe","Process ID":5808,"Threads":49,"Working Set":69906432,"Virtual Size":4924059648},{"Name":"aaaaa.exe","Process ID":14460,"Threads":18,"Working Set":49565696,"Virtual Size":5478469632},{"Name":"bbbbb.exe","Process ID":9584,"Threads":14,"Working Set":35012608,"Virtual Size":4496551936},{"Name":"ccccc.exe","Process ID":11944,"Threads":14,"Working Set":29609984,"Virtual Size":4481880064}],"gc":{"count":242,"heap_allocated_pages":1277,"heap_sorted_length":1279,"heap_allocatable_pages":9,"heap_available_slots":869213,"heap_live_slots":464541,"heap_free_slots":404672,"heap_final_slots":0,"heap_marked_slots":411311,"heap_swept_slots":457903,"heap_eden_pages":1268,"heap_tomb_pages":9,"total_allocated_pages":1278,"total_freed_pages":1,"total_allocated_objects":74364715,"total_freed_objects":73900174,"malloc_increase_bytes":640096,"malloc_increase_bytes_limit":16777216,"minor_gc_count":131,"major_gc_count":111,"remembered_wb_unprotected_objects":57031,"remembered_wb_unprotected_objects_limit":114062,"old_objects":349257,"old_objects_limit":698512,"oldmalloc_increase_bytes":640288,"oldmalloc_increase_bytes_limit":16777216},"os":{"System Name":"xxxxx","Description":"","Organization":"","Operating System":"Microsoft Windows 10 Enterprise LTSC","OS Version":"10.0.17763","OS Serial Number":"xxxxx-xxxxx-xxxxx-xxxxx","System Time":"2022-01-05T16:27:22.000-05:00","System Time Zone":-300,"Last Boot Time":"2021-12-15T23:26:38.000-05:00","System Drive":"C:","Total Physical Memory":34204393472,"Free Physical Memory":20056260608,"Total Virtual Memory":39304667136,"Free Virtual Memory":13915041792,"Number of Processes":307,"Number of Users":2,"volumes":[{"Drive":"C:\\","Type":"NTFS","Total Space":1023563264000,"Free Space":681182343168,"Block Size":4096}]},"symbol":{"size":28106},"stats_collection_time":387}
using |json will automatically create dynamic labels for all the key/values in the json log line:
gc_count = 123
os_Free_Virtual_Memory = 456789
etc.
Now I would like to plot one of these values in a grafana time series plot, but I am struggling to understand how to isolate one dynamic label and plot it.
Perhaps I'm using |json incorrectly. The documentation and examples I have read so far shows how to filter the logs using the dynamic labels, but I dont need that since I want to plot every log line.
thanks

I think this should help https://grafana.com/go/observabilitycon/2020/keynote-what-is-observability/ if you go to minute 41.
There's an example which is very similar to what you're trying to achieve.
Your query should look something like:
quantile_over_time(0,99, {job="railsdevlogs"}
| json
| unwrap gc_count [1m]}
by (job)

osmnx boundaries and admin_level

I hope someone here can help me to retrieve the correct administration level(s) from OSM. I am using the following code, but admin_level seems to be ignored:
tags = {"boundary":"administrative","admin_level":"4" }
gdf =ox.geometries.geometries_from_bbox(51.5, 51.0, 11.7, 11.2, tags)
gdf.shape
The bounding box seems to be used as a polygon to create an intersection with all the boundaries in the OSM database, the first tag is working because only administrative boundaries are returned, but the filter on level is ignored (gdf["admin_level"].head() shows level 6).
I would like to understand what I am doing wrong, and how I can use this package better; it seems like a very useful library.
Thanks,
Gijs
Result using the bounding box:

OK, rereading the documentation again made me realize that osmnx is using [OR] statements and not [AND] statements as I was presuming; removing the boundary request from the query is indeed giving only admin_level:4 results.
tags (dict) – Dict of tags used for finding objects in the selected area. Results returned are the union, not the intersection of each individual tag.
Some additional code: https://i.stack.imgur.com/Fw840.png

Twitter advanced search by date AND location for research purposes

I'm trying to research a topic and I need to get all tweets within 2013 and 2015 and a specific location for two keywords.
I tried to get the results via Advanced Search but I allways get no results.
I tried:
cannabis near:"España" within:15mi since:2013-10-07 until:2015-01-01
cannabis near:"Spain" within:15mi since:2013-10-07 until:2015-01-01
Basically, I have a database of scraped press articles sorted by date from a bunch of sources, and I want to know how the agenda of this news sources have an impact on the social media conversation.
I could do it over Reddit if it was the case for the US, but there's no Spanish alternative (well, we have Meneame, but the user base is very left-leaning and I think it will be very narrow).
So I wanted to either scrape the search results or get them via API, but It's not working, and AFAIK I can't do anything similar with Facebook.

One way to achieve this is by using Twitter's geocode operator. In the example below i took Madrid as a center and covered a radius of 600km around it like this:
(canabis OR cannabis) geocode:40.4381311,-3.8196196,600km since:2013-10-07 until:2015-12-31
Try it...
The syntax is as follows:
([your_boolean_search_query]) geocode:[latitude],[longitude],[radius]km since:[] until:[]
one easy way to find latitudes and longitudes of locations is to use Google Maps. Simply navigate to a place using the search box and then copy the latitude and longitude element from the URL line in the browser. Here it is for Madrid. The latitude and longitude are right after the # sign, separated by a comma:
https://www.google.com/maps/place/Madrid,+Spain/#40.4381311,-3.8196196,54451m/data=!3m2!1e3!4b1!4m5!3m4!1s0xd422997800a3c81:0xc436dec1618c2269!8m2!3d40.4167754!4d-3.7037902?hl=en
Try it...

importing website to google sheets

I have tried searching everywhere online for a good answer but cannot seem to find anything that matches specifically what i am looking for.
When i use the IMPORTHTML function in google sheets, i end up with data that looks like:
${player.name} (${player.position}, ${team.abbrev}) ${opponent.abbrev} #${opponent_rank} ${minutes} ${pts} ${fgm}-${fga} ${ftm}-${fta} ${p3m}-${p3a} ${treb} ${ast} ${stl} ${blk} ${tov} ${pf} ${fp} $${salary} ${ratio}
the code that i am using looks like this:
=IMPORTHTML("", "table",2)
When I use the same as above (=IMPORTHTML("", "table",2)) only with "0" as my index, it pulls this:
Opp Stats
Player Team Rank Min Pts FGM/A FTM/A 3PM/A Reb Ast Stl Blk Tov Foul FP Cost Value
Basically, I am attempting to pull the table data from this website:
https://www.numberfire.com/nba/fantasy/fantasy-basketball-projections
(because of my rep i cannot post more than two links, however my IMPORTHTML function has the above link input in both functions)
into a google sheet. Please help. any feedback is much appreciated... thanks!

Best advice is to find another Web table you can import. If you do "view source" on the page, you will find that the table content is dynamically populated from a variable named NF_DATA.
You need to create a document script to extract the data you want:
function this_is_test() {
var response = UrlFetchApp.fetch("https://www.numberfire.com/nba/fantasy/fantasy-basketball-projections");
raw_content = response.getContentText();
re = new RegExp('"daily_projections":\\[[^\\]]+','i');
proj = raw_content.match(re);
Logger.log(proj);
}
It will extract all text in-between "daily_projections":[ and ], which is (as of today):
"daily_projections":[{"nba_player_id":"77","nba_game_id":"20015","date":"2016-01-19","nba_team_id":"21","opponent_id":"7","season":"2016","game_play_probability":"1.00","game_start":"1.00","minutes":36.3,"fgm":"8.8","fga":"17.1","p3m":"1.9","p3a":"4.8","ftm":"6.2","fta":"6.9","oreb":"0.8","dreb":"7.2","ast":"4.7","stl":"1.1","blk":"1.2","tov":"2.7","pf":"1.8","pts":"25.3","ts":"0.628","efg":"0.655","oreb_pct":"2.6","dreb_pct":"21.4","treb_pct":"12.4","ast_pct":"23.4","stl_pct":"1.5","blk_pct":"2.4","tov_pct":"12.1","usg":"27.8","ortg":"122.2","drtg":"101.8","nerd":"22.34","star_street_fp":43.08,"star_street_salary":0,"star_street_ratio":0,"draft_street_daily_fp":39.75,"draft_street_daily_salary":0,"draft_street_daily_ratio":0,"fanduel_fp":43.85,"fanduel_salary":9900,"fanduel_ratio":4.43,"draft_kings_fp":46.55,"draft_kings_salary":9900,"draft_kings_ratio":4.7,"fantasy_feud_fp":39.75,"fantasy_feud_salary":153600,"fantasy_feud_ratio":0.26,"fanthrowdown_fp":45.2,"fanthrowdown_salary":0,"fanthrowdown_ratio":0,"fantasy_aces_fp":44.25,"fantasy_aces_salary":7250,"fantasy_aces_ratio":6.1,"draftday_fp":45.25,"draftday_salary":18800,"draftday_ratio":2.41,"fantasy_score_fp":45.6,"fantasy_score_salary":9600,"fantasy_score_ratio":4.75,"draftster_fp":43.75,"draftster_salary":9400,"draftster_ratio":4.65,"yahoo_fp":44.8,"yahoo_salary":52,"yahoo_ratio":0.86,"treb":8},{"nba_player_id":"397","nba_game_id":"20015","date":"2016-01-19","nba_team_id":"21","opponent_id":"7","season":"2016","game_play_probability":"1.00","game_start":"1.00","minutes":35,"fgm":"8.6","fga":"18.0","p3m":"1.3","p3a":"4.1","ftm":"5.9","fta":"7.2","oreb":"1.3","dreb":"5.0","ast":"8.8","stl":"2.0","blk":"0.4","tov":"3.6","pf":"2.2","pts":"24.4","ts":"0.576","efg":"0.592","oreb_pct":"4.6","dreb_pct":"15.3","treb_pct":"10.2","ast_pct":"44.4","stl_pct":"3.0","blk_pct":"0.8","tov_pct":"14.6","usg":"31.3","ortg":"117.8","drtg":"101.2","nerd":"19.75","star_street_fp":44.48,"star_street_salary":0,"star_street_ratio":0,"draft_street_daily_fp":41.33,"draft_street_daily_salary":0,"draft_street_daily_ratio":0,"fanduel_fp":46.36,"fanduel_salary":10500,"fanduel_ratio":4.42,"draft_kings_fp":49.13,"draft_kings_salary":10700,"draft_kings_ratio":4.59,"fantasy_feud_fp":41.33,"fantasy_feud_salary":169800,"fantasy_feud_ratio":0.24,"fanthrowdown_fp":47.33,"fanthrowdown_salary":0,"fanthrowdown_ratio":0,"fantasy_aces_fp":46.68,"fantasy_aces_salary":7800,"fantasy_aces_ratio":5.98,"draftday_fp":47.1,"draftday_salary":20500,"draftday_ratio":2.3,"fantasy_score_fp":48.48,"fantasy_score_salary":9900,"fantasy_score_ratio":4.9,"draftster_fp":45.38,"draftster_salary":9500,"draftster_ratio":4.78,"yahoo_fp":47.01,"yahoo_salary":59,"yahoo_ratio":0.8,"treb":6.3},{"nba_player_id":"279","nba_game_id":"20016","date":"2016-01-19","nba_team_id":"11","opponent_id":"24","season":"2016","game_play_probability":"1.00","game_start":"1.00","minutes":36.7,"fgm":"7.6","fga":"18.1","p3m":"2.5","p3a":"6.9","ftm":"5.5","fta":"6.5","oreb":"1.1","dreb":"5.8","ast":"5.3","stl":"1.8","blk":"0.4","tov":"3.6","pf":"2.4","pts":"22.5","ts":"0.537","efg":"0.610","oreb_pct":"3.3","dreb_pct":"17.6","treb_pct":"10.5","ast_pct":"25.1","stl_pct":"2.5","blk_pct":"0.9","tov_pct":"15.3","usg":"29.1","ortg":"104.1","drtg":"99.2","nerd":"5.26","star_street_fp":38.55,"star_street_salary":0,"star_street_ratio":0,"draft_street_daily_fp":34.13,"draft_street_daily_salary":0,"draft_street_daily_ratio":0,"fanduel_fp":39.53,"fanduel_salary":8700,"fanduel_ratio":4.54,"draft_kings_fp":42.93,"draft_kings_salary":9200,"draft_kings_ratio":4.67,"fantasy_feud_fp":34.13,"fantasy_feud_salary":138800,"fantasy_feud_ratio":0.25,"fanthrowdown_fp":41.13,"fanthrowdown_salary":0,"fanthrowdown_ratio":0,"fantasy_aces_fp":39.88,"fantasy_aces_salary":6500,"fantasy_aces_ratio":6.14,"draftday_fp":41.3,"draftday_salary":16600,"draftday_ratio":2.49,"fantasy_score_fp":41.68,"fantasy_score_salary":8400,"fantasy_score_ratio":4.96,"draftster_fp":39.45,"draftster_salary":8000,"draftster_ratio":4.93,"yahoo_fp":40.78,"yahoo_salary":47,"yahoo_ratio":0.87,"treb":6.9},{"nba_player_id":"2137","nba_game_id":"20014","date":"2016-01-19","nba_team_id":"38","opponent_id":"17","season":"2016","game_play_probability":"1.00","game_start":"1.00","minutes":35,"fgm":"8.0","fga":"16.6","p3m":"0.4","p3a":"1.3","ftm":"4.5","fta":"6.0","oreb":"2.6","dreb":"7.8","ast":"2.2","stl":"1.0","blk":"2.2","tov":"1.9","pf":"2.6","pts":"20.8","ts":"0.541","efg":"0.521","oreb_pct":"8.6","dreb_pct":"24.8","treb_pct":"16.6","ast_pct":"11.5","stl_pct":"1.4","blk_pct":"4.9","tov_pct":"9.4","usg":"27.0","ortg":"107.9","drtg":"103.1","nerd":"5.60","star_street_fp":41.05,"star_street_salary":0,"star_street_ratio":0,"draft_street_daily_fp":36.55,"draft_street_daily_salary":0,"draft_street_daily_ratio":0,"fanduel_fp":41.08,"fanduel_salary":10300,"fanduel_ratio":3.99,"draft_kings_fp":44.25,"draft_kings_salary":10000,"draft_kings_ratio":4.43,"fantasy_feud_fp":36.55,"fantasy_feud_salary":149400,"fantasy_feud_ratio":0.24,"fanthrowdown_fp":41.8,"fanthrowdown_salary":0,"fanthrowdown_ratio":0,"fantasy_aces_fp":41.6,"fantasy_aces_salary":7400,"fantasy_aces_ratio":5.62,"draftday_fp":40.43,"draftday_salary":18200,"draftday_ratio":2.22,"fantasy_score_fp":42.55,"fantasy_score_salary":9700,"fantasy_score_ratio":4.39,"draftster_fp":41.53,"draftster_salary":9200,"draftster_ratio":4.51,"yahoo_fp":41.28,"yahoo_salary":54,"yahoo_ratio":0.76,"treb":10.4},{"nba_player_id":"362","nba_game_id":"20013","date":"2016-01-19","nba_team_id":"15","opponent_id":"16","season":"2016","game_play_probability":"1.00","game_start":"1.00","minutes":34.9,"fgm":"7.3","fga":"15.4","p3m":"1.4","p3a":"3.7","ftm":"4.8","fta":"6.0","oreb":"1.7","dreb":"6.1","ast":"2.3","stl":"0.7","blk":"1.0","tov":"1.7","pf":"2.0","pts":"20.6","ts":"0.571","efg":"0.594","oreb_pct":"6.2","dreb_pct":"19.9","treb_pct":"13.3","ast_pct":"12.1","stl_pct":"1.1","blk_pct":"2.2","tov_pct":"8.5","usg":"26.5","ortg":"115.3","drtg":"104.5","nerd":"11.63","star_street_fp":34.93,"star_street_salary":0,"star_street_ratio":0,"draft_street_daily_fp":30.85,"draft_street_daily_salary":0,"draft_street_daily_ratio":0,"fanduel_fp":35.11,"fanduel_salary":7800,"fanduel_ratio":4.5,"draft_kings_fp":37.05,"draft_kings_salary":7600,"draft_kings_ratio":4.88,"fantasy_feud_fp":30.85,"fantasy_feud_salary":120400,"fantasy_feud_ratio":0.26,"fanthrowdown_fp":36.2,"fanthrowdown_salary":0,"fanthrowdown_ratio":0,"fantasy_aces_fp":35.5,"fantasy_aces_salary":5900,"fantasy_aces_ratio":6.02,"draftday_fp":35.43,"draftday_salary":13950,"draftday_ratio":2.54,"fantasy_score_fp":36.35,"fantasy_score_salary":7000,"fantasy_score_ratio":5.19,"draftster_fp":35.35,"draftster_salary":7200,"draftster_ratio":4.91,"yahoo_fp":35.81,"yahoo_salary":40,"yahoo_ratio":0.9,"treb":7.8},{"nba_player_id":"2249","nba_game_id":"20014","date":"2016-01-19","nba_team_id":"17","opponent_id":"38","season":"2016","game_play_probability":"1.00","game_start":"1.00","minutes":35.7,"fgm":"7.2","fga":"16.6","p3m":"0.9","p3a":"3.2","ftm":"4.6","fta":"6.3","oreb":"1.3","dreb":"2.9","ast":"2.1","stl":"0.9","blk":"0.5","tov":"2.2","pf":"2.4","pts":"20.2","ts":"0.521","efg":"0.530","oreb_pct":"4.3","dreb_pct":"9.6","treb_pct":"6.9","ast_pct":"10.2","stl_pct":"1.3","blk_pct":"1.0","tov_pct":"10.3","usg":"26.7","ortg":"101.6","drtg":"111.5","nerd":"-7.07","star_street_fp":28.68,"star_street_salary":0,"star_street_ratio":0,"draft_street_daily_fp":23.65,"draft_street_daily_salary":0,"draft_street_daily_ratio":0,"fanduel_fp":28.99,"fanduel_salary":6700,"fanduel_ratio":4.33,"draft_kings_fp":30.75,"draft_kings_salary":6900,"draft_kings_ratio":4.46,"fantasy_feud_fp":23.65,"fantasy_feud_salary":113500,"fantasy_feud_ratio":0.21,"fanthrowdown_fp":29.65,"fanthrowdown_salary":0,"fanthrowdown_ratio":0,"fantasy_aces_fp":29.2,"fantasy_aces_salary":4900,"fantasy_aces_ratio":5.96,"draftday_fp":28.43,"draftday_salary":12000,"draftday_ratio":2.37,"fantasy_score_fp":30.3,"fantasy_score_salary":6000,"fantasy_score_ratio":5.05,"draftster_fp":29.23,"draftster_salary":5900,"draftster_ratio":4.95,"yahoo_fp":29.44,"yahoo_salary":29,"yahoo_ratio":1.02,"treb":4.2},{"nba_player_id":"370","nba_game_id":
Note that even this is not complete. You need to somehow map nba_player_id to the appropriate name. Anyway, a lot coding will be involved...