import osmnx as ox
ox.__version__ # '0.13.0'
I would like to show the subway in Hannover as known in the German subway OSM data on a map using the great OSMNX module. But unlike the New York example no results are returned for:
G = ox.graph_from_place('Hannover, Germany',
retain_all=False, truncate_by_edge=True, simplify=True,
network_type='none', custom_filter='["railway"~"subway"]')
# EmptyOverpassResponse: There are no data elements in the response JSON
I do get results for other similar queries using 'Hannover, Germany' as region. I also do not get subway results for Paris or London. And I do not get results for similar queries like custom_filter='["railway"~"tram"]' or '["railway"~"s-bahn"]' or '["network"~"metro"]'.
Also, if I use the infrastructure keyword argument to select "railway", an extensive gdf is returned:
G = ox.graph_from_place('Hannover, Germany', retain_all=False, truncate_by_edge=True, simplify=True,
network_type='none', infrastructure='way["railway"]')
gdfox = ox.graph_to_gdfs(G, nodes=False, edges=True, node_geometry=True, fill_edge_geometry=True)
gdfox.shape # (4422, 14)
But I cannot identify the subway using the columns returned?:
['u', 'v', 'key', 'osmid', 'service', 'oneway', 'length',
'geometry', 'name', 'maxspeed', 'ref', 'bridge', 'tunnel',
'access']
What I also find strange is that there are only 2 LINESTRINGS returned if I (try to) retrieve all railways using the custom_filter:
G = ox.graph_from_place('Hannover, Germany', retain_all=False, truncate_by_edge=True,
simplify=True, network_type=None, custom_filter='["railway"~""]')
gdfox = ox.graph_to_gdfs(G, nodes=False, edges=True, node_geometry=True, fill_edge_geometry=True)
gdfox.shape # (2, 10) # returns only 2 LINESTRINGS: Altenbekener Damm
I am in the process of removing the infrastructure parameter in favor of a more consistent custom_filter parameter. Will be done in a couple days: https://github.com/gboeing/osmnx/pull/477 (EDIT: done and released in v0.14.0; code snippet below edited accordingly.)
In the meantime, I am not familiar with Hannover but it appears that its passenger rail system is tagged as "tram" and "rail" rather than "subway". Something like this seems to capture it:
import osmnx as ox
ox.config(use_cache=False,
log_console=True,
useful_tags_way=ox.settings.useful_tags_way + ['railway'])
G = ox.graph_from_place('Hannover, Germany',
retain_all=False, truncate_by_edge=True, simplify=True,
custom_filter='["railway"~"tram|rail"]')
len(G) #1776
Related
Let's say I have a directed graph (this is about relational tables). I want to find M:N tables to track relationships enabled thru M:N tables.
from pathlib import Path
import subprocess
import networkx as nx
def write_svg(g, name):
temp = "temp.dot"
suffix = "jpg"
nx.nx_agraph.write_dot(g, temp)
pa_img = Path(f"{name}.{suffix}")
li_cmd = f"/opt/local/bin/dot {temp} -T {suffix} -o {pa_img}".split()
subprocess.check_output(li_cmd)
G = nx.DiGraph()
G.add_edge("C1", "P1")
G.add_edge("C2", "P1")
G.add_edge("C21", "C2")
G.add_edge("MN12", "P1")
G.add_edge("MN12", "P2")
G.add_nodes_from([
("MN12", {"color" : "red"})
])
Run this, and I get:
So what I am considering here is that MN12 has as parents P1 and P2. So I want to consider P2 to be related to P1, with MN12 as the mapping table.
In other words, if I hard-code the relationship graph I want:
G = nx.DiGraph()
G.add_edge("C1", "P1")
G.add_edge("C2", "P1")
G.add_edge("C21", "C2")
G.add_edge("P2(using MN12)", "P1")
G.add_nodes_from([
("P2(using MN12)", {"color" : "green"})
])
Note that C21 remains a child of C2. Only MN12 is modified, because it has 2 parents.
Now, I know I can see the degree of a given node.
Going in back to my input graph:
(Pdb++) G.degree('MN12')
2
(Pdb++) G.degree('C1')
1
(Pdb++) G.degree('C2')
2
(Pdb++) G.degree('P1')
3
But how do I see that the arrows from MN12 go towards both P1 and P2? Is this even a question for networkx?
You can use both out_edges and successors.
>>> G.out_edges("MN12")
OutEdgeDataView([('MN12', 'P1'), ('MN12', 'P2')])
>>> list(G.successors("MN12"))
['P1', 'P2']
This code is specified to visualize the CALIPSO satellite atmospheric profiles
The input files are .HDF
The code is copyrighted to the HDF group.
In the begining, I struggled with installing the basemap,
finally I installed it using .whl file on my windows10.
Now, this error is reached when I run the script:
SystemError:
execution of module _geoslib raised unreported exception.
I have looked a lot in google, but nothing done.
Can you please help me?
Cheers
"Copyright (C) 2014-2019 The HDF Group
Copyright (C) 2014 John Evans
This example code illustrates how to access and visualize a LaRC CALIPSO file
in file in Python.
If you have any questions, suggestions, or comments on this example, please use
the HDF-EOS Forum (http://hdfeos.org/forums). If you would like to see an
example of any other NASA HDF/HDF-EOS data product that is not listed in the
HDF-EOS Comprehensive Examples page (http://hdfeos.org/zoo), feel free to
contact us at eoshelp#hdfgroup.org or post it at the HDF-EOS Forum
(http://hdfeos.org/forums).
Usage: save this script and run
$python CAL_LID_L2_VFM-ValStage1-V3-02.2011-12-31T23-18-11ZD.hdf.py
The HDF file must either be in your current working directory
or in a directory specified by the environment variable HDFEOS_ZOO_DIR.
Tested under: Python 2.7.15::Anaconda custom (64-bit)
Last updated: 2019-01-25
"""
import os
import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.basemap import Basemap
from matplotlib import colors
USE_NETCDF4 = False
def run(FILE_NAME):
# Identify the data field.
DATAFIELD_NAME = 'Feature_Classification_Flags'
if USE_NETCDF4:
from netCDF4 import Dataset
nc = Dataset(FILE_NAME)
# Subset the data to match the size of the swath geolocation fields.
# Turn off autoscaling, we'll handle that ourselves due to presence of
# a valid range.
var = nc.variables[DATAFIELD_NAME]
data = var[:,1256]
# Read geolocation datasets.
lat = nc.variables['Latitude'][:]
lon = nc.variables['Longitude'][:]
else:
from pyhdf.SD import SD, SDC
hdf = SD(FILE_NAME, SDC.READ)
# Read dataset.
data2D = hdf.select(DATAFIELD_NAME)
data = data2D[:,1256]
# Read geolocation datasets.
latitude = hdf.select('Latitude')
lat = latitude[:]
longitude = hdf.select('Longitude')
lon = longitude[:]
# Subset data. Otherwise, all points look black.
lat = lat[::10]
lon = lon[::10]
data = data[::10]
# Extract Feature Type only through bitmask.
data = data & 7
# Make a color map of fixed colors.
cmap = colors.ListedColormap(['black', 'blue', 'yellow', 'green', 'red', 'purple', 'gray', 'white'])
# The data is global, so render in a global projection.
m = Basemap(projection='cyl', resolution='l',
llcrnrlat=-90, urcrnrlat=90,
llcrnrlon=-180, urcrnrlon=180)
m.drawcoastlines(linewidth=0.5)
m.drawparallels(np.arange(-90.,90,45))
m.drawmeridians(np.arange(-180.,180,45), labels=[True,False,False,True])
x,y = m(lon, lat)
i = 0
for feature in data:
m.plot(x[i], y[i], 'o', color=cmap(feature), markersize=3)
i = i+1
long_name = 'Feature Type at Altitude = 2500m'
basename = os.path.basename(FILE_NAME)
plt.title('{0}\n{1}'.format(basename, long_name))
fig = plt.gcf()
# define the bins and normalize
bounds = np.linspace(0,8,9)
norm = mpl.colors.BoundaryNorm(bounds, cmap.N)
# create a second axes for the colorbar
ax2 = fig.add_axes([0.93, 0.2, 0.01, 0.6])
cb = mpl.colorbar.ColorbarBase(ax2, cmap=cmap, norm=norm, spacing='proportional', ticks=bounds, boundaries=bounds, format='%1i')
cb.ax.set_yticklabels(['invalid', 'clear', 'cloud', 'aerosol', 'strato', 'surface', 'subsurf', 'no signal'], fontsize=5)
# plt.show()
pngfile = "{0}.py.png".format(basename)
fig.savefig(pngfile)
if __name__ == "__main__":
# If a certain environment variable is set, look there for the input
# file, otherwise look in the current directory.
hdffile = 'CAL_LID_L2_VFM-ValStage1-V3-02.2011-12-31T23-18-11ZD.hdf'
try:
fname = os.path.join(os.environ['HDFEOS_ZOO_DIR'], ncfile)
except KeyError:
fname = hdffile
run(fname)
Please try miniconda and use basemap from conda-forge:
conda install -c conda-forge basemap
I have two Graphs.
Graph_1 is a Directed Acyclic Graph (DAG) which has the following edge list in df_1:
node_1 node_2
John Charity
John Constantine
Gordon John
Gordon Nick
Graph_1 = nx.from_pandas_edgelist(df_1, source="node_1",
target="node_2", create_using=nx.DiGraph())
Graph_2 is a random stochastic graph which is generated as follows:
Graph_2 = nx.erdos_renyi_graph(1000, 0.1)
I would like to join Graph_2 to Graph_1 by making the node with the highest betweenness centrality in Graph_2 a child node of the "Nick" node in Graph_1.
Does anyone have any ideas on how I could do this?
Following should work
import networkx as nx
import matplotlib.pylab as pl
edge_list = [
["John", "Charity"],
["John", "Constantine"],
["Gordon", "John"],
["Gordon", "Nick"], ]
Graph_1 = nx.from_edgelist(edge_list, create_using=nx.DiGraph())
# reduced the number for visualization
Graph_2 = nx.erdos_renyi_graph(10, 0.1)
node_with_highest_betweenness_centrality = max(nx.betweenness_centrality(Graph_2).items(), key=lambda x: x[1])[0]
joined_graph = nx.DiGraph(Graph_1)
joined_graph.add_edges_from(Graph_2.edges())
# not sure which direction you want
joined_graph.add_edge(node_with_highest_betweenness_centrality, "Nick")
nx.draw(joined_graph, with_labels=True)
pl.show()
I am trying to run a hyperparameters optimization with Talos. As I have a lot of parameters to test, I want to use a 'grid_downsample' argument that will select 30% of all possible hyperparameters combinations. However when I run my code I get: TypeError: __init__() got an unexpected keyword argument 'grid_downsample'
I tested the code below without the 'grid_downsample' option and with less hyperparameters.
#load data
data = pd.read_csv('data.txt', sep="\t", encoding = "latin1")
# split into input (X) and output (y) variables
Y = np.array(data['Y'])
data_bis = data.drop(['Y'], axis = 1)
X = np.array(data_bis)
p = {'activation':['relu'],
'optimizer': ['Nadam'],
'first_hidden_layer': [12],
'second_hidden_layer': [12],
'batch_size': [20],
'epochs': [10,20],
'dropout_rate':[0.0, 0.2]}
def dnn_model(x_train, y_train, x_val, y_val, params):
model = Sequential()
#input layer
model.add(Dense(params['first_hidden_layer'], input_shape=(1024,)))
model.add(Dropout(params['dropout_rate']))
model.add(Activation(params['activation']))
#hidden layer 2
model.add(Dense(params['second_hidden_layer']))
model.add(Dropout(params['dropout_rate']))
model.add(Activation(params['activation']))
# output layer with one node
model.add(Dense(1))
model.add(Activation(params['activation']))
# Compile model
model.compile(loss='binary_crossentropy', optimizer=params['optimizer'], metrics=['accuracy'])
out = model.fit(x_train, y_train,
batch_size=params['batch_size'],
epochs=params['epochs'],
validation_data=[x_val, y_val],
verbose=0)
return out, model
scan_object = ta.Scan(X, Y, model=dnn_model, params=p, experiment_name="test")
reporting = ta.Reporting(scan_object)
report = reporting.data
report.to_csv('./Random_search/dnn/report_talos.txt', sep = '\t')
This code works well. If I change the scan_object as the end to: scan_object = ta.Scan(X, Y, model=dnn_model, grid_downsample=0.3, params=p, experiment_name="test"), it gives me the error: TypeError: __init__() got an unexpected keyword argument 'grid_downsample' while I was expecting to have the same results format as a normal grid search but with less combinations. What am I missing? Did the name of the argument change? I'm using Talos 0.6.3 in a conda environment.
Thank you!
might be too late for you now but they've switched it to fraction_limit. It would give this for you
scan_object = ta.Scan(X, Y, model=dnn_model, params=p, experiment_name="test", fraction_limit = 0.1)
Sadly, the doc isn't well updated
Check out their examples on GitHub:
https://github.com/autonomio/talos/blob/master/examples/Hyperparameter%20Optimization%20with%20Keras%20for%20the%20Iris%20Prediction.ipynb
I am attempting to write a simple code to simply plot a bar graph of some fruit names in the x-axis vs corresponding sales units. The aim of this code is just to understand how to query postgres results from heroku hosted database through a dash app.
Below is the code,
from dash import dash
import dash_core_components as dcc
import dash_html_components as html
import plotly.graph_objs as go
import psycopg2
import os
DATABASE_URL = os.environ['DATABASE_URL']
conn = psycopg2.connect(DATABASE_URL, sslmode='require')
cur = conn.cursor()
cur.execute("SELECT fruits FROM pgrt_table")
fruits1=cur.fetchall()
#print(fruits1)
cur.execute("SELECT sales FROM pgrt_table")
sales1=cur.fetchall()
app = dash.Dash()
app.layout = html.Div(children=[
html.H1(
children='Hello Dash'
),
html.Div(
children='''Dash: A web application framework for Python.'''
),
dcc.Graph(
id='example-graph',
figure=go.Figure(
data=[
go.Bar(
x=fruits1, y=sales1, name='SF'),
#{'x': [1, 2, 3], 'y': [2, 4, 5], 'type': 'bar', 'name': u'Montréal'},
],
#'layout':{
# 'title': 'Dash Data Visualization'
#}
)
)
])
if __name__ == '__main__':
app.run_server(debug=True)
The output is below,
Output to the above code
The corresponding output is just the axes with no bar graphs. The connection with the db is working since printing fruits1 or sales1 gives me the values from the columns in postgres. The only issue is the plotting.
NOTE: This question has been heavily modified since the previous draft was extremely vague without any code to show for.
Example:
fruits1 = [('apple',), ('banana',),
('mango',), ('pineapple',),
('peach',), ('watermelon',)]
The output of your database cannot be used directly:
xData = [fruit[0] for fruit in fruits1]
# gives ['apple', 'banana', 'mango', 'pineapple', 'peach', 'watermelon']
yData = [sales[0] for sales in sales1]
You have to assign your data to the go.Bar object:
go.Bar(x=xData, y=yData, name='SF')