Following the instructions of ipycitoscape I am not able to plot a graph using ipycitoscape.
according to: https://github.com/QuantStack/ipycytoscape/blob/master/examples/Test%20NetworkX%20methods.ipynb
this should work:
import networkx as nx
import ipycytoscape
G2 = nx.Graph()
G2.add_nodes_from([*'ABCDEF'])
G2.add_edges_from([('A','B'),('B','C'),('C','D'),('E','F')])
print(G2.nodes)
print(G2.edges)
cytoscapeobj = ipycytoscape.CytoscapeWidget()
cytoscapeobj.graph.add_graph_from_networkx(nx_graph)
G2 is a networkx graph example and it looks ok since print(G2) gives the networkx object back and G2.nodes and G2.edges can be printed.
The error:
ValueError: invalid literal for int() with base 10: 'A'
Why should a node be an integer?
More general what to do if the starting data point if a pandas dataframe with a million rows edges those being strings like ProcessA-ProcessB, processC-processD etc
Also having a look to the examples it is to be noted that the list of nodes is composed of a dictionary data for every node. that data including an "id" per node and also "Atribute". The surprise here is that the networkx Graph should have all those properties.
thanks
This problem was fixed. See attachment.
Please let me know if it's still happening. Feel free to open an issue: https://github.com/QuantStack/ipycytoscape/
I'm just playing around with ipycytoscape myself, so I could be way off-base, but, shouldn't the line be:
cytoscapeobj.graph.add_graph_from_networkx(G2) # your graph name goes here
Trying to generate a cytoscape object built on a graph that doesn't exist might trigger a ValueError because it can't find any nodes.
Related
I use OSMnx for my project and I need to retrieve information about roads.
I know I can use osmnx.graph_from_place(place, network_type="all") to get the graph of highways of a certain type from Open Street Map. However it does not provides lots of features, only 15.
I know I can use osmnx.geometries_from_place(place, tags={"highway"=True}) , it gives more features however it provides different result than the previous function (less rows).
Using 'Etterbeek, Brussels, Belgium' as place, with graph_from_place, I get 12636 rows in my dataframe but with geometries_from_place, I get 3636
So can anyone explain me what is the difference between the two functions and why they provide different results?
Code :
import osmnx as ox
place = 'Etterbeek, Brussels, Belgium'
etterbeek_all_highway_info = ox.geometries_from_place(place, tags={"highway":True})
len(etterbeek_all_highway_info) # 3636
street_graph = ox.graph_from_place(place, network_type='all', simplify=False)
nodes , streets = ox.graph_to_gdfs(street_graph)
len(streets["highway"]) #12636
Documentation :
https://osmnx.readthedocs.io/en/stable/osmnx.html#osmnx.graph.graph_from_place
https://osmnx.readthedocs.io/en/stable/osmnx.html#osmnx.geometries.geometries_from_place
I am using OSMnx to get a graph and add a new edge attribute (w3) representing a custom weight for each edge. Then I can successfully find 2 different shortest paths between 2 points using NetworkX and 'length', 'w2'. Everything works fine, this is my code:
G = ox.graph_from_place(PLACE, network_type='all_private', retain_all = True, simplify=True,truncate_by_edge=False) ```
w3_dict = dict((zip(zip(lu, lv, lk),lw3)))
nx.set_edge_attributes(G, w3_dict, "w3")
route_1 = nx.shortest_path(G, node_start, node_stop, weight = 'length')
route_2 = nx.shortest_path(G, node_start, node_stop, weight = 'w3')
Now I would like to save G to disk and reopen it, to perform more navigation tasks later on. But after saving it with:
ox.save_graph_xml(G, filepath='DATA/network.osm')
and reopen it with:
G = ox.graph_from_xml('DATA/network.osm')
my custom attribute w3 has disappeared. I have followed the instructions in the docs but with no luck. It feels like I'm missing something really obvious but I don't understand what it is..
Use the ox.save_graphml and ox.load_graphml functions to save/load full-featured OSMnx/NetworkX graphs to/from disk for later use. The save xml function exists only to allow serialization to the .osm file format for applications that require it, and has many constraints to conform to that.
import networkx as nx
import osmnx as ox
ox.config(use_cache=True, log_console=True)
# get a graph, set 'w3' edge attribute
G = ox.graph_from_place('Piedmont, CA, USA', network_type='drive')
nx.set_edge_attributes(G, 100, 'w3')
# save graph to disk
ox.save_graphml(G, './data/graph.graphml')
# load graph from disk and confirm 'w3' edge attribute is there
G2 = ox.load_graphml('./data/graph.graphml')
nx.get_edge_attributes(G2, 'w3')
I am downloading all the images from the Neurotransmitter study of the Allen Brain Atlas using this script:
from allensdk.api.queries.image_download_api import ImageDownloadApi
from allensdk.config.manifest import Manifest
import pandas as pd
import os
#getting transmitter study
#product id from http://api.brain-map.org/api/v2/data/query.json?criteria=model::Product
nt_datasets = image_api.get_section_data_sets_by_product([27])
#an instance of Image Api for downloading
image_api = ImageDownloadApi()
for index, row in df_nt.iterrows():
#get section dataset id
section_dataset_id= row['id']
#each section dataset id has multiple image sections
section_images = pd.DataFrame(
image_api.section_image_query(
section_data_set_id=section_dataset_id)
)
for section_image_id in section_images['id'].tolist():
file_path = os.path.join('/path/to/save/dir/',
str(section_image_id) + '.jpg' )
Manifest.safe_make_parent_dirs(file_path)
image_api.download_section_image(section_image_id,
file_path=file_path,
downsample=downsample)
This script downloads presumably all the available ISH experiments. However, I am wondering what would be the best way to get more of the metadata as follows:
1) type of ISH experiment, known as "gene" (for example whether an image is MBP-stained, Nissl-stained or etc). Shown in red circle below.
2) Structure and correspondence to the atlas image (annotations, for example to see to which part of brain a section belongs to). I think this could be acquired with tree_search but not sure how. Shown in red circles below from two different webpages on Allen website.
3) The scale of the image, for example how big one pixel is in the downloaded image (e.g., 0.001x0.001 mm). I would require this for image analysis with respect to MRI, for example. Shown below in the red circle.
All the above information are somehow available on the website, my question is whether you could help me to do this programmatically via the SDK.
EDIT:
Also would be great to download "Nissl" stains programmatically, as they do not show using the above loop iteration. The picture is shown below.
To access this information, you'll need to formulate a somewhat complex API query.
from allensdk.api.queries.rma_api import RmaApi
api = RmaApi()
data_set_id = 146586401
data = api.model_query('SectionDataSet',
criteria='[id$eq%d]' % data_set_id,
include='section_images,treatments,probes(gene),specimen(structure)')
print("gene symbol: %s" % data[0]['probes'][0]['gene']['acronym'])
print("treatment name: %s" % data[0]['treatments'][0]['name'])
print("specimen location: %s" % data[0]['specimen']['structure']['name'])
print("section xy resolution: %f um" % data[0]['section_images'][0]['resolution'])
gene symbol: MBP
treatment name: ISH
specimen location: Cingulate Cortex
section xy resolution: 1.008000 um
Without doing a deep dive on the API data model, SectionDataSets have constituent SectionImages, Treatments, Probes, and source Specimens. Probes target Genes, and Specimens can be associated with a Structure. The query is downloading all of that information for a single SectionDataSet into a nested dictionary.
I don't remember how to find the specimen block extent. I'll update the answer if I find it.
I've just recently discovered that you can right-click an array in Spyder and get a quick plot of the data. With sample data like this:
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
# Some numbers in a data frame
nsample = 440
x1 = np.linspace(0, 100, nsample)
y = np.sin(x1)
dates = pd.date_range(pd.datetime(2016, 1, 1).strftime('%Y-%m-%d'), periods=nsample).tolist()
df = pd.DataFrame({'dates':dates, 'x1':x1, 'y':y})
df = df.set_index(['dates'])
df.index = pd.to_datetime(df.index)
you can go to the Variable explorer, right-click y and get the following directly in the console:
which will give you this:
The same option does not seem to be available to a pandas dataframe:
Sure, you could easily go for df.plot():
But I really like the right-click option to check whether the variables and dataframes look the way I expect them to when I'm messing around with a lot of data. So, is there any library I'd have to import? Or maybe something in the settings? I've also noticed that what happens in the console is this little piece of magic: %varexp --plot y, but can't seem to find an equivalent for data frames.
Thank you for any suggestions!
(Spyder developer here) This is just a bit of missing functionality for Dataframes, but it's very easy to implement.
Please open an issue in our issue tracker, so we don't forget to do it in a future release.
Hi I'm trying to import a shape file from
http://www.nyc.gov/html/dcp/html/bytes/bytesarchive.shtml
into a postgis database. the above files creates MULTIPOLYGONS when i import using shp2pgsql.
then i'm trying to simply determine if lat/long points are contained in my multipolygons
however my select's are not working, and when i print out the poitns of my the_geom column it seems to be very broken.
select st_astext(geom) from (select (st_dumppoints(the_geom)).* from nybb where borocode =1) foo;
gives the result...
st_astext
------------------------------------------
POINT(1007193.83859999 257820.786899999)
POINT(1007209.40620001 257829.435100004)
POINT(1007244.8654 257833.326199993)
POINT(1007283.3496 257839.812399998)
POINT(1007299.3502 257851.488900006)
POINT(1007320.1081 257869.218500003)
POINT(1007356.64669999 257891.055800006)
POINT(1007385.6197 257901.432999998)
POINT(1007421.94509999 257894.084000006)
POINT(1007516.85959999 257890.406100005)
POINT(1007582.59110001 257884.7861)
POINT(1007639.02150001 257877.217199996)
POINT(1007701.29170001 257872.893099993)
...
for points in nyc, this is very off.. what am i doing wrong?
The points are not of. The spatial data that is referred to is NOT in lat/long. This is why numbers are different from what you expect. If you need it to be in long/lat it must be reprojected. See more here: http://postgis.refractions.net/news/20020108/
The projection of the data seems to be in the NAD_1983_StatePlane_New_York_Long_Island_FIPS_3104_Feet coordinate system (according to the metadata - see code.).
<spref>
<horizsys>
<planar>
<planci>
<plance Sync="TRUE">coordinate pair</plance>
<coordrep>
<absres Sync="TRUE">0.000000</absres>
<ordres Sync="TRUE">0.000000</ordres>
</coordrep>
<plandu Sync="TRUE">survey feet</plandu>
</planci>
<mapproj><mapprojn Sync="TRUE">Lambert Conformal Conic</mapprojn><lambertc><stdparll Sync="TRUE">40.666667</stdparll><stdparll Sync="TRUE">41.033333</stdparll><longcm Sync="TRUE">-74.000000</longcm><latprjo Sync="TRUE">40.166667</latprjo><feast Sync="TRUE">984250.000000</feast><fnorth Sync="TRUE">0.000000</fnorth></lambertc></mapproj></planar>
<geodetic>
<horizdn Sync="TRUE">North American Datum of 1983</horizdn>
<ellips Sync="TRUE">Geodetic Reference System 80</ellips>
<semiaxis Sync="TRUE">6378137.000000</semiaxis>
<denflat Sync="TRUE">298.257222</denflat>
</geodetic>
<cordsysn>
<geogcsn Sync="TRUE">GCS_North_American_1983</geogcsn>
<projcsn Sync="TRUE">NAD_1983_StatePlane_New_York_Long_Island_FIPS_3104_Feet</projcsn>
</cordsysn>
</horizsys>
</spref>
If you work much with spatial data I suggest that you read more about map projection.
I think this is not issue with PostGIS. I checked input esri Shape file nybb.shp with AvisMap Free Viewer and as you see points are weird itself:
However there is something interesting in nybb.shp.xml metadata file:
<spdom>
<bounding>
<westbc Sync="TRUE">-74.257465</westbc>
<eastbc Sync="TRUE">-73.699450</eastbc>
<northbc Sync="TRUE">40.915808</northbc>
<southbc Sync="TRUE">40.495805</southbc>
</bounding>
<lboundng>
<leftbc Sync="TRUE">913090.770096</leftbc>
<rightbc Sync="TRUE">1067317.219904</rightbc>
<bottombc Sync="TRUE">120053.526313</bottombc>
<topbc Sync="TRUE">272932.050103</topbc>
</lboundng>
</spdom>
I am not familiar with those toolkit (ESRI ArcCatalog), but most probably you need to rescale your points after import using that metadata.