I have plotted the histogram of network (dataframe), with count of 'k' node connections, like so:
import seaborn as sns
parameter ='k'
sns.histplot(network[parameter])
But now I need to create a modular random graph using above group distribution with:
from networkx.generators.community import random_partition_graph
random_partition_graph(sizes, p_in, p_out, seed=None, directed=False)
And, instead of counts, I need this value p(k), which must be passed as p_in.
p_in (float)
probability of edges with in groups
How do I get p(k) from my network?
This is how I would handle what you described. First, you can normalize your histogram such that the integral of the histogram is equal to 1. This can be done by setting the weights argument of your histogram appropriately. This histogram can then be considered the probability distribution of your degrees. Now that you have this probability distribution, i.e. a list of probability (deg_prob in the code) you can randomly sample from it using np.random.choice(np.arange(np.amin(degrees),np.amax(degrees)+1), p=deg_prob, size=N_sampling). From this random sampling, you can then create a random expected_degree_graph by just passing your samples in the w argument.
You can then compare the degree distribution of your original graph with the one from your random graph.
See below for the code and more details:
import networkx as nx
from networkx.generators.random_graphs import binomial_graph
from networkx.generators.degree_seq import expected_degree_graph
import matplotlib.pyplot as plt
import numpy as np
fig=plt.figure()
N_nodes=1000
G=binomial_graph(n=N_nodes, p=0.01, seed=0) #Creating a random graph as data
degrees = np.array([G.degree(n) for n in G.nodes()])#Computing degrees of nodes
bins_val=np.arange(np.amin(degrees),np.amax(degrees)+2) #Bins
deg_prob,_,_=plt.hist(degrees,bins=bins_val,align='left',weights=np.ones_like(degrees)/N_nodes,
color='tab:orange',alpha=0.3,label='Original distribution')#Histogram
#Sampling from distribution
N_sampling=500
random_sampling=np.random.choice(np.arange(np.amin(degrees),np.amax(degrees)+1), p=deg_prob, size=N_sampling)
#Creating random graph from samples
G_random_sampling=expected_degree_graph(random_sampling,seed=0,selfloops=False)
degrees_random_sampling = np.array([G_random_sampling.degree(n) for n in G_random_sampling.nodes()])
deg_prob_random_sampling,_,_=plt.hist(degrees_random_sampling,bins=bins_val,align='left',
weights=np.ones_like(degrees_random_sampling)/N_sampling,color='tab:blue',label='Sample distribution',alpha=0.3)
#Plotting both histograms
plt.xticks(bins_val)
plt.xlabel('degree')
plt.ylabel('Prob')
plt.legend()
plt.show()
The output then gives:
Related
import numpy as np
import networkx as nx
import pylab as plt
A=np.array([[0,0,1,0],[1,0,0,0],[1,0,0,1],[1,0,0,0],[1,1,0,0]])
G = nx.DiGraph(A)
when iam trying to print graph of above matrix it shows error
pos=[[0,0],[0,1],[1,0],[1,1],[2,1]]
nx.draw(G,pos)
plt.savefig("2trial.png",format="PNG")
Networkx has a special function to construct a graph from numpy adjacency matrix:
G = nx.from_numpy_matrix(A)
However, an adjacency matrix must be square:
In graph theory and computer science, an adjacency matrix is a square matrix used to represent a finite graph. The elements of the matrix indicate whether pairs of vertices are adjacent or not in the graph.
So you can't create a graph with your matrix because it is not an adjacency matrix. You should convert it to a 5x5 matrix and then send to nx.from_numpy_matrix function:
import numpy as np
import networkx as nx
A=np.array([[0,0,1,0,0],[1,0,0,0,1],[1,0,0,1,0],[1,0,0,0,0],[1,1,0,0,1]])
G = nx.from_numpy_matrix(A, create_using=nx.DiGraph)
pos=[[0,0],[0,1],[1,0],[1,1],[2,1]]
nx.draw(G,pos)
Suppose you have a graph graph = nx.read_gml("x.gml") and you'd like to drop n edges. Is there any quick way to do so?
Here is one approach using the sample function from the random library. I set k, the number of edges to be sampled to 2.
import networkx as nx
import random
G=nx.Graph()
G.add_edges_from([[1,2],[1,3],[2,3],[2,4],[3,5],[4,5]])
to_remove=random.sample(G.edges(),k=2)
G.remove_edges_from(to_remove)
print(G.edges())
I want to know how to generate the same random (Normal Distribution) numbers in numpy as I do in MATLAB.
As an example when I do this in MATLAB
RandStream.setGlobalStream(RandStream('mt19937ar','seed',1));
rand
ans =
0.417022004702574
Now I can reproduce this with numpy:
import numpy as np
np.random.seed(1)
np.random.rand()
0.417022004702574
Which is nice, but when I do this with normal distribution I get different numbers.
RandStream.setGlobalStream(RandStream('mt19937ar','seed',1));
randn
ans =
-0.649013765191241
And with numpy
import numpy as np
np.random.seed(1)
np.random.randn()
1.6243453636632417
Both functions say in their documentation that they draw from the standard normal distribution, yet give me different results. Any idea how I can adjust my python/numpy to get the same numbers as MATLAB.
Because someone marked this as a duplicate:
This is about normal distribution, as I wrote in the beginning and end.
As I wrote uniform distribution works fine, this is about normal distribution.
None of the answers in the linked thread help with normal distribution.
My guess would be that the matlab and numpy may use different methods to get normal distribution of random numbers (which are obtained from uniform numbers in some way).
You can avoid this problem by writing a box-muller method to generate the random numbers yourself. For python,
import numpy as np
# Box-muller normal distribution, note needs pairs of random numbers as input
def randn_from_rand(rand):
assert rand.size == 2
#Use box-muller to get normally distributed random numbers
randn = np.zeros(2)
randn[0] = np.sqrt(-2.*np.log(rand[0]))*np.cos(2*np.pi*rand[1])
randn[1] = np.sqrt(-2.*np.log(rand[0]))*np.sin(2*np.pi*rand[1])
return randn
np.random.seed(1)
r = np.random.rand(2)
print(r, randn_from_rand(r))
which gives,
(array([ 0.417022 , 0.72032449]), array([-0.24517852, -1.29966152]))
and for matlab,
% Box-muller normal distribution, note needs pairs of random numbers as input
function randn = randn_from_rand(rand)
%Use box-muller to get normally distributed random numbers
randn(1) = sqrt(-2*log(rand(1)))*cos(2*pi*rand(2));
randn(2) = sqrt(-2*log(rand(1)))*sin(2*pi*rand(2));
which we call with
RandStream.setGlobalStream(RandStream('mt19937ar','seed',1));
r = [rand, rand]
rn = randn_from_rand(r)
with answer,
r =
0.4170 0.7203
rn =
-0.2452 -1.2997
Note, you can check the output is normally distributed, for python,
import matplotlib.pyplot as plt
ra = []
np.random.seed(1)
for i in range(1000000):
rand = np.random.rand(2)
ra.append(randn_from_rand(rand))
plt.hist(np.array(ra).ravel(),100)
plt.show()
which gives,
I try to fit my data to a poisson distribution:
import seaborn as sns
import scipy.stats as stats
sns.distplot(x, kde = False, fit = stats.poisson)
But I get this error:
AttributeError: 'poisson_gen' object has no attribute 'fit'
Other distribution (gamma, etc) de work well.
The Poisson distribution (implemented in scipy as scipy.stats.poisson) is a discrete distribution. The discrete distributions in scipy do not have a fit method.
I'm not very familiar with the seaborn.distplot function, but it appears to assume that the data comes from a continuous distribution. If that is the case, then even if scipy.stats.poisson had a fit method, it would not be an appropriate distribution to pass to distplot.
The question title is "How to fit a poisson distribution with seaborn?", so for the sake of completeness, here's one way to get a plot of the data and its fit. seaborn is only used for the bar plot, using #mwaskom's suggestion to use seaborn.countplot. The fitting is actually trivial, because the maximum likelihood estimation for the Poisson distribution is simply the mean of the data.
First, the imports:
In [136]: import numpy as np
In [137]: from scipy.stats import poisson
In [138]: import matplotlib.pyplot as plt
In [139]: import seaborn
Generate some data to work with:
In [140]: x = poisson.rvs(0.4, size=100)
These are the values in the x:
In [141]: k = np.arange(x.max()+1)
In [142]: k
Out[142]: array([0, 1, 2, 3])
Use seaborn.countplot to plot the data:
In [143]: seaborn.countplot(x, order=k, color='g', alpha=0.5)
Out[143]: <matplotlib.axes._subplots.AxesSubplot at 0x114700490>
The maximum likelihood estimation of the Poisson parameter is simply the mean of the data:
In [144]: mlest = x.mean()
Use poisson.pmf() to get the expected probability, and multiply by the size of the data set to get the expected counts, and then plot using matplotlib. The bars are the counts of the actual data, and the dots are the expected counts of the fitted distribution:
In [145]: plt.plot(k, poisson.pmf(k, mlest)*len(x), 'go', markersize=9)
Out[145]: [<matplotlib.lines.Line2D at 0x114da74d0>]
I have a 3000x50 feature vector matrix. I obtained a similarity matrix for this using sklearn.metrics.pairwise_distances as 'Similarity_Matrix'. Now I used networkx to create a graph using the similarity matrix generated in the previous step as G=nx.from_numpy_matrix(Similarity_Matrix). I want to perform spectral clustering on this graph G now but several google searches have failed to provide a decent example of scikit learn spectral clustering on this graph :( The official documentation shows how spectral clustering can be done on some image data which is highly unclear at least to a newbie like myself.
Can anyone give me a code sample for this or for graph cuts or graph partitioning using networkx, scikit learn etc.
Thanks a million!
adj_matrix = nx.from_numpy_matrix will help you create an adjacency matrix which will be your affinity matrix. You need to feed this to scikit-learn like this: SpectralClustering(affinity = 'precomputed', assign_labels="discretize",random_state=0,n_clusters=2).fit_predict(adj_matrix)
If you don't have any similarity matrix, you can change the value of 'affinity' param to 'rbf' or 'nearest_neighbors'. An example below explains the entire Spectral Clustering pipeline:
import sklearn
import networkx as nx
import matplotlib.pyplot as plt
'''Graph creation and initialization'''
G=nx.Graph()
G.add_edge(1,2) # default edge weight=1
G.add_edge(3,4,weight=0.2) #weight represents edge weight or affinity
G.add_edge(2,3,weight=0.9)
G.add_edge("Hello", "World", weight= 0.6)
'''Matrix creation'''
adj_matrix = nx.to_numpy_matrix(G) #Converts graph to an adj matrix with adj_matrix[i][j] represents weight between node i,j.
node_list = list(G.nodes()) #returns a list of nodes with index mapping with the a
'''Spectral Clustering'''
clusters = SpectralClustering(affinity = 'precomputed', assign_labels="discretize",random_state=0,n_clusters=2).fit_predict(adj_matrix)
plt.scatter(nodes_list,clusters,c=clusters, s=50, cmap='viridis')
plt.show()