In order to remove attributes from a networkx graph I have the following code:
for (n1,n2) in graph.edges(data=False):
for att in att_list:
graph[n1][n2].pop(att, None)
Is there a more pythonic way to do so?
If you only want to delete a few attributes in some list, say att_list
for n1, n2, d in graph.edges(data=True):
for att in att_list:
d.pop(att, None)
Or you can replace the last line with if att in d: del d[att] if it bothers you that pop returns something that you aren't using. The improvement compared to your code is that by having data=True I get d immediately, rather than having to reference graph[n1][n1] later.
See Removing multiple keys from a dictionary safely for how to remove multiple keys from a dictionary (which is what d is). Fundamentally that's what your problem reduces to once you've got d.
Alternately, if you want to clear all of the attributes, then note that if we set data=True, then graph.edges also returns a dictionary with the attributes. Clear this dictionary.
for (n1, n2, d) in graph.edges(data=True):
d.clear()
Here's a complete example
import networkx as nx
G=nx.Graph()
G.add_edge(1,2,weight=2)
G.edge[1][2]
> {'weight': 5}
for (n1, n2, d) in G.edges(data=True):
d.clear()
G.edge[1][2]
> {}
#just to check the edge in the opposite order
G.edge[2][1]
> {}
Related
Following the documentation, I tried to do the following:
t:([]a:1 2 3;b:4 5 6;c:`d`e`f) // some input table
`a`b _ t // works: delete NOT in place
(enlist `a) _ t // works: delete NOT in place
t _:`a`b // drop columns in place does not work; how to make it to work?
// 'type
// [0] t _:`a`b
Thank you very much for your help!
You should be able to use
delete a,b from `t
to delete in place (The backtick implies in place).
Alternatively, for more flexibility you could use the functional form;
![`t;();0b;`a`b]
The simplest way to achieve column deletion in place is using qSQL:
t:([]a:1 2 3;b:4 5 6;c:`d`e`f)
delete a,b from `t -- here, the backtick before t makes the change in place.
q)t
c
-
d
e
f
Michael & Kyle have covered the q-SQL options; for completeness, here are a couple of other options using _:
Using _ as in your question, you can re-assign this back to t e.g.
t:`a`b _ t
You can also use . amend with an empty list of indexes i.e. "amend entire", which can be done in-place by passing `t or not in-place by passing just t e.g.
q).[t;();`a`b _] / not in-place
c
-
d
e
f
q).[`t;();`a`b _] / in-place
`t
q)t
c
-
d
e
f
I am having a couple of issues to put this in a functional format.
select from tableName where i=fby[(last;i);([]column_one;column_two)]
This is what I got:
?[tableName;fby;enlist(=;`i;(enlist;last;`i);(+:;(!;enlist`column_one`column_two;(enlist;`column_one;`column_two))));0b;()]
but I get a type error.
Any suggestions?
Consider using the following function, adjust from the buildQuery function given in the whitepaper on Parse Trees. This is a pretty useful tool for quickly developing in q, this version is an improvement on that given in the linked whitepaper, having been extended to handle updates by reference (i.e., update x:3 from `tab)
\c 30 200
tidy:{ssr/[;("\"~~";"~~\"");("";"")] $[","=first x;1_x;x]};
strBrk:{y,(";" sv x),z};
//replace k representation with equivalent q keyword
kreplace:{[x] $[`=qval:.q?x;x;"~~",string[qval],"~~"]};
funcK:{$[0=t:type x;.z.s each x;t<100h;x;kreplace x]};
//replace eg ,`FD`ABC`DEF with "enlist`FD`ABC`DEF"
ereplace:{"~~enlist",(.Q.s1 first x),"~~"};
ereptest:{((0=type x) & (1=count x) & (11=type first x)) | ((11=type x)&(1=count x))};
funcEn:{$[ereptest x;ereplace x;0=type x;.z.s each x;x]};
basic:{tidy .Q.s1 funcK funcEn x};
addbraks:{"(",x,")"};
//where clause needs to be a list of where clauses, so if only one whereclause need to enlist.
stringify:{$[(0=type x) & 1=count x;"enlist ";""],basic x};
//if a dictionary apply to both, keys and values
ab:{$[(0=count x) | -1=type x;.Q.s1 x;99=type x;(addbraks stringify key x),"!",stringify value x;stringify x]};
inner:{[x]
idxs:2 3 4 5 6 inter ainds:til count x;
x:#[x;idxs;'[ab;eval]];
if[6 in idxs;x[6]:ssr/[;("hopen";"hclose");("iasc";"idesc")] x[6]];
//for select statements within select statements
//This line has been adjusted
x[1]:$[-11=type x 1;x 1;$[11h=type x 1;[idxs,:1;"`",string first x 1];[idxs,:1;.z.s x 1]]];
x:#[x;ainds except idxs;string];
x[0],strBrk[1_x;"[";"]"]
};
buildSelect:{[x]
inner parse x
};
We can use this to create the functional query that will work
q)n:1000
q)tab:([]sym:n?`3;col1:n?100.0;col2:n?10.0)
q)buildSelect "select from tab where i=fby[(last;i);([]col1;col2)]"
"?[tab;enlist (=;`i;(fby;(enlist;last;`i);(flip;(lsq;enlist`col1`col2;(enlist;`col1;`col2)))));0b;()]"
So we have the following as the functional form
?[tab;enlist (=;`i;(fby;(enlist;last;`i);(flip;(lsq;enlist`col1`col2;(enlist;`col1;`col2)))));0b;()]
// Applying this
q)?[tab;enlist (=;`i;(fby;(enlist;last;`i);(flip;(lsq;enlist`col1`col2;(enlist;`col1;`col2)))));0b;()]
sym col1 col2
----------------------
bah 18.70281 3.927524
jjb 35.95293 5.170911
ihm 48.09078 5.159796
...
Glad you were able to fix your problem with converting your query to functional form.
Generally it is the case that when you use parse with a fby in your statement, q will convert this function into its k definition. Usually you should just be able to replace this k code with the q function itself (i.e. change (k){stuff} to fby) and this should run properly when turning the query into functional form.
Additionally, if you check out https://code.kx.com/v2/wp/parse-trees/ it goes into more detail about parse trees and functional form. Additionally, it contains a script called buildQuery which will return the functional form of the query of interest as a string which can be quite handy and save time when a functional form is complex.
I actually got it myself ->
?[tableName;((=;`i;(fby;(enlist;last;`i);(+:;(!;enlist`column_one`column_two;(enlist;`column_one;`column_two)))));(in;`venue;enlist`venueone`venuetwo));0b;()]
The issues was a () missing from the statement. Works fine now.
**if someone wants to add a more detailed explanation on how manual parse trees are built and how the generic (k){} function can be replaced with the actual function in q feel free to add your answer and I'll accept and upvote it
I have a complex nested list (depth can be >2 also):
p:((`g;`d1`d2);(`r;enlist `e1);(`r;enlist `p1))
How to add an element to each element of the nested list but retaining the original structure; e.g. adding `h to each element of p to get the following :
((`g`h;(`d1`h;`d2`h));(`r`h;enlist `e1`h);(`r`h;enlist `p1`h))
I tried this but doesn't give what I want :
q)p,\:`h
((`g;`d1`d2;`h);(`r;enlist `e1;`h);(`r;enlist `p1;`h))
q)raze[p],\:`h
(`g`h;`d1`d2`h;`r`h;`e1`h;`r`h;`p1`h)
You can use .z.s to recursively go through the nested list and only append `h to lists of symbols:
q){$[0=type x;.z.s'[x];x,\:`h]}p
g h d1 h d2 h
`r`h ,`e1`h
`r`h ,`p1`h
For this function I have made the assumption that your nested lists will only contain symbols. It checks the type of the list, if it is not a mixed list then it appends `h to each element. If it is a mixed list then it passes each element of that list back into the function separately to check again.
Although not recursive (and so requires some knowledge about the shape of your nested list), a more conventional approach would be
q).[p;2#(::);,';`h]
g h d1 h d2 h
`r`h ,`e1`h
`r`h ,`p1`h
Though Thomas has already answered the question; In case you want to specify any other operation apart from append, you can use the following :
q)f:{`$ "_" sv string x,y}
q){[o;a;e] $[-11<>type e; .z.s [o;a] each e; o[e;a]] }[f;`h] each p
`g_h `d1_h`d2_h
`r_h ,`e1_h
`r_h ,`p1_h
or when f is assigned as append operation
q)f:{x,y}
q){[o;a;e] $[-11<>type e; .z.s [o;a] each e; o[e;a]] }[f;`h] each p
g h d1 h d2 h
`r`h ,`e1`h
`r`h ,`p1`h
Consider this example:
import numpy as np
a = np.array(1)
np.save("a.npy", a)
a = np.load("a.npy", mmap_mode='r')
print(type(a))
b = a + 2
print(type(b))
which outputs
<class 'numpy.core.memmap.memmap'>
<class 'numpy.int32'>
So it seems that b is not a memmap any more, and I assume that this forces numpy to read the whole a.npy, defeating the purpose of the memmap. Hence my question, can operations on memmaps be deferred until access time?
I believe subclassing ndarray or memmap could work, but don't feel confident enough about my Python skills to try it.
Here is an extended example showing my problem:
import numpy as np
# create 8 GB file
# np.save("memmap.npy", np.empty([1000000000]))
# I want to print the first value using f and memmaps
def f(value):
print(value[1])
# this is fast: f receives a memmap
a = np.load("memmap.npy", mmap_mode='r')
print("a = ")
f(a)
# this is slow: b has to be read completely; converted into an array
b = np.load("memmap.npy", mmap_mode='r')
print("b + 1 = ")
f(b + 1)
Here's a simple example of an ndarray subclass that defers operations on it until a specific element is requested by indexing.
I'm including this to show that it can be done, but it almost certainly will fail in novel and unexpected ways, and require substantial work to make it usable.
For a very specific case it may be easier than redesigning your code to solve the problem in a better way.
I'd recommend reading over these examples from the docs to help understand how it works.
import numpy as np
class Defered(np.ndarray):
"""
An array class that deferrs calculations applied to it, only
calculating them when an index is requested
"""
def __new__(cls, arr):
arr = np.asanyarray(arr).view(cls)
arr.toApply = []
return arr
def __array_ufunc__(self, ufunc, method, *inputs, **kwargs):
## Convert all arguments to ndarray, otherwise arguments
# of type Defered will cause infinite recursion
# also store self as None, to be replaced later on
newinputs = []
for i in inputs:
if i is self:
newinputs.append(None)
elif isinstance(i, np.ndarray):
newinputs.append(i.view(np.ndarray))
else:
newinputs.append(i)
## Store function to apply and necessary arguments
self.toApply.append((ufunc, method, newinputs, kwargs))
return self
def __getitem__(self, idx):
## Get index and convert to regular array
sub = self.view(np.ndarray).__getitem__(idx)
## Apply stored actions
for ufunc, method, inputs, kwargs in self.toApply:
inputs = [i if i is not None else sub for i in inputs]
sub = super().__array_ufunc__(ufunc, method, *inputs, **kwargs)
return sub
This will fail if modifications are made to it that don't use numpy's universal functions. For instance percentile and median aren't based on ufuncs, and would end up loading the entire array. Likewise, if you pass it to a function that iterates over the array, or applies an index to substantial amounts the entire array will be loaded.
This is just how python works. By default numpy operations return a new array, so b never exists as a memmap - it is created when + is called on a.
There's a couple of ways to work around this. The simplest is to do all operations in place,
a += 1
This requires loading the memory mapped array for reading and writing,
a = np.load("a.npy", mmap_mode='r+')
Of course this isn't any good if you don't want to overwrite your original array.
In this case you need to specify that b should be memmapped.
b = np.memmap("b.npy", mmap+mode='w+', dtype=a.dtype, shape=a.shape)
Assigning can be done by using the out keyword provided by numpy ufuncs.
np.add(a, 2, out=b)
I am going to start illustration using a code:
A = 'G1(General G1Airlines american G1Fungus )';
Using regexp (or any other function) in Matlab I want to distinctively locate: G1, G1A and G1F.
Currently if I try to do something as:
B = regexp( A, 'G1')
It is not able to distinguish G1 with the G1A and G1F i.e. I need to force the comparison to find me only case with G1 and ignore G1A and G1F.
However, when I am searching for G1A then it should still find me the location of G1A.
Can someone please help ?
Edit: Another case for A is:
A = 'R1George Service SmalR1Al C&I)';
And the expression this time I need to find is R1 and R1A instead.
Edit:
I have a giant array containing A's and another big vector containing G1, R1, etc I need to search for.
If you want to find 'G1' but not 'G1A' or 'G1F' you can use
>> B = regexp(A, 'G1[^AF]')
B =
1
This will find 'G1' and the ^ is used to specify that it should not match any characters contained with []. Then you could use
>> B = regexp(A, 'G1[AF]')
B =
12 32
to find both 'G1A' and 'G1F'.