I have tried to generate classes for WSDL (I need complex types classes)
--> WSDL here <--
but have many errors:
found unrecognised attribute {http://schemas.xmlsoap.org/wsdl/}arrayType (ignored) at /usr/share/perl5/SOAP/WSDL/Base.pm line 130.
found unrecognised attribute {http://schemas.xmlsoap.org/wsdl/}arrayType (ignored) at /usr/share/perl5/SOAP/WSDL/Base.pm line 130.
found unrecognised attribute {http://schemas.xmlsoap.org/wsdl/}arrayType (ignored) at /usr/share/perl5/SOAP/WSDL/Base.pm line 130.
found unrecognised attribute {http://schemas.xmlsoap.org/wsdl/}arrayType (ignored) at /usr/share/perl5/SOAP/WSDL/Base.pm line 130.
Creating complexType class MyTypes/AddressInfoStruct.pm
Creating complexType class MyTypes/AddressUserDataStruct.pm
Creating complexType class MyTypes/AddressesInfoStruct.pm
/usr/share/perl5/SOAP/WSDL/Generator/Template/XSD\complexType.tt undef error - no node at /usr/share/perl5/SOAP/WSDL/Generator/Template/Plugin/XSD.pm line 55
SOAP::WSDL::Generator::Template::Plugin::XSD::create_xsd_name('SOAP::WSDL::Generator::Template::Plugin::XSD=SCALAR(0xa17155c)', '') called at /usr/share/perl5/SOAP/WSDL/Generator/Template/XSD/complexType/attributeSet.tt line 42
eval {...} called at /usr/share/perl5/SOAP/WSDL/Generator/Template/XSD/complexType/attributeSet.tt line 42
eval {...} called at /usr/share/perl5/SOAP/WSDL/Generator/Template/XSD/complexType/attributeSet.tt line 6
eval {...} called at /usr/share/perl5/SOAP/WSDL/Generator/Template/XSD/complexType/attributeSet.tt line 7
Template::Document::__ANON__('Template::Context=HASH(0xa171728)') called at /usr/lib/perl5/Template/Document.pm line 151
eval {...} called at /usr/lib/perl5/Template/Document.pm line 149
Template::Document::process('Template::Document=HASH(0xa15ead4)', 'Template::Context=HASH(0xa171728)') called at /usr/lib/perl5/Template/Context.pm line 347
eval {...} called at /usr/lib/perl5/Template/Context.pm line 321
Template::Context::process('Template::Context=HASH(0xa171728)', 'complexType/attributeSet.tt', undef, 'localize me!') called at /usr/lib/perl5/Template/Context.pm line 409
Template::Context::include('Template::Context=HASH(0xa171728)', 'complexType/attributeSet.tt') called at /usr/share/perl5/SOAP/WSDL/Generator/Template/XSD/complexType/contentModel.tt line 19
eval {...} called at /usr/share/perl5/SOAP/WSDL/Generator/Template/XSD/complexType/contentModel.tt line 7
Template::Document::__ANON__('Template::Context=HASH(0xa171728)') called at /usr/lib/perl5/Template/Document.pm line 151
eval {...} called at /usr/lib/perl5/Template/Document.pm line 149
Template::Document::process('Template::Document=HASH(0xa149910)', 'Template::Context=HASH(0xa171728)') called at /usr/lib/perl5/Template/Context.pm line 347
eval {...} called at /usr/lib/perl5/Template/Context.pm line 321
Template::Context::process('Template::Context=HASH(0xa171728)', 'complexType/contentModel.tt', undef, 'localize me!') called at /usr/lib/perl5/Template/Context.pm line 409
Template::Context::include('Template::Context=HASH(0xa171728)', 'complexType/contentModel.tt') called at /usr/share/perl5/SOAP/WSDL/Generator/Template/XSD/complexType.tt line 18
eval {...} called at /usr/share/perl5/SOAP/WSDL/Generator/Template/XSD/complexType.tt line 7
Template::Document::__ANON__('Template::Context=HASH(0xa171728)') called at /usr/lib/perl5/Template/Document.pm line 151
eval {...} called at /usr/lib/perl5/Template/Document.pm line 149
Template::Document::process('Template::Document=HASH(0xa157e18)', 'Template::Context=HASH(0xa171728)') called at /usr/lib/perl5/Template/Context.pm line 347
eval {...} called at /usr/lib/perl5/Template/Context.pm line 321
Template::Context::process('Template::Context=HASH(0xa171728)', 'Template::Document=HASH(0xa157e18)') called at /usr/lib/perl5/Template/Service.pm line 94
eval {...} called at /usr/lib/perl5/Template/Service.pm line 91
Template::Service::process('Template::Service=HASH(0xa170bd4)', 'complexType.tt', 'HASH(0xa157558)') called at /usr/lib/perl5/Template.pm line 66
Template::process('Template=HASH(0xa170da0)', 'complexType.tt', 'HASH(0xa157558)', 'MyTypes/AddressesInfoStruct.pm') called at /usr/share/perl5/SOAP/WSDL/Generator/Template.pm line 66
SOAP::WSDL::Generator::Template::_process('SOAP::WSDL::Generator::Template::XSD=SCALAR(0x9f2a628)', 'complexType.tt', 'HASH(0x9f38434)', 'MyTypes/AddressesInfoStruct.pm') called at /usr/share/perl5/SOAP/WSDL/Generator/Template/XSD.pm line 232
SOAP::WSDL::Generator::Template::XSD::visit_XSD_ComplexType('SOAP::WSDL::Generator::Template::XSD=SCALAR(0x9f2a628)', 'SOAP::WSDL::XSD::ComplexType=SCALAR(0x9b309d0)') called at /usr/share/perl5/SOAP/WSDL/Base.pm line 62
SOAP::WSDL::Base::_accept('SOAP::WSDL::XSD::ComplexType=SCALAR(0x9b309d0)', 'SOAP::WSDL::Generator::Template::XSD=SCALAR(0x9f2a628)') called at /usr/share/perl5/SOAP/WSDL/Generator/Template/XSD.pm line 90
SOAP::WSDL::Generator::Template::XSD::generate_typelib('SOAP::WSDL::Generator::Template::XSD=SCALAR(0x9f2a628)', undef) called at /usr/share/perl5/SOAP/WSDL/Generator/Template/XSD.pm line 76
SOAP::WSDL::Generator::Template::XSD::generate('SOAP::WSDL::Generator::Template::XSD=SCALAR(0x9f2a628)') called at /usr/bin/wsdl2perl line 151
at /usr/share/perl5/SOAP/WSDL/Base.pm line 62
What's wrong?
Basically, SOAP::WSDL is currently unable to parse WSDL, this is a bug that's already been logged here so I think the best option would be to try and use SOAP::Lite or create the WSDL classes manually?!
Related
In the recent VS Code release, they added this feature to view the active variables in the Jupyter Notebook and also, view the values in the variable with Data Viewer.
However, every time I am trying to view the values in Data Viewer, VS Code is throwing error below. It says that the reason is that the object of data type is Int64 and not string, but I am sure that should not be the reason to not show the variable. Anyone facing similar issues. I tried with a simple data frame and it's working fine.
Error: Failure during variable extraction:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-16-eae5f1f55b35> in <module>
97
98 # Transform this back into a string
---> 99 print(_VSCODE_json.dumps(_VSCODE_targetVariable))
100 del _VSCODE_targetVariable
101
~/anaconda3/lib/python3.7/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
229 cls is None and indent is None and separators is None and
230 default is None and not sort_keys and not kw):
--> 231 return _default_encoder.encode(obj)
232 if cls is None:
233 cls = JSONEncoder
~/anaconda3/lib/python3.7/json/encoder.py in encode(self, o)
197 # exceptions aren't as detailed. The list call should be roughly
198 # equivalent to the PySequence_Fast that ''.join() would do.
--> 199 chunks = self.iterencode(o, _one_shot=True)
200 if not isinstance(chunks, (list, tuple)):
201 chunks = list(chunks)
~/anaconda3/lib/python3.7/json/encoder.py in iterencode(self, o, _one_shot)
255 self.key_separator, self.item_separator, self.sort_keys,
256 self.skipkeys, _one_shot)
--> 257 return _iterencode(o, 0)
258
259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,
~/anaconda3/lib/python3.7/json/encoder.py in default(self, o)
177
178 """
--> 179 raise TypeError(f'Object of type {o.__class__.__name__} '
180 f'is not JSON serializable')
181
TypeError: Object of type int64 is not JSON serializable
This is a complex question with regard to mro.pm, and the interplay with set_subname, and goto
In troubleshooting a problem, I think the core of my misunderstanding relates to how mro.pm works -- especially with regard to set_subname.
What is the difference between these three constructs,
Plain call to set_subname
*Foo::bar = set_subname( 'Foo::bar', $codeRef );
Anon sub which wraps a set_subname
*Foo::bar = sub {
my $codeRef2 = set_subname('Foo::bar', $codeRef);
goto $codeRef2
};
Anon sub which has its name set with set_subname
*Foo::bar = set_subname(
'Foo::bar',
sub { goto $codeRef }
);
Specifically, the Mojo test suits fails with either of these modifications with anon subs applied to Mojo::Utils's monkey_patch Running the two variants above against t/mojo/websocket_proxy.t,
With the 2 (second) option I have
*{"${class}::$k"} = sub {
my $cr = set_subname("${class}::$k", $patch{$k});
goto $cr;
};
And I get
Mojo::Reactor::Poll: Timer failed: Can't locate object method "send" via package "Mojo::Transaction::HTTP" at t/mojo/websocket_proxy.t line 66.
With the 3 (third) option I have,
*{"${class}::$k"} = set_subname("${class}::$k", sub { goto $patch{$k} })
And I get
No next::method 'new' found for Mojolicious::Routes at /usr/lib/x86_64-linux-gnu/perl/5.28/mro.pm line 30.
Obviously, the first version works (it's from the code I linked), the question is why are the other two variants giving me different errors (especially the second variant) and what's happening there -- why don't they work?
Your second option is not working because the sub you are using as a wrapper does not match the prototype of the inner sub. monkey_patch is not only used for methods, and this changes how some functions are parsed. In particular, Mojo::Util::steady_time has an empty prototype and is often called without using parenthesis.
*{"${class}::$k"} = Sub::Util::set_prototype(
Sub::Util::prototype( $patch{$k} ),
Sub::Util::set_subname(
"${class}::$k",
sub {
my $cr = Sub::Util::set_subname("${class}::$k", $patch{$k});
goto $cr;
}
)
);
The third construct is not working because you are using goto to remove the renamed wrapper sub from the call stack, leaving only the inner sub which has no name. This breaks next::method's ability to find the correct method name.
That is indeed complicated, but maybe you overcomplicated it?
Remember that MRO is only concerned with locating a method, which is just a symbol table entry to a coderef, through a defined order of package names. The internal subname only has to do with what caller() reports AFAIK.
From: Mojo
*{"${class}::$_"} = ## symbol table entry
set_subname("${class}::$_", ## an internal name
$patch{$_}) ## for a code ref
for keys %patch;
HTH
Edit After Seeing Error Messages:
The subroutines have not been validly installed. I suspect that since in option 2 and 3 you are deferring the calls to set_subname() to call time, the coderef $patch{$k} never has a subname assigned to it and that breaks a link in the chain of mro::_nextcan()'s XS magic. Particularly if $patch{$k} calls next::method. The closures seem to be valid though.
Although I must say my testing seems to show that option 2 is valid.
Enter command: my ($class, $k) = qw/W S/; my %patch = (S =>
sub {print "patch here\n"; decall;}); *{"${class}::$k"} =
sub { print "goto'r here\n"; my $cr = set_subname("${class}::$k",
$patch{$k}); goto $cr;};
Enter command: decall
0 "console"
1 "console.pl"
2 "114"
3 "(eval)"
4 "0"
5 0
6 "package W; decall"
7 ""
8 "256"
9 "\020\001\000\000\000P\004\000\000\000\000\000\000U\025U\005"
10 0
Enter command: S
goto'r here
patch here
0 "W"
1 "(eval 110)"
2 "1"
3 "W::S"
4 "1"
5 0
6 0
7 0
8 "256"
9 "\020\001\000\000\000P\004\000\000\000\000\000\000U\025U\005"
10 0
You might have to start looking farther afield for the problem with option 2.
After modifying Mojo/Util.pm with
foreach my $k (keys %patch) {
*{"${class}::$k"} = sub {
my $cr = set_subname("${class}::$k", $patch{$k});
goto $cr;
};
}
and isolating the test case, I get:
$ perl -MCarp::Always t/mojo/websocket_proxy2.t
Mojo::Reactor::Poll: Timer failed: Can't locate object method "send" via package "Mojo::Transaction::HTTP" at t/mojo/websocket_proxy2.t line 59.
main::__ANON__(Mojo::UserAgent=HASH(0x60280dad0), Mojo::Transaction::HTTP=HASH(0x6029ee7b0)) called at blib/lib/Mojo/UserAgent.pm line 252
Mojo::UserAgent::_finish(Mojo::UserAgent=HASH(0x60280dad0), "927210c53042c6142eda3f4010c8b17c", 1) called at blib/lib/Mojo/UserAgent.pm line 220
Mojo::UserAgent::_error(Mojo::UserAgent=HASH(0x60280dad0), "927210c53042c6142eda3f4010c8b17c", "Connect timeout") called at blib/lib/Mojo/UserAgent.pm line 128
Mojo::UserAgent::__ANON__(Mojo::IOLoop=HASH(0x601f9abb8), "Connect timeout", undef) called at blib/lib/Mojo/IOLoop.pm line 63
Mojo::IOLoop::__ANON__(Mojo::IOLoop::Client=HASH(0x601e34598)) called at blib/lib/Mojo/EventEmitter.pm line 15
Mojo::EventEmitter::emit(Mojo::IOLoop::Client=HASH(0x601e34598), "error", "Connect timeout") called at blib/lib/Mojo/IOLoop/Client.pm line 39
Mojo::IOLoop::Client::__ANON__(Mojo::Reactor::Poll=HASH(0x6001a8390)) called at blib/lib/Mojo/Reactor/Poll.pm line 143
eval {...} called at blib/lib/Mojo/Reactor/Poll.pm line 143
Mojo::Reactor::Poll::_try(Mojo::Reactor::Poll=HASH(0x6001a8390), "Timer", CODE(0x601e24ca0)) called at blib/lib/Mojo/Reactor/Poll.pm line 81
Mojo::Reactor::Poll::one_tick(Mojo::Reactor::Poll=HASH(0x6001a8390)) called at blib/lib/Mojo/Reactor/Poll.pm line 99
Mojo::Reactor::Poll::start(Mojo::Reactor::Poll=HASH(0x6001a8390)) called at blib/lib/Mojo/IOLoop.pm line 134
Mojo::IOLoop::start("Mojo::IOLoop") called at t/mojo/websocket_proxy2.t line 62
at blib/lib/Mojo/IOLoop.pm line 23.
Mojo::IOLoop::__ANON__(Mojo::Reactor::Poll=HASH(0x6001a8390), "Timer failed: Can't locate object method \"send\" via package \"Mojo::Transaction::HTTP\" at t/mojo/websocket_proxy2.t line 59.\x{a}\x{9}main::__ANON__(Mojo::UserAgent=HASH(0x60280dad0), Mojo::Transaction::HTTP=HASH(0x6029ee7b0)) called at blib/lib/Mojo/UserAgent.pm line 252\x{a}\x{9}Mojo::UserAgent::_finish(Mojo::UserAgent=HASH(0x60280dad0), \"927210c53042c6142eda3f4010c8b17c\", 1) called at blib/lib/Mojo/UserAgent.pm line 220\x{a}\x{9}Mojo::UserAgent::_error(Mojo::UserAgent=HASH(0x60280dad0), \"927210c53042c6142eda3f4010c8b17c\", \"Connect timeout\") called at blib/lib/Mojo/UserAgent.pm line 128\x{a}\x{9}Mojo::UserAgent::__ANON__(Mojo::IOLoop=HASH(0x601f9abb8), \"Connect timeout\", undef) called at blib/lib/Mojo/IOLoop.pm line 63\x{a}\x{9}Mojo::IOLoop::__ANON__(Mojo::IOLoop::Client=HASH(0x601e34598)) called at blib/lib/Mojo/EventEmitter.pm line 15\x{a}\x{9}Mojo::EventEmitter::emit(Mojo::IOLoop::Client=HASH(0x601e34598), \"error\", \"Connect timeout\") called at blib/lib/Mojo/IOLoop/Client.pm line 39\x{a}\x{9}Mojo::IOLoop::Client::__ANON__(Mojo::Reactor::Poll=HASH(0x6001a8390)) called at blib/lib/Mojo/Reactor/Poll.pm line 143\x{a}\x{9}eval {...} called at blib/lib/Mojo/Reactor/Poll.pm line 143\x{a}\x{9}Mojo::Reactor::Poll::_try(Mojo::Reactor::Poll=HASH(0x6001a8390), \"Timer\", CODE(0x601e24ca0)) called at blib/lib/Mojo/Reactor/Poll.pm line 81\x{a}\x{9}Mojo::Reactor::Poll::one_tick(Mojo::Reactor::Poll=HASH(0x6001a8390)) called at blib/lib/Mojo/Reactor/Poll.pm line 99\x{a}\x{9}Mojo::Reactor::Poll::start(Mojo::Reactor::Poll=HASH(0x6001a8390)) called at blib/lib/Mojo/IOLoop.pm line 134\x{a}\x{9}Mojo::IOLoop::start(\"Mojo::IOLoop\") called at t/mojo/websocket_proxy2.t line 62\x{a}") called at blib/lib/Mojo/EventEmitter.pm line 15
Mojo::EventEmitter::emit(Mojo::Reactor::Poll=HASH(0x6001a8390), "error", "Timer failed: Can't locate object method \"send\" via package \"Mojo::Transaction::HTTP\" at t/mojo/websocket_proxy2.t line 59.\x{a}\x{9}main::__ANON__(Mojo::UserAgent=HASH(0x60280dad0), Mojo::Transaction::HTTP=HASH(0x6029ee7b0)) called at blib/lib/Mojo/UserAgent.pm line 252\x{a}\x{9}Mojo::UserAgent::_finish(Mojo::UserAgent=HASH(0x60280dad0), \"927210c53042c6142eda3f4010c8b17c\", 1) called at blib/lib/Mojo/UserAgent.pm line 220\x{a}\x{9}Mojo::UserAgent::_error(Mojo::UserAgent=HASH(0x60280dad0), \"927210c53042c6142eda3f4010c8b17c\", \"Connect timeout\") called at blib/lib/Mojo/UserAgent.pm line 128\x{a}\x{9}Mojo::UserAgent::__ANON__(Mojo::IOLoop=HASH(0x601f9abb8), \"Connect timeout\", undef) called at blib/lib/Mojo/IOLoop.pm line 63\x{a}\x{9}Mojo::IOLoop::__ANON__(Mojo::IOLoop::Client=HASH(0x601e34598)) called at blib/lib/Mojo/EventEmitter.pm line 15\x{a}\x{9}Mojo::EventEmitter::emit(Mojo::IOLoop::Client=HASH(0x601e34598), \"error\", \"Connect timeout\") called at blib/lib/Mojo/IOLoop/Client.pm line 39\x{a}\x{9}Mojo::IOLoop::Client::__ANON__(Mojo::Reactor::Poll=HASH(0x6001a8390)) called at blib/lib/Mojo/Reactor/Poll.pm line 143\x{a}\x{9}eval {...} called at blib/lib/Mojo/Reactor/Poll.pm line 143\x{a}\x{9}Mojo::Reactor::Poll::_try(Mojo::Reactor::Poll=HASH(0x6001a8390), \"Timer\", CODE(0x601e24ca0)) called at blib/lib/Mojo/Reactor/Poll.pm line 81\x{a}\x{9}Mojo::Reactor::Poll::one_tick(Mojo::Reactor::Poll=HASH(0x6001a8390)) called at blib/lib/Mojo/Reactor/Poll.pm line 99\x{a}\x{9}Mojo::Reactor::Poll::start(Mojo::Reactor::Poll=HASH(0x6001a8390)) called at blib/lib/Mojo/IOLoop.pm line 134\x{a}\x{9}Mojo::IOLoop::start(\"Mojo::IOLoop\") called at t/mojo/websocket_proxy2.t line 62\x{a}") called at blib/lib/Mojo/Reactor/Poll.pm line 143
Mojo::Reactor::Poll::_try(Mojo::Reactor::Poll=HASH(0x6001a8390), "Timer", CODE(0x601e24ca0)) called at blib/lib/Mojo/Reactor/Poll.pm line 81
Mojo::Reactor::Poll::one_tick(Mojo::Reactor::Poll=HASH(0x6001a8390)) called at blib/lib/Mojo/Reactor/Poll.pm line 99
Mojo::Reactor::Poll::start(Mojo::Reactor::Poll=HASH(0x6001a8390)) called at blib/lib/Mojo/IOLoop.pm line 134
Mojo::IOLoop::start("Mojo::IOLoop") called at t/mojo/websocket_proxy2.t line 62
I can also confirm that setting the prototype fixes it.
I'm using IPython as a system shell.
Is there a way to make IPython to raise an exception when the shell command fails? (non-zero exit codes)
The default makes them fail silently.
As of IPython 4.0.1, !cmd is transformed into get_ipython().system(repr(cmd)) (IPython.core.inputtransformer._tr_system()).
In the source, it's actually InteractiveShell.system_raw(), as inspect.getsourcefile() and inspect.getsource() can tell.
It delegates to os.system() in Windows and subprocess.call() in other OSes. Not configurable, as you can see from the code.
So, you need to replace it with something that would call subprocess.check_call().
Apart from monkey-patching by hand, this can be done with the IPython configuration system. Available options (viewable with the %config magic) don't allow to replace TerminalInteractiveShell with another class but several TerminalIPythonApp options allow to execute code on startup.
Do double-check whether you really need this though: a look through the system_raw()'s source reveals that it sets the _exit_code variable - so it doesn't actually fail completely silently.
If you use ! to execute shell commands, errors will pass silently
!echo "hello" && exit 1
hello
If you use the %%sh cell magic to execute the shell command, errors will raise:
%%sh
echo "hello" && exit 1
hello
---------------------------------------------------------------------------
CalledProcessError Traceback (most recent call last)
<ipython-input-10-9229f76cae28> in <module>
----> 1 get_ipython().run_cell_magic('sh', '', 'echo "hello" && exit 1\n')
~/anaconda/envs/altair-dev/lib/python3.6/site-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
2360 with self.builtin_trap:
2361 args = (magic_arg_s, cell)
-> 2362 result = fn(*args, **kwargs)
2363 return result
2364
~/anaconda/envs/altair-dev/lib/python3.6/site-packages/IPython/core/magics/script.py in named_script_magic(line, cell)
140 else:
141 line = script
--> 142 return self.shebang(line, cell)
143
144 # write a basic docstring:
<decorator-gen-110> in shebang(self, line, cell)
~/anaconda/envs/altair-dev/lib/python3.6/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
185 # but it's overkill for just that one bit of state.
186 def magic_deco(arg):
--> 187 call = lambda f, *a, **k: f(*a, **k)
188
189 if callable(arg):
~/anaconda/envs/altair-dev/lib/python3.6/site-packages/IPython/core/magics/script.py in shebang(self, line, cell)
243 sys.stderr.flush()
244 if args.raise_error and p.returncode!=0:
--> 245 raise CalledProcessError(p.returncode, cell, output=out, stderr=err)
246
247 def _run_script(self, p, cell, to_close):
CalledProcessError: Command 'b'echo "hello" && exit 1\n'' returned non-zero exit status 1.
In the IPython and Jupyter documentation it says that get_ipython().magic() is deprecated. But when I changed my code to use run_line_magic it is failing to push to R (see below). Might be related to this problem
https://bitbucket.org/rpy2/rpy2/issues/184/valueerror-call-stack-is-not-deep-enough
I'm on Mac Yosemite, using Anaconda with Python 2.7. I just updated both Anaconda and rpy2 yesterday. The code below is from a Jupyter notebook.
%load_ext rpy2.ipython
import pandas as pd
'''Two test functions with rpy2.
The only difference between them is that
rpy2fun_magic uses 'magic' to push variable to R and
rpy2fun_linemagic uses 'run_line_magic' to push variable.
'magic' works fine. 'run_line_magic' returns an error.'''
def rpy2fun_magic(df):
get_ipython().magic('R -i df')
get_ipython().run_line_magic('R','df_cor <- cor(df)')
get_ipython().run_line_magic('R','-o df_cor')
return (df_cor)
def rpy2fun_linemagic(df):
get_ipython().run_line_magic('R','-i df')
get_ipython().run_line_magic('R','df_cor <- cor(df)')
get_ipython().run_line_magic('R','-o df_cor')
return (df_cor)
dataframetest = pd.DataFrame([[1,2,3,4],[6,3,4,5],[9,1,7,3]])
df_cor_magic = rpy2fun_magic(dataframetest)
print 'Using magic to push variable works fine\n'
print df_cor_magic
print '\nBut using run_line_magic returns an error\n'
df_cor_linemagic = rpy2fun_linemagic(dataframetest)
Using magic to push variable works fine
[[ 1. -0.37115374 0.91129318 -0.37115374]
[-0.37115374 1. -0.72057669 1. ]
[ 0.91129318 -0.72057669 1. -0.72057669]
[-0.37115374 1. -0.72057669 1. ]]
But using run_line_magic returns an error
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-1-e418b72a8621> in <module>()
28 print '\nBut using run_line_magic returns an error\n'
29
---> 30 df_cor_linemagic = rpy2fun_linemagic(dataframetest)
<ipython-input-1-e418b72a8621> in rpy2fun_linemagic(df)
15
16 def rpy2fun_linemagic(df):
---> 17 get_ipython().run_line_magic('R','-i df')
18 get_ipython().run_line_magic('R','df_cor <- cor(df)')
19 get_ipython().run_line_magic('R','-o df_cor')
/Users/alexmillner/anaconda/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in run_line_magic(self, magic_name, line)
2255 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
2256 with self.builtin_trap:
-> 2257 result = fn(*args,**kwargs)
2258 return result
2259
/Users/alexmillner/anaconda/lib/python2.7/site-packages/rpy2/ipython/rmagic.pyc in R(self, line, cell, local_ns)
/Users/alexmillner/anaconda/lib/python2.7/site-packages/IPython/core/magic.pyc in <lambda>(f, *a, **k)
191 # but it's overkill for just that one bit of state.
192 def magic_deco(arg):
--> 193 call = lambda f, *a, **k: f(*a, **k)
194
195 if callable(arg):
/Users/alexmillner/anaconda/lib/python2.7/site-packages/rpy2/ipython/rmagic.pyc in R(self, line, cell, local_ns)
657 val = self.shell.user_ns[input]
658 except KeyError:
--> 659 raise NameError("name '%s' is not defined" % input)
660 if args.converter is None:
661 ro.r.assign(input, self.pyconverter(val))
NameError: name 'df' is not defined
Some discussion of the same issue with %timeit first, followed by workaround answers at the bottom. I'm using IPython 3.1.0 with Anaconda 2.7.10, so my observations below could be different based on version differences alone.
This is not unique to the R extension, you can reproduce this with something simpler like %timeit:
In [47]: dfrm
Out[47]:
A B C
0 0.690466 0.370793 0.963782
1 0.478427 0.358897 0.689173
2 0.189277 0.268237 0.570624
3 0.735665 0.342549 0.509810
4 0.929736 0.090079 0.384444
5 0.210941 0.347164 0.852408
6 0.241940 0.187266 0.961489
7 0.768143 0.548450 0.604004
8 0.055765 0.842224 0.668782
9 0.717827 0.047011 0.948673
In [48]: def run_timeit(df):
get_ipython().run_line_magic('timeit', 'df.sum()')
....:
In [49]: run_timeit(dfrm)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-49-1e62302232b6> in <module>()
----> 1 run_timeit(dfrm)
<ipython-input-48-0a3e09ec1e0c> in run_timeit(df)
1 def run_timeit(df):
----> 2 get_ipython().run_line_magic('timeit', 'df.sum()')
3
/home/ely/anaconda/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in run_line_magic(self, magic_name, line)
2226 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
2227 with self.builtin_trap:
-> 2228 result = fn(*args,**kwargs)
2229 return result
2230
/home/ely/anaconda/lib/python2.7/site-packages/IPython/core/magics/execution.pyc in timeit(self, line, cell)
/home/ely/anaconda/lib/python2.7/site-packages/IPython/core/magic.pyc in <lambda>(f, *a, **k)
191 # but it's overkill for just that one bit of state.
192 def magic_deco(arg):
--> 193 call = lambda f, *a, **k: f(*a, **k)
194
195 if callable(arg):
/home/ely/anaconda/lib/python2.7/site-packages/IPython/core/magics/execution.pyc in timeit(self, line, cell)
1034 number = 1
1035 for _ in range(1, 10):
-> 1036 time_number = timer.timeit(number)
1037 worst_tuning = max(worst_tuning, time_number / number)
1038 if time_number >= 0.2:
/home/ely/anaconda/lib/python2.7/site-packages/IPython/core/magics/execution.pyc in timeit(self, number)
130 gc.disable()
131 try:
--> 132 timing = self.inner(it, self.timer)
133 finally:
134 if gcold:
<magic-timeit> in inner(_it, _timer)
NameError: global name 'df' is not defined
The issue is that the line magics are set to look for variable names at global scope, not at function scope. If the argument to your function rpy2fun_linemagic happened to coincide with a global variable name, the interior code would pick that up, for example:
In [52]: def run_timeit(dfrm):
get_ipython().run_line_magic('timeit', 'dfrm.sum()')
....:
In [53]: run_timeit(dfrm)
The slowest run took 5.67 times longer than the fastest. This could mean that an intermediate result is being cached
10000 loops, best of 3: 99.1 µs per loop
But this only works by accident, because the interior string passed to run_line_magic contains a name that is found globally.
However, I do get the same error even if using the plain magic function:
In [58]: def run_timeit(df):
get_ipython().magic('timeit df.sum()')
....:
In [59]: run_timeit(dfrm)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-59-1e62302232b6> in <module>()
----> 1 run_timeit(dfrm)
<ipython-input-58-e98c720ea7e8> in run_timeit(df)
1 def run_timeit(df):
----> 2 get_ipython().magic('timeit df.sum()')
3
/home/ely/anaconda/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in magic(self, arg_s)
2305 magic_name, _, magic_arg_s = arg_s.partition(' ')
2306 magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
-> 2307 return self.run_line_magic(magic_name, magic_arg_s)
2308
2309 #-------------------------------------------------------------------------
/home/ely/anaconda/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in run_line_magic(self, magic_name, line)
2226 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
2227 with self.builtin_trap:
-> 2228 result = fn(*args,**kwargs)
2229 return result
2230
/home/ely/anaconda/lib/python2.7/site-packages/IPython/core/magics/execution.pyc in timeit(self, line, cell)
/home/ely/anaconda/lib/python2.7/site-packages/IPython/core/magic.pyc in <lambda>(f, *a, **k)
191 # but it's overkill for just that one bit of state.
192 def magic_deco(arg):
--> 193 call = lambda f, *a, **k: f(*a, **k)
194
195 if callable(arg):
/home/ely/anaconda/lib/python2.7/site-packages/IPython/core/magics/execution.pyc in timeit(self, line, cell)
1034 number = 1
1035 for _ in range(1, 10):
-> 1036 time_number = timer.timeit(number)
1037 worst_tuning = max(worst_tuning, time_number / number)
1038 if time_number >= 0.2:
/home/ely/anaconda/lib/python2.7/site-packages/IPython/core/magics/execution.pyc in timeit(self, number)
130 gc.disable()
131 try:
--> 132 timing = self.inner(it, self.timer)
133 finally:
134 if gcold:
<magic-timeit> in inner(_it, _timer)
NameError: global name 'df' is not defined
One (super bad) way to get around this is to use globals to locate the item that is the same as the argument that was passed to your function, and then you'll have a global name for it.
For example:
In [68]: def run_timeit(df):
for var_name, var_val in globals().iteritems():
if df is var_val:
get_ipython().run_line_magic('timeit', '%s.sum()'%(var_name))
break
....:
In [69]: run_timeit(dfrm)
The slowest run took 5.72 times longer than the fastest. This could mean that an intermediate result is being cached
10000 loops, best of 3: 99.2 µs per loop
But this is very unstable, since it relies on pass-by-name in Python. If I passed an object like an integer or string, I would have to check whether it was interned or something, but otherwise couldn't find it "by name" in the global namespace.
Another way to do it that might be slightly better is to use the user_ns namespace dict that IPython stores. Then at least you're not looking at globals, and there is more stability over specific variables that have been named when assigned by the user in IPython:
In [71]: def run_timeit(df):
....: g = get_ipython()
....: for var_name, var_val in g.user_ns.iteritems():
....: if df is var_val:
....: g.run_line_magic('timeit', '%s.sum()'%(var_name))
....: break
....:
In [72]: run_timeit(dfrm)
The slowest run took 5.58 times longer than the fastest. This could mean that an intermediate result is being cached
10000 loops, best of 3: 99 µs per loop
In the case of your specific R function call, I would try:
def rpy2fun_linemagic(df):
g = get_ipython()
for var_name, var_val in g.user_ns.iteritems():
if df is var_val:
g.run_line_magic('R', '-i %s'%(var_name))
g.run_line_magic('R', 'df_cor <- cor(%s)'%(var_name))
g.run_line_magic('R', '-o df_cor')
return df_cor
You might also have to be careful on the return statement. You might need to use return g.user_ns['df_cor'] or something if the result of the output conversion back to Python is to create the variable at global scope as well, rather than function scope. Or, if that variable gets created as a side effect, you may not want to return anything at all. I'm not a big fan of relying on implicit mutation like that, but it could work for you.
I suspect that the code example you are providing is there only to demonstrate the issue with run_line_magic(), but for reference I am adding a way to do the same without ipython being involved.
from rpy2.robjects import globalenv
def rpy2cor(df):
fun = globalenv.get('cor', wantfun=True)
df_cor = fun(df)
return df_cor
A library that I'm using XML::Compile::Translate::Reader calls Log::Report's error method
or error __x"data for element or block starting with `{tag}' missing at {path}"
, tag => $label, path => $path, _class => 'misfit';
As I've got Log::Report set to debug mode, it returns a stack trace for an error.
[11 07 2014 22:17:39] [2804] error: data for element or block starting with `MSISDN' missing at {http://www.sigvalue.com/acc}TA
at /usr/local/share/perl5/XML/Compile/Translate/Reader.pm line 476
Log::Report::error("Log::Report::Message=HASH(0x2871cf8)") at /usr/local/share/perl5/XML/Compile/Translate/Reader.pm line 476
<snip>
XML::Compile::SOAP::Daemon::LWPutil::lwp_run_request("HTTP::Request=HASH(0x2882858)", "CODE(0x231ba38)", "HTTP::Daemon::ClientConn::SSL=GLOB(0x231b9c0)", undef) at /usr/local/share/perl5/XML/Compile/SOAP/Daemon/LWPutil.pm line 95
Any::Daemon::run("XML::Compile::SOAP::Daemon::AnyDaemon=HASH(0x7a3168)", "child_task", "CODE(0x2548128)", "max_childs", 36, "background", 1) at /usr/local/share/perl5/XML/Compile/SOAP/Daemon/AnyDaemon.pm line 75
XML::Compile::SOAP::Daemon::AnyDaemon::_run("XML::Compile::SOAP::Daemon::AnyDaemon=HASH(0x7a3168)", "HASH(0x18dda00)") at /usr/local/share/perl5/XML/Compile/SOAP/Daemon.pm line 99
(eval)("XML::Compile::SOAP::Daemon::AnyDaemon=HASH(0x7a3168)", "HASH(0x18dda00)") at /usr/local/share/perl5/XML/Compile/SOAP/Daemon.pm line 94
XML::Compile::SOAP::Daemon::run("XML::Compile::SOAP::Daemon::AnyDaemon=HASH(0x7a3168)", "name", "rizserver.pl", "background", 1, "max_childs", 36, "socket", [7 more]) at ./rizserver.pl line 95
There is lots of juicy data in those HASH, SCALAR, GLOB, and other elements that I want to get logged; as we are having trouble logging the original request in case it doesn't match.
I've explored using
Some leads that I don't know how to use are using Log::Dispatch, or some sort of Filter on Log::Report; but in the end, all I really want is to apply Data::Dumper to those elements.