Question is in the title.
I know that % usually denotes a "magic variable" in IPython. That's not a concept I'm terribly familiar with yet, but I have read about it.
However, today I saw a tutorial where someone was using it to run a shell command. Normally I have seen and used !.
Is there a difference? Both seem to be doing the same thing when I try them.
The difference is this:
When you run a command with !, it directly executes a bash command in a subshell.
When you run a command with %, it executes one of the magic commands defined in IPython.
Some of the magic commands defined by IPython deliberately mirror bash commands, but they differ in the implementation details.
For example, running the !cd bash command does not persistently change your directory, because it runs in a temporary subshell. However, running the %cd magic command will persistently change your directory:
!pwd
# /content
!cd sample_data/
!pwd
# /content
%cd sample_data/
!pwd
# /content/sample_data
Read more in IPython: Built-in Magic Commands.
Related
This question already has an answer here:
In Jupyter Notebooks on Google Colab, what's the difference between using % and ! to run a shell command?
(1 answer)
Closed 1 year ago.
I know that using ! before a command in python runs the command using the terminal.
For example:
!ls
!unzip zippedfile.zip
However, I noticed that using % also works for some bash commands but not for others.
So
%ls
will work, but
%unzip
will NOT work.
What is the difference between these two prefixes?
The ! escape runs an external shell command, like ping -c 3 www.google.com, not a Python command. Python or ipython has no idea what ping does, it just passes over control to it, and displays its output.
The % escape runs an ipython built-in command or extension, i.e. something that ipython itself understands.
To quote the documentation,
User-extensible ‘magic’ commands. A set of commands prefixed with % or %% is available for controlling IPython itself and provides directory control, namespace information and many aliases to common system shell commands.
The source of confusion here is probably that e.g. ls is also available as a "magic" command for portability and convenience. (It's portable in that it works even on platforms where there is no system ls command, like Windows.)
I am writing a perl script, and I want to run a simple shell command to use a certain version of NVM:
Here is my code snippet:
print "\n*** Switching to correct nvm environment for dashboard builds\n";
system("nvm use 8.12.0") == 0 or die $?;
But I am getting the following error:
Can't exec "nvm": No such file or directory
Can someone help?
Update (June 30, 2021):
I also tried adding the command:
my $nvm_version = "8.12.0";
system ("bash", "-lic", "nvm use $nvm_version");
But nothing happens:
I'm not familiar with nwm, but I think I get the gist of what it does. And if so, the attempt is fundamentally flawed. Even if you fixed this to run the proper shell so that nvm could run, I believe all the tool does is change the shell's environment variables, a shell you immediately exit. This means it would have no effect even if if it ran successfully.
Again, it this tool does what I think it does, such tool are meant to be used in interactive shells. In other instances, you simply use the path the to correct executable instead of relying on the PATH.
With that in mind, you can use the following to run the command in bash:
# Non-interactive shell.
system("bash", "-c", "nvm use 8.12.0")
or
# Interactive shell.
# This is improper and fragile as interactive shells
# often create aliases that override basic commands.
system("bash", "-ic", "nvm use 8.12.0")
Just to reiterate, at least one of these will allow the command to run (if it normally works from bash), but I believe it's unlikely this will produce the results you expect.
The nvm command is shell function which is different from a shell command. Also the nvm command is not an exported function so it will not be seen by sub shells. For example, in Bash shell:
$ nvm ls
-> v15.0.1
$ my-test-script.sh
./my-test-script.sh: line 3: nvm: command not found
where my-test-script.sh is:
#! /bin/bash
nvm use 16.4
The error nvm: command not found is because nvm is not exported. I can source the script in the current shell context to make it work:
$ source my-test-script.sh
Now using node v16.4.0 (npm v7.18.1)
$ node --version
v16.4.0
So a Perl script cannot change the node version of the current shell, but it can calculate the version and pass it back to shell, which can set the version. For example:
$ nvm use $(perl -E'$v=15.0; print $v')
Now using node v15.0.1 (npm v7.0.3)
(Note: I have plenty of python and unix shell experience, but fairly new to ipython -- using 7.5)
I'm trying to replicate a UNIX shell function that I use all the time, so that it works in the ipython shell.
The requirement is that I want to type something like to myproj, and then have ipython process the resulting text by doing a cd to the directory that comes back from to. (This is a quick-directory-change utility I use in unix)
The way it works in unix is that a shell function invokes an external command, that command prints its result to stdout, and the shell function then invokes the internal cd to the target dir.
I've been trying to wrap my head around %magic and macros and aliases in ipython, but so far I don't see how to get this done. Any ideas?
How can I put a comment at the end of an IPython magic command?
Eg, trying to use %cd:
%cd "dir" # comment
I want to change to directory dir but instead I get:
[Errno 2] No such file or directory: 'dir # comment'
It's not possible in general because IPython magic commands parse their own command lines.*
There are some magic commands where comments seem to work, but actually they're being handled in another way. For example:
%ls # comment works properly, but %ls is an alias to a shell command, so the shell is what's parsing the comment.
%automagic 0 # comment behaves like %automagic, toggling the setting instead of exclusively disabling it like %automagic 0 would do. It's as if the 0 # comment is all one string, which it ignores.
%run $filename # comment treats the comment as arguments, so ['#', 'comment'] ends up in sys.argv for $filename.
Workaround
You could call magic commands using exclusively Python syntax.
First get the running IPython instance with get_ipython():
ipython = get_ipython()
Then:
ipython.run_line_magic('cd', 'dir') # comment
Docs: InteractiveShell.run_line_magic()
Or for cell magics, you could go from this (which doesn't work):
%%script bash # comment
echo hello
to this:
ipython.run_cell_magic('script', 'bash', 'echo hello') # comment
Docs: InteractiveShell.run_cell_magic()
Another option, I suppose, is to run an entire cell, although, suppressing the output since it returns an ExecutionResult object:
ipython.run_cell('%cd dir'); # comment
Docs: InteractiveShell.run_cell()
* I'm not sure if this is documented explicitly, but it is documented implicitly under defining custom magics, and I found the clues in this GitHub issue: Allow comments after arguments in magic_arguments.
TL;DR: You cannot as things stand. This might or might not be an error in the implementation of %cd.
Until this is fixed, one way to avoid this is to use:
import os
os.chdir("dir") #some comment here
A second alternative is using bash commands.
However % is a magic command, not equivalent to a bash command. This is on purpose,
as it can change the current directory of the notebook.
This is not the same as e.g.
!cd dir #some comment here
Which will spawn a shell and execute the command there thus not changing the current directory. (You can verify using pwd before/after each command)
Note that if your goal is not to change the current jupyter notebook directory, you can issue multiple commands in one cell with the magic
%sh:
%%sh
cd dir #some comment here
ls #some more commands here
....
This command will spawn a shell and all bash commands will be executed there, so the current jupyter directory will not change.
I have a script that opens up different files at the same time.
My problem here is that when I run system() on the alias I've defined in bash which points to /home/user1/Software/nc, perl tells me that it can't execute the alias because there is no file/directory at the current location.
I know that the alias works because when I invoke it directly in a shell, it opens fine.
Funny enough, I can do system("firefox") within my script fine, but not the alias. How do I use this alias in this script without it breaking?
Perl won't run bash, it will execute the command directly. You can call
bash -c your_command
instead of calling the command itself in Perl.
As it is, this doesn't load your aliases. You need to open an interactive shell, as in #MortezaLSC's answer. There supposedly is a way of loading aliases correctly in a non-interactive shell, but I can't figure it out.
But why don't you just use the command you have aliased to directly in perl? The only reason I could see not to do this, is if your alias is going to change in the future, but you will still want to run whatever command it points to. This seems weird and dangerous to say the least.
Aliases are designed to reduce the typing you do if you invoke commands with the same options etc all the time. They're not all-purpose macros for bash. Bash has functions for doing more complicated stuff, but why would you want to call non-trivial bash code from a perl script? It doesn't seem like you really need this here. Keep the complexity, and the potential for modification and failure, in one place (the perl script).
See a couple of answers to similar questions:
https://unix.stackexchange.com/a/1499/41977
https://superuser.com/a/183980/187150
If you're smart, you made it so your alias is only defined for interactive shells, so you'll have to launch bash and specify that you want an interactive shell using -i.
system('bash', '-i', '-c', 'shell command');
Is it working?
system 'bash -i -c "your alias parameter"';