I'm trying to set a git hash value into an environment variable, i thought it would be as simple as doing this:
git log --oneline -1 | export GIT_HASH=$1
But the $1 doesn't contain anything. What am I doing wrong?
$1 is used to access the first argument in a script or a function. It is not used to access output from an earlier command in a pipeline.
You can use command substitution to get the output of the git command into an environment variable like this:
GIT_HASH=`git log --oneline -1` && export GIT_HASH
However...
This answer is specially in response to the question regarding the Bourne Shell and it is the most widely supported. Your shell (e.g. GNU Bash) will most likely support the $() syntax and so you should also consider Michael Rush's answer.
But some shells, like tcsh, do not support the $() syntax and so if you're writing a shell script to be as bulletproof as possible for the greatest number of systems then you should use the `` syntax despite the limitations.
Or, you can also do it using $(). (see What is the benefit of using $() instead of backticks shell scripts?)
For example:
export FOO_BACKWARDS=$(echo 'foo' | rev)
You can use an external file as a temporary variable:
TMPFILE=/var/tmp/mark-unsworth-bashvar-1
git log --oneline -1 >$TMPFILE; export GIT_HASH=$(cat $TMPFILE); rm $TMPFILE
echo GIT_HASH is $GIT_HASH
Related
This was asked here, but no solution was provided for PowerShell. The solution given does not work in PowerShell (specifically, PowerShell 5.1 within VSCode).
I have tried
git config --global mergetool.vscode.cmd '"code --wait $MERGED"'
but I lose the double-quotes (i.e. there are no double-quotes in the corresponding .gitconfig file). FYI, the single-quotes are necessary to pass $MERGED as a string literal and not have PowerShell attempt to expand it.
I've also tried
echo '"code --wait $MERGED"' | git config --global mergetool.vscode.cmd
git config --global mergetool.vscode.cmd '`"code --wait $MERGED`"'
git config --global mergetool.vscode.cmd #'
code --wait $MERGED
'#
but nothing works. Is there no way to do this from within PowerShell?
Additionally, I've researched this question, and the solution of
git config --global mergetool.vscode.cmd '\"code --wait $MERGED\"'
nor
git config --global mergetool.vscode.cmd "\"code --wait $MERGED\""
does not work either.
The closest I get is using
git config --global mergetool.vscode.cmd '" code --wait $MERGED "'
but this outputs the spaces in the string as such:
[mergetool "vscode"]
cmd = " code --wait $MERGED "
Unfortunately, an obscure workaround is required (see the bottom section for a generic solution via an installable module):
git config --global mergetool.vscode.cmd --% "\"code --wait $MERGED\""
Note: This assumes that you want to end up with the following in ~/.gitconfig, which probably isn't your intent:
[mergetool "vscode"]
cmd = \"code --wait $MERGED\"
Instead, as one of the answers you link to suggests, only the $MERGED part should be double-quoted with embedded double quotes, in which case you should use (see the alternative at the bottom):
git config --global mergetool.vscode.cmd --% "code --wait \"$MERGED\""
That would give you:
[mergetool "vscode"]
cmd = code --wait \"$MERGED\"
That is, only those individual arguments in the shell command that your value constitutes that need double-quoting for the shell's sake must be enclosed in "...".
Note that it is git that re-escapes verbatim " chars. in the given configuration value as \" in the config file. Unescaped " in the config file - which have syntactic function for git itself - are only used if you pass a value with leading and/or trailing spaces, so as to delimit the value as a whole; values with (interior) spaces do not themselves require manual double-quoting: git stores them as-is in the config file, except for escaping embedded \ as \\ and " as \".
Passing --% "code --wait \"$MERGED\"" from PowerShell signals the intent to pass verbatim value code --wait "$MERGED" to git config --global mergetool.vscode.cmd, which is the proper formulation of a sh command line[1] (sh is a POSIX-compatible shell that git uses even on Windows, via a bash implementation that ships with it) in which the "..." around (environment) variable reference $MERGED ensure that the expanded value is passed as-is to the target binary, code, even if it contains spaces, for instance.
You can verify this by querying the value afterwards:
# After running the command above:
PS> git config --global mergetool.vscode.cmd
code --wait "$MERGED"
git config and the configuration file format are documented here, also available locally by running git help config or, alternatively, on Unix-like platforms only, man git-config
Explanation of the workaround:
Use of --%, the stop-parsing symbol, allows you to control the exact quoting of the subsequent arguments passed to external programs (while also suppressing expansion of PowerShell variables, so that $MERGED is left as-is), whereas PowerShell by default performs re-quoting behind the scenes, after having performed its own parsing.
Leaving aside that double-quoting the entire command line is ultimately not the right approach, purely from a PowerShell syntax perspective what you tried, '"code --wait $MERGED"', should work - PowerShell should automatically translate that to "\"code --wait $MERGED\"" behind the scenes - but as of PowerShell 7.1 doesn't, due to a long-standing bug, described in detail in this answer.
While explicit \-escaping of embedded " chars.(!) is typically a better workaround and works reliably in PowerShell (Core) 7+, there are edge cases in Windows PowerShell where it doesn't, and you've hit one of them:
# Alternative workaround for *PowerShell (Core) v7+ only*
git config --global mergetool.vscode.cmd '\"code --wait $MERGED\"'
The reason this doesn't work in Windows PowerShell is that it mistakenly concludes that the \"...\" constitutes syntactic double-quoting and therefore doesn't enclose the argument in "..."; that is, instead of passing "\"...\"", it passes just \"...\" as part of the target process' command line.
Note that quoting arguments individually avoids the edge case discussed, so that explicit \-escaping works even in Windows PowerShell:
# Also works in Windows PowerShell, because the edge case is avoided.
git config --global mergetool.vscode.cmd 'code --wait \"$MERGED\"'
Generic solution via module Native:
If you want to solve quoting problems for most[2] (on Windows) / all (on Unix) calls to external programs, consider my Native module's ie function, which encapsulates all required workarounds:
# Install the module in the current user's scope.
Install-Module Native
# Simply prepend `ie` to your external-program calls, which correctly
# handles all behind-the-scenes re-quoting, allowing you to focus on
# PowerShell syntax only.
ie git config --global mergetool.vscode.cmd 'code --wait "$MERGED"'
[1] git appears to be invoking sh as follows (using sh syntax with placeholders for illustration):
sh -c '<config-value> "$#"' '<config-value>' <git-supplied args>
That is, the command line stored in the configuration value is invoked with git-supplied pass-through arguments appended. Note that the first first post -c argument is again the configuration value, which binds to $0, i.e. sets the invocation name, and is therefore not part of the array of pass-through arguments, "$#". An example of a git-supplied pass-through argument is the path of a file to edit that is passed to the command line stored in the core.editor configuration value.
[2] A solution that works with all external programs is fundamentally impossible on Windows, because each program can decide for itself how it parses the string that encodes the arguments being passed. However, there are a few widespread patterns that ie is aware of and applies appropriately, which notably makes it work robustly with batch files and msiexec-like executables.
I'm working with a command line utility that requires passing the name of a file to write output to, e.g.
foo -o output.txt
The only thing it writes to stdout is a message that indicates that it ran successfully. I'd like to be able to pipe everything that is written to output.txt to another command line utility. My motivation is that output.txt will end up being a 40 GB file that I don't need to keep, and I'd rather pipe the streams than work on massive files in a stepwise manner.
Is there any way in this scenario to pipe the real output (i.e. output.txt) to another command? Can I somehow magically pass stdout as the file argument?
Solution 1: Using process substitution
The most convenient way of doing this is by using process substitution. In bash the syntax looks as follows:
foo -o >(other_command)
(Note that this is a bashism. There's similar solutions for other shells, but bottom line is that it's not portable.)
Solution 2: Using named pipes explicitly
You can do the above explicitly / manually as follows:
Create a named pipe using the mkfifo command.
mkfifo my_buf
Launch your other command with that file as input
other_command < my_buf
Execute foo and let it write it's output to my_buf
foo -o my_buf
Solution 3: Using /dev/stdout
You can also use the device file /dev/stdout as follows
foo -o /dev/stdout | other_command
Named pipes work fine, but you have a nicer, more direct syntax available via bash process substitution that has the added benefit of not using a permanent named pipe that must later be deleted (process substitution uses temporary named pipes behind the scenes):
foo -o >(other command)
Also, should you want to pipe the output to your command and also save the output to a file, you can do this:
foo -o >(tee output.txt) | other command
For the sake of making stackoverflow happy let me write a long enough sentence because my proposed solution is only 18 characters long instead of the required 30+
foo -o /dev/stdout
You could use the magic of UNIX and create a named pipe :)
Create the pipe
$ mknod -p mypipe
Start the process that reads from the pipe
$ second-process < mypipe
Start the process, that writes into the pipe
$ foo -o mypipe
foo -o <(cat)
if for some reason you don't have permission to write to /dev/stdout
I use /dev/tty as the output filename, equivalent to using /dev/nul/ when you want to output nothing at all. Then | and you are done.
I am basically looking for a way to do this
list=$(command)
while read -r arg
do
...
done <<< "$list"
Using sh intead of bash. The code as it is doesn't run because of the last line:
syntax error: unexpected redirection
Any fixes?
Edit: I need to edit variables and access them outside the loop, so using | is not acceptable (as it creates a sub-shell with independent scope)
Edit 2: This question is NOT similar to Why does my Bash counter reset after while loop as I am not using | (as I just noticed in the last edit). I am asking for another way of achiving it. (The answers to the linked question only explain why the problem happens but do not provide any solutions that work with sh (no bash).
There's no purely syntactic way to do this in POSIX sh. You'll need to use either a temporary file for the output of the command, or a named pipe.
mkfifo output
command > output &
while read -r arg; do
...
done < output
rm output
Any reason you can't do this? Should work .. unless you are assigning any variables inside the loop that you want visible when it's done.
command |
while read -r arg
do
...
done
I'm trying to do a simple operation in ksh that I need to repeat in sh (Bourne shell)
All I want to do is append the contents of the first line of hte pay_period.txt file to the end of the new file name. This works great in ksh, but does not work in bourne. The program I'm using defaults to sh and I can't change that. Also I can't have actual shell scipts in the directories. So I have to issue commands.
How can I make the equivalent command below work in bourne
mv HEPAY.txt HE_PAY"$(/usr/bin/head -1 pay_period.txt)."txt
The results of $(/usr/bin/head -1 pay_period.txt) is 20140101.
If you are really talking about a real Bournce shell then you need to use backticks for command substitution ($() is POSIX and portable among "modern", POSIX-compliant shells but won't work in old, legacy shells), e.g.
mv HEPAY.txt HE_PAY`/usr/bin/head -1 pay_period.txt`.txt
Other than that I see no reason why this should not work.
PS: Note that head -1 isn't POSIX-compliant either (head -n 1 is).
I am asked to diff two directories using Perl but I think something is wrong with my command,
$diff = system("sudo diff -r '/Volumes/$vol1' '/Volumes/$vol2\\ 1/' >> $diff.txt");
It doesn't display and output. Can someone help me with this? Thanks!
It seems that you want to store all differences in a string.
If this is the case, the command in the question is not going to work for a few reasons:
It's hard to tell whether it's intended or not, but the $diff variable is being used to set the filename storing the differences. Perhaps this should be diff.txt, not $diff.txt
The result of the diff command is saved in $diff.txt. It doesn't display anything in STDOUT. This can be remedied by omitting the >> $diff.txt part. If it also needs to be stored in file, consider the tee command:
sudo diff -r dir1/ dir2/ | tee diff.txt
When a system call is assigned to a variable, it will return 0 upon success. To quote the documentation:
The return value is the exit status of the program as returned by the wait call.
This means that $diff won't store the differences, but the command exit status. A more sensible approach would be to use backticks. Doing this will allow $diff to store whatever is output to STDOUT by the command:
my $diff = `sudo diff -r dir1/ dir2/ | tee diff.txt`; # Not $diff.txt
Is it a must to use the sudo command? Avoid using it if even remotely possible:
my $diff = `diff -r dir1/ dir2/ | tee diff.txt`; # Not $diff.txt
A final recommendation
Let a good CPAN module take care of this task, as backtick calls can only go so far. Some have already been suggested here; it may be well worth a look.
Is sudo diff being prompted for a password?
If possible, take out the sudo from the invocation of diff, and run your script with sudo.
"It doesn't display and output." -- this is becuase you are saving the differences to a file, and then (presumably) not doing anything with that resulting file.
However, I expect "diff two directories using Perl" does not mean "use system() to do it in the shell and then capture the results". Have you considered doing this in the language itself? For example, see Text::Diff. For more nuanced control over what constitutes a "difference", you can simply read in each file and craft your own algorithm to perform the comparisons and compile the similarities and differences.
You might want to check out Test::Differences for a more flexible diff implementation.