I find myself unable to do a simple progress bar in a cshell script, without being able to output a form-feed. Is this totally unsupported by cshell? (or tcshell)?
Are there any alternatives to output a progress bar (and an ending point of progress), to run shell-commands between iterations of progress?
Could I use awk instead? Awk is pretty obscure to me.
You can use the printf command (which is not specific to csh or tcsh). To print a formfeed character (ctl-L):
printf '\xc'
To demonstrate:
% printf '\xc' | od -c
0000000 \f
0000001
But are you sure formfeed is the character you want? It typically has no effect in a VT-100 compatible terminal emulator.
You can print arbitrary control character sequences, as defined by termcap, with tcsh's built-in echotc command. man tcsh and search for echotc for details; man 5 termcap to see the 2-character codes recognized by echotc.
More portably (to shells other than csh and tcsh), the tput command, part of the ncurses package, is similar to echotc, but it uses terminfo capability names rather than termcap (though it also supports termcap if support is compiled into the package).
Related
I'm trying to use a perl one-liner to turn print0 output into quoted shell parameters, kind of like the trick that's something like .. | xargs -0 printf "%q" {} but I didn't want to require bash (whose printf implements %p). I was kind of amazed to, well, not find an easy way to do this in perl. For all of perl's quoting mechanisms, there's no way I saw for producing quoted strings. Surely I just haven't looked hard enough.
Hopefully the answer isn't a regular expression. Quoting an elaborate regular expression to put into a shell command-line is not my idea of fun (if only a simple perl program could quote it for me, oh back to the same problem).
You can roll your own quoting for POSIX-like shells fairly simply - no complicated regexes needed (just straightforward string substitution using literals):
$ echo "I'm \$HOME. 3\" of rain." | perl -lne "s/'/'\\\''/g; print q{'} . \$_ . q{'}"
'I'\''m $HOME. 3" of rain.'
The approach is modeled after AppleScript's quoted form of command:
The input string is broken into substrings by ', each substring is itself '-enclosed, with the original ' chars. spliced between the substrings as \' (an individually quoted ').
When passed to the shell, the shell rebuilds these parts into a single, literal string.
This multi-part string-concatenation approach is necessary, because POSIX-like shells categorically do not allow embedding ' itself inside single-quoted strings (there's not even an escape sequence).
Alternatively, you can install a CPAN module such as ShellQuote.
Optional background information
While it would be handy for Perl itself to support such a quoting mechanism for piecing together shell commands stored in a single string to pass to qx// (`...`), such a mechanism would have to operate platform-specifically.
Notably, quoting rules for Windows are very different from rules for Unix platforms, and except for simple cases shell commands as a whole will be incompatible too.
From inside Perl, you may be able to bypass the need for quoting altogether, by using the list forms of system() and open(), which allow you to pass the command arguments individually, as-is, but note that this is only an option if your command doesn't use any shell features; for a "shell-less" qx// (`...`) alternative, see this answer of mine, which also covers shell-quoting on Windows.
I came across this script which was not written by me, but because of an issue I need to know what it does.
What is the purpose of filtering the log file using this Perl one-liner?
cat log.txt | perl -pe 's/\e([^\[\]]|\[.*?[a-zA-Z]|\].*?\a)/ /g'
The log.txt file contains the output of a series of commands. I do not understand what is being filtered here, and why it might be useful.
It looks like the code should remove ANSI escape codes from the input, i.e codes to set colors, window title .... Since some of these code might cause harm it might be a security measure in case some kind of attack was able to include such escape codes into the log file. Since usually a log file does not contain any such escape codes this would also explain why you don't see any effect of this statement for normal log files.
For more information about this kind of attack see A Blast From the Past: Executing Code in Terminal Emulators via Escape Sequences.
BTW, while your question looks bad on the first view it is actually not. But you might try to improve questions by at least formatting it properly. Otherwise you risk that this questions gets down-voted fast.
First, the command line suffers from a useless use of cat. perl is fully capable of reading from a file name on the command line.
So,
$ perl -pe 's/\e([^\[\]]|\[.*?[a-zA-Z]|\].*?\a)/ /g' log.txt
would have done the same thing, but avoided spawning an extra process.
Now, -e is followed by a script for perl to execute. In this case, we have a single global substitution.
\e in a Perl regex pattern corresponds to the escape character, x1b.
The pattern following \e looks like the author wants to match ANSI escape sequences.
The -p option essentially wraps the script specified with -e in while loop, so the s/// is executed for each line of the input.
The pattern probably does the job for this simple purpose, but one might benefit from using Regexp::Common::ANSIescape as in:
$ perl -MRegexp::Common::ANSIescape=ANSIescape,no_defaults -pe 's/$RE{ANSIescape}/ /g' log.txt
Of course, if one uses a script like this very often, one might want to either use an alias, or even write a very short script that does this, as in:
#!/usr/bin/env perl
use strict;
use Regexp::Common 'ANSIescape', 'no_defaults';
while (<>) {
s/$RE{ANSIescape}/ /g;
print;
}
I am using following kind of script in my perl script and expecting entry at line 1. I am getting some error as below; any help?
plz ignore perl variable....
Error Messaage -
sed: -e expression #1, char 22: extra characters after command
# Empty file will not work for Sed line number 1
`"Security Concerns Report" > $outputFile`;
`sed '1i\
\
DATE :- $CDate \
Utility accounts with super access:- $LNumOfSupUserUtil \
Users not found in LDAP with super access: - $LNumOfSupUserNonLdap\
' $outputFile > $$`;
`mv $$ $outputFile`;
}
Your immediate problem is that the backslash character is interpreted by Perl inside the backtick operator, as is the dollar character. So your backslash-newline sequence turns into a newline in the shell command that is executed. If you replace these backslashes by \\, you'll go over this hurdle, but you'll still have a very brittle program.
Perl is calling a shell which calls sed. This requires an extra level of quoting for the shell which you are not performing. If your file names and data contain no special characters, you may be able to get away with this, until someone uses a date format containing a ' (among many things that would break your code).
Rather than fix this, it is a lot simpler to do everything in Perl. Everything sed and shells can do, Perl can do almost as easily or easier. It's not very clear from your question what you're trying to do. I'll focus on the sed call, but this may not be the best way to write your program.
If you really need to prepend some text to an existing file, there's a widely-used module on CPAN that already does this well. Use existing libraries in preference to reinventing the wheel. File::Slurp has a prepend_file method just for that. In the code below I use a here-document operator for the multiline string.
use File::Slurp; # at the top of the script with the other use directives
File::Slurp->prepend_file($outputFile, <<EOF);
DATE :- $CDate
Utility accounts with super access:- $LNumOfSupUserUtil
Users not found in LDAP with super access: - $LNumOfSupUserNonLdap
EOF
I have a file that has fairly long lines. The longest line has length 4609:
% perl -nle 'print length' ~/very_large_file | sort -nu | tail -1
4609
Now, when I just run cat ~/very_large_file it runs fine. But when I put inside backticks, it gives a 'word too long' error
% foreach line (`cat ~/very_large_file`)
Word too long.
% set x = `cat ~/very_large_file`
Word too long.
Is there an alternative to using backticks in csh to process each line of such a file?
Update My problem was solved by using a different language, but I still couldn't get the reason for the failing csh. Just came across this page that describes the manner of finding ARG_MAX. In particular, the getconf command is useful. Of course, I am still not sure whether this limit is the root cause, and if the limit applies to the languages other than csh.
I don't mean to beat an old horse, but if you're scripting do consider moving to bash, zsh or even Korn. csh has disadvantages.
What you can try without abandoning csh completely:
Move to tcsh if you're with regular old (very old) csh.
Recompile tcsh with a longer word length (the default is 1000 bytes, I think) or with dynamic allocation.
If possible move the line processing to a secondary script or program and write that loop like this:
cat ~/very_large_file | xargs secondary_script
I have a file like this:
my line - some words & text
oh lóok i've got some characters
I want to 'normalize' it and remove all the non-word characters. I want to end up with something like this:
mylinesomewordstext
ohlóokivegotsomecharacters
I'm using Linux on the command line at the moment, and I'm hoping there's some one-liner I can use.
I tried this:
cat file | perl -pe 's/\W//'
But that removed all the newlines and put everything one line. Is there someway I can tell Perl to not include newlines in the \W? Or is there some other way?
This removes characters that don't match \w or \n:
cat file | perl -C -pe 's/[^\w\n]//g'
#sth's solution uses Perl, which is (at least on my system) not Unicode compatible, thus it loses the accented o character.
On the other hand, sed is Unicode compatible (according to the lists on this page), and gives a correct result:
$ sed 's/\W//g' a.txt
mylinesomewordstext
ohlóokivegotsomecharacters
In Perl, I'd just add the -l switch, which re-adds the newline by appending it to the end of every print():
perl -ple 's/\W//g' file
Notice that you don't need the cat.
The previous response isn't echoing the "ó" character. At least in my case.
sed 's/\W//g' file
Best practices for shell scripting dictate that you should use the tr program for replacing single characters instead of sed, because it's faster and more efficient. Obviously use sed if replacing longer strings.
tr -d '[:blank:][:punct:]' < file
When run with time I get:
real 0m0.003s
user 0m0.000s
sys 0m0.004s
When I run the sed answer (sed -e 's/\W//g' file) with time I get:
real 0m0.003s
user 0m0.004s
sys 0m0.004s
While not a "huge" difference, you'll notice the difference when running against larger data sets. Also please notice how I didn't pipe cat's output into tr, instead using I/O redirection (one less process to spawn).