I have a file with a couple of lines and in each one there is process id and its generation.
I need to print how many processes there are in each generation and do it with only one loop and have to do it with grep and wc function but can't find how to use grep in my IDE and not as a terminal function.
I saw it should be something like:
grep -o '\<WORD\>' | wc -l
but it's not something I can write in IDE..
Any ideas?
Related
I need to check if local file is same as remote host file.
The file locations are like below:
File1 at Local machine
./remotehostname/home/a/b/scripts/xyz.cpp
File2 at remote machine
remotehostname:/home/a/b/scripts/xyz.cpp
I intend to compare these 2 files, using the command
diff ./remotehostname/home/a/b/scripts/xyz.cpp remotehostname:/home/a/b/scripts/xyz.cpp
find . -type f | grep -v .svn |xargs -I % diff %
I need to change % to take remotehost and compare the file.
Not sure how to apply sed on %. Or is there a better way to compare such files.
One way could be to save the list of files and then apply sed on that file, but I think there should be an even better way. Also the diff doesnt work on remote hosts, maybe I need to use output of dry rsync?
This can be done with xargs, but I prefer to use while read in bash.
xargs method
find . -type f | grep -v .svn | sed 's/.*/& remotehostname:&/' | xargs -n2 diff
The sed command duplicates the input and makes whatever modifications you need. The xargs then passes the inputs to diff two at a time. This will not work if any filename contain spaces.
bash method
find . -type f | grep -v .svn | while read line; do
diff "$line" "remotehostname:$line"
done
The bash read command reads a line from stdin, places it in the name variable, $line, and returns true. You can then put whatever you like inside the loop, so you get total freedom to rewrite the filename however you need. When the input runs out, read returns false, and the loop exits.
Note that piping things into loops has some interesting side effects that are not relevant here, but might bite you one day.
If you are interested in the actual difference (and not just whether they differ - which rsync is brilliant for telling you) then you can do this using GNU Parallel:
find . -type f | grep -v .svn |
parallel diff {} '<(ssh {= s:./::;s:/.*:: =} cat {= s:([^/]+/){2,2}::;$_=::shell_quote_scalar($_) =})'
s:./::;s:/.*:: = hostname from path
s:([^/]+/){2,2}:: = rest of path
::shell_quote_scalar = \-quote special chars as needed by the shell
GNU Parallel is a general parallelizer and makes is easy to run jobs in parallel on the same machine or on multiple machines you have ssh access to. It can often replace a for loop.
If you have 32 different jobs you want to run on 4 CPUs, a straight forward way to parallelize is to run 8 jobs on each CPU:
GNU Parallel instead spawns a new process when one finishes - keeping the CPUs active and thus saving time:
Installation
If GNU Parallel is not packaged for your distribution, you can do a personal installation, which does not require root access. It can be done in 10 seconds by doing this:
(wget -O - pi.dk/3 || curl pi.dk/3/ || fetch -o - http://pi.dk/3) | bash
For other installation options see http://git.savannah.gnu.org/cgit/parallel.git/tree/README
Learn more
See more examples: http://www.gnu.org/software/parallel/man.html
Watch the intro videos: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
Walk through the tutorial: http://www.gnu.org/software/parallel/parallel_tutorial.html
Sign up for the email list to get support: https://lists.gnu.org/mailman/listinfo/parallel
In Linux bash, I am trying to run a command and grep for an argument:
command | grep
However, I need to redirect the result of the commad to the stdout and simultaneously pipe it to grep (I need to see both the grep result and the command result in stdout).
I googled a bit and tried some variations, such as:
command | tee /dev/tty | grep
But, no luck.
I don't want to use sth like
command
command | grep
as it is ugly :)
Thanks,
Try
command | tee >(grep whatever)
Note that there's no space between these two symbols: >(.
I'm trying to find some stuff in a large number of text files, and I want the output to be in a file so I can read it at leisure:
grep -i 'alter table' *.sql >> tables.txt
grep (this is the Windows version of the Gnu tool) complains at the >>. I've tried piping and all the rest, and there doesn't saeem to be an option to define an output file either.
Any ideas?
Reviving this old question, but it's among the first Google results.
grep outputs differently, so I needed to add this option to ouptut the results to a file:
grep --line-buffered
Source
This works here:
grep -i "other something" *.txt >> tables.txt
I'm looking for help with a one-liner that I can run from the Mac OS X terminal. I use MAMP for web development on my Mac. I have a lot of CakePHP projects in my "/Applications/MAMP/htdocs" directory. For the sake of simplicity, let's just say that I had two CakePHP projects and that this was the output of the find /Applications/MAMP/htdocs -type d -iname Controller* command:
/Applications/MAMP/htdocs/my_cake1.3_project/app/controllers
/Applications/MAMP/htdocs/my_cake1.3_project/app/tests/cases/controllers
/Applications/MAMP/htdocs/my_cake1.3_project/cake/console/templates/skel/controllers
/Applications/MAMP/htdocs/my_cake1.3_project/cake/console/templates/skel/tests/cases/controllers
/Applications/MAMP/htdocs/my_cake1.3_project/cake/libs/controller
/Applications/MAMP/htdocs/my_cake1.3_project/cake/tests/cases/libs/controller
/Applications/MAMP/htdocs/my_cake1.3_project/cake/tests/test_app/controllers
/Applications/MAMP/htdocs/my_cake1.3_project/cake/tests/test_app/plugins/test_plugin/controllers
/Applications/MAMP/htdocs/my_cake2_project/app/Controller
/Applications/MAMP/htdocs/my_cake2_project/app/Test/Case/Controller
/Applications/MAMP/htdocs/my_cake2_project/lib/Cake/Console/Templates/skel/Controller
/Applications/MAMP/htdocs/my_cake2_project/lib/Cake/Console/Templates/skel/Test/Case/Controller
/Applications/MAMP/htdocs/my_cake2_project/lib/Cake/Controller
/Applications/MAMP/htdocs/my_cake2_project/lib/Cake/Test/Case/Controller
/Applications/MAMP/htdocs/my_cake2_project/lib/Cake/Test/test_app/Controller
/Applications/MAMP/htdocs/my_cake2_project/lib/Cake/Test/test_app/Plugin/TestPlugin/Controller
Now, sometimes I want to find a piece of code that I know I used in one of my CakePHP projects' controllers, but I can't remember which project it was, so I want to search all of them. I don't want to waste time searching in the "app/tests/cases/controllers" folder or any of the ones within "cake/", though. The find /Applications/MAMP/htdocs -type d -iname Controller* | grep -i /app/Controller command gives me the list of folders I want to search in:
/Applications/MAMP/htdocs/my_cake1.3_project/app/controllers
/Applications/MAMP/htdocs/my_cake2_project/app/Controller
I just need to find a way to take that output, add a slash and asterisk (/*) to the end of each line, and pipe each line to the grep -il "string to search for" command. Any help would be appreciated. Thanks!
solution 1
maybe you want to check two options of find command: (i)path and regex
with them you could narrow your find result and pass found files to your grep -il "searchString" for example by |xargs . it looks like:
find /Applications/MAMP/htdocs -type f -ipath "*/app/Controller/*.php"
| xargs grep -il 'foo'
with -regex would be more flexiable.
solution 2
however if you really really want to :
find a way to take that output, add a slash and asterisk (/*) to the
end of each line, and pipe each line to the grep -il "string to search
for" command.
(btw, here "pipe" won't work.)
you could do this:
find .(your original find).. |grep -i "/app/Controller"
|sed -r 's#^(.*)$#grep -il "foo" \1/*#g'|sh
the trick was done by the sed....|sh. the sed line will pick the result of your previous grep, add grep command and options :(grep -il "foo") and append "/*" in order to construct a complete grep command. finally pipe to sh, to execute it.
Have you tried this?
find /Applications/MAMP/htdocs -type d -iname Controller*
-exec grep -il "string to search for" {} /;
I would like to extract the highest revision number in a subversion dump file.
Besides parsing the file line by line, is there any easier (and hopefully faster) way using standard perl (no extra modules allowed on the server) or bash shell scripting?
if you created a dump file using
svnadmin dump /path/to/repo > Dump1.dump
The you can find the last revision number with this one-liner:
grep --binary-files=text "Revision-number" Dump1.dump | tail -n 1 | sed 's/Revision-number\:\ //g'
Alternately to avoid grepping the entire file, use tac (cat backwards) and stop on the first (last) match. Eliminates the need for tail on the large output of grep, and saves processing time.
tac Dump1.dump | grep -m1 --binary-files=text "Revision-number" | sed 's/Revision-number\:\ //g'
I can subvert this solution, no pun intended, by simply checking in a text file that contains something like
Revision-number: 720
The output of the grep looks like this:
-bash-3.2$ grep "Revision-number:" test.dump.2
Revision-number: 0
Revision-number: 1
Revision-number: 2
Revision-number: 720
To really do this properly the dump file needs to be parsed. I wrote a perl script using the SVN::Dumpfile module and just looped through the revisions until I got to the end.
#!/usr/bin/perl
use SVN::Dumpfile;
use strict;
my $df = new SVN::Dumpfile;
$df->open('/usr/tmp/test.dump.2');
my $revision=-1;
while (my $node = $df->read_node()) {
if ($node->is_rev) {
$revision = $node->header('Revision-number');
}
}
print "$revision\n";