How can I find and copy using gsutil? - google-cloud-storage

I know how to do it with for but is there an easy way?
touch temp/foo
for i in $(find my-dir/* -newer temp/foo);
do
gsutil -m cp -r "$i" gs://my-bucket/
done

The gsutil cp -I parameter should help with this. It allows you to read the list of files to copy from stdin.
I think this would work:
find my-dir/* -newer temp/foo | gsutil -m cp -I gs://my-bucket/

Related

Easy way to write cp -a in PowerShell

So since PowerShell seems to have alises to bash commands they still don't follow them 100%...
What is the easiest way to write bash's cp -a in PowerShell format?
The -r flag which stands for recursive.
cp -r

Copy files based on regex to another folder but keep folder structure

I want to copy files matching a regex to another folder but while keeping part of the folder structure, All the filepaths will start with src/main/java/ buth the path before that is different for most files
I know that I can use
find . -iregex ".*HeadersConstants\.java" -exec cp {} ./destination/ \;
To copy a file but then I lose the file path in the destination dir
Are you on linux? Historically, cpio would have been an obvious choice but these days rsync is likely to be better:
find . -iregex ".*HeadersConstants\.java" |\
rsync -v --files-from=- ./ ${destination}/
It's probaby not a good idea for the destination to be inside . as your question code suggests but we can stop find looking there with:
find . -path ./destination -prune \
-o -iregex ".*HeadersConstants\.java" -print |\
rsync -v --file-from=- ./ ./destination/
(You may want to investigate why the -print is required.)
In the meantime I got it to work (probably not the cleanest way)
javaRe='(.*)\/src\/main\/java\/(.*)\/'
find . -name "HeadersConstants\.java" | while read f
do
if [[ ${f} =~ ${javaRe} ]]; then
path=${BASH_REMATCH[2]}
fullpath=${destination}${srcDir}${path}
mkdir -p "$fullpath"
cp "$f" "$fullpath"
fi
done

Replace a given string in folders throughout a directory

I want to do the equivalent of this, but [maybe recursively] for all, say, .md files within a directory tree.
perl -pi -e 's/FOO/BAR/g' *.md
Use find:
find /path -name "*.md" -exec perl -pi -e 's/FOO/BAR/g' {} \;
A simple & pure bash one line solution by using bash parameter expansion:
$ cd ~/
$ mkdir test
$ cd test/
$ touch foo{1..10}.md
$ ls
foo1.md foo10.md foo2.md foo3.md foo4.md foo5.md foo6.md foo7.md foo8.md foo9.md
$ for file in ./*.md; do mv "$file" "${file/foo/bar}"; done
$ ls
bar1.md bar10.md bar2.md bar3.md bar4.md bar5.md bar6.md bar7.md bar8.md bar9.md
Of course it can be combined with find as suggested by devnull:
$ files=($(find ./test -name "*.md"))
$ for file in "${files[#]}"; do mv "$file" "${file/foo/bar}"; done
Or pipe the ouptut of find to the loop:
$ find ./test -name "*.md" | for file in $(xargs -0); do mv "$file" "${file/foo/bar}"; done

Regex with wget?

I'm using wget to download some useful website:
wget -k -m -r -q -t 1 http://www.web.com/
but I want replace some bad words with my own choice (like Yahoo pipes regex)
If you want to regexp out words from within the page you are fetching with wget, you should pipe the output through sed.
For example:
wget -k -m -r -q -t 1 -O - http://www.web.com/ | sed 's/cat/dog/g' > output.html
Use the -O - flag to write the output to stdout, and the -q flag to make wget run in quiet mode.
Haven't got a shell atm to check my syntax but that should set you on the right path!
You can use sed -i.
find www.web.com -type f -exec sed -i 's/word1\|word2\|word3//ig' {} +
word1, word2, word3, etc. are the words to delete.

How can you recursively copy all of the *.foo files in src to target using cp and/or find?

cp -v -ur path/to/jsps/ /dest/path/
The above command copies all of the files that have been updated from the source directory to the destination, preserving the directory structure.
What I can't figure out is how to copy only *.someExtention files. I know that you can use something like:
find -f -name *.jsp -exec some awesome commands {}
But I don't know how to do it (and I don't have time to read the info pages in detail).
All help is greatly appreciated.
Thanks,
LES
If you want to use find / cp then the following should do the trick:
find -f -name *.jsp -exec cp --parents {} /dest/path \;
but rsync is probably the better tool.
rsync might help - you can tell it to just copy certain files with a combination of include and exclude options, e.g.
rsync -a \
--include='*.foo' \
--include='*/' \
--exclude='*' \
path/to/jsps/ /dest/path/
See the manual and look at the section entitled FILTER RULES for more.