I download multiple BibTeX entries using wget <list of links>. Is there a way to concatenate all these entries into a single .bib file on the same command (i.e. without storing the files separately and then doing cat *.bib >> mybib.bib)?
Use wget's -O option:
wget -O everything_concatenated.bib <list of links>
Related
My source folders are on an external hard drive, but I want my thumbnails local. The following works, but it puts all the extracted files in the same folder as the source files, which requires another step to collect them and move them to a folder on my local machine.
exiftool -b -ThumbnailImage -w _thumb.jpg -ext CR2 -r source_folder_path\ > _extraction_results.txt
Is there any way to write them to a different folder in the same call to ExifTool?
Thanks!
Add the directory path to name given in the -w (textout) option (see examples at that link).
Example:
exiftool -b -ThumbnailImage -w /path/to/thumbdir/%f_thumb.jpg -ext CR2 -r source_folder_path\ > _extraction_results.txt
I am aware that you can download from multiple url using:
wget "url1" "url2" "url3"
Renaming the output file can be done via:
wget "url1" -O "new_name1"
But when I tried
wget "url1" "url2" "url3" -O "name1" "name2" "name3"
all the files are using name1.
what is the proper way to do so in a single command?
Yes something like this, You can add a file name next to each URL in the file then do:
while IFS= read -r url fileName;do
wget -O "$fileName" "$url"
done < list
where it is assumed you have added a (unique) file name after each URL in the file (separated by a space).
The -O option allows you to specify the destination file name. But if you're downloading multiple files at once, wget will save all of their content to the file you specify via -O. Note that in either case, the file will be truncated if it already exists. See the man page for more info.
You can exploit this option by telling wget to download the links one-by-one:
while IFS= read -r url;do
fileName="blah" # Add a rule to define a new name for each file here
wget -O "$fileName" "$url"
done < list
hope it useful.
I'm downloading so many images that their links are inside a file with the command:
wget -i file.txt
I suspect many of the files might have the same names. So I'm afraid they will be overwritten. is there anyway to make wget set sequential names to the file or handle similar names in any other way?
For wget 1.19.1, what you're looking for is the default behavior. Files that have the same names will be numbered when a matching file is found.
Assuming that file.txt looks like:
http://www.apple.com
http://www.apple.com
http://www.apple.com
http://www.apple.com
The output of wget -i file.txt will be four files, named:
index.html
index.html.1
index.html.2
index.html.3
I want to download an entire website using the wget -r command and change the name of the file.
I have tried with:
wget -r -o doc.txt "http....
hoping that the OS would have automatically create file in order like doc1.txt doc2.txt but It actually save the stream of the stdout in that file.
Is there any way to do this with just one command?
Thanks!
-r tells wget to recursively get resources from a host.
-o file saves log messages to file instead of the standard error. I think that is not what you are looking for, I think it is -O file.
-O file stores the resource(s) in the given file, instead of creating a file in the current directory with the name of the resource. If used in conjunction with -r, it causes wget to store all resources concatenated to that file.
Since wget -r downloads and stores more than one file, recreating the server file tree in the local system, it has no sense to indicate the name of one file to store.
If what you want is to rename all downloaded files to match the pattern docX.txt, you can do it with a different command after wget has end:
wget -r http....
i=1
while read file
do
mv "$file" "$(dirname "$file")/doc$i.txt"
i=$(( $i + 1 ))
done < <(find . -type f)
I understand that -i flag takes a file (which may contain list of URLs) and I know that -o followed by a name can be specified to rename a item being downloaded using wget.
example:
wget -i list_of_urls.txt
wget -o my_custom_name.mp3 http://example.com/some_file.mp3
I have a file that looks like this:
file name: list_of_urls.txt
http://example.com/some_file.mp3
http://example.com/another_file.mp3
http://example.com/yet_another_file.mp3
I want to use wget to download these files with the -i flag but also save each file as 1.mp3, 2.mp3 and so on.
Can this be done?
You can use any script language (PHP or Python) for generate batch file. In thin batch file each line will contains run wget with url and -O options.
Or you can try write cycle in bash script.
I ran a web search again and found https://superuser.com/questions/336669/downloading-multiple-files-and-specifying-output-filenames-with-wget
Wget can't seem to do it but Curl can with -K flag, the file supplied can contain url and output name. See http://curl.haxx.se/docs/manpage.html#-K
If you are willing to use some shell scripting then https://unix.stackexchange.com/questions/61132/how-do-i-use-wget-with-a-list-of-urls-and-their-corresponding-output-files has the answer.