I am trying to get a CSV file from Redmine in a shell script. WGET is complaining about an unacceptable. Any ideas what the magical incantation is, or how to find it?
$ wget --no-check-certificate --accept csv https://username:password#company.com/redmine/issues.csv?utf8=%E2%9C%93&columns=all&description=1
Resolving company.com (company.com)... 192.168.1.45
Connecting to company.com (company.com)|192.168.1.45|:443... connected.
WARNING: The certificate of ‘company.com’ is not trusted.
WARNING: The certificate of ‘company.com’ hasn't got a known issuer.
HTTP request sent, awaiting response... 406 Not Acceptable
2017-04-04 10:14:20 ERROR 406: Not Acceptable.
You can try to replace --accept csv with --accept "*.csv". See the wget manual: https://www.gnu.org/software/wget/manual/wget.html#Recursive-Accept_002fReject-Options
Related
I have Setup minikube in my Machine using Hyper-v in windows 10. All working fine, but when i tried to setup fannel network i execute following commannd.
wget http://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Oupput:-
Connecting to raw.githubusercontent.com (151.101.192.133:80)
wget: not an http or ftp url: https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
I tried some solution like to install wget and hash -r but not working.
Any idea or suggestion to solve this.
Thank you,
Note that the actual url your wget is trying to connect to is not an http but https url and the output you attached says:
wget: not an http or ftp url
which is true as https is neither http nor ftp url. It looks like your wget version supports only two mentioned protocols.
You can easily check it by issuing following commands:
wget -V | grep https
and
wget -V | grep ssl
I tried to reproduce it on a system possibly similar to the one you're using. For this purpose I created a buildroot Pod from advancedclimatesystems/docker-buildroot image:
kubectl run --generator=run-pod/v1 buildroot --image=advancedclimatesystems/docker-buildroot --command sleep 3600
and I attached to it by:
kubectl exec -ti buildroot /bin/sh
Once there, I tested out your wget command and it was successful. It's output in my system looks like this (note the 301 redirection to https url):
root#buildroot:~/buildroot# wget http://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
--2019-12-31 16:04:27-- http://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml [following]
--2019-12-31 16:04:27-- https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14416 (14K) [text/plain]
Saving to: 'kube-flannel.yml'
kube-flannel.yml 100%[=====================================================================================================================>] 14.08K --.-KB/s in 0s
2019-12-31 16:04:27 (53.3 MB/s) - 'kube-flannel.yml' saved [14416/14416]
As you can see it has built-in ssl and https support:
root#buildroot:~/buildroot# wget -V | grep ssl
+ntlm +opie +psl +ssl/openssl
-Wl,-z,relro -Wl,-z,now -lpcre -luuid -lidn2 -lssl -lcrypto -lpsl
ftp-opie.o openssl.o http-ntlm.o ../lib/libgnu.a
root#buildroot:~/buildroot# wget -V | grep https
-cares +digest -gpgme +https +ipv6 +iri +large-file -metalink +nls
I'm using this wget command to download all the .fits files from that URL:
wget -r -np -nd -l inf -A fits https://archive.stsci.edu/missions/tess/ete-6/tid/00/000/000/057/
This is based on an adaptation of this answer.
All I'm getting is a directory structures that mirrors the URI on the website all the way down to /057/, but there's no file.
If I add -nd, then I only get a robot.txt files which isn't very instructive, but still no files.
What am I not getting about how to use wget for this?
EDIT: based on Turgbek's answer below, I do see that the robot.txt file from that website actually has /missions/ in the "Disallow"... maybe this is what is preventing me from using the wget command? Is that the source of the problem? How can I get around that?
In robots.txt there's a statement:
Disallow: /missions/
Which your requested files are in. Since the url builds up as /missions/tess/ete-6/tid/00/000/000/057/ I believe that robots.txt is blocking you.
I've saved two of the files from that URL in my Raspberry Pi and ran a local test without robots.txt. With this command:
wget -r -np -nd -l inf -A fits 192.168.1.250/test/
It worked as intended and I've received both of the files.
--2018-05-03 23:46:51-- http://192.168.1.250/test/tess2019128220341-0000000005712108-0016-s_lc.fits
Reusing existing connection to 192.168.1.250:80.
HTTP request sent, awaiting response... 200 OK
Length: 2090880 (2.0M)
Saving to: `192.168.1.250/test/tess2019128220341-0000000005712108-0016-s_lc.fits'
100%[==============================================================================>] 2,090,880 3.77M/s in 0.5s
2018-05-03 23:46:51 (3.77 MB/s) - `192.168.1.250/test/tess2019128220341-0000000005712108-0016-s_lc.fits' saved [2090880/2090880]
--2018-05-03 23:46:51-- http://192.168.1.250/test/tess2019128220341-0000000005715814-0016-s_lc.fits
Reusing existing connection to 192.168.1.250:80.
HTTP request sent, awaiting response... 200 OK
Length: 2090880 (2.0M)
Saving to: `192.168.1.250/test/tess2019128220341-0000000005715814-0016-s_lc.fits'
100%[==============================================================================>] 2,090,880 4.61M/s in 0.4s
2018-05-03 23:46:52 (4.61 MB/s) - `192.168.1.250/test/tess2019128220341-0000000005715814-0016-s_lc.fits' saved [2090880/2090880]
i have this error
--2018-02-14 13:45:42-- http://www.webmin.com/jcameron-key.asc
Resolving www.webmin.com (www.webmin.com)... 216.105.38.10
Connecting to www.webmin.com (www.webmin.com)|216.105.38.10|:80... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Emplacement : https://sourceforge.net/error-404.html [next]
--2018-02-14 13:45:43-- https://sourceforge.net/error-404.html
Resolving sourceforge.net (sourceforge.net)... 216.105.38.10
Connecting to sourceforge.net (sourceforge.net)|216.105.38.10|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2018-02-14 16:45:44 ERROR 404: Not Found.
When i do this
root#server:/tmp$ wget http://www.webmin.com/jcameron-key.asc
Any solution please
404 not found means that the link you provided did not resolve to anything and thus it doesn’t exist anymore. It’s server side and thus you have no control.
I have the solution:
Install via shell command
If you like to install and update Webmin via APT, Do like this:
$~: sudo nano /etc/apt/sources.list
Add this at the bottom of the file, last line.
deb http://download.webmin.com/download/repository sarge contrib
deb http://webmin.mirror.somersettechsolutions.co.uk/repository sarge contrib
Install Webmin
:~$ sudo -i
:~$ wget https://www.techandme.se/wp-content/uploads/2015/01/jcameron-key.asc
:~$ apt-key add jcameron-key.asc
:~$ apt-get update && apt-get install webmin --force-yes -y && rm jcameron-key.asc
Login
https://your-ip-adress:10000
This happens due to maintenance of Webmin servers moving to the other location. It will be back in a bit.
Sorry about that.
I was trying to download tomcat using wget command and getting below error.
Error- sudheer#sudheer:~$ wget https://www-eu.apache.org/dist/tomcat/tomcat-9/v9.0.11/bin/apache-tomcat-9.0.11.tar.gz -P /tmp --2018-12-02 00:49:10-- https://www-eu.apache.org/dist/tomcat/tomcat-9/v9.0.11/bin/apache-tomcat-9.0.11.tar.gz Resolving www-eu.apache.org (www-eu.apache.org)... 2a01:4f9:2a:185f::2, 95.216.24.32 Connecting to www-eu.apache.org (www-eu.apache.org)|2a01:4f9:2a:185f::2|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2018-12-02 00:49:12 ERROR 404: Not Found.
Solution-
check the url "https://www-eu.apache.org/dist/tomcat/tomcat-9/v9.0.11/bin/apache-tomcat-9.0.11.tar.gz" is correct or not
I found this url doesn't exist so I corrected it and its working fine. Correct url should be
"https://www-eu.apache.org/dist/tomcat/tomcat-9/v9.0.13/bin/apache-tomcat-9.0.13-deployer.tar.gz"
Output:-
sudheer#sudheer:~$ wget https://www-eu.apache.org/dist/tomcat/tomcat-9/v9.0.13/bin/apache-tomcat-9.0.13-deployer.tar.gz -P /tmp --2018-12-02 00:53:18-- https://www-eu.apache.org/dist/tomcat/tomcat-9/v9.0.13/bin/apache-tomcat-9.0.13-deployer.tar.gz Resolving www-eu.apache.org (www-eu.apache.org)... 2a01:4f9:2a:185f::2, 95.216.24.32 Connecting to www-eu.apache.org (www-eu.apache.org)|2a01:4f9:2a:185f::2|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 2636635 (2.5M) [application/x-gzip] Saving to: ‘/tmp/apache-tomcat-9.0.13-deployer.tar.gz’
apache-tomcat-9.0.13-deployer.tar.gz 100%[=======================================================================>] 2.51M 370KB/s in 7.0s
2018-12-02 00:53:27 (370 KB/s) - ‘/tmp/apache-tomcat-9.0.13-deployer.tar.gz’ saved [2636635/2636635]
I am trying to download all the zip files from the website http://www.minetad.gob.es/energia/balances/Publicaciones/ElectricasMensuales/Paginas/ElectricasMensuales.aspx
Note that the .zip files listed have the route (By using right click over the link and selecting copy link address) in the form, for example the first one http://www.minetad.gob.es/energia/balances/Publicaciones/ElectricasMensuales/2015/Abril_2015.zip
I have tried all the combinations listed here How to download multiple .zip files?, but none of them works. They either download an empty structure of directories, or I get this error message:
For example, this one
wget -r -l1 -nd -H -t1 -nd -N -np -A.zip -erobots=off -U mozilla --random-wait http://www.minetad.gob.es/energia/balances/Publicaciones/ElectricasMensuales/2015
gives me the following error message:
Resolving www.minetad.gob.es (www.minetad.gob.es)... 193.146.1.81, 2001:720:438:400::81
Connecting to www.minetad.gob.es (www.minetad.gob.es)|193.146.1.81|:80... connected.
HTTP request sent, awaiting response... 302 Redirect
Location: http://www.minetad.gob.es/energia/balances/Publicaciones/ElectricasMensuales/2015/Forms/AllItems.aspx [following]
--2017-02-05 18:52:40-- http://www.minetad.gob.es/energia/balances/Publicaciones/ElectricasMensuales/2015/Forms/AllItems.aspx
Reusing existing connection to www.minetad.gob.es:80.
HTTP request sent, awaiting response... 401 Unauthorized
Username/Password Authentication Failed.
Note that I can download every file individually by using the complete url, like this:
wget http://www.minetad.gob.es/energia/balances/Publicaciones/ElectricasMensuales/2015/Abril_2015.zip
I get the following error when trying to connect to a http URL through a proxy using wget
the error:
wget "http://pro.fastmarkets.com/feeds/default.aspx?usr=anzbank&pwd=587345bv98735vb2b56&feed=physicals-csv" -O /tmp/test.csv
**wget: Error in /home/acdbaqa/.wgetrc at line 3.**
--06:04:15-- http://pro.fastmarkets.com/feeds/default.aspx?usr=anzbank&pwd=587345bv98735vb2b56&feed=physicals-csv
=> `/tmp/test.csv'
Connecting to 59.154.134.109:80... connected.
Proxy request sent, awaiting response... 407 Proxy Authentication Required
06:04:15 ERROR 407: Proxy Authentication Required.
my .wgetrc file contents:
http_proxy=59.154.134.109:80
proxy_user=ACPROXYPROD
proxy_password=test
#password='Ev*luti*n0456789'
#proxy-password='Ev*luti*n0456789'
ftp_proxy=204.2.23.10:80
line 3 is the password. am i missing anything here? not sure why it says error at line 3
Τhis was solved by changing the .wgetrc file with the below parameters
proxy-user=ACPROXYPROD
proxy-passwd=test1245
it is NOT proxy_user and proxy_password.
You are using an older wget release where the password setting used the following syntax (note the missing "or" in password):
proxy_passwd=test