I'm trying to use wget with subprocess to download a tar file.
In my Ide I enter web_paage = subprocess.run('wget -O', web_dnld_file.tar url,capture=True)
I have also tried args = ['wget -O', web_dnld_file.tar url)
The ide indicates error cannot assign to literal
Related
What I am trying to achieve.
Run a python script saved On pythonanywhere host from google sheets on a button press.
Check the answer by Dustin Michels
Task of Each File?
app.py: contains code of REST API made using Flask.
runMe.py: contains code for that get values from(google sheet cell A1:A2). And sum both values send sum back to A3.
main.py: contains code for a GET request with an argument as name(runMe.py).filename may change if the user wants to run another file.
I Made an API by using Flask.it works online and offline perfectly but still, if you want to recommend anything related to the app.py.Code Review App.py
from flask import Flask, jsonify
from flask_restful import Api, Resource
import os
app = Flask(__name__)
api = Api(app)
class callApi(Resource):
def get(self, file_name):
my_dir = os.path.dirname(__file__)
file_path = os.path.join(my_dir, file_name)
file = open(file_path)
getvalues = {}
exec(file.read(), getvalues)
return jsonify({'data': getvalues['total']})
api.add_resource(callApi, "/callApi/<string:file_name>")
if __name__ == '__main__':
app.run()
Here is the Code of runMe2.py
import gspread
from oauth2client.service_account import ServiceAccountCredentials
# use creds to create a client to interact with the Google Drive API
scopes =['https://www.googleapis.com/auth/spreadsheets',"https://www.googleapis.com/auth/drive.file","https://www.googleapis.com/auth/drive"]
creds = ServiceAccountCredentials.from_json_keyfile_name('service_account.json', scopes)
client = gspread.authorize(creds)
# Find a workbook by name and open the first sheet
# Make sure you use the right name here.
sheet = client.open("Demosheet").sheet1
# Extract and print all of the values
list_of_hashes = sheet.get_all_records()
print(list_of_hashes)
below is the main.py code
import requests
BASE = 'https://username.pythonanywhere.com/callApi/test.py'
response = requests.get(BASE)
print(response.json())
main.py output
{'data': 54}
Test.py code
a = 20
b = 34
total = a+b
print(total)
PROBLEM IS
if I request runMe2.py at that time I am got this error.
check runMe2.py code above
app.py is hosted on https://www.pythonanywhere.com/
ModuleNotFoundError: No module named 'gspread'
However, I installed gspread on pythonanywhere why using the command. but it's not working.
You either haven't installed the gspread package on your current python environment or it is installed somewhere (e.g. in a diff. virtual env) and your script cant find it.
Try installing the package inside the environment your running your script in using pip3:
pip3 install gspread
You can try something like this on Windows
pip install gspread
or on Mac
pip3 install gspread
If you're running on Docker, or building with a requirements.txt you can try adding this line you your requirements.txt file
gspread==3.7.0
Any other instructions for this package can be found here => https://github.com/burnash/gspread
Download gspread here
Download the tar file: gspread-3.7.0.tar.gz from the above link
Extract file and convert folder in zip then upload it back on server
Open bash console and use command as
$ unzip gspread-3.7.0
$ cd gspread-3.7.0
$ python3.7 setup.py install --user
I am trying to extract images in CentOS machine using ImageMagick's convert command as below:
convert -coalesce http://cdn.abcdf.com/p/f7/81/d3/40/f781d34031e68828eaasdwc937cf3f8.gif /mnt/temp/123.png
I am getting the following error:
convert: unable to open image `//cdn.adnxs.com/p/f7/81/d3/40/f781d34031e6882840eaa6dc937cf3f8.gif': No such file or directory # error/blob.c/OpenBlob/2701.
convert: no decode delegate for this image format `HTTP' # error/constitute.c/ReadImage/504.
convert: no images defined `/mnt/ephemeral2/creative_report/temp/123.png' # error/convert.c/ConvertImageCommand/3258
I tried reinstalling ImageMagick from source but it was of no use.
I resolved the issue by reinstalling IM and the missing libxml2-devel library. Below are the steps I followed:
1)cd (ImageMagick folder)
2)make uninstall
3)yum install tcl-devel libpng-devel libjpeg-devel ghostscript-devel bzip2-devel freetype-devel libtiff-devel libxml2-devel
4)wget ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-6.9.9-0.tar.gz
5)tar xvfz ImageMagick-6.9.9-0.tar.gz
6)cd (the folder created)
7)./configure --prefix=/usr/local --with-bzlib=yes --with-fontconfig=yes --with-freetype=yes --with-gslib=yes --with-gvc=yes --with-jpeg=yes --with-jp2=yes --with-png=yes --with-tiff=yes
8)make clean
9)make
10)make install
On windows I would put double quotes around the path as it looks like it is breaking up the image path.
Your link is bad. My browser tells me it cannot find the server. In ImageMagick, you should read the input image first. So if you fix your URL, then
convert http://cdn.abcdf.com/p/f7/81/d3/40/f781d34031e68828eaasdwc937cf3f8.gif -coalesce /mnt/temp/123.png
I'm trying to install Spark on my local machine. I have been following this guide. I have installed JDK-7 (also have JDK-8) and Scala 2.11.7. A problem occurs when I try to use sbt to build Spark 1.4.1. I get the following exception.
NOTE: The sbt/sbt script has been relocated to build/sbt.
Please update references to point to the new location.
Invoking 'build/sbt assembly' now ...
Attempting to fetch sbt
Launching sbt from build/sbt-launch-0.13.7.jar
Error: Invalid or corrupt jarfile build/sbt-launch-0.13.7.jar
I have searched for a solution to this problem. I have found a nice guide https://stackoverflow.com/a/31597283/2771315 which uses a pre-built version. Other than using the pre-built version, is there a way to install Spark using sbt? Further, is there a reason as to why the Invalid or corrupt jarfile error occurs?
I met the same problem. I have fixed it now.
This probably because sbt-launch-0.13.7.jar has a unsuccessful download, although you can see the file is exist, but it's not correct file. The file is about 1.2MB in size. If less than that, you can get into the build/ , use "vim sbt-launch-0.13.7.jar" or other tools to open sbt-launch-0.13.7.jar file.
If the file have the content like this:
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
It implys that sbt-launch-0.13.7.jar is not downloaded.
Then open sbt-launch-lib.bash in the same directory,check the line 41 and 42, it gives two urls. Open it to check if they work well.
If url1 doesn't work，download the sbt-launch.jar manually(you can use url2, it may works,or you can download from sbt official website), put it in the same directory, rename it to sbt-launch-0.13.7.jar, then you shoud comment lines in relation to the downloading(may be between line 47 and 68)， avoid the script download it again. Like this:
acquire_sbt_jar () {
SBT_VERSION=`awk -F "=" '/sbt\.version/ {print $2}' ./project/build.properties`
URL1=http://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/${SBT_VERSION}/sbt-launch.jar
URL2=http://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/${SBT_VERSION}/sbt-launch.jar
JAR=build/sbt-launch-${SBT_VERSION}.jar
sbt_jar=$JAR
# if [[ ! -f "$sbt_jar" ]]; then
# # Download sbt launch jar if it hasn't been downloaded yet
# if [ ! -f "${JAR}" ]; then
# # Download
# printf "Attempting to fetch sbt\n"
# JAR_DL="${JAR}.part"
# if [ $(command -v curl) ]; then
# (curl --silent ${URL1} > "${JAR_DL}" || curl --silent ${URL2} > "${JAR_DL}") && mv "${JAR_DL}" "${JAR}"
# elif [ $(command -v wget) ]; then
# (wget --quiet ${URL1} -O "${JAR_DL}" || wget --quiet ${URL2} -O "${JAR_DL}") && mv "${JAR_DL}" "${JAR}"
# else
# printf "You do not have curl or wget installed, please install sbt manually from http://www.scala-sbt.org/\n"
# exit -1
# fi
# fi
# if [ ! -f "${JAR}" ]; then
# # We failed to download
# printf "Our attempt to download sbt locally to ${JAR} failed. Please install sbt manually from http://www.scala-sbt.org/\n"
# exit -1
# fi
# printf "Launching sbt from ${JAR}\n"
# fi
}
Then use "build/sbt assembly" to build the spark again.
Hope you will succeed.
If I didn't express clearly, the following links may be helpful.
https://www.mail-archive.com/user#spark.apache.org/msg34358.html
Error: Invalid or corrupt jarfile sbt/sbt-launch-0.13.5.jar the answer by prabeesh
https://groups.google.com/forum/#!topic/predictionio-user/fllCh8n-0d4
Download the sbt-launch.jar file manually (you can use url2, it may work, or you can download from the sbt official website), put it in the same directory, rename it to sbt-launch-0.13.7.jar, then run the sbt/sbt assembly command.
On a website, after logging in with my credentials I am able to download daa by changing the url address to variations of this:
https://data.somewhere.com/DataDownload/getfile.jsp?ccy=AUDUSD&df=BBO&year=2014&month=02&dllater=Download
This put a zip file in my downlaod directory.
If I try to automate it with wget using:
wget "https://data.somewhere.com/DataDownload/getfile.jsp?ccy=AUDUSD&df=BBO&year=2014&month=02&dllater=Download" --no-check-certificate --ignore-length
$ ~/dnloadHotSpot.sh
--2014-03-22 16:05:16-- https://data.somewhere.com/DataDownload/getfile.jsp?ccy=AUDUSD&df=BBO&year=2014&month=02&dllater=Download
Resolving data.somewhere.com (data.somewhere.com)... 209.191.250.173
Connecting to data.somewhere.com (data.somewhere.com)|209.191.250.173|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: ignored [text/html]
Saving to: `getfile.jsp#ccy=AUDUSD&df=BBO&year=2014&month=02&dllater=Download'
[ <=> ] 8,925 --.-K/s in 0.001s
2014-03-22 16:05:18 (14.4 MB/s) - `getfile.jsp#ccy=AUDUSD&df=BBO&year=2014&month=02&dllater=Download' saved [8925]
What else to I need to add to make wget actually download the file?
If you want to specify the name of the output file into which wget places the contents of the file is is downloading, then use the capital O parameter, something like:
wget -O myfilename ......
I'm trying to build a very simple PostgreSQL client in C over Cygwin.
Here's what I've done so far:
I've downloaded the PostgreSQL source code version 9.1.2 (to match the same version that is running on my server)
I've configured and compiled the source code from Cygwin. The compilation seemed to go smoothly.
From what I can tell, the header files are in:
/cygdrive/c/workspace/src/postgresql-9.1.2/src/interfaces/libpq, and
/cygdrive/c/workspace/src/postgresql-9.1.2/src/include
The libraries are in:
/cygdrive/c/workspace/src/postgresql-9.1.2/src/interfaces/libpq
From here, I compiled and linked the client program using the makefile below:
testlibpq: testlibpq.c
gcc -o testlibpq -I /cygdrive/c/workspace/src/postgresql-9.1.2/src/interfaces/libpq -I /cygdrive/c/workspace/src/postgresql-9.1.2/src/include -L /cygdrive/c/workspace/src/postgresql-9.1.2/src/interfaces/libpq testlibpq.c -Bstatic -lpq
The compilation and the linking succeeded without errors or warnings.
However, when I try to run the program, I get the following error:
$ ./testlibpq
/cygdrive/c/Users/dleclair/Dropbox/denis/src/testlibpq/testlibpq.exe: error while loading shared libraries: cygpq.dll: cannot open shared object file: No such file or directory
I haven't figured out how to fix this. Any pointers would be greatly appreciated.
Oh, one more thing, I found the folder that cygpq.dll was sitting in and set my LD_LIBRARY_PATH to point to it but it still gave me the same result.
dleclair#dleclair-win7l ~/Dropbox/denis/src/testlibpq
$ ls /cygdrive/c/workspace/src/postgresql-9.1.2/src/interfaces/libpq
bcc32.mak encnames.o fe-connect.o fe-misc.o fe-protocol3.o ip.o libpq-events.c md5.c pgstrcasecmp.c pqsignal.c thread.o
blibpqdll.def exports.txt fe-exec.c fe-print.c fe-secure.c libpq.a libpq-events.h md5.o pgstrcasecmp.o pqsignal.h wchar.c
chklocale.c fe-auth.c fe-exec.o fe-print.o fe-secure.o libpq.rc.in libpq-events.o nls.mk po pqsignal.o wchar.o
chklocale.o fe-auth.h fe-lobj.c fe-protocol2.c inet_net_ntop.c libpqddll.def libpq-fe.h noblock.c pqexpbuffer.c pthread-win32.c win32.c
cygpq.dll fe-auth.o fe-lobj.o fe-protocol2.o inet_net_ntop.o libpq-dist.rc libpq-int.h noblock.o pqexpbuffer.h README win32.h
encnames.c fe-connect.c fe-misc.c fe-protocol3.c ip.c libpqdll.def Makefile pg_service.conf.sample pqexpbuffer.o thread.c win32.mak
dleclair#dleclair-win7l ~/Dropbox/denis/src/testlibpq
$ echo $LD_LIBRARY_PATH
/cygdrive/c/workspace/src/postgresql-9.1.2/src/interfaces/libpq
dleclair#dleclair-win7l ~/Dropbox/denis/src/testlibpq
Normally on unix/linux systems after building the source a make install is done which copies the headers to standard locations like /usr/local/include and /usr/local/lib. You may have to do that on cygwin to to get the DLL in the search path.
Or you can just locate the DLL yourself and put it on the search path or in the same folder as your executable.