I'm using the flask-user extension (https://flask-user.readthedocs.io/) along with the mongoengine adapter and flask-wtf to create an HTML form that'll accept several files (images, mp4). The goal is to directly store those files in MongoDB using GridFS but I can't get it to work without storing the image locally on the flask server first...
Here's my non-working code:
#bp.route('/video1', methods=['GET', 'POST'])
#login_required
def video1():
video1Form = Video1Form()
if request.method == 'POST' and video1Form.validate_on_submit():
newVideo1 = Video1(
slogan = video1Form.slogan.data,
objectif = video1Form.objectif.data,
stat1 = video1Form.stat1.data,
stat2 = video1Form.stat2.data
)
name = secure_filename(str(current_user.id) + "__" + "logo.png")
fh = open(video1Form.logo.data, 'rb')
newVideo1.logo.put(fh, filename=name, content_type = "image/jpeg")
newVideo1.save()
return redirect(url_for('video.download'))
if video1Form.errors:
flash(video1Form.errors, 'alert-danger')
return render_template('video/video1.html', video1Form=video1Form)
`
I get the following error on:
[...]File "/home/lucaspierru/Documents/IKADA/sos_sponsors/app/video/routes.py", line 35, in video1
fh = open(video1Form.logo.data, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'name_of_the_uploaded_file.ext'
Is it possible to point directly to the file's content and not just get the filename without saving it to the server first ?
I found my mistake:
I forgot to add the enctype="multipart/form-data" attribute to my HTML <form> tag and apparently it is needed to properly upload files, otherwise they'll be considered as text data.
Related
Having the following binary file (mp3) that send audio to a service in Azure to be trascripted.
The following code works in Databricks.
import os
import requests
url = "https://endpoint_service"
headers = {
'Ocp-Apim-Subscription-Key': 'MyKey',
'Content-Type': 'audio/mpeg'
}
def send_audio_transcript(url, payload, header):
"""Send audio.mp3 to a Azure service to be transcripted to text."""
response = requests.request("POST", url, headers=headers, data=payload)
return response.json()
full_path = <my_path>file.mp3
with open(full_path, mode='rb') as file: # b is important -> binary
fileContent = file.read()
send_audio_transcript(url, fileContent, headers) # a POST request its works
But my audio files are in a sensitive storage in Data lake and the only way to access them is by spark read.
looking for the documentation the way to read a binary file is.
df = spark.read.format("binaryFile").load(full_path)
display(df)
path || modificationTime || length || content
path || sometime || some_lenght || 2dnUwAC
first try:
content = df.content
test_service = send_audio_transcript(url, content , headers)
ValueError: Cannot convert column into bool: please use '&' for 'and', '|' for 'or', '~' for 'not' when building DataFrame boolean expressions.
Second try(convert spark to pandas):
pandas_df = df.toPandas()
content = pandas_df["content"]
test_service = send_audio_transcript(url, content , headers)
Valuerror:ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
What is the exactly translate in python-pyspark to:
with open(full_path, mode='rb') as file: # b is important -> binary
fileContent = file.read()
Your content data comming from Spark is not the same as the content data comming from open file.
From spark and later pandas you have a pandas series but from open the file you will have a class bytes
with open(full_path, mode='rb') as file: # b is important -> binary
fileContent = file.read()
print(type(fileContent)) # will return <class 'bytes'>
but from Spark
input_df = spark.read.format("binaryFile").load(full_path)
pandas_df = input_df.toPandas()
content = pandas_df['content']
print(type(content)) # return <class 'pandas.core.series.Series'>
In your case to fix your problem you need to take just the first element of the series.
content_good = content[0]
print(content_good) # you have your <class 'bytes'> wich is what you need
I'm trying to copy paste an html table received by outlook email to a new excel spreadsheet but I get a "pywintypes.com_error." Seeking a more pythonic way to do a the equivalent of a "Control+A" on an email body and paste to a new spreadsheet.
The relevant pieces of code are:
import win32com.client
outlook = win32com.client.Dispatch("Outlook.Application").GetNamespace("MAPI")
# Select main Inbox
inbox = outlook.GetDefaultFolder(6)
messages = inbox.Items
try:
for message in messages:
try:
if message.subject == 'myemailed Report':
print('Sender:' , message.sender)
print(message.subject)
mailItem = message.HTMLBody # <----Attempting to copy the body of the selected email.
# Start an instance of Excel
Xlsx = win32com.client.Dispatch("Excel.Application")
# Prevent Excel from asking questions.
Xlsx.DisplayAlerts = True # will change to False
Xlsx.Visible = True # will change to False
# Create a new Excel Workbook
workbook = Xlsx.Workbooks.Add()
ws = workbook.Sheets("Sheet1")
ws.Range('a7').select
ws.Paste(mailItem) # <--------------- Generates Error
workbook.SaveAs(mydesktop+'UpdatedSheet.xlsx')
# Quit Excel
Xlsx.Quit()
except:
x=1
except:
x=1
I get a message: Traceback (most recent call last):
File "", line 1, in
ws.Paste(mailItem)
File ">", line 3, in Paste
pywintypes.com_error: (-2147352567, 'Exception occurred.', (0, 'Microsoft Excel', 'Paste method of Worksheet class failed', 'xlmain11.chm', 0, -2146827284), None)
Is there a better way? Help is appreciated!
ws.Paste(mailItem) is the main part of the problem. Code should be:
ws.Paste()
However - copying the email body as if hitting "Ctrl-A" is a little more involved and there are a lot of almost answers. I managed to get the following work but I don't know why it works.
I used import pyperclip which requires pip install pyperclip, along with the following code:
import pyperclip
def copy(text):
win32clipboard.OpenClipboard()
win32clipboard.EmptyClipboard()
win32clipboard.SetClipboardText(text, win32clipboard.CF_UNICODETEXT)
win32clipboard.CloseClipboard()
def paste():
win32clipboard.OpenClipboard()
data = win32clipboard.GetClipboardData(win32clipboard.CF_UNICODETEXT)
win32clipboard.CloseClipboard()
return data
Then later on...
mailItem = message.HTMLBody
pyperclip.copy(mailItem)
# Start an instance of Excel
Xlsx = win32com.client.Dispatch("Excel.Application")
# Create a new Excel Workbook
workbook = Xlsx.Workbooks.Add()
ws = workbook.Sheets("Sheet1")
ws.Range('a1').select
ws.Paste()
ws.Range('a1').select
workbook.SaveAs(myexcel.xlsx')
I tried doing this without using pyperclip but the combination of the two def's in the beginning and the ws.Paste() worked.
I am trying to upload a file through the post request in flask restfulplus
I followed the example in this link
Here is my code
#ns.route('/upload/logo/')
#ns.doc('Upload Logo')
class UploadImage(Resource):
#ns.param('picture', 'image file path')
# #ns.route('/upload/')
def post(self):
parse = reqparse.RequestParser()
parse.add_argument('picture', type=FileStorage, location='files', required=True)
argsd = parse.parse_args()
print(argsd)
imgFile = argsd['picture']
print(imgFile)
imgFile.save("your_file_name.jpg")
return {'successfully' : 'loaded'}
For some reason the file is not getting uploaded. Can someone help me to solve this problem.
[Input Upload][2]
[2]: https://i.stack.imgur.com/BQc4W.png
I get the following error when trying to process uploaded images with imagick.
Fatal error: Uncaught exception 'ImagickException' with message 'unable to open image `9eK59iu.jpg': No such file or directory # error/blob.c/OpenBlob/2644' in D:\PATH\upload.php on line 77
The code looks like this:
<?php
$new_folder_name = "D:/PATH/content";
mkdir("$new_folder_name",0700);
$tmp_img = $_FILES["upload_file"]["tmp_name"];
$img = new Imagick($tmp_img);
$img->thumbnailImage(100 , 100 , TRUE);
$img->writeImage($new_folder_name);
?>
Without imagick the image upload works just fine.
Only imagick won't open the image given to $_FILES
I also tried to open the image with imagick, after move_uploaded_file, like this:
<?php
$extension = pathinfo($upload_file_name, PATHINFO_EXTENSION);
$new_upload_file_name = rand(00000, 99999).".".$extension;
$new_folder_name = "D:/PATH/content".time();
mkdir("$new_folder_name",0700);
$path_to_file = $new_folder_name."/".$new_upload_file_name;
move_uploaded_file($_FILES["upload_file"]["tmp_name"],$path_to_file);
$img = new Imagick($path_to_file);
$img->thumbnailImage(100 , 100 , TRUE);
$img->writeImage($new_folder_name);
?>
neither works.. :-(
Any suggestion?
Read the file upload docs. The server-side temporary filename assigned by PHP to store the uploaded file in ['tmp_name'] in the $_FILES array. You're trying to use the client-side user-provided ['name'], which DOES NOT exist anywhere on your server.
$tmp_img = $_FILES["upload_file"]["tmp_name"];
^^^^
You are also simply assuming that the upload has succeed. That is NOT a good thing. Never EVER assume success with dealing with remote resources (web apis, file uploads, database operations, etc...). ALWAYS check for errors:
if ($_FILES['upload_file']['error'] !== UPLOAD_ERR_OK) {
die("Upload failed with error code " . $_FILES['upload_file']['error']);
}
I'm new to web app and I want to check when there's a new version of dota map, I'll check links in getdota.com.
How can I do this and which language, I want it checks every time you start warcraft, and auto download new map to specific folder.
My question is : Can you give a link to a specific article about web automation or something like that.
Thanks first :)
Below is an example in Python.
It parses getdota.com page, reads parameters for POST request for downloading a map, gets the file and saves it in configured directory (by default current directory).
#!/usr/bin/env python
import urllib
import urllib2
import sgmllib
from pprint import pprint
import os.path
import sys
url = 'http://www.getdota.com/'
download_url = 'http://www.getdota.com/app/getmap/'
chunk = 10000
directory = '' #directory where file should be saved, if empty uses current dir
class DotaParser(sgmllib.SGMLParser):
def parse(self, s):
self.feed(s)
self.close()
def __init__(self, verbose=0):
sgmllib.SGMLParser.__init__(self, verbose)
self.URL = ''
self.post_args = {}
def getArgs(self):
return self.post_args
def start_input(self, attributes):
d = dict(attributes)
if d.get('id', None) == None:
return
if d['id'] in ["input_mirror2", "input_file_name2", "input_map_id2", "input_language2", "input_language_id2"]:
self.post_args[d['name']] = d['value']
if __name__ == '__main__':
dotap = DotaParser()
data = urllib2.urlopen(urllib2.Request('http://www.getdota.com/')).read()
dotap.parse(data)
data = urllib.urlencode(dotap.getArgs())
request = urllib2.Request(download_url, data)
response = urllib2.urlopen(request)
page = response.read()
#download file
fname = directory + page.split('/')[-1]
if os.path.isfile(fname):
print "No newer file available"
sys.exit(0)
f = open(fname, 'w')
print "New file available. Saving in: %s" % fname
webFile = urllib.urlopen(page)
c = webFile.read(chunk)
while(c):
f.write(c)
c = webFile.read(chunk)
f.close()
webFile.close()