can't get pictures from gridFS using Tornado - mongodb

my problem is that i can get the save picture to gridFS even if it's there, i've verified and it show me the piture and its name and the size from the console.
here is the code:
conn = Connection()
the classe that saves to the database:
class Profile(tornado.web.RequestHandler):
def post(self):
self.db = conn["essog"]
avat = self.request.files['avatar'][0]["body"]
avctype = self.request.files['avatar'][0]["content_type"]
nomfich = self.request.files['avatar'][0]["filename"]
#..operation using PIL to decide to save the picture or not
self.fs = GridFS(self.db)
avatar_id = self.fs.put(avat, content_type=avctype, filename=nomfich) #change the name later to avoid delete using the same name, so generating a different name...
.....
user={..., "avatar":avatar_id}
self.db.users.insert(user)
self.db.users.save(user)
the class that reads from the database:
class Profil(tornado.web.RequestHandler):
def get(self):
self.db = conn["essog"]
self.fs = GridFS(self.db)
avatar_id = self.db.users.find_one()["avatar"]
...
avatar = self.fs.get(avatar_id).read()
self.render("profile.html",..., avatar=avatar)
and in the view (profile.html)
img src="{{avatar}}" />
but nothing is displayed!

Unless you want to use a base64 URI for the source of the image, you should use a url and then create a view for returning the data from that view. If you are using nginx you might be interested in the nginx-gridfs module for better performance.

The src attribute of an img tag does not (typically) contain the image data itself, but rather the URL of the image. I think you're confusing two separate requests and responses:
HTML page that contains an <img src="..." /> tag:
class Profil(tornado.web.RequestHandler):
self.render('profile.html',
avatar=self.reverse_url('avatar', avatar_id))
image itself (which needs a separate handler):
class Avatar(tornado.web.RequestHandler):
def get(self, avatar_id):
avatar = self.fs.get(avatar_id)
self.set_header('Content-Type', avatar.content_type)
self.finish(avatar.read())

Related

How to store an image in a mongodb document using python and pillow?

I have the following code that creates a thumbnail from a request to a url:
r = requests.get(image_url, stream=True, headers=headers)
size = 500, 500
img = Image.open(r.raw)
thumb = ImageOps.fit(img, size, Image.ANTIALIAS)
At this point I would like to store the image inside a mongo document like so:
photo = {
'thumbnail': img,
'source': source,
'tags': tags,
'creationDate': datetime.now(),
}
Obviously that won't work so what kind of transformation do I need to apply before I can do this?
Okay here are my thoughts on this (I am not certain it will work though; some thoughts adopted from here).
I think you can achieve what you need using the Binary BSON type in pymongo library. Try loading the image in binary. Say using PILLOW (pil.image) or
image_file = open('1.bmp', 'rb')
or as
image_file = StringIO(open("test.jpg",'rb').read())
and then send it to Binary(image_file) type in pymongo
Binary_image_file = Binary(image_file) #pymongo libary
Then do a normal insert in mongo.
To read. do a normal find(). Then load the value from key and convert the data stored to image as:
image_data = StringIO.StringIO(Stringio_image_file)
image = Image.open(image_data)
I hope that helps a little. (also you could go with Aydin base64 proposition).
All the best.

How do I save the file name in the database

When I add a new record in a database using a form, I also can upload an image. These two are not linked together; record goes in database and the image goes in an folder on my desktop, so to know which image belongs to which record, I want to put the filename in a column. How do i approach this?
Im using PlayFramework 2.4, Scala, H2 Database and Anorm for my Project
In your html form you need to have an input tag of file type, something like:
<input type="file" name="picture">
And in your scala method Controller, where you are getting the form submit, something like:
def save = Action(parse.multipartFormData) { request =>
request.body.file("picture").map { picture =>
import java.io.File
val filename = picture.filename
println(filename)
Ok("saved")
}
You could retrieve the absolute path of the file like so:
scala> val filePath = getClass.getResource("myImage.png")
filePath: java.net.URI = file:/home/robert/myImage.png

Scrape different pages using Scrapy

I've been trying to scrape different pages. First, I scrape a URL from the first page using the xpath(#href) at the parse function. And then I try to scrape the article at that URL, from the parse function request callback. But it doesn't work.
How can I solve this issue? Here is my code:
import scrapy
from string import join
from article.items import ArticleItem
class ArticleSpider(scrapy.Spider):
name = "article"
allowed_domains = ["http://joongang.joins.com"]
j_classifications = ['politics','money','society','culture']
start_urls = ["http://news.joins.com/politics",
"http://news.joins.com/society",
"http://news.joins.com/money",]
def parse(self, response):
sel = scrapy.Selector(response)
urls = sel.xpath('//div[#class="bd"]/ul/li/strong[#class="headline mg"]')
items = []
for url in urls:
item = ArticleItem()
item['url'] = url.xpath('a/#href').extract()
item['url'] = "http://news.joins.com"+join(item['url'])
items.append(item['url'])
for itm in items:
yield scrapy.Request(itm,callback=self.parse2,meta={'item':item})
def parse2(self, response):
item = response.meta['item']
sel = scrapy.Selector(response)
articles = sel.xpath('//div[#id="article_body"]')
for article in articles:
item['article'] = article.xpath('text()').extract()
items.append(item['article'])
return items
The problem here is that you restrict the domains: allowed_domains = ["http://joongang.joins.com"]
If I change this to allowed_domains = ["joins.com"] I get results in parse2 and article text is extracted -- as unicode but this is OK since the site is not written in latin characters.
And by the way: you can use response.xpath() instead of creating a selector over the response object. This requires some less code and makes it easier to code.

How to obtain the image path/ url from a file uploaded on an openshift data dir to be stored in a postgresql database?

I am a self-taught programming newbie here so please bear with me. I am trying to create a site which allows users to upload their images on. With the patience of another user, I was able to get some answers as to how to create and allow users to upload their images onto the data drive on openshift. However, now I need to be able to store the image path or url onto a postgresql database (which can be called on later) so that each user will be able to keep track of the images that they have uploaded. I am currently stymied by this.
Here are the fragments of code which I feel plays a big role in answering this question:
class Todo(db.Model):
__tablename__ = 'todos'
id = db.Column('todo_id', db.Integer, primary_key=True)
title = db.Column(db.String(60))
text = db.Column(db.String)
done = db.Column(db.Boolean)
pub_date = db.Column(db.DateTime)
user_id = db.Column(db.Integer, db.ForeignKey('users.user_id'))
image_url = db.Column(db.String)
def __init__(self, title, text, image_url):
self.title = title
self.text = text
self.image_url = image_url
self.done = False
self.pub_date = datetime.utcnow()
def allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1] in app.config['ALLOWED_EXTENSIONS']
#app.route('/upload', methods=['POST'])
def upload():
# Get the name of the uploaded file
file = request.files['file']
# Check if the file is one of the allowed types/extensions
if file and allowed_file(file.filename):
# Make the filename safe, remove unsupported chars
filename = secure_filename(file.filename)
# Move the file form the temporal folder to
# the upload folder we setup
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
# Redirect the user to the uploaded_file route, which
# will basicaly show on the browser the uploaded file
return redirect(url_for('uploaded_file',
filename=filename))
#app.route('/uploads/<filename>')
def uploaded_file(filename):
return send_from_directory(app.config['UPLOAD_FOLDER'],
filename)
#app.route('/new', methods=['GET', 'POST'])
#login_required
def new():
if request.method == 'POST':
if not request.form['title']:
flash('Title is required', 'error')
elif not request.form['text']:
flash('Text is required', 'error')
else:
todo = Todo(request.form['title'], request.form['text'])
todo.user = g.user
db.session.add(todo)
db.session.commit()
flash('Todo item was successfully created')
return redirect(url_for('index'))
return render_template('new.html')
The current code that I have is pieced together by various tutorials and examples.
Currently, I am trying to merge the "Todo" db, "upload" function and "new" function and am having very little success. Using the little knowledge I have, I have merely added the "image_url" portions in which will be a column that is intended to house the image path. I would greatly appreciate it if somebody could shed some light on this conundrum. Thanks a million.
Respectfully,
Max

store (binary) file - play framework using scala in heroku

I'm trying to store user-uploaded images in my application which is written by scala and play framework 2.2.x
I've deployed my app in heroku.
Heroku does not allow me to save my file in file system.
So I've tried to store my file in data base.
here is the code that I use for storing image :
def updateImage(id: Long, image: Array[Byte]) = {
val selected = getById(id)
DB.withConnection {
implicit c =>
SQL("update subcategory set image={image} where id = {id}").on('id -> id, 'image -> image).executeUpdate()
}
selected }
and here is the code that I use to retreive my image :
def getImageById(id: Long): Array[Byte] = DB.withConnection {
implicit c =>
val all = SQL("select image from subcategory where id = {id}").on('id -> id)().map {
case Row(image: Array[Byte]) => image
case Row(Some(image: Array[Byte])) => image
case Row(image: java.sql.Blob )=> image.getBytes(0 , image.length().toInt)
}
all.head
}
The problem is: when I use H2 database and blob column, I get the "Match Error" exception.
When I use Postgresql and bytea column, I got no error but when I retrieve the image, It's in hex format and some of the bytes in the beginning of the array are missing.
According to the PostgreSQL documentation, bytea stores the length of the array in the four bytes at the beginning of the array. These are stripped when you read the row, so that's why they seem to be "missing" when you compare the data in Scala with the data in the DB.
You will have to set the response's content-type to the appropriate value if you want the web browser to display the image correctly, as otherwise it does not know it is receiving image data. The Ok.sendFile helper does it for you. Otherwise you will have to do it by hand:
def getPicture = Action {
SimpleResult(
header = ResponseHeader(200),
body = Enumerator(pictureByteArray))
.as(pictureContentType)
}
In the example above, pictureByteArray is the Array[Byte] containing the picture data from your database, and pictureContentType is a string with the appropriate content type (for example, image/jpeg).
This is all quite well explained in the Play documentation.