Django REST framework let you set a Pagination.
When we make a request it returns a JSON object, with a 'paging' attribute that has 'next' and/or 'previous' attributes containing urls of other pages (if they exist).
Does Tornado provide an option to set Pagination?
I have searched the web for tornado pagination example, but could not find any.
Any help is appreciated.
Thanking you in advance.
Nope, because pagination needs to work pretty closely with the database, and Tornado is DB agnostic.
Here's a UI module that I use; you need a count of your total results, as well as the results for the page (this doesn't limit those results for you, if just builds the pagination links):
from __future__ import division
import math
import urlparse
import urllib
import tornado.web
def update_querystring(url, **kwargs):
base_url = urlparse.urlsplit(url)
query_args = urlparse.parse_qs(base_url.query)
query_args.update(kwargs)
for arg_name, arg_value in kwargs.iteritems():
if arg_value is None:
if query_args.has_key(arg_name):
del query_args[arg_name]
query_string = urllib.urlencode(query_args, True)
return urlparse.urlunsplit((base_url.scheme, base_url.netloc,
base_url.path, query_string, base_url.fragment))
class Paginator(tornado.web.UIModule):
"""Pagination links display."""
def render(self, page, page_size, results_count):
pages = int(math.ceil(results_count / page_size)) if results_count else 0
def get_page_url(page):
# don't allow ?page=1
if page <= 1:
page = None
return update_querystring(self.request.uri, page=page)
next = page + 1 if page < pages else None
previous = page - 1 if page > 1 else None
return self.render_string('uimodules/pagination.html', page=page, pages=pages, next=next,
previous=previous, get_page_url=get_page_url)
Here's the module template (uimodules/pagination.html in the above example):
{% if pages > 1 %}
<div class="pagination pagination-centered">
<ul>
<li{% if previous %}>«{% else %} class="disabled"><span>«</span></li>{% end %}
{% for page_num in xrange(1, pages + 1) %}{# 1-index range #}
<li{% if page_num != page %}>{{ page_num }}{% else %} class="active"><span>{{ page_num }}</span></li>{% end %}
{% end %}
<li{% if next %}>»{% else %} class="disabled"><span>»</span></li>{% end %}
</ul>
</div>
{% end %}
Don't forget to tell your tornado app about the module.
Finally, to actually use it:
{% for result in results %}
<p>{{ result }}</p>
{% end %}
{% module Paginator(page, page_size, results_count) %}
Hope that helps!
Related
I'm new to flask world and wondering what causes the program to fail. every time i run my app i'm getting interfaceError. i was able to print the data on command line but this error displayed when i'm trying to render on html. thank you so much for your help
here is my mainApp.py file
from flask import Flask, render_template
from flask_sqlalchemy import SQLAlchemy
from models import People, db
from flask_migrate import Migrate
import psycopg2
# creates an application that is named after the name of the file
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = "postgresql://student_user:1234#localhost:5432/studentdb"
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
db.init_app(app)
migrate = Migrate(app, db)
#conn = "postgresql://student_user:1234#localhost:5432/studentdb"
conn = psycopg2.connect(dbname="teacherdb", host="localhost", user="teacher_user", password=1234)
cur = conn.cursor()
#app.route('/')
def index():
# Execute a query
cur.execute("SELECT * FROM teachertb")
# Retrieve query results
records = cur.fetchall()
print(records)
conn.close()
return render_template("index.html", records=records)
if __name__ == "__main__":
app.run(debug=True)
index.html
<!DOCTYPE html>
<html>
<head>
<meta charset='utf-8'>
<meta http-equiv='X-UA-Compatible' content='IE=edge'>
<title>Page Title</title>
<meta name='viewport' content='width=device-width, initial-scale=1'>
</head>
<body>
{% for record in range(0, len(records)) %}
<p>{{ records[record][1] }}</p>
{% endfor %}
</body>
</html>
Lastly, if name == "main": checks that the current file is being run as the main program and calls app.run(debug=True) to start the application with the debug mode enabled.
Please note that the configuration of the database(like user name,password,host) may have to be updated accordingly. It also seems like the connection is closed before sending the records variable to the template, which may cause the program to not work properly.
I have rigged up a custom "Advanced Search" view by slightly extending the flask admin index view. This takes the user to a custom form rendered using the flask admin form rendering machinery to do most of the work.
The form is defined as follows:
class PaperSearchForm(FlaskForm):
return_url = HiddenField()
title = StringField()
abstract = StringField()
start_date = DateTimeField()
end_date = DateTimeField()
doi = StringField()
pubmed_id = StringField()
link = StringField()
journals = QuerySelectMultipleField(
query_factory=_get_model(Journal),
)
authors = QuerySelectMultipleField(
query_factory=_get_model(Author),
)
keywords = QuerySelectMultipleField(
query_factory=_get_model(Keyword),
)
chapters = QuerySelectMultipleField(
query_factory=_get_model(Chapter),
)
printed = BooleanField(default=True)
unprinted = BooleanField(default=True)
submit = SubmitField('Search')
The Advanced Search model view is defined like this:
from flask import flash
from flask import redirect
from flask import request
from flask_admin import BaseView
from flask_admin import expose
from flask_wtf import FlaskForm
from flask_login import current_user
from .forms import PaperSearchForm
class AdvancedPaperSearchView(BaseView):
form_base_class = FlaskForm
def __init__(self,
name=None,
category=None,
endpoint=None,
url=None,
template='auth/model/paper/advanced_search.html',
menu_class_name=None,
menu_icon_type=None,
menu_icon_value=None
):
super(AdvancedPaperSearchView, self).__init__(
name,
category,
endpoint,
url or '/',
'static',
menu_class_name=menu_class_name,
menu_icon_type=menu_icon_type,
menu_icon_value=menu_icon_value)
self._template = template
def is_visible(self):
return False
def is_accessible(self):
if current_user.is_authenticated:
return current_user.can_view_papers()
return False
#expose('/', methods=['GET', 'POST'])
def index(self):
form = PaperSearchForm()
form.return_url.data = request.args['return_url']
self._template_args['form'] = form
self._template_args['cancel_url'] = request.args['return_url']
return self.render(self._template)
#expose('/search', methods=['POST'])
def search(self):
# List view generates list of models based on 'term'= from request.args.get('term', default=None)
# Manually setting these arguments will serve as the advanced search functionality
form = PaperSearchForm() # ???
search = None # ???
filter = None # ???
flash('How to apply multiple filters?', 'error')
return redirect('papermodelview.index', search=search, filter=filter) # ???
Then, the template is defined like this:
{% extends "admin/master.html" %}
{% import 'admin/lib.html' as lib with context %}
{% from 'admin/lib.html' import extra with context %} {# backward compatible #}
{% from 'admin/lib.html' import render_field with context %}
{% block head %}
{{ super() }}
{{ lib.form_css() }}
{% endblock %}
{% block body %}
{% block navlinks %}
<ul class="nav nav-tabs">
<li>
List
</li>
<li class="active">
Advanced Search
</li>
</ul>
{% endblock %}
<form method="post" action="{{ url_for('advancedpapersearchview.search') }}">
{{ form.return_url }}
{{ form.csrf_token }}
{{ render_field(form, form.title) }}
{{ render_field(form, form.abstract) }}
{{ render_field(form, form.start_date) }}
{{ render_field(form, form.end_date) }}
{{ render_field(form, form.doi) }}
{{ render_field(form, form.pubmed_id) }}
{{ render_field(form, form.link) }}
{{ render_field(form, form.journals) }}
{{ render_field(form, form.authors) }}
{{ render_field(form, form.chapters) }}
{{ render_field(form, form.keywords) }}
{{ render_field(form, form.printed) }}
{{ render_field(form, form.unprinted) }}
<div class="row">
<div class="col-xs-12">
{{ form.submit(class="btn") }}
</div>
</div>
<div class="row">
<div class="col-xs-12">
<a href="{{ cancel_url }}" class="btn warning">
Cancel
</a>
</div>
</div>
</form>
{% endblock %}
{% block tail %}
{{ super() }}
{{ lib.form_js() }}
<script src="/static/vendor/jquery.min.js" type="text/javascript">/script>
{# use /static/bootstrap2/js/bootstrap.min.js if you are using bootstrap2 #}
<script src="/static/bootstrap3/js/bootstrap.min.js" type="text/javascript"></script>
<script src="/static/vendor/moment.min.js" type="text/javascript"></script>
<script src="/static/vendor/select2/select2.min.js" type="text/javascript"></script>
{% endblock %}
In the Paper Model View, the filters are defined like this:
class PaperModelView(MainModelView):
# ...
column_filters = [
'chapter_paper_assoc.printed',
'journal_paper_assoc.publication_date',
'chapters.name',
'chapters.number',
'journals.name',
'authors.last_name',
'keywords.keyword',
]
So, I commented a bunch of ??? where I don't know what to do. How do I map the fields of my form (specific attributes of the models selected) to the filters that are defined by 'column_filters' in the model view.
I.e -- rather than overriding the index view search handling to actually perform a search, I could instead apply a bunch of filters by passing this information to the index view, which retrieves this information with:
filters=response.args.get('filter', None)
Is there a better approach?
Thanks
Alright, things are about to get ugly, so hold onto your hat.
This is what I ended up writing to accomplish this functionality, and it is code that I am not particularly proud to have written. It works, but that said, please suggest a cleaner way to do this if you feel like it.
Here is the advanced search view:
from flask import flash
from flask import redirect
from flask import request
from flask import url_for
from flask_admin import BaseView
from flask_admin import expose
from flask_wtf import FlaskForm
from flask_login import current_user
from app import admin
from .forms import PaperSearchForm
class AdvancedPaperSearchView(BaseView):
form_base_class = FlaskForm
def __init__(self,
name=None,
category=None,
endpoint=None,
url=None,
template='auth/model/paper/advanced_search.html',
menu_class_name=None,
menu_icon_type=None,
menu_icon_value=None
):
super(AdvancedPaperSearchView, self).__init__(
name,
category,
endpoint,
url or '/',
'static',
menu_class_name=menu_class_name,
menu_icon_type=menu_icon_type,
menu_icon_value=menu_icon_value)
self._template = template
def is_visible(self):
return False
def is_accessible(self):
if current_user.is_authenticated:
return current_user.can_view_papers()
return False
#expose('/', methods=['GET', 'POST'])
def index(self):
form = PaperSearchForm()
form.return_url.data = request.args['return_url']
self._template_args['form'] = form
self._template_args['cancel_url'] = request.args['return_url']
return self.render(self._template)
#expose('/search', methods=['POST'])
def search(self):
form = PaperSearchForm()
# The goal here is to get the paper model view from the currently running app (and its admin extension). Once
# the model view is here, use it to get the available filters (get their keys and operations). Use the existing
# request args and add filters to them using the key and operations defined in the model view.
paper_model_view = None
for view in admin._views:
# There must be a better way to do this, and I know this is a WTF, but I don't have the vocabulary to search
# the flask admin documentation for the right way to get the instance of the model view from the admin
# object. I need the *instance*, with the filters created and added to that instance by the init... so...
# not clean or pretty ... and permanently restricts name of paper model view ... TODO: Fix? Rewrite?
# - Chris, March 2017
if "PaperModelView object" in view.__repr__():
paper_model_view = view
# ._filters contains the list of all filters
# ._filter_args contains a dictionary of keys and filter objects for url construction
# each filter is persisted with request.args, the query string is <flt[position]_[key]=[value]> or
# <flt[position]_[key]> for filters without values
# The filter is accessed by looking up the filter object with the key value, and then the filters are listed
# in the order of the position provided in the query string. I am unsure whether or not they are applied in
# this order, but that seems like what is happening.
filters = {}
i = 0
str = "flt{pos}_{key}"
def __check(column, table, filter):
return (column in filter.column.name and table in filter.column.table.name.__repr__())
# Sorry for this...
# Iterate through each filter available for the view. Check if it's name and operation are something that
# will enact a portion of the search, then add it's filter (in the format expected) to a dictionary. The index
# variable i keeps track of the "count" of filters that have been added and uses this as the position of the
# filter.
for key, key_filter in paper_model_view._filter_args.items():
filter = key_filter[1]
if hasattr(filter, 'column'):
if __check("title", "papers", filter):
if "FilterLike" in filter.operation.__repr__():
if form.title.data:
filters[str.format(pos=i, key=key)] = form.title.data
i += 1
if __check("abstract", "papers", filter):
if "FilterLike" in filter.operation.__repr__():
if form.abstract.data:
filters[str.format(pos=i, key=key)] = form.abstract.data
i += 1
if __check("publication_date", "journal_paper", filter):
if "DateSmaller" in filter.operation.__repr__():
if form.end_date.data:
filters[str.format(pos=i, key=key)] = form.end_date.data.date() # Only keeps the date for the filter
i += 1
elif "DateGreater" in filter.operation.__repr__():
if form.start_date.data:
filters[str.format(pos=i, key=key)] = form.start_date.data.date()
i += 1
if __check("doi", "papers", filter):
if "FilterLike" in filter.operation.__repr__():
if form.doi.data:
filters[str.format(pos=i, key=key)] = form.doi.data
i += 1
if __check("pubmed_id", "papers", filter):
if "FilterLike" in filter.operation.__repr__():
if form.pubmed_id.data:
filters[str.format(pos=i, key=key)] = form.pubmed_id.data
i += 1
if __check("link", "papers", filter):
if "FilterLike" in filter.operation.__repr__():
if form.link.data:
filters[str.format(pos=i, key=key)] = form.link.data
i += 1
if __check("name", "journal", filter):
if "FilterLike" in filter.operation.__repr__():
if form.journals.data:
for journal in form.journals.data:
filters[str.format(pos=i, key=key)] = journal.name
i += 1
if __check("first_name", "authors", filter):
if "FilterLike" in filter.operation.__repr__():
for author in form.authors.data:
filters[str.format(pos=i, key=key)] = author.first_name
i += 1
if __check("last_name", "authors", filter):
if "FilterLike" in filter.operation.__repr__():
for author in form.authors.data:
filters[str.format(pos=i, key=key)] = author.last_name
i += 1
if __check("keyword", "keywords", filter):
if "FilterLike" in filter.operation.__repr__():
for keyword in form.keywords.data:
filters[str.format(pos=i, key=key)] = keyword.keyword
i += 1
if __check("name", "chapters", filter):
if "FilterLike" in filter.operation.__repr__():
for chapter in form.chapters.data:
filters[str.format(pos=i, key=key)] = chapter.name
i += 1
if __check("printed", "chapter_paper", filter):
if "FilterEqual" in filter.operation.__repr__():
if form.printed.data == True:
if form.unprinted.data == False: # Printed only
filters[str.format(pos=i, key=key)] = 1 # True
i += 1
else:
pass # Both are True
else:
if form.unprinted.data == True: # Unprinted only
filters[str.format(pos=i, key=key)] = 0 # False
i += 1
else:
pass # Both are False
else:
continue
flash('Filters successfully applied', 'success')
return redirect(url_for('paper.index_view', **filters))
I am working on a project, to control a RC car with Raspberry Pi and have done it with he help of WeiOPi.
Now I wanted to add a Ultrasonic Range Finder on it (I have HC-SR04).
I googled in internet about it and didn't find much information, I have a question.
How I can combine the python code with java-script and html to print the Distance on the web page.
Thanks in advance,
Any Help will be appreciated.
Regards,
H.M
P.S, I am only 13 and new to Programming.
EDIT:
Unfinished Code:
The python code is:
import webiopi
import RPi.GPIO as GPIO
import time
GPIO.setmode(GPIO.BCM)
TRIG = 23
ECHO = 24
print "Distance Measurement In Progress"
GPIO.setup(TRIG,GPIO.OUT)
GPIO.setup(ECHO,GPIO.IN)
GPIO.output(TRIG, False)
print "Waiting For Sensor To Settle"
time.sleep(2)
GPIO.output(TRIG, True)
time.sleep(0.00001)
GPIO.output(TRIG, False)
while GPIO.input(ECHO)==0:
pulse_start = time.time()
while GPIO.input(ECHO)==1:
pulse_end = time.time()
pulse_duration = pulse_end - pulse_start
distance = pulse_duration * 17150
# _____________________________________________________________________
# I want to print the following string (distance) to print on html page
# ---------------------------------------------------------------------
#webiopi.macro
def getSensor(channel):
percent = round(distance, 2)
return "%.2f%%" % percent
#Instantiate the server on the port 8000, it starts immediately in its own thread
server = webiopi.Server(port=8000)
# Run our loop until CTRL-C is pressed or SIGTERM received
webiopi.runLoop()
# -------------------------------------------------- #
# Termination part #
# -------------------------------------------------- #
# Stop the server
server.stop()
The html code:
<script type="text/javascript" src="/webiopi.js"></script>
<script type="text/javascript">
webiopi().ready(init);
// defines function passed previously to webiopi().ready()
function init() {
// automatically refresh UI each seconds
setInterval(updateUI, 1000);
}
// function called through setInterval
function updateUI() {
webiopi().callMacro("getSensor", 0, sensorCallback);
webiopi().callMacro("getSensor", 1, sensorCallback);
webiopi().callMacro("getSensor", 2, sensorCallback);
webiopi().callMacro("getSensor", 3, sensorCallback);
}
// callback function used to display sensor data
function sensorCallback(macroName, channel, data) {
// use jQuery to change spans content
$("#sensor"+channel).text(data);
}
</script>
</head>
<body>
<div align="center">
<div>Sensor 0: <span id="sensor0"></span></div>
<div>Sensor 1: <span id="sensor1"></span></div>
<div>Sensor 2: <span id="sensor2"></span></div>
<div>Sensor 3: <span id="sensor3"></span></div>
</div>
</body>
</html>
I'd like to escape a HTML in my template, but no matter what I try it won't work
Here is my template:
<ul>
#for(index <- 0 until appointments.size){
<li>
#(utils.DateUtil.getLocalDate(appointments(index).getStartDate())) - #appointments(index).getReason()
<ul>
#{val procedure = appointments(index).getProcedures()
{if(procedure == null){
<b>Empty</b>
} else {
">b/<NotEmpty>/b<" +
procedure.size().toString+
procedure.size().toString+
<b>NotEmpty</b>+
"<b>NotEmpty</b>"+
"<b>NotEmpty</b>".toString;
}
}
}
</ul>
</li>
}
</ul>
The problematic code is in the else branch
I'm trying to print <b>NotEmpty</b> as NotEmpty but I've got just a plaintext, not html
I've tried #Html("<strong>Do not escape</strong>") but it says expected start of definition
if I delete the else branch contents and leave just
else {
<b>NotEmpty</b>;
}
It prints out fine.
I'm using play framework 2.2.1 built with Scala 2.10.2 (running Java 1.7.0_25)
How about:
#{
val x = Some(Seq("hi there"))
if(x.isDefined)
<b>size = {x.get.size}</b><br/>
<b>Not Empty</b>
else
<b>Empty</b>
}
I just created the following model:
class Categoria(models.Model):
nombre=models.CharField(max_length=30)
padre=models.ForeignKey('self', blank=True, null=True)
def __unicode__(self):
return self.nombre
Then registered to the admin interface and syncdb'd
Everything ok if I just add plain ASCII chars. But if I add a "Categoria" named "á" (to say something) I get:
Environment:
Request Method: GET
Request URL: http://192.168.2.103:8000/administracion/locales/categoria/
Django Version: 1.1.1
Python Version: 2.6.4
Installed Applications:
['django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.admin',
'cruzandoelsuquiaDJ.locales']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware')
Template error:
In template /usr/lib/pymodules/python2.6/django/contrib/admin/templates/admin/change_list.html, error at line 78
Caught an exception while rendering: ('ascii', '\xc3\xa1', 0, 1, 'ordinal not in range(128)')
68 : {% endif %}
69 : {% endblock %}
70 :
71 : <form action="" method="post"{% if cl.formset.is_multipart %} enctype="multipart/form-data"{% endif %}>
72 : {% if cl.formset %}
73 : {{ cl.formset.management_form }}
74 : {% endif %}
75 :
76 : {% block result_list %}
77 : {% if action_form and actions_on_top and cl.full_result_count %}{% admin_actions %}{% endif %}
78 : {% result_list cl %}
79 : {% if action_form and actions_on_bottom and cl.full_result_count %}{% admin_actions %}{% endif %}
80 : {% endblock %}
81 : {% block pagination %}{% pagination cl %}{% endblock %}
82 : </form>
83 : </div>
84 : </div>
85 : {% endblock %}
86 :
Traceback:
File "/usr/lib/pymodules/python2.6/django/core/handlers/base.py" in get_response
92. response = callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/pymodules/python2.6/django/contrib/admin/options.py" in wrapper
226. return self.admin_site.admin_view(view)(*args, **kwargs)
File "/usr/lib/pymodules/python2.6/django/views/decorators/cache.py" in _wrapped_view_func
44. response = view_func(request, *args, **kwargs)
File "/usr/lib/pymodules/python2.6/django/contrib/admin/sites.py" in inner
186. return view(request, *args, **kwargs)
File "/usr/lib/pymodules/python2.6/django/contrib/admin/options.py" in changelist_view
986. ], context, context_instance=context_instance)
File "/usr/lib/pymodules/python2.6/django/shortcuts/__init__.py" in render_to_response
20. return HttpResponse(loader.render_to_string(*args, **kwargs), **httpresponse_kwargs)
File "/usr/lib/pymodules/python2.6/django/template/loader.py" in render_to_string
108. return t.render(context_instance)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
178. return self.nodelist.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
779. bits.append(self.render_node(node, context))
File "/usr/lib/pymodules/python2.6/django/template/debug.py" in render_node
71. result = node.render(context)
File "/usr/lib/pymodules/python2.6/django/template/loader_tags.py" in render
97. return compiled_parent.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
178. return self.nodelist.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
779. bits.append(self.render_node(node, context))
File "/usr/lib/pymodules/python2.6/django/template/debug.py" in render_node
71. result = node.render(context)
File "/usr/lib/pymodules/python2.6/django/template/loader_tags.py" in render
97. return compiled_parent.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
178. return self.nodelist.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
779. bits.append(self.render_node(node, context))
File "/usr/lib/pymodules/python2.6/django/template/debug.py" in render_node
71. result = node.render(context)
File "/usr/lib/pymodules/python2.6/django/template/loader_tags.py" in render
24. result = self.nodelist.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
779. bits.append(self.render_node(node, context))
File "/usr/lib/pymodules/python2.6/django/template/debug.py" in render_node
71. result = node.render(context)
File "/usr/lib/pymodules/python2.6/django/template/loader_tags.py" in render
24. result = self.nodelist.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
779. bits.append(self.render_node(node, context))
File "/usr/lib/pymodules/python2.6/django/template/debug.py" in render_node
81. raise wrapped
Exception Type: TemplateSyntaxError at /administracion/locales/categoria/
Exception Value: Caught an exception while rendering: ('ascii', '\xc3\xa1', 0, 1, 'ordinal not in range(128)')
My django version is 1.1 and my database is 5.1.37-1ubuntu5 with utf8 charset and the table is using a utf8_bin collation.
This problem seems too basic to be true, and I'm a django newbie so I'm sorry in advance if I'm missing something very simple :)
Django generally has very good Unicode support (see the Django 1.1 "Unicode data" documentation for details). In my code I find that, if I'm having a problem with simple Unicode features, the problem usually is that I'm not understanding Django's details well, not that Django has a bug in its Unicode support.
The "Unicode Data" page tells us that "All of Django’s database backends ... automatically convert strings retrieved from the database into Python Unicode strings. You don’t even need to tell Django what encoding your database uses: that is handled transparently." So your simple return self.nombre should return a Python Unicode string.
However, the Django 1.1 "Databases" page has an important note about how the MySQL backend handles the utf8_bin collation:
...if you really want case-sensitive
comparisons on a particular column or
table, you would change the column or
table to use the utf8_bin collation.
The main thing to be aware of in this
case is that if you are using MySQLdb
1.2.2, the database backend in Django will then return bytestrings (instead
of unicode strings) for any character
fields it returns receive from the
database. This is a strong variation
from Django's normal practice of
always returning unicode strings. It
is up to you, the developer, to handle
the fact that you will receive
bytestrings if you configure your
table(s) to use utf8_bin collation.
Django itself should work smoothly
with such columns, but if your code
must be prepared to call
django.utils.encoding.smart_unicode()
at times if it really wants to work
with consistent data ...
So, in your original example, the column "nombre" used utf8_bin collation. This meant that self.nombre was returning a Python byte string. When you put it in an expression that required a Python Unicode string, Python performed its default conversion. This is the equivalent of self.nombre.decode('ascii'). And of course, .decode('ascii') fails when it encounters any byte above 0x7F, such as the UTF-8 bytes which encode "á".
You discovered the two ways to solve this problem. The first is to convert the Python byte string returned by self.nombre into a Python Unicode string explicitly. I'll bet the following simpler code would have worked:
return self.nombre.decode('utf8')
The second approach is to change the MySQL collation for column "nombre", which causes Django's MySQL backend to return Python Unicode strings instead of the unusual byte strings. Then your original expression gives a Python Unicode string:
return self.nombre
Hope this helps.
This problem can be solved by a little bit changing of django's code.
Add code below in django/utils/encoding.py
import sys
reload(sys)
sys.setdefaultencoding('utf-8')
I had this issue in production and never on the development server.
Then I realized that new tables were created with utf8_bin collation instead of utf8_general_ci.
To see which tables require conversion, type
SHOW TABLE STATUS;
Then convert those with utf8_bin collation by typing
ALTER TABLE app_table CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
Finally, change the default collation so this doesn't happen again:
ALTER DATABASE my_database character set utf8 collate utf8_general_ci;
Ok...
return u"%s"%(self.nombre.decode('utf8'),)
does the trick.
But also found that changing utf8_bin to utf8_general_ci does the trick, i.e. self.nombre works as expected.
I solved by problem by simply creating an Admin Model for the Model and including all variables in "list_display".
I had similar issue when recently changed a MySQL table to use collation utf8_bin in staging while no problem in dev (python2.7, Django1.4.2 in both environments). I found out that in dev I have MySQL-python 1.2.4c1 and in staging I have 1.2.3. Upgrading to MySQL-python 1.2.4 solved the problem for me.