constructing a mongo doc using pymodm from_document - mongodb

I am trying to construct a pymodm document from a python dictionary using from_document api. I am receiving a error "ValueError: Unrecognized field name 'prim_key'"
Here is the sample code that I have written to make this happen with minimum code.
from pymodm import connect, EmbeddedMongoModel, MongoModel, fields
from pymodm.errors import ValidationError, ConfigurationError
from pymongo.write_concern import WriteConcern
import pprint
from datetime import datetime
class sample_doc(MongoModel):
prim_key = fields.CharField(primary_key=True)
another_field = fields.CharField()
class Meta:
write_concern = WriteConcern(j=True)
connection_alias = 'pymodm-tester'
connect("mongodb://localhost:27017/pymodm-tester", alias="pymodm-tester")
q = {'prim_key' : 'prim_val', "another_field" : "another_filed1" }
sample_doc.from_document(q).save()

In the docs:
document: A Python dictionary describing a MongoDB document. Keys
within the document must be named according to each model field’s
mongo_name attribute, rather than the field’s Python name.
It looks that, in order to instantiate a Model as you try, it needs to be specified the mongo_name Field. It would work for you something like:
class sample_doc(MongoModel):
prim_key = fields.CharField(primary_key=True, mongo_name="prim_key")
another_field = fields.CharField(mongo_name="another_field")
class Meta:
write_concern = WriteConcern(j=True)
connection_alias = 'pymodm-tester'

Related

Cython read large binary file using class definition

My code tries to read a binary file using a Structure class:
import os
import mmap
from ctypes import Structure, c_uint
class StructFile(Structure):
_pack_ = 1
_fields_ = [('Xmin', c_uint),('Ymax', c_uint)]
fd = os.open(filePath, os.O_RDWR | os.O_BINARY)
mmap_file = mmap.mmap(fd, length=ln, access=mmap.ACCESS_WRITE, offset=0)
d_arrayFile = StructFile * numItems
data = d_arrayFile.from_buffer(mmap_file)
for i in data:
print i.Xmin, i.Ymax
It works correctly, but I would like to speed it up using a cython class. The problem is when I use cdef class StructFile(Structure): the compilation says: "'Structure' is not a type name".
How could I do in order to cython recognise 'Structure' as a type name like 'int', 'object', etc.? Or any other solution to optimise the code (not using struct.unpack).
Thanks!

how to make a custom filter in flask admin

What am I doing wrong in this?? I am getting an exception thrown.
Exception: Invalid field : does not contains any columns.
I actually want to filter results from three different columns containing the cost of rooms.If any room satisfies a certain condition from the three rooms then it should appear on the list. That's why I want to make a custom filter.
Please help me. Thanks in advance.
from flask.ext.admin.babel import lazy_gettext
from flask.ext.admin.model import filters
class MyBaseFilter(filters.BaseFilter):
"""
Base SQLAlchemy filter.
"""
def __init__(self, column, name, options=None, data_type=None):
"""
Constructor.
:param column:
Model field
:param name:
Display name
:param options:
Fixed set of options
:param data_type:
Client data type
"""
super(MyBaseFilter, self).__init__(name, options, data_type)
self.column = column
class Views():
class HotelAdmin(ModelView):
class FilterCost(MyBaseFilter):
def apply(self, query, value):
return query.filter(self.column > value)
def operation(self):
return lazy_gettext('Cost')
column_labels = {'hotel_name':'Hotel Details'}
column_list = ('hotel_name','website')
column_searchable_list = ('city',)
column_filters = (FilterCost(Table.Hotel.deluxe_room,'Cost'),'state')
I did the following and it worked:
added:
from flask.ext.admin.contrib.sqla import filters
removed:
from flask.ext.admin.model import filters AND
class MyBaseFilter(filters.BaseFilter)
and then I inherited "filters.BaseSQLAFilter" in my filter class.
But still, How can I use "class MyBaseFilter(filters.BaseFilter)", It is still throwing an error

how to use composite data types (e.g. geomval) in SQLAlchemy?

I'm trying to replicate a nested raw PostGreSQL/PostGIS raster query using SQLAlchemy(0.8)/GeoAlchemy2(0.2.1) and can't figure how to access the components of a geomval data type. It's a compound data type that returns a 'geom' and a 'val'.
Here is the raw query that works:
SELECT (dap).val, (dap).geom
FROM (SELECT ST_DumpAsPolygons(rast) as dap FROM my_raster_table) thing
And the SQLAlchemy query I'm currently working with:
import geoalchemy2 as ga2
from sqlalchemy import *
from sqlalchemy.orm import sessionmaker
metadata = MetaData()
my_raster_table = Table('my_raster_table', metadata,
Column('rid', Integer),
Column('rast', ga2.Raster))
engine = create_engine(my_conn_str)
session = sessionmaker(engine)()
q = session.query(ga2.func.ST_DumpAsPolygons(my_raster_table.c.rast).label('dap'))
And then I'd like to access that in a subquery, something like this:
q2 = session.query(ga2.func.ST_Area(q.subquery().c.dap.geom))
But that syntax doesn't work, or I wouldn't be posting this question ;). Anyone have ideas? Thanks!
The solution ended up being fairly simple:
First, define a custom GeomvalType, inheriting geoalchemy2's CompositeType and specifying a typemap specific to geomval:
class GeomvalType(ga2.types.CompositeType):
typemap = {'geom':ga2.Geometry('MULTIPOLYGON'),'val':Float}
Next, use type_coerce to cast the ST_DumpAsPolygons result to the GeomvalType in the initial query:
q = session.query(type_coerce(ga2.func.ST_DumpAsPolygons(my_raster_table.c.rast), GeomvalType()).label('dap'))
Finally, access it (successfully!) from the subquery as I was trying to before:
q2 = session.query(ga2.func.ST_Area(q.subquery().c.dap.geom))

Django-Nonrel with Mongodb listfield

I am trying to implement manytomany field relation in django-nonrel on mongodb. It was suggessted at to:
Django-nonrel form field for ListField
Following the accepted answer
models.py
class MyClass(models.Model):
field = ListField(models.ForeignKey(AnotherClass))
i am not sure where the following goes, it has been tested in fields.py, widgets,py, models.py
class ModelListField(ListField):
def formfield(self, **kwargs):
return FormListField(**kwargs)
class ListFieldWidget(SelectMultiple):
pass
class FormListField(MultipleChoiceField):
"""
This is a custom form field that can display a ModelListField as a Multiple Select GUI element.
"""
widget = ListFieldWidget
def clean(self, value):
#TODO: clean your data in whatever way is correct in your case and return cleaned data instead of just the value
return value
admin.py
class MyClassAdmin(admin.ModelAdmin):
form = MyClassForm
def __init__(self, model, admin_site):
super(MyClassAdmin,self).__init__(model, admin_site)
admin.site.register(MyClass, MyClassAdmin)
The following Errors keep popping up:
If the middle custom class code is used in models.py
name 'SelectMultiple' is not defined
If custom class code is taken off models.py:
No form field implemented for <class 'djangotoolbox.fields.ListField'>
You just need to import SelectMultiple by the sound of it. You can put the code in any of those three files, fields.py would make sense.
Since it's pretty usual to have:
from django import forms
at the top of your file already, you probably just want to edit the code below to:
# you'll have to work out how to import the Mongo ListField for yourself :)
class ModelListField(ListField):
def formfield(self, **kwargs):
return FormListField(**kwargs)
class ListFieldWidget(forms.SelectMultiple):
pass
class FormListField(forms.MultipleChoiceField):
"""
This is a custom form field that can display a ModelListField as a Multiple Select GUI element.
"""
widget = ListFieldWidget
def clean(self, value):
#TODO: clean your data in whatever way is correct in your case and return cleaned data instead of just the value
return value
You probably also want to try and learn a bit more about how python works, how to import modules etc.

Connecting to MongoDB from MATLAB

I'm trying to use MongoDB with MATLAB. Although there is no supported driver for MATLAB, there is one for Java. Fortunately I was able to use it to connect to db, etc.
I downloaded the latest (2.1) version of jar-file and install it with JAVAADDPATH. Then I tried to follow the Java tutorial.
Here is the code
javaaddpath('c:\MATLAB\myJavaClasses\mongo-2.1.jar')
import com.mongodb.Mongo;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.BasicDBObject;
import com.mongodb.DBObject;
import com.mongodb.DBCursor;
m = Mongo(); % connect to local db
db = m.getDB('test'); % get db object
colls = db.getCollectionNames(); % get collection names
coll = db.getCollection('things'); % get DBCollection object
doc = BasicDBObject();
doc.put('name', 'MongoDB');
doc.put('type', 'database');
doc.put('count', 1);
info = BasicDBObject();
info.put('x', 203);
info.put('y', 102);
doc.put('info', info);
coll.insert(doc);
Here is where I stacked. coll supposed to be DBCollection object, but actually is object of com.mongodb.DBApiLayer$MyCollection class. So the last command returns the error:
??? No method 'insert' with matching signature found for class 'com.mongodb.DBApiLayer$MyCollection'.
In the tutorial the coll variable is created explicitly as DBCollection object:
DBCollection coll = db.getCollection("testCollection")
Am I doing something wrong in MATLAB? Any ideas?
Another minor question about colls variable. It's com.mongodb.util.OrderedSet class and contains list of names of all collections in the db. How could I convert it to MATLAB's cell array?
Update: In addition to Amro's answer this works as well:
wc = com.mongodb.WriteConcern();
coll.insert(doc,wc)
A quick check:
methodsview(coll) %# or: methods(coll, '-full')
shows that it expects an array:
com.mongodb.WriteResult insert(com.mongodb.DBObject[])
Try this instead:
doc(1) = BasicDBObject();
doc(1).put('name', 'MongoDB');
doc(1).put('type', 'database');
...
coll.insert(doc);
Note: If you are using Java in MATLAB, I suggest you use the CheckClass and UIInspect utilities by Yair Altman
There is now a driver built expressly to connect MongoDB and Matlab. It is built on top of the mongo-c-driver. Source can be found on github:
https://github.com/gerald-lindsly/mongo-matlab-driver
For the minor question about converting the list of collections use the toArray() method.
>> cList=cell(colls.toArray())
cList =
'foo'
'system.indexes'
'things'