I have an issue with defining model in spyne to generate several levels "in" SOAP11.
I used example at first, but my task is to generate service for tables already existing, so I got stuck and try to understand wheter to seek in Spyne properties or Sqlalchemy.
To be precise, i'll take example from site and show what i'm trying to reach:
class Permission(TableModel):
__tablename__ = 'permission'
id = UnsignedInteger32(pk=True)
application = Unicode(values=('usermgr', 'accountmgr'))
operation = Unicode(values=('read', 'modify', 'delete'))
perm_user_id = integer
last field is the FK for user table, but its name is different from user_id
class User(TableModel):
__tablename__ = 'user'
id = UnsignedInteger32(pk=True)
user_name = Unicode(32, min_len=4, pattern='[a-z0-9.]+', unique=True)
full_name = Unicode(64, pattern='\w+( \w+)+')
email = Unicode(64, pattern=r'[a-z0-9._%+-]+#[a-z0-9.-]+\.[A-Z]{2,4}')
last_pos = Point(2, index='gist')
permissions = Array(Permission).store_as('table')
--- SQL generated tries to add "WHEN user.id = permission.user_id" but I need another field (perm_user_id) to be filtered
Help me to define class to get correct inner tags.. actually it'll be about 3 more classes deep.
Thanx in Advance, Yury
Your answer is correct. Just as an alternative for simple tables, you can omit column definitions and let sqlalchemy's reflection engine figure it out.
meta = TableModel.Attributes.sqla_metadata
meta.reflect()
class User(TableModel):
__table__ = meta.tables['user']
The User class will be reconstructed using as much information as possible from the table columns and their types.
Found it myself, sorry to disturb anyone,
from spyne.model.complex import table
Permissions= Array(permission).customize(store_as=table(right='perm_user_id'))
Related
I did a bunch of research but I did not get through a sophisticated example helping me out to use ArrayOfEnum() to store a class-specific Enum in a PostgreSQL database. I do use Flask, Flask-SQLAlchemy, as well SQLAlchemy.
My example originates by a WTForm using a MultiSelect which I want to store in the database:
genres = SelectMultipleField(
'genres', validators=[DataRequired()], choices=[
('Alternative', 'Alternative'),
('Blues', 'Blues'),
...
])
I would like to define an enum in python like:
class Genres(enum.Enum):
alternative ='Alternative'
blues ='Blues'
classical = 'Classical'
And use it as an authorized data type in the model, like this:
class Artist(db.Model):
__tablename__ = 'Artist'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String)
city = db.Column(db.String(120))
genres = db.Column(pg.ArrayOfEnum(Genres))
My general setup is: requirements.txt:
alembic==1.7.7
Babel==2.9.0
click==8.1.2
colorama==0.4.4
Flask==2.1.1
Flask-Migrate==3.1.0
Flask-Moment==0.11.0
Flask-SQLAlchemy==2.4.4
Flask-WTF==0.14.3
greenlet==1.1.2
importlib-metadata==4.11.3
itsdangerous==2.1.2
Jinja2==3.0.3
Mako==1.2.0
MarkupSafe==2.1.1
postgres==4.0
psycopg2-binary==2.9.3
psycopg2-pool==1.1
python-dateutil==2.6.0
pytz==2022.1
six==1.16.0
SQLAlchemy==1.4.35
Werkzeug==2.0.0
WTForms==3.0.1
zipp==3.8.0
In this question I do specifically ask for the solution using the ENUM with ARRAY, but I'm also open to use other solutions that might fit.
I am doing something fairly straightforward...I am creating a record with a unique identifier using the UUID field in my POSTRGRESQL database. It's working fine. I'm trying to use that value and do a compare when the user is updating records. However, even though the value is in the database, when I try to get it via a ModelFORM in my CLEAN it is showing up as NONE. I'm guessing there's a special way to get this value but I can't figure out what it is. I tried to use this as a reference....https://stackoverflow.com/questions/54914961/django-form-with-a-modelchoicefield-on-a-uuid-do-not-show-its-content-in-cleaned but I can't work it out.
My Model...
class Model(models.Model):
unique_identifier = models.UUIDField(primary_key=False, default=uuid.uuid4, editable=False)
My FORM...
class FORM(forms.ModelForm):
class Meta:
model = Model
def clean(self):
cleaned_data = super(UpdateSavedNewProcedureForm, self).clean()
unique_identifier = cleaned_data.get('unique_identifier')
print(unique_identifier)
When I print the unique_identifier it comes back as NONE even though I can see the value in the database. If I store the value as a CHARFIELD I can get the value...but I'm trying to avoid duplicating this field just for readability. I'm certain there's a special way to get this value but I'm stumped at the moment.
Thanks in advance for any thoughts.
My final answer thanks to #Willem Van Onsem
My Model...
class Model(models.Model):
unique_identifier = models.UUIDField(primary_key=False,
default=uuid.uuid4, editable=False)
My Form...
class FORM(forms.ModelForm):
class Meta:
model = Model
def clean(self):
cleaned_data = super(UpdateSavedNewProcedureForm, self).clean()
unique_identifier = self.instance.unique_identifier
If unique_identifier:
Do Something Cool.....
I've created views in my Postgres Database that have links in Django Rest Framework.
I have about 20 views altogether. On 7 of them - I keep getting this programming error:
ProgrammingError at /api/reports/
column report.event_type_id_id does not exist
LINE 1: SELECT "report"."id", "report"."ev...
All 7 have the same exact error message. All of the views are based off one table with all the same column names. The column referenced in the table is event_type_id. NOT event_type_id_id. So I'm not sure where it's getting that from. If it had the same error on all of them - it would make a little more sense but it's only 7.
I'm not sure where to begin even correcting this issue because I can't pinpoint what the exact problem is. I'm assuming it's on the database side and how django expects to receive something from the database - but I'm not entirely sure. Any ideas??
Thanks in advance!
UPDATE:
Exception Location:
/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py in execute
def execute(self, sql, params=None):
self.db.validate_no_broken_transaction()
with self.db.wrap_database_errors:
if params is None:
return self.cursor.execute(sql)
else:
return self.cursor.execute(sql, params) ...
def executemany(self, sql, param_list):
self.db.validate_no_broken_transaction()
with self.db.wrap_database_errors:
return self.cursor.executemany(sql, param_list)
Report Model Update:
class Report(models.Model):
event_type_id = models.ForeignKey(EventTypeRef, default='01')
event_at = models.DateTimeField("Event Time")
xxxxxx_id = models.ForeignKey(xxxxxx)
xxxx = models.BigIntegerField(blank=True, null=True)
color = models.IntegerField(blank=True, null=True)
xxxxx_info = models.CharField(db_column='xxxxxInfo', max_length=56, blank=True)
xxxx_tag = models.ForeignKey(xxxxxxxx, blank=True, null=True)
hidden = models.BooleanField(default=False)
def __unicode__(self): # __unicode__ on Python 2
return self.event_type_id
class Meta:
managed= False,
db_table = u'report'
verbose_name_plural = "XXXXX Reports"
According to the Django ForeignKey documentation:
Behind the scenes, Django appends "_id" to the field name to create
its database column name. In the above example, the database table for
the Car model will have a manufacturer_id column. (You can change this
explicitly by specifying db_column) However, your code should never
have to deal with the database column name, unless you write custom
SQL. You’ll always deal with the field names of your model object.
So, you should rename event_type_id to event_type
This seems a silly, simple question. I'm going round in circles trying to get this work, but I can't see the wood for the trees.
Given a simple model such as (I've skipped the imports):
class Location(models.Model):
description = model.CharField(max_length=40)
address1 = model.CharField(max_length=40)
# ..... (and so on)
tel = model.CharField(max_length=12)
and another with a relationship to it:
class InformationRequest(models.Model):
source = models.ForeignKey(Location)
request_date = Models.DateField(default=datetime.now())
# ..... (and so on)
How do I add a field that references the 'tel' field from the Location model in such a way that it can be populated automatically or from a select list in Django admin.
OK, if I get this right than you are, nomen est omen, thoroughly confusing the way that relational databases work :] One of key principles is to eliminate redundancy. There shouldn't be the very same piece of data stored in two tables that are related to one another.
I think that your current models are correct. Given these instances (I'm ignoring the fact that you have other, non-nullable fields)...
>>> loc = Location()
>>> loc.tel = "123"
>>> loc.save()
>>> info = InformationRequest()
>>> info.source = loc
>>> info.save()
...you can access tel from InformationRequest instance just like this:
>>> info.source.tel
'123'
You can also create a method...
class InformationRequest(models.Model):
source = models.ForeignKey(Location, related_name="information_requests")
request_date = Models.DateField(default=datetime.now())
# ..... (and so on)
def contact_tel(self):
return self.source.tel
... and get it like this:
>>> info.contact_tel()
'123'
You can even trick it into being an attribute...
class InformationRequest(models.Model):
source = models.ForeignKey(Location, related_name="information_requests")
request_date = Models.DateField(default=datetime.now())
# ..... (and so on)
#property
def contact_tel(self):
return self.source.tel
... and get it without parentheses:
>>> info.contact_tel
'123'
Anyway, you should work your way around it programatically. Hope that helps.
I have a very simple mapping function called "BuildEntity" that does the usual boring "left/right" coding required to dump my reader data into my domain object. (shown below) My question is this - If I don't bring back every column in this mapping as is, I get the "System.IndexOutOfRangeException" exception and wanted to know if ado.net had anything to correct this so I don't need to bring back every column with each call into SQL ...
What I'm really looking for is something like "IsValidColumn" so I can keep this 1 mapping function throughout my DataAccess class with all the left/right mappings defined - and have it work even when a sproc doesn't return every column listed ...
Using reader As SqlDataReader = cmd.ExecuteReader()
Dim product As Product
While reader.Read()
product = New Product()
product.ID = Convert.ToInt32(reader("ProductID"))
product.SupplierID = Convert.ToInt32(reader("SupplierID"))
product.CategoryID = Convert.ToInt32(reader("CategoryID"))
product.ProductName = Convert.ToString(reader("ProductName"))
product.QuantityPerUnit = Convert.ToString(reader("QuantityPerUnit"))
product.UnitPrice = Convert.ToDouble(reader("UnitPrice"))
product.UnitsInStock = Convert.ToInt32(reader("UnitsInStock"))
product.UnitsOnOrder = Convert.ToInt32(reader("UnitsOnOrder"))
product.ReorderLevel = Convert.ToInt32(reader("ReorderLevel"))
productList.Add(product)
End While
Also check out this extension method I wrote for use on data commands:
public static void Fill<T>(this IDbCommand cmd,
IList<T> list, Func<IDataReader, T> rowConverter)
{
using (var rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
list.Add(rowConverter(rdr));
}
}
}
You can use it like this:
cmd.Fill(products, r => r.GetProduct());
Where "products" is the IList<Product> you want to populate, and "GetProduct" contains the logic to create a Product instance from a data reader. It won't help with this specific problem of not having all the fields present, but if you're doing a lot of old-fashioned ADO.NET like this it can be quite handy.
Although connection.GetSchema("Tables") does return meta data about the tables in your database, it won't return everything in your sproc if you define any custom columns.
For example, if you throw in some random ad-hoc column like *SELECT ProductName,'Testing' As ProductTestName FROM dbo.Products" you won't see 'ProductTestName' as a column because it's not in the Schema of the Products table. To solve this, and ask for every column available in the returned data, leverage a method on the SqlDataReader object "GetSchemaTable()"
If I add this to the existing code sample you listed in your original question, you will notice just after the reader is declared I add a data table to capture the meta data from the reader itself. Next I loop through this meta data and add each column to another table that I use in the left-right code to check if each column exists.
Updated Source Code
Using reader As SqlDataReader = cmd.ExecuteReader()
Dim table As DataTable = reader.GetSchemaTable()
Dim colNames As New DataTable()
For Each row As DataRow In table.Rows
colNames.Columns.Add(row.ItemArray(0))
Next
Dim product As Product While reader.Read()
product = New Product()
If Not colNames.Columns("ProductID") Is Nothing Then
product.ID = Convert.ToInt32(reader("ProductID"))
End If
product.SupplierID = Convert.ToInt32(reader("SupplierID"))
product.CategoryID = Convert.ToInt32(reader("CategoryID"))
product.ProductName = Convert.ToString(reader("ProductName"))
product.QuantityPerUnit = Convert.ToString(reader("QuantityPerUnit"))
product.UnitPrice = Convert.ToDouble(reader("UnitPrice"))
product.UnitsInStock = Convert.ToInt32(reader("UnitsInStock"))
product.UnitsOnOrder = Convert.ToInt32(reader("UnitsOnOrder"))
product.ReorderLevel = Convert.ToInt32(reader("ReorderLevel"))
productList.Add(product)
End While
This is a hack to be honest, as you should return every column to hydrate your object correctly. But I thought to include this reader method as it would actually grab all the columns, even if they are not defined in your table schema.
This approach to mapping your relational data into your domain model might cause some issues when you get into a lazy loading scenario.
Why not just have each sproc return complete column set, using null, -1, or acceptable values where you don't have the data. Avoids having to catch IndexOutOfRangeException or re-writing everything in LinqToSql.
Use the GetSchemaTable() method to retrieve the metadata of the DataReader. The DataTable that is returned can be used to check if a specific column is present or not.
Why don't you use LinqToSql - everything you need is done automatically. For the sake of being general you can use any other ORM tool for .NET
If you don't want to use an ORM you can also use reflection for things like this (though in this case because ProductID is not named the same on both sides, you couldn't do it in the simplistic fashion demonstrated here):
List Provider in C#
I would call reader.GetOrdinal for each field name before starting the while loop. Unfortunately GetOrdinal throws an IndexOutOfRangeException if the field doesn't exist, so it won't be very performant.
You could probably store the results in a Dictionary<string, int> and use its ContainsKey method to determine if the field was supplied.
I ended up writing my own, but this mapper is pretty good (and simple): https://code.google.com/p/dapper-dot-net/