I have my reflection configured like so:
meta = MetaData()
meta.reflect(bind=db.engine, schema='web')
Base = automap_base(metadata=meta)
Base.prepare()
I thought the schema keyword would limit the reflection to the specified schema, but running the app I see that it reflects tables in all schemas anyway, which leads to some conflicts given I have tables of the same name in different schemas. So Base.classes will contain the wrong classes given that it uses the wrong schema.
What are my options here?
You can limit the tables reflected in MetaData.reflect using the only keyword.
meta.reflect(bind=db.engine, schema='web', only=tables)
If you don't know the tables in the particular schema ahead of time, you can use an Inspector object to get them.
insp = reflection.Inspector.from_engine(db.engine)
tables = insp.get_table_names(schema='web')
Related
i would like to know if you have any idea how i can achieve this, considering a query stored as string in the configuration file.
I tried to use SqlQuery applied to the DBSet, but the problem is that SqlQuery requires me to select all properties of the required entities in my query. If i don't consider any column, it will complain because is not able to map the query to the entities.
I don't want to select all properties of the entities i want to query.
Thanks
If you are using EF then why not use Database.ExecuteSqlCommand()? It's in the System.Data.Entity namespace.
For example:
int result = db.Database.ExecuteSqlCommand("Non SELECT SQL etc...");
Well, I ended up implementing a mechanism using reflection that basically receives a group of fields to select, and constructs dynamic objects with those fields, so when applied the query with the joins between the entities, will only bring the fields I am looking for.
So, considering Entity1, Entity2, Entity3 with the following relationship
<b>Entity1</b>{
<br/> Entity1Name, <br/> List<*Entity2*> Entity2Items, <br/> etc..
<br/>}
and
<b>Entity2</b> { <br/> Entity2Name, <br/> List<*Entity3*> Entity3Items <br/>}
I can store e.g. the following query in the configuration file, and retrieve the information:
"Entity1.Entity1Name", <br/>
"Entity1.Entity2Items.Entity2Name", <br/>
"Entity1.Entity2Items.Entity3Items.Entity3Name"
Anyway, I was just trying to see if there would be any solution out-of-the-box that would require minimal code changes.
Thank you.
I have a legacy database with a particular table -- I will call it ItemTable -- that can have billions of rows of data. To overcome database restrictions, we have decided to split the table into "silos" whenever the number of rows reaches 100,000,000. So, ItemTable will exist, then a procedure will run in the middle of the night to check the number of rows. If numberOfRows is > 100,000,000 then silo1_ItemTable will be created. Any Items added to the database from now on will be added to silo1_ItemTable (until it grows to big, then silo2_ItemTable will exist...)
ItemTable and silo1_ItemTable can be mapped to the same Item entity because the table structures are identical, but I am not sure how to set this mapping up at runtime, or how to specify the table name for my queries. All inserts should be added to the latest siloX_ItemTable, and all Reads should be from a specified siloX_ItemTable.
I have a separate siloTracker table that will give me the table name to insert/read the data from, but I am not sure how I can use this with entity framework...
Thoughts?
You could try to use the Entity Inheritance to get this. So you have a base class which has all the fields mapped to ItemTable and then you have descendant classes that inherit from ItemTable entity and is mapped to the silo tables in the db. Every time you create a new silo you create a new entity mapped to that silo table.
[Table("ItemTable")]
public class Item
{
//All the fields in the table goes here
}
[Table("silo1_ItemTable")]
public class Silo1Item : Item
{
}
[Table("silo2_ItemTable")]
public class Silo2Item : Item
{
}
You can find more information on this here
Other option is to create a view that creates a union of all those table and map your entity to that view.
As mentioned in my comment, to solve this problem I am using the SQLQuery method that is exposed by DBSet. Since all my item tables have the exact same schema, I can use the SQLQuery to define my own query and I can pass in the name of the table to the query. Tested on my system and it is working well.
See this link for an explanation of running raw queries with entity framework:
EF raw query documentation
If anyone has a better way to solve my question, please leave a comment.
[UPDATE]
I agree that stored procedures are also a great option, but for some reason my management is very resistant to make any changes to our database. It is easier for me (and our customers) to put the sql in code and acknowledge the fact that there is raw sql. At least I can hide it from the other layers rather easily.
[/UPDATE]
Possible solution for this problem may be using context initialization with DbCompiledModel param:
var builder = new DbModelBuilder(DbModelBuilderVersion.V6_0);
builder.Configurations.Add(new EntityTypeConfiguration<EntityName>());
builder.Entity<EntityName>().ToTable("TableNameDefinedInRuntime");
var dynamicContext = new MyDbContext(builder.Build(context.Database.Connection).Compile());
For some reason in EF6 it fails on second table request, but mapping inside context looks correct on the moment of execution.
I have many EntityManager, one per schema that I have (I use entity-mappings file to map EMs with schemas). It works.
When I use #NamedQuery it's working like a charm but when I use #NamedNativeQuery schema is not used. I have to qualify with it SELECT foo FROM schema.table.
Is it the right behaviour ?
I think it's not possible to parameter #NamedNativeQuery to dynamically pass schema (I believe only columns can be dynamics not tables or schemas or anything else) so how can I use #NamedNativeQuery with dynamic schema please ?
Prefix your table name with "{h-schema}", e.g.SELECT foo FROM {h-schema}table
(courtesy of getting hibernate default schema name programmatically from session factory?)
Excerpts from documentation :
NamedNativeQuery : Specifies a named native SQL query. Query names are scoped to the persistence unit.
NamedQuery : Specifies a static, named query in the Java Persistence query language. Query names are scoped to the persistence unit.
It isn't specified directly that NamedNativeQuery is static, but both are same scoped & can't be altered afterwards & it's the desired behaviour.
Named queries are mean to be accessed by multiple modules - application wide, identified by unique name, so they are static & constant. You can try building a query string dynamically & can create a native query from it, instead of named native query.
I'm giving EF Model first a go. I'm using EF 4.1
Pretty much followed this article
I've set PluraliseNewObjects to False on the Model and also in Options->Database Tools ->O/R Designer set Pluralization of names to false.
Neither have any effect - when I generate a new schema from the model the table names are always pluralised - is it possible to disable this?
OK - I've found one way to achieve what I want - but it's a pretty daft route.
Generated db with the plural names (interesting that it only pluralised the tables mapping to types - not the auto-generated linking tables for many to many joins).
Manually renamed the tables in the database
Deleted Model from the project and recreated based on existing database schema (the one I've just renamed).
Model is now correctly mapped to singularly names tables.
I'll wait and see if anyone comes up with a more sensible way of achieving this....
The names of the tables in the generated DDL seem to match the "Entity Set Name" values (different than the "Entity Name"). If you singularize the Entity Set Names, the table names in the DDL are singularized as well.
This will have the possibly undesired effect of singularizing the EntitySet property names in your code, though. Instead of:
myDatabase
.Products
.Where...
.Select...
your code will look like:
myDatabase
.Product
.Where...
.Select...
may or may not be an issue
I'm new to CakePHP and using version 1.3.
How can I dynamically change the 'schema' property as found in DATABASE_CONFIG prior to any database operation? What is the class where I could have the postgres-specific command "set search_path to 'schema_xyz'" executed before any database interaction?
I want to use Postgres' ability to maintain multiple distinct namespaces (aka schema in postgres parlance) within a single database to implement multi-tenancy in my application. That is, every namespace will contain the same set of tables, but evidently with different content. Here, it's important not to understand schema as meaning table metadata, but rather as the postgres-specific concept of namespace where a schema is a container for tables. The exact Postgres command isn't important. What is, is the mechanism by which it can be invoked, and steering clear of Cake's typical meaning of table description, as seen in the SchemaShell. The only place I have found where Cake exposes the concept of namespace is in the database.php file, which is then used when the DB connection is first established. See: api13.cakephp.org/view_source/dbo-postgres/#line-113 (new user link limit, sorry)
if ($this->connection) {
$this->connected = true;
$this->_execute("SET search_path TO " . $config['schema']);
I want to set that search_path before ALL DB queries, not just at connection time as is currently done.
As a proof of concept, I have tried setting $useDbConfig in my models, but according to the debug output where the SQL commands are printed, this only seems to affect a subset of all queries. I've moved this up into app_model.php with the same result. As did augmenting that with creating a db_config instance on the fly and passing to the ConnectionManager through loadDataSource. Maybe I should slap that code in all flavors of before... methods.
I have seen many posts online where people discuss using one of several DB configurations in database.php to use different databases for dev, lab and production environments. But I have a single database with multiple namespaces/schemas. Also, my number of such namespaces will be too high and dynamic to make hardcoding a new variable in database.php practical.
Thus, where is the spot in CakePHP where I could insert something to set the search_path prior to any database command? I'll deal with optimizing that later. Remember that I'm new to Cake, so try to be as specific as you can. Let me know if I can clarify this question.
Thanks in advance. Here's the partially working code snippet:
class AppModel extends Model {
function beforeFind($queryData)
{
App::import("ConnectionManager");
$cm = &ConnectionManager::getInstance();
$namespace = 'xyz_namespace'; //name of the new schema/namespace/search path
$new_db_config_name = 'new_config'; //name for the new DB config to be used in the ConnectionManager
$new_db_config = $cm->config->default; //copy the 'default' DB config into an array
$new_db_config['schema'] = $namespace; //change/add the new schema/namespace/search path
$cm->create($new_db_config_name, $new_db_config); //turn the array into a DbConfig object
$cm->loadDataSource($new_db_config_name); //load the new DbConfig into the ConnectionManager
$this->useDbConfig = $new_db_config_name; //tell the model to new use the Db Config
return $queryData;
}
}
There is a very simple way in PostgreSQL if you want to switch schema per login role:
ALTER ROLE foo SET search_path=bar, public;
ALTER ROLE baz SET search_path=bam, public;
Thus a connection initiated by that role has that search_path set automatically.
If your login names are the same as the desired schema names, there is an even simpler way, I quote the fine manual:
If one of the list items is the special value $user, then the schema
having the name returned by SESSION_USER is substituted, if there is
such a schema. (If not, $user is ignored.)
But be advised that - the fine manual again:
Role-specific variable settings take effect only at login; SET ROLE
and SET SESSION AUTHORIZATION do not process role-specific variable
settings.
If I understand your question correctly, (bear with me, I know little about Postgres but basically I think you mean, reloading the schema whenever the table perspective changes?), here's how to dynamically switch schemas in your controller:
// Model::getDataSource()->configKeyName holds whichever db config you're using
if ($this->Model->getDataSource()->configKeyName != 'default') {
// do something...
$this->loadModel("Special")
$this->Model->table = "extras";
$this->Model->schema(true);
} else {
// predictably, Model::setDataSource($configKey) changes configs
$this->Model->setDataSource("offsite"); // this could be a string variable
}
Or from the model, $this->getDataSource()->configKeyName and $this->schema(true) and so forth. Note $this->schema(true) actually reloads the model schema and registers it with cake. app_model, a component, or config/bootstrap might be an appropriate place for this. I'm not sure where Cake would have defined the search_path the first time, but it would almost certainly be a property of the dataSource object and could be redefined there just like the table name, etc. And then reload Cake's schema to register the changed path. It is necessary to ensure Cake unloads any default it may have picked up, and load the correct schema based on the currently defined table. (It sounds like this may have been the only step you were missing.)
If this does not answer your question or if I misunderstood, let me know. HTH. :)