I have read many questions and answers about dynamic implementation of EntityManager with different schemas. I found also a solution using alter database session to set current schema (see below).
ALTER SESSION SET CURRENT_SCHEMA= "SCHEMA_NAME"
As the number of users using the application increases, this isn't a suitable way of routing anymore. Also, it would change to EntityManager on every time a new user login. The situation is as follows
1 database server
many schemas on that server, every user has their own schema.
I only need to change the schema name during runtime the schema name is retainable by logged in user
I'm using Spring with Hibernate 4.0 and Spring data JPA
I can't find a way to change the schema completely dynamically. Does someone know how to do it in spring?
Related
I'm playing around with spring-data-jdbc and discovered a problem, with I can't solve using Google.
No matter what I try to do, I just can't push a trivial object into the database (Bean1.java:25):
carRepository.save(new Car(2L, "BMW", "5"));
Both, without one and with a TransactionManager +#Transactional the database (apparently) does not commit the record.
The code is based on a Postgres database, but you might also simply use a H2 below and get the same result.
Here is the (minimalistic) source code:
https://github.com/bitmagier/spring-data-jdbc-sandbox/tree/stackoverflow-question
Can somebody tell me, why the car is not inserted into the database?
This is not related to transactions not working.
Instead, it's about Spring Data JDBC considering your instance an existing instance that needs updating (instead of inserting).
You can verify this is the problem by activating logging for org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate. You should see an update but no insert.
By default, Spring Data JDBC considers an entity as new when it has an id of an object type and a value of null or of a primitive type (e.g. int or long) and a value of 0.
If your entity has an attribute with #Version annotation that attribute will be used to determine if the instance is a new one.
You have the following options in order to make it work:
Set the id to null and configure your database schema so that it will automatically create a new value on insert. After the save your entity instance will contain the generated value from the database.
Note: Spring Data JDBC will set the id even if it is final in your entity.
Leave the id null and set it in a Before-Convert listener to the desired value.
Let your entity implement Persistable. This allows you to control when an entity is considered new. You'll probably need a listener as well so you can let the entity know it is not new any longer.
Beginning with version 1.1 of Spring Data JDBC you'll also be able to use a JdbcAggregateTemplate to do a direct insert, without inspecting the id, see https://jira.spring.io/browse/DATAJDBC-282. Of course, you can do that in a custom method of your repository, as is done in this example: https://github.com/spring-projects/spring-data-examples/pull/441
We are migrating our Software from our own and legacy PHP framework to Symfony.
Our application has different Schema for every user. So beside public Schema, we have about 2.000 user Schemas.
Let's say that we have a system similar to the marketplace, and every user can customize some tables inside their own Schema, but let's not focus on that.
We are currently searching for the best solutions for two problems:
When a new user is registered, our Symfony should run migrations only for that newly created user Schema.
When we create migration we need that that migration affects ALL existing user Schemas.
Currently, in our old system, we do these kinds of migration with a bunch of PHP logic, and I would like to avoid customizing DoctrineMigrationsBundle or have hundreds of lines of PHP code in a single migration.
So I would like to know is there any easier way to handle all user Schemas in Doctrine/Symfony, so when I run for example:
php bin/console doctrine:migrations:migrate
all our existing user Schemas are updated.
Our user Schema names are like app234234, app453453 and except those Schemas we have only default public Schema
We are using:
Symfony 4.3
PostgreSQL 9.1
PHP 7.3
1 - For every schema you have, you can define a schema filter. Let's say you prefix user table with schema name app234234_ . To prevent migrations from removing your other tables, you can use doctrine dbal schema_filter property in your config.yml/doctrine.yml or wherever you define new users's schema
## doctrine.yml
doctrine:
dbal:
connections:
app234234:
schema_filter: "/^app234234_/" ## more likely a %schema_name% parameter
According to symfony doc: This ignores the tables on the DBAL level and they will be ignored by the diff command.
Note that if you have multiple connections configured then the schema_filter configuration will need to be placed per-connection
2- Regarding your second point, i don't know if this is something you can/should handle with symfony.
Situation:
For our SaaS API we use schema-based multitenancy, which means every customer (~tenant) has its own separate schema within the same (postgres) database, without interfering with other customers. Each schema consists of the same underlying entity-model.
Everytime a new customer is registered to the system, a new isolated schema is automatically created within the db. This means, the schema is created at runtime and not known in advance. The customer's schema is named according to the customer's domain.
For every request that arrives at our API, we extract the user's tenancy-affiliation from the JWT and determine which db-schema to use to perform the requested db-operations for this tenant.
Problem
After having established a connection to a (postgres) database via TypeORM (e.g. using createConnection), our only chance to set the schema for a db-operation is to resort to the createQueryBuilder:
const orders = await this.entityManager
.createQueryBuilder()
.select()
.from(`${tenantId}.orders`, 'order') // <--- setting schema-prefix here
.where("order.priority = 4")
.getMany();
This means, we are forced to use the QueryBuilder as it does not seem to be possible to set the schema when working with the EntityManager API (or the Repository API).
However, we want/need to use these APIs, because they are much simpler to write, require less code and are also less error-prone, since they do not rely on writing queries "manually" employing a string-based syntax.
Question
In case of TypeORM, is it possible to somehow set the db-schema when working with the EntityManager or repositories?
Something like this?
// set schema when instantiating manager
const manager = connection.createEntityManager({ schema: tenantDomain });
// should find all matching "order" entities within schema
const orders = manager.find(Order, { priority: 4 })
// should find a matching "item" entity within schema using same manager
const item = manager.findOne(Item, { id: 321 })
Notes:
The db-schema needs to be set in a request-scoped way to avoid setting the schema for other requests, which may belong to other customers. Setting the schema for the whole connection is not an option.
We are aware that one could create a whole new connection and set the schema for this connection, but we want to reuse the existing connection. So simply creating a new connection to set the schema is not an option.
To answer my own question:
At the moment there is no way to instantiate TypeORM repositories with different schemas at runtime without creating new connections.
So the only two options that a developer is left with for schema-based multi tenancy are:
Setting up new connections to connect with different schemas within the same db at runtime. E.g. see NestJS Request Scoped Multitenancy for Multiple Databases. However, one should definitely strive for reusing connections and and be aware of connection limits.
Abandoning the idea of working with the RepositoryApi and reverting to using createQueryBuilder (or executing SQL queries via query()).
For further research, here are some TypeORM GitHub issues that track the idea of changing the schema for a existing connections or repositories at runtime (similar to what is requested in the OP):
Multi-tenant architecture using schema. #4786 proposes something like this.photoRepository.useSchema('customer1').find()
Handling of database schemas #3067 proposes something like getConnection().changeDefaultSchema('myschema')
Run-time change of schema #4473
Add an ability to set postgresql schema per call #2439
P.S. If TypeORM decides to support the idea discussed in the OP, I will try to update this answer.
Here is a global overview of the issues with schema-based multitenancy along with a complete walkthrough a Github repo for it.
Most of the time, you may want to use Postgres Row Security Policy instead. It gives most of the benefits of schema-based multitenancy (especially on developer experience), without the issues related to the multiplication of connections.
Since commenting does not work for me, here a hint from the documentation of NestJS:
https://docs.nestjs.com/techniques/database#async-configuration
I am not using NestJS but reading the docs at the moment to decide, if it's a fitting framework for us. We have an app where only some modules have multi tenancy with schema per tenant, so using TypeOrmModule.forRootAsync(dynamicCreatedDbConfig) might be an option for me too.
This may help you if you have an interceptor or middleware, which prepares the dynamicCreatedDbConfig data before...
I'm currently using APEX 18.1 and we created in our first database schema (db1) a RESTful Webservice (we provide a URL so other Application can send us data, so that we save them in the database). Now I changed the schema, to a new one (db2) and also already changed that in the shared components.
But the data is still saving in the old schema and I dont know how I change the schema for the REST services. I already tried to put the schema name in front of my statements like "INSERT INTO "db2".test VALUES (xyz) but its not working.
Does someone have the same problem and a solution ?
We are attempting to use CFE to generate one schema for each tenant as outlined in the CodeFluent blog post (http://blog.codefluententities.com/2014/12/04/multi-tenant-using-multiple-schema/). In this scenario, we are expecting that each schema generated should be identical and we are using the ICodeFluentPersistence Hook system to identify the company for a user and then properly set the schema to be used. All of that works fine, but when we run the code to generate the multiple schemas (https://github.com/SoftFluent/CodeFluent-Entities/tree/master/Extensions/SoftFluent.MultiTenantGenerator), it is removing the constraints. I then tried to see if there was an issue with my configuration, but running the sample program from GitHub produces the same results. After running the sample program, the Primary key was not present in the contoso schema, even though is was properly defined in the dbo schema (and in the model).
Has anyone used the CFE Multi-Schema generator or have any insight into what the issue may be?
Thanks for your response, but I am not sure that I agree. The whole reason (at least of me) to use the Multi-Tenant generator is to create as many database schemas as needed (one per client) from a single CFE model. The idea that you would lose the constraints in all but one of them didn't feel right so I did a bit more investigation and found the following in "Microsoft SQL Server 2012 Internals" by Kalen Delaney and Craig Freeman (through Google Books):
And in fact was able to do a quick test to prove this out by creating two identical tables with identical PK names:
So it would appear to me that CFE should be able to create the two identical databases from the same model and seems to point to a deficiency in the SQLServer diff engine.
The multi-schema generator loads the model and change it dynamically to modify the schema of the entities. Then it call the standard code production process with only the database producers (SQL Server, Oracle, etc.).
So if you want to generate 2 differents schema (dbo and contoso) against an empty database, the process is the following:
Generate the database for the dbo schema from a blank database
Generate the database for the contoso schema from the previously generated database
Before creating a constraint, the SQL Server diff engine drops the constraint with the same name. In fact SQL Server does not allow 2 constraints to have the same name (I can't find a page on MSDN with more details about that). So in your case the existing PK is dropped when you generate the contoso schema because the name of the PK is the same as the one that exists in the dbo schema. Maybe this can be improved, but the diffs engine tries to generate a code that works for SQL Server 2000 to SQL Server 2016.
Workarounds
You can generate each schema in a different database, so the diffs engine will generate the code you expect. Then you can run the generated scripts on the production database. Not the easiest way but it should work.
You can use the patch producer to replace the name of the schema in the file. For SQL files you should use the SqlServerPatchProducer as explain in the KnowledgeBase:
namespace Sample
{
public class SqlServerPatchProducer : SqlServerProducer
{
public SqlServerPatchProducer()
{
}
protected override void RunProceduresScript()
{
string path = GetPath(Project.DefaultNamespace + "_procedures.sql");
ProduceFrom(path, "before");
SearchAndReplaceProducer.ProducePatches(Project, null, this, null, ProductionFlags, Element);
Utilities.RunFileScript(path, Database, OutputEncoding);
ProduceFrom(path, "after");
}
}
}