I'm currently using APEX 18.1 and we created in our first database schema (db1) a RESTful Webservice (we provide a URL so other Application can send us data, so that we save them in the database). Now I changed the schema, to a new one (db2) and also already changed that in the shared components.
But the data is still saving in the old schema and I dont know how I change the schema for the REST services. I already tried to put the schema name in front of my statements like "INSERT INTO "db2".test VALUES (xyz) but its not working.
Does someone have the same problem and a solution ?
Related
I want to implement a multi-tenant solution where I have one webserver and one database shared across all tenants. Regarding to this blog post from AWS it is "pooled multi tenancy model".
I'm using nest.js and sequelize. If sequelize is not a good fit for this I also could switch to another library like typeORM if necessary.
How can this be implemented? I'm absolutely clueless how I can use a different connection (different database user) for each HTTP request and also I don't know how to set a runtime context variable for the connection in a good way.
What I get currently is that every HTTP requests contains a header tenant-id. This should be used for all queries.
There is also the concept of scopes in sequelize. But this is something that is implemented on the client side and not on the database directly. Also, this is something that is specific to sequelize. I would prefer a solution that is independent from sequelize and maybe more specific to PostgreSQL.
Is there any way to implement this with sequelize? A hint or a basic approach would be sufficient.
That seems that this approach is similar. https://learn.microsoft.com/en-us/microsoft-365/education/deploy/design-multi-tenant-architecture.
I'm studding for create a similar architecture, but i will use the "silo" model or "physical database". I think that at first you need to create a internal database called "catalog" that will contains the information of the user (this user already have a login? if true select this information) where have to contains a previous credentials how tenant-id. About the Sequelize, i guess that is necessary to use RAW queries for create ROLE|GRANT|DATA BASE etc and the MIGRATIONS to create the same DB for each new clients.
Situation:
For our SaaS API we use schema-based multitenancy, which means every customer (~tenant) has its own separate schema within the same (postgres) database, without interfering with other customers. Each schema consists of the same underlying entity-model.
Everytime a new customer is registered to the system, a new isolated schema is automatically created within the db. This means, the schema is created at runtime and not known in advance. The customer's schema is named according to the customer's domain.
For every request that arrives at our API, we extract the user's tenancy-affiliation from the JWT and determine which db-schema to use to perform the requested db-operations for this tenant.
Problem
After having established a connection to a (postgres) database via TypeORM (e.g. using createConnection), our only chance to set the schema for a db-operation is to resort to the createQueryBuilder:
const orders = await this.entityManager
.createQueryBuilder()
.select()
.from(`${tenantId}.orders`, 'order') // <--- setting schema-prefix here
.where("order.priority = 4")
.getMany();
This means, we are forced to use the QueryBuilder as it does not seem to be possible to set the schema when working with the EntityManager API (or the Repository API).
However, we want/need to use these APIs, because they are much simpler to write, require less code and are also less error-prone, since they do not rely on writing queries "manually" employing a string-based syntax.
Question
In case of TypeORM, is it possible to somehow set the db-schema when working with the EntityManager or repositories?
Something like this?
// set schema when instantiating manager
const manager = connection.createEntityManager({ schema: tenantDomain });
// should find all matching "order" entities within schema
const orders = manager.find(Order, { priority: 4 })
// should find a matching "item" entity within schema using same manager
const item = manager.findOne(Item, { id: 321 })
Notes:
The db-schema needs to be set in a request-scoped way to avoid setting the schema for other requests, which may belong to other customers. Setting the schema for the whole connection is not an option.
We are aware that one could create a whole new connection and set the schema for this connection, but we want to reuse the existing connection. So simply creating a new connection to set the schema is not an option.
To answer my own question:
At the moment there is no way to instantiate TypeORM repositories with different schemas at runtime without creating new connections.
So the only two options that a developer is left with for schema-based multi tenancy are:
Setting up new connections to connect with different schemas within the same db at runtime. E.g. see NestJS Request Scoped Multitenancy for Multiple Databases. However, one should definitely strive for reusing connections and and be aware of connection limits.
Abandoning the idea of working with the RepositoryApi and reverting to using createQueryBuilder (or executing SQL queries via query()).
For further research, here are some TypeORM GitHub issues that track the idea of changing the schema for a existing connections or repositories at runtime (similar to what is requested in the OP):
Multi-tenant architecture using schema. #4786 proposes something like this.photoRepository.useSchema('customer1').find()
Handling of database schemas #3067 proposes something like getConnection().changeDefaultSchema('myschema')
Run-time change of schema #4473
Add an ability to set postgresql schema per call #2439
P.S. If TypeORM decides to support the idea discussed in the OP, I will try to update this answer.
Here is a global overview of the issues with schema-based multitenancy along with a complete walkthrough a Github repo for it.
Most of the time, you may want to use Postgres Row Security Policy instead. It gives most of the benefits of schema-based multitenancy (especially on developer experience), without the issues related to the multiplication of connections.
Since commenting does not work for me, here a hint from the documentation of NestJS:
https://docs.nestjs.com/techniques/database#async-configuration
I am not using NestJS but reading the docs at the moment to decide, if it's a fitting framework for us. We have an app where only some modules have multi tenancy with schema per tenant, so using TypeOrmModule.forRootAsync(dynamicCreatedDbConfig) might be an option for me too.
This may help you if you have an interceptor or middleware, which prepares the dynamicCreatedDbConfig data before...
I have read many questions and answers about dynamic implementation of EntityManager with different schemas. I found also a solution using alter database session to set current schema (see below).
ALTER SESSION SET CURRENT_SCHEMA= "SCHEMA_NAME"
As the number of users using the application increases, this isn't a suitable way of routing anymore. Also, it would change to EntityManager on every time a new user login. The situation is as follows
1 database server
many schemas on that server, every user has their own schema.
I only need to change the schema name during runtime the schema name is retainable by logged in user
I'm using Spring with Hibernate 4.0 and Spring data JPA
I can't find a way to change the schema completely dynamically. Does someone know how to do it in spring?
I need to read data from existing database is it possible using
compile "org.grails.plugins:db-reverse-engineer:4.0.0"?
My operations are: user should read data from existing table, create new record, create new coulmn, edit coulmn name, edit records.
View format will be in grid like xml grid.
Which technology is the best for these operations in grails, I have plan to work on javascript using jaxrs, is it good to do?
DB Reverse Engineering Plugin (org.grails.plugins:db-reverse-engineer:4.0.0) allows you to generate domain classes using existing DB. After you generate them - just use GORM to perform CRUD operations. You can read about GORM here
You can implement REST api in Grails using standart ways, check this answer to get high level understanding. If you need jaxrs- there is a plugin for that.
We are attempting to use CFE to generate one schema for each tenant as outlined in the CodeFluent blog post (http://blog.codefluententities.com/2014/12/04/multi-tenant-using-multiple-schema/). In this scenario, we are expecting that each schema generated should be identical and we are using the ICodeFluentPersistence Hook system to identify the company for a user and then properly set the schema to be used. All of that works fine, but when we run the code to generate the multiple schemas (https://github.com/SoftFluent/CodeFluent-Entities/tree/master/Extensions/SoftFluent.MultiTenantGenerator), it is removing the constraints. I then tried to see if there was an issue with my configuration, but running the sample program from GitHub produces the same results. After running the sample program, the Primary key was not present in the contoso schema, even though is was properly defined in the dbo schema (and in the model).
Has anyone used the CFE Multi-Schema generator or have any insight into what the issue may be?
Thanks for your response, but I am not sure that I agree. The whole reason (at least of me) to use the Multi-Tenant generator is to create as many database schemas as needed (one per client) from a single CFE model. The idea that you would lose the constraints in all but one of them didn't feel right so I did a bit more investigation and found the following in "Microsoft SQL Server 2012 Internals" by Kalen Delaney and Craig Freeman (through Google Books):
And in fact was able to do a quick test to prove this out by creating two identical tables with identical PK names:
So it would appear to me that CFE should be able to create the two identical databases from the same model and seems to point to a deficiency in the SQLServer diff engine.
The multi-schema generator loads the model and change it dynamically to modify the schema of the entities. Then it call the standard code production process with only the database producers (SQL Server, Oracle, etc.).
So if you want to generate 2 differents schema (dbo and contoso) against an empty database, the process is the following:
Generate the database for the dbo schema from a blank database
Generate the database for the contoso schema from the previously generated database
Before creating a constraint, the SQL Server diff engine drops the constraint with the same name. In fact SQL Server does not allow 2 constraints to have the same name (I can't find a page on MSDN with more details about that). So in your case the existing PK is dropped when you generate the contoso schema because the name of the PK is the same as the one that exists in the dbo schema. Maybe this can be improved, but the diffs engine tries to generate a code that works for SQL Server 2000 to SQL Server 2016.
Workarounds
You can generate each schema in a different database, so the diffs engine will generate the code you expect. Then you can run the generated scripts on the production database. Not the easiest way but it should work.
You can use the patch producer to replace the name of the schema in the file. For SQL files you should use the SqlServerPatchProducer as explain in the KnowledgeBase:
namespace Sample
{
public class SqlServerPatchProducer : SqlServerProducer
{
public SqlServerPatchProducer()
{
}
protected override void RunProceduresScript()
{
string path = GetPath(Project.DefaultNamespace + "_procedures.sql");
ProduceFrom(path, "before");
SearchAndReplaceProducer.ProducePatches(Project, null, this, null, ProductionFlags, Element);
Utilities.RunFileScript(path, Database, OutputEncoding);
ProduceFrom(path, "after");
}
}
}