I'm having an issue, when I want to perform a MongoDB database operation every time I boot SailsJs (v1.0.0-49), declaring db like this:
const db = sails.getDatastore().manager
But I get the following:
TypeError: sails.getDatastore is not a function
UPDATE: I am not using sails (Waterline) Models.
Now, I've tried using it in config/bootstrap.js, as a Hook and as a Service, with no success. I've also asked in a related issue in their repository without any response since last year.
Here's the link of my original post:
https://github.com/balderdashy/sails/issues/4156
Actually, you are allowed to call getDatastore() directly on the Sails object OR against the model. But this is a Waterline feature wired into Sails:
https://sailsjs.com/documentation/reference/waterline-orm/datastores
https://sailsjs.com/documentation/reference/application/sails-get-datastore
You have to call getDatastore().manager on a model not the sails object.
if you have some model named MyModel then you access the datastore manager like this :
const db = MyModel.getDatastore().manager;
check the docs.
Related
Situation:
For our SaaS API we use schema-based multitenancy, which means every customer (~tenant) has its own separate schema within the same (postgres) database, without interfering with other customers. Each schema consists of the same underlying entity-model.
Everytime a new customer is registered to the system, a new isolated schema is automatically created within the db. This means, the schema is created at runtime and not known in advance. The customer's schema is named according to the customer's domain.
For every request that arrives at our API, we extract the user's tenancy-affiliation from the JWT and determine which db-schema to use to perform the requested db-operations for this tenant.
Problem
After having established a connection to a (postgres) database via TypeORM (e.g. using createConnection), our only chance to set the schema for a db-operation is to resort to the createQueryBuilder:
const orders = await this.entityManager
.createQueryBuilder()
.select()
.from(`${tenantId}.orders`, 'order') // <--- setting schema-prefix here
.where("order.priority = 4")
.getMany();
This means, we are forced to use the QueryBuilder as it does not seem to be possible to set the schema when working with the EntityManager API (or the Repository API).
However, we want/need to use these APIs, because they are much simpler to write, require less code and are also less error-prone, since they do not rely on writing queries "manually" employing a string-based syntax.
Question
In case of TypeORM, is it possible to somehow set the db-schema when working with the EntityManager or repositories?
Something like this?
// set schema when instantiating manager
const manager = connection.createEntityManager({ schema: tenantDomain });
// should find all matching "order" entities within schema
const orders = manager.find(Order, { priority: 4 })
// should find a matching "item" entity within schema using same manager
const item = manager.findOne(Item, { id: 321 })
Notes:
The db-schema needs to be set in a request-scoped way to avoid setting the schema for other requests, which may belong to other customers. Setting the schema for the whole connection is not an option.
We are aware that one could create a whole new connection and set the schema for this connection, but we want to reuse the existing connection. So simply creating a new connection to set the schema is not an option.
To answer my own question:
At the moment there is no way to instantiate TypeORM repositories with different schemas at runtime without creating new connections.
So the only two options that a developer is left with for schema-based multi tenancy are:
Setting up new connections to connect with different schemas within the same db at runtime. E.g. see NestJS Request Scoped Multitenancy for Multiple Databases. However, one should definitely strive for reusing connections and and be aware of connection limits.
Abandoning the idea of working with the RepositoryApi and reverting to using createQueryBuilder (or executing SQL queries via query()).
For further research, here are some TypeORM GitHub issues that track the idea of changing the schema for a existing connections or repositories at runtime (similar to what is requested in the OP):
Multi-tenant architecture using schema. #4786 proposes something like this.photoRepository.useSchema('customer1').find()
Handling of database schemas #3067 proposes something like getConnection().changeDefaultSchema('myschema')
Run-time change of schema #4473
Add an ability to set postgresql schema per call #2439
P.S. If TypeORM decides to support the idea discussed in the OP, I will try to update this answer.
Here is a global overview of the issues with schema-based multitenancy along with a complete walkthrough a Github repo for it.
Most of the time, you may want to use Postgres Row Security Policy instead. It gives most of the benefits of schema-based multitenancy (especially on developer experience), without the issues related to the multiplication of connections.
Since commenting does not work for me, here a hint from the documentation of NestJS:
https://docs.nestjs.com/techniques/database#async-configuration
I am not using NestJS but reading the docs at the moment to decide, if it's a fitting framework for us. We have an app where only some modules have multi tenancy with schema per tenant, so using TypeOrmModule.forRootAsync(dynamicCreatedDbConfig) might be an option for me too.
This may help you if you have an interceptor or middleware, which prepares the dynamicCreatedDbConfig data before...
I would like to hook an event after an object is retrieved from the database. This event should be fired regardless if the object is retrieved from .find() or via a .populate() call. I would think it should be similar to the other lifecycle callbacks like afterCreate or afterUpdate, etc.
Is this possible to do with Waterline and if so, how can it be accomplished?
To give an idea of what I'm trying to accomplish: I'm using Mongo to
store my data. There is the potential that the model schema has
changed since the record was last saved. I'd like to "upgrade" the
record retrieved from the database to the latest version using a
function defined on the model itself. I would like this process to
happen as transparently as possible after a query returns (ie. Not
have to explicitly call the function in each query callback function).
I can't create MongoDb models using Gii module code generator. Gets the below error :
Unknown Method – yii\base\UnknownMethodException
Calling unknown method: common\components\MongoConnection::getTableSchema()
Gii should not be used to create MongoDB Models.
MongoDB documents may change from one to another and there is no "collection definiton", as oposed to "table schemas" where you can get all column names and types.
This is the reason why a model cannot be created automatically and should be created manually.
There is a Gii component to help you generate MongoDB ActiveRecords.
Here's a little YouTube video that shows you exactly how it modify Yii2 configuration, create an active record and generate a CRUD controller.
http://www.youtube.com/watch?v=1urmws7hz5Q
I would like to be able to track the mutated (modified) attributes of an object in sails during afterUpdate lifecycle callback.
Assuming an object is updated I would like to know which attribute has been modified during the update operation, I've been using sails-mongo and I believe I could write a proxy adapter that keep a local instance and attach it to the one thats going to be modified and do a diff on save but there might be an already existing way of doing so :)
Thanks !
Waterline doesn't currently have a built in diff mechanism. To accomplish this you can explore the use of native queries where some databases will allow you to return the values being updated or store the previous records in a diff on the record in the database.
afterUpdate won't be able to handle this because by then the results have already been updated in the database. You could write a controller method that uses the same criteria to capture all the records before you issue the update criteria.
I'm trying to use Entity Framework to query database and I have following code that I'm using to get some data.
var students= MyEntities.Students.Where(s => s.Age > 20).ToList();
This code works fine and returns correct data. However, if I run this code, then go to database and update records to change the data this code should return, and then re-run this code without shutting down the app, I'm getting original data.
I'm pretty sure it used to work fine, but now this doesn't refresh data.
No it never worked. This is essential behavior of entity framework (and many ORM tools) called identity map (explanation here).
If you run the query on the same context you will not get your entities updated. It will only append new entities created in the database between running those two queries. If you want to force EF to reload entities you must do something like this:
ObjectQuery query = MyEntities.Students;
query.MergeOption = MergeOption.OverwriteChanges;
var students = query.Where(s => s.Age > 20).ToList();
You are running into issues because EF caches data, so if the data changes behind the scenes, and you dont dispose/reopen your context you are going to run into issues.
The general rule of thumb is to keep the lifetime of the context as short as possible, to circumvent issues like you just mentioned.
Please do not disregard what I said above, but if you would like to force a refresh from the DB you could use the Refresh() method.