Sort populated record in sails waterline - postgresql

I created a Sails application with two models Publication and Worksheet. They are having a one-to-one relationship. Sails-postgresql is the adapter I'm using. I'm using waterline orm to fire query to the database. I'm When I am trying to load publications data along with worksheet and then sort the records depending on a field in the Worksheet using sort() I'm getting an error.
My model is:
Publication.js
module.exports = {
attributes: {
id: {
type: 'integer'
unique: true
},
worksheetId: {
type: 'integer',
model : 'worksheet'
},
status: {
type: 'string',
defaultsTo: 'active',
in : ['active', 'disabled'],
}
}
}
Worksheet.js
module.exports = {
attributes: {
id: {
type: 'integer',
unique: true
},
name: 'string',
orderWeight: {
type: 'integer',
defaultsTo: 0
}
}
}
So now I want to load all the publication where status is "active" and populate worksheet in the data.
So I'm executing the query:
Publication.find({
where: {
status: 'active'
}
})
.populate('worksheetId').limit(1)
.exec(function (error, publications) {
})
And I'm getting a data like :
{
id : 1,
status : "active",
worksheetId : {
id : 1
name : "test",
orderWeight : 10
}
}
So till now it's all working fine. Now I want to increase the limit to 10 and want to sort the data depending on "orderWeight" which is in the populated data. Initially I sorted the whole data depending on publication id and the query worked.
Publication.find({
where: {
status: 'active'
}
})
.populate('worksheetId').sort('id ASC').limit(10)
.exec(function (error, publications) {
})
So I fired similar query to sort the data on "orderWeight"
Publication.find({
where: {
status: 'active'
}
})
.populate('worksheetId').sort('worksheetId.orderWeight ASC').limit(10)
.exec(function (error, publications) {
})
And this query is giving me error that worksheetId.orderWeight is not a column on the publication table. So I want to fire this sort query on the populated data not on the publication table.
Please let me know how I can get my expected result.
Apart from sort() method I also want to run some find command to the populated data to get those publication where the worksheet name matches with certain key as well.

Basically, what you're trying to do, is query an association's attribute. This has been in the waterline roadmap since 2014, but it's still not supported, so you'll have to figure out a workaround.
One option is to query the Worksheet model, and populate the Publication, since sails doesn't let you query across models without using raw queries (i.e. .sort('worksheetId.orderWeight ASC') doesn't work). Unfortunately, you might have to move the active flag to the Worksheet. For example:
Worksheet.find({
status: 'active'
})
.populate('publication') // you should also add publication to Worksheet.js
.sort('orderWeight ASC')
.limit(10)
Alternatively, you could combine Worksheet and Publication into one model, since they're one-to-one. Probably not ideal, but sails.js and Waterline make it very difficult to work with relational data - I'd estimate that half of the queries in the project I'm working on are raw queries due to sails' poor support of postgres. The framework is pretty biased towards using MongoDB, although it claims to "just work" with any of the "supported" DBs.

Related

Mongo DB Time Series Performance on just secondary Index

I am building a mongoose model to store survey response data. There are, however, different types of surveys with different response rates. One type of survey has frequent answers (perhaps every few seconds) and data is normally queried in chunks of time, eg from startDate to endDate of the response. However, some surveys only get responses maybe a 20 times a month, and sometimes I would want to get all the data for that survey just based on the survey_id, and, not using any time field constraints.
So my question is, do secondary indexes on time series collections work as well as they would on a non-time series collection?
My model looks like this:
const responseSchema = mongoose.Schema(
{
metaData: {
type: new mongoose.Schema({
survey_id: { type: mongoose.Schema.Types.ObjectId, ref: "survey", required: true },
}),
required: true,
},
createdAt: Date,
answers: { type: Map, of: mongoose.Mixed },
},
{
timeseries: {
timeField: "createdAt",
metaField: "metaData",
granularity: "seconds",
},
}
);
responseSchema.plugin(ts);
responseSchema.index({ "metaData.survey_id": 1, "createdAt": 1 });
I would expect normal querys using the createdAt field as filters to work well, but what if I only query by survey_id and don't use the time field. Will that still work well? or do I get performance degradation by not using the time field with a time series collection.
querys of this collection will always be based on the survey_id

sailsjs one-way associations confusion

I am having a hard time wrapping my head around associations with sailsjs.
I have 2 models
Services
attributes: {
status: {
defaultsTo: 'inactive'
},
userId:{
model: 'users',
via: 'id',
},
},
Users
attributes: {
email: {
type: 'string',
required: true,
unique: true
},
password: {
type: 'string'
}
},
So, a service is tied to a user (matching the id of the user).
I used to do a call like http://localhost:1337/Services?userId=userId
Now I would like to transition to associations using the above model attributes.
This works by calling the ID of the service just fine (it includes the users data as well), however if all i have is the user, how could I get the service
Doing the same call (http://localhost:1337/Services?userId=userId) returns and empty object.
Am I forced to actually have a one-to-one or one-to-many association? I don't understand why I can no longer use the userId field (stored in the DB) to do queries once I start using associations. I guess I am looking for the best of both worlds here.
EDIT:
Let me try make this more clear. Before trying to do associations, I could call this URL (using blueprint)
http://localhost:1337/Services?userId=userId
The Services model used to look like this
attributes: {
status: {
defaultsTo: 'inactive'
},
userId:{
type: 'string',
required: true,
},
},
Then when a user is created, a service for that user is created with the userId matching the ID in the Users table.
Now I would like to implement associations using the above model scheme.
However, because (my best guess) the userId field of the service is mapped to the Users model, I am unable to search for a Server using the userId field that is stored.
I hope that makes sense? In another words, tryin to call
http://localhost:1337/Services?userId=userId
returns nothing when using associations but does return a value when I don't use associations

Can I use Sails native query with populate?

I am using the native method of sails-mongo to query a collection. I need to use native to access some of the Mongo geospatial query features.
I would like to use the sails populate syntax to include associated models.
Is there a way to do this?
Here is an example of my existing code:
Trip.native(function(err, collection) {
collection.find(
{
"locationTo": {
"$near": {
"$maxDistance": 80467.35439432222,
"$geometry": {
"type": "Point",
"coordinates": [-117.133655, 32.720519]
}
}
}
}
)
.toArray(function(err, trips) {
console.log("Trips nearby:", trips);
});
});
Here is my Trip model for reference.
var Trip = {
attributes: {
owner: {
model: 'User'
},
title: 'string',
addressFrom: 'string',
locationFrom: 'json', // geoJson
dateTimeDepart: 'datetime',
dateTimeArrive: 'datetime',
dateTimeReturn: 'datetime',
addressTo: 'string',
locationTo: 'json', // geoJson
driver: {
model: 'User'
},
status: {
type: 'string',
defaultsTo: 'PENDING'
}
}
}
Would be helpful if you share the Trip model as well. If the field you wish to populate has type "collection" (not "array"), you should be able to populate it just fine.
Update: Alright, I got your question wrong. There doesn't seem to be any way of populating directly after a native call. There's really not much you can do with a native call as far as Waterline functions are concerned. I would suggest either running another query(Waterline) after you've fetched locationTo or populating the fields yourself since you only need to populate two of them(and that too from the same model). I can't think of anything that would suffice with a single query.
Thanks, I ended up doing it in two queries for now.
First, I build an array of matching ID's via the native query.
var tripIdList = trips.map(function (trip) {
return trip._id
});
Second, I do a normal find query using the ID list. It's not a single query but works well. Thanks for the help
Trip.find(filter)
.where({id: tripIdList})
.populate('driver')
.exec(function (err, trips) {
console.log("Trips:", trips);
}

Sails.js - Many-to-Many fails to save

I have a simple need to add tags to patients. I followed the Sails and Waterline documentation concerning many-to-many associations, but it's failing at some point (no errors). I'm using MongoDB for data storage. Code below:
Tag Model
module.exports = {
attributes: {
name: 'STRING',
color: {
type: 'STRING',
defaultsTo: '#777777'
},
tagged: {
collection: 'patient',
via: 'tags',
dominant: true
}
}
};
Patient Model
module.exports = {
attributes: {
name: 'STRING',
tags: {
collection: 'tag',
via: 'tagged'
}
}
};
And this is the controller method that tries to associate data:
module.exports = {
addToPatient: function(req, res) {
Patient.findOne({id: req.param('patientId')}).exec(function(err, patient) {
// Queue up a record to be inserted into the join table
patient.tags.add(req.param('tagId'));
// Save the user, creating the new associations in the join table
patient.save(function(err) {});
});
res.send("tag assigned");
}
};
I've inspected the responses at various breaks and everything seems to be passing just fine. The patient is found. The save function shows a tag association in the patient object, but nothing is added in the database. I assume I will see either a join table being created or something in the patient/tag collections to signify an association, but I see nothing. I'm so very confused. If I do an HTTP get, I'm presented with a "tag assigned" response. What am I missing?
Works fine for me, but you're right in the tag and patient collections, you won't see a populated field with the associations. You'll see new join collections that are created that contains the relationships like #sgress454 pointed out.

Mongoose: How to model a foreign key/inverse relationship?

I am using Mongoose to model Person and Transaction collections, where each Transaction will have references to two different Person instances:
var TransactionSchema = new Schema({
, amount : { type: Number, required: true }
, from : { type: ObjectId, required: true }
, to : { type: ObjectId, required: true }
, date : Date
});
var PersonSchema = new Schema({
name : { type: String, required: true }
, transactions : [ObjectId]
});
I'd like each Person to have a collection of all the Transactions that they are either the to or from value for. So far, this is the best way I've been able to figure out how to do it:
TransactionSchema.pre('save', function(next, done) {
var transaction = this;
Person.findById(this.to, function (err, person) {
person.transactions.push(transaction);
person.save();
});
Person.findById(this.from, function (err, person) {
person.transactions.push(transaction);
person.save();
});
next();
});
This seems excessive. Is there a better way to do it, or am I trying to use MongoDB too much like a relational database? Instead of having a collection of Transactions associated with each Person instance, should I just be querying the Translation collection directly?
Thank you.
You've got to think more on the queries you are going to execute on the database when you design the MongoDB schema.
Try to duplicate data for speed and reference it for integrity. What does that mean?
Well, for example when you make a query for a Transaction, I guess you don't need all the user details from the first time no? (do you need the user's email, location when displaying info on a Transaction?)
I think you just probably need the user id and the username, so you should do something like this:
var TransactionSchema = new Schema({
, amount : { type: Number, required: true }
, from : {
user_id: {
type: ObjectId
, required: true
}
, username: {
type: String
, required: true
}
}
, to : {
user_id: {
type: ObjectId
, required: true
}
, username: {
type: String
, required: true
}
}
, date : Date
});
So instead of doing 3 queries for the page displaying the Transaction details (one for the transaction and 2 additional queries for the usernames), you'll have just one.
This is just an example, you could apply the same logic for the User schema, depending on what you're trying to achieve.
Anyway I don't think your middleware is ok, since you are not checking for errors there (you are always calling next no matter what). This is how I would write the middleware (didn't test, but the idea is important):
TransactionSchema.pre('save', function(next, done) {
var transaction = this;
Person.where('_id').in([this.to, this.from]).run(function (err, people) {
if (people.length != 2) { next(new Error("To or from doesn't exist")); return; }
Step(
function save_to() {
people[0].transactions.push(transaction);
people[0].save(this);
},
function save_from(err) {
if (err) { next(err); return; }
people[1].transactions.push(transaction);
people[1].save(this);
},
function callback(err) {
next(err);
}
);
});
});
In the code above I'm using the Step library for flow control and I'm only using one query instead of two (when searching for "to" and "from").