I have:
ThinkingSphinx::Index.define :new_post, with: :real_time do
indexes title, sortable: true
indexes text
has state, type: :string
has forum_hidden, type: :boolean
has created_at, type: :timestamp
has publish_at, type: :timestamp
has reminde_at, type: :timestamp
has deleted_at, type: :timestamp
has content_category_ids, type: :integer, multi: true
end
And let's i need to get all the records where #title=query with any value publish_at or #text=query with publish_at = 1.month.ago..Time.current
That is, I need to combine these two requests:
NewPost.search(conditions: { title: query })
NewPost.search(conditions: { text: query }, with: { publish_at: 1.month.ago..Time.current })
The result is needed with excerpts
UPDATE
published_at interval for the #title and #text fields is always different and depends on the user's rights. For example, there can be such a situation:
NewPost.search(conditions: { title: query }, with: { publish_at: 1.year.ago..Time.current })
NewPost.search(conditions: { text: query }, with: { publish_at: 6.month.ago..Time.current })
and all results that do not fall under these conditions should not be displayed at all
The main thing when wanting to 'combine' multiple criteria is deciding on a 'calculation' how to compute a weight that gives that effect. Sphinx computes a weight and orders by that. Rather than unions of multiple distinct queries
For example
https://freelancing-gods.com/thinking-sphinx/searching.html#sorting
ThinkingSphinx.search(
:select => '*, WEIGHT() + IF(publish_at>NOW()-2592000,1000,0) AS custom_weight',
:order => 'custom_weight DESC'
)
would use a the 'full-text' search weight, but add 1000 if in last month.
Uses the sphinx NOW() function
http://sphinxsearch.com/docs/current.html#expr-func-now
You might also want field weights
https://freelancing-gods.com/thinking-sphinx/searching.html#fieldweights
to boost title matches over 'text' matches
:field_weights => { :title=>10 }
Bringing all togehter (if got the ruby syntax right... )
NewPost.search( query ,
:select => '*, weight() + IF(publish_at>NOW()-2592000,1000,0) as custom_weight',
:order => 'custom_weight DESC',
:field_weights => { :title=>10 }
)
... in theory title matches will be first. And recent ones towards start too. Not EXACTLY what you asked for, but close.
Related
I'm kinda new to mongoose, and I'm not sure if it's a right term.
what I'm building is a community site (like redit), and I have a schema like below
const postSchema = new mongoose.Schema({
content: {
type: String,
required: true,
},
title: {
type: String,
required: true,
},
userId: {
type: mongoose.Schema.Types.ObjectId,
required: true,
ref: 'User',
},
board: {
type: String,
required: true,
enum: ['board1','board2'],
},
created_at: {
type: Date,
default: Date.now,
},
updated_at: {
type: Date,
},
})
there are many kinds of 'board'
and I'm not sure if it can be 'indexed'.
purpose of it is for getting posts faster
for example in sql (assume that board column is indexed)
--> select * from post where board = 'board1' ;
I'm confusing about the terms, need some direction..
Short answer:
You need to create an index on the field board by doing:
db.post.createIndex(
{ board: 1 } ,
{ name: "borad index" }
)
Long answer:
Indexing in mongodb uses memory in order to save running time.
Let's take an example: say you have all words in English in your DB. And you are reading a book and from time to time you need to search for a word to check its meaning.
How would you do that? A dictionary. You'll sort the words alphabetically and then you could easily search for every word you wanted.
Indexing apply the same concept. When you create an index on the field board it takes all its values, sort them and save it in a table (and reference for each entry the full document from your collection).
Now when you search for select * from post where board = 'board1' it first use the memorized table of sorted boards, finds the ones that equal to board1 and then by the reference gives you the full documents that belongs to it. You can continue reading here.
I am using KeystoneJS with PostgreSQL as my backend and Apollo on the frontend for my app.
I have a schema that has a list that is linked to another list.
I want to be able to allow users to change the order of the second list.
This is a simplified version of my schema
keystone.createList(
'forms',
{
fields: {
name: {
type: Text,
isRequired: true,
},
buttons: {
type: Relationship,
ref: 'buttons.attached_forms',
many: true,
},
},
}
);
keystone.createList(
'buttons',
{
fields: {
name: {
type: Text,
isRequired: true,
},
attached_forms: {
type: Relationship,
ref: 'forms.buttons',
many: true,
},
},
}
);
So what I would like to do, is allow users to change the order of buttons so when I fetch them in the future from forms:
const QUERY = gql`
query getForms($formId: ID!) {
allforms(where: {
id: $formId,
}) {
id
name
buttons {
id
name
}
}
}
`;
The buttons should come back from the backend in a predefined order.
{
id: 1,
name: 'Form 1',
buttons: [
{
id: 1,
name: 'Button 1',
},
{
id: 3,
name: 'Button 3',
},
{
id: 2,
name: 'Button 2',
}
]
}
Or even just have some data on that returns with the query that will allow for sorting according to the user-defined sort order on the frontend.
The catch is that this relationship is many to many.
So it wouldn't be enough to add a column to the buttons schema as the ordering needs to be relationship-specific. In other words, if a user puts a particular button last on a particular form, it shouldn't change the order of that same button on other forms.
In a backend that I was creating myself, I would add something to the joining table, like a sortOrder field or similar and then change those values to change the order, or even order them on the frontend using that information.
Something like this answer here.
The many-to-many join table would have columns like formId, buttonId, sortOrder.
I have been diving into the docs for KeystoneJS and I can't figure out a way to make this work without getting into the weeds of overriding the KnexAdapter that we are using.
I am using:
{
"#keystonejs/adapter-knex": "^11.0.7",
"#keystonejs/app-admin-ui": "^7.3.11",
"#keystonejs/app-graphql": "^6.2.1",
"#keystonejs/fields": "^20.1.2",
"#keystonejs/keystone": "^17.1.2",
"#keystonejs/server-side-graphql-client": "^1.1.2",
}
Any thoughts on how I can achieve this?
One approach would be to have two "button" lists, one with a template for a button (buttonTemplate below) with common data such as name etc, and another (button below) which references one buttonTemplate and one form. This allows you to assign a formIndex property to each button, which dictates its position on the corresponding form.
(Untested) example code:
keystone.createList(
'Form',
{
fields: {
name: {
type: Text,
isRequired: true,
},
buttons: {
type: Relationship,
ref: 'Button.form',
many: true,
},
},
}
);
keystone.createList(
'Button',
{
fields: {
buttonTemplate: {
type: Relationship,
ref: 'ButtonTemplate.buttons',
many: false,
},
form: {
type: Relationship,
ref: 'Form.buttons',
many: false,
},
formIndex: {
type: Integer,
isRequired: true,
},
},
}
);
keystone.createList(
'ButtonTemplate',
{
fields: {
name: {
type: Text,
isRequired: true,
},
buttons: {
type: Relationship,
ref: 'Button.buttonTemplate',
many: true,
},
},
}
);
I think this is less likely to cause you headaches (which I'm sure you can see coming) down the line than your buttonOrder solution, e.g. users deleting buttons that are referenced by this field.
If you do decide to go with this approach, you can guard against such issues with the hook functionality in Keystone. E.g. before a button is deleted, go through all the forms and rewrite the buttonOrder field, removing any references to the deleted button.
I had a similar challenge once, so after some research and found this answer, I implemented a solution to a project using PostgreSQL TRIGGER.
So you can add a trigger where on an update, it should shift the buttonOrder.
Here is the SQL I had on me, this was the test code, I regex replaced the terms to fit your question :)
// Assign order
await knex.raw(`
do $$
DECLARE form_id text;
begin
CREATE SEQUENCE buttons_order_seq;
CREATE VIEW buttons_view AS SELECT * FROM "buttons" ORDER BY "createdAt" ASC, "formId";
CREATE RULE buttons_rule AS ON UPDATE TO buttons_view DO INSTEAD UPDATE buttons SET order = NEW.order WHERE id = NEW.id;
FOR form_id IN SELECT id FROM form LOOP
ALTER SEQUENCE buttons_order_seq RESTART;
UPDATE buttons_view SET order = nextval('buttons_order_seq') WHERE "formId" = form_id;
END LOOP;
DROP SEQUENCE buttons_order_seq;
DROP RULE buttons_rule ON buttons_view;
DROP VIEW buttons_view;
END; $$`);
// Create function that shifts orders
await knex.raw(`
CREATE FUNCTION shift_buttons_order()
RETURNS trigger AS
$$
BEGIN
IF NEW.order < OLD.order THEN
UPDATE buttons SET order = order + 1, "shiftOrderFlag" = NOT "shiftOrderFlag"
WHERE order >= NEW.order AND order < OLD.order AND "formId" = OLD."formId";
ELSE
UPDATE buttons SET order = order - 1, "shiftOrderFlag" = NOT "shiftOrderFlag"
WHERE order <= NEW.order AND order > OLD.order AND "formId" = OLD."formId";
END IF;
RETURN NEW;
END;
$$
LANGUAGE 'plpgsql'`);
// Create trigger to shift orders on update
await knex.raw(`
CREATE TRIGGER shift_buttons_order BEFORE UPDATE OF order ON buttons FOR EACH ROW
WHEN (OLD."shiftOrderFlag" = NEW."shiftOrderFlag" AND OLD.order <> NEW.order)
EXECUTE PROCEDURE shift_buttons_order()`);
One option that we came up with is to add the order to the form table.
keystone.createList(
'forms',
{
fields: {
name: {
type: Text,
isRequired: true,
},
buttonOrder: {
type: Text,
},
buttons: {
type: Relationship,
ref: 'buttons.attached_forms',
many: true,
},
},
}
);
This new field buttonOrder could contain a string representation of the order of the button Ids, like in a JSON stringified array.
The main issue with this is that it will be difficult to keep this field in-sync with the actual linked buttons.
I am unable to get the case insensitive search from the database using Sails.js V1.0 + Waterline ORM. I am using sails-postgresql adapter. The running environment is Heroku + Heroku PostgreSQL.
Is there any way to turn off the following setting in database adapter - For performance reasons, case-sensitivity of contains depends on the database adapter.
Tried the method:
Datastore configuration is:
default: {
adapter: 'sails-postgresql',
url: 'postgres://....',
ssl: true,
wlNext: {
caseSensitive: true
}
}
The code block is:
var meetings = await Meeting.find({
select: ['id', 'uid', 'name', 'deleted', 'complete'],
where: {
owner: user.id,
name: { contains: searchJson.name } : ""
},
skip: (inputs.page > 0) ? (inputs.page) : 0,
limit: (inputs.limit > 0) ? (inputs.limit) : 10,
sort: 'date DESC'
});
The easiest way I've found to handle this and case-insensitive unique indexes with PG/Sails is to use the citext column type instead of text/character varying types (compared to forcing everything to lowercase which stinks).
citext is a case insensitive text datatype. "Essentially, it internally calls lower when comparing values. Otherwise, it behaves almost exactly like text."
An example model attribute pulled from a working app:
username: {
type: 'string',
columnType: 'citext',
required: true,
unique: true,
description: 'A users.. Username',
// ...
},
According to this (somewhat irellevant) Heroku docs page this looks like it should work, but you may need to run create extension citext; on your database first.
For MongoDB https://github.com/balderdashy/sails/issues/7014.
From sails-mongo#1.2.0, you can chain on .meta({makeLikeModifierCaseInsensitive: true}) for a case-insensitive query.
Example
await User.find(criteria).meta({makeLikeModifierCaseInsensitive: true});
You can use native queries, for example:
const eventTypesRows = await Meeting.getDatastore()
.sendNativeQuery(\`SELECT "name" FROM meeting WHERE LOWER(name) = LOWER($1)\`, [searchName]);
I created a Sails application with two models Publication and Worksheet. They are having a one-to-one relationship. Sails-postgresql is the adapter I'm using. I'm using waterline orm to fire query to the database. I'm When I am trying to load publications data along with worksheet and then sort the records depending on a field in the Worksheet using sort() I'm getting an error.
My model is:
Publication.js
module.exports = {
attributes: {
id: {
type: 'integer'
unique: true
},
worksheetId: {
type: 'integer',
model : 'worksheet'
},
status: {
type: 'string',
defaultsTo: 'active',
in : ['active', 'disabled'],
}
}
}
Worksheet.js
module.exports = {
attributes: {
id: {
type: 'integer',
unique: true
},
name: 'string',
orderWeight: {
type: 'integer',
defaultsTo: 0
}
}
}
So now I want to load all the publication where status is "active" and populate worksheet in the data.
So I'm executing the query:
Publication.find({
where: {
status: 'active'
}
})
.populate('worksheetId').limit(1)
.exec(function (error, publications) {
})
And I'm getting a data like :
{
id : 1,
status : "active",
worksheetId : {
id : 1
name : "test",
orderWeight : 10
}
}
So till now it's all working fine. Now I want to increase the limit to 10 and want to sort the data depending on "orderWeight" which is in the populated data. Initially I sorted the whole data depending on publication id and the query worked.
Publication.find({
where: {
status: 'active'
}
})
.populate('worksheetId').sort('id ASC').limit(10)
.exec(function (error, publications) {
})
So I fired similar query to sort the data on "orderWeight"
Publication.find({
where: {
status: 'active'
}
})
.populate('worksheetId').sort('worksheetId.orderWeight ASC').limit(10)
.exec(function (error, publications) {
})
And this query is giving me error that worksheetId.orderWeight is not a column on the publication table. So I want to fire this sort query on the populated data not on the publication table.
Please let me know how I can get my expected result.
Apart from sort() method I also want to run some find command to the populated data to get those publication where the worksheet name matches with certain key as well.
Basically, what you're trying to do, is query an association's attribute. This has been in the waterline roadmap since 2014, but it's still not supported, so you'll have to figure out a workaround.
One option is to query the Worksheet model, and populate the Publication, since sails doesn't let you query across models without using raw queries (i.e. .sort('worksheetId.orderWeight ASC') doesn't work). Unfortunately, you might have to move the active flag to the Worksheet. For example:
Worksheet.find({
status: 'active'
})
.populate('publication') // you should also add publication to Worksheet.js
.sort('orderWeight ASC')
.limit(10)
Alternatively, you could combine Worksheet and Publication into one model, since they're one-to-one. Probably not ideal, but sails.js and Waterline make it very difficult to work with relational data - I'd estimate that half of the queries in the project I'm working on are raw queries due to sails' poor support of postgres. The framework is pretty biased towards using MongoDB, although it claims to "just work" with any of the "supported" DBs.
I have the following pg_search scope on my stories.rb model:
pg_search_scope :with_text,
:against => :title,
:using => { :tsearch => { :dictionary => "english" }},
:associated_against => { :posts => :contents }
I want the query to return the results ignoring any ranking (I care only about the date the story was last updated order DESC). I know that this is an easy question for most of the people who view it, but how do I turn off the rank ordering in pg_search?
I'm the author of pg_search.
You could do something like this, which uses ActiveRecord::QueryMethods#reorder
MyModel.with_text("foo").reorder("updated_at DESC")