How to nest .or() and .and() conditions in Mongoid - mongodb

I have a 'search' function where I want to pass in an arbitrary 'filter' condition and have matches returned
The following matches any name/email where the filter string is a match:
#people = Person.all
#people = #people.or(
{'name.first_name' => /#{filter}/i},
{'name.last_name' => /#{filter}/i},
{'email' => /#{filter}/i }
)
The following correctly does the same on the 'tags' array on the Person record:
#people = Person.all
#people = #people.any_in('tags' => [/#{filter}/i])
Can anyone tell me how to combine the two queries, so that a Person is matched if the filter text is found in the name, email or any of the tags?

It turns out there is a method I was missing here ... found indirectly via https://github.com/mongoid/mongoid/issues/2845
Given these two queryables:
a=Person.where({'name.first_name'=> /a/i})
b=Person.where({'name.first_name'=> /j/i})
You can combine them using .selector
Person.or(a.selector, b.selector).to_a
=> selector={"$or"=>[{"name.first_name"=>/a/i}, {"name.first_name"=>/j/i}]}
or
Person.and(a.selector, b.selector).to_a
=> selector={"$and"=>[{"name.first_name"=>/a/i}, {"name.first_name"=>/j/i}]}

You don't need to use any_in at all. If say:
:array_field => regex
then MongoDB will automatically check each element of array_field against regex for you, you don't have care about the arrayness at all. That means that you can toss the :tags check in with the other conditions:
regex = /#{filter}/i
#people = Person.where(
:$or => [
{ 'name.first_name' => regex },
{ 'name.last_name' => regex },
{ 'email' => regex },
{ 'tags' => regex }
]
)
I also pull the regex out of the query into a variable to make it clear that you're using the same one for each check and I switched to where as that's more common (at least in my experience).

Related

ApiPlatform Regex property filter with Postgresql

I'm trying to add a custom filter to an entity in my ApiPlatform project that allows me to filter on specific property given a regex pattern.
Following the ApiPlatform documentation I came up with the following class (this is a near copy of the their example, only the where-clause is different):
<?php
namespace App\Filter;
use ApiPlatform\Core\Bridge\Doctrine\Orm\Filter\AbstractContextAwareFilter;
use ApiPlatform\Core\Bridge\Doctrine\Orm\Util\QueryNameGeneratorInterface;
use Doctrine\ORM\QueryBuilder;
final class RegexFilter extends AbstractContextAwareFilter
{
protected function filterProperty(
string $property,
$value,
QueryBuilder $queryBuilder,
QueryNameGeneratorInterface $queryNameGenerator,
string $resourceClass,
string $operationName = null
) {
// otherwise filter is applied to order and page as well
if (
!$this->isPropertyEnabled($property, $resourceClass) ||
!$this->isPropertyMapped($property, $resourceClass)
) {
return;
}
$parameterName = $queryNameGenerator->generateParameterName($property); // Generate a unique parameter name to avoid collisions with other filters
$queryBuilder
->andWhere(sprintf('(o.%s ~ :%s) = true', $property, $parameterName))
->setParameter($parameterName, $value);
}
// This function is only used to hook in documentation generators (supported by Swagger and Hydra)
public function getDescription(string $resourceClass): array
{
if (!$this->properties) {
return [];
}
$description = [];
foreach ($this->properties as $property => $strategy) {
$description["regexp_$property"] = [
'property' => $property,
'type' => 'string',
'required' => false,
'swagger' => [
'description' => 'Filter using a regex. This will appear in the Swagger documentation!',
'name' => 'Custom name to use in the Swagger documentation',
'type' => 'Will appear below the name in the Swagger documentation',
],
];
}
return $description;
}
}
When I run my code this results in the following DQL:
SELECT o FROM App\Entity\Vehicle o WHERE (o.licensePlate ~ :licensePlate_p1) = true ORDER BY o.id ASC
However I cannot get the Lexer to understand the tilde ~ character:
Doctrine\ORM\Query\QueryException
[Syntax Error] line 0, col 56: Error: Expected Doctrine\ORM\Query\Lexer::T_CLOSE_PARENTHESIS, got '~'
How can I make Doctrine understand the tilde?
Turns out I was pretty close, simPod's answer helped me fill the gaps.
I copied his custom DQL function and added it in my Doctrine yaml configuration
In my RegexFilter I had to slightly modify the where-clause:
->andWhere(sprintf('REGEX(o.%s, :%s) = true', $property, $parameterName))

How to do atomic operations like $push $pull $set etc. in Ecto MongoDB

I'm using mongodb_ecto and I want to know how can I do operations like $push or $pull on a deeply nested field? At the moment I write back the whole document which sometimes causes false data to be in the DB due to a race-condition.
Ok, I kind of figured it out. Do not use Ecto for this. In some cases you need the MongoDB positional operator and this can only be done directly via the Mongo-Adapter. Now for some usage examples:
I have a doucment with a list of options. Options have an ID, a label and list of userIDs who voted for this option.
BTW to generate an ObjectID (which is needed for talking directly to the MongoDB-Adapter) use this:
id = "584a5b9419d51d724d146e3f" # string form
value = (for <<hex::16 <- id>>, into: <<>>, do: <<String.to_integer(<<hex::16>>, 16)::8>>)
object_id = %BSON.ObjectId{value: value}
And now for some examples:
# update label of option
Mongo.update_one(Repo.Pool, "polls",
%{"_id" => document_id, "options.id" => option_id}, # query
%{"$set" => %{"data.options.$.label" => new_label}} # update
)
# add new option to poll
Mongo.update_one(Repo.Pool, "polls",
%{"_id" => document_id},
%{"$addToSet" => %{"options" => %{label: label, id: random_new_id, votes: []}}}
)
# add user_id to option
Mongo.update_one(Repo.Pool, "polls",
%{"_id" => document_id, "options.id" => option_id},
%{"$addToSet" => %{"options.$.votes" => user_id}}
)
# remove user_id form option
Mongo.update_one(Repo.Pool, "polls",
%{"_id" => document_id, "options.id" => option_id},
%{"$pull" => %{"data.options.$.votes" => user_id}}
)

DBIx::Class: How can I sort on multiple substrings of a column?

I have a SQLite-database with table with a document number following this schema:
16-145-45
16-127-30
16-141-42
16-122-14
15-090-04
15-089-15
15-089-05
I'd like to sort the ResultSet on the first and last part of the number, like this. First, all documents starting with the highest two-digit prefix (16) sorted by the last 2 digits and then the same with the next block, and so on.
16-145-45
16-141-42
16-127-30
16-122-14
15-089-15
15-089-05
15-090-04
Is there a way to do this in DBIx::Class with some sort of custom order_by clause, or what would be the approach?
I have tried the following, which does not work, because the middle part of the number is also considered for sorting:
my #rs = $self->search(undef,
{
order_by => { -desc => 'me.number' }
}
);
If you want the database to sort the results, you have to use literal SQL.
Here's an example for Postgres (I added a space after the backslash to fix the syntax highlighting):
my #rs = $self->search(undef,
{
order_by => \ "split_part(number, '-', 1) || split_part(number, '-', 3) DESC",
}
);
Or, by creating an output column with the +select result set attribute:
my #rs = $self->search(undef,
{
'+select' => [
{ sort_key => \ "split_part(number, '-', 1) || split_part(number, '-', 3)" },
],
'+as' => [ qw(sort_key) ], # Make sort key accessible from DBIC.
order_by => { -desc => 'sort_key' },
}
);
Another approach is to retreive the whole unsorted result set, and sort it on the client side. DBIC doesn't have any specific features to help you with that, so simply use Perl's sort function.
Since the answer from #nwellnhof works like a charm, I just wanted to provide the corresponding syntax for SQLite, which does not know the split_part() function.
# SQL for filtering the doc number in SQLite
my #rs = $self->search(undef,
{
order_by => \ "SUBSTR(me.number, 1, 2) || SUBSTR(me.number, -2, 2) DESC"
}
);
You need to extract additional columns from the result set which are equal to the value of the function that you want to sort by. Then you can just put those columns in an order_by clause as normal
This assumes that your document number field is called docnum. It fetches all the columns from Table plus the two substrings of docnum called docnum1 and docnum3
my $rs = $schema->resultset('Table')->search(undef,
{
'+select' => [
{ substr => [ 'docnum', 1, 2 ], -as => 'docnum1' },
{ substr => [ 'docnum', -2 ], -as => 'docnum3' },
],
order_by => [ { -desc => 'docnum1' }, { -desc => 'docnum3' } ],
}
);

Find query with and operator in PHP

Hi i am working on backend of web application & want to find the documents from mongodb database that contain key active_status with value set to both 1 & 2. With mongodb PHP i am confused of how to find with both parameters in single query.
My query was this:
$mongoDb = MongoDbConnector::getCollection("endusers");
$endUserData = $mongoDb->find(array('active_status' => 1, '$and' => array('active_status' => 2)));
I have to fetch the users whose active_status should be 1 & 2. The above query doesnt seems to work. What is the right one for that?
Thanks on advance for quick response.
You have $and the wrong way around. Both arguments need to be included:
$endUserData = $mongoDb->find(array(
'$and' => array(
array( 'active_status' => 1 )
array( 'active_status' => 2 )
)
));
And since that would only make sense when looking for both elements within an array element, then you should instead use $all, which is shorter syntax:
$endUserData = $mongoDb->find(array(
'active_status' => array( '$all' => array(1,2) )
));
I should add that unless you intend to match a document like this:
{ "active_status" => [1,2] }
The you do not in fact want $and at all, but rather you want $or or better yet $in for multiple possible values on the same field:
$endUserData = $mongoDb->find(array(
'active_status' => array( '$in' => array(1,2) )
));
This matches documents like this:
{ "active_status": 1 },
{ "active_status": 2 }

How to keep orders in MongoDB?

In my MongoDB document I have object like this
[_id] => MongoId Object (
[$id] => 52a46b44aabacb5c218b4567
)
[results] => Array (
[http://google.com] => Array (
[position] => 1
[data] => 42672
)
[http://bing.com] => Array (
[position] => 2
[data] => 9423
)
[http://yandex.com] => Array (
[position] => 3
[data] => 5513
)
)
I would like to change data parameter in "bing.com" from 9423 to for instance 300. Moreover, I have to keep order of the sites. It have to looks like this
[_id] => MongoId Object (
[$id] => 52a46b44aabacb5c218b4567
)
[results] => Array (
[http://google.com] => Array (
[position] => 1
[data] => 42672
)
[http://bing.com] => Array (
[position] => 2
[data] => 300
)
[http://yandex.com] => Array (
[position] => 3
[data] => 5513
)
)
Is this achievable in Mongo?
The reordering of fields issue has been fixed as of MongoDB v2.5.2 (2.6 release). Having said that one way you can avoid the issue completely is having results as an array instead of a (sub)document. Also note you should not use "." as part of the key name either.
With 2.4, with the following you will see there is reodering in the case of _id=1 (subdocument) but not in the case of _id=2 (array).
$document = array("_id" => 1, "results" => array('http://google.com' => array('position' => 1, 'data' => 42672),
'http://bing.com' => array('position' => 2, 'data' => 9423),
'http://yandex.com' => array('position' => 3, 'data' => 5513)));
$coll->insert($document);
$document = array("_id" => 2, "results" => array(array('site' => 'http://google.com', 'data' => 42672),
array('site' => 'http://bing.com', 'data' => 9423),
array('site' => 'http://yandex.com', 'data' => 5513)));
$coll->insert($document);
$coll->update(array("_id" => 1), array('$set'=>array("results.http://bing.com.data"=>300)));
$coll->update(array("_id" => 2, 'results.site' => 'http://bing.com'), array('$set'=>array('results.$.data'=>300)));
I've included examples below using the mongo shell for clarity, but the PHP equivalent should be straightforward to work out.
I notice you originally modelled your list of sites as an embedded document, however the order of fields within an embedded document is currently not guaranteed to be preserved so you should instead use an array.
Additionally, you cannot use field names with embedded dots (.) in MongoDB so you should not plan to store urls as field names (see: Field name restrictions).
In order to find an element in an array you need to search by a value (not a field name) so your schema should look more like:
{
_id: ObjectId("52a46b44aabacb5c218b4567"),
results: [
{
site: 'http://google.com',
position: 1,
data: 42762
},
{
site: 'http://bing.com',
position: 2,
data: 9423
},
{
site: 'http://yandex.com',
position: 3,
data: 5513
}
]
}
Assuming the array site elements are unique, you can use the positional operator $ to find and update the matching embedded document in place.
For example, to perform your update of the "bing.com" data value:
db.sites.update(
// Match criteria
{
_id:ObjectId("52a46b44aabacb5c218b4567"),
'results.site':'http://bing.com'
},
// Update
{ $set: {
'results.$.data': 300 }
}
)
In MongoDB 2.4+ you have the option of pushing to a sorted array which could also be a useful approach to maintaining your array in sorted order when you add new entries.
It's worth noting that if you plan to store many (i.e. thousands) of items in an array this can impose a significant performance penalty due to document growth and the complexity of updating large arrays.
I am pretty sure that (as every other DBMS) you can't and should't rely on records orders.
Instead I would advice you to add index (on position, i.e. db.people.ensureIndex( { position: 1 } )) and query your record sorted by that field, i. e.: db.collection.find().sort( { position: 1 } )