DBIx::Class: How can I sort on multiple substrings of a column? - perl

I have a SQLite-database with table with a document number following this schema:
16-145-45
16-127-30
16-141-42
16-122-14
15-090-04
15-089-15
15-089-05
I'd like to sort the ResultSet on the first and last part of the number, like this. First, all documents starting with the highest two-digit prefix (16) sorted by the last 2 digits and then the same with the next block, and so on.
16-145-45
16-141-42
16-127-30
16-122-14
15-089-15
15-089-05
15-090-04
Is there a way to do this in DBIx::Class with some sort of custom order_by clause, or what would be the approach?
I have tried the following, which does not work, because the middle part of the number is also considered for sorting:
my #rs = $self->search(undef,
{
order_by => { -desc => 'me.number' }
}
);

If you want the database to sort the results, you have to use literal SQL.
Here's an example for Postgres (I added a space after the backslash to fix the syntax highlighting):
my #rs = $self->search(undef,
{
order_by => \ "split_part(number, '-', 1) || split_part(number, '-', 3) DESC",
}
);
Or, by creating an output column with the +select result set attribute:
my #rs = $self->search(undef,
{
'+select' => [
{ sort_key => \ "split_part(number, '-', 1) || split_part(number, '-', 3)" },
],
'+as' => [ qw(sort_key) ], # Make sort key accessible from DBIC.
order_by => { -desc => 'sort_key' },
}
);
Another approach is to retreive the whole unsorted result set, and sort it on the client side. DBIC doesn't have any specific features to help you with that, so simply use Perl's sort function.

Since the answer from #nwellnhof works like a charm, I just wanted to provide the corresponding syntax for SQLite, which does not know the split_part() function.
# SQL for filtering the doc number in SQLite
my #rs = $self->search(undef,
{
order_by => \ "SUBSTR(me.number, 1, 2) || SUBSTR(me.number, -2, 2) DESC"
}
);

You need to extract additional columns from the result set which are equal to the value of the function that you want to sort by. Then you can just put those columns in an order_by clause as normal
This assumes that your document number field is called docnum. It fetches all the columns from Table plus the two substrings of docnum called docnum1 and docnum3
my $rs = $schema->resultset('Table')->search(undef,
{
'+select' => [
{ substr => [ 'docnum', 1, 2 ], -as => 'docnum1' },
{ substr => [ 'docnum', -2 ], -as => 'docnum3' },
],
order_by => [ { -desc => 'docnum1' }, { -desc => 'docnum3' } ],
}
);

Related

Why `select` does not replace existing columns?

DOC for select attribute:
select indicates which columns should be selected from the storage
This works as expected:
$rs->search( undef, { select => [ 'me.id', 'me.user_id' ] } )
->search( undef, { select => [ 'me.role_id' ] } )
$rs->as_query; # SELECT "me"."role_id" FROM "users_roles" "me"
But this does not:
$rs->search( undef, { prefetch => [ 'User' ] } )
->search( undef, { select => [ 'User.name' ] } )
$rs->as_query; # SELECT "User"."name", "User"."id", "User"."email", "User"."name" FROM "users_roles" "me" JOIN "users" "User" ON "User"."id" = "me"."user_id"
The prefetch implies +columns ( which is: +select and +as ).
I alter select in second search by requesting only one column: name
What did I miss?
Why I still get columns name, id, email, name instead of only one name?
If I rewrite prefetch by join. This works fine:
$rs->search( undef, {
,join => [ 'User' ]
,collaplse => 1
,'+columns' => [map
{ +{ "cds.$_" => "cds.$_" } }
$c->db->source('Right')->related_source('User')->columns
]
})->search( undef, { select => [ 'User.name' ] } )
$rs->as_query; # SELECT "User"."name" FROM "users_roles" "me" JOIN "users" "User" ON "User"."id" = "me"."user_id"
So it seems there is a bug with prefetch. Because join+collapse++columns is not same as prefetch:
DOC
This attribute is a shorthand for specifying a "join" spec, adding all columns from the joined related sources as "+columns" and setting "collapse" to a true value.
Also docs states:
For example, the following two queries are equivalent
As you can see join and prefetch are not equivalent

Hash not passing properly between functions

I set up a user hash in a function with the following,
push #{$profile{$index}{$infoName}}, $information
and print it using print Dumper(\%profile); index++; in the function it was set up it prints each of indexes
`$VAR1 = { '374' => { 'degree' => [ 'CS' ], 'birthdate' => [ '1973/12/13' ], 'gender' => [ 'M' ],...}
$VAR1 = { '375' => { 'degree' => [ 'CS' ], 'birthdate' => [ '1933/02/03' ], 'gender' => [ 'F' ],...}`
when i try to access this within another function's foreach loop using print "${$profile{$currIndex}{'gender'}}"; i get odd behaviour where the print returns an empty string and get some random numbers appear in the hash: '$VAR1 = { '4' => {}, '1' => {}, '3' => {}, '2' => {}, '378' => { 'birthdate' => [ '1961/03/29' ], 'gender' => ['F'],..}
How can i properly access the gender feild from within a loop?
push #{$profile{$index}{$infoName}}, $information;
print "${$profile{$currIndex}{'gender'}}";
I'm not even sure what the second line actually does. On my Ubuntu, perl produces an error: not a scalar reference.
What you want is, to print all array elements:
print "#{$profile{$currIndex}{'gender'}}\n";
or, to print the first:
print $profile{$currIndex}->{'gender'}->[0], "\n";
The leaf element is an array references and has to be dereferenced as such.
I'm not sure why you use there array references. In your sample data there are no multiple elements in the arrays. Probably you wanted to write simply this? -
$profile{$index}{$infoName} = $information;
...
print "$profile{$currIndex}{'gender'}\n";

Alias the sum of two columns in a DBIx::Class resultset

SELECT me.id, me.date_created, me.date_updated, me.yes,
me.name, me.description, me.currency, me.locked, me.skip,
me.uri_part, me.user_id,
yes + currency as weight
FROM ideas me having ((weight < 5)) order by weight;
How can I generate that query in DBIx::Class without using literal SQL like this:
my $query = $rs->search({},
{
'+select' => \[
'yes + currency as weight',
],
rows => 1,
order_by => { -desc => [qw/weight name/] },
having => {
weight => { '<' => $self->yes + $self->currency },
},
});
use Data::Dumper;
warn Dumper($query->as_query);
I tried using -as, however, it seems to only be useful for working with columns generated from functions, as this:
'+select' => {
'yes + currency', '-as' => 'weight'
}
generates an error
"Odd number of elements in anonymous hash at
/data/TGC/lib/TGC/DB/Result/Idea.pm line 105, line 1000.
DBIx::Class::SQLMaker::_recurse_fields(): Malformed select argument -
too many keys in hash: SCALAR(0xbf14c40),weight"
Probably the most idiomatic thing I can think of in SQL Abstract expression, without straining too hard:
#!/usr/bin/env perl
use Modern::Perl;
use MySchema;
use Data::Dumper;
my $schema = MySchema->connect('dbi:SQLite:example.db');
my $rs = $schema->resultset('Sample')->search(
{
weight => { '<' => 5 },
},
{
'+select' => [
{ '' => \'`me`.`yes` + `me`.`currency`', -as => 'weight' }
]
}
);
say Dumper( $rs->as_query() );
Which is a contrived wrapping of the column names, but it does the job, sort of. Just don't know of any way to abstract the + here. But stil:
'(SELECT me.name, me.yes, me.currency, ( me.yes + me.currency ) AS weight FROM sample me WHERE ( weight < ? ))',
Unless you are just going for idiomatic perl, in which case:
{ '' => \(join " + ", qw/`me`.`yes` `me`.`currency`/), -as => 'weight' }
But either way seems a little contrived considering both forms are longer than the literal string.
Also note that weight is referenced in WHERE and not HAVING which is because that will blow up on various SQL engines.
The '+as' should be at the same level as '+select'
$idea = $rs->search({-and => [
name => { '<' => $self->name },
]},
{
'+select' => [
'yes + currency'
],
'+as' => [qw/weight/],
rows => 1,
order_by => ['weight', 'name'],
having => {
weight => { '<' => $self->yes + $self->currency },
},
})->single;

How to nest .or() and .and() conditions in Mongoid

I have a 'search' function where I want to pass in an arbitrary 'filter' condition and have matches returned
The following matches any name/email where the filter string is a match:
#people = Person.all
#people = #people.or(
{'name.first_name' => /#{filter}/i},
{'name.last_name' => /#{filter}/i},
{'email' => /#{filter}/i }
)
The following correctly does the same on the 'tags' array on the Person record:
#people = Person.all
#people = #people.any_in('tags' => [/#{filter}/i])
Can anyone tell me how to combine the two queries, so that a Person is matched if the filter text is found in the name, email or any of the tags?
It turns out there is a method I was missing here ... found indirectly via https://github.com/mongoid/mongoid/issues/2845
Given these two queryables:
a=Person.where({'name.first_name'=> /a/i})
b=Person.where({'name.first_name'=> /j/i})
You can combine them using .selector
Person.or(a.selector, b.selector).to_a
=> selector={"$or"=>[{"name.first_name"=>/a/i}, {"name.first_name"=>/j/i}]}
or
Person.and(a.selector, b.selector).to_a
=> selector={"$and"=>[{"name.first_name"=>/a/i}, {"name.first_name"=>/j/i}]}
You don't need to use any_in at all. If say:
:array_field => regex
then MongoDB will automatically check each element of array_field against regex for you, you don't have care about the arrayness at all. That means that you can toss the :tags check in with the other conditions:
regex = /#{filter}/i
#people = Person.where(
:$or => [
{ 'name.first_name' => regex },
{ 'name.last_name' => regex },
{ 'email' => regex },
{ 'tags' => regex }
]
)
I also pull the regex out of the query into a variable to make it clear that you're using the same one for each check and I switched to where as that's more common (at least in my experience).

SQL::Abstract::Limit failing at OR logic

I'm trying to create an OR logic query using Class::DBI/Class::DBI::AbstractSearch. My code looks something like this:
my $results = Example::CDBI::Quote->search_where(
{ field_1 => {'like', $search_string},
field_2 => {'like', $search_string}},
{logic => 'or'}
);
According to the documentation this should work. It says that the information is passed to SQL::Abstract::Limit, which shows as taking the logic parameter. Unfortunately, MySQL shows the following in the query log (edited for brevity, and assuming a search of "123"):
SELECT * FROM quote WHERE ((field_1 LIKE '123' AND field_2 LIKE '123' ))
I have trying changing 'or' to 'OR' (silly, but worth a shot) which did not work. I also tried hunting down the logic in SQL::Abstract::Limit, but this operator is being passed to SQL::Abstract instead.
How do I get SQL::Abstract::Limit to accept OR logic from Class::DBI?
How Class::DBI calls SQL::Abstract::Limit
I was able to determine how SQL::Abstract::Limit is being constructed. I put values in instead of the variable names so it is easier to read.
my $sql = SQL::Abstract::Limit->new({'logic' => 'OR'});
my($phrase, #bind) = $sql->where(
{'field_1'=>{'like' => '123'},'field_2'=>{'like'=>'123'}},
undef, undef, undef);
You can apply OR locally like this:
use SQL::Abstract;
my $sql = SQL::Abstract->new;
my ($stmt, #bind) = $sql->where(
{ -or => [ { field_1 => { 'like', 'John' }},
{ field_2 => { 'like', 'John' }},
],
}, []);
gives in $stmt:
WHERE ( ( field_1 LIKE ? OR field_2 LIKE ? ) )
The logic property can be set in SQL::Abstract constructor, but I don't have idea how to propagate from Class::DBI.
Edit: I don't know if it is bug or feature, but it the operators changed by logic clause seems apply only when you define with arrayrefs. With hashrefs, you get always AND:
my $sql_and = SQL::Abstract::Limit->new(logic => 'AND');
my $sql_or = SQL::Abstract::Limit->new(logic => 'OR');
say $sql_and->where(['field_1'=>{'like' => '123'},'field_2'=>{'like'=>'123'}]);
# WHERE ( ( field_1 LIKE ? AND field_2 LIKE ? ) )
say $sql_or->where (['field_1'=>{'like' => '123'},'field_2'=>{'like'=>'123'}]);
# WHERE ( ( field_1 LIKE ? OR field_2 LIKE ? ) )
Or, to work with Class::DBI:
my $results = Example::CDBI::Quote->search_where(
[ field_1 => {'like', $search_string},
field_2 => {'like', $search_string}],
{logic => 'or'}
);