DBIx::Class infinite results - perl

Before I describe the details, the problem is, I run a $c->model('ResultName')->search({k=>v}) and when I loop on the results of it's has_many relation, there's only one in the database, yet it loops forever. I've tried googling and found one person who solved the problem, but with too brief an explanation for me. His post was here.
Basically I have 3 tables
Orders, OrderItems and Items. Items are what's available. Orders are collections of Items that one person wants. So I can tie them all together with something like
select oi.order_item_id,oi.order_id,i.item_id from orders as o inner join order_items as oi on oi.order_id = o.order_id inner join items as i on i.item_id = oi.item_id where blah blah blah....
I ran DBIx::Class::Schema::Loader and got what seemed like proper relationships
MyApp::Schema::Result::Order->has_many('order_items'...)
MyApp::Schema::Result::Items->has_many('order_items'...)
MyApp::Schema::Result::OrderItems->belongs_to('items'...)
in a test I try
my $orders = $schema->resultset('Order')->search({
'user_id'=>1
});
while(my $o = $orders->next) {
while(my $oi = $o->order_items->next) {
warn('order_item_id: '.$oi->order_item);
}
}
It loops infinitely on the inner loop

Your solution works but it loses the niceties of next in that it is an iterator. You are in effect loading all the rows as objects into memory and looping over them.
The issue, as you said is that $o->order_items->next recreates the order_items resultset each time. You should do this:
my $orders = $schema->resultset('Order')->search({
'user_id'=>1
});
while(my $o = $orders->next) {
my $oi_rs = $o->order_items;
while(my $oi = $oi_rs->next) {
warn('order_item_id: '.$oi->order_item);
}
}

Reading more carfully in the ResultSet documentation for "next" I found
"Note that you need to store the
resultset object, and call next on it.
Calling resultset('Table')->next
repeatedly will always return the
first record from the resultset."
from here
When I changed the loops to
for my $o ($orders->all) {
for my $oi ($o->order_items->all) {
# stuff
}
}
all is well.

Related

Iterating the results returned from fetchall_arrayref

I have a sql wherein I am fetching few records, sorted by full name.
My requirement is to extract chunks of similar names and then do some operation on it.
Say, the sql returns some records containing names like [name1,name1,name2,name3,name3]
I need to split them to [name1,name1] , [name2] , [name3,name3]
I am able to do it, but I am not happy with my implementation as I have to call doSomethingWithNames()twice.
while (my $paRecs = $pStatementHandle->fetchall_arrayref({}, 3)){
foreach my $phRec(#{$paRecs}){
my $pCurrentName = $phRec->{'FULL_NAME'};
if ((scalar(#aParentRecords) eq 0) or ($aParentRecords[-1] eq $pCurrentName)){
push(#aParentRecords, $pCurrentName);
} else {
doSomethingWithNames(\#aParentRecords);
#aParentRecords= ();
push(#aParentRecords, $pCurrentName);
}
}
};
doSomethingWithNames(\#aParentRecords); # This should be inside while loop
I believe am running into this issue because while doesn't go into the loop for
the last iteration as fetch* returns undef.
Sounds basic PERL stuff, but tried many loop constructs with no luck.
Any pointer will be a great help
The trick is to postpone existing the loop by converting it into an infinite loop. This requires checking the loop-terminating condition (!$rows) twice, though.
my $rows = [];
my $prev_name = '';
my #group;
while (1) {
$rows = $sth->fetchall_arrayref({}, 3) if !#$rows;
if (!$rows || $rows->[0]->{FULL_NAME} ne $prev_name)
if (#group) {
do_something(\#group);
#group = ();
}
last if !$rows;
}
$prev_name = $rows->[0]->{FULL_NAME};
push #group, shift(#$rows);
}

How can I get Perl DBI's selectrow_hashref to return a new row each iteration?

I am trying to use DBI's selectrow_hashref instead of fetchrow_hashref in order to save a couple lines of code, but it keeps returning the same row of data over and over.
my $select="SELECT * FROM table";
while (my ($user_ref) = $dbh->selectrow_hashref()) {
# $user_ref is the same each time!
}
When I use fetchrow_hashref, everything is fine, and each iteration I get new data.
my $select="SELECT * FROM table";
my $sth = $dbh->prepare($select) || die "prepare: $select: $DBI::errstr";
$sth->execute() || die "execute: $select: $DBI::errstr";
while (my ($user_ref) = $sth->fetchrow_hashref()) {
# works great, new data in $user_ref each iteration
}
Pray tell, what am I doing wrong? Is selectrow_hashref only intended to retrieve a single record? It doesn't seem that way in the doc.
Is selectrow_hashref only intended to retrieve a single record?
Yes.
It doesn't seem that way in the doc.
Well, that documentation says:
It returns the first row of data from the statement.
Which seems pretty clear to me.
Are you looking for selectall_hashref instead?
Update: Actually, I think you want selectall_array:
my $select='SELECT * FROM table';
foreach my $user_ref ($dbh->selectall_array($select, { Slice => {} })) {
# $user_ref is a hash ref
say $user_ref->{some_column};
}

Perl -> Avoiding unnecessary method calls

I have to read log files of a store. The log shows the item id and the word "sold" after it. So I made a script to read this file, counting how many times a word "sold" appears for each item id. Turns out that there are many "owners" for the items. That is, there is a relation between "owner_id" (a data in my DB) and "item_id". Im interested in knowing how many items owners sell per day, so I create a "%item_id_owner_map":
my %item_id_sold_times;
my %item_id_owner_map;
open my $infile, "<", $file_location or die("$!: $file_location");
while (<$infile>) {
if (/item_id:(\d+)\s*,\s*sold/) {
my $item_id = $1;
$item_id_sold_times{$item_id}++;
my $owner_ids =
Store::Model::Map::ItemOwnerMap->fetch_by_keys( [$item_id] )
->entry();
for my $owner_id (#$owner_ids) {
$item_id_owner_map{$owner_id}++;
}
}
}
close $infile;
The "Store::Model::Map::ItemOwnerMap->fetch_by_keys( [$item_id] )->entry();" method takes item_id or ids as input, and gives back owner_id as output.
Everything looks great but actually, you will see that every time Perl finds a regex match (that is, every time the "if" condition applies), my script will call "Store::Model::Map::ItemOwnerMap->fetch_by_keys" method, which is very expensive, as these log files are very very long.
Is there a way to make my script more efficient? If possible, I only want to call my Model method once.
Best!
Separate your logic into two loops:
while (<$infile>) {
if (/item_id:(\d+)\s*,\s*sold/) {
my $item_id = $1;
$item_id_sold_times{$item_id}++;
}
}
my #matched_items_ids = keys %item_id_sold_times;
my $owner_ids =
Store::Model::Map::ItemOwnerMap->fetch_by_keys( \#matched_item_ids )
->entry();
for my $owner_id (#$owner_ids) {
$item_id_owner_map{$owner_id}++;
}
I don't know if the entry() call is correct, but the general shape of that code should do it for you.
In general databases are good at fetching sets of rows, so you're right to minimise the calls to fetch from the DB.

Moodle Database API error : Get quiz marks for all sections of one course for one user

I am trying to get total marks obtained by a particular user, for a particular course for all the sections of that course.
The following query works and gives correct results with mysql, but not with Databse API calls
$sql = "SELECT d.section as section_id,d.name as section_name, sum(a.sumgrades) AS marks FROM mdl_quiz_attempts a, mdl_quiz b, mdl_course_modules c, mdl_course_sections d WHERE a.userid=6 AND b.course=4 AND a.quiz=b.id AND c.instance=a.quiz AND c.module=14 AND a.sumgrades>0 AND d.id=c.section GROUP BY d.section"
I tried different API calls, mainly I would want
$DB->get_records_sql($sql);
The results from API calls are meaningless. Any suggestion?
PS : This is moodle 2.2.
I just tried to do something similar, only without getting the sections. You only need the course and user id. I hope this helps you.
global $DB;
// get all attempts & grades from a user from every quiz of one course
$sql = "SELECT qa.id, qa.attempt, qa.quiz, qa.sumgrades AS grade, qa.timefinish, qa.timemodified, q.sumgrades, q.grade AS maxgrade
FROM {quiz} q, {quiz_attempts} qa
WHERE q.course=".$courseid."
AND qa.quiz = q.id
AND qa.userid = ".$userid."
AND state = 'finished'
ORDER BY qa.timefinish ASC";
$exams = $DB->get_records_sql($sql);
// calculate final grades from sum grades
$grades = array();
foreach($exams as $exam) {
$grade = new stdClass;
$grade->quiz = $exam->quiz;
$grade->attempt = $exam->attempt;
// sum to final
$grade->finalgrade = $exam->grade * ($exam->maxgrade / $exam->sumgrades);
$grade->grademax = $exam->maxgrade;
$grade->timemodified = $exam->timemodified;
array_push($grades, $grade);
}
This works in latest moodle version. Moodle 2.9. Although I am still open for better solution as this is really hacky way of getting deeper analytics about user's performance.

Delete read-only Zend row

What causes a row (Zend_Db_Table_Row) to be set to "readOnly?" I'm having trouble deleting rows in a loop:
// this is set to some integers
$ids = array();
// get the results
$results = $table->fetchAll($select);
foreach ($results as $result)
{
$value = $result->value;
if (!in_array($value, $ids))
{
// throws a "row is read-only" error
$result->delete();
}
}
Here's my select:
$table = $options->joinModel;
$select = $table->select();
$select->from($table->getTableName(), array("id", "value" => $options->joinForeignKey))
->where("`{$options->foreignKey}` = ?", $row->id)
->group($options->joinForeignKey);
I want to delete the row's that aren't in the $ids array, but it throws an error saying the row is read only. I haven't set that flag or done anything with the row. Any idea why it's read-only?
A row is readOnly if the $select is such that prevents you from directly mapping fields back to a single origin row.
For example, if the $select involves a JOIN or a GROUP BY, it's not clear which row(s) would be affected if you change a value of a field in the row object.
You might say "I know which row is the source, why can't Zend_Db_Table_Row tell?" But there are many corner cases, so it's a hard problem to solve in general.
Keep in mind all of Zend_Db is under 3000 lines of code. It can't have a lot of magic in it.
A row object can also be readOnly if you've serialized and then deserialized it.