In my Event model, I have the following function to retrieve all the events with status = 1 with a 12 limit and order according to event created DESC:
public function latestEvents() {
$this->Behaviors->load('Containable');
$result = $this->find('all' ,array('recursive' => -1, 'conditions'=> array('Event.status' => 1), 'limit' => 12, 'order' => array('Event.created DESC')));
debug($result); die();
return $result;
}
This function is not returning any data. When I change my limit to 6 and debug it returns six records but when I change my limit to more than 6 it returns (empty) this :
I even checked in my database by doing this query :
SELECT * FROM `events` WHERE `status` = 1 ORDER BY `created` DESC LIMIT 12
and this returns the desired data that I want. I even tried :
$result = $this->query('SELECT * FROM `events` WHERE `status` = 1 ORDER BY `created` DESC LIMIT 12');
but the same thing is happening with the limit (6 returns the data but more than 6 does not).
I found out that the data I was trying to debug had special characters and I had to include 'encoding' => 'utf8' in my database.php and worked like a charm. This post helped me.
Related
I need make search on billion records in MySQL and it's very long process (it's works now). May be Sphinx help me? How correctly to configure Sphinx for search numbers? Should I use integer attribute for searching (not string field)?
I need to get only row where the timestamp 'nearest or equal' to query:
CREATE TABLE test ( date TIMESTAMP(6) UNIQUE, num INT(32) );
| 2018-07-02 05:50:33.084011 | 282 |
| 2018-07-02 05:50:33.084028 | 475 |
...
(40 M such rows... all timestamps is unique, so this column are unique index so I no need in create additional index I suppose.)
sphinx.conf:
source src1
{
type = mysql
...
sql_query = SELECT * FROM test
}
indexer...
Sphinx 3.0.3
...
indexing index 'test'...
collected 40000000 docs, 0.0 MB
In my test I find nearest timestamp to query:
$start = microtime(true);
$query = '2018-07-02 05:50:33.084011';
$connMySQL = new PDO('mysql:host=localhost;dbname=test','','');
$sql = "SELECT * FROM test WHERE date <= '$search' ORDER BY date DESC LIMIT 1";
$que = $connMySQL->query($sql);
$result = $que->fetchAll(PDO::FETCH_ASSOC);
$query = $connMySQL->query('reset query cache');
$connMySQL = null;
print_r ($result);
echo 'Time MySQL:'.(microtime(true) - $start).' sec.';
$start = microtime(true);
$query = '2018-07-02 05:50:33.084029';
$connSphinxQL = new PDO('mysql:host=localhost;port=9306;dbname=test','root','');
$sql = "SELECT * FROM test WHERE date <= '$search' ORDER BY date DESC LIMIT 1";
$que = $connSphinxQL->query($sql);
$result = $que->fetchAll(PDO::FETCH_ASSOC);
$query = $connSphinxQL->query('reset query cache');
$connSphinxQL = null;
print_r ($result);
echo 'Time Sphinx:'.(microtime(true) - $start).' sec.';
Output:
[date] => 2018-07-02 05:50:33.084011 [num] => 282 Time MySQL: 0.00193 sec.
[date] => 2018-07-02 05:50:33.084028 [num] => 475 Time Sphinx: 0.00184 sec.
I suggested to see some different resuts, but noticed that before indexing I have got the same result, so I think Sphinx searches directy in MySQL by the reason of my wrong configuration.
Only ask here I found: no text search
Should I use integer attribute for searching (not string field)?
Yes. But an added complication, is a index NEEDS at least one field (sphinx isnt really designed as a general database, its intended for text queries!)
Can synthesize a fake one.
sql_query = SELECT unix_timestamp(`date`) AS id, 'a' AS field, num FROM test
sql_attr_uint = num
Also shows that need a unique integer as the first column, to be a document_id, seems as your timestamp is unique, can use that. a UNIX_TIMESTAMP is a nice easy way to represent a timestamp as a plain integer.
Can use id in queries too, for filtering, so would need to convert to a timestamp at the same time.
$query = '2018-07-02 05:50:33.084011';
$id = strtotime($query)
$sql = "SELECT * FROM test WHERE id <= '$id' ORDER BY id DESC LIMIT 1";
In my controller, I have the following code:
$threads = MessageThread::whereExists(function($query) {
$messid = '`' . env('DB_PREFIX') . 'message_threads`.`id`';
$query->select(\DB::raw(1))
->from('message_thread_participants')
->where('message_thread_participants.message_thread_id', '=', $messid); // AND message_thread_participants.user_id = ' . $user_id);
})
->orderBy('message_threads.created_at', 'DESC')
->take(30)
->get();
If I print out the resulting query using DB::getQueryLog() I get the following:
[ {
"query": "select * from bd45_message_threads where exists (select 1 from bd45_message_thread_participants where
bd45_message_thread_participants.message_thread_id = ?) order by
bd45_message_threads.created_at desc limit 30",
"bindings": [
"bd45_message_threads.id"
],
"time": 1.06 } ]
And I run that query manually (substituting the binding for the ?, I get the expected 30 results..... but the code itself returns no results... I assume it's some problem between using Models and Where Exists... but it's just a guess right now.... anyone have any clue why the query would return results in Navicat or PHPMyAdmin, but the controller returns a blank array?
I'm a little late to the party, but in case someone has the same problem, it must be beacause you have to use
->whereColumn('bd45_message_thread_participants.message_thread_id', 'bd45_message_threads.id')
to compare two columns values. The ->where expects to compare a column and a value, not another column.
In C# code with EF6 and Sql Server, my goal is to use this query :
Select MAX(columnA) from myTable WHERE columnB>5 AND ColumnC=1
by using C# code.
Example :
SELECT Max(ColumnA) from myTable
becomes :
int max = DbContext.myTable.Max(t => t.ColumnA); => works fine, OK
But do you know how to add the where clause into this C# code ???
Erixx
You put Where first and then Max later like this
int max = DbContext.myTable.Where(it=>it.columnB>5 && it.ColumnC=1).Max(t => t.ColumnA);
Just add the Where before Max (or after, depending on your logic).
int max = DbContext.myTable.Where(t => t.ColumnB > 5 && ColumnC == 1).Max(t => t.ColumnA)
I have tables with articles and users, both have many-to-many mapping to third table - reads.
What I am trying to do here is to get all unread articles for particular user ( user_id not present in table reads ).
My query is getting all articles but those read are marked, which if fine as I can filter them out (user_id field contains id of user in question).
I have an SQL query like this:
SELECT articles.id, reads.user_id
FROM articles
LEFT JOIN
reads
ON articles.id = reads.article_id AND reads.user_id = 9
ORDER BY articles.last_update DESC LIMIT 5;
Which yields following:
articles.id | reads.user_id
-------------------+-----------------
57125839 | 9
57065456 |
56945065 |
56945066 |
56763090 |
(5 rows)
This is fine. This is what I want.
I'd like to get same result in Catalyst using my article model, but I cannot find any option to add conditions to a JOIN clause.
Do you know any way how to add AND X = Y to DBIx JOIN?
I know this can be done with custom resoult source and virtual view, but I have some other queries that could benefit from it and I'd like to avoid creating virtual view for each of them.
Thanks,
Canto
I don't even know what Catalyst is but I can hack the SQL query:
select articles.id, reads.user_id
from
articles
left join
(
select *
from reads
where user_id = 9
) reads on articles.id = reads.article_id
order by articles.last_update desc
limit 5;
I got an solution.
It's not straight forward, but it's better than virtual view.
http://search.cpan.org/dist/DBIx-Class/lib/DBIx/Class/Relationship/Base.pm#condition
Above describes how to use conditions in JOIN clause.
However, my case needs an variable in those conditions, which is not available by default in model.
So getting around a bit of model concept and introducing variable to it, we have the following.
In model file
our $USER_ID;
__PACKAGE__->has_many(
pindols => "My::MyDB::Result::Read",
sub {
my $args = shift;
die "no user_id specified!" unless $USER_ID;
return ({
"$args->{self_alias}.id" => { -ident => "$args->{foreign_alias}.article_id" },
"$args->{foreign_alias}.user_id" => { -ident => $USER_ID },
});
}
);
in controller
$My::MyDB::Result::Article::USER_ID = $c->user->id;
$articles = $channel->search(
{ "pindols.user_id" => undef } ,
{
page => int($page),
rows => 20,
order_by => 'last_update DESC',
prefetch => "pindols"
}
);
Will fetch all unread articles and yield following SQL.
SELECT me.id, me.url, me.title, me.content, me.last_update, me.author, me.thumbnail, pindols.article_id, pindols.user_id FROM (SELECT me.id, me.url, me.title, me.content, me.last_update, me.author, me.thumbnail FROM articles me LEFT JOIN reads pindols ON ( me.id = pindols.article_id AND pindols.user_id = 9 ) WHERE ( pindols.user_id IS NULL ) GROUP BY me.id, me.url, me.title, me.content, me.last_update, me.author, me.thumbnail ORDER BY last_update DESC LIMIT ?) me LEFT JOIN reads pindols ON ( me.id = pindols.article_id AND pindols.user_id = 9 ) WHERE ( pindols.user_id IS NULL ) ORDER BY last_update DESC: '20'
Of course you can skip the paging but I had it in my code so I included it here.
Special thanks goes to deg from #dbix-class on irc.perl.org and https://blog.afoolishmanifesto.com/posts/dbix-class-parameterized-relationships/.
Thanks,
Canto
I am developing an application in MongoDB, CakePHP. I have 145,000 record in my database. When I query for records using following command then it takes 12seconds. Which is very bad for my application.
$params= array('aggregate'=>array(
array('$project'=>array('as'=>1,'pid'=>1,'st'=>1)),
array('$unwind'=>'$as'),
array('$match' => array('pd'=>array('$gt'=>$f,'$lt'=>$t),'pid'=>$project_id)),
array('$group'=>array('_id'=>'$as')),
array('$sort'=>array('_id'=>1)),
array('$limit'=>10)
)
);
$results = $this->Detail->find('all',array('conditions'=>$params));
Can anyone help me in reducing time for query.
I have indexes on as & pid. My system RAM is 1.5GB.
I got following data as result
[1] => Array
(
[Detail] => Array
(
[_id] => Array
(
[0] => "superfone" Llc,moscow,ru
)
)
)
[2] => Array
(
[Detail] => Array
(
[_id] => Array
(
[0] => "superphone" Llc,moscow,ru
)
)
)
Before performing the find, make sure to set the recursive level to the level you need, then report back on your performance.
$this->Detail->recursive = -1; // no joins
results = $this->Detail->find('all',array('conditions'=>$params));
or
$this->Detail->recursive = 0; // data + domain
results = $this->Detail->find('all',array('conditions'=>$params));
Not sure what level of recursive you need, so give them a test, but default is 1 and it can be a costly exercise.
ref
http://book.cakephp.org/2.0/en/models/model-attributes.html#recursive
The reference also says to make -1 your default, and raise the recursive levels as you need them, this can be done by adding the following to AppModel
public $recursive = -1;