I have a query which looks up data in a huge collection (over 48M), yet even if I add timeout=-1 to it, it still throws MongoCursorTimeoutException exception..
return \DB::connection('mongodb')->collection('stats')->timeout(-1)
->where('ip','=',$alias)
->where('created_at','>=', new \DateTime( $date ) )
->where('created_at','<=', new \DateTime( $date . ' 23:59:59' ) )
->count();
I am using this library: https://github.com/jenssegers/laravel-mongodb
Any ideas?
There is an issue PHP-1249 - MongoCursor::count() should use cursor's socket timeout submitted for PHP MongoDB driver v1.5.7 which was fixed in v1.5.8 in October, 2014.
The reply from the support:
Looking into the code a bit, it appears that both socket timeout and maxTimeMS is not passed along to the count command.
If you need an immediate work-around, you should be able to get by with MongoDB::command() for now (which can support both timeouts).
The workaround posted by one of the users is:
$countComand = $mongo->command(
array(
'count' => 'collection',
'query' => $query
),
array('socketTimeoutMS' => -1)
);
if($countComand['ok']){
$count = $countComand['n'];
} else {
// ... error ...
}
It seems that laravel-mongodb doesn't use MongoDB::command(). You either have to write your query explicitly without help of where methods as shown above or upgrade to v.1.5.8.
Related
I have a Perl script that pulls a table from a SQL database ($row variable) and attempts to do a MongoDB update like so:
my $res = $users->update({"meeting_id" => $row[0]},
{'$set' => {
"meeting_id" => $row[0],
"case_id" => $row[1],
"case_desc" => $row[2],
"date" => $row[3],
"start_time" => $row[4],
"end_time" => $row[5],
#"mediator_LawyerID" => $row[6],
"mediator_LawyerIDs" => \#medLawIds,
"case_number" => $row[6],
"case_name" => $row[7],
"location" => $row[8],
"number_of_parties" => $row[9],
"case_manager" => $row[10],
"last_updated" => $row[11],
"meeting_result" => $row[12],
"parties" => \#partyList
}},
{'upsert' => 1}) or die "I ain't update!!!";
My client now wants ICS style calendar invites sent to their mediators. Thus, I need to know whether an update or insert happened. The documentation for MongoDB::UpdateResult implies that this is how you access such a property:
my $id = $res->upserted_id;
So I tried:
bless ($res,"MongoDB::UpdateResult");
my $id = $res->upserted_id;
After this code $id is like:
HASH(0x356f8fc)
Are these the actual IDs? If so, how do I convert to a hexadecimal string that can be cast to Mongo's ObjectId type? It should be noted I know absolutely nothing about perl; if more of the code is relevant, at request I will post any section ASAP. Its 300 lines so I didn't want to include the whole file off the bat.
EDIT: I should mention before anyone suggests this that using update_one instead of update returns the exact same result.
HASH(0x356f8fc) is a Perl Hash reference. It's basically some kind of (internal) memory address of some data.
The easiest way to get the contents is Data::Dumper:
use Data::Dumper
[...]
my $result = $res->upserted_id;
print Dumper($result);
HASH(0x356f8fc) is just the human readable representation of the real pointer. You must dump it in the same process and can't pass it from one to another.
You'll probably end up with something like
`my $id = $result->{_id};`
See the PerlRef manpage for details.
See also the MongoDB documentation about write concern.
PS: Also remember that you could use your own IDs for MongoDB. You don't need to work with the generated ones.
I'm trying to work out how I can use WWW::Mailchimp ( http://search.cpan.org/~arcanez/WWW-Mailchimp/ ) to sign someone up to our list, but also assign the language of the person (i.e english, french, german, spanish, etc).
Here is what I have thus far:
my $mailchimp = WWW::Mailchimp->new(apikey => 'xxxx' );
$mailchimp->listSubscribe( id => "xxx", email_address => $in->{Email}, merge_vars => [ FNAME => $name[0], LNAME => $name[1], mc_language => "fr", LANG => "fr", LANGUAGE => "fr" ] );
mc_language => "fr", LANG => "fr", LANGUAGE => "fr" doesn't seem to do anything (been trying all the params I see laying around, in the vain hope one of them works!)
While it works (and asks you to confirm your subscription), all the language variables are ignored. Looking at their documents, I'm a bit confused as to what to use:
https://apidocs.mailchimp.com/api/2.0/lists/subscribe.php
The code "fr" is ok, but I'm unsure what params to pass along to it.
Has anyone had any experience with this before? Apart from the language, it works fine (but I need to be able to send the confirmation emails in their own language, and then also filter down when doing mailings)
UPDATE: Ok, so it looks like its not going to be a simple case of updating to the newer API. I've been looking into the v3.0 API, and its a total overhaul of the older one (new function names, new ways of sending requests, etc). What I'm going to do is look into a "Curl" method, so we can at least get it going with that. Once I've got that going, I'll probably have a look at coding something to work with LWP::UserAgent, as that'd be cleaner than doing lots of curl requests. Shame there isn't anything out there already for Perl and MailChimp (with the new API, or even version 2.0!)
From looking at the source, it defaults to API 1.3:
has api_version => (
is => 'ro',
isa => Num,
lazy => 1,
default => sub { 1.3 },
);
The documentation for that shows you need to use MC_LANGUAGE:
string MC_LANGUAGE Set the member's language preference. Supported
codes are fully case-sensitive and can be found here.
It looks like the module just shoves whatever data structure you provide into JSON and POSTs it to Mailchimp, so the appropriate Mailchimp API doc version for the API you target should be referenced as a primary source.
Ok, so I got there in the end! I have been talking with MailChimp support, and they were very helpful. Turns out it was a double issue.
1) Auto-Translate needed to be enabled for the list in question. This was their answer around that:
After taking a look at the call, it appears to be set up properly now, so you are all good on that front. That being said, I am seeing
that the Auto-translate option doesn't seem to be enabled for any of
your lists. In order for the Confirmation and all other response
emails to automatically translate, this will need to be enabled for
all of the lists being used.
We have a bit of additional information on that, here, if you'd like to check that out:
http://kb.mailchimp.com/lists/signup-forms/translate-signup-forms-and-emails#Auto-Translate-Forms
2) When making the request via the API, you need to specifically set the Accept-Language: xx value. For example, en, fr, es, de, etc.
Here is a working function for anyone who needs it in the future. Just be sure to update the apikey,listId and endpoint URL.
do_register_email_list('foo#bar.com','Andrew Test',"en")
sub do_register_email_list {
# (email,name,lang)
use WWW::Curl::Easy;
use Digest::MD5;
use JSON;
my #name = split /\s+/, $_[1];
my $apikey = 'xxxx-us6';
my $listid = 'xxxx';
my $email = $_[0];
my $endpoint = "https://us6.api.mailchimp.com/3.0/lists";
my $lang = $_[2]||'en';
my $json = JSON::encode_json({
'email_address' => $email,
'status' => 'pending',
'language' => $lang,
'merge_fields' => {
'FNAME' => $name[0]||'',
'LNAME' => $name[1]||''
}
});
my $curl = WWW::Curl::Easy->new;
my $url = "$endpoint/$listid/members/" . Digest::MD5::md5(lc($email));
$curl->setopt(CURLOPT_HEADER,1);
$curl->setopt(CURLOPT_URL, $url);
# $curl->setopt(CURLOPT_VERBOSE, 1);
$curl->setopt(CURLOPT_USERPWD, 'user:' . $apikey);
$curl->setopt(CURLOPT_HTTPHEADER, ['Content-Type: application/json',"Accept-Language: $lang"]);
$curl->setopt(CURLOPT_TIMEOUT, 10);
$curl->setopt(CURLOPT_CUSTOMREQUEST, 'PUT');
$curl->setopt(CURLOPT_SSL_VERIFYPEER, 0);
$curl->setopt(CURLOPT_POSTFIELDS, $json);
# A filehandle, reference to a scalar or reference to a typeglob can be used here.
my $response_body;
$curl->setopt(CURLOPT_WRITEDATA,\$response_body);
# Starts the actual request
my $retcode = $curl->perform;
#print "FOO HERE";
# Looking at the results...
if ($retcode == 0) {
print "Transfer went ok\n";
my $response_code = $curl->getinfo(CURLINFO_HTTP_CODE);
print "Received response: $response_body\n";
} else {
# Error code, type of error, error message
print "An error happened: $retcode ".$curl->strerror($retcode)." ".$curl->errbuf."\n";
}
}
Hopefully this saves someone else from all the grief I had with it :) (the MailChimp support lady also said she will get their team to add something about this in the developer notes, so its made a bit clearer!)
I cant see my error of Sql. It cuts my queries, and the error is meaningless except it informs there is an error.
Like
error in statement "select * from o.."
how can I get full query, so i can investigate how the error occured?
I previously wrote a sql function that throws debugging string if sql has error. I though zend'ers have needed and there should exists a code:
if(($error = mysql_error($conn)){
$cagiri=debug_backtrace();
$i=count($cagiri);
And says
[Caller__Function__] => dbSave
[Caller__Class__] => classBasic
[Arguments] => Array
(
[0] => function : loadLinks
[1] =>
[1] => sql error: Duplicate entry 'http://www.istanbulboncugu.com/Lokma' for key 'url'
[2] => query: insert into downloadLinks set `title`= 'Lokma : İstanbul - Avrupa', `url`= 'http://www.istanbulboncugu.com/Lokma', `site`= 'rssSehirFirsati', `status`= 'new'
So I dont dig 30 query, if there is error,
Error string shows function that composed the query. Etc.
I think Zend have lacks that kind of awareness.
I know zend not prefers hard and log path against smart ways ?!?
I start using that code, if there is error.
function results($q){
$results = Array();
try{
$results = $this->_db->query($q)->fetchAll();
}catch(Exception $e){
var_dump($q);
die($e->getMessage());
}
return $results;
}
If you are using Zend_Db_Select you can get the full created query by doing
$select->__toString();
In reply to your comment, this is how you can get the SQL error message:
try {
$this->db->insert('Users', $array );
} catch (Exception $e){
echo $e->getMessage();
}
When using the Perl module Net::Cassandra::Easy to interface with Cassandra I use the following code to read colums col[123] from rows row[123] in the column-family Standard1:
my $cassandra = Net::Cassandra::Easy->new(keyspace => 'Keyspace1', server => 'localhost');
$cassandra->connect();
my $result = $cassandra->get(['row1', 'row2', 'row3'], family => 'Standard1', byname => ['col1', 'col2', 'col3']);
This works as expected.
However, when trying to insert row row1 with ..
$result = $cassandra->mutate(['row1'], family => 'Standard1', insertions => { "col1" => "Value to set." });
.. I get the error message Can't use string ("0") as a SCALAR ref while "strict refs" in use at .../Net/GenThrift/Thrift/BinaryProtocol.pm line 376.
What am I doing wrong?
It looks like a bug in the library:
sub readByte
{
my $self = shift;
my $value = shift;
my $data = $self->{trans}->readAll(1);
my #arr = unpack('c', $data);
$$value = $arr[0]; # <~ line 376
return 1;
}
(from Net::GenThrift::Thrift::BinaryProtocol)
Apparently that sub is being called from somewhere in the library where $value is not a variable, but a constant scalar. I'd report the bug to the authors.
The code works as expected under Cassandra 0.6.x, but fails under Cassandra 0.5.x.
It appears as if Net::Cassandra::Easy is targeting Cassandra 0.6.x only.
Upgrading to Cassandra 0.6.x solves the problem.
hmm, it looks more like a Perl binding bug when handling exception to me.
I believe that 0.6 fixes it for you because the interface has indeed changed, so 0.6 is not raising a thrift exception anymore, but the bug in thrift remains. I've opened a JIRA case, we'll see that thrift team says about it:
https://issues.apache.org/jira/browse/THRIFT-758
Following the sample code on http://framework.zend.com/manual/en/zend.db.profiler.html I have set up db profiling of my Zend Framework app.
application.ini:
db.profiler.enabled = true
View Helper:
$totalTime = $profiler->getTotalElapsedSecs();
$queryCount = $profiler->getTotalNumQueries();
$longestTime = 0;
$longestQuery = null;
foreach ($profiler->getQueryProfiles() as $query) {
if ($query->getElapsedSecs() > $longestTime) {
$longestTime = $query->getElapsedSecs();
$longestQuery = $query->getQuery();
}
}
echo 'Executed ' . $queryCount . ' queries in ' . $totalTime . ' seconds' . "\n";
echo 'Average query length: ' . $totalTime / $queryCount . ' seconds' . "\n";
echo 'Queries per second: ' . $queryCount / $totalTime . "\n";
echo 'Longest query length: ' . $longestTime . "\n";
echo "Longest query: \n" . $longestQuery . "\n";
It works fine for select/insert/update/delete queries.
But I cannot find anyway to get the profiler to show the time taken to initiate the actual db connection, despite the documenation implying that it does log this.
I suspect that Zend_Db simply does not log the connection to the db with the profiler.
Does anyone know what is going on here?
I am using the Oracle database adapter, and ZF 1.10.1
UPDATE:
I understand it is possible to filter the profiler output, such that it will only show certain query types, e.g. select/insert/update. There also appears to be an option to filter just the connection records:
$profiler->setFilterQueryType(Zend_Db_Profiler::CONNECT);
However, my problem is that the profiler is not logging the connections to begin with, so this filter does nothing.
I know this for a fact, because if I print the profiler object, it contains data for many different queries - but no data for the connection queries:
print_r($profiler);
//output
Zend_Db_Profiler Object
(
[_queryProfiles:protected] => Array
(
[0] => Zend_Db_Profiler_Query Object
(
[_query:protected] => select * from table1
[_queryType:protected] => 32
[_startedMicrotime:protected] => 1268104035.3465
[_endedMicrotime:protected] => 1268104035.3855
[_boundParams:protected] => Array
(
)
)
[1] => Zend_Db_Profiler_Query Object
(
[_query:protected] => select * from table2
[_queryType:protected] => 32
[_startedMicrotime:protected] => 1268104035.3882
[_endedMicrotime:protected] => 1268104035.419
[_boundParams:protected] => Array
(
)
)
)
[_enabled:protected] => 1
[_filterElapsedSecs:protected] =>
[_filterTypes:protected] =>
)
Am I doing something wrong - or has logging of connections just not been added to Zend Framework yet?
The profiler 'bundles' connection and other operations in with the general queries.
There's three ways you might examine the connection specifically:
Set a filter on the profiler during setup:
$profiler->setFilterQueryType(**Zend_Db_Profiler::CONNECT**);
Then the resultant profiles will only include 'connect' operations.
Specify a query type when you retrieve the Query Profiles:
$profiles = $profiler->getQueryProfiles(**Zend_Db_Profiler::CONNECT**);
Examine the query objects directly during the iteration:
foreach($profiler->getQueryProfiles() as $query) {
if ($query->getQueryType() == Zend_Db_Profiler::CONNECT &&
$query->getElapsedSecs() > $longestConnectionTime) {
$longestConnectionTime = $query->getElapsedSecs();
}
}
You'll not find great detail in there, it's logged just as a 'connect' operation along with the time taken.