I need to run a collection on Mongoid but this collection does not have a model, so i cant use it like: Project.collection.map_reduce( ..., :query => scoped.selector)
Does anyone know how could i do this?
Thanks
Here's an answer/example that works for Mongoid 3.1.5 / Moped 1.5.1, hope that it helps.
Please respond with your version info if you want something more specific to your environment.
Note that since you don't have a Mongoid model,
you loose the facilities of ODM level and have to drop down to the Moped driver level.
test/unit/session_test.rb
require 'test_helper'
require 'pp'
class SessionTest < ActiveSupport::TestCase
def setup
#session = Mongoid.default_session
#collection_name = 'project'
#collection = #session[#collection_name]
#collection.drop
end
test "collection map-reduce without model" do
puts "\nMongoid::VERSION:#{Mongoid::VERSION}\nMoped::VERSION:#{Moped::VERSION}"
docs = [
{'name' => 'Charlie', 'gender' => 'M', 'age' => 11},
{'name' => 'Lucy', 'gender' => 'F', 'age' => 13},
{'name' => 'Sally', 'gender' => 'F', 'age' => 15},
{'name' => 'Linus', 'gender' => 'M', 'age' => 11},
{'name' => 'Snoopy'},
]
#collection.insert(docs)
assert_equal docs.size, #collection.find.to_a.size
pp #session.command(
:mapReduce => #collection_name,
:map => 'function(){ emit(this.gender, this.age); }',
:reduce => 'function(key, values){ return Array.sum(values)/values.length; }',
:out => { :inline => 1 },
:query => { 'age' => { '$exists' => true } }
)['results']
end
end
$ rake test
Run options:
# Running tests:
[1/1] SessionTest#test_collection_map-reduce_without_model
Mongoid::VERSION:3.1.5
Moped::VERSION:1.5.1
[{"_id"=>"F", "value"=>14.0}, {"_id"=>"M", "value"=>11.0}]
Finished tests in 0.111151s, 8.9968 tests/s, 8.9968 assertions/s.
1 tests, 1 assertions, 0 failures, 0 errors, 0 skips
Related
I have a database with 50 million documents and each document has a object field which has dozens of other nested objects within. Basically this field is a XML file converted to an object. I was making some tests with aggregate() and find() but I noticed that projecting and filtering this field takes much performance loss.
For example
$cursor = $db->find(array(
'id_empresa' => 7,
'tipo_doc' => 'nfe',
'xml.NFe.infNFe.emit.xNome' => ['$regex' => 'auto', '$options' => 'i'],
'$and' => array(['$or' => array(
['xml.NFe.infNFe.emit.xNome' => ['$regex' => 'me', '$options' => 'i']],
['xml.NFe.infNFe.dest.xNome' => ['$regex' => 'me', '$options' => 'i']],
['xml.NFe.infNFe.transp.transporta.xNome' => ['$regex' => 'me', '$options' => 'i']]
)])
), array(
'limit' => 100,
'sort' => ['xml.NFe.infNFe.emit.xNome' => 1],
'projection' => array(
'id_empresa' => 1,
'tipo_doc' => 1,
'_id' => 1,
'xml.NFe.infNFe.emit.xNome' => 1,
'xml.NFe.infNFe.ide.nNF' => 1,
'xml.NFe.infNFe.dest.xNome' => 1,
'xml.NFe.infNFe.transp.transporta.xNome' => 1,
)
));
In this example there is only 4 projected fields of this object but in most cases it will be around 20~40 fields.
Is there any way to get better performance when working with those nested objects?
How to use aggregate and $lookup with php 7 mongo driver manager.
How to convert the following mongo command to php
db.a.aggregate([{$lookup:{from:"b",localField:"business_id",foreignField:"_id",as:"contact"}}])
Also recommend good reference tutorials.
This code worked
$mng = new MongoDB\Driver\Manager("mongodb://localhost:27017");
$command = new MongoDB\Driver\Command([
'aggregate' => 'a',
'pipeline' => [
['$lookup' => ["from" => "b","localField" => "business_id","foreignField" => "_id","as" => "contact"]],
],
]);
$cursor = $mng->executeCommand('test', $command);
Try something like this:
$mongo->db->a->aggregate([
['$lookup' => ['from' => 'b', 'localField' => 'business_id', 'foreignField' => '_id', 'as' => 'contact'] ]
]);
PHP7.2
MongoDB 4.2
I took reference from #Jibin Mathew' answer but I got some errors.
then I added
'cursor' => new stdClass,
this line in code.
Now it's working properly.
//mongo database join example
public function test_join() {
global $mng;
global $dbname;
$pipeline = [['$lookup' => ["from" => "b","localField" => "business_id","foreignField" => "_id","as" => "contact"]]];
$aggregate = new \MongoDB\Driver\Command([
'aggregate' => 'a',
'cursor' => new stdClass,
'pipeline' => $pipeline,
]);
$cursor = $mng->executeCommand($dbname, $aggregate);
return $cursor;
}
I've got several calls working already, but for the life of me I can't figure out how to make product_attribute.create work. I'm always getting a 102 Invalid request parameters or 623 Wrong Method Signature.
making the call like this my $res = $self->_useragent->call( call => $self->_session, #{$payload} ); (note: useragent is a XML::RPC object.
This Dumper $payload;
$VAR1 = [
'product_attribute.create',
[
'test',
{
'frontend_label' => [
{
'label' => 'Test ME',
'store_id' => 0
}
],
'scope' => 'store',
'frontend_input' => 'text'
}
]
];
I've read the API Documentation but figuring out what the call should look like in Perl is tricky.
I'm not familiar with the XML-RPC library you're using in perl, but he error you're seeing is a Magento API exception, configured in
<!--File: app/code/core/Mage/Catalog/etc/api.xml -->
<!-- ... -->
<invalid_parameters>
<code>102</code>
<message>Invalid request parameters.</message>
</invalid_parameters>
<!-- ... -->
Using the exception's name, you can find the place Magento threw it
#File: app/code/core/Mage/Catalog/Model/Product/Attribute/Api.php
//...
if (empty($data['attribute_code']) || !is_array($data['frontend_label'])) {
$this->_fault('invalid_parameters');
}
//...
So, my guess is your call is correct, you're just missing an attribute_code.
After some digging through Magento's code, I copied this from the test suite and it converted to perl, it appears to work. maybe all attributes are required.
$VAR1 = [
'product_attribute.create',
[
{
'default_value' => '1',
'is_configurable' => 0,
'used_in_product_listing' => 0,
'is_visible_on_front' => 0,
'apply_to' => [
'simple'
],
'is_comparable' => 0,
'is_used_for_promo_rules' => 0,
'is_required' => 0,
'scope' => 'store',
'is_unique' => 0,
'frontend_input' => 'text',
'is_searchable' => 0,
'attribute_code' => 'unique_code',
'is_visible_in_advanced_search' => 0,
'frontend_label' => [
{
'label' => 'some label',
'store_id' => '0'
}
]
}
]
];
Further experimentation somewhat based on Alan Storm's Answer, suggest that the following fields are required, as I was not able to successfully create a request without all of these fields at minimum being defined.
$VAR1 = [
'product_attribute.create',
[
{
'frontend_input' => 'text',
'attribute_code' => 'test1374438470',
'frontend_label' => [
{
'store_id' => 0,
'label' => 'Test ME'
}
]
}
]
];
I'm using MongoDB through Mongoid with Rails 3 and observe this strange behavior when doing query in rails console:
> Table.where(:field => {"$exists" => true}).count
=> 3735
> Table.where(:field => {"$exists" => true}, :field => {"$ne" => ""}).count
=> 14878 # wtf???
> Table.where(:field => {"$exists" => true}, :field => "").count
=> 0 # at least it's not negative
> Table.where(:field => {"$exists" => false}).count
=> 11143
Since 11143 + 3735 = 14878, I assume that where(:field => {"$exists" => true}, :field => {"$ne" => ""}) also counts those records in which :field is not present (because nil != ""?). However, I believed conditions listed in #where would be joined with and, so it should match only those records where :field is not empty string AND is present.
You say "However, I believed conditions listed in #where would be joined with 'and'," but this is not correct. The conditions are a hash, and you have a collision on the key :field. Ruby silently uses the last value.
Please review the documentation for selection in Mongoid http://mongoid.org/en/origin/docs/selection.html, and use #and for a proper 'and' conjunction. Note that you can #inspect your query and examine the returned Criteria object. For example:
puts Table.where(:field => {"$exists" => true}, :field => {"$ne" => ""}).inspect
Hope that this helps.
Hi everyone,
This is very simple for perl programmers but not beginners like me,
I have one xml file and I processed using XML::Simple like this
my $file="service.xml";
my $xml = new XML::Simple;
my $data = $xml->XMLin("$file", ForceArray => ['Service','SystemReaction',
'Customers', 'Suppliers','SW','HW'],);
Dumping out $data, it looks like this:
$data = {
'Service' => [{
'Suppliers' => [{
'SW' => [
{'Path' => '/work/service.xml', 'Service' => 'b7a'},
{'Path' => '/work/service1.xml', 'Service' => 'b7b'},
{'Path' => '/work/service2.xml', 'Service' => 'b5'}]}
],
'Id' => 'SKRM',
'Customers' =>
[{'SW' => [{'Path' => '/work/service.xml', 'Service' => 'ASOC'}]}],
'Des' => 'Control the current through the pipe',
'Name' => ' Control unit'
},
{
'Suppliers' => [{
'HW' => [{
'Type' => 'W',
'Path' => '/work/hardware.xml',
'Nr' => '18',
'Service' => '1'
},
{
'Type' => 'B',
'Path' => '/work/hardware.xml',
'Nr' => '7',
'Service' => '1'
},
{
'Type' => 'k',
'Path' => '/work/hardware.xml',
'Nr' => '1',
'Service' => '1'
}]}
],
'Id' => 'ADTM',
'Customers' =>
[{'SW' => [{'Path' => '/work/service.xml', 'Service' => 'SDCR'}]}],
'Des' => 'It delivers actual motor speed',
'Name' => ' Motor Drivers and Diognostics'
},
# etc.
],
'Systemreaction' => [
# etc.
],
};
How to access each elements in the service and systemReaction(not provided). because I am using "$data" in further processing. So I need to access each Id,customers, suppliers values in each service. How to get particular value from service to do some process with that value.for example I need to get all Id values form service and create nodes for each id values.
To get Type and Nr value I tried like this
foreach my $service (#{ $data->{Service}[1]{Suppliers}[0]{HW}[0] }) {
say $service->{Nr};
}
foreach my $service (#{ $data->{Service}[1]{Suppliers}[0]{HW}[0] }) {
say $service->{Type};
}
can you help me how to get all Nr and Type values from Supplier->HW.
I suggest reading perldocs Reference Tutorial and References and Nested Data Structures. They contain an introduction and full explanation of how to access data like that.
But, for example, you can access the service ID by doing:
say $data->{Service}[0]{Id} # prints SKRM
You could go through all the services, printing their ID, with a loop:
foreach my $service (#{ $data->{Service} }) {
say $service->{Id};
}
In response to your edit
$data->{Service}[1]{Suppliers}[0]{HW}[0] is an hash reference (you can check this quickly by either using Data::Dumper or Data::Dump on it, or just the ref function). In particular, it is { Nr => 18, Path => "/work/hardware.xml", Service => 1, Type => "W" }
In other words, you've almost got it—you just went one level too deep. It should be:
foreach my $service (#{ $data->{Service}[1]{Suppliers}[0]{HW} }) {
say $service->{Nr};
}
Note the lack of the final [0] that you had.