Integrate MONGODB and SOLR - mongodb

Question

I have tried to integrate MONGODB and SOLR by using MONGO CONNECTOR provided by mongodb which is running in a replica set configuration.
**python2.7 mongo_connector.py -m localhost:27017 -t http://localhost:8983/solr -u_id -d ./doc_managers/solr_doc_manager.py**
My output is
2013-06-19 16:19:10,943 - INFO - Finished 'http://localhost:8983/solr/update/?commit=true' (post) with body 'u'<commit ' in 0.012 seconds.
But I could not configure SOLR to get the documents from MONGODB. Please help me how to configure SOLR to get the documents from MONGODB. Should I use a SolrMongoImporter ?

Answer 1

I was facing the same problem. I was not able to solved so I found an interesting link:
http://derickrethans.nl/mongodb-and-solr.html
It connect mongo with solr through a php script. A

Answer 2

Step 1: Installing the Mongo Connector
To install the mongo connector run
Pip install mongo-connector
Step 2: Creating a Solr Core
./bin/solr create -c <corename>-p 8983 -s 3 -rf 3
Step 3:Configuring to Solr
The fields in the mongodb documents to be indexed are specified in the schema.xml configuration file . open the schema.xml in a vi editor. As
vi/solr/solr-6.6.2/server/solr/configsets/data_driven_schema_configs/
conf/schema.xml
Step 4: Mongo Connector also stores the metadata associated with the each mongodb document it indexes in fields ns and _ts. Also add the ns and _ts fields to the schema.xml.
<schema>
<?xml version="1.0" encoding="UTF-8" ?>
<schema name="example" version="1.5">
<field name="time_stamp" type="string" indexed="true"  stored="true" 
multiValued="false" />
<field name="category" type="string" indexed="true"  stored="true" 
multiValued="false" />
<field name="type" type="string" indexed="true"  stored="true" 
multiValued="false" />
<field name="servername" type="string" indexed="true"  stored="true" 
multiValued="false" />
<field name="code" type="string" indexed="true"  stored="true" 
multiValued="false" />
<field name="msg" type="string" indexed="true"  stored="true" 
multiValued="false" />
<field name="_ts" type="long" indexed="true" stored="true" />
<field name="ns" type="string" indexed="true" stored="true"/>
<field name="_version_" type="long" indexed="true" stored="true"/>
</schema>
Step 5:We also need to configure the
org.apache.solr.handler.admin.LukeRequestHandler request handler in the solrconfig.xml. 
Open the solrconfig.xml in the vi editor.
vi ./solr-5.3.1/server/solr/configsets/basic_configs/conf/solrconfig.xml
Specify the request handler for the Mongo Connector.
*<requestHandler name="/admin/luke"
class="org.apache.solr.handler.admin.LukeRequestHandler" />*
Also configure the auto commit to true so that Solr auto commits the data from MongoDB after the configured time.
<autoCommit>
<maxTime>15000</maxTime>
<openSearcher>true</openSearcher>
</autoCommit>
Step 6:Solr needs to be restarted
Bin/solr restart -force
Starting MongoDB Server
Mongo Connector requires a MongoDB replica set to be running for it to index MongoDB data in Solr. A replica set is a cluster of MongoDB servers that implements replication and automatic failover. A replica set could consist of only a single server and  the port is specified as 27017, the data directory for MongoDB is specified as /data/db and the replica set name is specified as rs0 with the –replSet option.
Sudo mongod --port 27017 --dbpath /data/db --replSet rs0
Step 7: Starting MongoDB Shell
Start the Mongodb shell with the following command
Mongo
MongoDB shell gets started. We need to initiate the replica set. Run the following command to initiate the replica set.
rs.initiate()
Step 8:Starting MongoDB Connector and Indexing MongoDB Database with Solr
Run Mongo-connector command as below
mongo-connector --unique-key=id –n solr.wlslog -m localhost:27017 -t
http://xx.xxx.xxx.xx:8983/solr/wlslog -d solr_doc_manager
In above statement
solr.wlslog--> solr is a databasename wlslog is a collection name
Solr/wlslog--> wlslog is a CORE name
For Future reference use the below link
https://www.toadworld.com/platforms/nosql/b/weblog/archive/2017/02/03/indexing-mongodb-data-in-apache-solr