Steps to connect MongoDB and Solr using DataImportHandler - mongodb

Question

I am new to SOLR and MONGODB.
I am trying to index data from mongodb into SOLR using DataImportHandler but I could not find the exact steps that I need to follow.
Could you please help me in getting the exact steps to index MongoDB into Solr using DataImportHandler?
SolrVersion - solr-4.6.0
MongoDB version- 2.2.7

Answer 1

Late to answer, however thought people might find it useful.
Below are the steps for importing data from mongodb to Solr 4.7.0 using DataImportHandler.
Step 1:
Assume that your Mongodb has following database and collection
Database Name: Test
Collection Name: sample
The sample collection has following documents
db.sample.find()
{ "_id" : ObjectId("54c0c6666ee638a21198793b"), "Name" : "Rahul", "EmpNumber" : 452123 }
{ "_id" : ObjectId("54c0c7486ee638a21198793c"), "Name" : "Manohar", "EmpNumber" : 784521 }
Step 2:
Create a lib folder in your solrhome folder( which has bin and collection1 folders)
add below jar files to lib folder. You can download solr-mongo-importer from here!
- solr-dataimporthandler-4.7.0.jar
- solr-mongo-importer-1.0.0.jar
- mongo-java-driver-2.10.1.jar (this is the mongo java driver)
Step 3:
Declare Solr fields in schema.xml(assumed that id is already defined by default)
add below fields in schema.xml inside the <fields> </fields> tag.
<field name="Name" type="text_general" indexed="true" stored="true"/>
<field name="EmployeeNumber" type="int" indexed="true" stored="true"/>
Step 4:
Declare data-config file in solrconfig.xml by adding below code inside <config> </config> tag.
<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">data-config.xml</str>
</lst>
</requestHandler>
Step 5:
Create a data-config.xml file in the path collection1\conf\ (which by default holds solrconfig.xml and schema.xml)
data-config.xml
<?xml version="1.0"?>
<dataConfig>
<dataSource name="MyMongo" type="MongoDataSource" database="Test" />
<document name="import">
<!-- if query="" then it imports everything -->
<entity processor="MongoEntityProcessor"
query="{Name:'Rahul'}"
collection="sample"
datasource="MyMongo"
transformer="MongoMapperTransformer" name="sample_entity">
<!-- If mongoField name and the field declared in schema.xml are same than no need to declare below.
If not same than you have to refer the mongoField to field in schema.xml
( Ex: mongoField="EmpNumber" to name="EmployeeNumber"). -->
<field column="_id" name="id"/>
<field column="EmpNumber" name="EmployeeNumber" mongoField="EmpNumber"/>
</entity>
</document>
</dataConfig>
Step 6:
Assuming solr (I have used port 8080) and mongodb are running, open the following link http://localhost:8080/solr/dataimport?command=full-import in your browser for importing data from mongodb to solr.
fields imported are _id,Name and EmpNumber(MongoDB) as id,Name and EmployeeNumber(Solr).
You can see the result in http://localhost:8080/solr/query?q=*

Answer 2

You can try using SolrMongoImporter, it ask you to import 2 libraries into your solr proyect and create a data-config.xml.
You probably will need to import in your solrconfig.xml the following libraries if you don't have it
<lib dir="../../../contrib/dataimporthandler/lib" regex=".*\.jar" />
<lib dir="../../../dist/" regex="solr-dataimporthandler-.*\.jar" />

Answer 3

If you have followed everything above and still facing the issue: check whether you have multiple jars in different locations.
i.e.
{solr-home}/dist/solr-dataimporthandler-8.10.0.jar
{solr-home}/server/libs/solr-dataimporthandler-8.10.0.jar
{solr-home}/server/solr/core/libs/solr-dataimporthandler-8.10.0.jar
If so, remove the jar files from everywhere but the location you have configured in the solrconfig.xml file.