I am doing data import from Mongo to Solr. For this I need to add new fields into the Solr schema. I am using solr server 5.1.0.
Schema.xml is not available in solrhome/server/solr/core/config directory so I copied one from solr-5.1.0\server\solr\configsets\basic_configs\conf to core\config and added fields into this.
Now when we start Solr server I get WARN message that I should remove the schema.xml file as this is now managed-schema. These fields should be moved to managed-schema file, but this didn't happen.
How Solr will move the database table to indexing. Does it convert each row into a document? Or I have to specify document structure somewhere?
How I can re index the data previously imported?
For Data import you need db-data-config.xml file where you will put your sql queries and configure it with SOLR For information about how to achieve the configuration go to this link
I have find a way to add the fields into schema. Please provide your input if there is any other proper way of doing it.
1. add schema.xml file into your solr/core1/conf directory.
2. copy all content of managed-schema into schema.xml
3. add the new fields into schema.xml.
4. delete the managed-schema.xml
5. restart the solr it will create a managed-schema.xml and will move schema.xml to scheam.bkp
References-
solrconfig.xml - Read the doc for schemaFactory tag.
Related
I am new to mongodb and my task is to export metadata about all collections in a database. I know little about shell scripting and i am not able to understand the documents provided on mongodb website.
I am using Mongodb compass 4.2.14 and though i am able to export the collection, i am not able to just export the column names.
How do i export just the column names and also how to view the metadata?
In Compass GUI, select the Schema TAB, and click the Analyze button. This will show the data types for each field for a collection.
To get a copy of the schema, from the top-left menu, select the Collection -> Share Schema as JSON menu item. This will copy the schema to the Clipboard. You can then paste from the clipboard into any text file/editor of your choice.
Reference: Compass - Analyze Schema.
I create a schema and i add 1TB data to druid schema. then the log file version was upgraded and new two columns was added. then i want to add that data to druid schema. but couldn't yet.
In order to add a new column to existing datasource you need to follow the below steps:
Go to the Tasks menu in druid console.
From the listed datasources, go to the 'Actions' column in the last of the datasource in which you want to add the column.
There will be a magnifying glass like button, click on that to copy the existing payload.
Copy the payload in notepad and add the 2 columns to "dimensions" array.
Copy the updated payload and submit it via Submit Supervisor button.
You'll find the new columns in the datasource which you can verify by querying the datasource in query section of druid.
I create a schema and i add 1TB data to druid schema. then the log file version was upgraded and new two columns was added. then i want to add that data to druid schema. but couldn't yet.
In order to add a new column to existing datasource you need to follow the below steps:
Go to the Tasks menu in druid console.
From the listed datasources, go to the 'Actions' column in the last of the datasource in which you want to add the column.
There will be a magnifying glass like button, click on that to copy the existing payload.
Copy the payload in notepad and add the 2 columns to "dimensions" array.
Copy the updated payload and submit it via Submit Supervisor button.
You'll find the new columns in the datasource which you can verify by querying the datasource in query section of druid.
I am using Apache drill in embedded mode and when I am able to connect to mongo and query in drill successfully.
However when I create a schema in saiku schema designer using driver as "org.apache.drill.jdbc.Driver" and URL as "jdbc:drill:drillbit=hostname:31010" the connection is successful and all collections are also fetched and shown as tables in saiku, but in place of column names "*" is coming and actual column names are not coming.
Dont know what I am missing on.
I figured out the solution and posting in case anyone could benefit. I had created a view in drill with select * from table. When I created view as select col1,col2... from table the issue got resolved.
I am currently using the Kentico Import Toolkit to create documents in the tree.
At this point, I have imported around 100 documents using the toolkit, and they are all located at the correct place in the tree. Now the issue/concern that I had was, as I have imported these documents, my spreadsheet has been updated, so extra fields and data were added, so how do I go about importing this extra data into the currently existing documents? Also just bear in mind I don't want other fields or data to be affected by this, as some of the documents were updated with some other content by the content editors using CMS Desk, which isn't available in the spreadsheet.
Import toolkit is not the right tool to achieve this task. Even if you select "Import new and overwrite existing pages" it'll overwrite most of your columns. Actually it only preserves system and id columns from the existing documents - all other columns get overwritten.
Either you can write a piece of custom code or you can try following:
Open SSMS and navigate to the coupled table of your page type (something like CONTENT_MyDocType). This is where your custom columns are stored.
Right click -> Edit top 200 rows
Click "Show SQL Pane"
Adjust the columns, ORDER BY and WHERE clause to match your excel file, re-run the query
Select desired rows in your excel file and copy them to clipboard
Paste the data in the SSMS
rocky is right, Import Toolkit is meant for importing complete objects, not partial/continuous update.
You could map the fields that you know are not changed in the spreadsheet to a SQL query selecting the value from the target database.
To achieve this, just insert #<target> at the beginning of the SQL select statement you will be mapping the field to.
It will be rather laborious though and it also requires certain knowledge about the nature of the spreadsheet changes.