I am very new to Apache Nifi. I am trying to Migrate data from Oracle to Mongo DB as per the screenshot in Apache NiFi. I am failing with the reported error. Pls help.
Till PutFile i think its working fine, as i can see the below Json format file in my local directory.
Simple setup direct from Oracle Database to MongoDb without SSL or username and password (not recommended for Production)
Just keep tinkering on PutMongoRecord Processor until you resolve all outstanding issues and exclamation mark is cleared
I am first using an ExecuteSQL processor which is resulting the dataset in Avro, I need the final data in JSON. In DBconnection pooling Service, you need to create a controller with the credentials of your Orcale database. Post that I am using Split Avro and then Transform XML to convert it into JSON. In Transform XML, you need to use XSLT file. After that, I use PutMongo Processor for ingestion in Json which gets automatically converted in BSON
Related
I am having one requirement for SQL-like interface to query S3 object storage(It's not actually AWS S3, but S3 based protocol storage. We are having server name, bucket name, access_key and secret_access_key available with us).
I need to know whether there exists any PostgreSQL extension which can leverage foreign data wrapper(FDW) feature to run query against S3 data.
Here, we don't want to copy the data(csv files) from S3 to postgreSQL server. We have external PostgreSQL server, using any extension, we directly want to query that S3 data using that PostgreSQL engine and get the result on PostgreSQL server.
A very similar requirement link was found as below:
https://www.cdata.com/kb/tech/amazons3-jdbc-postgresql-fdw.rst
But it is using it's own cdata JDBC driver. We don't want anything proprietary.
Is there anything opensource available? If yes, how can we achieve that?
We are using 12.2 PostgreSQL version.
I'm hoping to read data from a CSV file, process the data, and upload it to a remote database. I'm using Spring's starter repo as a base. Code is here
I tried putting this information in the properties file:
spring.data.cassandra.contact-points=IP, IP
spring.data.cassandra.port=PORT
spring.data.cassandra.keyspace-name=KEYSPACE_NAME
spring.data.cassandra.username=USER
spring.data.cassandra.password=PASS
spring.data.cassandra.ssl=TRUE
However, I think it keeps defaulting to pushing to some local tomcat jdbc. I'm not really sure where to start. Any help is appreciated! Thanks.
Your code doesn't have anything to use Cassandra. It doesn't have any of the dependencies and the ItemWriter implementation is a JdbcBatchItemWriter which I don't think will work for Cassandra. You need to configure your application to actually use Cassandra (the spring data starter as well as an ItemWriter implementation that can write to Cassandra).
I am using Mule 3.6 and would like to use the bulk insert option on the Generic Database Configuration to load data into mongodb 3.0.8.
I have entered the URL as:
jdbc:mongo://localhost:27017/test
and have tried a number of different Mongo and JDBC drivers but keep receiving the message "Test connection failed. Error trying to load driver..."
How can I configure the Generic Database Connector in Mule to connect to Mongo?
As already stated in this post, there is no official JDBC driver for MongoDB but one the suggested alternatives is using UnityJDBC.
If you decide to follow the UnityJDBC approach, then:
Download and install the driver by executing the following command:
java -jar UnityJDBC_Trial_Install.jar
Go to the installation folder and copy mongodb_unityjdbc_full.jar to the classpath of your Mule app.
Configure the URL and Driver in the Global Element of your Generic Database component (the values you provided are OK):
URL: jdbc:mongo://<host>:<port>/<database>
Driver Class Name: mongodb.jdbc.MongoDriver
If not, use the MongoDB Connector as suggested by #JoostD.
You need to use the MongoDB connector, it should be included in studio.
Otherwise install it from the Anypoint Exchange:
https://www.mulesoft.com/exchange/#!/mongodb-integration-connector
Also see some example on it:
https://www.mulesoft.com/exchange/#!/importing-csv-into-mongodb
I have a Jenkins job that runs the Selenium tests. The results are then stored in a CSV file and then fed to Cassandra. My requirement is to create JIRA request if the test fails either by analyzing the CSV file or from Cassandra. Please suggest the possible approaches.
Jira API + CSV Reader or Cassandra API
https://docs.atlassian.com/jira/REST/latest/
How do I add new Database support (MongoDB) in 2.6.3 version of WSO2 Data Service Server.
You can use DSS (2.6.3) with any database type if the database connectivity is exposed via JDBC. In other words, if your preferred database type exposes a JDBC driver/adapter for the users to connect to it via JDBC then you can use DSS to expose your data stored in your data store as a web service. Similarly, if MongoDB too has a JDBC adapter you wouldn't have any (or too many :) ) issues integrating that with DSS. However, there are some exceptions when it comes to exposing flat files such as google spreadsheeets, excel sheets, csv files as DSS uses the relevant client APIs such as Google gdate client API, Apache POI, etc to connect to those datasources and extract data. However, if we consider the general case is you need to have an adapter or a similar mechanism to connect to your datasource via JDBC.
But in the upcoming version of DSS (v3.0.0), it is planned to introduce custom datasource support so you can easily write an adapter to any datasource and use it with DSS.
Regards,
Prabath
I am not sure about this, but I suppose that if is not supported by default you can always download the jar library for MongoDB and put it in CARBON_HOME\repository\components\lib and restart. For example for mysql I have the mysql-connector-java-5.1.7-bin.jar in that folder.
Hope this help