The project in which I am working requires synchronization of data between MongoDB and SQL Server, so what is the best way for it? This synchronization should be handled by MongoDB side; does MongoDB support job scheduling?
I am new in MongoDB and I want to know few things about it. I know it is not an RDBMS but in case it is possible, then how?
Like we can write program in Oracle can we write in MongoDB. I mean directly in MongoDB, if no then which client side language can used?
As #Vasanth said, MongoDB has no native job management.
As to which client side language: well that's entirely upto you.
You can always get get a prebuilt replicator like: http://code.google.com/p/tungsten-replicator/ that might do the job for you.
As further reference, this question maybe of help to you: https://softwareengineering.stackexchange.com/questions/125980/can-sql-server-and-mongo-be-used-together
Related
I was wondering if there is something like a script that I can write on the MongoDB side that would do something like delete an item in a list if it is older than a week old etc.
I want to have the DB do this check every day. Are there some kind of automated functions that I can setup on the DB to do this?
I could just write a few small methods to do it on the userside myself, but I remember my old SQL DB having this feature. Any help would be appreciated. Thanks,
If you are using Self-Managed MongoDB, The answer is NO.
Like SQL, MongoDB doesn't support scheduled transactions. However you can run the scheduled jobs in your programming language and perform the operations. For example, https://thecodebarbarian.com/node.js-task-scheduling-with-agenda-and-mongodb
If you are using MongoDB Atlas, Then you need to check this https://docs.mongodb.com/realm/triggers/scheduled-triggers/
Then you might want to check out the MongoDB TTL feature :
https://docs.mongodb.com/manual/tutorial/expire-data/
It won't run every day like a script, but it will automatically remove data after some time.
Hope it can help !
If a system is already running SQL Server, is it possible to use a NoSQL database (i,e MongoDb in particular) as the failover database in a SQL Server failover environment? Such that if the primary SQL node fails the secondary node running/hosting MongoDb takes the primary place.
The short answer to this question is "no". The long answer is anything is possible given enough code and resources.
SQL and MongoDB do not speak the same language, so there would need to be an intermediary that can translate. But this adds another failure mode to the system. It also needs to be complex enough to understand such concepts as "primary". There are connectors out there that will handle either SQL -> MongoDB or MongoDB -> SQL, but I'm not aware of any that are capable of syncing the two in real time. Additionally, it would be up to your application to determine where to query data from and where to write data to. This would be outside something a connector like these will do.
As my employer makes the big jump to MongoDB, Redshift and Spark. I am trying to be proactive and get hands on with each of these technologies. Could you please refer me to any resources that will be helpful in performing this task -
"Creating a data pipeline using Apache Spark to move data from MongoDB to RedShift"
So, far I have been able to download a dev version of MongoDB and create a test Redshift instance. How do I go about setting the rest of the process and get my feet wet.
I understand to create the data pipeline using Apache Spark, one has to either code in Scala or Python or Java. I have a solid understanding of SQL, so feel free to suggest which language out of Scala, Python or Java would be easy for me to learn.
My background is in data warehousing, traditional ETL (Informatica, Datastage etc.).
Thank you in advance :)
A really good approach may be to use AWS Data Migration Services
http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MongoDB.html
you can specify mongodb as a source endpoint and redshift as the target endpoint
I am seriously considering Meteor framework for building every POC and apps in the future...but, I can't get ride of an ACID compliant database as I have few usages of multi-documents atomic transaction that require this compliance.
Meteor strongly rely on MongoDB syntax and storage engine at the moment (it means there are no "Transaction" related syntax available...)
I am currently evaluating any solution allowing this ACID capability :
Using a MySQL native driver for Meteor (different syntax than MongoDB?)
Using a PostgreSQL native driver for Meteor (SQL syntax)
Using a TokuMX (a MongoDB fork with ACID compliance...same syntax than MongoDB appart from transaction related commands that would be required to add)
Those 3 solutions are good candidates for the Meteor roadmap as shown here
What pros/cons about those solution ? Which are the most advanced one ?
What would you although consider as a solution to keep Meteor while storing documents in a NoSQL like ACID compliant db ?
sqlAndMeteor
If you are like me, you love Meteor but hate Mongo. In Meteor's Trello Roadmap (https://trello.com/b/hjBDflxp/meteor-roadmap), the most voted feature is SQL Support, either PostgreSQL or MySQL.
Since there is no date for that in Meteor, here I summarize the partial solutions I have found.
1.- Use SQL only for client-side querys.
Let's face it, Mongo sucks on common data operations, so having the ability to use SQL to query data (with JOINS, GRUP BY and so on) would relief a lot of pain. There are packages which let you use SQL in the client, at least for querys: The simplest one is a old (2010) utility, SqlLike (http://www.thomasfrank.se/sqlike.html). The new player in town is alaSQL, which is actively developed by #agershun (https://github.com/agershun/alasql). The SqlLike advantage is that it only has 10k. AlaSQL, is a lot more powerful, of course, but for using SQL to replace mongo sintax in unions and aggregations, SqlLike is OK.
With both of them you can do something like this in your helper:
productsSold:function(){
var customerSalesHistory=salesHistory.find({cutomerId:Session.get('currentCustomer')}).fetch();
var items=products.find().fetch();
return alasql("select item.name, sales.ordered as sumaVentas from ? sales, ? items
where items.Id=sales.itemId",[customerSalesHistory,items]);
}
2.- Experiment with direct SQL support.
Some packages try to replace Mongo (and minimongo) with MySql or PostgreSQL. #numtel's MySql package is Meteor-MySql https://github.com/numtel/meteor-mysql, and PostgreSQL is Meteor-pg (https://github.com/numtel/meteor-pg). Both are good attempts to solve the problem, but have some issues yet and are somehow cumbersome to adapt.
A team from Hack Reactor has formed Meteor Stream, and its first product is a PostgreSql integration with Meteor, meteor-postgres (https://github.com/meteor-stream/meteor-postgres). It looks very good and uses alaSql on the client to replace minimongo.
Both approaches are good, but they have some problems:
They broke deployment to meteor.
They are very, very young and not near production ready AFAIK
They still require tweaks to the usual pub-sub sintax we are used to, which could raise compatibility issues with other meteor packages.
3.- Still use Mongo, but as simple repository for your MySql database.
This option maintains all Meteor's characteristics and uses Mongo as a temporal repository for your MySql or PostgreSql databases.
A brilliant attempt to that is mysql-shadow by #perak (https://github.com/perak/mysql-shadow). It does what it says, keeps Mongo synchronized both ways with MySql and let's you work your data in MySql.
The bad news is that the developer will not continue maintaining it, but what is done is enough to work with simple scenarios where you don't have complex triggers that update other tables or stuff like that.
For a full featured synchronization you can use SymmetricsDS (http://www.symmetricds.org), a very well tested database replicator. This involves setting up a new java server, of course, but is by far the best way to be sure that you will be able to convert your Mongo database in a simple repository of your real MySql, PostgreSQL, SQL Server , Informix database. I have to check it myself yet.
For now MySQL Shadow seems like a good enough solution.
One advantage of this approach is that you can still use all standard Meteor features, packages, meteor deployment and so on. You don´t have to do anything but set up the synch mechanism, and you are not breaking anything.
Also, if someday the Meteor team uses some of the dollars raised in SQL integration, your app is more likely to work as is.
If MySQL works for you, I've used the meteor-mysql package and it works well.
I finally forged my own conclusion...
I will have TWO platforms :
a Meteor front with business data in PostgreSQL and some front data or easy-to-replicate data in MongoDB
a Java data backend (server2server only) handling all atomic operations on my business data in PostgreSQL...plus technical adapters (SAP, Salesforce), a BPMN 2.0 workflow engine (Actility), and any registered SOA needed from other systems
Any comments are still very welcomed and will be considered and answered
I have some collection and I want to perform action on every insert into that collection. The problem is that the code, that will do this actions is in Java. In Oracle it was possible to wrap Java or even C code into PL/SQL procedure, and then use this procedure in trigger. In CouchDB we could write a view. What would be the closest analog for MongoDB?
The best possibility I can think of is to wrap my code into REST server, and then interact with it using stored javascript.
I've already seen this question, but due to dependency on java libs, I can't use just javascript in my workflow, neither I don't want to run a new heavy service along with mongodb if there is some other way to do this.
There are a number of things to say about your request:
I have some collection and I want to perform action on every insert into that collection.
1) What you're asking for here is not really a "stored procedure", but really is a "database trigger". MongoDB does not provide any sort of "database trigger" functionality.
This is consistent with the general design goals of MongoDB, which is to provide a very fast, scalable data store without the heavy weight of traditional DBMS systems. See this presentation for more details about the design goals of MongoDB: http://www.10gen.com/presentations/mongosf2011/whymongodb
2) If there is some data processing that you'd like to perform on every insert, you'll need to do it on the client side of the MongoDB connection. This will necessarily involve writing some code in your application.
3) I'd suggest that you avoid running JavaScript within the mongod server if at all possible. The JavaScript is interpreted on the server side, so the speed of your queries will be affected. In addition, all JavaScript run in the mongod server is single-threaded, so there is no concurrency of any JavaScript execution.
I wish I had a better answer for you.