I am switching from MongoDB to AWS DocumentDB. However, in MongoDB, I used GridFs to store and retrieve files larger than 16MB. But this is not supported in AWS DocumentDb. Is there any way on how to store or process large files(>16MB) in AWS DocumentDB?
Any help or leads would be appreciated. Thanks!
GridFS is a client construct and should just work with DocumentDB. However, we do not test it and thus don't officially support it. Did you encounter any issues when using GridFS with DocumentDB?
If you switched to MongoDB Atlas which runs very nicely on to of AWS you would still be able to used GridFS to store your files.
Related
Hi I'd like to move some data from an AWS DocumentDB to a PostgreSQL, I've found on the web ToroDB that would be the best tool for my task. It's implemented for MongoDB I'd like to know if I can use it also in AWS DocumentDB (with MongoDB compatibility)
Thank you in advice!
From what I can see in ToroDB documentation , is using the oplog to tail the changes in MongoDB, not change streams. Amazon DocumentDB doesn't have the oplog collection, but does supports change streams. Unless ToroDB gets updated to support change streams (which I doubt it will, documentation says only MongoDB 3.2 and 3.4 are supported and last Github update is from 5 years ago), is not going to be able to live replicate from DocumentDB.
However, if the destination is AWS RDS PostgreSQL or AWS Aurora PostgreSQL, then AWS DMS can be used to replicate from DocumentDB.
I've tried multiple things to try to connect AWS glue to MongoDB atlas. Has someone been successful in doing so and if so, please can someone help me with the steps.
The AWS documentation claims that it should work with any compatible MongoDB link but it doesn't.
I am facing a similar issue. I checked with the AWS support team and it seems like they have a huge backlog of similar issues where customers have requested the ability to connect to MongoDB Atlas. Unfortunately, they don't have an ETA for this.
Either you can opt to migrate to AWS Document DB and then use Glue to crawl your data store or you probably have to think of some other way to get your data from atlas to a layer that is supported properly by Glue.
I have a project which has the following characteristics:
Local MongoDB replica set on an on-premise database
Cloud MongoDB instance in MongoDB atlas
On-premise MongoDB should keep in sync with MongoDB atlas
Local MongoDB instance may be offline several days
Once its online, it should start synchronizing with MongoDB atlas
Basically, I'm looking for something similar to Realm, except that this solution runs on an actual local server and not a mobile device.
I have looked into live migrations, see here. But this doesn't seem to fit this use-case entirely, as its intended for an eventual cutover, which I don't want.
Therefore, how can I achieve the following with MongoDB atlas? What am I missing?
Can I treat MongoDB atlas, as if its a part of my local replica set, and use the standard replication capability of MongoDB? I.e. Atlas will always be a secondary.
This functionality with native MongoDB Atlas not possible. You need to look for customised solution.
Is it possible, and if so how, to use an Azure MongoDB as the backend for my Meteor Application.
I have added the connection string from my database into the MONGO_URL variable with no success. I have found some previous threads over Stack Overflow and on here about incompatibility related to oplog errors, but they seem to be using DocumentDB instead of Azure's MongoDB (which I think is newer than a few years ago).
In your example, you're actually using DocumentDB with MongoDB compatibility. You're not using native MongoDB (nor is this native MongoDB as-a-service).
DocumentDB (even with MongoDB compat) does not provide an oplog. And since Meteor has a dependency on reading the oplog, you wouldn't be able to point Meteor at DocumentDB.
In your case, you'd need to either run native MongoDB on your own (e.g. in VMs) or take advantage of a 3rd-party MongoDB hosting solution which provides MongoDB support within the same region as your app. (ok, yes, you can run your app in a different region, but you'd see latency along with data egress charges).
I am deciding to use MongoDB as a Document management DB in my application. Initially I was thinking to use S3 as a data store but it seems mongoDB uses local file system to store data. Can I use S3 as data store in MongoDB.
thanx
Provisioned IOPS in AWS is ideal for MongoDB.
This link has notes about running MongoDB on AWS and is rather useful.