I am building an app that allows people to share items with other people in the community. I wanted to use AWS as my platform.
My idea was to use react Native for the app. AWS Cognito for the authentication. AWS lambda for the server calls. Relational database for storing data about the items and user data such as geolocation. Dynamodb for real-time chat, requests for borrowing and transaction data between users. My primary focus is low cost and I was thinking of using PostgresSQL for relational database.
What do you guys think of my database choices. Of course the PostgresSQL database on rds. Is there a flaw in database plan so far? Any help would be greatly appreciated.
I would probably just use DynamoDB for everything in your application. I don't see a real need to storing some of your data in an RDS database here. However if you definitely need a relational database, I would suggest AWS Aurora Serverless so that your entire application would be using serverless AWS services. Also, normal relational database connection pools don't work that well in AWS Lambda, so I would suggest using the new Data API.
Related
We certainly want to use separate databases since the front-end team finds it robust to work with MongoDB Atlas and AWS cloud architects find it easy to work with DynamoDB.
Our architecture:
Web application uses MongoDB to insert, update and retrieve data.
The MongoDB is synced in real-time with DynamoDB.
Background AWS services use DynamoDB for inserting, updating and retrieving data.
The changes in either DynamoDB or MongoDB are replicated to each other.
Tried so far:
We currently do have a sync in place with DyanmoDB streams and MongoDB atlas trigger to listen to changes on each database and forward them to the other. We use lambas for this, but our replication logic is not robust yet.
AWS Database Migration Service with ongoing replication has been suggested but haven't been able to get it to work in our use case. Perhaps, this is one option.
3rd party services like: https://www.cdata.com/sync/
Ideal Fit
The most ideal solution would be an AWS-based solution if not a reliable 3rd party service.
Greatly appreciate any resources or thoughts on this! :)
I'm very new in the world of "NoSQL", Recently I started using Cosmos DB for JSON documents and am able to store and get data using CosmosDB "SQL" and "Mongo" API.
Which API is best for me? (requirement: Performance, Low Latency, Huge Collection, Low cost)
Now I have spent more than 6 months with Cosmos DB project. I can give my opinion on this question.
Azure Cosmos DB offer the same feature with SQL API vs Mongo API, hence you can't compare by feature.
Reason using Mongo API in Cosmos DB
If you have existing Mongo DB database and you want to migrate on Azure.
If you or your team members have Mongo DB experience.
If you are developing cloud agnostic application, you can move your application on-premise or any other cloud vendor.
Reason using SQL API in Cosmos DB
If you have SQL query experience, it would be easy to start.
If you are not looking cloud agnostic application. (can't move Cosmos DB application on-premise or other clouds vendors).
Cosmos DB Change Feed only available with SQL API.
--- UPDATED 2020 ---
Change Feed is available for all the APIs across all available SDKs except the Table API
Both the API’s provide you high throughput, low latency, elastic scale, geo-replication, etc.
It boils down to what you are comfortable with. Many people preferred Mongo, as they are already familiar with Mongo, it’s syntax, tools, and they already had an application built on Mongo which they migrated to CosmosDB.
If you are starting new, and don’t have an existing investment in Mongo, I will suggest starting with SQL. SQL is the oldest API of CosmosDB, and more battle hardened.
As a software engineer using MongoDB API. I can confirm that my team and I suffered when handling 426 code errors from CosmosDB. So we end up developing our own custom solution with Polly to handles these throttles. Whereas if we used SQL API we would have access to CosmosDB sdk which pretty much handles these situations. This is just my feedback.
Currently I need to create admin ui, using google cloud datastore as user storage, and google functions as controllers.
I found the Keystone JS CMS platform, which uses mongoDB for storing user data, so I am looking for the chance to use it with google cloud datastore.
Is it possible to create some driver that will redirect all the keystone db requests not to mongodb, but in Google Datastore?
The core of Keystone is built on top of Mongo (and Mongoose) functions and code. At the moment, there isn't another way to use Keystone with another database type or provider. See this GitHub issue for some more information that may be of use.
I've a google big table instance that need to be populate with data that are in a Postgres Database. My product team give a URL's that allow me to replicate the database. So using simple words I need to duplicate the Postgres database into the google instance and the way that my product team give me is using this url, how can I do this? any tutorial that can help me?
If you are already running PostgreSQL and would like to have a mirror of it on Google Cloud Platform, the best and simplest approach may be to run your own PostgreSQL instance on a Google Compute Engine virtual machine which can be done via several approaches, e.g.,
tutorial for launching PostgreSQL, or
click-to-deploy solution for PostgreSQL by Bitnami
Then, you would want to continuously mirror data from your local instance to the PostgreSQL instance running in Google Cloud to be able to query it. Another SO answer suggests that there are two major approaches to this:
Master/Master replication (Bucardo)
Master/Slave replication (Slony)
Based on your use case where you want to keep your local PostgreSQL instance as the canonical one, and just replicate to Google Cloud for the purpose of querying it, you want a Master/Slave replication, and have the PostgreSQL instance be the read-only replica, so you probably want to use the Slony approach.
For a more in-depth look at PostgreSQL solutions for high availability, load balancing, and replication, see the comparison in the manual.
I am creating a simple application and getting stuck with data storage option. To begin with I would like to use SQL Server as my data storage. I will not be using any special features of SQL Server, its pure tables with CRUD operations.
Now I should be able to switch the underlying data store to either SQL Data Services or Amazon S3 by changing few configuration parameters.
Is this possible??? If yes, can anyone provide high level guidance on how to go about it? Do I need to use Entity Framework to begin with SQL Server? Does Entity Framework supports SQL Data Services? Any common component which supports both SQL Data Services and Amazon S3?
Too many questions!!!
Thanks for the help in advance.
The closest ORM I know of is LightSpeed. I've never used it though. Personally, if the end goal is to use cloud storage, I'd probably just use cloud storage from the get go...
If you are going to use Amazon's SimpleDB, M/Gateway has a open source db that mimics their API.