We are currently looking into using tableau for analysis of app user data.
Some concerns were raised in my company about connecting tableau to our production database.
Security: We are not happy to share sensitive user data with tableau.
Database load: tableau queries may have a negative effect on the performance of our production database.
The idea came up to create a intermediary database. A script would regularly pull data from our production database and insert it into the intermediary database.
This may also make it easier to access data from our redis, elasticseatch, ??? and our main database in postgres.
Does anyone have experience with such a intermediary database? Is this common practice?
Related
I am creating accounting/invoicing software and my database is in postgreSQL. Should I create a separate database for each user since the data is sensitive financial data? Or is having a user foreign key secure enough? If I am hosting the database on aws I understand that I could have a few db servers across multiple availability zones and regions so that if one is compromised it wouldn't effect everyone even if many users have info stored in a single database. Is this safe enough? Thanks!
In general no. Encrypt the data so that if someone exfiltrates a dump they can't actually use it without the decryption key. If you're worried that someone with admin access can see the user's information then you might want to consider a user-level encryption for all fields related to personally identifiable information.
There are few ways you could go about it but I wouldn’t create a new DB for every customers. It will be too expensive and a pain to maintain and evolve.
To me, this sounds like you are creating a multi-tenant application.
I’d personally use the row-level security feature in Postgres (see this article) or create a separate Schema for each Customer.
You can add an extra layer of protection with encryption at rest. AWS support it (link)
I am in charge of the database for an application that we are developing and I am starting to get confused on how to use my development database.
I understand that having two separate databases is useful, helps while developing new features or change the database structure and this is why we actually have a production database and a development database. However as the project grows I am slowly getting confused on how I should use the development database and the development environment as a whole.
Our data is stored in Firestore which is a NoSQL database. We use it to store real time data that need to be accessed both by users and by a growing number of scripts that process the data to create some more. This real time data is also useful while developing to monitor the behavior related to the changes we made for a specific feature (on our test app).
So my question is :
Should my development database be a copy of the production database (copy every insert, update, delete ...) and should we duplicate all our scripts (one on the production environment and one on the development environment) that process the data ? In which case I would need to create connexions between each database and the costs related to storing and processing the data would double.
Or should I just use the development database as a database with the same structure as my production database, with less data and just pull some data or activate some pipelines to redirect some real time data when I need to test a new feature or a change in my database.
Also if you know a good book that I could read on the subject I'd take it !
Thank you,
I am building an app that allows people to share items with other people in the community. I wanted to use AWS as my platform.
My idea was to use react Native for the app. AWS Cognito for the authentication. AWS lambda for the server calls. Relational database for storing data about the items and user data such as geolocation. Dynamodb for real-time chat, requests for borrowing and transaction data between users. My primary focus is low cost and I was thinking of using PostgresSQL for relational database.
What do you guys think of my database choices. Of course the PostgresSQL database on rds. Is there a flaw in database plan so far? Any help would be greatly appreciated.
I would probably just use DynamoDB for everything in your application. I don't see a real need to storing some of your data in an RDS database here. However if you definitely need a relational database, I would suggest AWS Aurora Serverless so that your entire application would be using serverless AWS services. Also, normal relational database connection pools don't work that well in AWS Lambda, so I would suggest using the new Data API.
I have created a backend for a mobile app. The database uses Postgres with fairly complex relationships.
Is there a way to recreate the database in Realm? I saw that there was an (enterprise) real-time sync tool that links Realm to Postgres instances, but I'm unsure how to mirror the database in the first place. Do I simply write a schema, step-by-step, on the mobile client to match the Postgres database? The complex relationships involved would make that file very complicated to write.
The Enterprise Edition of Realm Platform contains a PostgreSQL data connector that can perform real-time synchronization between Postgres and Realm Platform, including creating the schema and loading the initial data.
I have an application where security and data theft are primary concerns. I am using Postgres 9.4 on RDS by AWS.
I have several users who need read permission on the db. I know that these users can essentially write a script to scrape all the data from the db but is there a way to deny them from using the pg_dump utility.
I am not sure what code examples I can provide for the same.
Is there any alternate strategy to use here? To share db data with developers without allowing them to take dumps of the same?