I can read in thingsboard configuration documentation (https://thingsboard.io/docs/user-guide/install/config/), in "Common database parameters" section, that database.ts.type can be sql or cassandra, and cassandra should be used for hybrid mode. what's that hybrid mode?
Do you mean that database.entities.type can be sql (postgres) and database.ts.type can be cassandra? and
vice versa?
what's the recommanded install? All on cassandra?
Many thanks,
Best regards
Found the answer here:
https://thingsboard.io/docs/reference/ , section "SQL vs NoSQL vs Hybrid database approach"
ThingsBard uses database to store entities (devices, assets, customers, dashboards, etc) and telemetry data (attributes, timeseries sensor readings, statistics, events). Platform supports three database options at the moment:
SQL - Stores all entities and telemetry in SQL database. ThingsBoard authors recommend to use PostgreSQL and this is the main SQL database that ThingsBoard supports. It is possible to use HSQLDB for local development purposes. We do not recommend to use HSQLDB for anything except running tests and launching dev instance that has minimum possible load.
NoSQL - Stores all entities and telemetry in NoSQL database. ThingsBoard authors recommend to use Cassandra and this is the only NoSQL database that ThingsBoard supports at the moment. However, due to a lot of interest to deployments with managed databases, we plan to introduce support on AWS DynamoDB in v2.3.
Hybrid - Stores all entities in SQL database and all telemetry in NoSQL database.
Related
We are in the process of refactoring our databases. As part of that we have modified split our data which used to exist in single Postgres table into a new Postgres table schema and DynamoDB tables. What is the best way to migrate the data from the old schema into this new hybrid schema? We are thinking of writing a Java program to do it. But wanted to check if we can leverage some AWS offering to do this in an efficient way.
Check out the AWS Database Migration Service.
AWS Database Migration Service helps you migrate databases to AWS quickly and securely. The source database remains fully operational during the migration, minimizing downtime to applications that rely on the database. The AWS Database Migration Service can migrate your data to and from most widely used commercial and open-source databases.
See https://aws.amazon.com/dms/
I have a database in PostgreSQL. Now we need to import whole database into DynamoDB. Data Migration service need to be used for this purpose or any other service can be used. Please explain in detail.
What is the strategy to be followed? I have studied many blogs but I couldn't get any proper way to migrate the whole database from PostgreSQL to DynamoDB. Only through DMS Service is it possible to migrate or any other service can be used to migrate or any script should be run to migrate the PostgreSQL to to DynamoDB.
The AWS Database Migration Service offers everything you would need to migrate data from any relational DB into AWS - whether the target is DynamoDB (NoSQL offering from AWS) or any of the DB flavors from AWS RDS service.
You can find multiple migration playbooks and step-by-step guides on the Resources page of this AWS service
I have an OLTP DB in Oracle and a downstream OLAP System in PostgreSQL on-premises. The data from Oracle is pumped into PostgreSQL using Oracle_FDW.
I am exploring the possibility of moving the PostgreSQL to AWS, but none of the RDS have Oracle_fdw capability. One way out is to install PG on an EC2 instance but that would leave some of the features like read-replica provided natively by AWS. Is there a better workaround?
Also is there a way to fetch the data in Oracle RDS from Postgres RDS in AWS?
With PostgreSQL on Amazon RDS your choice of extensions is limited to the extensions they explicitly support. As far as I'm aware there's no way around this limitation.
Like you mentioned, the general option in this case would be to host PostgreSQL yourself on EC2 instead of RDS. You lose automatic backup/replication/management features, but you get the power and flexibility you need. This will certainly work but will require some leg work to replace what you're losing by not using RDS.
The only alternative to this I can think of is that you may be able to host a different (otherwise empty) PostgreSQL server with the oracle-fdw extension installed and use the postgres-fdw extension (which is supported by RDS) to proxy requests from your RDS hosted database, through your proxy PostgreSQL database, to your Oracle database and back. If the amount of data you're retrieving is substantial, or if the number of queries per minute is high this is probably a terrible idea. But it might be worth testing to see if it works for your use case.
I did a quick search around and I haven't been able to find any references to anyone actually layering foreign data wrappers like this but I also couldn't find anything in the manual or online saying it wasn't supported either. In theory it should work, but if you do try it make sure you thoroughly test it prior to using it to do anything important.
Oracle_FDW is now supported in recent versions - https://aws.amazon.com/about-aws/whats-new/2021/07/amazon-rds-for-postgresql-supports-oracle-fdw-extension-for-accessing-data-in-oracle-databases/
Which IBM Bluemix service allows you to convert data into an SQL database?
IBM DB2, object storage, MongoDB by compos, or Apache Spark Boiler Plate?
If you mean "store" in a SQL database, then there are a couple options. The "sqldb" service is based on DB2 and is for the regular (traditional) relational processing of data. The "dashDB" service is for analytic processing of relational data and is also based on DB2 and uses the BLU Acceleration technology. There are also services offered that feature the full DB2 functionality or make use of the Informix TimeSeries feature.
The "sqldb" and "dashDB" services can be interfaced via input and output nodes in the Node-RED starter kit. All services have suitable drivers for the offered runtimes (node.js, Python, Java, ...).
The Postgres Plus Cloud Database was announced early this year.
As I do not have a lot of experience on database clustering, I am wondering which one I should use. Can anyone give me a compare between Postgres Plus Cloud Database and Amazon RDS in terms of features, cost and maintenance?
The requirements for our applications are:
high availability.
There will be very low load for our application.
Thanks!
PostgreSQL is not (yet) supported by Amazon RDS. So it's not easy to answer your question.
In any case: Heroku could be an alternative to Postgres Plus Cloud Database.
PostgreSQL is now supported in Amazon RDS!
See the blogpost here:
http://aws.typepad.com/aws/2013/11/amazon-rds-for-postgresql-now-available.html?sc_ichannel=EM&sc_icountry=Global&sc_icampaign=reinvent1114&sc_idetail=55698910&ref_=13