I am working on a project that involves me dynamically creating a database for each client. I could do this with PHP and MySQL, but for optimization, the project language is been changed to Go and PostgreSQL. So i am trying to see if there is a way to achieve this with go-pg or any method in Go.
Related
On the side of my Gooddata project, I maintain a small PostgreSQL database that contains a few tables.
I would like to be able to integrate both my ETL processes using the same tool, and it seems to me cloudconnect would be the easiest way, since I already have my whole GoodData ETL in it.
Here are the ways I tried to do it without success:
I tried to have a look in the documentation, and it seems to me that all the functionalities of CloverETL that enabled this (DBOutput, PostGreSQLDataWriter) are not available in Cloudconnect.
I managed to connect to the Agile Datawarehouse Service (Database attached to GoodData), but it seems that only the ADS database is able to understand the request:
COPY MyDataBaseTable (field1,field2) FROM LOCAL '${DATA_TMP_DIR}/CIforADS.csv'
even when I adapt the syntax to PostgreSQL because the dynamic addressing I use here does not seem to work.
Is there any way to proceed that I'm missing? Can anyone think of a workaround?
In general this could be achieved by using of "DBExecute" component, but
I'm not sure if I understand it well - do you want to load data into your own Postgres instance using CloudConnect?
As a webapp novice, I'm not sure if I need to define models.py.
I already have a working Postgres database on Heroku that I've linked up with Postico and pgAdmin. Using these GUIs, it seems I can get and post data, and make structure changes very simply.
Most tutorials seem to glaze over the details and reasoning for having a models.py. Thanks!
Web frameworks typically enforce or encourage a Model-View-Controller (MVC) patterns that structures code such that the database code is kept separate to the presentation layer.
Frameworks like django come with and are more integrated with ORM functionality which is used to implement an MVC framework. The ORM allows you to programatically interact with your database without having to write sql code. It can let you create a schema as well as interact with it by mapping programming classes to tables and objects to rows.
Flask can be distinguished from many other web frameworks, like django, in that it is considered a micro framework. It is light weight and can be extended by adding extensions. If you need the database integration then you can use it with an external ORM tool like sqlalchemy (and optionally flask-sqlalchemy extension). You can then define a sqlalchemy model, for instance, in a file called model.py or schema.py, or any other name you find appropriate.
If you only need to run one or two queries against an existing postgres database and feel you have no need for the use of an ORM, you can simply use flask with the postgres driver and write the sql yourself. It is not mandatory to have a model.
A model/ORM can be beneficial. For example if you want to recreate an integration/test instance of the database, you can instruct the ORM tool to create a new instance of the database on another host by deploying the model. A model also provides a programming abstraction to the database, which in theory should make your code more database independent (well in theory, its hard to achieve this as databases can have subtle differences), and should make your code less tied to a specific database solution. Also, it alleviates the need of writing a language within a language (sql text strings within python code).
I am using sails.js with postgres as the database. Although most of the actions can be easily handled via waterline ORM, there are certain cases I prefer to use native queries and sometimes even postgres' native stored functions. However, the challenge with stored functions is that they come with an overhead of code maintenance.
In my project repo, I have created a directory sql which contains all the SQL functions. Currently, I have to manually make sure that whenever I am making some changes to a function, I need to recompile it on database.
I need to configure it such that these are compiled whenever I restart the server, just like all the models are re-created. Is it possible to do this and how ?
Sails doesn't have any built-in support for compiling Postgres stored procedures, but you could make a Grunt task for this. Take a look at the documentation for tasks. These are run every time you lift Sails (and in some cases, when files are changed).
A quick Google found the grunt-pg-utils package, which might help you on your way.
is there a way to connect to a postgreSQL Database? I've noticed that the PHP module doesn't have the postgres extensions and its not possible to load it dynamically.
Is there any other way? An external PHP scripts will be the worst solution.
Alex
I don't have direct experience with Tide, but reviewing their docs here are the places I would look:
Javascript-> PostgreSQL Is there a possibility of connecting postgreSQL directly to Javascript?
Python -> PostgreSQL, like psychopg.
If that fails, along with other similar languages, MAY be able to find another driver written fully in a language like this.
The PostgreSQL protocol is extensively documented and several implementations exist for it. You could probably adapt either the Javascript driver or implement in another language to give you a direct driver to PostgreSQL http://www.postgresql.org/docs/9.1/static/protocol-flow.html
Using Model First, what is the best way to approach preservation of existing database data when the model changes and the database has to be regenerated?
The Database Power Pack extension no longer works (I've been trying to contact the author). I can't find anything that provides similar functionality.
R.
If database power pack doesn't work there is no other automatic way. Manual way requires running created SQL script on another database and using Visual Studio Database tools to create difference script between the current and the newly created database.