i set the PERSISTENCE to true and DATA_DIR also exist,
i even confirmed that it was set by issuing "localstack config show" and there it is seen that the settings are indeed set.
i ran localstack with "localstack start -d" then i created an S3 bucket by "awslocal s3api create-bucket --bucket mybucket"
querying it after to confirm it has been created.
I tested the persistence by shuting down the docker image and restarting it.
i queried it again by "awslocal s3api list-buckets" but it doesnt looks like it was there and i need to recreate it.
how can i make localstack persist S3, SNS and database tables? we are trying to build our own local dev environment and it seems it is timed consuming to always recreate our data everytime.
Persistence works for LocalStock Pro users only
https://github.com/localstack/localstack/issues/6281
Related
I'm trying to avoid race conditions with gcloud / gsutil authentication on the same system but different CI/CD jobs on my Gitlab-Runner on a Mac Mini.
I have tried setting the auth manually with
RUN gcloud auth activate-service-account --key-file="gitlab-runner.json"
RUN gcloud config set project $GCP_PROJECT_ID
for the Dockerfile (in which I'm performing a download operation from a Google Cloud Storage bucket).
I'm using a configuration in the bash script to run the docker command and in the same script for authenticating I'm using
gcloud config configurations activate $TARGET
Where I've previously done the above two commands to save them to the configuration.
The configurations are working fine if I start the CI/CD jobs one after the other has finished. But I want to trigger them for all clients at the same time, which causes race conditions with gcloud authentication and one of the jobs trying to download from the wrong project bucket.
How to avoid a race condition? I'm already authenticating before each gsutil command but still its causing the race condition. Do I need something like CloudBuild to separate the runtime environments?
You can use Cloud Build to get separate execution environments but this might be an overkill for your use case, as a Cloud Build worker uses an entire VM which might be just too heavy, linux containers / Docker can provide necessary isolation as well.
You should make sure that each container you run has a unique config file placed in the path expected by gcloud. The issue may come from improper volume mounting (all the containers share the same location from the host/OS), or maybe you should mount a directory containing their configuration file (unique for each bucket) on running an image, or perhaps you should run gcloud config configurations activate in a Dockerfile step (thus creating image variants for different buckets if it’s feasible).
Alternatively, and I think this solution might be easier, you can switch from Cloud SDK distribution to standalone gsutil distribution. That way you can provide a path to a boto configuration file through an environment variable.
Such variables can be specified on running a Docker image.
I have an AWS RDS postgresql database that was provisioned via terraform with encryption disabled: storage_encrypted = false
This database needs to be encrypted now but I can see from the docs that enabling encryption is something that can only be done during DB creation.
I was considering creating a read replica of this instance with encryption enabled and then promoting this replica to be a standalone instance and finally pointing my app to this new instance. Is there a simpler way?
One of the ways to achieve this in a non-production environment is as follows -
Stop writes on the instance, ie. stop the applications writing to the RDS tables
Create a manual snapshot of the unencrypted RDS instance
Go to Snapshots from the left panel and choose the snapshot just created
From the Actions, choose Copy snapshot option and enable encryption
Select the new encrypted snapshot
Go to Actions and select Restore snapshot
For a minimal downtime switch follow this -
https://aws.amazon.com/premiumsupport/knowledge-center/rds-encrypt-instance-mysql-mariadb/
We are experimenting with Kubernetes and Confluence in the cloud and have deployed Confluence connected to a pgsql database. When applying an update, something happened that caused the pgsql pod to tank and lose the persistent volume connections.
Thankfully the volume was set to retain, so we have the volume and I have since been able to point a new pgsql instance to this volume, but I can't find a way to get Confluence to see this existing database. Confluence just proceeds to the initial fresh install screens. I've tried installing it on a temporary database and then modifying the confluence.cfg.xml file to point to the old data once completed but Confluence will not restart when I try this.
Any help is appreciated.
Using the web installer you should have a step to select "My own database". From there you can configure the database credentials and host. Go ahead and let the installer run, it will overwrite the default settings but will retain your existing data.
Also, you may want to get on the psql shell via console and check to make sure that your data actually exists and you haven't ended up with an empty database.
If you're still stuck, reach out here and we can check out the next steps.
In my case the original solution posted here is accurate:
However I had to do this in a non containerized environment. I installed Confluence on a VM using a blank database, then modified the confluence.cfg.xml file to point to the pgsql database in the kubernetes cluster and restarted confluence. I was able to see my data, so I then used confluence's XML export feature to grab the dataset. I then blew away the kubernetes environment and re-created it from scratch and imported the backed up XML into the new instance. Not a super clean way of doing it, but got where I needed to.
I would like to deploy an application using docker and would like to use a postgresql container to hold my data.
However I am worried about losing data, so I need back-ups.
I know I could run a cron job on the host to dump the data out from the container, however this approach is not containerized and when I deploy to a new location, I have to remember to add the cronjob.
What is a good , preferably containerized, approach to implement rotating data backups from a postgresql docker container?
Why not deploy a second container that is linked to the PostgreSQL one that does the backups?
It can contain a crontab within, together with instructions on how to upload the backup to Amazon S3, or some other secure storage in the cloud that will not fail even in case of an atomic war :)
Here's some basic information on linking containers: https://docs.docker.com/userguide/dockerlinks/
You can also use Docker Compose in order to deploy a fleet of containers (at least 2, in your case). If your "backup container" uploads stuff to the cloud, make sure you don't put your secrets (such as AWS keys) into the image at build time. Put them into the container at run-time. Here's more information on managing secrets using Docker.
I have read that you can replicate a Cloud SQL database to MySQL. Instead, I want to replicate from a MySQL database (that the business uses to keep inventory) to Cloud SQL so it can have up-to-date inventory levels for use on a web site.
Is it possible to replicate MySQL to Cloud SQL. If so, how do I configure that?
This is something that is not yet possible in CloudSQL.
I'm using DBSync to do it, and working fine.
http://dbconvert.com/mysql.php
The Sync version do the service that you want.
It work well with App Engine and Cloud SQL. You must authorize external conections first.
This is a rather old question, but it might be worth noting that this seems now possible by Configuring External Masters.
The high level steps are:
Create a dump of the data from the master and upload the file to a storage bucket
Create a master instance in CloudSQL
Setup a replica of that instance, using the external master IP, username and password. Also provide the dump file location
Setup additional replicas if needed
Voilà!