I created a spring project in AWS codestar.
I would like to pass environment properties to my application (e.g. DATA_SOURCE_URL). I can do it in elastic beanstalk in "Configuration" -> "Software" "modify" and adding the properties. But whenever a new deployment is triggered this configuration gets reseted.
I was wondering what is the way of setting environment properties when using AWS codestar.
As it may help other people that search a solution
I finally get it to work by using the Saved Configuration function in Beanstalk, and calling it via the cloud formation template.yml : EBConfigurationTemplate (from the autogenerated template.yml by codestar)
EBConfigurationTemplate:
[...]
SourceConfiguration:
ApplicationName: !Ref 'EBApplication'
TemplateName: "Saved Configuration Name"
After that my django application was able to read the os.environ['ENV_VAR_NAME']
as well as django.config that was able to connect to an RDS (Non-managed by beanstalk) to do the migration as a container_command
Related
I want to load env variables from .env file, when running locally,
So here's my serverless.yaml file,
functions:
health:
handler: src/api/health.check
name: ${self:provider.stackName}-health
environment:
USER_POOL_APP_CLIENT_ID: !Ref UserPoolClient
You see, it sets a userpool id that gets created in the resources section as an environment variable to a lambda. This works perfectly fine, when deployed as expected.
However, when I try to run it locally via serverless-offline, no matter how I set the env variables, via dotenv or manually, it seems to get overriden by serverless, in this case all I see is "[object object]".
Other workflows that I see, load all env variables from files, like below
functions:
health:
handler: src/api/health.check
name: ${self:provider.stackName}-health
environment:
USER_POOL_APP_CLIENT_ID: {env:USER_POOL_APP_CLIENT_ID}
But wouldn't this require us to have variables of all stages, stored locally as files?
I was hoping to store only the dev version, locally, and have all the remaining fetched from the serverless file itself, automatically via !Ref like shown at the beginning.
So, how do I prevent, serverless from populating/polluting my env variables when I run locally, while sticking to the first format?
Or are there other better ways to handle this?
It happened here with the new version of serverless-offline (v12.0.4).
My solution was to use: https://www.serverless.com/plugins/serverless-plugin-ifelse
See the example below:
serverlessIfElse:
- If: '"${env:NODE_ENV}" == "development"'
Set:
functions.health.environment.USER_POOL_APP_CLIENT_ID: "${env:USER_POOL_APP_CLIENT_ID, ''}"
You can change it for your use case.
Mainly wondering if anyone has encountered before (and how to resolve).
Transfer job from s3-compatible source to Cloud Storage fails. Unable to list objects from the source.
Agent logs show
W0111 06:49:13.002952 8 helpers.go:465 got unknown FailureType: UNKNOWN_FAILURE, MissingRegion: could not find region configuration
I have been following the guide https://cloud.google.com/storage-transfer/docs/s3-compatible
Source is non-Amazon S3-compatible storage service. Source has been tested with other S3 client tools (including gsutil).
Appears AWS SDK for Go is in use
https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/
The SDK requires a region to operate but does not have a default Region.
Amazon S3-compatible storage services (MinIO, Hitichai Content Platform) do not require a region.
I have tried passing AWS_REGION, AWS_DEFAULT_REGION environment variables to Agent running in Docker. Mounting ~/.aws/config in the container.
SDK may require both of these to be done
Set the AWS_REGION environment variable to the default Region
Set the region explicitly using config.WithRegion as an argument to config.LoadDefaultConfig when loading configuration.
Thanks
I want to use dev services to start postgres db on local application startup (quarkus profile = local).
Dev services are active in test and dev mode, so by not defining quarkus.datasource.jdbc.url it is actually working already with following config:
quarkus.datasource.db-kind=postgresql
quarkus.datasource.devservices.port=5432
quarkus.datasource.username=postgres
quarkus.datasource.password=postgres
However, I don't know how to configure this besides a prod configuration which has a quarkus.datasource.jdbc.url defined e.g.
%local.quarkus.datasource.devservices.enabled=true
%local.quarkus.datasource.db-kind=postgresql
%local.quarkus.datasource.devservices.port=5432
%local.quarkus.datasource.username=postgres
%local.quarkus.datasource.password=postgres
quarkus.datasource.db-kind=postgresql
quarkus.datasource.jdbc.url=jdbc:postgresql://myProdDB.com:5432/mydb
If I start the application with quarkus profile = local, instead of spinning up postgres on docker, quarkus falls back to the prod property quarkus.datasource.jdbc.url which I don't want for my local dev environment.
If I prefix the prod properties with a prod profile like %prod.quarkus.datasource.jdbc.url this fallback can be prevented, however, I want to follow the convention that prod properties are default, thus not prefixed with profile.
I already tried without success to somehow set %local.quarkus.datasource.jdbc.url to an empty value to prevent fallback to quarkus.datasource.jdbc.url:
%local.quarkus.datasource.jdbc.url="" -> java.sql.SQLException: Driver does not support the provided URL: ""
%local.quarkus.datasource.jdbc.url=null -> java.sql.SQLException: Driver does not support the provided URL: null
%local.quarkus.datasource.jdbc.url= -> io.quarkus.runtime.configuration.ConfigurationException: Model classes are defined for the default persistence unit default but configured datasource default not found: the default EntityManagerFactory will not be created. To solve this, configure the default datasource. Refer to https://quarkus.io/guides/datasource for guidance.
I am using ASP.NET Core 5.0 and I have a Web API app deployed to internal cloud where few settings like DB are controlled via environment variables on the host cloud. In my Startup.cs I have the below code
string projectDbConnection = Configuration.GetSection("ProjectDatabaseSettings").GetValue<string>("PROJECT_DB_CONNECTION");
string projectDbName = Configuration.GetSection("ProjectDatabaseSettings").GetValue<string>("PROJECT_DB_NAME");
Here as I understand, when running locally in IIS Express it looks for appsettings.<Environment>.json and they take precedence over appsettings.json values.
But this app is always connecting to the wrong DB when I deployed to Cloud where I mentioned the PROJECT_DB_CONNECTION & PROJECT_DB_NAME as Environment variables for the app.
To make the app read from the Environment variables I had to change the above Code in Startup.cs as
string projectDbConnection = Configuration.GetValue<string>("PROJECT_DB_CONNECTION");
string projectDbName = Configuration.GetValue<string>("PROJECT_DB_NAME");
I am unable to understand the difference between the GetSection.GetValue and just GetValue and why I should use Configuration.GetValue() to direct app to read from Env variables.
what am I missing and when should we use what?
Naming of environment variables
There is kind of a naming convention in the environment variables for nested appsettings to env vars, see naming of environment variables.
Each element in the hierarchy is separated by a double underscore.
In your case it would work if you name the env variable: ProjectDatabaseSettings__PROJECT_DB_CONNECTION.
Config Order
Regarding to Microsoft Documentation there is a order in which the config sources are checked.
ChainedConfigurationProvider : Adds an existing IConfiguration as a source. In the default configuration case, adds the host configuration and setting it as the first source for the app configuration.
appsettings.json using the JSON configuration provider.
appsettings.Environment.json using the JSON configuration provider. For example, appsettings.Production.json and appsettings.Development.json.
App secrets when the app runs in the Development environment.
Environment variables using the Environment Variables configuration provider.
Command-line arguments using the Command-line configuration provider.
The usecase
This is useful when you are developing local using appsettings.json, but run in a cluster or cloud in production where it is more convenient to use environment variables (f.e.: in kubernetes environment variables are set via config maps).
I have created a ASP.NET Core WebAPI and deployed in an Dev Environment (Kubernetes) using Azure Pipelines. How can I update the configuration in the pipeline if I need to publish the same API in another environment (eg. SIT). Since I have different settings/configuration for Dev and SIT environments.
Kindly guide me.
You can use release variables to do this. Feel free to reachout if you need any assistance.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch
https://learn.microsoft.com/en-us/azure/devops/pipelines/release/?view=azure-devops#how-do-i-specify-variables-i-want-to-edit-when-a-release-is-created
The problem occurs when I want to deploy the same API to other
environments like QA/UAT/Prod. Since each environment is having
separate databases
For this issue , there are several ways to achieve this. You can add Replace Tokens extension to the job to replace the database connection string in appsettings.json.
You can define your variable like below:
{
"ConnectionStrings": {
"DefaultConnection": "#{connectstring}#"
}
}
You can refer to this case and lab for details.
Here are some reference for the same issue:
Replacing database connection strings in the Docker image
Set Json Property task to replace the ConnectionStrings