For dev containers, it is possible to either provide env variables directly in decontainer.json or to provide a decontainer.env file. I wish to provide most of the variables via the env file but be prompted for input for a few sensitive ones to be available inside the container. This can be achieved for launch.json files using VAR="{input:var_id}". I could not find a similar thing for devcontainer.json. I tried asking for user input in postCreateCommand and exporting it but it was not available when I attached a shell to the container.
Related
I have a Jenkins job and in that job the first thing that runs is a powershell script that I want to capture user inputs values and set them as global variables that are used through out the Jenkins job.
Now i want the user to be able to put these values in from their machine and then run the job with these values ?
How can i do this ?
EDIT: In case anybody else finds this answer. Please see the comments below. This should not be used for credentials! As the communication can be secured by TLS, the credentials will still be visible in build logs etc.
You need to check the This project is parameterized checkbox in the settings of your job in Jenkins. Then define the name, type etc.
The given name is already accessible via standard syntax.
In shell script ${nameOfParam} or %nameOfParam (depending on your shell / os).
In pipelines they are also accessible via params.nameOfParam.
You can set these variables via GUI using Build with parameters or via API call http://<JENKINS_URL>/job/<JOB_NAME>/buildWithParameters/nameOfParam=foo
See also: https://www.baeldung.com/ops/jenkins-parameterized-builds
Only thing I quiet don't get from your question is, what you exactly want to do with the powershell script. A pipeline script in Jenkins is executed on a node, so if the job starts it should be running without any user interaction. To set values from the user input as global variables in a powershell script, you already need to have them available within the jenkins node, hence it's nonsense to set them in the powershell script because they are already available.
I use cloud storage (eg gcs, s3) in addition to the local file system for data analysis.
My question is: are there tools that enable tab completion (in a shell environment) for file paths or URIs that aren't on local, mounted file systems? Eg for a file URI like gs://path/to/file.txt, I'd like to have tab completion when typing part of the path.
Note: I don't want to use FUSE (or mount a file system or volume in some way). I'm wondering if there are bash or zsh extensions that enable this functionality for non-file-system URIs, presumably with API calls in the background or something.
gcloud interactive shell has auto-complete and auto prompting for any command that has a manual page, including the gcloud, bq, gsutil, and kubectl command-line tools.
To enable the gcloud interactive shell:
Install the gcloud beta components first by running the command gcloud components install beta.
Run gcloud beta interactive to enter gcloud interactive mode.
You can now use the tab to complete a file path or resource assignment
Check this link for the official documentation of gcloud auto-complete.
I have to think this is a solved issue but I am just not getting it to work. So I have come to you StackOverflow with this issue:
I have a windows server 2016 machine running in amazon ec2. I have a machine.ps1 script in a config directory.
I create an image of the box. (I have tried with checking noreboot and unchecking it)
When I create a new instance of the image I want it to run machine.ps1 at launch to set the computer name and then set routes and some config settings for the box. The goal is to do this without logging into the box.
I have read and tried:
Running Powershell scripts at Start up
and used this to ensure user data was getting passed in:
EC2 Powershell Launch Tools
I have tried setting up a scheduled task that runs the machine.ps1 on start up (It just hangs)
I see the initializeInstance.ps1 on start up task and have tried to even coop that replacing the line to run userdata with the line to run my script. Nothing.
If I log into the box and run machine.ps1, it will restart the computer and set the computer name and then I need to run it once more to set routes. This works manually. I just need to find a way to do it automagically.
I want to launch these instances from powershell not with launch configurations and auto scale.
You can use User data
Whenever you deploy a new server, workstation or virtual machine there is nearly always a requirement to make final changes to the system before it’s ready for use. Typically this is normally done with a post-deployment script that might be triggered manually on start-up or it might be a final step in a Configuration Manager task sequence or if you using Azure you may use the Custom Script Extension. So how do you achieve similar functionality using EC2 instances in Amazon Web Services (AWS)? If you’ve created your own Amazon Machine Image (AMI) you can set the script to run from the Runonce registry key, but then can be a cumbersome approach particularly if you want to make changes to the script and it’s been embedded into the image. AWS offers a much more dynamic method of injecting a script to run upon start-up through a feature called user data.
Please refer following link for ther same:
Poershell User data
Windows typically won't let a powershell script call another powershell script unless it is being run as Administrator. It is a weird 'safety' feature. But it is perfectly okay to load the ps1 files and use any functions inside them.
The UserData script is typically run as "system". You would THINK that would pass muster. But it fails...
The SOLUTION: Make ALL of your scripts into powershell functions instead.
In your machine.ps1 - wrap the contents with function syntax
function MyDescriptiveName { <original script contents> }
Then in UserData - use the functions like this
# To use a relative path
Set-Location -Path <my location>
# Load script file into process memory
. <full-or-relpath>/machine.ps1
# Call function
MyDescriptiveName <params-if-applicable>
If the function needs to call other functions (aka scripts), you'll need to make those scripts into functions and load the script file into process memory in UserData also.
When running a guest executable in Service Fabric I have noticed that some environment variables do not seem to be mapped to where I would expect them to be.
A few examples of these are that %appdata% didnt resolve to the usual:
C:\Users\\AppData\Roaming
but instead resolved to somewhere deep inside C:/windows
I have also noticed that when running applications using Erlang that the '.erlang.cookie' file is usually placed in the user root:
C:\Users\.erlang.cookie
but instead is trying to be created in C:\Windows
Is there a reason to why these are changed in these ways and currently I am having to make the guest executable not use 'appdata' and grant it administrative privileges using a policy in the application manifest to give it write access to C:\windows to write the '.erlang.cookie'.
That's because services run under the NETWORKSERVICE account by default.
You can do runas, or use the configuration system for settings.
https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-application-runas-security
https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-reliable-services-configuration
We are dockerizing an application (written in Node.js) that will need to access some sensitive data at run-time (API tokens for different services) and I can't find any recommended approach to deal with that.
Some information:
The sensitive information is not in our codebase, but it's kept on another repository in encrypted format.
On our current deployment, without Docker, we update the codebase with git, and then we manually copy the sensitive information via SSH.
The docker images will be stored in a private, self-hosted registry
I can think of some different approaches, but all of them have some drawbacks:
Include the sensitive information in the Docker images at build time. This is certainly the easiest one; however, it makes them available to anyone with access to the image (I don't know if we should trust the registry that much).
Like 1, but having the credentials in a data-only image.
Create a volume in the image that links to a directory in the host system, and manually copy the credentials over SSH like we're doing right now. This is very convenient too, but then we can't spin up new servers easily (maybe we could use something like etcd to synchronize them?)
Pass the information as environment variables. However, we have 5 different pairs of API credentials right now, which makes this a bit inconvenient. Most importantly, however, we would need to keep another copy of the sensitive information in the configuration scripts (the commands that will be executed to run Docker images), and this can easily create problems (e.g. credentials accidentally included in git, etc).
PS: I've done some research but couldn't find anything similar to my problem. Other questions (like this one) were about sensitive information needed at build-time; in our case, we need the information at run-time
I've used your options 3 and 4 to solve this in the past. To rephrase/elaborate:
Create a volume in the image that links to a directory in the host system, and manually copy the credentials over SSH like we're doing right now.
I use config management (Chef or Ansible) to set up the credentials on the host. If the app takes a config file needing API tokens or database credentials, I use config management to create that file from a template. Chef can read the credentials from encrypted data bag or attributes, set up the files on the host, then start the container with a volume just like you describe.
Note that in the container you may need a wrapper to run the app. The wrapper copies the config file from whatever the volume is mounted to wherever the application expects it, then starts the app.
Pass the information as environment variables. However, we have 5 different pairs of API credentials right now, which makes this a bit inconvenient. Most importantly, however, we would need to keep another copy of the sensitive information in the configuration scripts (the commands that will be executed to run Docker images), and this can easily create problems (e.g. credentials accidentally included in git, etc).
Yes, it's cumbersome to pass a bunch of env variables using -e key=value syntax, but this is how I prefer to do it. Remember the variables are still exposed to anyone with access to the Docker daemon. If your docker run command is composed programmatically it's easier.
If not, use the --env-file flag as discussed here in the Docker docs. You create a file with key=value pairs, then run a container using that file.
$ cat >> myenv << END
FOO=BAR
BAR=BAZ
END
$ docker run --env-file myenv
That myenv file can be created using chef/config management as described above.
If you're hosting on AWS you can leverage KMS here. Keep either the env file or the config file (that is passed to the container in a volume) encrypted via KMS. In the container, use a wrapper script to call out to KMS, decrypt the file, move it in to place and start the app. This way the config data is not exposed on disk.