dotcloud supervisord.conf file environment specification - dotcloud

http://docs.dotcloud.com/guides/daemons/ states:
Configuring The Environment
You can easily modify the environment of execution of your daemon with the “directory” and “environment” directives to change the directory where the command is executed and to define additional environment variable. For example:
[program:daemonname]
command = php my_daemon.php
directory = /home/dotcloud/current/
environment = QUEUE=*, VERBOSE=TRUE
However, I'm finding my PYTHONPATH environment variable is not being set:
dotcloud.yml:
www:
type: python
db:
type: postgresql
worker:
type: python-worker
supervisord.conf:
[program:apnsd]
command=/home/dotcloud/current/printenv.py
environment=PYTHONPATH=/home/dotcloud/current/apnsd/
printenv.py
#! /home/dotcloud/env/bin/python
import os
print "ENVIRONMENT"
print os.environ
the logs:
ENVIRONMENT
{'SUPERVISOR_ENABLED': '1', 'SUPERVISOR_SERVER_URL': 'unix:///var/dotcloud/super
visor.sock', 'VERBOSE': 'no', 'UPSTART_INSTANCE': '', 'PYTHONPATH': '/', 'PREVLE
VEL': 'N', 'UPSTART_EVENTS': 'runlevel', '/': '/', 'SUPERVISOR_PROCESS_NAME': 'a
pnsd', 'UPSTART_JOB': 'rc', 'PWD': '/', 'SUPERVISOR_GROUP_NAME': 'apnsd', 'RUNLE
VEL': '2', 'PATH': '/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
', 'runlevel': '2', 'previous': 'N'}
Do not show a modified python variable!

There is a bug in Supervisor; some variables (like those containing a /) have to be quoted.
In that case, you need:
[program:apnsd]
command=/home/dotcloud/current/printenv.py
environment= PYTHONPATH="/home/dotcloud/current/apnsd/"
(The space in = PYTHONPATH is not mandatory, it's just to make the file slightly more readable; the quotes around the value of PYTHONPATH are, however, required!)
I will update dotCloud's documentation to mention this issue.

Related

How to retrieve PACKAGECONFIG variable from another recipe to a bbclass

For one of my requirements, I need to call a specific tasks based on whether a packageconfig variable is defined in another recipes or not.
For example:
We have a recipe called recipes-crypto where, in the .bb file we have:
PACKAGECONFIG[veritysetup] = "--enable-veritysetup,--disable-veritysetup"
BBCLASSEXTEND = "native nativesdk"
Then, in my meta-qti-bsp/classes, I have qimage.class, where I wanted to do like this:
if ${#bb.utils.contains('PACKAGECONFIG', 'veritysetup', 'true', 'false', d)}; then
#Call some function
fi
But it gives errors:
ERROR: ParseError at /local/mnt/workspace/PINTU/WORK/Y2021/NAD-CORE-WORK/NEW_C10_30Nov/poky/meta-qti-bsp/classes/qimage.bbclass:102: unparsed line: 'if ${#bb.utils.contains('PACKAGECONFIG', 'veritysetup', 'true', 'false', d)}; then'
How to make veritysetup variable get recognised in my class file?
I saw some examples and added this on top:
PACKAGECONFIG_append_class-native = " veritysetup"
But with this also it gives the same error.
I am using this veritysetup command only during build time.
So, I wanted to execute this command if and only if this PACKAGECONFIG variable is defined.
What is the best way to do it ?
veritysetup is not a value of PACKAGECONFIG, but it is a flag.
PACKAGECONFIG has many flags and each flag has its value.
For more information about variable flags check this link.
So, here is an example of how to check if that flag is activated:
verity-example.bb
LICENSE = "CLOSED"
PACKAGECONFIG[veritysetup] = "--enable-veritysetup,--disable-veritysetup"
do_check_verity(){
if [ ${#d.getVarFlag('PACKAGECONFIG', 'veritysetup', False)} ]; then
bbwarn "veritysetup is activated with value: ${#d.getVarFlags('PACKAGECONFIG').get('veritysetup')}"
else
bbwarn "veritysetup is not activated."
fi
}
addtask do_check_verity
If you run:
bitbake verity-example -c check_verity
You will get the warning:
WARNING: verity-example-1.0-r0 do_sample: veritysetup is activated
with value: --enable-veritysetup,--disable-veritysetup
Actually, I did it in this way and it worked for me.
The following is already enabled in recipes-crypto like this:
PACKAGECONFIG[veritysetup] = "--enable-veritysetup,--disable-veritysetup"
Now, in our .bbclass I just called like this:
DEPENDS += "cryptsetup-native openssl-native"
PACKAGECONFIG_append = " veritysetup"
**==> This is the main part how we can check, if a packageconfig variable is enabled elsewhere or not**
Then I can check the condition like this:
if not bb.utils.contains('PACKAGECONFIG', 'veritysetup', True, False, d):
//dome something
else:
//done something else

Rundeck stop running steps based on global variable

I have a Rundeck job that executes multiple steps, each of which are Job References to other small jobs. The first step selects a server to upgrade, and sets a global variable with the server name. The remaining steps perform upgrade tasks. It is possible though for the first step to return NONE as the server name, and if that's the case I would like to halt execution right there without running the remaining steps, and I'd like the whole job to be marked as Successful.
I could just make that first job exit with an error code, but then the whole job looks failed, and it looks like there is something wrong with it, even though it successfully ran and found there was nothing to upgrade.
Any ideas? I'm finding "use a flow control step" everywhere, but I can't see how to make that work for my use case.
The best way to create complex workflows depending on some output value is to use the Ruleset Strategy (Rundeck Enterprise). Take a look at this.
On the community version you can save the result of the first step on a key-value variable and do some "script-fu" in the following steps:
Step 1: print the status and save it on a data variable using the key-value data log filter.
Steps 2,3,4: capture the key-value data and then the step can continue or not.
I made an example easy to import to your instance for testing:
- defaultTab: nodes
description: ''
executionEnabled: true
id: 27de501a-8bb2-4c6e-a5f9-0676e80ca75a
loglevel: INFO
name: HelloWorld
nodeFilterEditable: false
options:
- enforced: true
name: opt1
required: true
value: 'true'
values:
- 'true'
- 'false'
valuesListDelimiter: ','
plugins:
ExecutionLifecycle: null
scheduleEnabled: true
sequence:
commands:
- exec: echo "url=${option.opt1}"
plugins:
LogFilter:
- config:
invalidKeyPattern: \s|\$|\{|\}|\\
logData: 'true'
name: result
regex: .*=\s*(.+)$
type: key-value-data
- fileExtension: .sh
interpreterArgsQuoted: false
script: |-
# data/value evaluation
if [ "#data.result#" = "true" ]; then
echo "step two"
fi
scriptInterpreter: /bin/bash
- fileExtension: .sh
interpreterArgsQuoted: false
script: |-
# data/value evaluation
if [ "#data.result#" = "true" ]; then
echo "step three"
fi
scriptInterpreter: /bin/bash
- fileExtension: .sh
interpreterArgsQuoted: false
script: |-
# data/value evaluation
if [ "#data.result#" = "true" ]; then
echo "step four"
fi
scriptInterpreter: /bin/bash
keepgoing: false
strategy: node-first
uuid: 27de501a-8bb2-4c6e-a5f9-0676e80ca75a
MegaDrive68k's answer is what you can do best with the basic opensource version or if you have the Enterprise version.
But you can also create your own plugin or make a fork out of an existing one.
Which I did with the official flow control puglin and add conditions.
You can fork this plugin and add in the java code 2 new #PluginProperty (That add two new field in a plugin parameter in rundeck interface) and make a comparison of values.
Example:
#PluginProperty(title = "First Value", description = "Compare this", required = true)
String value1;
#PluginProperty(title = "Second Value", description = "To this", required = true)
String value2;
Comparison of Strings values (in your case it is)
if (value1.equals(value2)) {...}
Comparison of Numeric values
if (value1 == value2) {...}
If you want to stop the job with successful (it does not stop the parent job, just actual):
context.getFlowControl().Halt(true);
If you want to stop the job with a failed status:
context.getFlowControl().Halt(false);
If you want to stop the job with a customized status:
context.getFlowControl().Halt("MY CUSTOM STATUS");
And finally, if you want to continue and not stop:
context.getFlowControl().Continue();
So a complete example (add this to your public class):
#PluginProperty(title = "First Value", description = "Compare this", required = true)
String value1;
#PluginProperty(title = "Second Value", description = "To this", required = true)
String value2;
#Override
public void executeStep(final PluginStepContext context, final Map<String, Object> configuration)
throws StepException
{
if (value1.equals(value2)) {
//Halt actual JOB without failed
context.getFlowControl().Halt(true);
} else {
//Continue
context.getFlowControl().Continue();
}
}
Then create your jar file and place it in the libext folder.
Now you can add your custom step. Put your global var in the first field and "NONE" in the second field.
If global var contain "NONE" the job stop successful at this step.
If you call a job with this step from oterh job (parent), the parent job continue.
If you want you can use this fork plugin which already includes these modifications. Look like this

Passing environment variables to NOW

I am trying to pass firebase environment variables for deployment with now.
I have encoded these variables manually with base64 and added them to now with the following command:
now secrets add firebase_api_key_dev "mybase64string"
The encoded string was placed within speech marks ""
These are in my CLI tool and I can see them all using the list command:
now secrets ls
> 7 secrets found under project-name [499ms]
name created
firebase_api_key_dev 6d ago
firebase_auth_domain_dev 6d ago
...
In my firebase config, I am using the following code:
const config = {
apiKey: Buffer.from(process.env.FIREBASE_API_KEY, "base64").toString(),
authDomain: Buffer.from(process.env.FIREBASE_AUTH_DOMAIN,"base64").toString(),
...
}
In my now.json file I have the following code:
{
"env": {
"FIREBASE_API_KEY": "#firebase_api_key_dev",
"FIREBASE_AUTH_DOMAIN": "#firebase_auth_domain_dev",
...
}
}
Everything works fine in my local environment (when I run next) as I also have a .env file with these variables, yet when I deploy my code, I get the following error in my now console:
TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be one of type string, Buffer, ArrayBuffer, Array, or Array-like Object. Received type undefined
Does this indicate that my environment variables are not being read? What's the issue here? It looks like they don't exist at all
The solution was to replace my existing now.json with:
{
"build":{
"env": {
"FIREBASE_API_KEY": "#firebase_api_key",
"FIREBASE_AUTH_DOMAIN": "#firebase_auth_domain",
"FIREBASE_DATABASE_URL": "#firebase_database_url",
"FIREBASE_PROJECT_ID": "#firebase_project_id",
"FIREBASE_STORAGE_BUCKET": "#firebase_storage_bucket",
"FIREBASE_MESSAGING_SENDER_ID": "#firebase_messaging_sender_id",
"FIREBASE_APP_ID": "#firebase_app_id",
"FIREBASE_API_KEY_DEV": "#firebase_api_key_dev",
"FIREBASE_AUTH_DOMAIN_DEV": "#firebase_auth_domain_dev",
"FIREBASE_DATABASE_URL_DEV": "#firebase_database_url_dev",
"FIREBASE_PROJECT_ID_DEV": "#firebase_project_id_dev",
"FIREBASE_STORAGE_BUCKET_DEV": "#firebase_storage_bucket_dev",
"FIREBASE_MESSAGING_SENDER_ID_DEV": "#firebase_messaging_sender_id_dev",
"FIREBASE_APP_ID_DEV": "#firebase_app_id_dev"
}
}
}
I was missing the build header.
I had to contact ZEIT support to help me identify this issue.

How to filter files by extension with Perl File::RsyncP

I am using Perl library File::RsyncP.
This moment script copies all files, but I need to use
extension and later pattern for example 2019*.xml
I don´t know how to filter files by extension.
I have tried following     
I take a connection into localhost
I have
/home/raimo/A/SRC/srcDirectory/1.xml
/home/raimo/A/SRC/srcDirectory/2.xml
/home/raimo/A/SRC/srcDirectory/3.txt
here
I would like to copy only xml files into    
I have tried:
rsyncCmd => "/bin/rsync -avz --include '*.xml' srcDirectory destDirectory ", and I have tired rsyncCmd => "/bin/rsync -avz --include '*.xml' destDirectory srcDirectory "
/home/raimo/A/SRC/destDirectory
my $rs = File::RsyncP->new({
logLevel => 5,
rsyncCmd => "/bin/rsync --include '*.xml'", #check if possible to filter
rsyncArgs => [
"--numeric-ids",
"--perms",
"--owner",
"--group",
"--devices",
"--links",
"--ignore-times",
"--block-size=700",
"--relative",
"--recursive",
"--verbose"
],
});
...
# Receive files from remote srcDirectory to local destDirectory
# # by running rsyncCmd with rsyncArgs.
$rs->remoteStart(1, srcDirectory);

Puppet-hiera-Function lookup() did not find a value-Windows

I installed dsc module and added AD user to Domain controller using puppet. Code below works fine when hard-coding password as plain text. Is it possible somehow to encrypt those passwords.
I read that hiera-eyaml is solution for this so i encrypted password
[root#PUPPET puppet]# /opt/puppetlabs/puppet/bin/eyaml encrypt -p
Enter password: **********
string: ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAl/+uUACl6WpGAnA1sSqEuTp39SVYfHc7J0BMvC+a2C0YzQg1V]
Then stored that encrypted pass in /etc/common.eyaml file (specified in hiera config file)
/opt/puppetlabs/puppet/bin/eyaml edit /etc/common.eyaml
I can decrypt the file successfully:
/opt/puppetlabs/puppet/bin/eyaml decrypt -f /etc/common.eyaml
Then i specified encrypted pass to manifest file
/etc/puppetlabs/code/environments/production/manifests/site.pp:
dsc_xADUser {'FirstUser':
dsc_ensure => 'present',
dsc_domainname => 'ad.contoso.com',
dsc_username => 'tfl',
dsc_userprincipalname => 'tfl#ad.contoso.com',
dsc_password => {
'user' => 'Administrator#ad.contoso.com',
'password' => Sensitive('pass')
},
dsc_passwordneverexpires => true,
dsc_domainadministratorcredential => {
'user' => 'Administrator#ad.contoso.com',
'password' => Sensitive(lookup('password'))
},
}
On windows node i got error
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Function lookup() did not find a value for the name 'password' on node windows.example.com
Hiera config file:
cat /etc/puppetlabs/puppet/hiera.yaml
---
# Hiera 5 Global configuration file
---
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
- name: "Eyaml hierarchy"
lookup_key: eyaml_lookup_key # eyaml backend
paths:
- "/etc/common.eyaml"
options:
pkcs7_private_key: "/etc/puppetlabs/puppet/keys/private_key.pkcs7.pem"
pkcs7_public_key: "/etc/puppetlabs/puppet/keys/public_key.pkcs7.pem"
cat /etc/common.eyaml
password: ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAl/+uUACl6WpGAnA1sSqEuTp39SVYfHc7J0BMvC+a2C0YzQg1V]
I'm new to Puppet and this hiera is confusing me
For starters, there is a typo in your Hiera config file. The path to the data should be:
paths:
- "/etc/common.eyaml"
After fixing that, you need to retrieve the value from Hiera. This is performed with the puppet lookup function. Since you have a single key value pair here in a single data file, this can be performed with a minimal number of arguments.
dsc_xADUser {'FirstUser':
dsc_ensure => 'present',
dsc_domainname => 'ad.contoso.com',
dsc_username => 'tfl',
dsc_userprincipalname => 'tfl#ad.contoso.com',
dsc_password => {
'user' => 'Administrator#ad.contoso.com',
'password' => Sensitive('pass')
},
dsc_passwordneverexpires => true,
dsc_domainadministratorcredential => {
'user' => 'Administrator#ad.contoso.com',
'password' => lookup('string'),
},
}
However, you also really want to redact that password from your logs and reports. You would want to wrap that password String in a Sensitive data type.
'password' => Sensitive(lookup('string')),
You seem to already be doing that for your other password that is being passed in as a String pass.
A side note to all of this is that Puppet has intrinsic support for lookup retrievals from Vault and Conjur in version 6, so that will become best practices instead of hiera-eyaml soon.
Ufff, after much struggling finally got it working:
cat /etc/puppetlabs/puppet/hiera.yaml
---
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
- name: "Eyaml hierarchy"
lookup_key: eyaml_lookup_key # eyaml backend
paths:
- "nodes/%{trusted.certname}.yaml"
- "windowspass.eyaml"
options:
pkcs7_private_key: "/etc/puppetlabs/puppet/keys/private_key.pkcs7.pem"
pkcs7_public_key: "/etc/puppetlabs/puppet/keys/public_key.pkcs7.pem
Created password:
/opt/puppetlabs/puppet/bin/eyaml encrypt -l 'password' -s 'Pass' --pkcs7-public-key=/etc/puppetlabs/puppet/keys/public_key.pkcs7.pem --pkcs7-private-key=/etc/puppetlabs/puppet/keys/private_key.pkcs7.pem
Added it to /etc/puppetlabs/puppet/data/windowspass.eyaml file:
/opt/puppetlabs/puppet/bin/eyaml edit windowspass.eyaml --pkcs7-public-key=/etc/puppetlabs/puppet/keys/public_key.pkcs7.pem --pkcs7-private-key=/etc/puppetlabs/puppet/keys/private_key.pkcs7.pem
cat /etc/puppetlabs/puppet/data/windowspass.eyaml
---
password: ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAUopetXenh/+DN1+VesIZUI5y4k3kOTn2xa5uBrtGZP3GvGqoWfwAbYsfeNApjeMG+lg93/N/6mE9T59DPh]
Tested decryption:
/opt/puppetlabs/puppet/bin/eyaml decrypt -f windowspass.eyaml --pkcs7-public-key=/etc/puppetlabs/puppet/keys/public_key.pkcs7.pem --pkcs7-private-key=/etc/puppetlabs/puppet/keys/private_key.pkcs7.pem
As Matt suggested, mapped content of windowspass.eyaml to manifest file
'password' => Sensitive(lookup('password'))
Debugging command helped me a lot:
puppet master --debug --compile windows.example.com --environment=production
Thanks everyone, especially to Matt