Puppet onlyif and unless conditional test from boolean data in Hiera and CLI script output - diff

I am running Puppet v3.0 on RHEL 6 and am doing package management via the exec resource.
I would like to add a number of control gates into my manifest via onlyif and unless.
First I would like to use booleans as defined in Hiera [ auto lookup function ].
Secondly I would like to use booleans from a bash script running diff <() <().
Im using the following hiera data :
---
my-class::package::patch_now:
0
my-class::package::package_list:
acl-2.2.49-6.el6-x86_64
acpid-1.0.10-2.1.el6-x86_64
...etc
and my manifest are as follows :
# less package.pp
class my-classs::package(
$package_list,
$patch_now,
){
exec {'patch_packages':
provider => shell,
path => [ "/bin/", "/usr/bin/" ],
logoutput => true,
timeout => 100,
command => "yum update -e0 -d0 -y $package_list",
unless => "/path/to/my-diff.script 2>&1 > /dev/null",
onlyif => "test 0 -eq $patch_now",
}
}
How would I test the booleans (0|1) from Hiera and a CLI diff.script with unless and onlyif in the context above ?

I'm assuming that you mean to install all listed packages in one sweep if $patch_now is set.
You should not test for that using onlyif. That is meant to verify some state on the agent system. If the master is aware of your data, you should use conditionals in the manifest structure.
if $patch_now {
exec { ... }
}
But do use true and false instead of 1 and 0 as the value for the flag - both 1 and 0 are equal to true in boolean context!
Your YAML looks funny, anyway.
To define a single value:
my-class::package::patch_now: false
To define an array:
my-class::package::package_list:
- acl-2.2.49-6.el6-x86_64
- acpid-1.0.10-2.1.el6-x86_64
- ...
When you use the array in your class, you cannot just put it in a string such as "yum update -e0 -d0 -y $package_list", for that will expand to "yum update -e0 -d0 -y acl-2.2.49-6.el6-x86_64acpid-1.0.10-2.1.el6-x86_64...", without spaces between the elements.
To concatenate the elements with spaces, use the join function from the stdlib
module.
$packages = join($package_list, ' ')
...
"yum update -e0 -d0 -y $packages"
I honestly don't get how your diff <() <() is supposed to work. The whole approach looks a little convoluted. I suspect that with a little tweaking, your diff script could probably perform the updates on its own (so that the exec just runs this script with different parameters).
EDIT after receiving more info in your comment.
To make this work cleanly, I recommend the following.
have Puppet transfer your Hiera data to the agent
file { '/opt/wanted-packages': content => inline_template('<%= package_list * "\n" %>') }
The diff will then work like you suggested, only simpler.
diff /opt/wanted-packages <(facter ...)
Just make sure that the exec requires the file and you should be fine.

Related

Passing parameters to puppet manifest via command line

I have been searching for an answer to this question with no luck, but is there a way to pass parameters into puppet manifests when running the 'apply' command, in a similar way to the way you pass parameters when running a UNIX script on the command line?
The suggestions I see mention either keeping variables at the top of the manifest for use later, or to store them in a hiera file. But neither really answer the question I am posing?
Any guidance on how to do this would be greatly appreciated?
Edit:
An example of what I have been doing is:
$doc_root = "/var/www/example"
exec { 'apt-get update':
command => '/usr/bin/apt-get update'
}
package { 'apache2':
ensure => "installed",
require => Exec['apt-get update']
}
file { $doc_root:
ensure => "directory",
owner => "www-data",
group => "www-data",
mode => 644
}
file { "$doc_root/index.html":
ensure => "present",
source => "puppet:///modules/main/index.html",
require => File[$doc_root]
}
As you can see the variable is hardcoded at the top, whereas whilst I am trying to use the variable in the same way, I need to be able to pass the value in when running the apply command.
Using lookup functions in conjunction with hiera.yaml files doesn't fulfil my requirements for the same reason.
The only thing I can think may be a work around is to create a UNIX script that accepts parameters, saves those values in a yaml file, and then have the script execute the .pp file.
But I'm hoping that puppet has a way to do this directly.
The common procedure for passing variables into a classless manifest for use with the puppet apply subcommand would be to assign the value to a Facter fact from the CLI, and then resolve its value inside the manifest. You would begin with removing the hardcoded variable doc_root from the head of the manifest. Then, you would modify the variable into a fact like:
file { $facts['doc_root']:
...
file { "${facts['doc_root']}/index.html":
...
require => File["${facts['doc_root']}"] <-- interpolation required due to Puppet DSL inability to resolve hash value as first class expression
You would then pass the Facter value from the puppet apply subcommand like:
FACTER_doc_root=/var/www/example puppet apply manifest.pp
Note this also causes FACTER_doc_root to be temporarily set as an environment variable as a side effect.

jboss-cli : How do I read one specific system property using jboss-cli?

I'm new to jboss-cli and working through the 'jboss-cli recipes'.
Question
How do I read one specific property using jboss-cli? E.g.
jboss.home.dir (e.g. "-Djboss.home.dir=/path/to/my/jboss")
Xmx ("-Xmx=4G")
Context
The "CLI Recipes" documentation has this helpful example to get all system properties. However its 'too much infomration'. I want to script reading one specific property.
https://docs.jboss.org/author/display/WFLY10/CLI+Recipes#CLIRecipes-
Overview of all system properties in JBoss AS7+ including OS system
properties and properties specified on command line using -D, -P or
--properties arguments.
Standalone
[standalone#IP_ADDRESS:9999 /] /core-service=platform-mbean/type=runtime:read-attribute(name=system-properties)
Thanks in advance
You could do a :
:resolve-expression(expression=${jboss.home.dir})
You can use the cli like this:
$JBOSS_HOME/bin/jboss-cli.sh -c --command=/system-property=MY_PROPERTY:read-resource
you get an output like this:
$JBOSS_HOME/bin/jboss-cli.sh -c --command=/system-property=MY_PROPERTY:read-resource
{
"outcome" => "success",
"result" => {"value" => "4.0"}
}
which you can extract by piping into something like this:
<cli command> | grep "{\"value\"" | sed "s/.*value\" => \"\([^\"]*\)\".*/\1/"
its a bit ugly, and there are some nasty edge cases if the values were to be something like "value" => "value =" or something hideous.
In general this works OK.
Change the sed command to be a bit more specific to fix that.
This link pointed me to the answer: I can use a groovy script to get the values. From what I see the "jboss-cli command line" does not offer this flexibility.
https://developer.jboss.org/wiki/AdvancedCLIScriptingWithGroovyRhinoJythonEtc
Solution
Here's a solution for jboss home.
[For memory you can get results from "/core-service=platform-mbean/type=memory/:read-attribute(name=heap-memory-usage)"
bash
#!/bin/sh
# Note: must set jbbin to 'jboss home /bin'
groovy -cp $jbbin/client/jboss-cli-client.jar readJbossHome.groovy
Groovy
Note: this is 'quick and dirty'.
import org.jboss.as.cli.scriptsupport.*
cli = CLI.newInstance()
cli.connect()
// Define properties
myParentProp="system-properties"
myProp="jboss.home.dir"
// Retrieve and pluck values
result = cli.cmd("/core-service=platform-mbean/type=runtime:read-resource(recursive=true,include-runtime=false)")
myResult = result.getResponse().get("result")
myParentVal = myResult.get(myParentProp)
myVal = myParentVal.get(myProp)
// Print out results
println "Property detail ${myProp} is ${myVal}"
cli.disconnect()
You can also do it via Wildfly management rest call.
http://localhost:9990/management
POST
Headers = Content-Type:application/json
Body =
{
"operation":"resolve-expression",
"expression":"${jboss.home.dir}"
}
With newer Teiid DOCs I have found some useful information I thought this might be helpful to share to people coming across a similar usecase
https://access.redhat.com/documentation/en-us/jboss_enterprise_application_platform/6.3/html/administration_and_configuration_guide/configure_system_properties_using_the_management_cli
Helps Adding, Removing & Reading System Properties with jboss-cli
jboss-cli
If you have a cli command like ehsavoie suggested :resolve-expression(expression=${jboss.home.dir}) and want to use the content of the "result" property within jboss-cli you can save it in a variable. You can use backticks (`) to evaluate expressions.
simple expression
[standalone#localhost:9990 /] :resolve-expression(expression=${jboss.home.dir})
{
"outcome" => "success",
"result" => "/home/user/wildfly"
}
use in valiable
[standalone#localhost:9990 /] set wildflydirectory=`:resolve-expression(expression=${jboss.home.dir})`
[standalone#localhost:9990 /] echo $wildflydirectory
/home/user/wildfly
PowerShell
If you happen to use the PowerShell you can use a one-liner to extract even deeply nested results with the help of the cli's --output-json option and PowerShell's ConvertFrom-Json cmdlet. In this way the parsing problem from James Roberts's approach with grep and sed are gone.
$value=(Invoke-Expression "./jboss-cli.ps1 -c --command=':resolve-expression(expression=`${jboss.home.dir})' --output-json" | ConvertFrom-Json).result
It is a bit tricky to quote the command and escape the correct PowerShell special characters.

Using __END__ and DATA in Chef recipes (to run legacy shell scripts)

I'm migrating some shell scripts to Chef recipes. Some of these scripts are fairly involved, so just to make life easier in the short term and to avoid introducing bugs in rewriting everything in Chef/Ruby, I'd like to just run some of them as-is. They're all well-written and idempotent, so honestly there's no rush, but of course, the eventual goal is to rewrite them.
One cool feature of Ruby is its __END__ keyword/method: Lines below __END__ will not be executed. Those lines will be available via the special filehandle DATA.
It would be cool to ship the shell scripts as-is inside the the recipe after __END__, maybe something like the following, which I placed in chef-repo/cookbooks/ruby-data-test/recipes/default.rb:
file = Tempfile.new(File.basename(__FILE__))
file << DATA.read
bash file.path
file.unlink
__END__
echo "Hello, world"
However when I run this (with chef-solo -c solo.rb --override-runlist 'recipe[ruby-data-test]'), I get the following error:
[2014-10-03T17:14:56+00:00] ERROR: uninitialized constant Chef::Recipe::DATA
I'm pretty new to Chef, but I'm guessing the above is something about Chef wrapping my recipe in a class, and there's something simple preventing me from accessing DATA. Since it's "global" (?) I tried putting a dollar sign ($DATA) in front of it but that failed with:
NoMethodError
-------------
undefined method `read' for nil:NilClass
So the question is: How do I access DATA in my Chef recipe? Thanks!
It appears you don't have access to DATA, but you can fake it by reading in the current file yourself and splitting on __END__, like Sinatra does.
I ended up making a Chef LWRP for reuse. I don't know if I'll actually end up using this, but I wanted to figure it out. Like I said, I'm a Chef/Ruby noob, so any better ideas or suggestions welcome!
ruby_data_test/recipes/default.rb:
ruby_data_test_execute_ruby_data __FILE__
__END__
#!/bin/bash
set -o errexit
date
echo "Hello, world"
ruby_data_test/resources/execute_ruby_data.rb:
actions :execute_ruby_data
default_action :execute_ruby_data
attribute :source, :name_attribute => true, :required => true
attribute :args, :kind_of => Array
attribute :ignore_errors, :kind_of => [TrueClass, FalseClass], :default => false
ruby_data_test/providers/execute_ruby_data.rb:
def whyrun_supported?
true
end
use_inline_resources
action :execute_ruby_data do
converge_by("Executing #{#new_resource}") do
Chef::Log.info("Executing #{#new_resource}")
file_who_called_me = #new_resource.source
io = ::IO.respond_to?(:binread) ? ::IO.binread(file_who_called_me) : ::IO.read(file_who_called_me)
app, data = io.gsub("\r\n", "\n").split(/^__END__$/, 2)
data.lstrip!
file = Tempfile.new('execute_ruby_data')
file << data
file.chmod(0755)
file.close
exit_status = ::Open3.popen2e(file.path, *#new_resource.args) do |stdin, stdout_and_stderr, wait_thr|
stdout_and_stderr.each { |line| puts line }
wait_thr.value # exit status
end
if exit_status != 0 && !#new_resource.ignore_errors
throw RuntimeError
end
end
end
Here's the output:
$ chef-solo -c solo.rb --override-runlist 'recipe[ruby_data_test]'
Starting Chef Client, version 11.12.4
[2014-10-03T21:50:29+00:00] WARN: Run List override has been provided.
[2014-10-03T21:50:29+00:00] WARN: Original Run List: []
[2014-10-03T21:50:29+00:00] WARN: Overridden Run List: [recipe[ruby_data_test]]
Compiling Cookbooks...
Converging 1 resources
Recipe: ruby_data_test::default
* ruby_data_test_execute_ruby_data[/root/chef/chef-repo/cookbooks/ruby_data_test/recipes/default.rb] action execute_ruby_dataFri Oct 3 21:50:29 UTC 2014
Hello, world
- Executing ruby_data_test_execute_ruby_data[/root/chef/chef-repo/cookbooks/ruby_data_test/recipes/default.rb]
Running handlers:
Running handlers complete
Chef Client finished, 1/1 resources updated in 1.387608 seconds

Insecure dependency with Inline::Python

What could explain this compile-time error message when running Inline::Python in -T mode?
Insecure dependency in open while running with -T switch at /usr/local/lib/perl/5.14.2/Inline/Python.pm line 193.
Line 193 is where Inline::Python opens $o->{API}{location}, which I take to be the "Inline DIRECTORY".
I have, of course, used the required options:
use constant _INLINE_DIR_ => '/var/myapp/inline';
use Inline Config => UNTAINT => 1,
NO_UNTAINT_WARN => 1,
DIRECTORY => _INLINE_DIR_;
I have made sure that /var/myapp/inline and everything inside it is writable by everyone, obviously including both root and the user that the application is setuid'ed to at run-time.
The very same script works without problem on my computer, whether I start it as root or not, running Inline 0.50 Inline::Python 0.43, but gives me this error when I try running it on a server that uses the same version of Inline::Python and either version 0.49 or 0.55 of Inline.
Since this is different in different environments, my bet is that somehow there's an environment variable that either Inline or Inline::Python is reading before it does the step requested by the UNTAINT config parameter.
(Contrary to the comment, I don't think that file permissions could cause this message, only insecure dependencies on command-line parameters or environment variables)
Given that, I'd start your script by forcibly clearing the environment and then adding in only those environmental variables you know you need:
%ENV = ();
$ENV{'PATH'} = '/bin:/usr/bin:/usr/local/bin'; # Or whatever's appropriate
$ENV{'PYTHONPATH'} = '/usr/local/lib/python'; # Optional, if appropriate
# ... etc ...

How to change the order of rsync in symfony deployment task

I want to deploy a part of my symfony application, say, it's like a module.
I want to exclude all files first, and then include only the files of
my new module.
For deployment I use the following symfony task
php symfony project:deploy production -t
The parameter -t prints all files to the output that are included in this dry run of rsync.
Content of config/rsync_exclude.txt is only *, since I like to exclude everthing:
*
In config/rsync_include.txt I list all the files and folders for the inclusion:
config/
config/mysupermodule.yml
lib/model/doctrine/
lib/model/doctrine/MySuperclass.php
lib/model/doctrine/MySuperclassTable.php
lib/
lib/MySuperLibrary/
lib/MySuperLibrary/*
The symfony task builds the following rsync command:
rsync --dry-run -azC --force --delete --progress --exclude-from=config/rsync_exclude.txt --include-from=config/rsync_include.txt -e "ssh -p22" ./ user#www.server.com:/test_deployment/
Problem 1: The the task doesn't sync any files.
Solution to 1: Change order: Include first, then exclude.
I figured out, that if I change my need to this one:
I want to include all files of my new module and exclude then all
other.
This means using the following command:
rsync --dry-run -azC --force --delete --progress --include-from=config/rsync_include.txt --exclude-from=config/rsync_exclude.txt -e "ssh -p22" ./ user#www.server.com:/test_deployment/
The rsync works.
Problem 2: How can I change the order of the rsync when using the symfony task?
The symfony task first excludes than includes.
Solution 2: ?
It is NOT possible.
But you can edit the deployment task in lib/task/project/sfProjectDeployTask.class.php.
Replace this (line 145 to 154 in SF 1.4):
if (file_exists($options['rsync-dir'].'/rsync_exclude.txt'))
{
$parameters .= sprintf(' --exclude-from=%s/rsync_exclude.txt', $options['rsync-dir']);
}
if (file_exists($options['rsync-dir'].'/rsync_include.txt'))
{
$parameters .= sprintf(' --include-from=%s/rsync_include.txt', $options['rsync-dir']);
}
with this:
if (file_exists($options['rsync-dir'].'/rsync_include.txt'))
{
$parameters .= sprintf(' --include-from=%s/rsync_include.txt', $options['rsync-dir']);
}
if (file_exists($options['rsync-dir'].'/rsync_exclude.txt'))
{
$parameters .= sprintf(' --exclude-from=%s/rsync_exclude.txt', $options['rsync-dir']);
}
In short: switch this two IF statements.
Let's change the way you want to do.
You should use only the exclude file. Exclude only directories that changed but you don't want to sync.
Because anyway if you modules/, app/, ... directories haven't change, you don't have to put them in the exclude file because they will remain the same on both server.