I want to write an Inspec Profile, in which I can add an input to an existing string.
For example, something like:
inspec exec test --input user=abcinst
Input: abcinst
Test: its('home') { should eq '/db2/{input}' }
I didn't find anything yet to this topic.
for inputs you can refer : https://docs.chef.io/inspec/inputs/
you have to use input resource to read the argument passed to
Related
I have a pyspark job available on GCP Dataproc to be triggered on airflow as shown below:
config = help.loadJSON("batch/config_file")
MY_PYSPARK_JOB = {
"reference": {"project_id": "my_project_id"},
"placement": {"cluster_name": "my_cluster_name"},
"pyspark_job": {
"main_python_file_uri": "gs://file/loc/my_spark_file.py"]
"properties": config["spark_properties"]
"args": <TO_BE_ADDED>
},
}
I need to supply command line arguments to this pyspark job as show below [this is how I am running my pyspark job from command line]:
spark-submit gs://file/loc/my_spark_file.py --arg1 val1 --arg2 val2
I am providing the arguments to my pyspark job using "configparser". Therefore, arg1 is the key and val1 is the value from my spark-submit commant above.
How do I define the "args" param in the "MY_PYSPARK_JOB" defined above [equivalent to my command line arguments]?
I finally managed to solve this conundrum.
If we are making use of ConfigParser, the key has to be specified as below [irrespective of whether the argument is being passed as command or on airflow]:
--arg1
In airflow, the configs are passed as a Sequence[str] (as mentioned by #Betjens below) and each argument is defined as follows:
arg1=val1
Therefore, as per my requirement, command line arguments are defined as depicted below:
"args": ["--arg1=val1",
"--arg2=val2"]
PS: Thank you #Betjens for all your suggestions.
You have to pass a Sequence[str]. If you check DataprocSubmitJobOperator you will see that the params job implements a class google.cloud.dataproc_v1.types.Job.
class DataprocSubmitJobOperator(BaseOperator):
...
:param job: Required. The job resource. If a dict is provided, it must be of the same form as the protobuf message.
:class:`~google.cloud.dataproc_v1.types.Job`
So, on the section about job type pySpark which is google.cloud.dataproc_v1.types.PySparkJob:
args Sequence[str]
Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
I have the following credentials.yml file :
test: 123
test2: ((test))
When I upload a pipeline, feeding it with the credentials file , whenever test2 variable is used it is not interpolated and I'm getting a "undefined vars : test" error in Concourse.
Is it even possible to refer to another variable in the very same yaml or do you have to always refer to variables in a configured credentials manager (e.g. Vault) ?
Solved it using anchors and aliases . Sadly keys containing dots or hyphens do not work at all.
e.g. :
test: &test 123
test2: *test
I have been searching for an answer to this question with no luck, but is there a way to pass parameters into puppet manifests when running the 'apply' command, in a similar way to the way you pass parameters when running a UNIX script on the command line?
The suggestions I see mention either keeping variables at the top of the manifest for use later, or to store them in a hiera file. But neither really answer the question I am posing?
Any guidance on how to do this would be greatly appreciated?
Edit:
An example of what I have been doing is:
$doc_root = "/var/www/example"
exec { 'apt-get update':
command => '/usr/bin/apt-get update'
}
package { 'apache2':
ensure => "installed",
require => Exec['apt-get update']
}
file { $doc_root:
ensure => "directory",
owner => "www-data",
group => "www-data",
mode => 644
}
file { "$doc_root/index.html":
ensure => "present",
source => "puppet:///modules/main/index.html",
require => File[$doc_root]
}
As you can see the variable is hardcoded at the top, whereas whilst I am trying to use the variable in the same way, I need to be able to pass the value in when running the apply command.
Using lookup functions in conjunction with hiera.yaml files doesn't fulfil my requirements for the same reason.
The only thing I can think may be a work around is to create a UNIX script that accepts parameters, saves those values in a yaml file, and then have the script execute the .pp file.
But I'm hoping that puppet has a way to do this directly.
The common procedure for passing variables into a classless manifest for use with the puppet apply subcommand would be to assign the value to a Facter fact from the CLI, and then resolve its value inside the manifest. You would begin with removing the hardcoded variable doc_root from the head of the manifest. Then, you would modify the variable into a fact like:
file { $facts['doc_root']:
...
file { "${facts['doc_root']}/index.html":
...
require => File["${facts['doc_root']}"] <-- interpolation required due to Puppet DSL inability to resolve hash value as first class expression
You would then pass the Facter value from the puppet apply subcommand like:
FACTER_doc_root=/var/www/example puppet apply manifest.pp
Note this also causes FACTER_doc_root to be temporarily set as an environment variable as a side effect.
I'm new to jboss-cli and working through the 'jboss-cli recipes'.
Question
How do I read one specific property using jboss-cli? E.g.
jboss.home.dir (e.g. "-Djboss.home.dir=/path/to/my/jboss")
Xmx ("-Xmx=4G")
Context
The "CLI Recipes" documentation has this helpful example to get all system properties. However its 'too much infomration'. I want to script reading one specific property.
https://docs.jboss.org/author/display/WFLY10/CLI+Recipes#CLIRecipes-
Overview of all system properties in JBoss AS7+ including OS system
properties and properties specified on command line using -D, -P or
--properties arguments.
Standalone
[standalone#IP_ADDRESS:9999 /] /core-service=platform-mbean/type=runtime:read-attribute(name=system-properties)
Thanks in advance
You could do a :
:resolve-expression(expression=${jboss.home.dir})
You can use the cli like this:
$JBOSS_HOME/bin/jboss-cli.sh -c --command=/system-property=MY_PROPERTY:read-resource
you get an output like this:
$JBOSS_HOME/bin/jboss-cli.sh -c --command=/system-property=MY_PROPERTY:read-resource
{
"outcome" => "success",
"result" => {"value" => "4.0"}
}
which you can extract by piping into something like this:
<cli command> | grep "{\"value\"" | sed "s/.*value\" => \"\([^\"]*\)\".*/\1/"
its a bit ugly, and there are some nasty edge cases if the values were to be something like "value" => "value =" or something hideous.
In general this works OK.
Change the sed command to be a bit more specific to fix that.
This link pointed me to the answer: I can use a groovy script to get the values. From what I see the "jboss-cli command line" does not offer this flexibility.
https://developer.jboss.org/wiki/AdvancedCLIScriptingWithGroovyRhinoJythonEtc
Solution
Here's a solution for jboss home.
[For memory you can get results from "/core-service=platform-mbean/type=memory/:read-attribute(name=heap-memory-usage)"
bash
#!/bin/sh
# Note: must set jbbin to 'jboss home /bin'
groovy -cp $jbbin/client/jboss-cli-client.jar readJbossHome.groovy
Groovy
Note: this is 'quick and dirty'.
import org.jboss.as.cli.scriptsupport.*
cli = CLI.newInstance()
cli.connect()
// Define properties
myParentProp="system-properties"
myProp="jboss.home.dir"
// Retrieve and pluck values
result = cli.cmd("/core-service=platform-mbean/type=runtime:read-resource(recursive=true,include-runtime=false)")
myResult = result.getResponse().get("result")
myParentVal = myResult.get(myParentProp)
myVal = myParentVal.get(myProp)
// Print out results
println "Property detail ${myProp} is ${myVal}"
cli.disconnect()
You can also do it via Wildfly management rest call.
http://localhost:9990/management
POST
Headers = Content-Type:application/json
Body =
{
"operation":"resolve-expression",
"expression":"${jboss.home.dir}"
}
With newer Teiid DOCs I have found some useful information I thought this might be helpful to share to people coming across a similar usecase
https://access.redhat.com/documentation/en-us/jboss_enterprise_application_platform/6.3/html/administration_and_configuration_guide/configure_system_properties_using_the_management_cli
Helps Adding, Removing & Reading System Properties with jboss-cli
jboss-cli
If you have a cli command like ehsavoie suggested :resolve-expression(expression=${jboss.home.dir}) and want to use the content of the "result" property within jboss-cli you can save it in a variable. You can use backticks (`) to evaluate expressions.
simple expression
[standalone#localhost:9990 /] :resolve-expression(expression=${jboss.home.dir})
{
"outcome" => "success",
"result" => "/home/user/wildfly"
}
use in valiable
[standalone#localhost:9990 /] set wildflydirectory=`:resolve-expression(expression=${jboss.home.dir})`
[standalone#localhost:9990 /] echo $wildflydirectory
/home/user/wildfly
PowerShell
If you happen to use the PowerShell you can use a one-liner to extract even deeply nested results with the help of the cli's --output-json option and PowerShell's ConvertFrom-Json cmdlet. In this way the parsing problem from James Roberts's approach with grep and sed are gone.
$value=(Invoke-Expression "./jboss-cli.ps1 -c --command=':resolve-expression(expression=`${jboss.home.dir})' --output-json" | ConvertFrom-Json).result
It is a bit tricky to quote the command and escape the correct PowerShell special characters.
I am generating the client scala code for an API using the Swagger Edtior. I pasted the json then did a Generate Client/Scala. It gives me a default root package of
io.swagger.client
I can't see any obvious way of specifying something different. Can this be done?
Step (1): Create a file config.json and add following lines and define package names:
{
"modelPackage" : "com.xyz.model",
"apiPackage" : "com.xyz.api"
}
Step (2): Now, pass the above file name along with codegen command with -c option:
$ java -jar swagger-codegen-cli.jar generate -i path/swagger.json -l java -o Code -c path/config.json
Now, it will generate your java packages like com.xyz… instead of default one io.swagger.client…
Run the following command to get information about the supported configuration options
java -jar swagger-codegen-cli.jar config-help -l scala
This will give you information about supported by this generator (Scala in this example):
CONFIG OPTIONS
sortParamsByRequiredFlag
Sort method arguments to place required parameters before optional parameters. (Default: true)
ensureUniqueParams
Whether to ensure parameter names are unique in an operation (rename parameters that are not). (Default: true)
modelPackage
package for generated models
apiPackage
package for generated api classes
Next, define a config.json file with the above parameters:
{
"modelPackage": "your package name",
"apiPackage": "your package name"
}
And supply config.json as input to swagger-codegen using the -c flag.