I am using the Typescript API of pulumi. I noticed that when I invoke console.log("\n\n"), pulumi strips out the newlines. I want to keep these newlines to improve the readability of the deployment log.
Is there a way to instruct pulumi to keep newlines in the output log?
The current behavior of the Pulumi CLI is to break your messages into lines (split by \n), trim every line, drop empty lines, and display the result.
Although ugly, you could force your line breaks with an extra "zero-width space" character:
console.log("Top line");
console.log("\u200B\n\u200B\n\u200B");
console.log("There will be three empty lines before this line");
You could use something more trivial like _ instead of the zero-width space. Obviously, underscores will be visible.
Track this issue for further progress.
Pulumi should not be stripping newlines or otherwise manipulating your console.log() output. I just tested this and my string with newlines was printed as expected with newlines.
Code
import * as aws from "#pulumi/aws";
const bucket = new aws.s3.Bucket("main", {
acl: "private",
})
bucket.onObjectCreated("logger", new aws.lambda.CallbackFunction<aws.s3.BucketEvent, void>("loggerFn", {
memorySize: 128,
callback: (e) => {
for (const rec of e.Records || []) {
const [buck, key] = [rec.s3.bucket.name, rec.s3.object.key];
console.log(`Object created: ${buck}/${key}`);
}
},
}));
console.log(`My
multi-line
string`);
export const bucketName = bucket.bucket;
Output
$ pulumi up -y
Previewing update (dev):
Type Name Plan Info
+ pulumi:pulumi:Stack demo-aws-ts-serverless-dev create 3 ...
+ └─ aws:lambda:Function loggerFn create
Diagnostics:
pulumi:pulumi:Stack (demo-aws-ts-serverless-dev):
My
multi-line
string
Resources:
+ 8 to create
Updating (dev):
Type Name Status Info
+ pulumi:pulumi:Stack demo-aws-ts-serverless-dev created ...
+ └─ aws:lambda:Function loggerFn created
Diagnostics:
pulumi:pulumi:Stack (demo-aws-ts-serverless-dev):
My
multi-line
string
Outputs:
bucketName: "main-b568df3"
Resources:
+ 8 created
...
Related
i am currently checking out tanka + jsonnet. But evertime i think i understand it... sth. new irritates me. Can somebody help me understand how to do a loop-reference? (Or general better solution?)
Trying to create multiple deployments with a corresponding configmapVolumeMount and i am not sure how to reference to the according configmap object here?
(using a configVolumeMount it works since it refers to the name, not the object).
deployment: [
deploy.new(
name='demo-' + instance.name,
],
)
+ deploy.configMapVolumeMount('config-' + instance.name, '/config.yml', k.core.v1.volumeMount.withSubPath('config.yml'))
for instance in $._config.demo.instances
],
configMap: [
configMap.new('config-' + instance.name, {
'config.yml': (importstr 'files/config.yml') % {
name: instance.name,
....
},
}),
for instance in $._config.demo.instances
]
regards
Great to read that you're making progress with tanka, it's an awesome tool (once you learned how to ride it heh).
Find below a possible answer, see inline comments in the code, in particular how we ab-use tanka layout flexibility, to "populate" deploys: [...] array with jsonnet objects containing each paired deploy+configMap.
config.jsonnet
{
demo: {
instances: ['foo', 'bar'],
image: 'nginx', // just as example
},
}
main.jsonnet
local config = import 'config.jsonnet';
local k = import 'github.com/grafana/jsonnet-libs/ksonnet-util/kausal.libsonnet';
{
local deployment = k.apps.v1.deployment,
local configMap = k.core.v1.configMap,
_config:: import 'config.jsonnet',
// my_deploy(name) will return name-d deploy+configMap object
my_deploy(name):: {
local this = self,
deployment:
deployment.new(
name='deploy-%s' % name,
replicas=1,
containers=[
k.core.v1.container.new('demo-%s' % name, $._config.demo.image),
],
)
+ deployment.configMapVolumeMount(
this.configMap,
'/config.yml',
k.core.v1.volumeMount.withSubPath('config.yml')
),
configMap:
configMap.new('config-%s' % name)
+ configMap.withData({
// NB: replacing `importstr 'files/config.yml';` by
// a simple YAML multi-line string, just for the sake of having
// a simple yet complete/usable example.
'config.yml': |||
name: %(name)s
other: value
||| % { name: name }, //
}),
},
// Tanka is pretty flexible with the "layout" of the Kubernetes objects
// in the Environment (can be arrays, objects, etc), below using an array
// for simplicity (built via a loop/comprehension)
deploys: [$.my_deploy(name) for name in $._config.demo.instances],
}
output
$ tk init
[...]
## NOTE: using https://kind.sigs.k8s.io/ local Kubernetes cluster
$ tk env set --server-from-context kind-kind environments/default
[... save main.jsonnet, config.jsonnet to ./environments/default/]
$ tk apply --dry-run=server environments/default
[...]
configmap/config-bar created (server dry run)
configmap/config-foo created (server dry run)
deployment.apps/deploy-bar created (server dry run)
deployment.apps/deploy-foo created (server dry run)
I am trying to pass firebase environment variables for deployment with now.
I have encoded these variables manually with base64 and added them to now with the following command:
now secrets add firebase_api_key_dev "mybase64string"
The encoded string was placed within speech marks ""
These are in my CLI tool and I can see them all using the list command:
now secrets ls
> 7 secrets found under project-name [499ms]
name created
firebase_api_key_dev 6d ago
firebase_auth_domain_dev 6d ago
...
In my firebase config, I am using the following code:
const config = {
apiKey: Buffer.from(process.env.FIREBASE_API_KEY, "base64").toString(),
authDomain: Buffer.from(process.env.FIREBASE_AUTH_DOMAIN,"base64").toString(),
...
}
In my now.json file I have the following code:
{
"env": {
"FIREBASE_API_KEY": "#firebase_api_key_dev",
"FIREBASE_AUTH_DOMAIN": "#firebase_auth_domain_dev",
...
}
}
Everything works fine in my local environment (when I run next) as I also have a .env file with these variables, yet when I deploy my code, I get the following error in my now console:
TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be one of type string, Buffer, ArrayBuffer, Array, or Array-like Object. Received type undefined
Does this indicate that my environment variables are not being read? What's the issue here? It looks like they don't exist at all
The solution was to replace my existing now.json with:
{
"build":{
"env": {
"FIREBASE_API_KEY": "#firebase_api_key",
"FIREBASE_AUTH_DOMAIN": "#firebase_auth_domain",
"FIREBASE_DATABASE_URL": "#firebase_database_url",
"FIREBASE_PROJECT_ID": "#firebase_project_id",
"FIREBASE_STORAGE_BUCKET": "#firebase_storage_bucket",
"FIREBASE_MESSAGING_SENDER_ID": "#firebase_messaging_sender_id",
"FIREBASE_APP_ID": "#firebase_app_id",
"FIREBASE_API_KEY_DEV": "#firebase_api_key_dev",
"FIREBASE_AUTH_DOMAIN_DEV": "#firebase_auth_domain_dev",
"FIREBASE_DATABASE_URL_DEV": "#firebase_database_url_dev",
"FIREBASE_PROJECT_ID_DEV": "#firebase_project_id_dev",
"FIREBASE_STORAGE_BUCKET_DEV": "#firebase_storage_bucket_dev",
"FIREBASE_MESSAGING_SENDER_ID_DEV": "#firebase_messaging_sender_id_dev",
"FIREBASE_APP_ID_DEV": "#firebase_app_id_dev"
}
}
}
I was missing the build header.
I had to contact ZEIT support to help me identify this issue.
When using ruamel.yaml to round-trip some YAML I see the following issue. Given this input:
root:
matchers:
- select: "response.body#state"
test: all
expected: "on"
I see this output:
root:
matchers:
- select: response.body#state
test: all
expected: on
Note that in YAML, on parses as a boolean true value while off parses as false.
The following code is used to read/write:
# Use the default (round-trip) settings.
yaml = YAML()
if args.source == '-':
src = sys.stdin
else:
src = open(args.source)
doc = yaml.load(src)
process(args.tag, set(args.keep.split(',')), doc)
if args.destination == '-':
dest = sys.stdout
else:
dest = open(args.destination, 'w')
yaml.dump(doc, dest)
The process function is not modifying values. It only removes things with a special tag in the input after crawling the structure.
How can I get the output to be a string rather than a boolean?
You write that:
Note that in YAML, on parses as a boolean true value while off parses as false.
That statement is not true (or better:
has not been true for ten years). If you have an unquoted on in your
YAML, like in your output, that is obviously not the case when using ruamel.yaml:
import sys
import ruamel.yaml
yaml_str = """\
root:
matchers:
- select: response.body#state
test: all
expected: on
"""
yaml = ruamel.yaml.YAML()
data = yaml.load(yaml_str)
expected = data['root']['matchers'][0]['expected']
print(type(expected), repr(expected))
which gives:
<class 'str'> 'on'
This is because in the YAML 1.2 spec on/off/yes/no are no
longer mentioned as having the same meaning as true
resp. false. They are mentioned in the YAML 1.1 spec, but that was
superseded in 2009. Unfortunately there are YAML libraries out in the
wild, that have not been updated since then.
What is actually happening is that the suprefluous quotes in your
input are automatically discarded by the round-trip process. You can
also see that happen for the value "response.body#state". Although
there the character that starts comments (#) is included, to
actually start a comment that character has to be proceded by
white-space, and since it is isn't, the quotes are not necessary.
So your output is fine, but if you are in the unfortunate
situation where you have to deal with other programs relying on
outdated YAML 1.1, then you can e.g. specify that you want to preserve
your quotes on round-trip:
yaml_str = """\
root:
matchers:
- select: "response.body#state"
test: all
expected: "on"
"""
yaml = ruamel.yaml.YAML()
yaml.indent(sequence=4, offset=2)
yaml.preserve_quotes = True
data = yaml.load(yaml_str)
yaml.dump(data, sys.stdout)
as this gives your exact input:
root:
matchers:
- select: "response.body#state"
test: all
expected: "on"
However maybe the better option would be that you actually specify that your
YAML is and has to conforming to the YAML 1.1 specification by making
your intensions, and the output document, explicit:
yaml_str = """\
root:
matchers:
- select: response.body#state
test: all
expected: on
"""
yaml_in = ruamel.yaml.YAML()
yaml_out = ruamel.yaml.YAML()
yaml_out.indent(sequence=4, offset=2)
yaml_out.version = (1, 1)
data = yaml_in.load(yaml_str)
yaml_out.dump(data, sys.stdout)
Notice that the "unquoted" YAML 1.2 input, gives output where on is quoted:
%YAML 1.1
---
root:
matchers:
- select: response.body#state
test: all
expected: 'on'
I was trying to create some basic inspec tests to validate a set of HTTP URLs. The way I started is like this -
control 'http-url-checks' do
impact 1.0
title 'http-url-checks'
desc '
Specify the URLs which need to be up and working.
'
tag 'http-url-checks'
describe http('http://example.com') do
its('status') { should eq 200 }
its('body') { should match /abc/ }
its('headers.name') { should eq 'header' }
end
describe http('http://example.net') do
its('status') { should eq 200 }
its('body') { should match /abc/ }
its('headers.name') { should eq 'header' }
end
end
We notice that the URLs are hard-coded in the controls and isn't a lot of fun. I'd like to move them to some 'attributes' file of some sort and loop through them in the control file.
My attempt was to use the 'files' folder structure inside the profile.I created a file - httpurls.yml and had the following content in it -
- url: http://example.com
- url: http://example.net
..and in my control file, I had the construct -
my_urls = yaml(content: inspec.profile.file('httpurls.yml')).params
my_urls.each do |s|
describe http(s['url']) do
its('status') { should eq 200 }
end
end
However, when I execute the compliance profile, I get an error - 'httpurls.yml not found' (not sure about the exact error message though though). The following is the folder structure I had for my compliance profile.
What I am doing wrong?
Is there a better way to achieve what I am trying to do?
The secret is to use profile attributes, as defined near the bottom of this page:
https://www.inspec.io/docs/reference/profiles/
First, create a profile attributes YML file. I name mine profile-attribute.yml.
Second, put your array of values in the YML file, like so:
urls:
- http://example.com
- http://example.net
Third, create an attribute at the top of your InSpec tests:
my_urls = attribute('urls', description: 'The URLs that I am validating.')
Fourth, use your attribute in your InSpec test:
my_urls.each do |s|
describe http(s['url']) do
its('status') { should eq 200 }
end
end
Finally, when you call your InSpec test, point to your YML file using --attrs:
inspec exec mytest.rb --reporter=cli --attrs profile-attribute.yml
There is another way to do this using files (instead of the profile attributes and the --attrs flag). You can use JSON or YAML.
First, create the JSON and/or YAML file and put them in the files directory. A simple example of the JSON file might look like this:
{
"urls": ["https://www.google.com", "https://www.apple.com"]
}
And a simple example of the YAML file might look like this:
urls:
- https://www.google.com
- https://www.apple.com
Second, include code at the top of your InSpec file to read and parse the JSON and/or YAML, like so:
jsoncontent = inspec.profile.file("tmp.json")
jsonparams = JSON.parse(jsoncontent)
jsonurls = jsonparams['urls']
yamlcontent = inspec.profile.file("tmp.yaml")
yamlparams = YAML.load(yamlcontent)
yamlurls = yamlparams['urls']
Third, use the variables in your InSpec tests, like so:
jsonurls.each do |jsonurl|
describe http(jsonurl) do
puts "json url is " + jsonurl
its('status') { should eq 200 }
end
end
yamlurls.each do |yamlurl|
describe http(yamlurl) do
puts "yaml url is " + yamlurl
its('status') { should eq 200 }
end
end
(NOTE: the puts line is for debugging.)
The result is what you would expect:
json url is https://www.google.com
json url is https://www.apple.com
yaml url is https://www.google.com
yaml url is https://www.apple.com
Profile: InSpec Profile (inspec-file-test)
Version: 0.1.0
Target: local://
http GET on https://www.google.com
✔ status should eq 200
http GET on https://www.apple.com
✔ status should eq 200
http GET on https://www.google.com
✔ status should eq 200
http GET on https://www.apple.com
✔ status should eq 200
I am trying to checkout code from SVN repo for which I am accepting the URL as argument. I have quoted the URL as shown below because it contains spaces. I also checked the parameter by redirecting the $svn_url in file (shown below). If I pick the URL from the file and pass it as is on the command line to the given script, it works fine but somehow when invoked from Puppet, it's not working.
Puppet manifests:
repo_checkout.pp:
define infra::svn::repo_checkout ($svn_url_params) {
$svn_url = $svn_url_params[svn_url]
include infra::params
$repo_checkout_ps = $infra::params::repo_checkout_ps
file { $repo_checkout_ps:
ensure => file,
source => 'puppet:///modules/infra/repo_checkout.ps1',
}
util::executeps { 'Checking out repo':
pspath => $repo_checkout_ps,
argument => "\'\"$svn_url\"\'",
}
}
params.pp:
$repo_checkout_ps = 'c:/scripts/infra/repo_checkout.ps1',
site.pp:
$svn_url_ad = {
svn_url => 'https:\\\\some_repo.abc.com\svn\dir with space\util',
}
infra::svn::repo_checkout { "Checking out code in C:\build":
svn_url_params => $svn_url_ad
}
executeps.pp:
define util::executeps ($pspath, $argument) {
$powershell = 'C:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe -NoProfile -NoLogo -NonInteractive'
exec { "Executing PS file \"$pspath\" with argument \"$argument\"":
command => "$powershell -file $pspath $argument",
timeout => 900,
}
}
PowerShell code:
$svn_url = $args[0]
Set-Location C:\build
echo "svn co --username user --password xxx --non-interactive '$svn_url'" | Out-File c:\svn_url
svn co --username user --password xxx --non-interactive '$svn_url'
Puppet output on agent node:
Util::Executeps[Checking out repo]/Exec[Executing PS file "c:/scripts/infra/repo_checkout.ps1" with argument "'"https:\\some_repo.abc.com\svn\dir with space\util"'"]/returns: executed successfully
Notice: Applied catalog in 1.83 seconds
Content of c:\svn_url:
'https:\\\\some_repo.abc.com\svn\dir with space\util'
UPDATE: Sorry for the confusion but i was trying out several permutations and combinations and in doing that, i forgot to mention that when the $svn_url contains backslash (\), it does NOT work on the command line too if i copy the SVN URL from the text file where i am redirecting the echo output.
Based on #Ansgar's suggestion, i changed '$svn_url' to "$svn_url" in powershell code but the output in text file then contained ' quote twice around the URL. So i changed the argument parameter from "\'\"$svn_url\"\'" to "\"$svn_url\"". Now the output file had only single quote present around the URL. I copied only the URL (along with single quotes around it) from the output file and tried passing it to the powershell script. I now get the following error:
svn: E020024: Error resolving case of 'https:\\some_repo.abc.com\svn\dir with space\util'
Another thing to note is that if i change the back slashes in URL to forward slashes, it works fine on the command line. Invoking from Puppet still doesn't work.
Posting the final configuration that worked out for me based on #AnsgarWiechers' suggestion.
[tom#pe-server] cat repo_checkout.pp
define infra::svn::repo_checkout ($svn_url_params) {
$svn_url = $svn_url_params[svn_url]
...
...
util::executeps { 'Checking out repo':
pspath => $repo_checkout_ps,
argument => "\"$svn_url\"",
}
}
[tom#pe-server] cat repo_checkout.ps1
$svn_url = $args[0]
Set-Location C:\build
svn co --username user --password xxx --non-interactive "$svn_url"
[tom#pe-server] cat params.pp
$repo_checkout_ps = 'c:/scripts/infra/repo_checkout.ps1',
[tom#pe-server] cat site.pp
$svn_url_ad = {
svn_url => 'https://some_repo.abc.com/svn/dir with space/util',
}
infra::svn::repo_checkout { "Checking out code in C:\build":
svn_url_params => $svn_url_ad
}
Thanks a lot #AnsgarWiechers! :)
Note:
In site.pp: Used forwardslashes (/) when specifying svn_url
In repo_checkout.ps1: Changed '$svn_url' to "$svn_url"
In repo_checkout.pp: Changed double-nested (' and ") quoting in argument to single (") nested i.e., from "\'\"$svn_url\"\'" to "\"$svn_url\""