AWS CDK asset path is incorrect - aws-cloudformation

On September 6, I ran a build using CodePipeline. It generates a CloudFormation template for a project's stack using CDK. The stack has assets (a Lambda Layer), and the assets are correctly placed in the cdk.out folder. This can be seen in the CloudFormation template:
"Metadata": {
"aws:cdk:path": "MyStack/MyLayer/Resource",
"aws:asset:path": "asset.ccb8fd8b4259a8f517879d7aaa083679461d02b9d60bfd12725857d23567b70f",
"aws:asset:property": "Content"
}
Starting yesterday, builds were failing with "Uploaded file must be a non-empty zip". When I investigated further, I noticed that the template was no longer correct. It has the asset path set to the source code of the Lambda instead:
"Metadata": {
"aws:cdk:path": "MyStack/MyLayer/Resource",
"aws:asset:path": "/codebuild/output/src216693626/src/src/lambdas/layers",
"aws:asset:property": "Content"
}
When I build, I've added additional commands to the buildspec file which shows that the assets.abcdef folder has the layer and its dependencies, while the src folder does not. Yet the template is now different.
No code was changed in this time period, and I've tried both CDK version 1.105.0 and 1.119.0.
This code declares the Layer:
new lambdapython.PythonLayerVersion(this.stack, 'MyLayer', {
entry: path.join(__dirname, '../../src/lambdas/layers'),
description: 'Common utilities for the Lambdas',
compatibleRuntimes: [lambda.Runtime.PYTHON_3_8],
layerVersionName: `${Aws.STACK_NAME}Utils`,
});
Is there a known way for me to force the stack to use the assets in the cdk.out folder? Has something changed in the last couple of days with respect to how CDK generates the template's asset path?

It turns out that I had added a cdk ls to print out additional debugging information while troubleshooting another problem. That command re-synthesized the stack, but with the incorrect asset path.
build: {
commands: [
'cd ' + config.cdkDir,
'cdk synth',
'cdk ls --long'
]
}
The solution was to delete the cdk ls --long from the buildspec definition.

Related

how to regenerate meson for newly added yaml files

I have added yaml files to add new dbus objects and I added PHOSPHOR_MAPPER_SERVICE_append = " com/newCoName"
(newCoName is the name of my company)
But when I run bitbake, do_configure for phosphor_mapper bails when it passes the option -Ddata_com_newCoName to meson. The following readme says I need to run ./regenerate_meson from the gen directory when I add new YAML files. But how do I do that from a recipe file?
https://github.com/openbmc/phosphor-dbus-interfaces
One option is to generate these files outside ot yocto environment (i.e. not involving bitbake). Thus
clone that git repo
place your yaml file where you cloned repo
do what readme tells, i.e. go to gen directory and execute meson-regenerate script
collect changes that are done by script and create patch
add patch to your layer and reference it in .bbappend file (meta-/recipes-phosphor/dbus/phosphor-dbus-interfaces_git.bbappend)
Another option would be to add to .bbappend file additional task that runs before do_configure - and call that script from there:
do_configure_prepend() {
cd ${S}/gen && ./meson-regenerate
}
Along this .bbappend you should add your yaml so that it lands inside gen folder in patch or directly in your layer (check FILESEXTRAPATHS).
In both cases you'll need to patch meson_options.txt: add option
option('data_com_newCoName', type: 'boolean', value: true)

React-Snap with Create-React-App and Service Workers

So, my understanding is that react-snap as per its features "Works out-of-the-box with create-react-app - no code-changes required."
I read through the documentation and I see that it required some adjusting to work with Google Analytics which I implemented.
However, it also suggests changes to be made if one is going to use the default service worker that comes with CRA.
https://github.com/stereobooster/react-snap#service-workers
However, what is confusing is that it seems one has to perform a EJECT in order to make the necessary change.
navigateFallback: publicUrl + '/index.html',
You need to change this to an un-prerendered version of index.html - 200.html, otherwise you will see index.html flash on other pages (if you have any). See Configure sw-precache without ejecting for more information.
My question is - and note I am quite novice - does one have to eject? I kinda want to keep things simple. The only place I could find this line was in WebPack. navigateFallback
Also, if I don't see the negative side of the flashes on pages as per the documentation, is it okay to omit this step or will it have issues on other things?
Although this question is more than a year old, I'd like to take the opportunity as I've been able to implement service workers in react-snap (although with a varying degree of success).
Here's stereobooster's reference in GitHub:
https://github.com/stereobooster/react-snap/blob/master/doc/recipes.md#configure-sw-precache-without-ejecting
You can configure it without ejecting. What you need to do is the following:
Download and install sw-precache and ugfify-js:
npm install sw-precache uglify-js --save-dev
or
yarn add sw-precache uglify-js -D
Then, in your package.json add the following entries:
(Replace the build script with the following)
"scripts": {
"generate-sw": "sw-precache --root=build --config scripts/sw-precache-config.js && uglifyjs build/service-worker.js -o build/service-worker.js",
"build": "react-scripts build && react-snap && yarn run generate-sw"
}
Then, create a folder in the root level (next to your package.json) called scripts
and add sw-precache-config.js file.
module.exports = {
// a directory should be the same as "reactSnap.destination",
// which default value is `build`
staticFileGlobs: [
"build/static/css/*.css",
"build/static/js/*.js",
"build/shell.html",
"build/index.html"
],
stripPrefix: "build",
publicPath: ".",
// there is "reactSnap.include": ["/shell.html"] in package.json
navigateFallback: "/shell.html",
// Ignores URLs starting from /__ (useful for Firebase):
// https://github.com/facebookincubator/create-react-app/issues/2237#issuecomment-302693219
navigateFallbackWhitelist: [/^(?!\/__).*/],
// By default, a cache-busting query parameter is appended to requests
// used to populate the caches, to ensure the responses are fresh.
// If a URL is already hashed by Webpack, then there is no concern
// about it being stale, and the cache-busting can be skipped.
dontCacheBustUrlsMatching: /\.\w{8}\./,
// configuration specific to this experiment
runtimeCaching: [
{
urlPattern: /api/,
handler: "fastest"
}
]
};
Note, if you're not using an app-shell but you're loading the whole page (Meaning there's no dyanmic content), replace where it says navigateFallback: "/shell.html" with navigateFallback: "/200.html"
This basically allows you to cache the entire page
You can look for more information here:
https://github.com/stereobooster/an-almost-static-stack
One thing that I'd recommend to check (I'm close to start that process as well) is the workbox-sw.
What to do if React-Snap fails
error at / TypeError: Cannot read property 'ok' of null
Or
ERROR: The process with PID 38776 (child process of PID 26920) could not be terminated. \node_modules\minimalcss\src\run.js:13:35)
Reason: There is no running instance of the task.
You may get these infamous errors. I don't know exactly what causes them, but I know they're mentioned here, and here. In this case, delete the build folder, open a new terminal window, and try again.
If the problem still persists, then break down the script:
Do:
"scripts": {
"build": "react-scripts build"
"postbuild": "react-snap",
"generate-sw": "sw-precache --root=build --config scripts/sw-precache-config.js && uglifyjs build/service-worker.js -o build/service-worker.js",
}
And try running them independently.

How can I get "HelloWorld - BitBake Style" working on a newer version of Yocto?

In the book "Embedded Linux Systems with the Yocto Project", Chapter 4 contains a sample called "HelloWorld - BitBake style". I encountered a bunch of problems trying to get the old example working against the "Sumo" release 2.5.
If you're like me, the first error you encountered following the book's instructions was that you copied across bitbake.conf and got:
ERROR: ParseError at /tmp/bbhello/conf/bitbake.conf:749: Could not include required file conf/abi_version.conf
And after copying over abi_version.conf as well, you kept finding more and more cross-connected files that needed to be moved, and then some relative-path errors after that... Is there a better way?
Here's a series of steps which can allow you to bitbake nano based on the book's instructions.
Unless otherwise specified, these samples and instructions are all based on the online copy of the book's code-samples. While convenient for copy-pasting, the online resource is not totally consistent with the printed copy, and contains at least one extra bug.
Initial workspace setup
This guide assumes that you're working with Yocto release 2.5 ("sumo"), installed into /tmp/poky, and that the build environment will go into /tmp/bbhello. If you don't the Poky tools+libraries already, the easiest way is to clone it with:
$ git clone -b sumo git://git.yoctoproject.org/poky.git /tmp/poky
Then you can initialize the workspace with:
$ source /tmp/poky/oe-init-build-env /tmp/bbhello/
If you start a new terminal window, you'll need to repeat the previous command which will get get your shell environment set up again, but it should not replace any of the files created inside the workspace from the first time.
Wiring up the defaults
The oe-init-build-env script should have just created these files for you:
bbhello/conf/local.conf
bbhello/conf/templateconf.cfg
bbhello/conf/bblayers.conf
Keep these, they supersede some of the book-instructions, meaning that you should not create or have the files:
bbhello/classes/base.bbclass
bbhello/conf/bitbake.conf
Similarly, do not overwrite bbhello/conf/bblayers.conf with the book's sample. Instead, edit it to add a single line pointing to your own meta-hello folder, ex:
BBLAYERS ?= " \
${TOPDIR}/meta-hello \
/tmp/poky/meta \
/tmp/poky/meta-poky \
/tmp/poky/meta-yocto-bsp \
"
Creating the layer and recipe
Go ahead and create the following files from the book-samples:
meta-hello/conf/layer.conf
meta-hello/recipes-editor/nano/nano.bb
We'll edit these files gradually as we hit errors.
Can't find recipe error
The error:
ERROR: BBFILE_PATTERN_hello not defined
It is caused by the book-website's bbhello/meta-hello/conf/layer.conf being internally inconsistent. It uses the collection-name "hello" but on the next two lines uses _test suffixes. Just change them to _hello to match:
# Set layer search pattern and priority
BBFILE_COLLECTIONS += "hello"
BBFILE_PATTERN_hello := "^${LAYERDIR}/"
BBFILE_PRIORITY_hello = "5"
Interestingly, this error is not present in the printed copy of the book.
No license error
The error:
ERROR: /tmp/bbhello/meta-hello/recipes-editor/nano/nano.bb: This recipe does not have the LICENSE field set (nano)
ERROR: Failed to parse recipe: /tmp/bbhello/meta-hello/recipes-editor/nano/nano.bb
Can be fixed by adding a license setting with one of the values that bitbake recognizes. In this case, add a line onto nano.bb of:
LICENSE="GPLv3"
Recipe parse error
ERROR: ExpansionError during parsing /tmp/bbhello/meta-hello/recipes-editor/nano/nano.bb
[...]
bb.data_smart.ExpansionError: Failure expanding variable PV_MAJOR, expression was ${#bb.data.getVar('PV',d,1).split('.')[0]} which triggered exception AttributeError: module 'bb.data' has no attribute 'getVar'
This is fixed by updating the special python commands being used in the recipe, because #bb.data was deprecated and is now removed. Instead, replace it with #d, ex:
PV_MAJOR = "${#d.getVar('PV',d,1).split('.')[0]}"
PV_MINOR = "${#d.getVar('PV',d,1).split('.')[1]}"
License checksum failure
ERROR: nano-2.2.6-r0 do_populate_lic: QA Issue: nano: Recipe file fetches files and does not have license file information (LIC_FILES_CHKSUM) [license-checksum]
This can be fixed by adding a directive to the recipe telling it what license-info-containing file to grab, and what checksum we expect it to have.
We can follow the way the recipe generates the SRC_URI, and modify it slightly to point at the COPYING file in the same web-directory. Add this line to nano.bb:
LIC_FILES_CHKSUM = "${SITE}/v${PV_MAJOR}.${PV_MINOR}/COPYING;md5=f27defe1e96c2e1ecd4e0c9be8967949"
The MD5 checksum in this case came from manually downloading and inspecting the matching file.
Done!
Now bitbake nano ought to work, and when it is complete you should see it built nano:
/tmp/bbhello $ find ./tmp/deploy/ -name "*nano*.rpm*"
./tmp/deploy/rpm/i586/nano-dbg-2.2.6-r0.i586.rpm
./tmp/deploy/rpm/i586/nano-dev-2.2.6-r0.i586.rpm
I have recently worked on that hands-on hello world project. As far as I am concerned, I think that the source code in the book contains some bugs. Below there is a list of suggested fixes:
Inheriting native class
In fact, when you build with bitbake that you got from poky, it builds only for the target, unless you mention in your recipe that you are building for the host machine (native). You can do the latter by adding this line at the end of your recipe:
inherit native
Adding license information
It is worth mentioning that the variable LICENSE is important to be set in any recipe, otherwise bitbake rises an error. In our case, we try to build the version 2.2.6 of the nano editor, its current license is GPLv3, hence it should be mentioned as follow:
LICENSE = "GPLv3"
Using os.system calls
As the book states, you cannot dereference metadata directly from a python function. Which means it is mandatory to access metadata through the d dictionary. Bellow, there is a suggestion for the do_unpack python function, you can use its concept to code the next tasks (do_configure, do_compile):
python do_unpack() {
workdir = d.getVar("WORKDIR", True)
dl_dir = d.getVar("DL_DIR", True)
p = d.getVar("P", True)
tarball_name = os.path.join(dl_dir, p+".tar.gz")
bb.plain("Unpacking tarball")
os.system("tar -x -C " + workdir + " -f " + tarball_name)
bb.plain("tarball unpacked successfully")
}
Launching the nano editor
After successfully building your nano editor package, you can find your nano executable in the following directory in case you are using Ubuntu (arch x86_64):
./tmp/work/x86_64-linux/nano/2.2.6-r0/src/nano
Should you have any comments or questions, Don't hesitate !

Old feature file path is used even after updating a new path

I am new to cucumber and I am automating a scenario. Initially I kept my features files in the path C:\Users\test\eclipse-workspace\Automation\src\test\resources\featureFile. Then I moved the feature files to a different path (C:\Users\test\eclipse-workspace\Automation\src\test\com\test]automation\features). I have updated the same in CucumberOptions as shown below.
#CucumberOptions(features = {
"src/test/java/com/test/automation/features/CO_Self_Service_Home_Page_Personalizations.feature" }, glue = {
"src/test/java/com/oracle/peoplesoft/HCM/StepDefinitions" })
But when I try to run the feature, I am getting the below exception stating the feature file is not found. Here the path shown in the exception is the old path. I am not sure from where it is fetched as I have updated the new path in Cucumber options. Can you please help me understand the cause of this issue.
Exception in thread "main" java.lang.IllegalArgumentException: Not a
file or directory:
C:\Users\test\eclipse-workspace\Automation\src\test\resources\featureFile\Self_Service_Home_Page_Personalizations.feature
at
cucumber.runtime.io.FileResourceIterator$FileIterator.(FileResourceIterator.java:54)
at
cucumber.runtime.io.FileResourceIterator.(FileResourceIterator.java:20)
at
cucumber.runtime.io.FileResourceIterable.iterator(FileResourceIterable.java:19)
at
cucumber.runtime.model.CucumberFeature.loadFromFeaturePath(CucumberFeature.java:103)
at
cucumber.runtime.model.CucumberFeature.load(CucumberFeature.java:54)
at
cucumber.runtime.model.CucumberFeature.load(CucumberFeature.java:34)
at
cucumber.runtime.RuntimeOptions.cucumberFeatures(RuntimeOptions.java:235)
at cucumber.runtime.Runtime.run(Runtime.java:110) at
cucumber.api.cli.Main.run(Main.java:36) at
cucumber.api.cli.Main.main(Main.java:18)
There are a couple of points you need to take care as follows :
As per Best Practices cerate the directory features which will contain the featurefile(s) strictly through your IDE only (not through other softwares Notepad or Textpad or SubLime3) as per the image below (New -> File) :
Create the featurefile i.e. CO_Self_Service_Home_Page_Personalizations.feature within features directory strictly through your IDE only.
Keep your Project Structure simple by placing the directory containing the featurefile(s) just under Project Workspace. For Featurefiles Cucumber works with directory names. So create the features directory just under your project space Automation (same hierarchy as src). So the location of the Self_Service_Home_Page_Personalizations.feature will be :
C:\Users\test\eclipse-workspace\Automation\features\Self_Service_Home_Page_Personalizations.feature
Again, as in your Class file containing #CucumberOptions you have mentioned glue = {"StepDefinitions" } ensure that the Class file containing #CucumberOptions must be in the similar hierarchy as the figure below :
So your CucumberOptions will be as follows :
#CucumberOptions(features = {"features" }, glue = {"StepDefinitions" })
Execute your Test
Note : Do not move/copy feature file(s)/directory(ies). Delete the unwanted and create a new one through your IDE only.

Packer with chef-solo provisioning does nothing

I'm starting up with Packer using chef-solo to provision. I have a very simple recipe (default.rb) that contains the lines below:
package "git"
package "ruby"
package "rubygems"
I was able to provision an image using Vagrant with this successfully. I'm now trying to move this provision step to Packer but when I execute packer build it doesn't seem to run the recipe.
virtualbox-iso: Running handlers complete
virtualbox-iso: Chef Client finished, 0/0 resources updated in 1.818437359 seconds
My Packer template's provision section is:
{
"type": "chef-solo",
"cookbook_paths": ["/cookbooks"]
}
My second part to this question (I'm assuming it's going to be related) is what is the run_list configuration option?
The Packer documentation says it goes in the same file, is called run_list, and by default is empty. So you should give the name of your cookbook as a single-element string array, in a param called run_list.
For beginners like myself who are looking for an answer, I addressed this by adding a run_list like the following:
"run_list": ["git::default"]
'git' is the name in the metadata.rb file and 'default' is the filename or recipe (if my terminology is correct). My cookbook directory structure is as follows:
~/Projects/Packer-Templates/Cookbooks $ find .
.
./ruby-environment
./ruby-environment/metadata.rb
./ruby-environment/recipes
./ruby-environment/recipes/default.rb