s3cmd reads configuration file but claims it doesn't exist - ubuntu-16.04

I'm using s3cmd to access the Ceph storage provided by my cloud provider. I created a minimal .s3cfg file on my VM to do so. When I try to list buckets, it claims that it can't find the configuration file despite --debug output showing that it's parsing it:
$ s3cmd ls --debug
DEBUG: s3cmd version 1.6.1
DEBUG: ConfigParser: Reading file '/home/ubuntu/.s3cfg'
DEBUG: ConfigParser: access_key->...-3_chars...
DEBUG: ConfigParser: secret_key->...-3_chars...
DEBUG: ConfigParser: enable_multipart->True
DEBUG: ConfigParser: host_base->storage.datacentred.io
DEBUG: ConfigParser: host_bucket->%(bucket)s.storage.datacentred.io
DEBUG: ConfigParser: use_https->True
ERROR: /home/ubuntu/.s3cfg: None
ERROR: Configuration file not available.
ERROR: Consider using --configure parameter to create one.
I am using Ubuntu 16.04.2 LTS. Any idea what I'm doing wrong? Is there a key property that is missing that would make s3cmd believe the file doesn't exist?

The prior line /home/ubuntu/.s3cfg: None, says there is some error parsing your configuration file. The None unfortunately means whatever the error was doesn't have a nice string to display :-(.
Here's the relevant block of s3cmd source:
try:
cfg = Config(options.config, options.access_key, options.secret_key, options.access_token)
except IOError, e:
if options.run_configure:
cfg = Config()
else:
error(u"%s: %s" % (options.config, e.strerror))
error(u"Configuration file not available.")

Related

Running druid script for injestion spec failed - bin/post-index-task --file quickstart/tutorial/wikipedia-index.json

I am trying to follow the quickstart for running the script but it is failing with this error in druid router cli:
~ $ bin/post-index-task --file quickstart/tutorial/wikipedia-index.json
bin/post-index-task: /opt/druid/bin/post-index-task-main: /usr/bin/env: bad interpreter: No such file or directory
bin/post-index-task: line 31: /opt/druid/bin/post-index-task-main: No error information
bin/post-index-task is already present but it seem to throw an error of No such file or directory.
Please guide, thanks.

VS Code and pytest: Where is the default junit-xml output path defined and why is there one?

I am trying to run a debug the tests of my Python project inside the VS Code interface.
I followed the instruction from VS Code's website using pytest but when trying to run a test the output fails:
============================= test session starts =============================
platform win32 -- Python 3.8.10, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: c:\Users\myUserName\Projects\myProjectName
plugins: localserver-0.5.0
collected 1 item
myProjectName\tests\test_static_analysis.py . [100%]
- generated xml file: C:\Users\MYUSERNAME\AppData\Local\Temp\tmp-26696NTXW7ChqfMEN.xml
-
============================== 1 passed in 0.18s ==============================
Error: Error: cannot open file:///c%3A/Users/myUserName/Projects/myProjectName/C. Detail:
Unable to read file 'c:\Users\myUserName\Projects\myProjectName\C' (Error: Unable to resolve
non-existing file 'c:\Users\myUserName\Projects\myProjectName\C')
Error: Error: cannot open file:///c%3A/Users/myUserName/Projects/myProjectName/C. Detail:
Unable to read file 'c:\Users\myUserName\Projects\myProjectName\C' (Error: Unable to resolve
non-existing file 'c:\Users\myUserName\Projects\myProjectName\C')
Maybe VS Code doesn't have the authorization to write in AppData.
What concerns me is why is VS Code launching pytest with a junit-xml output option?
The command actually executed by VS Code is:
c:; cd 'c:\Users\myUserName\Projects\myProjectName'; & 'C:\ProgramData\Anaconda3\envs\venv-myProjectName\python.exe' 'c:\Users\myUserName\.vscode\extensions\ms-python.python-2021.8.1105858891\pythonFiles\lib\python\debugpy\launcher' '53774' '--' 'c:\Users\myUserName\.vscode\extensions\ms-python.python-2021.8.1105858891\pythonFiles\testlauncher.py' 'c:\Users\myUserName\Projects\myProjectName' 'pytest' '--override-ini' 'junit_family=xunit1' '--rootdir' 'c:\Users\myUserName\Projects\myProjectName' '--junit-xml=C:\Users\MYUSERNAME\AppData\Local\Temp\tmp-26696pYh1w3pF2cXx.xml' './myProjectName/tests/test_static_analysis.py::TestStaticAnalysisVesselForceOnLateralCenter::test_pos_surge_on_vessel'
When going in Settings > python.testing.pytestArgs it is empty.
Where is this output path defined?
How can I change it to be in the local working directory?
Do I need to have a junit-xml output? Is it mandatory for VS Code UI to work?

How Can I Solve "Failed To Source Bitbake" With Xilinx Petalinux SDK

I am running Ubuntu 16.04 with Xilinx Petalinux 2018.03 SDK. After a number of successful compilations I am now facing this error
$ petalinux-build
[INFO] building project
[INFO] sourcing bitbake
ERROR: Failed to source bitbake
ERROR: Failed to build project
How can I solved this issue?
Another reason to get the errors "ERROR: Failed to source bitbake" as well as "ERROR: Failed to build project" is a possible upgrade of Python on the build machine. The Petalinux SDK requires python v2 (>= 2.7.3) for the 2018.3 edition.
You can check under [YOUR_PROJECT]/build/build.log and you might see a log similar to this one below:
[INFO] building project
[INFO] sourcing bitbake
SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.
OpenEmbedded requires 'python' to be python v2 (>= 2.7.3), not python v3.
Please set up python v2 as your default 'python' interpreter.
ERROR: Failed to source bitbake
ERROR: Failed to build project
To remedy this issue remove the symbolic link under /usr/bin and make sure to create a new one that is pointing to Python 2.7:
sudo rm /usr/bin/python
sudo ln -s /usr/bin/python2.7 /usr/bin/python
First you need to investigate the error a little further, do this:
source /opt/pkg/petalinux/2018.3/settings.sh
It will return something similar to this below:
PetaLinux environment set to '/opt/pkg/petalinux/2018.3'
INFO: Checking free disk space
INFO: Checking installed tools
INFO: Checking installed development libraries
INFO: Checking network and other services
Source the environment setup:
source /opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/environment-setup-aarch64-xilinx-linux
followed by:
devtool --help
In my case I can see more about the actual error:
NOTE: Starting bitbake server...
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/bitbake-cookerdaemon.log):
Traceback (most recent call last):
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/daemonize.py", line 77, in createDaemon
function()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/server/process.py", line 433, in _startServer
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/cooker.py", line 178, in __init__
self.configwatcher = pyinotify.WatchManager()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/pyinotify.py", line 1764, in __init__
raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, Errno=Too many open files (EMFILE)
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/bitbake-cookerdaemon.log):
Traceback (most recent call last):
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/daemonize.py", line 77, in createDaemon
function()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/server/process.py", line 433, in _startServer
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/cooker.py", line 178, in __init__
self.configwatcher = pyinotify.WatchManager()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/pyinotify.py", line 1764, in __init__
raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, **Errno=Too many open files (EMFILE)**
This is pointing to the /proc/sys/fs/inotify/max_user_instances that need to be increased. In my case I went from 128 to 256 by doing this:
sudo su
echo 256 > /proc/sys/fs/inotify/max_user_instances
You need to become root with "su" and change the mac_user_instances.

How to solve error on docker:layers_calculator to compute the Merkle tree on private tangle?

I want to setup a private tangle on my own virtual machine with Ubuntu 18.04, 4GB RAM and 20GB memory.
I have follow this instructions: https://docs.iota.org/docs/compass/0.1/how-to-guides/set-up-a-private-tangle. Every command works fine until reach this one: bazel run //docker:layers_calculator.
It shows an error as follows:
Starting local Bazel server and connecting to it...
ERROR: /home/istabraq/compass/third-party/maven_deps.bzl:3:5: Traceback (most recent call last):
File "/home/istabraq/compass/WORKSPACE", line 42
maven_jars()
File "/home/istabraq/compass/third-party/maven_deps.bzl", line 3, in maven_jars
native.maven_jar(<4 more arguments>)
type 'struct' has no method maven_jar()
ERROR: error loading package '': Encountered error while reading extension file 'protobuf_deps.bzl': no such package '#com_google_protobuf_deps//': error loading package 'external': Could not load //external package
ERROR: error loading package '': Encountered error while reading extension file 'protobuf_deps.bzl': no such package '#com_google_protobuf_deps//': error loading package 'external': Could not load //external package
INFO: Elapsed time: 4.743s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)
How can I solve this problem? what I have missed?
read carefully the message given after running bazel installer:
Make sure you have "/home/yourusername/bin" in your path. You can also activate bash completion by adding the following line to your :
source /home/yourusername/.bazel/bin/bazel-complete.bash
You can check with: "bazel info" or "bazel version"
Unfortunately, there are further errors:
https://github.com/iotaledger/compass/issues/142
I have solve this issue by using this commands:
Step 3: Set up your environment
If you ran the Bazel installer with the --user flag as above, the Bazel executable is installed in your $HOME/bin directory. It’s a good idea to add this directory to your default paths, as follows:
export PATH="$PATH:$HOME/bin"
You can also add this command to your ~/.bashrc or ~/.zshrc file to make it permanent.
reference:
https://docs.bazel.build/versions/master/install-ubuntu.html

Unable to start bitbake server

I am trying to learn yocto by following the video tutorials on their main website. I installed the poky-rocko-18.0.0 and after setting up the build environment I tried to build the linux image using the following command:
bitbake core-image-minimal
However, I am getting the following error:
I am unsure how to start the bitbake server and so far have not found any good references for the same.
We also faced same issue with our bitbake server. It will worked after remove bitbacke.lock
file. Use below command for solution.
rm -rf bitbake.lock
###/build$ bitbake core-image-sato
Loading cache: 100% |#########################################################################################################################################################################################################| Time: 0:00:01
Loaded 3867 entries from dependency cache.
Parsing recipes: 100%
My problem was some missing packages on my build system.
Fixed it by installing the following packages (Debian):
sudo apt-get install chrpath
sudo apt-get install texinfo
On my Arch system:
sudo pacman -S rpcsvc-proto chrpath texinfo cpio diffstat
Just try this in your build folder:
rm -rf bitbake.lock
this shoud work
The reason for this is the state of the bitbake is locked during last bitbake execution. Once you stop intermittently, we need to remove the bitbake.lock
In my case it was solved with this answer from https://stackoverflow.com/a/45880855/5350353 (Unable to connect to bitbake server):
This is because new function findTopdir (Submitted on July 18, 2017) does not handle errors. For example, the lack of BBPATH environment variable and the inability to find conf/bblayers.conf in BBPATH. findTopdir just returns None in case of that errors.
Maybe caused by the absence of host application(s), like gawk, chrpath and texinfo.
Below is one example.
ERROR: Unable to start bitbake server (None)
ERROR: Server log for this session (/home/zephyr/workspace/w031/openembedded-core/build/bitbake-cookerdaemon.log):
--- Starting bitbake server pid 22675 at 2019-03-16 00:28:44.447008 ---
Traceback (most recent call last):
File "/home/zephyr/workspace/w031/bitbake/lib/bb/cookerdata.py", line 290, in parseBaseConfiguration
bb.event.fire(bb.event.ConfigParsed(), self.data)
File "/home/zephyr/workspace/w031/bitbake/lib/bb/event.py", line 225, in fire
fire_class_handlers(event, d)
File "/home/zephyr/workspace/w031/bitbake/lib/bb/event.py", line 134, in fire_class_handlers
execute_handler(name, handler, event, d)
File "/home/zephyr/workspace/w031/bitbake/lib/bb/event.py", line 106, in execute_handler
ret = handler(event)
File "/home/zephyr/workspace/w031/openembedded-core/meta/classes/base.bbclass", line 238, in base_eventhandler
setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS', d)
File "/home/zephyr/workspace/w031/openembedded-core/meta/classes/base.bbclass", line 142, in setup_hosttools_dir
bb.fatal("The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed:\n %s" % " ".join(notfound))
File "/home/zephyr/workspace/w031/bitbake/lib/bb/__init__.py", line 120, in fatal
raise BBHandledException()
bb.BBHandledException
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/zephyr/workspace/w031/bitbake/lib/bb/daemonize.py", line 83, in createDaemon
function()
File "/home/zephyr/workspace/w031/bitbake/lib/bb/server/process.py", line 469, in _startServer
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/home/zephyr/workspace/w031/bitbake/lib/bb/cooker.py", line 210, in __init__
self.initConfigurationData()
File "/home/zephyr/workspace/w031/bitbake/lib/bb/cooker.py", line 375, in initConfigurationData
self.databuilder.parseBaseConfiguration()
File "/home/zephyr/workspace/w031/bitbake/lib/bb/cookerdata.py", line 317, in parseBaseConfiguration
raise bb.BBHandledException
bb.BBHandledException
ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed:
gawk
First, change local.conf, bblayers.conf to previous configuration.
Then, bitbake -c cleanall recipe_name.
It will be all right now!
Like the OP stated a package was missing on the build host. ( makeinfo in his case)
To properly prepare the build host, look into the documentation for your yocto version and your Distro.
latest
In my case some playing with devtool caused a duplicated definition in bblayers.conf:
BBLAYERS ?= " \
${TOPDIR}/../meta \
${TOPDIR}/../meta-poky \
${TOPDIR}/../meta-yocto-bsp \
${TOPDIR}/../meta-atmel \
${TOPDIR}/../meta-libgpiod \
${TOPDIR}/../meta-libuio \
${TOPDIR}/../meta-lsuio \
${TOPDIR}/workspace \
/home/me/yocto/poky/build/workspace \
"
I had to manually remove one of the two last lines as follows:
BBLAYERS ?= " \
${TOPDIR}/../meta \
${TOPDIR}/../meta-poky \
${TOPDIR}/../meta-yocto-bsp \
${TOPDIR}/../meta-atmel \
${TOPDIR}/../meta-libgpiod \
${TOPDIR}/../meta-libuio \
${TOPDIR}/../meta-lsuio \
${TOPDIR}/workspace
"
Then I retried and the issue was resolved.
In my case (PLNX 2018.2) I was not getting this problem because of the .Xil folder that was hidden in the root directory of the project, deleting it solves the problem.
I had a similar issue; with an additional Unicode Decode Error: 'ascii' codec can't decode byte 0xe2 in position 5305: ordinal not in range(128) at the bottom of the list.
I resolved this by checking my 'locale' setting in Ubuntu 18.04. and running the following:
export LC_ALL="en_US.UTF-8"
export LANG="en_US.UTF-8"
export LANGUAGE="en_US.UTF-8"
bitbake-layers command worked perfectly after this.
Please shutdown and rerun that bitbake command then it will solve.