Using openembedded-core Morty branch:
For whatever reason, python 2 has these package but when attempting to run:
bitbake python3-glob
and
bitbake python3-shutil
These packages are not available. When installing just python3 as part of a project they are missing as well.
However, another "standard package" (not sure if this is an accurate statement but this comes included in python3 on debian) subprocess can be installed by adding python3-subprocess as a dependency.
Is glob and shutil part of some larger python3 standard packages recipe?
Found it:
the openembedded-core Python3 recipe comes with a file called python-3.5-manifest. This file outlines different sub-recipes of python3.
glob and shutil are available by building python3-shell.
It appears for these to show up in an image they must be added to IMAGE_INSTALL in your image recipe.
Related
When using setup.py in a Python project, we can simply run
$ python3 setup.py --version
And this will give us the version field that is set in the setup.py file. This saves us using sed or something alike to read the version when we need to do so in a script.
I was wondering, is there an equivalent way in a setup.cfg project?
When you have only a setup.cfg, and no setup.py file, you can do this:
$ cat setup.cfg
[metadata]
name = mypkg
version = "0.1.2.3" # or `file: VERSION.txt` or `attr: mypkg.__version__` etc
$ python3 -c 'import setuptools; setuptools.setup()' --version
"0.1.2.3"
If the build backend is setuptools, then this approach will also work for a source tree with a pyproject.toml file.
Allow me to deviate from your actual question to offer what I thing is a better solution.
If you have to get the package version while developing (from an outside script) then you should split such information from your setup.cfg/setup.py.
You want setup.cfg (i.e, setuptools) to get that information from a file, for instance. Then, you can have the version just by reading the corresponding file.
Check the docs for details: https://setuptools.pypa.io/en/latest/userguide/declarative_config.html. Notice the #meta-1 anchor/note in that page.
Basically, you'll use attr or file attributes (in your setup.cfg metadata section) to get the version from another place:
version = file: VERSION.txt
There are plenty of perl packages missing in Centos 8 and Rocky Linux. So, I try to get the rpm spec by cpanspec and build rpm by myself. But, it seems like that rpmbuild could not find the rpm I built.
This is the script for me to build rpm.
cd /root/rpmbuild
cpanspec --packer 'Example <example#example.com>' <Perl-Package-Name>
mkdir SOURCES
cp <Perl-Package-Name>.tar.gz SOURCES
rpmbuild -ba perl-<Package-Name>.spec
Let's say we have two package A and B. A is needed by B.
I try to build both of the packages through the script above. I build A first, switch into /root/rpmbuild/RPMS/noarch and install A.rpm. Then, I try to build package B.
I got
error: Failed build dependencies:
perl(A) is needed by perl-<B>
I try to check the existence of package A.
yum list installed | grep A
and
perldoc -l A
Both of the commands show that A exists.
Did I miss something?
update 2022/06/07
I just gave up and commented the BuildRequires: A in B package. This is not a good approach but it works.
Last time I've followed recommended src layout (https://hynek.me/articles/testing-packaging/) with using tox with great success.
However VSCODE tests discovery fails because src package cannot be imported. That is expected as we want to test installed package.
But how to debug my tests in vscode?
(Q author here: I've done research on that before posting the question, so sharing what I found)
Not solution
You could modify your PYTHONPATH to point to your src directory, but it breaks the main benefit from having separate src directory (read the link from OP).
Solution
Use pip install -e path/to/your/package (usually pip install -e .) to enable development mode and test versus your codebase as it would be installed.
After that your tests should be discovered properly. Otherwise it is different issue - read vs code OUTPUT console.
Note: this requires setup.py as a build backend
workaround for poetry
pyproject.toml
[build-system]
requires = [
"poetry-core>=1.0.0",
"setuptools" # to support local installations
]
then
poetry build --format sdist && tar --wildcards -xvf dist/*.tar.gz -O '*/setup.py' > setup.py
pip install -e .
Source: https://github.com/python-poetry/poetry/issues/34
TLDR: proper solution is outside of poetry scope, links to python-list discussions: https://github.com/python-poetry/poetry/issues/34#issuecomment-732478605
my setup.py is like below:
from setuptools import find_packages, setup
tests_require = ['pytest-env', 'pytest-mock', 'pytest-cov', 'pytest-xdist', 'pytest', 'mock', 'moto<=1.3.10']
setup(
name='repo_name',
version='0.1.0',
description='repo_name',
keywords=['?'],
packages=find_packages('src', exclude=['tests', 'venv']),
package_dir={'': 'src'},
package_data={'': ['schema/*.yaml']},
install_requires=[
'boto3<=1.10.19',
'requests<=2.22.0',
'jsonschema<=3.0.1',
'objectpath<=0.6.1',
'pyyaml<=5.1.1',
'sqlalchemy<=1.3.5',
'psycopg2-binary<=2.8.3',
'auth-client<=1.0.23', # internal package
'policy_client<=1.0.9', # internal package
'audit-client<=1.1.20', # internal package
'flask<=1.1.1',
'click<=7.0',
'Werkzeug<=0.15.5',
'itsdangerous<=1.1.0',
'Jinja2<=2.10.1',
'MarkupSafe<=1.1.1',
'structlog<=19.2.0',
'python-rapidjson<=0.9.1'
],
setup_requires=['pytest-runner'],
tests_require=tests_require,
extras_require={'test': tests_require},
include_package_data=True,
zip_safe=False
)
When I run python setup.py test, it will always reinstall the most of the packages that I already installed into .eggs folder, which I understand. While it's installing the package with different versions that I provided in the install_requires section, which results in below error:
pkg_resources.ContextualVersionConflict: (urllib3 1.25.10 (/Users/***/Desktop/repo/.eggs/urllib3-1.25.10-py3.7.egg), Requirement.parse('urllib3<1.25,>=1.21.1'), {'requests'})
I have to pin the requests version to 2.22.0 which requires urllib3 version no larger than 1.25, which caused my issue. So, is there anyway I can pin the urllib3 version in somewhere so when it runs, it will only install the provided version? I have tried to add the urllib3 version in install_requires, tests_require and in requirements.txt, but no luck so far.
Probably not what you want to hear but... setup_requires and tests_require are deprecated. Although they (probably) still work (mostly) fine, you might want to look for alternative solutions. As far as I know the most common alternative solutions are to use Tox instead of tests_require and PEP 517 to replace setup_requires.
In your case, it seems that your usage of setup_requires is only a consequence of your usage of tests_require. So by switching to something like Tox you get rid of both tests_require and setup_requires.
I got it resolve to simply replace python setup.py test to be pip instll -e . [test], and then pytest directly.
It will still install all the tests_require packages and then run pytest directly. Instead of going through all the list packages and find the best match versions for all packages.
UPDATES:
The real problem is I did not remove the old .eggs/ and venv/ folder when I made the packages version change. So the solution is updating the requests version to be 2.21.0 in the setup.py file, then remove the .eggs/ and venv/ folder and rerun everything.
How do I install PIL?
>pip install PIL
Downloading/unpacking PIL
Could not find any downloads that satisfy the requirement PIL
Some externally hosted files were ignored (use --allow-external PIL to allow).
Cleaning up...
No distributions at all found for PIL
Storing debug log for failure in /root/.pip/pip.log
>pip uninstall PIL
Can't uninstall 'PIL'. No files were found to uninstall.
pip install PIL --allow-external PIL --allow-unverified PIL
This is due to changes in the new version of Pip. Run pip --version and I'm willing to bet you are running 1.5. See the changelog here. This new default behavior enhances security. In PIL's case, the file you are installing actually comes from effbot.org (thus --allow-external) and PyPi doesn't have a checksum to guarantee validity (thus --allow-unverified).
Also, you might consider using the Pillow replacement to PIL.
Updated info for those reading in 2016:
--allow-external
and
--allow-unverified
were recently deprecated. Installing packages external to PyPi using pip is no longer supported: http://www.python.org/dev/peps/pep-0470/
As an alternative, when you really need to install that external package, you can download the source code and run its setup.py. For example, for PIL 1.1.7, download from http://www.pythonware.com/products/pil/, then:
$ tar xvfz Imaging-1.1.7.tar.gz
$ cd Imaging-1.1.7
$ python setup.py install
(^ from the PIL README)
If you only want to install the package to a specific virtualenv, you can just activate your virtualenv first. ** thanks #Caumons
Alternatively, substitute the path to your virtualenv for 'python' in the third line, e.g.:
$ /home/username/virtualenv-name/bin/python setup.py install