According to docs https://packaging.python.org/en/latest/distributing/#data-files
setuptools will honor data_files configed in setup.py. But i can't make it work. This is my setup.py:
setup(
name='booking_order',
version=version,
packages=find_packages(),
package_data={
'booking_order': ['fake_backends/static/*',
'scripts/*',
'*.sample'],
},
data_files=[
('/etc/booking', ['etc/booking.conf'])
],
This is the project's file tree:
.
├── booking_order
│ ├── __init__.py
│ ├── tests
│ │ ├── __init__.py
├── etc
│ ├── booking.conf
├── README.md
├── setup.py
The behavior is, if i run python setup.py install, file etc/booking.conf will got installed to /etc/booking. But if i first python setup.py sdist upload, then pip install booking_order, there will be an error "error: can't copy 'etc/booking.conf': doesn't exist or not a regular file".
I checked python setup.py sdist doesn't include files in etc at all.
EDIT:
it seems this is the answer: https://github.com/pypa/setuptools/issues/521
Answer it myself.
According to pypa, and non-package-data-files。"Setuptools doesn't support installing data files to some arbitrary location on a user’s machine; this is a feature, not a bug."
If one need to install files to locations like /etc, /usr/share, eg, then he/she may use data_files flag from distutils, which feature is not totally cleaned up from setuptools. "Not totally cleaned up" means you need to add those files to MANIFEST.in manually, which is different as in distutils.
Of course, it will be better if one can manage these configuration files with rpm or deb package system. For me it's just a temporary solution to use pip here.
Related
In a Dist::Zilla-based distribution I would like to have some files that are only used for testing, but do not get installed. These are mockup libs that aren't needed for runtime.
How do I do that?
CPAN distributions never install the t and xt directories. You can put your tests and your mock libs into t.
As an example, take my module MooseX::LocalAttribute. In the dist, there is a t/, a t/lib and an xt/.
If you install this using cpanm -l into a local lib dir, you will see there are no tests installed. This happens automatically. It's just how CPAN works.
$ cpanm -l mylib MooseX::LocalAttribute
--> Working on MooseX::LocalAttribute
Fetching http://www.cpan.org/authors/id/S/SI/SIMBABQUE/MooseX-LocalAttribute-0.05.tar.gz ... OK
Configuring MooseX-LocalAttribute-0.05 ... OK
Building and testing MooseX-LocalAttribute-0.05 ... OK
Successfully installed MooseX-LocalAttribute-0.05
1 distribution installed
$ tree mylib
mylib
├── lib
│ └── perl5
│ ├── MooseX
│ │ └── LocalAttribute.pm
│ └── x86_64-linux
│ ├── auto
│ │ └── MooseX
│ │ └── LocalAttribute
│ └── perllocal.pod
└── man
└── man3
└── MooseX::LocalAttribute.3
9 directories, 3 files
Note that as long as stuff is in t/lib (or anywhere under t/, really), you do not have to hide the package names from the PAUSE indexer. It's smart enough to not find it.
I misunderstood the question. This answer is for the following question:
How do I exclude files from a Dist::Zilla based distribution so they don't get shipped at all?
You are probably using either the GatherDir or Git::GatherDir plugin to build your bundle. Both of them have an option exclude_filename that you can set in your dist.ini to not include a file in a bundle.
A common pattern is to exclude auto-generated files such as LICENSE or META.json, and then add them later with another plugin. But you don't have to do that, you can just exclude files completely.
A good example is the URI distribution. On metacpan, it does not include any text files in the bundle. But if you look at the repository on github, you can see there are various .txt files such as rfc2396.txt. The dist.ini contains the following lines.
[Git::GatherDir]
exclude_filename = LICENSE
exclude_filename = README.md
exclude_filename = draft-duerst-iri-bis.txt
exclude_filename = rfc2396.txt
exclude_filename = rfc3986.txt
exclude_filename = rfc3987.txt
As mentioned before, the LICENSE and README.md files will still appear in the final bundle, because they get added later via #Git::VersionManager.
This is a much asked question, but none of the solutions mentioned on SO have worked so far.
The folder structure is as follows:
project/
└── tests/
├── conftest.py
├── __init__.py
└── int_tests/
└── test_device.py
└── project_core/
└── tests/
├── conftest.py
├── __init__.py
└── int_tests/
└── test_device.py
import file mismatch:
imported module 'test_device' has this __file__ attribute:
/home/.../project/project_core/tests/int_tests/test_device.py
which is not the same as the test file we want to collect:
/home/.../project/tests/int_tests/test_device.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
Steps tried so far:
Removing pycache and pyc files.
Adding _init to each folder. (As is stated in pytest GIP)
Removing _init from each folder.
Do i need init files in each tests/subfolder?
The same error occurs with conftest.py as well. This error is not limited to vscode-pytest plugin, also occurs on the terminal.
PS : For CI purposes, the system is configured with docker & tox. Development is done in venv.
I have used vscode 1.41.1 on my mac for a few months and it worked good until I started to use go modules for dependency management. At the moment I am rewriting a simple tool and introduce packages for separate functionalities.
My code structure looks like this:
├── bmr.go -> package main & main(), uses below packages
├── check
│ ├── check.go -> package check
│ └── check_test.go
├── go.mod
├── go.sum
├── push
│ ├── push.go -> package push
│ └── push_test.go
└── s3objects
├── s3objects.go -> package s3objects
└── s3objects_test.go
My go.mod file:
module github.com/some-org/business-metrics-restore
go 1.13
require (
github.com/aws/aws-sdk-go v1.28.1
github.com/go-redis/redis v6.15.6+incompatible
github.com/sirupsen/logrus v1.4.2
github.com/spf13/viper v1.6.1
github.com/stretchr/testify v1.4.0
golang.org/x/sys v0.0.0-20200113162924-86b910548bc1
)
All is fine when I invoke go test/run/build commands from the shell. But when I use 'Debug' -> 'Run Without Debugging' I get:
go: finding github.com/some-org/business-metrics-restore/push latest
go: finding github.com/some-org/business-metrics-restore latest
go: finding github.com/some-org/business-metrics-restore/check latest
go: finding github.com/some-org/business-metrics-restore/s3objects latest
build command-line-arguments: cannot load github.com/some-org/business-metrics-restore/check: module github.com/some-org/business-metrics-restore#latest found (v0.0.0-20191022092726-d1a52439dad8), but does not contain package github.com/some-org/business-metrics-restore/check
Process exiting with code: 1
My code currently is in a feature branch and d1a52439dad8 is the first (init) and only commit on master. No code for the tool (incl. 3 mentioned non main packages) is in the master branch.
The problem here is that for some reason as you see above vscode fetches state from master and I cannot override this behaviour.
Can anyone help me?
Thanks!
Best Regards,
Rafal.
I realized that if the go.mod is not at the root of your project VSCode does not work properly. I have an AWS SAM project with the following structure:
├── Makefile
├── README.md
├── nic-update
│ ├── go.mod
│ ├── go.sum
│ ├── main.go
│ ├── main_test.go
│ └── r53service
│ └── r53.go
├── samconfig.toml
└── template.yaml
and the only way it works if by starting VSCode from the nic-update directory.
My go.mod has the following content:
require (
github.com/aws/aws-lambda-go v1.13.3
github.com/aws/aws-sdk-go v1.32.12
)
module github.com/jschwindt/ddns-route53/nic-update
go 1.14
I realized that if the go.mod is not at the root of your project VSCode does not work properly
That might now (Oct. 2020) be supported, as a consequence of gopls v0.5.1 and its experimental feature Multi-module workspace support from the proposal 32394.
Even if you don't have multiple modules, a go.mod in a sub-folder (instead of the root folder of your project) will be better managed (if you activate the gopls.experimentalWorkspaceModule setting).
As noted by kayochin in the comments:
The setting should be "gopls": {"build.experimentalWorkspaceModule": true}
See the documentation "gopls / Settings / experimentalWorkspaceModule bool".
I have also had trouble with VS Code and modules. The current status of VS Code support for Go Modules is kept up to date here: https://github.com/golang/vscode-go#Set-up-your-environment
In that link they suggest ditching most of the existing extensions VS Code encourages you to install with Go and instead using the language server gopls with these directions:
Add the below in your settings to use it.
"go.useLanguageServer": true
Note: You will be prompted to install the latest stable version of gopls as and when the Go tools team tag a new version as stable.
You should also fix autoimporting:
Add the setting "go.formatTool": "goimports" and then use Go: Install/Update Tools to install/update goimports as it has recently added support for modules.
When you do these things, keep in mind that you'll also lose a couple of features:
Completion of unimported packages doesnt work
Find references and rename only work in a single package
Suppose I have the following directory structure:
src/
└── python/
└── generated/
├── __init__.py
├── a.py
└── lib/
├── __init__.py
└── b.py
What does my setup.py need to look like in order to create a dist with a directory layout like:
src/
└── python/
├── __init__.py
├── a.py
└── lib/
├── __init__.py
└── b.py
The goal is to simply eliminate the generated folder. I've tried endless variations with package_dir and can't get anything produced other than the original directory structure.
Your setup.py should be placed in your src directory and should look like this:
#!/usr/bin/env python3
import setuptools
setuptools.setup(
name='Thing',
version='1.2.3',
packages=[
'python',
'python.lib',
],
package_dir={
'python': 'python/generated',
},
)
Note the package_dir setting. It instructs setuptools to get the code for the python package from the directory python/generated. In the built distributions you will then find the right directory structure.
First, here is my solution:
#!/usr/bin/env python
import os, shutil
from setuptools import setup
from setuptools.command.build_py import build_py
class BuildPyCommand(build_py):
"""Custom build command."""
def run(self):
shutil.rmtree('src.tmp', ignore_errors=True)
os.mkdir('src.tmp')
shutil.copytree('src/python/generated', 'src.tmp/python')
build_py.run(self)
setup(cmdclass={ 'build_py': BuildPyCommand },
name='Blabla',
version='1.0',
description='best desc ever',
author='Me',
packages=['python', 'python.lib'],
package_dir={'': 'src.tmp'},
setup_requires=['wheel']
)
And you can generate your distribution with:
python setup.py build bdist_wheel
The idea is perform a two steps build:
I generate a valid source structure
I build this temporary structure
And I deliver it in a wheel because it doesn't require future users to understand my trick. If you give it a try with a source distribution, you will notice that you need to publish the generated files as data (not difficult, but troublesome, and, I guess you will want to hide your tricks from your users).
But, I think that there is a design flaw in your process. The file src/python/generated/__init__.py, assumed to be a module <something>.generated eventually becomes your <something>.python, which is troublesome. It would be much simpler and more robust to generate a valid Python structure: src/generated/python/__init__.py. The setup.py would become trivial and your generator wouldn't be more complex.
I want to distribute a gzipped tarball containing a program binary.
No problem making a package containing the binary plus additional script/man files because these are files I install with CMake INSTALL command during a source installation.
While the need for such a functionality seems quite obvious to me, I didn't find a way to include text files in CPack tarball packages.
Here is an example of the final tarball I'd like to get:
myprogram-1.0.0rc-Darwin-i386/
├── LICENSE <- not installed by CMake
├── INSTALL <- not installed by CMake
├── README <- not installed by CMake
├── bin
│ └── myprogram
└── share
└── myprogram
└── man
└── man1
└── myprogram.1
Any help would be very much appreciated.
Cheers
You can use install too with FILES or DIRECTORY option.
install(FILES ${CMAKE_CURRENT_SOURCE_DIR}/README.txt DESTINATION .)