I have trouble creating a package with setuptools. I have a repository which I'm cleaning up to make it a package. The directory structure looks something like this
my-proj
├── setup.py
├── MANIFEST.in
├── MakeFile
├── README.rst
├── setup.py
└── myproj
├── __init__.py
├── my_code.py
├── templates
│ ├── template1.yaml
│ ├── template2.yaml
Initial version of "my_code.py" had code snippet which would directly reference the files withing templates folder to do some processing. If I package this using setup tools, I provide the following information in these files:
MANIFEST.in:
include README.rst
include requirements.txt
include LICENSE.txt
recursive-include myproj/templates *
setup.py:
setup(
name='myproj',
package_dir={'testbed_init': 'testbed_init'},
package_data={'templates': ['templates/*'], 'configs': ['configs/*']},
include_package_data=True,
)
My question is as follows. In "my_Code.py" I used to reference templates directly without any problem as I would run script from the myproj folder. If I package this, how can I make sure, I include the templates as part of package and when script runs, I need to open the templates relative to where the package is installed.
Code snippet from my_code.py:
if _type == "a":
temp_file = f"templates/template1.yaml"
else:
temp_file = f"templates/template2.yaml"
build_config(deploy_esx_file, output_file, data)
Code snippet of what happens in build_config:
def build_config(template_file, output_file, inputs):
templateLoader = jinja2.FileSystemLoader(searchpath="./")
templateEnv = jinja2.Environment(loader=templateLoader)
template = templateEnv.get_template(template_file)
outputText = template.render(inputs)
with open(output_file, 'w') as h:
h.write(outputText)
Related
I have the next tree:
root_project/
├── app
│ ├── default_photo_profile.jpg
│ ├── config.py
│ ├── __main__.py # My app are python package, I'm runnig it via "python -m"
│ └── ...
├── tests
│ ├── test_unit.py # import config.py inside
│ ├── functional # import config.py inside
│ ├── pytest.ini
│ └── ...
...
Currently default_photo_profile causing error because tests doesn't have this file.
Reading file in config.py:
DEFAULT_PHOTO_FILE_PATH = Path('default_photo.jpg')
with open(file=DEFAULT_PHOTO_FILE_PATH, mode='rb') as file_obj:
DEFAULT_PHOTO_BYTES = file_obj.read()
How I can solve this?
I tried:
Patch access to default_photo.jpg with fixture - not helped, error during import stage, not executiion.
set flag to pytest comamnd line: --rootdir app - not helped (don't know why).
try/except for reading the file in app.config.py - may help but it's not my intention, I really want raise error if file not found
Put default_photo.jpg inside EVERY test directory - will help bit dirty.
Patch os.path like suggested in https://stackoverflow.com/a/43003192/11277611 - dirty
Include tests into package (move __main__.py into root_project - not sure that it's a good idea (have not enough experience to decide).
Set absolut path to default_photo.jpg - will fail on the production server.
Probably adoptable solutions (What I want):
Set root dir to root_project.app somehow inside pytest.ini to immitate regular execution.
Set root dir to root_project.tests somehow to place file in root of tests and access from any of tests folder.
Try to use following code in config.py:
DEFAULT_PHOTO_FILE_PATH = Path(__file__).parent / 'default_photo.jpg'
with open(file=DEFAULT_PHOTO_FILE_PATH, mode='rb') as file_obj:
DEFAULT_PHOTO_BYTES = file_obj.read()
Is it what you are trying to achieve?
In a Dist::Zilla-based distribution I would like to have some files that are only used for testing, but do not get installed. These are mockup libs that aren't needed for runtime.
How do I do that?
CPAN distributions never install the t and xt directories. You can put your tests and your mock libs into t.
As an example, take my module MooseX::LocalAttribute. In the dist, there is a t/, a t/lib and an xt/.
If you install this using cpanm -l into a local lib dir, you will see there are no tests installed. This happens automatically. It's just how CPAN works.
$ cpanm -l mylib MooseX::LocalAttribute
--> Working on MooseX::LocalAttribute
Fetching http://www.cpan.org/authors/id/S/SI/SIMBABQUE/MooseX-LocalAttribute-0.05.tar.gz ... OK
Configuring MooseX-LocalAttribute-0.05 ... OK
Building and testing MooseX-LocalAttribute-0.05 ... OK
Successfully installed MooseX-LocalAttribute-0.05
1 distribution installed
$ tree mylib
mylib
├── lib
│ └── perl5
│ ├── MooseX
│ │ └── LocalAttribute.pm
│ └── x86_64-linux
│ ├── auto
│ │ └── MooseX
│ │ └── LocalAttribute
│ └── perllocal.pod
└── man
└── man3
└── MooseX::LocalAttribute.3
9 directories, 3 files
Note that as long as stuff is in t/lib (or anywhere under t/, really), you do not have to hide the package names from the PAUSE indexer. It's smart enough to not find it.
I misunderstood the question. This answer is for the following question:
How do I exclude files from a Dist::Zilla based distribution so they don't get shipped at all?
You are probably using either the GatherDir or Git::GatherDir plugin to build your bundle. Both of them have an option exclude_filename that you can set in your dist.ini to not include a file in a bundle.
A common pattern is to exclude auto-generated files such as LICENSE or META.json, and then add them later with another plugin. But you don't have to do that, you can just exclude files completely.
A good example is the URI distribution. On metacpan, it does not include any text files in the bundle. But if you look at the repository on github, you can see there are various .txt files such as rfc2396.txt. The dist.ini contains the following lines.
[Git::GatherDir]
exclude_filename = LICENSE
exclude_filename = README.md
exclude_filename = draft-duerst-iri-bis.txt
exclude_filename = rfc2396.txt
exclude_filename = rfc3986.txt
exclude_filename = rfc3987.txt
As mentioned before, the LICENSE and README.md files will still appear in the final bundle, because they get added later via #Git::VersionManager.
This is a much asked question, but none of the solutions mentioned on SO have worked so far.
The folder structure is as follows:
project/
└── tests/
├── conftest.py
├── __init__.py
└── int_tests/
└── test_device.py
└── project_core/
└── tests/
├── conftest.py
├── __init__.py
└── int_tests/
└── test_device.py
import file mismatch:
imported module 'test_device' has this __file__ attribute:
/home/.../project/project_core/tests/int_tests/test_device.py
which is not the same as the test file we want to collect:
/home/.../project/tests/int_tests/test_device.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
Steps tried so far:
Removing pycache and pyc files.
Adding _init to each folder. (As is stated in pytest GIP)
Removing _init from each folder.
Do i need init files in each tests/subfolder?
The same error occurs with conftest.py as well. This error is not limited to vscode-pytest plugin, also occurs on the terminal.
PS : For CI purposes, the system is configured with docker & tox. Development is done in venv.
Suppose I have the following directory structure:
src/
└── python/
└── generated/
├── __init__.py
├── a.py
└── lib/
├── __init__.py
└── b.py
What does my setup.py need to look like in order to create a dist with a directory layout like:
src/
└── python/
├── __init__.py
├── a.py
└── lib/
├── __init__.py
└── b.py
The goal is to simply eliminate the generated folder. I've tried endless variations with package_dir and can't get anything produced other than the original directory structure.
Your setup.py should be placed in your src directory and should look like this:
#!/usr/bin/env python3
import setuptools
setuptools.setup(
name='Thing',
version='1.2.3',
packages=[
'python',
'python.lib',
],
package_dir={
'python': 'python/generated',
},
)
Note the package_dir setting. It instructs setuptools to get the code for the python package from the directory python/generated. In the built distributions you will then find the right directory structure.
First, here is my solution:
#!/usr/bin/env python
import os, shutil
from setuptools import setup
from setuptools.command.build_py import build_py
class BuildPyCommand(build_py):
"""Custom build command."""
def run(self):
shutil.rmtree('src.tmp', ignore_errors=True)
os.mkdir('src.tmp')
shutil.copytree('src/python/generated', 'src.tmp/python')
build_py.run(self)
setup(cmdclass={ 'build_py': BuildPyCommand },
name='Blabla',
version='1.0',
description='best desc ever',
author='Me',
packages=['python', 'python.lib'],
package_dir={'': 'src.tmp'},
setup_requires=['wheel']
)
And you can generate your distribution with:
python setup.py build bdist_wheel
The idea is perform a two steps build:
I generate a valid source structure
I build this temporary structure
And I deliver it in a wheel because it doesn't require future users to understand my trick. If you give it a try with a source distribution, you will notice that you need to publish the generated files as data (not difficult, but troublesome, and, I guess you will want to hide your tricks from your users).
But, I think that there is a design flaw in your process. The file src/python/generated/__init__.py, assumed to be a module <something>.generated eventually becomes your <something>.python, which is troublesome. It would be much simpler and more robust to generate a valid Python structure: src/generated/python/__init__.py. The setup.py would become trivial and your generator wouldn't be more complex.
I want to distribute a gzipped tarball containing a program binary.
No problem making a package containing the binary plus additional script/man files because these are files I install with CMake INSTALL command during a source installation.
While the need for such a functionality seems quite obvious to me, I didn't find a way to include text files in CPack tarball packages.
Here is an example of the final tarball I'd like to get:
myprogram-1.0.0rc-Darwin-i386/
├── LICENSE <- not installed by CMake
├── INSTALL <- not installed by CMake
├── README <- not installed by CMake
├── bin
│ └── myprogram
└── share
└── myprogram
└── man
└── man1
└── myprogram.1
Any help would be very much appreciated.
Cheers
You can use install too with FILES or DIRECTORY option.
install(FILES ${CMAKE_CURRENT_SOURCE_DIR}/README.txt DESTINATION .)