I am trying to make a python wheel package using setup.py. The folder structure looks like the one mentioned below. (I am using windows 10)
D:.
│ setup.py
│
│
└───folder_2
├───folder_3
└───folder_4
├───folder_5
└───folder_6
Note: setup.py is on the same level as folder_2.
Here, while making a wheel package for folder_2, I want to include the folder_4 but, I do not want to include the folder_5. I only want folder_6.
For folder_5 I want to create a different wheel package in which folder_6 will not be included.
For this, I have tried many methods from the setuptool documentation. None of them seem to work.
The program that I am using right now is also given below.
import setuptools
setuptools.setup(name="example",
version="0.1.0",
packages=setuptools.find_packages(where="folder_2",
exclude=['folder_4.folder_5']),
package_dir={"": "folder_2"},
include_package_data=True
)
I have also tried to use include() instead of exclude().
Can someone please help me with this?
import setuptools
setuptools.setup(name="example",
version="0.1.0",
packages=setuptools.find_packages(where="folder_2",
exclude=['folder_4.folder_5']),
package_dir={"": "folder_2"},
include_package_data=True
)
Related
I'm working on a pybind11 extension written in C++ but I'm having a hard time understanding how should it be distributed.
The project links to a number of third party libraries (e.g. libpng, glew etc.).
The project builds fine with CMAKE and it generates a .so file. Now I am not sure what is the right way of installing this extension. The extension seems to work, as if I try copy the file into the python lib directories it is picked up (I can import it, and it works correctly). However, this is clearly not the way to go I think.
I also tried the setuptools route (from https://pybind11.readthedocs.io/en/stable/compiling.html) by creating a setup.py files like this:
import sys
# Available at setup time due to pyproject.toml
from pybind11 import get_cmake_dir
from pybind11.setup_helpers import Pybind11Extension, build_ext
from setuptools import setup
from glob import glob
files = sorted(glob("*.cpp"))
__version__ = "0.0.1"
ext_modules = [
Pybind11Extension("mylib",
files,
# Example: passing in the version to the compiled code
define_macros = [('VERSION_INFO', __version__)],
),
]
setup(
name="mylib",
version=__version__,
author="fab",
author_email="fab#fab",
url="https://github.com/pybind/python_example",
description="mylib",
long_description="",
ext_modules=ext_modules,
extras_require={"test": "pytest"},
cmdclass={"build_ext": build_ext},
zip_safe=False,
python_requires=">=3.7",
)
and now I can build the extension by simply calling
pip3 install
however it looks like all the links are broken because whenever I try importing the extension in Python I get linkage errors, as if setuptools does not link correctly the extension with the 3rd party libs. For instance errors in linking with libpng as in:
>>> import mylib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /home/fabrizio/.local/lib/python3.8/site-packages/mylib.cpython-38-x86_64-linux-gnu.so: undefined symbol: png_sig_cmp
However I have no clue how to add this link info to setuptools, and don't even know if that's possible (it should be the setuptools equivalent of CMAKE's target_link_libraries).
I am really at a loss after weeks of reading documentation, forum threads and failed attempts. If anyone is able to point me in the right way or to clear some of the fog it would be really appreciated!
Thanks!
Fab
/home/fabrizio/.local/lib/python3.8/site-packages/mylib.cpython-38-x86_64-linux-gnu.so: undefined symbol: png_sig_cmp
This line pretty much says it clearly. Your local shared object file .so can't find the libpng.so against which it is linked.
You can confirm this by running:
ldd /home/fabrizio/.local/lib/python3.8/site-packages/mylib.cpython-38-x86_64-linux-gnu.so
There is no equivalent of target_link_libraries() in setuptools. Because that wouldn't make any sense. The library is already built and you've already linked it. This is your system more or less telling you that it can't find the libraries it needs. And those most likely need to be installed.
This is also one of the reasons why Linux distributions provide their own package managers and why you should use the developer packages provided by said distributions.
So how do you fix this? Well your .so file needs to find the other .so files against which you linked to understand how this works I will refer you to this link.
My main guess is based on the fact that when you manually copy the files it works - That during the build process you probably specify the rpath to a local directory. Hence what you most likely need to do is specify to your setuptools that it needs to copy those files when installing.
My flutter project depends on several local flutter and dart packages to keep things separated and clean.
My folder structure is like this:
main-flutter-project
│ lib
| test
│ pubspec.yaml
│
└── local-packages
│ └── dart-package-1
│ │ pubspec.yaml
│ │
│ └── flutter-package-1
│ │ pubspec.yaml
│ │
│ └── flutter-package-2
│ pubspec.yaml
...
Each local package is self contained and can be maintained without touching the main project.
This structure means that I have many pubspec.yaml files where I have to keep the dependencies updated.
When I use e.g. the bloc libaray bloc: ^7.2.1 in say 5 packages, I have to update the version in each pubspec file separately when a new version is released.
Is there a possibility to specify those shared package dependency versions in only one place where the other pubspec.yaml files refer to?
I've seen this e.g. with Maven where you can specify a property <junit.version>4.12</junit.version> and access it from somewhere else <version>${junit.version}</version>.
We were solving a similar problem.
AFAIK, there's no built-in or recommended way to do this, so we were inventing some hacks.
In our case, we have core package that has some shared functionality and common dependencies, if you don't have it, you can still create an artificial one, let's say, shared_dependencies package, and specify all the shared dependencies there.
Now, let's say, package foo depends on shared_dependencies package, and there's dependency bar defined in shared_dependecies package that foo needs to use. There are some ways to do that:
Import dependency directly. Since foo depends transitively on bar, you can just write import package:bar/bar.dart and it will work. It's not the best way though:
it's bad practice to import transitive dependency (there's even a linter rule for that);
auto-import won't work;
Export package in shared_dependencies package. I.e. shared_dependencies.dart can contain the following lines:
export 'package:bar/bar.dart'
That means that in your foo package you can just write import 'shared_dependencies/shared_dependencies.dart' and get access to bar content.
Pros:
auto-imports work.
Contras:
if you export several packages, there can be name conflicts (you'll have to hide some names in export);
if foo package depends on one bar package only, it could be weird to import all shared_dependencies.
Export in separate libraries of shared_dependencies package. You can group some related packages together in different files, e.g.:
bar.dart:
export 'package:bar/bar.dart'
bloc.dart:
export 'package:bloc_concurrency/bloc_concurrency.dart';
export 'package:flutter_bloc/flutter_bloc.dart';
In that case, if you need bar package in foo, you write import 'package:shared_dependencies/bar.dart'; if you need bloc, you write import 'package:shared_dependencies/bloc.dart'. Auto-imports work as well.
Add direct dependency to foo package, but don't specify version constraints:
bar:
This basically means that you need any bar package, but since foo also depends on shared_dependencies, its constraints will be taken into account. This may be needed if you're using some executables from bar package, as there's a limitation in Dart SDK that doesn't allow to run executables in transitive dependencies.
In our project, we ended up using 2 for the most commonly used packages, 3 for other packages, and 4 for packages with executables that we need to run.
This question is specific to PyDev. The package structure looks like this:
app
├── __init__.py
├── sub1
│ ├── __init__.py
│ └── mod1.py
└── sub2
├── __init__.py
└── mod2.py
The mod1.py module:
from __future__ import print_function
def f():
print('It works!')
The mod2.py module:
from __future__ import absolute_import
from ..sub1 import mod1
if __name__ == '__main__':
mod1.f()
Everything works beautifully from the shell, the python -m app.sub2.mod2 command prints:
It works!
as expected, all is fine. (The from __future__ import absolute_import line seems to have no effect: I can comment it out and everything still works just fine.)
If I click on mod2 in the PyDev IDE and try to Run As > Python Run, I get
ValueError: Attempted relative import in non-package
which is not surprising as the -m switch is not turned on by default. If I edit the Run/Debug settings for mod2: Arguments > VM Arguments and add -m here; the -m is most likely passed to the python interpreter but now I get:
/usr/bin/python: Import by filename is not supported.
The from __future__ import absolute_import line seems to have no effect; it does not matter whether I comment it out or not; I am using Python 2.7.
I am out of ideas at this point.
In PyDev, how can I run a module inside a package that uses relative
imports?
How should I change the settings once (globally) such that whenever I
try to run a module inside a package, PyDev does the right thing?
(That is, I don't have to individually specify the settings for each
module that I wish to run.)
The developer in person confirmed that it is not possible in PyDev yet; I have opened a ticket for it.
Running a module inside a package, using relative imports
UPDATE: As of Dec 2, 2016, the issue is resolved, see the accepted answer.
Edit:
In PyDev 5.4.0, there's now an option to run using the -m flag (which will import the module through its regular name and not as it was __main__ so that relative imports will work there).
You can enable it at: Preferences > PyDev > Run (i.e.: this will enable it for all runs -- maybe in the future there'll be an option to make it per run, but for now it's set globally for all launches).
Original answer:
The problem is that you have relative imports in your main module and PyDev executes the file with python path/to/file_to_execute.py instead of python -m my.module.
A simple fix is doing a separate main module which in turn imports a main() function from that module and runs it (although again: it can't have relative imports in the module executed as __main__ (this happens because the module is called __main__ and thus cannot resolve a relative import because it wasn't actually imported with a name which can be used to resolve the relative import).
Another fix would be changing the launch configuration to add the '-m my.module' in the VM arguments (go to run > run configurations to do that -- but you have to do that for each main module you want to run, including unit-tests).
And the last fix would be changing PyDev itself (so, please create a ticket for that in the PyDev tracker: https://www.brainwy.com/tracker/PyDev/ -- or submit a pull request, which would make adding that feature much faster ;) )
I am trying to import a module in my python script and I can't make it work.
So I have my python script: /home/user/pythonscript/oneDir/onescript.py
And I would like to use a script that is a directory higher in hierarchy:
/home/user/pythonscript/common.py
So I did the following at the top of my onescript.py:
import sys
sys.path.insert(1,'/home/user/pythonscript')
import common
In my common.py file, I have a function onecConnect, and when I try to run onescript.py, which uses onecConnect function, I get the following error: nameError: name 'onecConnect' is not defined
Anyone can see what I do wrong or forgot to do?
Thanks
Make sure there are __init__.py in all directories, go to /home/user/pythonscript and run Python code from there. So:
python oneDir/onescript.py
In onescript.py you can do:
from common import onecConnect
The rules are:
Always run a Python script from the highest possible directory (not the deepest into the project).
Always have full import lines, no relative imports.
This keeps the problems away.
I can't use functions of custom subdirectories.
My Code Organziation
I have under "src" a path hierarchy like
a/b
with all my directories and go-Files (it is the "root" of my project). The directories contain no subdirectories and it works fine. So the deepest path is "a/b/c". E.g. I have
a/b/c
and
a/b/d
with some go-files. Import of "a/b/d" and calling a function with "d.DoSomething()" from a file in "a/b/c" works fine.
Problem description
Now I want ot reorganize "a/b/d". I move some files from "a/b/d" to
a/b/d/e
and the rest of the files to
a/b/d/f
If try to import "a/b/d/e" with import-statement
import ( "a/b/d/e" )
from the same file in "/a/b/c" and want to call "e.DoSomething()" (it is the place, where the file with the "DoSomething-function" moved to), I get an error at the line, where I call "e.DoSomething()": "undefined: e".
While searching for a result, I've nowhere seen examples with deeper path hierarchies. Is it generally not possible to use/import subdirectories or what's the problem?
go-version I used: go1.2.2 linux/amd64
Thanks for any advices
Your approach is completely wrong. Go has absolutely no concept of importing files or directories, all you can import in Go are packages. It now happens that the name of a package is it's path relative to GOPATH and you import packages by that name. But the identifier under which an imported package is available in the importing code depends on the package declaration of the package. You cannot simply "move" files between directories as each directory (for the go tool) is a single package without changing the package declaration.
You can have package x under path a/b/c. When you import package x with import ( "a/b/c" ) all the exported stuff from package x is available to you as x.ExportedName.
Please read http://blog.golang.org/organizing-go-code carefully.
Try and do a go build in a/b/d/e first, before trying to build in a/b: that will generate the compiled classes you want to import.