Console scripts outside the main module aren't packaged along (ModuleNotFoundError) - setuptools

Behold, my tree:
src/
├── bin
│ ├── cli.py
├── mypackage
│ ├── __init__.py
│ ├── api.py
│ ├── models
│ └── utils.py
Now, observe, my setup.cfg:
[options]
zip_safe = False
packages = find:
package_dir =
=src
[options.package_data]
src/bin = *
[options.packages.find]
where =
src
[options.entry_points]
console_scripts =
mycommand = bin.cli:main
That seemed fairly reasonable to me. Yet when I install this package in a new virtual environment (pip install /path/to/package), and I run mycommand, I get a very unreasonable answer: ModuleNotFoundError: No module named 'bin'.
NOTE this does work if I do an editable install!
What am I doing wrong?

Related

How to change the default directory structure of dh_make so that dpkg-buildpackage does not throw any errors

I am trying to create a debian package for a postgreSQL extension Apache-age release 1.1.1 and created the directory structure using dh_make command.
The directory structure is as follows:
age-1.1.1 (project root)
├── debian
│   ├── changelog
│   ├── compat
│   ├── control
│   ├── docs
│   ├── examples
│   ├── links
│   ├── manpages
│   ├── menu
│   ├── postinst
│   ├── postrm
│   ├── preinst
│   ├── prerm
│   ├── rules
│   ├── source
│   └── watch
├── src
└── Makefile
The dpkg-buildpackage -b when run from project-root folder it looks for debian folder, then reads the rule file, then reads the Makefile located in the project root to build the package.
I want to change the directory structure to the following:
.project root
├── packaging
│ ├── debian
│ │ ├── control
│ │ ├── control.in
│ │ ├── changelog
│ │ ├── copyright
│ │ ├── pgversions
│ │ ├── rules
│ │ └── ...
│ └──
├── src
├── LICENSE
├── README.md
├── Makefile
└── ...
I want to change the directory structure so that the dpkg-buildpackage -b command can be run from the packaging folder and it should build the package.
Inside your Makefile
Modify the install paths accordingly. It should point to your packaging/debian/* where * is the filename.
This way the Makefile can point to the correct file path target inside the new folder structure.
I'm not sure if this is the best way to do this but it's working for me:
Here are the steps:
First run the dh_make_pgxs command from the project root directory.
Create a packaging directory in the project root and move the debian directory created in step 1 to this directory along with the Makefile, age.control and the age--1.1.1.sql.
Your file structure should look like this:
.project root
├── packaging
│   ├── debian
│   │   ├── control
│   │   ├── control.in
│   │   ├── changelog
│   │   ├── copyright
│   │   ├── pgversions
│   │   ├── rules
│   │   └── ...
│   ├── age--1.1.1.sql
│   ├── age.control
│   ├── Makefile
│   └── ...
├── src
├── LICENSE
├── README.md
└── ...
Change the file paths in the Makefile like:
src/backend/age.o should be ../src/backend/age.o.
./tools/ should be ./../tools/.
and so on.
Now you can simply run the dpkg-buildpackage -b command from the packaging directory to build the debian package.
Note: In step 1 we are running dh_make_pgxs in the project root first, this is to make sure that the project name in the control files and the version in the changelog file are correct. In this case the name/source in control, control.in & changelog files should be apache-age and the version number in changelog file should be 1.1.1-1.
Alternatively, you can run the command from the packaging directory and manually change the name and version in the control and changelog files.

Adding a custom resource directory to SBT's default unmanagedResourceDirectories

I have two SBT projects as outlined below.
├── Project 1
│ │
│ └── src
│ └── main
│ ├── scala
│ │ └── com
│ │ └── xyz
│ │ └── <*.scala>
│ └── resources
│ └── <Typesafe & Log4J config files>
│
│
└── Project 2
│
├── src
│ └── main
│ ├── scala
│ │ └── com
│ │ └── xyz
│ │ └── <*.scala>
│ └── resources
│ └── <Typesafe & Log4J config files>
│
├── resources
│ └── <JS, HTML, Image files etc.>
├── other-dir-1
│
├── other-dir-2
│
└── other-dir-3
Compiling Project 1 (actually running SBT exportedProducts task) produces the following directory structure. unmanagedResourceDirectories points to Project1/src/main/resources. I believe this is the default resourceDirectory (as mentioned in Customizing Paths). In other words, files in default resource directory are automatically added by exportedProducts
├── Project 1
└── target
└── scala-2.10
└── classes
├── com
│ └── xyz
│ └── <*.class>
└── <Typesafe & Log4J config files>
For Project 2, I want the following directory structure to be produced by exportedProducts.
├── Project 2
└── target
└── scala-2.10
└── classes
├── com
│ └── xyz
│ └── <*.class>
├── <Typesafe & Log4J config files>
│
└── resources
└── <JS, HTML, Image files etc.>
To do this I added the following to SBT build file in the appropriate project definition.
unmanagedResourceDirectories in Compile += baseDirectory.value
excludeFilter in unmanagedResources := HiddenFileFilter || "other-dir-*"
includeFilter in unmanagedResources :=
new SimpleFileFilter(_.getCanonicalPath.startsWith((baseDirectory.value / "resources").getCanonicalPath))
This correctly includes resources directory but doesn't include the files from Project2\src\main\resources. The target directory looks like the
├── Project 2
└── target
└── scala-2.10
└── classes
├── com
│ └── xyz
│ └── <*.class>
└── resources
└── <JS, HTML, Image files etc.>
Adding a custom resource directory in some way masks the content of the default resource directory. I tried something along the lines of what was mentioned in this SO post but wasn't successful.
The other thing that I tried was to set unmanagedResourceDirectories in Compile += baseDirectory.value / "resources" and remove both includeFilter and excludeFilter. This adds the files from Project2\src\main\resources correctly but adds the files & directories from Project2\resources directly to Project2\target\scala-2.10\classes. The target directory looks like the following
├── Project 2
└── target
└── scala-2.10
└── classes
├── com
│ └── xyz
│ └── <*.class>
├── <Typesafe & Log4J config files>
│
└── <JS, HTML, Image files etc.>

How to automatically create README.md markdown of directory tree listing

I want to create markdown with a directory tree listing automatically, in order to be shown in online repos like GitHub in every directory.
So, given that I'm going to use the Linux tree command that can be installed on MacOS X using brew install tree (see here for details), I came out with this solution:
tree --dirsfirst --noreport -I README.md > README.md;sed -i '' '1s/^/```/' README.md;echo \ >> README.md; sed -i '' -e '$s/$/```/' README.md
where the first sed -i '' '1s/^/```/' README.md is prepending the ``` chars – see Mastering (Github) Markdown for details about supported markdown.
The echo echo \ >> README.md; is adding a newline. Note here that I'm not using the sed equivalent sed -i '' -e '$a\' filename since this only adds a newline when it does not exists due to the $a pattern (see here).
While the last sed -i '' -e '$s/$/```/' README.md is adding a trailing ``` to the file.
The tree command tree --dirsfirst --noreport -I README.md is going to exclude some patterns, put directory first, ignore reporting file and dir count.
The result is going to be something like the following:
```.
├── bin
│   ├── Debug
│   │   ├── SampleLibrary.jar
│   │   ├── cooper.jar
│   │   ├── sugar.data.jar
│   │   ├── sugar.jar
│   │   └── swift.jar
│   └── Release
│   ├── SampleLibrary.jar
│   ├── cooper.jar
│   ├── sugar.data.jar
│   ├── sugar.jar
│   └── swift.jar
├── obj
│   ├── Debug
│   │   └── Android
│   │   ├── ClassLibrary2.elements.FilesWrittenAbsolute.txt
│   │   └── samplelibrary.jar
│   └── Release
│   └── Android
│   ├── ClassLibrary2.elements.FilesWrittenAbsolute.txt
│   └── samplelibrary.jar
├── ClassLibrary2.elements
└── ClassLibrary2.elements.user
```
You can see this markdown README.md here.
This solution is not so efficient and it is limited to -I pattern options of tree to filter out unwanted dirs (let's say build directories) or file names, etc. Also it does not work properly to update an existing README.md markdown.
The solution should work on MacOS X (where sed has some differences to that on Linux).
One way to get source code markup is to indent everything by four spaces:
tree --dirsfirst --noreport -I README.md | sed 's/^/ /' > README.md
To do it your way, adding a new first and last line with ``` on each, we can do
tree --dirsfirst --noreport -I README.md |
sed '1s/^/```'$'\n''/;$s/$/'$'\n''```/' > README.md
where inserting a newline in the replacement string is done with a C-style escape. Alternatively, we can use "$(printf '\n')":
tree --dirsfirst --noreport -I README.md |
sed '1s/^/```'"$(printf '\n')"'/;$s/$/'"$(printf '\n')"'```/' > README.md
These should both work with the sed on macOS.
With GNU sed, it would be a little simpler:
tree --dirsfirst --noreport -I README.md |
sed '1s/^/```\n/;$s/$/\n```/' > README.md
Have a look at this python module (I am the author).
It generats descriptive directory trees dynamically, and it has a markdown export format like the following:
.
├── example_folder\
│ ├── first_subfolder\ a documented folder
│ │ ├── sub-sub1\
│ │ │ └── file3.sh this is file 3
│ │ ├── sub-sub2\
│ │ │ └── file4.cpp this is file 4
│ │ └── random_file.rdm a documented file
│ ├── second_subfolder\ _a documented folder
│ ├── a_text_file.txt a text file
│ ├── my_javascript.js this is file 1
│ └── test.py a python script
└── README.md The main readme\
and an ASCII one of course
.
├── example_folder\
│ ├── first_subfolder\ (a documented folder)
│ │ ├── sub-sub1\
│ │ │ └── file3.sh (this is file 3)
│ │ ├── sub-sub2\
│ │ │ └── file4.cpp (this is file 4)
│ │ └── random_file.rdm (a documented file)
│ ├── second_subfolder\ (a documented folder with no documented files)
│ ├── a_text_file.txt (a text file)
│ ├── my_javascript.js (this is file 1)
│ └── test.py (a python script)
└── README.md (The main readme)
Create your markdown in VSCode and add Ascii Tree Generator extension from Marketplace.

ImportError: No module named myapp.settings

I created a Django app locally, and I would like to host it on PythonAnywhere.com. I've followed the directions at https://help.pythonanywhere.com/pages/VirtualEnvForNewerDjango and created a virtualenv with 1.9 installed. However, when I try to run my app, I get the error ImportError: No module named myapp.settings
Here is my username_pythonanywhere_com_wsgi.py
import os
import sys
# add your project directory to the sys.path
project_home = u'/home/rhpt'
if project_home not in sys.path:
sys.path.append(project_home)
# set environment variable to tell django where your settings.py is
os.environ['DJANGO_SETTINGS_MODULE'] = 'myapp.settings'
# serve django via WSGI
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
I also tried myapp.my_app.settings without success.
My tree
myapp
├── my_app
│ ├── __init__.py
│ ├── settings.py
│ ├── urls.py
│ └── wsgi.py
├── get_data
│ ├── __init__.py
│ ├── admin.py
│ ├── models.py
│ ├── tests.py
│ ├── urls.py
│ └── views.py
└── manage.py
if your settings.py file is in /home/rhpt/myapp/my_app/settings.py
then this part
# add your project directory to the sys.path
project_home = u'/home/rhpt'
needs to be
# add your project directory to the sys.path
project_home = u'/home/rhpt/myapp'
and also keep this
os.environ['DJANGO_SETTINGS_MODULE'] = 'my_app.settings'
Maybe if you put an empty
__init__.py
in the same folder as the settings.py it worked for me

How do I create a multi-module distribution?

I want to create my own Perl module, but the problem is that it contain multiple .pm files. The structure is:
lib
├── A_Z.pm
└── T_test
├── A.pm
├── B.pm
├── C.pm
├── D.pm
└── E.pm
I used h2xs -XA -n T_test::A T_test::B T_test::C T_test::D T_test::E. It compiled only A.pm; the other B.pm, C.pm, D.pm, E.pm are not considered. Is there any solution to execute all the .pm file at the same time?
Use Module::Starter::PBP instead.
$ module-starter --builder=Module::Build --module=A_Z,T_test::{A,B,C,D,E}
Added to MANIFEST: Build.PL
Added to MANIFEST: Changes
Added to MANIFEST: lib/A_Z.pm
Added to MANIFEST: lib/T_test/A.pm
Added to MANIFEST: lib/T_test/B.pm
Added to MANIFEST: lib/T_test/C.pm
Added to MANIFEST: lib/T_test/D.pm
Added to MANIFEST: lib/T_test/E.pm
Added to MANIFEST: MANIFEST
Added to MANIFEST: README
Added to MANIFEST: t/00.load.t
Created starter directories and files
$ tree A_Z
A_Z
├── Build.PL
├── Changes
├── lib
│ ├── A_Z.pm
│ └── T_test
│ ├── A.pm
│ ├── B.pm
│ ├── C.pm
│ ├── D.pm
│ └── E.pm
├── MANIFEST
├── README
└── t
└── 00.load.t
3 directories, 11 files
You don't have to do anything special. Just makes sure all the files are listed in MANIFEST as usual. Both ExtUtils::MakeMaker and Module::Build consider all .pm to be modules to install.