I have a project structure in VSCode like this:
Project/
.venv/
virtual environment containing pip packages like numpy
config/
__init__.py
useful scripts
src/
program.py
I want program.py to be able to import packages from the virtual environment as well as the config package I made.
The issue is that it's not detecting the "config" module. I'm afraid that if I do something like append the system path, it will just make the virtual environment modules undetected instead. Do I have to change my project structure or is there an easier way?
Related
I recently started experimenting with Poetry for package and dependency management, and I am still getting used to the differences between it and my experience with setuptools. Specifically, I would appreciate help in understanding how to handle the following scenario.
I have a data file that I want to bundle with my package stored in a package subdirectory. Using setup.py I would specify the file and directory names in the setup.py file and then access the file in my code using the pkg_resources API.
What is the equivalent approach using Poetry and pyproject.toml?
Unlike setuptools poetry bundles automatically all files within your package folder into your package, unless you haven't explicit excluded them in your .gitignore or the pyproject.toml.
So after the package is installed, you can access them with pkg_resources.
You could still use setuptools's pkg_resources.
You could also use one of those from Python's own standard library:
https://docs.python.org/3/library/pkgutil.html#pkgutil.get_data
https://docs.python.org/3/library/importlib.html#importlib.resources.read_binary
I have code in /home/user/Documents/code/workspace/ (Debian Jessie) in subfolders like java/ for java and python/ for python I've written. For golang in Eclipse, should I use the workspace ~/Documents/code/workspace or ../workspace/go or ../workspace/go/src?
Use workspace/go as your workspace. From code organization guidelines on golang.org
Go code must be kept inside a workspace. A workspace is a directory
hierarchy with three directories at its root:
src contains Go source files organized into packages (one package per directory),
pkg contains package objects, and
bin contains executable commands.
The go tool builds source packages and installs the resulting binaries
to the pkg and bin directories.
The src subdirectory typically contains multiple version control
repositories (such as for Git or Mercurial) that track the development
of one or more source packages.
I'm having trouble with relative imports, but I think its because i'm not understanding package structure completely.
For example, here is my package structure.
neo_autorig/ Source folder, Top level
__init__.py
basic/ Subpackage for basic utiltites for the script
__init__.py
name.py
name_test.py
module_locator.py
Theres more than this, but this is basically what i'm using for imports
In name.py i'm importing module locator using
from .. import module_locator
But it says
# Error: line 1: Attempted relative import beyond toplevel package
Are top level scripts (like my main script/ui used to execute everything) supposed to go in the top source folder in the eclipse package? Or am i setting this up wrong. There are also other sub packages in the source folder, each with scripts in them.
Edit: If i putanother package in a sub package, I can relative import, its only the case where i cant relative import from a sub package to a top level package, and the scripts source is in my python path.
The python import mechanism works with the __name__ of the file. Executing a file directly gives the file the name "__main__" instead of its usual name. The common answer to questions like this would be to run the program with the -m option. I recommend reading pep 366 and maybe this or this question as well.
Since I first saw a dist/ directory in many open source projects, usually on GitHub, I've been wondering what it means.
With dist, vendor, lib, src, and many other folder names that we see quite often, I sometimes wonder how I should name my own folders.
Correct me if I'm wrong!
src: Contains the sources. Sometimes only the pure sources, sometimes with the minified version, depends on the project.
vendor: Contains other dependencies, like other open source projects.
lib: Good question, it's really close to vendor actually, depending on the project we can see one or another or both...
dist: From what I saw, it contains the "production" files, the one we should use if we want to use the library.
Why is open source so confusing? Isn't it possible to do things clearer? At least per language because some languages use specific names.
To answer your question:
/dist means "distributable", the compiled code/library.
Folder structure varies by build system and programming language. Here are some standard conventions:
src/: "source" files to build and develop the project. This is where the original source files are located, before being compiled into fewer files to dist/, public/ or build/.
dist/: "distribution", the compiled code/library, also named public/ or build/. The files meant for production or public use are usually located here.
There may be a slight difference between these three:
build/: is a compiled version of your src/ but not a production-ready.
dist/: is a production-ready compiled version of your code.
public/: usually used as the files runs on the browser. which it may be the server-side JS and also include some HTML and CSS.
assets/: static content like images, video, audio, fonts etc.
lib/: external dependencies (when included directly).
test/: the project's tests scripts, mocks, etc.
node_modules/: includes libraries and dependencies for JS packages, used by Npm.
vendor/: includes libraries and dependencies for PHP packages, used by Composer.
bin/: files that get added to your PATH when installed.
Markdown/Text Files:
README.md: A help file which addresses setup, tutorials, and documents the project. README.txt is also used.
LICENSE.md: any rights given to you regarding the project. LICENSE or LICENSE.txt are variations of the license file name, having the same contents.
CONTRIBUTING.md: how to help out with the project. Sometimes this is addressed in the README.md file.
Specific (these could go on forever):
package.json: defines libraries and dependencies for JS packages, used by Npm.
package-lock.json: specific version lock for dependencies installed from package.json, used by Npm.
composer.json: defines libraries and dependencies for PHP packages, used by Composer.
composer.lock: specific version lock for dependencies installed from composer.json, used by Composer.
gulpfile.js: used to define functions and tasks to be run with Gulp.
.travis.yml: config file for the Travis CI environment.
.gitignore: Specification of the files meant to be ignored by Git.
To answer your original question about the meaning of the dist folder:
The shortform dist stands for distributable and refers to a directory where files will be stored that can be directly used by others without the need to compile or minify the source code that is being reused.
Example: If I want to use the source code of a Java library that someone wrote, then you need to compile the sources first to make use of it. But if a library author puts the already compiled version into the repository, then you can just go ahead. Such an already compiled version is saved into the dist directory.
Something similar applies to JavaScript modules. Usually JavaScript code is minified and obfuscated for use in production. Therefore, if you want to distribute a JavaScript library, it's advisable to put the plain (not minified) source code into an src (source) directory and the minified and obfuscated version into the dist (distributable) directoy, so others can grab the minified version right away without having to minify it themselves.
Note: Some developers use names like target, build or dest (destination) instead of dist. But the purpose of these folders is identical.
Summary of the folders:
bin: binaries
contrib: contributions to the project
dist: -- see 1. and 2.
doc/s: documentation
include: headers (C/C++)
lib: libraries (C/C++)
man: short for man/manual pages (Unix/Linux), c.f. man(1)
src: source
"/dist means "distributable", the compiled code/library." ref.
"The shortform dist stands for distributable and refers to a directory where files will be stored that can be directly used by others without the need to compile or minify the source code that is being reused." ref.
Actually! "dist folder" is the result you get after modifying a source code with "npm run build" or "ng build" or "ng build --prod" for production.
Meanwhile! After getting "dist folder" there might still be few things that you still need to do depending on your project type ✌️
There are many suggestions on the web what the structure of the Python project can/shall be, e.g. What is the best project structure for a Python application?.
"proj-dir"
+- doc
+- apidoc
+- scripts
+- "src-dir"
+- tests
It seems that many people in the Python world prefer the "src-dir" and "proj-dir" to be equal (or very similar). However, the scripts and tests will surely have to import some modules from src-dir, so either I have to
run all scripts and tests when proj-dir as my current working dir, or
the PYTHONPATH must contain the proj-dir, right?
Or, is there any other possibility which I overlook now?
Suppose I have such structure and have set up the PYTHONPATH correctly. Now I commit the project into VCS and another developer checks the project out. On his machine the PYTHONPATH will not be set properly and the imports in scripts and in tests will not work.
Is there any possibility to somehow make the PYTHONPATH definition part of the project, so that it can be part of the version control?
Is there a different project structure that would allow me to make checkout and start using the project without modifying the PYTHONPATH?
Is to solution suggested here: Python - How to PYTHONPATH with a complex directory structure? (i.e. to include a short sys.path modifying snippet in every script and test) the right one?
Would a site module http://docs.python.org/2/library/site.html be of any help?
What is your workflow when developing a project with a similar structure?
Even through project structure is complex and testcases uses src in different package, you can add PYTHONPATH in Environmental Variables in Windows
My Computer --> Properties --> System Properties --> Advanced Tab --> Environmental Variables. Add new System Variable PYTHONPATH and append your projects dir to that variable with delimiter ';'
I suggest you to keep readme.txt to your project which clearly says to how to add PYTHONPATH in order to run the scripts. It is as similar as adding PATH of android-sdk to make adb shell to work, adding Java path to make Java to work in command prompt
Thus readme.txt helps to run the script when downloaded in different machine with only one change(adding PYTHONPATH variable with value = projectpath)
No need to add all modules of src in order to make testcases to work which is using src
Here is an example proj structure:
SAMPLEPROJECT
src
com
sample
example
utils
a.py
b.py
testcases
test1.py
test2.py
res
docs
If test1.py test case is using a.py and and b.py:
Dont add utils module to PYTHONPATH, as you already added project path to python path your import statement looks something like this in test1.py
from src.com.sample.example.utils import a.py
from src.com.sample.example.utils import b.py