Inorder to deploy my flask postgresql based app, i want to initialize my database with few items. Hence i created a manage.py for this. This is because i want to enter item into database only once. After that, i want to go as usual(git push my project).
My project works fine, problem is i don't know where should i place my manage.py and how do i use it as an admin or make sure it just run once.
I use manage.py as
python manage.py create_tables.
this is my wsgi folder:
├── application
├── app.py
├── auth.py
├── flaskapp.cfg
├── forms.py
├── main.py
├── models.py
├── result.json
├── templates
│ ├── base.html
│ ├── evaluate.html
│ └── show_all.html
└── views.py
and this is my project folder :
manage.py readme.md requirements.txt setup.py venv wsgi wsgi.py
Obviously, i want to initialize database in such a way that everytime i change code and push, i don't want to re-initialize the database with same entries but only when i want to and also, as an admin.
This is my application file looks like :
#!/usr/bin/python
import os
virtenv = os.environ['OPENSHIFT_PYTHON_DIR'] + '/virtenv/'
os.environ['PYTHON_EGG_CACHE'] = os.path.join(virtenv, 'lib/python2.7/site-packages')
virtualenv = os.path.join(virtenv, 'bin/activate_this.py')
try:
execfile(virtualenv, dict(__file__=virtualenv))
except IOError:
pass
#
# IMPORTANT: Put any additional includes below this line. If placed above this
# line, it's possible required libraries won't be in your searchable path
#
from main import app as application
from main import *
db.create_all()
The application file in openshift isn't documented well and its not clear as to what's the right place to do one time database initialization.
Place this code under .openshift/action_hooks in deploy file:
source $OPENSHIFT_HOMEDIR/python/virtenv/bin/activate
python "$OPENSHIFT_REPO_DIR"wsgi/manage.py
This script will run manage.py every time that you deploy your app.
Related
I have the next tree:
root_project/
├── app
│ ├── default_photo_profile.jpg
│ ├── config.py
│ ├── __main__.py # My app are python package, I'm runnig it via "python -m"
│ └── ...
├── tests
│ ├── test_unit.py # import config.py inside
│ ├── functional # import config.py inside
│ ├── pytest.ini
│ └── ...
...
Currently default_photo_profile causing error because tests doesn't have this file.
Reading file in config.py:
DEFAULT_PHOTO_FILE_PATH = Path('default_photo.jpg')
with open(file=DEFAULT_PHOTO_FILE_PATH, mode='rb') as file_obj:
DEFAULT_PHOTO_BYTES = file_obj.read()
How I can solve this?
I tried:
Patch access to default_photo.jpg with fixture - not helped, error during import stage, not executiion.
set flag to pytest comamnd line: --rootdir app - not helped (don't know why).
try/except for reading the file in app.config.py - may help but it's not my intention, I really want raise error if file not found
Put default_photo.jpg inside EVERY test directory - will help bit dirty.
Patch os.path like suggested in https://stackoverflow.com/a/43003192/11277611 - dirty
Include tests into package (move __main__.py into root_project - not sure that it's a good idea (have not enough experience to decide).
Set absolut path to default_photo.jpg - will fail on the production server.
Probably adoptable solutions (What I want):
Set root dir to root_project.app somehow inside pytest.ini to immitate regular execution.
Set root dir to root_project.tests somehow to place file in root of tests and access from any of tests folder.
Try to use following code in config.py:
DEFAULT_PHOTO_FILE_PATH = Path(__file__).parent / 'default_photo.jpg'
with open(file=DEFAULT_PHOTO_FILE_PATH, mode='rb') as file_obj:
DEFAULT_PHOTO_BYTES = file_obj.read()
Is it what you are trying to achieve?
This is a much asked question, but none of the solutions mentioned on SO have worked so far.
The folder structure is as follows:
project/
└── tests/
├── conftest.py
├── __init__.py
└── int_tests/
└── test_device.py
└── project_core/
└── tests/
├── conftest.py
├── __init__.py
└── int_tests/
└── test_device.py
import file mismatch:
imported module 'test_device' has this __file__ attribute:
/home/.../project/project_core/tests/int_tests/test_device.py
which is not the same as the test file we want to collect:
/home/.../project/tests/int_tests/test_device.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
Steps tried so far:
Removing pycache and pyc files.
Adding _init to each folder. (As is stated in pytest GIP)
Removing _init from each folder.
Do i need init files in each tests/subfolder?
The same error occurs with conftest.py as well. This error is not limited to vscode-pytest plugin, also occurs on the terminal.
PS : For CI purposes, the system is configured with docker & tox. Development is done in venv.
I have trouble creating a package with setuptools. I have a repository which I'm cleaning up to make it a package. The directory structure looks something like this
my-proj
├── setup.py
├── MANIFEST.in
├── MakeFile
├── README.rst
├── setup.py
└── myproj
├── __init__.py
├── my_code.py
├── templates
│ ├── template1.yaml
│ ├── template2.yaml
Initial version of "my_code.py" had code snippet which would directly reference the files withing templates folder to do some processing. If I package this using setup tools, I provide the following information in these files:
MANIFEST.in:
include README.rst
include requirements.txt
include LICENSE.txt
recursive-include myproj/templates *
setup.py:
setup(
name='myproj',
package_dir={'testbed_init': 'testbed_init'},
package_data={'templates': ['templates/*'], 'configs': ['configs/*']},
include_package_data=True,
)
My question is as follows. In "my_Code.py" I used to reference templates directly without any problem as I would run script from the myproj folder. If I package this, how can I make sure, I include the templates as part of package and when script runs, I need to open the templates relative to where the package is installed.
Code snippet from my_code.py:
if _type == "a":
temp_file = f"templates/template1.yaml"
else:
temp_file = f"templates/template2.yaml"
build_config(deploy_esx_file, output_file, data)
Code snippet of what happens in build_config:
def build_config(template_file, output_file, inputs):
templateLoader = jinja2.FileSystemLoader(searchpath="./")
templateEnv = jinja2.Environment(loader=templateLoader)
template = templateEnv.get_template(template_file)
outputText = template.render(inputs)
with open(output_file, 'w') as h:
h.write(outputText)
Suppose I have the following directory structure:
src/
└── python/
└── generated/
├── __init__.py
├── a.py
└── lib/
├── __init__.py
└── b.py
What does my setup.py need to look like in order to create a dist with a directory layout like:
src/
└── python/
├── __init__.py
├── a.py
└── lib/
├── __init__.py
└── b.py
The goal is to simply eliminate the generated folder. I've tried endless variations with package_dir and can't get anything produced other than the original directory structure.
Your setup.py should be placed in your src directory and should look like this:
#!/usr/bin/env python3
import setuptools
setuptools.setup(
name='Thing',
version='1.2.3',
packages=[
'python',
'python.lib',
],
package_dir={
'python': 'python/generated',
},
)
Note the package_dir setting. It instructs setuptools to get the code for the python package from the directory python/generated. In the built distributions you will then find the right directory structure.
First, here is my solution:
#!/usr/bin/env python
import os, shutil
from setuptools import setup
from setuptools.command.build_py import build_py
class BuildPyCommand(build_py):
"""Custom build command."""
def run(self):
shutil.rmtree('src.tmp', ignore_errors=True)
os.mkdir('src.tmp')
shutil.copytree('src/python/generated', 'src.tmp/python')
build_py.run(self)
setup(cmdclass={ 'build_py': BuildPyCommand },
name='Blabla',
version='1.0',
description='best desc ever',
author='Me',
packages=['python', 'python.lib'],
package_dir={'': 'src.tmp'},
setup_requires=['wheel']
)
And you can generate your distribution with:
python setup.py build bdist_wheel
The idea is perform a two steps build:
I generate a valid source structure
I build this temporary structure
And I deliver it in a wheel because it doesn't require future users to understand my trick. If you give it a try with a source distribution, you will notice that you need to publish the generated files as data (not difficult, but troublesome, and, I guess you will want to hide your tricks from your users).
But, I think that there is a design flaw in your process. The file src/python/generated/__init__.py, assumed to be a module <something>.generated eventually becomes your <something>.python, which is troublesome. It would be much simpler and more robust to generate a valid Python structure: src/generated/python/__init__.py. The setup.py would become trivial and your generator wouldn't be more complex.
According to docs https://packaging.python.org/en/latest/distributing/#data-files
setuptools will honor data_files configed in setup.py. But i can't make it work. This is my setup.py:
setup(
name='booking_order',
version=version,
packages=find_packages(),
package_data={
'booking_order': ['fake_backends/static/*',
'scripts/*',
'*.sample'],
},
data_files=[
('/etc/booking', ['etc/booking.conf'])
],
This is the project's file tree:
.
├── booking_order
│ ├── __init__.py
│ ├── tests
│ │ ├── __init__.py
├── etc
│ ├── booking.conf
├── README.md
├── setup.py
The behavior is, if i run python setup.py install, file etc/booking.conf will got installed to /etc/booking. But if i first python setup.py sdist upload, then pip install booking_order, there will be an error "error: can't copy 'etc/booking.conf': doesn't exist or not a regular file".
I checked python setup.py sdist doesn't include files in etc at all.
EDIT:
it seems this is the answer: https://github.com/pypa/setuptools/issues/521
Answer it myself.
According to pypa, and non-package-data-files。"Setuptools doesn't support installing data files to some arbitrary location on a user’s machine; this is a feature, not a bug."
If one need to install files to locations like /etc, /usr/share, eg, then he/she may use data_files flag from distutils, which feature is not totally cleaned up from setuptools. "Not totally cleaned up" means you need to add those files to MANIFEST.in manually, which is different as in distutils.
Of course, it will be better if one can manage these configuration files with rpm or deb package system. For me it's just a temporary solution to use pip here.