How to run protoc correctly with different file import? - proto

My main file to generate has these import :
import "protos/google_annotations.proto";
import "protos/nakama_annotations.proto";
import "protos/nakama_api.proto";
The folder structure :
├── lib
├── protos
├── google_annotations.proto
├── nakama_annotations.proto
├── nakama_api.proto
├── apigrpc.proto <--- this is the file to generate.
The highlight syntax is ok.(Android studio)
The 2 cases that i got error are :
1.
Command run in protos directory
Run protoc apigrpc.proto --java_out=. --proto_path=.
Get this error
protos/google_annotations.proto: File not found.
protos/nakama_annotations.proto: File not found.
protos/nakama_api.proto: File not found.
Specify all import files
Command run in protos directory
Run
protoc apigrpc.proto --java_out=. --proto_path=google_annotations.proto --proto_path=nakama_annotations.proto --proto_path=nakama_api.proto
Get this error apigrpc.proto: File does not reside within any path specified using --proto_path
What did i do wrong?

I just found what's wrong. It's about import.
I have to remove the prefex protos because the import file is in the same level of directory.
So the import become this :
import "google_annotations.proto";
import "nakama_annotations.proto";
import "nakama_api.proto";
The reason that I put protos in front before because the Android Studio plugin doesn't show red highlight when I put like that. Now After remove that, it highlight red, but it works.

Related

import from parent directory python

I am trying to run a pytest test for filea.py using the following directory structure
test_filea.py
from filea import *
def test_one_p_one():
r = one_p_one()
assert r == 2
filea.py
def one_p_one():
return 1 + 1
When i have to following directory structure every thing works fine.
├── filea.py
├── test_filea.py
but when i move my tests into a sub directory like this
├── filea.py
└── tests
└── test_filea.py
i get the error:
test_filea.py:1: in <module>
from filea import *
E ModuleNotFoundError: No module named 'filea'
My editor seems to indicate the import in the file in the sub directory is ok.. (no read squiggly lines)
but when i run this using "pytest"
i get the error indicated above.
As per pytest documentation about test discovery, try like this:
add an empty __init__.py file in testsdirectory;
make sure that, when you run pytest ., the parent directory of filea.py and tests is the current working directory.
It depends where you run the tests from, and how you invoke pytest. Calling pytest tests is different than calling python -m pytest tests, the later adds the current working directory into the sys.path, which makes filea module importable.

learning python packaging, the old ModuleNotFoundErrro

What am I doing wrong here???
My structure :-
├── tst
│   ├── setup.py
│   └── tst
│   ├── __init__.py
│   ├── mre.py
│   └── start.py
contents of start.py
from mre import mre
def proc1():
mre.more()
return ('ran proc1')
if __name__ == "__main__":
print('test')
print(proc1())
contents of mre.py
class mre(object):
def more():
print('this is some more')
contents of setup.py
from setuptools import setup
setup(name='tst',
version='0.1',
description='just a test',
author='Mr Test',
author_email='test#example.com',
entry_points={'console_scripts': ['tst=tst.start:proc1']},
license='MIT',
packages=['tst'],
zip_safe=False)
nothing in __init__.py
When I run this from the command line all is fine, runs as expected.
However when I package this up using PIP and run using tst I get:-
Traceback (most recent call last):
File "/home/simon/.local/bin/tst", line 5, in <module>
from tst.start import proc1
File "/home/simon/.local/lib/python3.8/site-packages/tst/start.py", line 1, in <module>
from mre import mre
ModuleNotFoundError: No module named 'mre'
I've read numerous posts and I just can't seem to figure this out, if I go into the installed code and change the line
from mre import mre
to
from tst.mre import mre
then it works, but then that doesn't work when running it from the dir for development purposes... I'm obviously missing something obvious :) is it a path issue or am I missing a command in the setup.py?
If someone could point me in the right direction?
edit: do I need to do something different while developing a module thats going to be packaged, perhaps call the code some different way?
cheers
From my point of view, the absolute import from tst.mre import mre is the right thing. You could eventually use from .mre import mre, but the absolute import is safer.
For development purposes:
Use pip's editable mode:
path/to/pythonX.Y -m pip install --editable .
Similar to setuptools develop mode which is slowly going towards deprecation path/to/pythonX.Y setup.py develop.
And run the console script, or the executable module:
tst
path/to/pythonX.Y -m tst.start
Without installation, it is often sill possible to run the executable module:
path/to/pythonX.Y -m tst.start.

Customizing python package directory layout with setup.py

Suppose I have the following directory structure:
src/
└── python/
└── generated/
├── __init__.py
├── a.py
└── lib/
├── __init__.py
└── b.py
What does my setup.py need to look like in order to create a dist with a directory layout like:
src/
└── python/
├── __init__.py
├── a.py
└── lib/
├── __init__.py
└── b.py
The goal is to simply eliminate the generated folder. I've tried endless variations with package_dir and can't get anything produced other than the original directory structure.
Your setup.py should be placed in your src directory and should look like this:
#!/usr/bin/env python3
import setuptools
setuptools.setup(
name='Thing',
version='1.2.3',
packages=[
'python',
'python.lib',
],
package_dir={
'python': 'python/generated',
},
)
Note the package_dir setting. It instructs setuptools to get the code for the python package from the directory python/generated. In the built distributions you will then find the right directory structure.
First, here is my solution:
#!/usr/bin/env python
import os, shutil
from setuptools import setup
from setuptools.command.build_py import build_py
class BuildPyCommand(build_py):
"""Custom build command."""
def run(self):
shutil.rmtree('src.tmp', ignore_errors=True)
os.mkdir('src.tmp')
shutil.copytree('src/python/generated', 'src.tmp/python')
build_py.run(self)
setup(cmdclass={ 'build_py': BuildPyCommand },
name='Blabla',
version='1.0',
description='best desc ever',
author='Me',
packages=['python', 'python.lib'],
package_dir={'': 'src.tmp'},
setup_requires=['wheel']
)
And you can generate your distribution with:
python setup.py build bdist_wheel
The idea is perform a two steps build:
I generate a valid source structure
I build this temporary structure
And I deliver it in a wheel because it doesn't require future users to understand my trick. If you give it a try with a source distribution, you will notice that you need to publish the generated files as data (not difficult, but troublesome, and, I guess you will want to hide your tricks from your users).
But, I think that there is a design flaw in your process. The file src/python/generated/__init__.py, assumed to be a module <something>.generated eventually becomes your <something>.python, which is troublesome. It would be much simpler and more robust to generate a valid Python structure: src/generated/python/__init__.py. The setup.py would become trivial and your generator wouldn't be more complex.

Kotlin cannot import packages

I'm using kotlin on command line and I'm getting import errors error: unresolved reference: ConnectionHandler
Following is my directory tree:
$ tree .
.
├── LICENSE
├── main.jar
├── main.kt
└── server
├── ConnectionHandler.class
├── ConnectionHandler.kt
├── HttpRequest.class
└── HttpRequest.kt
1 directory, 7 files
When I run kotlinc main.kt -include-runtime -d main.jar I get
main.kt:2:8: error: unresolved reference: server
import server.*
I have declared package server in both of server/ConnectionHandler.kt and server/HttpRequest.kt
Note: The META-INF folder is missing. It is not regenerated either on subsequent compilations.
What am I doing wrong? If it has anything to do with META-INF folder, how can I regenerate it?
You need to include all your source files when using kotlinc (or at least include the compiled classes on the classpath). e.g.
kotlinc main.kt server/ConnectionHandler.kt server/HttpRequest.kt -include-runtime -d main.jar

How to import python modules from parent and sibling packages

This (or similar) question has been asked many times before, but none of the solutions offered work in my case.
My project structure is like this :
| project_2
main.py
__init__.py
systems.py
| config
__init__.py
options.py
| database
__init__.py
database.py
entity.py
| tests
__init__.py
test_systems.py
test_options.py
test_database.py
test_entity.py
Obviously I need to import all the modules in the test modules under the tests package. I tried relative imports with the dot syntax:
from ..systems import System
from ..config import options
from ..database.entity import Entity
Returns a ValueError: Attempt relative import in non-package. I have tried that with a package structure where everything (including systems) is in its own package. It fails with the same message.
What really bothers me is that this is supposed to work: PEP 328, but it does not. I really want to avoid having to append the packages to $PYTHONPATH or to use some insane method such as loading the modules with imp from the file path.
I read that part of the problem might be that systems.py is in the main package, but that does not explain why the rest of the relative imports do not work either.
P.S. I actually recreated the example from PEP 328 just to test it and it does not work.
You get that when a python file does a relative import, but that file not loaded as a module via import in another module (but e.g. from the commandline). Given this structure:
.
├── main.py
└── test
├── __init__.py
├── a.py
└── b.py
main.py:
from test.a import A
print A
a.py:
from .b import B
A = B
if __name__ == '__main__':
print A
b.py:
B = 'b'
Now try:
python main.py
result is
b
and with
python test/a.py
you get:
Traceback (most recent call last):
File "test/a.py", line 1, in <module>
from .b import B
ValueError: Attempted relative import in non-package
What does work is:
python -m test.a
If you simply add . to your python path, if you run the script from the project_2 folder relative paths such as config.options will work. This requires an update to PYTHONPATH on every machine, unfortunately.
Tested on Python 2.7.14