How to make sphinx look for modules in virtualenv while building html? - virtualenv

I want to build html docs using a virtualenv instead of the native environment on my machine.
I've entered the virtualenv but when I run make html I get errors saying the module can't be imported - I know the errors are due to the module being unavailable in my native environment.
How can I specify which environment should be used when searching for docs (eg the virtualenv)?

The problem is correctly spotted by Mathijs.
$ which sphinx-build
/usr/local/bin/sphinx-build
I solved this issue installing sphinx itself in the virtual environment.
With the environment activated:
$ source /home/migonzalvar/envs/myenvironment/bin/activate
$ pip install sphinx
$ which sphinx-build
/home/migonzalvar/envs/myenvironment/bin/sphinx-build
It seems neat enough.

The problem here is that make html uses the sphinx-build command as a normal shell command, which explicitly specifies which Python interpreter to use in the first line of the file (ie. #!/usr/bin/python). If Python gets invoked in this way, it will not use your virtual environment.
A quick and dirty way around this is by explicitly calling the sphinx-build Python script from an interpreter. In the Makefile, this can be achieved by changing SPHINXBUILD to the following:
SPHINXBUILD = python <absolute_path_to_sphinx-build-file>/sphinx-build
If you do not want to modify your Makefile you can also pass this parameter from the command line, as follows:
make html SPHINXBUILD='python <path_to_sphinx>/sphinx-build'
Now if you execute make build from within your VirtualEnv environment, it should use the Python interpreter from within your environment and you should see Sphinx finding all the goodies it requires.
I am well aware that this is not a neat solution, as a Makefile like this should not assume any specific location for the sphinx-build file, so any suggestions for a more suitable solution are warmly welcomed.

I had the same problem, but I couldn't use the accepted solution because I didn't use the Makefile. I was calling sphinx-build from within a custom python build file. What I really wanted to do was to call sphinx-build with the exact same environment that I was calling my python build script with. Fiddling with paths was too complicated and error prone, so I ended up with what seems to me like an elegant solution, which is to "manually" load the console script entry point and call it:
from pkg_resources import load_entry_point
cmd = load_entry_point('Sphinx', 'console_scripts', 'sphinx-build')
cmd(['sphinx-build', basepath, destpath])

Related

Distribute shell scripts using setuptools and pyproject.toml

I'm trying to distribute a shell script along with a Python package. Ideally, the shell script is installed when I run pip install my_package. I read from this SO that, my expected behavior is exactly what the scripts keyword of setuptools.setup provides. E.g. the script my_script will be installed with the following setup.py script:
setup(
...
scripts=['my_script'],
...
)
However, I cannot use the above method for two reasons:
the official doc did not mention this behavior. I don't know if I can continue to do this way.
my whole project is built on pyproject.toml, without setup.py. Although pyproject.toml has provided a [project.scripts] table, as explained in the setuptools official doc, the scripts can only be python functions instead of shell scripts.
For completeness, in my case, the shell script reads git status and sets environment variables, which will be read from within my python project. The shell script and my python project are bonded so tightly that I would rather not split them into two projects.
I have also tried to use a python function to execute the shell script, e.g.
[project.scripts]
my_script = 'my_project:my_func'
def my_func():
subprocess.run(...)
The problem with this solution is that every time I run my_script, my_project is loaded and the loading process is really slow.
Maybe a link in the comments leads to this information already. Anyway, I think it is worth posting that scripts = [...] in setup.py can be written in pyproject.toml as:
[tool.setuptools]
script-files = ["scripts/myscript1", "scripts/myscript2"]
However, this feature is deprecated. I hope the authors of the packaging tools will recognize the problem with shell scripts and deal with it.
Link: setuptools docs
I'm not exactly sure it will work for you case, but I solved this by creating a "shim" setup.py file (it has an added benefit of being able to install your project in edit mode).
It usually just calls setup(), but it was possible to pass the scripts argument:
"""Shim setup file to allow for editable install."""
from setuptools import setup
if __name__ == "__main__":
setup(scripts=["myscript"])
Everything else was loaded from pyproject.toml.

How can you run a config script when installing a perl module?

I've been searching for a couple of hours and I'm coming up empty trying to find a solution. I'm using Dist::Zilla. I have a module that uses a simple config file in .ini format located in the module's share/ directory. When my module is installed, I'd like the install script to prompt the user for configuration options and save the user's options in the config file. Then, using File::UserConfig, it will copy the file over to the user's configuration directory where it can be loaded by the module when it runs.
Someone had suggested the Dist::Zilla::Plugin::MakeMaker::Custom module but I know next to nothing about MakeMaker and how I might write a custom one to kick off the configuration script.
I'm surprised I can't find anything that makes this easy to do. Perhaps I'm searching on the wrong keywords?
You had discussed this in IRC, and the gist is:
You cannot rely on the installation process allowing any interaction, as a large amount of installations are via cpanm which is non-interactive and hides output from Makefile.PL by default. This is because users don't like having to configure things, and as an example, a Carton deployment is frequently non-interactive by its nature. You can allow configuration via environment variables recognized by your Makefile.PL to get around this.
You can document to install using the --interactive option for cpanm in order to respond to prompts in your Makefile.PL, injected into the generated file using the [MakeMaker::Awesome] plugin.
You can include a script with the distribution that will set up the configuration so the user can do it themselves separate from the installation.

How to handle external dependencies in perl's ExtUtils:MakeMaker

I have a series of perl scripts for which I'm writing a Makefile.PL script, but I'm rather inexperienced with ExtUtils::MakeMaker.
One of the scripts I wrote makes a system call to a command line utility that must be installed in order for the script to run properly. My script can gracefully detect that the utility is missing and issue an error about installing it and putting it in the user's path, but is there some standard way to handle this in the Makefile.PL script? Could it even gasp attempt to install the third-party utility if I enter the download link in the Makefile.PL script?
At the very least, I'd like the script to warn the user if the external dependency was not found. I know I can write a test case that uses it. Is this as simple as copying and pasting the subroutine I wrote in the script itself that checks for the third party utility and prints an error if it's not found or would that be the "wrong way to do it"?
Let's call this external dependency foobar, for sake of argument.
As per #KeepCalmAndCarryOn's comment, firstly consider whether foobar could be replaced by something from CPAN (maybe Foo::Bar), or a few lines of Perl.
Otherwise, the best course of action is:
Create a new CPAN distribution called Alien::Foobar. The job of Alien::Foobar is to download, perhaps compile, and then install foobar, as part of Alien::Foobar's Makefile.PL or Build.PL.
(There exists a module called Alien::Base which aims to make doing this sort of thing easier. It's mostly aimed at installing libraries rather than binaries, though I've had some success using it for the latter.)
Now the Makefile.PL you were originally working on can declare a dependency on Alien::Foobar.
If you have an external dependency on a command-line utility (i.e. there's no perl module that does what the utility does), ExtUtils::MakeMaker is not designed to handle such a dependency. What you need to do is write an install script or edit the make file to handle the dependency. Here are the considerations in doing so:
Check if the dependency exists and if the version is sufficient.
Download the dependent package
Configure, compile, & install the dependent package
Test to make sure it works
Update the user's environment setup if necessary
Run your perl package's installation steps (e.g. perl makefile.PL;make;sudo make install)
Note, you may need to know whether your script is running as root or not, which you can verify using id -u to check if the user ID is root (i.e. '0').

Can't load 'C:/strawberry/perl/site/lib/auto/XML/LibXML/LibXML.dll' for module XML::LibXML

I have downloaded strawberry PERL and writing one application with CGI Perl Apache on Winxp sp3).
One of the libraries (written by someone else) which I using uses XML::LibXML. When i load the page it gives Internal Server Error. From Apache error log i can see this error:
Can't load 'C:/strawberry/perl/site/lib/auto/XML/LibXML/LibXML.dll' for module XML::LibXML: load_file:The specified module could not be found at C:/strawberry/perl/lib/DynaLoader.pm line 190.
C:/strawberry/perl/site/lib/auto/XML/LibXML/LibXML.dll exists with all permissions.
Also this library works properly on Linux. My application also works fine if I remove all code that needs LibXML.
Can anyone tell me when can be possible issue here.
If you peek into the source for DynaLoader you'll find
Many dynamic extension loading problems will appear to come from
this section of code: XYZ failed at line 123 of DynaLoader.pm.
Often these errors are actually occurring in the initialisation
C code of the extension XS file. Perl reports the error as being
in this perl code simply because this was the last perl code
it executed.
You should have also gotten (but may not have noticed) the following dialog, which provides a more accurate error message:
The problem isn't that perl can't find LibXML.dll; it's that LibXML.dll can't find the real libxml. (The former is just a wrapper that provides Perl bindings for the latter.) To fix that you need to ensure that Strawberry Perl's c\bin folder is in your PATH. In your case, that would be C:\strawberry\c\bin.
You might have to check the environment variable settings in the windows,
make sure that the installation path of the module is present in the PATH variable.
The reason it works in linux is that make files usually set the environment variables for you in linux in windows it may not have set properly.
For eg;
go to Control Panel\System and Security\System click change settings then advanced tab in user variable section see if there is a variable called perl5lib.
if not create an new perl5lib variable and add the path of your library ( usuall C:\Perl\site\lib but may be different in your case)
I had the same issue after installing Strawberry perl. It was working fine when I run the script from server, but not remotely from a automation tool. The issue was because of the Environment variables not updated when we run it remotely. So I did server reboot, which resolved the issue.
I encountered the same problem recently, in my case it was not related to PATH variable (it was already correct). The thing is I was executing my script from Git Bash console and as it turned out git-bash perl was being used instead of Strawberry (see git-bash perl should not be first in path). Switching to standard Windows CMD terminal helped.

What is a quick way to run a single script to automatically install missing modules using only Perl core?

I inherited a project which is supposed to be able to be deployed to other servers. This project has a number of simple module dependencies which however might not be present on all target machines.
As such I'd like to be able to run a single command line script that checks which Perl modules are installed and tries to automatically install missing ones via CPAN.
Since this should be very basic (i.e. needing to install stuff to run the module installer would defeat the point) said script should only use Perl 5.8.8 core modules.
Does something like that exist already or would i need to write it myself?
Creating a Bundle package is one possible answer.
You can then look at something like CPAN::Shell (see CPAN module) to automate the process.
/I3az/
Update re: brian's comment about Task:: - Here are some pertinent links:
Writing a CPAN Task (using Module::Install)
"Task:: or Bundle::"? (Perlmonks)
Use Module::Install, it will be bundled with your module/program. You can use "auto_install" command to automatically install dependencies.