track history of installed python packages - virtualenv

The production code runs in a virtualenv.
The packages and the version numbers should be dumped regularly, to see if there was a version change.
Use case: the application does not behave like it should, and you need to find the source of the problem. It should be easy to see if there were changes (for example new packages).
I know pip --freeze and I know how to write a cronjob .... but somehow I think there could be a better solution.

Related

Chocolatey: Simple Automatic Upgrades of Local Folder Packages

I have a folder like c:\chocopkg where I put a couple of packages which I can't find on the official repo.
Creating nupkg archives was really simple and fun. Instead, the Automatic Updater (AU) is too much for me: there is no simple cinst au; one should clone a git repo and also setup a new one even for a local run.
What I need is very simple. I added a script <package>\tools\chocolateyBeforeUpgrade.ps1, with trivial Invoke-WebRequest regexps. It checks for new versions on the vendor's site and can update chocolateyInstall.ps1.
My first question is: Is there some config option to have cup all running a script like this before checking for package status?
If this is not possible, it would be also simple to wrap cup in a, say, cup2 checking and running automatic upgrades, but what file should this wrapper edit before giving control to actual cup?
cup page just says it "upgrades a package or a list of packages", but I don't understand how. I can speculate it looks at the .nuspec version. However in a local share there is no such info without unzipping the .nupkg file and for remote packages this would require a possibly large download.
AU essentially does what you'll need, even if it is a little more setup and work. I know it may feel like too much to start, but you can just start with the files that run the updates.
What I need is very simple. I added a script <package>\tools\chocolateyBeforeUpgrade.ps1, with trivial Invoke-WebRequest regexps. It checks for new versions on the vendor's site and can update chocolateyInstall.ps1.
This isn't going to help with upgrades as it is a chicken and egg scenario. You need the updated package first to be able to upgrade a package. So putting something into beforemodify or the install script is only going to help you on installation. BeforeModify only runs from the already installed package on upgrade/uninstall, so unless there

DrRacket can't check for updates

system : MAC OS 10.11.6
The error message :
Error: could not connect to website
(url->port: bad reply from server: #<eof>)
I don't know if I can simply drag the "Racket v6.2.1" folder in Application to Trash and then install the newest version of DrRacket.
This was, unfortunately, a bug in Racket that was present in v6.2.1. It has since been fixed (by me, in this commit), but given that you have a version with the bug, you’ll have to update it manually.
Each installation of Racket basically exists within its own sandbox, though, so you should be able to install the new version alongside v6.2.1 without any problems, and you can check if it works. Things like settings will be preserved across versions, but installed packages will not be.
I think DrRacket may prompt you to migrate your packages when you launch a new version, but I honestly forget. However, you can manually migrate your old packages from another version using the raco pkg migrate command. Otherwise, it’s not much of a loss—you can always just reinstall them.
Once you’ve migrated your packages (or decided you don’t want or need to), you can safely delete your old installation’s directory.

Perl module installation structure and version control

I am just starting to organize some stuff on the cluster and would like some advice on it. I posted a recent question How to organize Perl modules and got some good answers about what I was doing incorrectly. I was trying to install each perl module independently by setting the PREFIX for Makefile.PL each time to /path/to/lib/module-name/module-version/installation happens here.
For e.g. for a module JSON, I installed it like this:
perl Makefile.PL --PREFIX=/path/to/lib/perl5/5.22.1/JSON/2.53
make
make test
make install
For module Data-UUID, I did it like this:
perl Makefile.PL --PREFIX=/path/to/lib/perl5/5.22.1/Data-UUID/1.221
make
make test
make install
So it made a directory JSON/2.53 in /path/to/lib/perl5/5.22.1 and that's where it installed the package. But because I change the PREFIX for each individual module, I have to set the PATH in the bash_profile accordingly, which is kind of messy.
My main goal to do this was for version control. In a hypothetical scenario where different versions work for two of your teammates, say JSON/2.52 works for X and JSON/2.53 works for Y, how do you control for versions without having to ask them to install their versions locally? In another scenario, what if a version worked for you 3 months back and the updated version doesn't work for you anymore? How do you keep track of versions if you install everything in one directory?
I also have more questions on the module local::lib but I will post it as another question.
Thanks!
Maintaining concurrent versions of CPAN modules is asking for grief. I would suggest instead - don't, use docker for anything that's got any sort of deployment. That way you can have local installation of stuff + deps in an isolated container.
It's a bit like early days yet on docker, but they're a significant amount of enthusiasm and support for it from some very big names.
Personally I'm just using it to bundle up "mojolicious" perl webapps behind a reverse proxy, and maintain their dependencies as a self contained installation (which I can run/test/deploy autonomously)

How can I revert to the previous Perl setup after installing Bundle::CPAN?

After installation of latest Bundle::CPAN, I realized that certain legacy code will not run in this latest version. I wish to remove (and not just unlink) this latest version and revert back to using the old version. How can I do that?
Perl nor CPAN.pm have a way to go back to what you had before you installed later versions of modules. See How can I de-install a Perl module installed via cpan?.
Next time, however, you can fix this problem.
Here's what I recommend:
If you can figure out which modules are causing the problem, install those in separate directories and adjust #INC to use those directories. Do that until you figure out the module versions you need.
When you figure out the distributions you need, grab those from CPAN or BackPAN. Keep the distributions locally and set up your own MiniCPAN with just those distros. Since CPAN has as a design feature to track only the latest versions of any distribution, you want to make your local MiniCPAN pretend that the older distributions are the latest versions. That's why I created MyCPAN::App::DPAN. I write a bit about this in Manage multiple MiniCPANs, and version them
There's also Pinto and Stratopan, which serve like source control for your private CPAN-like repository. Rollback if you get something you don't like.
Once you get things fixed, put your Perl libraries into source control so you can always roll back an installation. See Manage your Perl modules with git.
Come up with an installation and testing policy where you don't disturb anything before you know that everything works. Part of that is testing new installations outside of your main Perl library directories. When you think you're satisfied, then you can install into those directories.
The easiest way is by visiting metacpan.
Search for your module. In your case Bundle::CPAN.
You then have to select the relevant version of the module that you want to install by clicking on the arrow to the left of the module name:
Once you click on the version you want from the drop-down you can then download it or just do for example:
cpanm http://cpan.metacpan.org/authors/id/A/AN/ANDK/Bundle-CPAN-1.861.tar.gz
Hope that helps

What do you expect from a package manager for Emacs? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
Although several thousand Emacs Lisp libraries exist, GNU Emacs, until version 24.1 did not have an (internal) package manager.
I guess that most users would agree that it is currently rather inconvenient to find, install and especially keep up-to-date Emacs Lisp libraries.
Pages that make life a bit easier
For versions of Emacs older than 24.1:
Emacs Lisp List - Problem: I see dead people (links).
Emacswiki - Problem: May contain traces of nuts (malicious code).
Emacsmirror - The package repository I am working on. Problem: No package manager supports it natively yet.
Some package managers
It's not that nobody has tried yet. (Some of these did not exist when this question was asked.)
auto-install
borg.el - Assimilate Emacs packages using Git submodules.
el-get.el - Supports many sources.
elinstall.el
epackage aka DELPS - Debian packaging concepts applied to Emacs Lisp packages.
epkg.el - This is now just a tool for browsing the Emacsmirror.
install.el
install-elisp.el
jem-pkg.el
package.el - ELPA. Seems like it will be included in Emacs 24.
UPDATE -- package.el is included in GNU Emacs, starting with version 24.1
pases.el
pelm - Command line installer; using php.
plugin.el
straight.el - Recent and experimental, has not reached 1.0 release yet.
use-package.el
XEmacs package manager
package has been included in the Emacs trunk. epkg is not ready yet and also currently not available. At least install-elisp, plugin and use-package do not seem to be actively maintained anymore.
I have created a git repository containing all these package managers as submodules.
Some utilities that might be useful
Package managers could use these utilities and/or they could be used to maintain a mirror of packages.
date-calc.el - Date calculation and parsing routines.
ell.el - Browse the Emacs Lisp List.
elm.el, elx.el, xpkg.el - Used to maintain the Emacsmirror.
genauto.el - Helps generate autoloads for your elisp packages.
inversion.el - Require specific package versions.
loadhist.el, lib-requires.el, elisp-depend.el - Commands to list Emacs Lisp library dependencies.
project-root.el - Define a project root and take actions based upon it.
strptime.el - Partial implementation of POSIX date and time parsing.
wikirel.el - Visit relevant pages on the Emacs Wiki.
Discussions about the subject at hand
emacs-devel 20080801
comp.emacs 20021121
RationalElispPackaging
The question (finally)
So - I would like to know from you what you consider important/unimportant/supplementary etc. in a package manager for Emacs.
Some ideas
Many packages (the Emacsmirror provides that largest available collection of packages, but there is no explicit support in any package manager yet).
Only packages that have been tested.
Support for more than one package archive (so people can choose between many/tested packages).
Dependency calculated based on required features only.
Dependencies take particular versions into account.
Only use versions that have been released upstream.
Use versions from version control systems if available.
Packages are categorized.
Packages can be uninstalled and updated not only installed.
Support creating fork of upstream version of packages.
Support publishing these forks.
Support choosing a fork.
After installation packages are activated.
Generate autoload files.
Integration with Emacswiki (see wikirel.el).
Users can tag, comment etc. packages and share that information.
Only FSF-assigned/GPL/FOSS software or don't care about license.
Package manager should be integrated be distributed with Emacs.
Support for easily contacting author.
Lots of metadata.
Suggest alternatives before installing a particular package.
I am hoping for these kinds of answers
Pointers to more implementations, discussions etc.
Lengthy descriptions of a set of features that make up your ideal package manager.
Descriptions of one particular desired/undesired feature. Feel free to elaborate on my ideas from above.
Surprise me.
I'm still learning Emacs, so I haven't had a chance to look into package managers, but a great feature would be to inform the user that the package is available if they try to use it but it's not on their system. For example, I wanted to edit a PHP file on a server once, and I tried
M-x php-mode
and Emacs was all like
M-x php-mode [no match]
when it should have been like
php-mode available from ftp.gnu.org. install? (y/n)
and then it would have installed and loaded up php-mode for me. That would have made my day right there.
Automatic publishing from version control
I'd love to see a standard, central, and single Emacs package manager. Right now, I'd put my money on ELPA, but there is still a long way to go.
The biggest thing that would help an Emacs package manager would be to make it super trivial to publish packages. In my opinion, I'd like to see this happen in combination with a version control system like git on a central hosted platform like GitHub -- something that would make it easy for authors to publish their packages and would make it easy for others to contribute back.
Similar to how GitHub (used to) make it easy to publish RubyGems, I'd like to see something similar in an Emacs package manager. For example, tag your repository with "vX.Y.Z" and have your elisp goodness automatically available to all.
The added benefit of using a popular backend like GitHub is that you'd immediately get a lot of exposure which should help drive its success.
What I expect most is that everything useful is on it, and works well. This requires you (or a team of maintainers) to aggressively pursue packaging everything for it, and doing whatever that involves — emailing every author of a useful package, and so on.
For instance, the reason Debian (and its derivatives: Ubuntu etc.) is so good is that you can happily use your system without ever having to install something outside the repositories, and that everything on it is thoroughly tested. The actual features of the package manager are important, but secondary to the managed packages themselves.
Easy configuration synchronization: I, like many people, use Emacs on many different computers and servers, some of them my own and some not. It would be amazing if the package manager had some sort of file which I could transfer from one computer to another; then, on the latter computer, the package manager would bring my Emacs into the state I like it in -- all the packages installed and configurations set. Combined with the ability to be able to easily install either site-wide (if one has root permissions) or as a single user, I could synchronize all of Emacsen everywhere.
I'm nearly positive that the best solution involves submitting more packages to ELPA and adding multi-source support to package.el. The Emacs maintainers have said that they would consider including package.el in version 24 as long as it pointed to an FSF repository by default.
Of course, submission also needs to be an automated process too; the current method of mailing the ELPA maintainer only works on a small scale.
No matter how this is done, the most important thing in my opinion is that it should be trivial to submit packages to the repository. At the same time, we do not want those packages to be instantly available, to guard against malicious code(and due to licensing issues). Unless there is a "trust" system in place, based on crypto signatures.
Also useful:
"metapackages", to install several packages at once.
In the same way, we should be able to install a set of elisp files, for maintainability
"Broken" packages should not be allowed to disrupt Emacs startup. This is easy and I have it implemented in my own .emacs
Ability to install files other than scripts. This is often overlooked, but very useful. You'd be able to, for instance, ship images, for icons, toolbars, etc.
Versioning:package X requires package Y > 1.0
Testing: perform basic sanity checks, testing for conflicts (keybindings, function redefinitions, functions that are expected to be present but aren't, etc).
BUG TRACKING: I can't stress the importance of this enough. Having a centralized place to report package bugs (and being able to track them) is extremely important to assure the quality of the packages.
Some sort of compressed archive seems to be best to do some of the above.
So far, a much improved ELPA seems the way to go.
I once spent some time writing a small package manager for Emacs.
http://gmarceau.qc.ca/plugin.el
I wrote:
Plugin is my attempt at creating a
package manager for Emacs. Plugin
will automatically downloads Emacs
extensions, unpacks them in a
directory, adds that directory to the
load-path, generates auto-load
annotations, and modify your dot-emacs
file. The auto-load annotations are a
little-known feature of Emacs. Once
they are generated, Emacs extensions
load quickly and incrementally, which
is really nice if you have as many
extensions installed as I do.
You will need two library files to get it to run, loop-constructs.el and record.el
I think the hackers for the iPhone got quite close to what I want, as does Ubuntu's "apt".
I like to be able to:
add
remove (package only)
remove user settings
view documentation
upgrade ( after reading the change log)
add new archive ( aka add repository )
see dependencies
see version
search for name, keyword
browse by (date added, date modified, name)
save all installed packages & settings
load set of packages & settings
I would like a main set of things that all work nicely and are the recommended way of doing whatever. Then a global set where everything working gets in. Then the ability for anyone to host their own archive.
It would be nice if this was all tied into git/svn/whatever so that you could install old versions. Make your own patches by forking off etc etc etc....
Besides the mentioned above, i expect something like debian, and other repositories - set of the stable, experemental, untested packages. Ability to add my own repositories - i use lot of the packages directly from VCS, so it could be useful to create my own packages
I think that the package manager should take a lot of inspiration from Rubygems. I also think that it should have a site like Gemcutter.
A central repository could also be nice (like Emacsmirror). This however may not be necessary if a site like Gemcutter exists that collects all packages.
I think these things are important for this to work.
Central location of some kind that collects all packages
Easy to add packages
Easy to maintain packages
Easy to contribute to other packages
Easy to install, uninstall and update packages
Possibility to add package dependencies
Common structure for all packages
So a package manager like Rubygems with a site like Gemcutter and a central repository like Emacsmirror (preferably on Github because of it's social coding) would do Emacs really good.
All in all I think that much inspiration should be taken from Rails and how Rails handles Gems.
I don't know how fresh this question is...
but the model I would like to see is CPAN. I also don't know Rubygems but it sounds similar to CPAN.
CPAN is a perl archive + library management system. When I need to write a perl program that requires... FTP or SOAP or JSON or XML or ZIP, or...etc, I can run the CPAN package manager, select the requisite package for download, view and verify the dependencies, then install everything. CPAN is mirrored .."everywhere".
CPAN works wonderfully for my purposes, and something similar for emacs would be nice to have. It also supports building C/C++ code on demand.
That's what I would like to see in emacs.
Some further comment on requirements.
explicit download of packages. No auto install. No invisible downloads. I want to ask for new libraries or new function.
I should be able to list the name/version/timestamp of installed packages.
If my friend gives me his list, I should be able to diff his emacs state against mine.
check-for-updates function. What updates are available? What do they fix?
depedency checking, verification, and download. If I install csharp-mode and it requires v5.0.28 of cc-mode, then it should confirm with me, that I must also download cc-mode.
there should be some sort of community ranking of these packages, like ranking torrents on isohunt. I want to see if a package has 3 upvotes or 3000.
"transactional" behavior. If an install goes boom, it must unwind to a last-known-good state.
failsafes. If I've put custom mods in linum.el, it should refuse to install a new version over my changes, unless I explicitly allow it. It should warn me before even starting. Do this with checksums/md5's over the existing install.
have the option of running some packages from compressed archives, like zip files. So I never have any doubt that I have not updated any of the embbedded elisp.
ability to use mirrored hosts for package distribution.
all this function should be accessible through M-x library-manageemnt or something.
Finally, it would be nice to have a way to segregate or organize libraries of functions. Hierarchical namespaces. Emacs' flat namespace is very dated. This is sort of independent but complementary to the core function of package management. I'm not a lisp guru so I don't know how hard this would be; maybe there is already a way to do it.
Package managers don't offer anything I value w.r.t. single-file elisp packages with simple dependencies: adding and deleting from site-lisp has never caused problems. It's packages that depend on external programs (e.g., ispell), multi-file packages (e.g., auctex, org-mode) that can be tricky. Can't think of any single-file elisp package with nontrivial dependencies, offhand.
For these, short of a package manager, I'd like emacs' elisp-packages to acquire test suites which can be run en masse, and which provide useful information in the event of dependency failures.