I would like to benefit from the excellent package management capabilities and neuro/scientific support of Gentoo Linux when I am setting up continuous integration (CI) testing for my software (e.g. SAMRI).
Sadly, my favorite platform, TravisCI, seems to offer Ubuntu, Ubuntu, and MacOS environments only. I would like to know how to best (i.e. with minimal overhead and delay) set up a Gentoo Linux environment on TravisCI (or if need be, on another CI platform).
I have tried using the Gentoo Prefix bootstrap script, however:
simply calling it in - yes "" | ./bootstrap-prefix.ch will overflow the 4MB log limit
redirecting it to a file - yes "" | ./bootstrap-prefix.sh >> prefix_log.txt will cause the job to time out
I can't figure out how to increase the time out for such a command. -
travis_wait 60 |& yes "" | ./bootstrap-prefix.sh >> prefix_log.txt times out after 10 minutes.
In excess of all these issues, I feel bootstrapping an entire Gentoo install from scratch is a waste of time and resources. I am thinking maybe this can be more elegantly addressed by using one of the Gentoo OpenStack images - or something else?
If you want to test on Gentoo, I suggest using Docker (https://docs.travis-ci.com/user/docker), pull down a small Gentoo image (e.g., https://hub.docker.com/r/gentoo/stage3-amd64/) and run your tests on that image.
Related
Is it possible to set the path where the berkshelf plugin puts the cookbooks it installs? (As in the .berkshelf folder)
I am running Windows 7.
I am currently trying to install a mysql server using an opscode cookbook to a vm and here at work they have the %HOMEDRIVE% system variable set to a network drive. So when .berkshelf starts at the beginning of the Vagrantfile, it pushes the cookbooks to the network drive and it causes it to be slow and well, its not where it should be. Is there a fix to this?
VirtualBox did this as well, but I fixed it by altering the settings. I tried looking for some sort of equivalent settings for berkshelf, but the closest I got was for the standard berkshelf (thats not a vagrant plugin), it appears you can set this environment variable:
ENV['BERKSHELF_PATH']
Found here:
http://www.rubydoc.info/github/RiotGames/berkshelf/Berkshelf#berkshelf_path-class_method
I need to be able to have the cookbooks it reads from the berksfile store to my laptops local drive instead, as in my scenario I cannot have the mobility of the VM limited to the building because of files that are stored on the network.
Any incite would be much appreciated.
Perhaps its better to use the actual berkshelf over the vagrant plugin?
Thanks.
If you want to have the portability - a full chef-repo ready for chef-solo runs, better off using standalone berkshelf instead of the vagrant-berkshelf plugin - which is NOT that flexibly.
For complex cookbooks, I prefer to use standalone berkshelf as it allows me to do berks install --path chef/cookbooks to copy all cookbooks required from ~/.berkshelf/cookbooks, then I can just tar the whole thing and transfer to other machines for the same chef-solo run. some people use capistrano automate the tar and scp/rsync over the network. I just use rysnc/scp;-)
HTH
I'm writing some new web services in Go.
What are some deployment strategies I can use, regardless of the target platform? For example, I'm developing on a Mac, but the staging/production servers will be running Linux.
Are there some existing deployment tools I can use that support Go? If not, what are some things I can do to streamline the process?
I use LiteIDE for development. Is there any way to hook LiteIDE into the deployment process?
Unfortunately since Go is such a young language not much exists yet, or at least they've been hard to find. I would also be interested in the development of such tools for Go.
What I have found is that some people have been doing it themselves, or they've adapted other tools, such as Capistrano, to do it for them.
Most likely it's something you'll have to do yourself. And you don't have to limit yourself to shell scripts - do it in Go! In fact many of the Go tools are written in Go. You should avoid compiling on the target system as it's usually a bad practice to have build tools on your production system. Go makes it really easy to cross compile binaries. For example, this is how you compile for ARM & Linux:
GOARCH=arm GOOS=linux go build myapp
One thing you could do is hop on the #go-nuts freenode IRC channel or join the Go mailing list and ask other Gophers what they're doing.
Capistrano sounds like a good idea for deployment alone. You can also do cross-compilation as Luke suggested. Both will work just fine.
More generally though... I'm also kind of torn between OS X (development) and Linux (deployment) and in fact I ended just developing in a virtual machine via VirtualBox and Vagrant. I'm using TextMate 2 for text editing but installing many of development tools on a Mac is just a major PITA and I'm just more comfortable with having Debian or the like running somewhere in the background. The bonus is - this virtual environment can mirror deployment environment so I can avoid surprises when I deploy my code, whatever the language.
I haven't tried it myself, but it appears you can cross compile golang (either with goxc or Dave Cheney's golang-crosscompile), albeit with some caveats.
But if you need to match the environment with production, which probably you should most of the time, it's safest to go as Marcin suggested.
You can find some prebuilt VirtualBox images on http://virtualboxes.org/images/ although creating one yourself is pretty easy.
what are some things I can do to streamline the process?
The cross-compilation idea should be even more appealing with Go 1.5 (Q3 2015), as Dave Cheney details in "Cross compilation just got a whole lot better in Go 1.5":
Before:
For successful cross compilation you would need
compilers for the target platform, if they differed from your host platform, ie you’re on darwin/amd64 (6g) and you want to compile for linux/arm (5g).
a standard library for the target platform, which included some files generated at the point your Go distribution was built.
After (Go 1.5+):
With the plan to translate the Go compiler into Go coming to fruition in the 1.5 release the first issue is now resolved.
package main
import "fmt"
import "runtime"
func main() {
fmt.Printf("Hello %s/%s\n", runtime.GOOS, runtime.GOARCH)
}
build for darwin/386
% env GOOS=darwin GOARCH=386 go build hello.go
# scp to darwin host
$ ./hello
Hello darwin/386
Or build for linux/arm
% env GOOS=linux GOARCH=arm GOARM=7 go build hello.go
# scp to linux host
$ ./hello
Hello linux/arm
I'm developing on a Mac, but the staging/production servers will be running Linux.
Considering the compiler for Go is in Go, the process to produce a Linux executable from your Mac should become straightforward.
I am wanting to create an install script in the fashion of npm's (curl http://example.com/install.sh | sh) but it leaves me asking the question: can I just write the script in perl? As far as I know, perl is installed by default on at least ubuntu, RHEL & OS X - so I'm wondering in the year 2011, can I not write shell and still be generic enough for everyone? Is there a third and better option?
This would be targeting a user's development box, not staging or production.
What I want to do specifically is use this install script to bootstrap a development environment easily without the overhead of creating and maintaining packages. The script would have 4 steps:
check and make sure git is installed
use git to clone a repo to cwd
pull down and save a perl control script to /usr/bin, make it executable
add some environment variables (related post: linux cross-distro environment variable modification via script?)
That's it. My thinking is this is simple and generic enough to use a bootstrap script rather than a package. And my target audience is a user's unix or linux local development system.
The best option is to simply use the existing, well-oiled and -used (development) toolchain for the language the target app is written in. Not doing so frivolously discards the network effects gained from the ecologies that have grown around them.
C: GNU autotools
Haskell: Cabal
Perl: EU::MM, M::B, M::I
etc. etc.
Installing from the Web should be reserved for conveniently bootstrapping a user's system into the development environment.
Do tell more details about your software to get less general advice.
Edit: response to your addendum.
I dissuade you from a Web installer. This isn't bootstrapping an installation tool, this is plain installation of software and it should be done with with e.g. a Module::Build subclass.
I think perl is ubiquitous enough for you to write your installer in it. Shell is a lot more awkward anyway.
You might want to consider actually packaging your application as a deb or rpm or even using makeself rather than providing a raw script.
Here's a list of the various distributions of perl:
https://www.socialtext.net/perl5/distributions
Even if perl doesn't ship on every little obscure distro it's just an apt-get (or whatever) away. You might run into problems due to the various versions of perl installed however.
Use something like:
perl -E "$( wget -q -O - http://host/intall.pl )"
Also you can use
`cmd`
instead of
$(cmd)
but anyway, double-quote your choice.
I am using the Ruby Daemon-kit to setup a services that does various background operations for my Rails application.
It works fine when I call in on the commandline:
./bin/bgservice
How do I go about creating a daemon initd starter script for it, so that it will autostart on reboot?
There's a few approaches:
You could write /etc/init.d/ scripts that could be placed into the /etc/rc?.d/ directories (or wherever they live on your target distributions). Some details on this mechanism can be found in the Debian policy guidelines and openSUSE initscript tutorial. There's an annoying number of distribution-specific idiosyncrasies in initscripts, so don't feel about writing a simple one and asking distributions to contribute 'better' ones tailored for their environment. (For example, any Debian-derived distribution will provide the immensely useful start-stop-daemon(8) helper, but it is sorely missing from other distributions.)
You could write upstart job specifications for the distributions that support upstart (which I think is Ubuntu, Google ChromeOS, Fedora, .. more?). upstart documentation is still pretty weak, but there are some details and plenty of examples in /etc/init/ on Ubuntu, probably the same location in other distributions that use upstart. Getting the dependencies correct may be some work across all distributions, but upstart job specifications look far simpler to write and maintain than initscripts.
You could add lines to /etc/inittab on distributions that still support the standard SysV-init inittab(5) file. This would only be useful if your program doesn't do the usual daemon fork(2)/setsid(2)/fork(2) incantation, as init uses the pid it gets from fork(2) to determine if your program needs to be restarted.
Modern Vixie cron(8) supports a #reboot specifier in crontab(5) files. This can be used both by the system crontab as well as user crontabs, which might be nice if you just want to run the program as your usual login account.
As the author of daemon-kit I've avoided making any init-style scripts due to coping with the various distributions and they're migrations from old init-V style to newer upstart/insserv, saving myself a nightmare.
How I recommend to do this is to use the god config-generator, and ensure god is started on boot (by runit or some other means), and god starts the daemon up initially and keeps it running.
At best I'll expand daemon-kit to be able to generate runit scripts for boot...
HTH.
I want to install some plugins and upgrade some to the latest version. They are not included in the Eclipse SDK downloads. So I must install the individuals manually.
The problem is, the network connection to the update site is very slow. (Maybe 1k bytes per second, or lower) Currently, I have an Amazon EC2 box in us-east-1 region, which is a lot faster to the update site. So, I'd like to install the plugins in EC2 box, and then copy them back to my local machine (in China).
That will look like:
very slow, or inaccessible
My machine #China <-----------------------------------> Update Sites
(1k bytes/sec)
|
|
|
V
much faster fast
My machine #China <-----------------> EC2 #US-east <------------------> Update Sites
(100k bytes/sec) (1M bytes/sec)
Now, my EC2 box is running Debian linux, I'm not going to install X windows there, so it can't start Eclipse in GUI mode. Though, I hope I can get updates from command line only.
EDIT
To clarify the question: how to "Install Software" for Eclipse in command line? Since Eclipse is OSGi-based, I guess I can do it using some kind of osgi shell, maybe?
You can install and delete plugins for eclipse through a CLI. This is possible through the p2.director:
https://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.platform.doc.isv%2Fguide%2Fp2_director.html
In regards to updating eclipse through CLI, I stumbled across this question looking for that answer.