SSHKit command or Capistrano task to filter/replace tokens on upload - capistrano

I'm using Capistrano to deploy configuration files for a legacy non-Ruby application, that for arcane legacy reasons need to be parameterized with the fully-qualified name of the target host, e.g.
name: myservice-stg
identifier: myservice-stg-1.example.org:8675
baseURI: http://myservice-stg-1.example.org:8675
Apart from that, for a given environment, there's no difference between the config files, so I'd like to be able to just define a template (example uses Mustache but could be ERB or whatever):
name: myservice-stg
identifier: {{fqhn}}:8675
baseURI: http://{{fqhn}}:8675
My current idea for a hack is just to use gsub and a StringIO:
config_tmpl = File.open('/config/src/config.txt')
config_txt = config_tmpl.gsub('{{fqhn}}', host.hostname)
upload!(StringIO.new(config_txt), 'dest/config.txt')
But it seems like there ought to be a more standard, out-of-the box solution.

Tools like Ansible and Chef are great for this, but might be overkill if this is all you're trying to do.
Your proposed solution looks fairly standard. Using ERB (or other templating system) wouldn't be that much more work and provides flexibility/reusability down the road:
template_path = File.open('/config/src/config.txt.erb')
config_txt = ERB.new(File.new(template_path).read).result(binding)
upload! StringIO.new(config_txt), 'dest/config.txt', mode: 0644
ERB:
name: myservice-stg
identifier: <%= host.hostname %>:8675
baseURI: http://<%= host.hostname %>:8675

Related

vSphere Build Version via API

Is there a way do get the vsphere build versing using any API/SDK/REST?
I know it's possible using powershell on vcenter for that, but it'd be great if there was another option.
Like described here: https://www.virtuallyghetto.com/2017/08/powercli-script-to-help-correlate-vcenter-esxi-vsan-buildversions-wo-manual-vmware-kb-lookup.html
It looks like you should be able to using the VMware vSphere API Python Bindings, as you can just simulate going through the Managed Object Browser.
Parent Managed Object ID: ServiceInstance
Property Path: content.about
And then there is a build string which is what you are looking for.
I figure out how to do this, my need is having as much information about vsphere as possible, so get datacenter, cluster and host details is mantaroy.
For that I used the official ruby api, rbvmomi, but I believe it's exactly the same thing for python one and golang.
It's needed to interact through host folder under root/children object, which is not that clear on wmware api docs, to get it easier follow a piece of code:
vim = RbVmomi::VIM.connect host: host, user: 'user', password: 'pass', insecure: true, debug: false
vim.root.children.each do |root_child|
root_child.hostFolder.children.each do |child|
child.host.each do |host|
prod = host.config.product
puts host.name,
prod.apiType,
prod.apiVersion,
prod.build,
prod.fullName,
prod.instanceUuid,
prod.licenseProductName,
prod.localeBuild,
prod.localeVersion,
prod.name,
prod.osType,
prod.productLineId,
prod.vendor,
prod.version
end
end
end

What is SINATRA_ENV?

Am I correct that within Sinatra there is no use or mention of SINATRA_ENV? For some reason I thought there was. Here's what I think is true now:
Sinatra bases its sense of environment on RACK_ENV. If RACK_ENV is not defined it defaults to development.
If you use ActiveRecord you will also need to set RAILS_ENV because Rails modules don't pay attention to RACK_ENV and certainly not to SINATRA_ENV
Can someone corroborate this analysis?
You are mostly correct. Some developers will build a SINATRA_ENV into their applications (and set RACK_ENV to it), but it is not built into the library. Sinatra 2.0 is introducing APP_ENV, which will be preferred over RACK_ENV.
ActiveRecord appears to check RACK_ENV if RAILS_ENV is not set, so you shouldn't need to set RAILS_ENV manually. This could potentially be a problem with Sinatra 2.0, so that's something to keep in mind. A simple workaround would be to set RACK_ENV to APP_ENV for backwards compatibility with other gems, or vice-versa.
Within Sinatra applications, classic and modular, you should be using environment (or self.environment in helpers and routes) and convenience methods such as development?, production?, etc. instead of inspecting the environment variables explicitly. I've taken to using Sinatra::Base.environment (and the aforementioned convenience methods) outside of my application(s) once Sinatra is loaded. For example:
# Gemfile
gem 'sinatra', require: 'sinatra/base' # in default group
group :development do
# ..
end
group :production do
# ..
end
And:
# config.ru
require 'bundler/setup'
Bundler.require :default # this loads Sinatra, which inspects e.g. RACK_ENV
Bundler.require Sinatra::Base.environment
It bears keeping in mind that Sinatra is a tiny, tiny library (last time I checked, about 2K LOC), and it really does leave a lot of these application concerns in the hands of the developer. There aren't many hard and fast rules!

Setting :deploy_to from server config in Capistrano3

In my Capistrano 3 deployment, I would like to set the set :deploy_to, -> { "/srv/www/#{fetch(:application)}" } so the :deploy_to is different for each server it deploys to.
In my staging.rb file I have:
server 'dev.myserver.com', user: 'deploy', roles: %w{web app db}, install_path: 'mycustom/path'
server 'dev.myserver2.com', user: 'deploy', roles: %w{web app db}, install_path: 'mycustom/other/path'
My question is: would it possible to use the "install_path" I defined, in my :deploy_to? If that's possible, how would you do it?
Finally, after looking around, I came onto an issue from one of the developer of Capistrano, stating specifically that it can't be done
Quote from the Github issue:
Not possible, sorry. fetch() (as is documented widely) reads values
set by set(), the only reason to use set() and fetch() over regular
ruby variables is to provide a consistent API between plugins and
extensions, and because set() can take a Proc to be resolved later.
The variables you are setting in the host object via the server()
command belong to an individual host, some of them, user, roles, etc
have special meanings. For more information see
https://github.com/capistrano/sshkit/blob/master/EXAMPLES.md#do-something-different-on-one-host-or-another-depending-on-a-host-property.
If you specifically need to deploy to a different directory on each
machine you probably should not be using the built-in tasks (they
don't fit your needs), and rather copy the deploy.rake from the Gem
into your own project, and modify it as you need. Which in this case
might be to not take fetch(:deploy_to), but to read that from a host
property.
You could try to do something where before doing anything that relies
on calling fetch(:deploy_to), you set() it using the value from
host.someproperty but I'm pretty sure that'll break in exciting and
interesting ways.

Chef LWRP, Definition, or Cookbook for abstracting creation of Nginx virtual hosts

I'm trying to figure out the correct way to architect a solution to automatically configure new Rails App servers.
I've looked at the chef-rails cookbook and it seems a little verbose. In our case we always deploy Nginx a certain way, always perform backups a certain way, etc, so much of the configuration would be redundant from one node definition to the next.
My goal is to be able to create a new Rails App server by defining just the following information.
wh_webhead "test_app" do
ssl :enable
backups :enable
passenger :enable
ruby_version 2.0.0
db_type :mysql
db_user "testuser"
db_pass "3207496r9w6"
nagios_ssl_string_match "login"
end
Then I would like Chef to perform the following actions:
Create user accounts
Setup box and install
Install Nginx w/wildcard SSL cert
Configure log rotation
Setup firewall rules to allow traffic to ports 80 and 443
Install Passenger and RVM with Ruby 2.0.0
Create Rails app dirs following template (e.g. /opt/local/test_app)
Create new database on MySQL server, grant access, and setup firewall rules
Create firewall rules for Nagios and configure Nagios to monitor:
port 80 for redirection to port 443
port 443 for HTTP 200 status
port 443 for the text "login"
Configure backups for app dir (e.g. /opt/local/test_app)
I'm already using the community cookbooks for Nginx, Nagios, Ufw, etc and have created recipes in a custom cookbook to configure Mysql and Nginx. There's just a lot of duplicate code from one app's Nginx/Mysql cookbook to the next.
What I'm struggling with is where to use Cookbooks, Recipes, LWRPs and Definitions to properly abstract this.
Should I put the default configuration for Nginx and Mysql in Definitions and then use those in recipes or create custom wrapper cookbooks with the defaults?
First, take a look at the application_ruby and artifact cookbook, both of which can automate these workflows for you.
I specifically enjoy using the artifact cookbook, as it provides a lot of flexibility, but the application_ruby cookbook has built-in support for Passenger, Unicorn and other tools you'd normally find in a Rails application requirements.
As for your question regarding Cookbooks, Recipes, LWRPs and Definitions I would definitely look at #sethvargo's answer at https://stackoverflow.com/a/21733093/747032. It provides a good guide on what to use when, from an employee at Opscode (now called Chef (the company)), and someone who is constantly involved in the Chef community and thus has excellent knowledge on this topic.
As far as my advice (which I'll keep concise):
Use LWRP's to wrap a lot of resources that are always called together, for example, we use an "AWS EBS" LWRP, to create, mount and format new EBS'.
Use recipes to call on all your LWRP's (both custom and public) and resources.
Don't use definitions, they are really deprecated by LWRP's in my opinion.

What's the best Perl module for hierarchical and inheritable configuration?

If I have a greenfield project, what is the best practice Perl based configuration module to use?
There will be a Catalyst app and some command line scripts. They should share the same configuration.
Some features I think I want ...
Hierarchical Configurations to cleanly maintain different development and live settings.
I'd like to define "global" configurations once (eg, results_per_page => 20), have those inherited but override-able by my dev/live configs.
Global:
results_per_page: 20
db_dsn: DBI:mysql;
db_name: my_app
Dev:
inherit_from: Global
db_user: dev
db_pass: dev
Dev_New_Feature_Branch:
inherit_from: Dev
db_name: my_app_new_feature
Live:
inherit_from: Global
db_user: live
db_pass: secure
When I deploy a project to a new server, or branch/fork/copy it somewhere new (eg, a new development instance), I want to (one time only) set which configuration set/file to use, and then all future updates are automatic.
I'd envisage this could be achieved with a symlink:
git clone example.com:/var/git/my_project . # or any equiv vcs
cd my_project/etc
ln -s live.config to_use.config
Then in the future
git pull # or any equiv vcs
I'd also like something that akin to FindBin, so that my configs can either use absolute paths, or relative to the current deployment. Given
/home/me/development/project/
bin
lib
etc/config
where /home/me/development/project/etc/config contains:
tmpl_dir: templates/
when my perl code looks up the tmpl_dir configuration it'll get:
/home/me/development/project/templates/
But on the live deployment:
/var/www/project/
bin
lib
etc/config
The same code would magically return
/var/www/project/templates/
Absolute values in the config should be honoured, so that:
apache_config: /etc/apache2/httpd.conf
would return "/etc/apache2/httpd.conf" in all cases.
Rather than a FindBin style approach, an alternative might be to allow configuration values to be defined in terms of other configuration values?
tmpl_dir: $base_dir/templates
I'd also like a pony ;)
Catalyst::Plugin::ConfigLoader supports multiple overriding config files. If your Catalyst app is called MyApp, then it has three levels of override: 1) MyApp.pm can have a __PACKAGE__->config(...) directive, 2) it next looks for MyApp.yml in the main directory of the app, 3) it looks for MyApp_local.yml. Each level may override settings in each other level.
In a Catalyst app I built, I put all of my immutable settings in MyApp.pm, my debug settings in MyApp.yml, and my production settings in MyApp_<servertype>.yml and then symlinked MyApp_local.yml to point at MyApp_<servertype>.yml on each deployed server (they were all a little different...).
That way, all of my config was in SVN and I just needed one ln -s step to manually config a server.
Perl Best Practices warns against exactly what you want. It states that config files should be simple and avoid the sort of baroque features you desire. It goes on to recommend three modules (none of which are Core Perl): Config::General, Config::Std, and Config::Tiny.
The general rational behind this is that the editing of config files tends to be done by non-programmers and the more complicated you make your config files, the more likely they will screw them up.
All of that said, you might take a look at YAML. It provides a full featured, human readable*, serialization format. I believe the currently recommend parser in Perl is YAML::XS. If you do go this route I would suggest writing a configuration tool for end users to use instead of having them edit the files directly.
ETA: Based on Chris Dolan's answer it sounds like YAML is the way to go for you since Catalyst is already using it (.yml is the de facto extension for YAML files).
* I have heard complaints that blind people may have difficulty with it
YAML is hateful for config - it's not non-programmer friendly partly because yaml in pod is by definition broken as they're both white-space dependent in different ways. This addresses the main problem with Config::General. I've written some quite complicated config files with C::G in the past and it really keeps out of your way in terms of syntax requirements etc. Other than that, Chris' advice seems on the money.