How to upgrade HAProxy on OpsWorks - haproxy

We are using an AWS OpsWorks stack with Chef Version 11.10. with the default HAProxy Layer. We would like to upgrade the HAProxy to the latest version 1.6-stable (from the default 1.4-stable).
There seems to be a dedicated PPA for our Ubuntu version.
But where can we make OpsWorks use this PPA to install HAProxy?
In the default cookbook there is a default attributes file that has the following lines:
default[:haproxy][:version] = '1.4.22'
default[:haproxy][:patchlevel] = '1'
default[:haproxy][:rpm] = "haproxy-#{node[:haproxy][:version]}-#{node[:haproxy][:patchlevel]}.#{rhel_arch}.rpm"
default[:haproxy][:rpm_url] = "#{node[:opsworks_commons][:assets_url]}/packages/#{node[:platform]}/#{node[:platform_version]}/#{node[:haproxy][:rpm]}"
Overwriting the file in our cookbooks and naïvely changing the version number here does not have the desired effect.

We ended up overwriting the recipe haproxy/recipes/default.rb like so:
#Install software-properties-common if not installed
package 'software-properties-common' do
action :install
end
#Add PPA for haproxy 1.6 and update repo
execute "add-ppa-update" do
command "add-apt-repository ppa:vbernat/haproxy-1.6 && apt-get update -y"
action :run
end
package "haproxy" do
retries 3
retry_delay 5
version '1.6.4-3ppa1~trusty'
action :install
end
if platform?('debian','ubuntu')
template '/etc/default/haproxy' do
source 'haproxy-default.erb'
owner 'root'
group 'root'
mode 0644
end
end
include_recipe 'haproxy::service'
template '/etc/haproxy/haproxy.cfg' do
source 'haproxy.cfg.erb'
owner 'root'
group 'root'
mode 0644
notifies :restart, "service[haproxy]"
end
template "/etc/haproxy/server.pem" do
source "server.pem.erb"
owner 'root'
group 'root'
mode 0600
notifies :restart, "service[haproxy]"
end
service 'haproxy' do
action [:enable, :start]
end
Additionally we needed to update the haproxy.conf to work with the new version.
Everything runs beautifully now.

Related

Bazel Kubernetes Object Error: no objects passed to apply (Google Container Registry)

I have a k8s_object rule to apply a deployment to my Google Kubernetes Cluster. Here is my setup:
load("#io_bazel_rules_docker//nodejs:image.bzl", "nodejs_image")
nodejs_image(
name = "image",
data = [":lib", "//:package.json"],
entry_point = ":index.ts",
)
load("#io_bazel_rules_k8s//k8s:object.bzl", "k8s_object")
k8s_object(
name = "k8s_deployment",
template = ":gateway.deployment.yaml",
kind = "deployment",
cluster = "gke_cents-ideas_europe-west3-b_cents-ideas",
images = {
"gcr.io/cents-ideas/gateway:latest": ":image"
},
)
But when I run bazel run //services/gateway:k8s_deployment.apply, I get the following error
INFO: Analyzed target //services/gateway:k8s_deployment.apply (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //services/gateway:k8s_deployment.apply up-to-date:
bazel-bin/services/gateway/k8s_deployment.apply
INFO: Elapsed time: 0.113s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
$ /snap/bin/kubectl --kubeconfig= --cluster=gke_cents-ideas_europe-west3-b_cents-ideas --context= --user= apply -f -
2020/02/12 14:52:44 Unable to publish images: unable to publish image gcr.io/cents-ideas/gateway:latest
error: no objects passed to apply
error: no objects passed to apply
It doesn't push the new image to the Google Container Registry.
Strangely, this worked a few days ago. But I didn't change anything.
Here is the full code if you need to take a closer look: https://github.com/flolude/cents-ideas/blob/069c773ade88dfa8aff492f024a1ade1f8ed282e/services/gateway/BUILD
Update
I don't know if this has something to do with this issue but when I run
gcloud auth configure-docker
I get some warnings:
WARNING: `docker-credential-gcloud` not in system PATH.
gcloud's Docker credential helper can be configured but it will not work until this is corrected.
WARNING: Your config file at [/home/flolu/.docker/config.json] contains these credential helper entries:
{
"credHelpers": {
"asia.gcr.io": "gcloud",
"staging-k8s.gcr.io": "gcloud",
"us.gcr.io": "gcloud",
"gcr.io": "gcloud",
"marketplace.gcr.io": "gcloud",
"eu.gcr.io": "gcloud"
}
}
Adding credentials for all GCR repositories.
WARNING: A long list of credential helpers may cause delays running 'docker build'. We recommend passing the registry name to configure only the registry you are using.
gcloud credential helpers already registered correctly.
I had google-cloud-sdk installed via snap install. What I did to make it work is to remove google-cloud-sdk via
snap remove google-cloud-sdk
and then followed those instructions to install it via
sudo apt install google-cloud-sdk
Now it works fine

install mongo with chef

I've tried to figure out how to install a mongodb 3.4 instance using this chef cookbook. Nevertheless, I've not able to get to install it.
This is my mongodb.rb file content:
node.default['mongodb']['package_version'] = '3.4'
include_recipe 'mongodb::default'
And my metadata.db: depends 'mongodb', '~> 0.16.2'.
I've tried to verify it on centos-72 platform using kitchen verify centos-72. I'm getting this message:
ERROR: yum_package[mongodb-org] (mongodb::install line 77) had an error: Chef::Exceptions::Package: Version ["3.4"] of ["mongodb-org"] not found. Did you specify both version and release? (version-release, e.g. 1.84-10.fc6)
I'm realizing this cookbook tries to add this yum_repository:
yum_repository 'mongodb' do
description 'mongodb RPM Repository'
baseurl "http://downloads-distro.mongodb.org/repo/redhat/os/#{node['kernel']['machine'] =~ /x86_64/ ? 'x86_64' : 'i686'}"
action :create
gpgcheck false
enabled true
end
And according to this mongo documentation the link repository should have to be:
https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/3.4/x86_64/
instead of
"http://downloads-distro.mongodb.org/repo/redhat/os/..."
The repo you are using does not have version 3.4 available. You can verify this manually by just looking at the RPMs in http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/RPMS/

chef recipe doesn't find ohai plugin defined resource

I want to add additional resource 'version of installed openssh' to Ohai to use it in my openssh maintaining recipe.
On RHEL 5.11 Chef 12.4.1 Ohai 8.5.0 test workstation I have created and tested Ohai plugin
$ cat cookbooks/test/files/default/plugins/openssh.rb
Ohai.plugin(:Openssh) do
provides "openssh"
Ohai::Log.debug('plugin start')
def create_objects
openssh Mash.new
end
collect_data do
create_objects
openssh[:version] = 'ssh -V 2>&1 |head -1| cut -d, -f1| cut -d_ -f2 '
end
end
Local test of ohai plugin in irb is working fine.
Now I'm trying to check resource visibility in Chef recipe
$ cat test/recipes/default.rb
file "#{ENV['HOME']}/x.txt" do
content 'HELLO WORLD'
end
output="#{Chef::JSONCompat.to_json_pretty(node.to_hash)}"
file '/tmp/node.json' do
content output
end
Chef::Log.info("============ test cookbook ** #{openssh['version']} **")
\#Chef::Log.info("============ test cookbook ** #{node['kernel']} **")
by running local chef-client
$ chef-client -z -m test/recipes/default.rb
To make additional plugin visible line is added to config files
$grep Ohai ~/.chef/*.rb
~/.chef/client.rb:Ohai::Config[:plugin_path] << '~/chef/cookbooks/test/files/default/plugins/'
~/.chef/knife.rb:Ohai::Config[:plugin_path] << '~/chef/cookbooks/test/files/default/plugins/'
(I understand that this is too explicit )
Although running with printing node['kernel'] is working fine , openssh version is not running with debug log that shows:
[2016-01-27T11:48:21-08:00] DEBUG: Cookbooks detail: []
[2016-01-27T11:48:21-08:00] DEBUG: Cookbooks to compile: []
[2016-01-27T11:48:21-08:00] DEBUG: **Loading Recipe File XXX/cookbooks/test/recipes/default.rb**
[2016-01-27T11:48:21-08:00] DEBUG: Resources for generic file resource enabled on node include: [Chef::Resource::File]
[2016-01-27T11:48:21-08:00] DEBUG: Resource for file is Chef::Resource::File
[2016-01-27T11:48:21-08:00] DEBUG: Resources for generic file resource enabled on node include: [Chef::Resource::File]
[2016-01-27T11:48:21-08:00] DEBUG: Resource for file is Chef::Resource::File
[2016-01-27T11:48:21-08:00] DEBUG: Resources for generic openssh resource enabled on node include: []
[2016-01-27T11:48:21-08:00] DEBUG: **Dynamic resource resolver FAILED to resolve a resource for openssh**
[2016-01-27T11:48:21-08:00] DEBUG: Re-raising exception: NameError - No resource, method, or local variable named `openssh' for `Chef::Recipe "XXX/cookbooks/test/recipes/default.rb"'
Questions:
How properly chef out additional plugin to recipe for local and remote execution? How to check that it is cheffed out and ready?
How properly notify chef-client to execute ohai additional plugin for local single recipe run and for remote run as well?
Any explanations and suggestions are welcomed.
Alex
A few issues: first check out https://github.com/coderanger/ohai-example to see how to package an ohai plugin in a cookbook for distribution. Second, node attributes from custom plugins still need to be accessed via the node object: node['openssh']['version']. Third, remember how execution ordering works in Chef (https://coderanger.net/two-pass/) and that the custom attributes won't be available until after the plugin is loaded and run.
Checkout mainstream before google!
This project describes how to deploy you plugin in 2017 year!
https://github.com/chef-cookbooks/ohai

Can't run Capistrano - dependency error

I've installed Capistrano on my Ubuntu with the following commands
sudo gem install capistrano
sudo gem install railsless-deploy # I want to deploy a PHP website
sudo gem install capistrano-ext
When I run "capify ." in my deployment folder, have got an error:
Same error with other Capistrano command like a simple "cap -h" ...
/usr/lib/ruby/1.9.1/rubygems/dependency.rb:247:in `to_specs': Could not find capistrano (>= 0) amongst [POpen4-0.1.4, Platform-0.4.0, addressable-2.3.4, addressable-2.3.2, bundler-1.3.5, bundler-1.2.3, bundler-1.0.21, chunky_png-1.2.8, chunky_png-1.2.7, coderay-1.0.9, coderay-1.0.8, commonjs-0.2.6, compass-0.12.2, compass-normalize-1.4.3, compass-normalize-1.4.2, compass-recipes-0.3.0, em-websocket-0.5.0, em-websocket-0.4.0, eventmachine-1.0.3, eventmachine-1.0.0, ffi-1.7.0, ffi-1.3.1, forge-0.5.0, formatador-0.2.4, fssm-0.2.10, fssm-0.2.9, guard-1.8.0, guard-1.7.0, guard-1.6.1, guard-1.3.3, guard-livereload-1.3.0, guard-livereload-1.2.1, guard-livereload-1.1.3, guard-livereload-1.0.1, hike-1.2.2, hike-1.2.1, http_parser.rb-0.5.3, json-1.7.7, json-1.7.6, json-1.6.5, less-2.3.2, less-2.2.2, less-2.0.8, libv8-3.16.14.1, libv8-3.11.8.13-x86-linux, libv8-3.3.10.4-x86-linux, listen-1.0.2, listen-0.7.3, listen-0.7.2, lumberjack-1.0.3, lumberjack-1.0.2, method_source-0.8.1, multi_json-1.7.2, multi_json-1.5.0, open4-1.3.0, pry-0.9.12.1, pry-0.9.12, pry-0.9.11.4, rack-1.5.2, rack-1.5.0, rack-1.3.5, rake-10.0.4, rake-10.0.3, rake-0.9.2.2, rb-fsevent-0.9.3, rb-inotify-0.9.0, rb-kqueue-0.2.0, ref-1.0.4, ref-1.0.2, rubygems-update-2.0.3, rubygems-update-1.8.25, rubyzip-0.9.9, sass-3.2.8, sass-3.2.7, sass-3.2.5, sassy-buttons-0.1.4, slick-buttons-0.0.6, slop-3.4.4, slop-3.4.3, sprockets-2.9.3, sprockets-2.9.2, sprockets-2.8.2, sprockets-2.0.3, sprockets-helpers-1.0.1, sprockets-helpers-1.0.0, sprockets-helpers-0.8.0, sprockets-sass-1.0.0, sprockets-sass-0.9.1, sprockets-sass-0.3.0, stitch-0.1.6, therubyracer-0.11.4, therubyracer-0.11.3, therubyracer-0.9.9, thor-0.18.1, thor-0.17.0, thor-0.15.4, tilt-1.3.7, tilt-1.3.3, yui-compressor-0.9.6] (Gem::LoadError)
from /usr/lib/ruby/1.9.1/rubygems/dependency.rb:256:in `to_spec'
from /usr/lib/ruby/1.9.1/rubygems.rb:1231:in `gem'
from /usr/local/bin/cap:22:in `<main>'
I'm a noob with Capistrano and Ruby
Any idea to resolve this problem ?
Thx
Probably what's going on is that you installed Capistrano as the root user. However, running capify from a non-root user account is trying to find Capistrano installed for that specific user rather than a system-wide install.
Can you run simply
gem install capistrano railsless-deploy capistrano-ext
As the user, without Sudo?
To answer your comment: It's generally not best practice to be installing and running cap deploy scripts as the root user. For your purposes maybe it will be OK, since you are doing a very limited task with Ruby/Cap; in the future, I would recommend looking into something like RVM.
RVM (Ruby Version Manager) is a tool you will be able to install system-wide, that then will manage your ruby version and gems per user.

Chef Deployment with irrelevant default symlinks

I am trying to deploy my application code with Chef, which is working for one node and failing on another. I cannot determine why it works for one node and not another when they have the exact same config, but I can at least try to debug the problem on the node that fails.
deploy_revision app_config['deploy_dir'] do
scm_provider Chef::Provider::Git
repo app_config['deploy_repo']
revision app_config['deploy_branch']
if secrets["deploy_key"]
git_ssh_wrapper "#{app_config['deploy_dir']}/git-ssh-wrapper" # For private Git repos
end
enable_submodules true
shallow_clone false
symlink_before_migrate({}) # Symlinks to add before running db migrations
purge_before_symlink [] # Directories to delete before adding symlinks
create_dirs_before_symlink [] # Directories to create before adding symlinks
# symlinks()
action :deploy
restart_command do
service "apache2" do action :restart; end
end
end
This is my recipe for deploying the code. Notice that I have tried disabling symlinking entirely, as Chef always jams its own defaults in. Even with this I get the error:
================================================================================
Error executing action `deploy` on resource 'deploy_revision[/var/www]'
================================================================================
Chef::Exceptions::FileNotFound
------------------------------
Cannot symlink /var/www/shared/config/database.yml to /var/www/releases/7404041cf8859a35de90ae72091bea1628391075/config/database.yml before migrate: No such file or directory - /var/www/shared/config/database.yml or /var/www/releases/7404041cf8859a35de90ae72091bea1628391075/config/database.yml
Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/kapture/recipes/api.rb
68:
69: deploy_revision app_config['deploy_dir'] do
70: scm_provider Chef::Provider::Git
71: repo app_config['deploy_repo']
72: revision app_config['deploy_branch']
73: if secrets["deploy_key"]
74: git_ssh_wrapper "#{app_config['deploy_dir']}/git-ssh-wrapper" # For private Git repos
75: end
76: enable_submodules true
Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/kapture/recipes/api.rb:69:in `from_file'
deploy_revision("/var/www") do
destination "/var/www/shared/cached-copy"
symlink_before_migrate {"config/database.yml"=>"config/database.yml"}
updated_by_last_action true
restart_command #<Proc:0x00007f40f366e5a0#/var/chef/cache/cookbooks/kapture/recipes/api.rb:82>
repository_cache "cached-copy"
retries 0
keep_releases 5
create_dirs_before_symlink ["tmp", "public", "config"]
updated true
provider Chef::Provider::Deploy::Revision
enable_submodules true
deploy_to "/var/www"
current_path "/var/www/current"
recipe_name "api"
revision "HEAD"
scm_provider Chef::Provider::Git
purge_before_symlink ["log", "tmp/pids", "public/system"]
git_ssh_wrapper "/var/www/git-ssh-wrapper"
remote "origin"
shared_path "/var/www/shared"
cookbook_name "kapture"
symlinks {"log"=>"log", "system"=>"public/system", "pids"=>"tmp/pids"}
action [:deploy]
repo "git#github.com:kapture/api.git"
retry_delay 2
end
[2012-09-24T15:42:07+00:00] ERROR: Running exception handlers
[2012-09-24T15:42:07+00:00] FATAL: Saving node information to /var/chef/cache/failed-run-data.json
[2012-09-24T15:42:07+00:00] ERROR: Exception handlers complete
[2012-09-24T15:42:07+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2012-09-24T15:42:07+00:00] FATAL: Chef::Exceptions::FileNotFound: deploy_revision[/var/www] (kapture::api line 69) had an error: Chef::Exceptions::FileNotFound: Cannot symlink /var/www/shared/config/database.yml to /var/www/releases/7404041cf8859a35de90ae72091bea1628391075/config/database.yml before migrate: No such file or directory - /var/www/shared/config/database.yml or /var/www/releases/7404041cf8859a35de90ae72091bea1628391075/config/database.yml
Here you can see it mention database.yml, tmp/, system/ and pids folders, all of which are defaults that Chef likes to set (see related bug)
Question 1
What are these symlinks for and how do I know if I even need any. What sort of things am I symlinking? I will be using migrations, so if they are useful for the migration then I'll need them working.
I have read the documentation many times and it just doesn't explain this is plain English - at least not that I have found.
Question 2
If I do not require them, how can I disable symlinking entirely? Following the examples in that bug report have not helped.
Clear out all the symlink attributes.
deploy_revision("/var/www") do
# ...
symlink_before_migrate.clear
create_dirs_before_symlink.clear
purge_before_symlink.clear
symlinks.clear
end
Make sure the deployment directory in shared has the proper directory structure (/var/www/shared/[log,pids,system,config]) and that all config files necessary for your application are in the config directory.
Your recipe for your application's cookbook should have an array of directory names to create (recursively) so that you won't run into this error again.
The symlinks are there so that while your application code will continue to evolve, you can share the log, pids, and system folder by symlinking shared/log to current/log and so on and so forth..
Chef happens to cache the directory structure - somehow, I haven't dug into that - with this troll application cookbook. It's something in the deploy resource I believe - I never use that - but you can fix it by deleting the directory structure it creates in /var/derp or whatever. Also ensure your tmp directory is setup.
A couple reasons this may be an issue:
File permissions are incorrect
The currently running chef user cannot access the file
You are using the application cookbook in a configuration for rails deploys and your application does not have the same directory structure.
It's definitely caused by Chef caching the state of the deploy somewhere, then reading that state out of it's cache - wherever that is - and then reusing that. I would look at either the application cookbook to see for any persistance, and if you fail at finding it there, look in the deploy resource from chef itself.