Dynamically bootstrapping Bare-Metal Nodes to Chef during Kickstart Post Script - centos

I'm working on some automation to build my bare-metal hosts. The automation goes through a kickstart script to install CentOS 7.3, wrapping up by running some post-kickstart install scripts. One of these scripts does the necessary work to install the Chef package, copy over the certs I need, generate the appropriate files, join the node the the org, place it in an appropriate environment and role and then finally run a chef-client.
Everything seems to be working up until I get to the point where I run my chef-client --once command, which is failing on its first run. When I manually login to the box after the fact and run a manual chef-client, I have no issue, runs just fine. The only thing in my run-list is chef-client::default and chef-client::config. Chef Version 19.33.36.
Here is the snippet of bash code where I am bootstrapping to my org and then running chef-client:
cd /opt/chef/bin/
chef-client -c /etc/chef/client.rb
echo "---------------------------------------" >> /tmp/build.log
echo "chef-client -c /etc/chef/client.rb" >> /tmp/build.log
echo "---------------------------------------" >> /tmp/build.log
(cd /etc/chef && /usr/bin/knife node environment set $HOST $CHEF_ENV)
echo "---------------------------------------" >> /tmp/build.log
echo "/usr/bin/knife node environment set $HOST $CHEF_ENV" >> /tmp/build.log
echo "---------------------------------------" >> /tmp/build.log
(cd /etc/chef && /usr/bin/knife node run_list set $HOST role[$CHEF_ROLE])
echo "---------------------------------------" >> /tmp/build.log
echo "/usr/bin/knife node run_list set $HOST role[$CHEF_ROLE]" >>
/tmp/build.log
echo "---------------------------------------" >> /tmp/build.log
CHEF_RUN_OUTPUT=`chef-client --once`
echo "---------------------------------------" >> /tmp/build.log
echo "chef-client --once" >> /tmp/build.log
echo "---------------------------------------" >> /tmp/build.log
echo $CHEF_RUN_OUTPUT >> /tmp/build.log
Here is the log of what is occurring after my post script finishes:
From /var/log/chef/client.log:
[2017-08-18T10:51:51-05:00] INFO: Processing directory[/var/run/chef] action create (chef-client::systemd_service line 56)
[2017-08-18T10:51:51-05:00] INFO: directory[/var/run/chef] created directory /var/run/chef
[2017-08-18T10:51:51-05:00] INFO: directory[/var/run/chef] owner changed to 0
[2017-08-18T10:51:51-05:00] INFO: directory[/var/run/chef] group changed to 0
[2017-08-18T10:51:51-05:00] INFO: Processing directory[/var/cache/chef] action create (chef-client::systemd_service line 56)
[2017-08-18T10:51:51-05:00] INFO: directory[/var/cache/chef] created directory /var/cache/chef
[2017-08-18T10:51:51-05:00] INFO: directory[/var/cache/chef] owner changed to 0
[2017-08-18T10:51:51-05:00] INFO: directory[/var/cache/chef] group changed to 0
[2017-08-18T10:51:51-05:00] INFO: Processing directory[/var/lib/chef] action create (chef-client::systemd_service line 56)
[2017-08-18T10:51:51-05:00] INFO: directory[/var/lib/chef] created directory /var/lib/chef
[2017-08-18T10:51:51-05:00] INFO: directory[/var/lib/chef] owner changed to 0
[2017-08-18T10:51:51-05:00] INFO: directory[/var/lib/chef] group changed to 0
[2017-08-18T10:51:51-05:00] INFO: Processing directory[/var/log/chef] action create (chef-client::systemd_service line 56)
[2017-08-18T10:51:51-05:00] INFO: Processing directory[/etc/chef] action create (chef-client::systemd_service line 56)
[2017-08-18T10:51:51-05:00] INFO: Processing template[/etc/sysconfig/chef-client] action create (chef-client::systemd_service line 27)
[2017-08-18T10:51:51-05:00] INFO: template[/etc/sysconfig/chef-client] created file /etc/sysconfig/chef-client
[2017-08-18T10:51:51-05:00] INFO: template[/etc/sysconfig/chef-client] updated file contents /etc/sysconfig/chef-client
[2017-08-18T10:51:51-05:00] INFO: template[/etc/sysconfig/chef-client] mode changed to 644
[2017-08-18T10:51:51-05:00] INFO: Processing systemd_unit[chef-client.service] action create (chef-client::systemd_service line 56)
[2017-08-18T10:51:51-05:00] INFO: Processing file[/etc/systemd/system/chef-client.service] action create (dynamically defined)
[2017-08-18T10:51:51-05:00] INFO: Running queued delayed notifications before re-raising exception
[2017-08-18T10:51:51-05:00] INFO: template[/etc/sysconfig/chef-client] sending restart action to service[chef-client] (delayed)
[2017-08-18T10:51:51-05:00] INFO: Processing service[chef-client] action restart (chef-client::systemd_service line 62)
[2017-08-18T10:51:51-05:00] INFO: service[chef-client] restarted
[2017-08-18T10:51:51-05:00] ERROR: Running exception handlers
[2017-08-18T10:51:51-05:00] ERROR: Exception handlers complete
[2017-08-18T10:51:51-05:00] INFO: Sending resource update report (run-id: 2a04bb13-bd0e-4fec-b6e0-093bd0bb3ab7)
[2017-08-18T10:51:51-05:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2017-08-18T10:51:51-05:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2017-08-18T10:51:51-05:00] ERROR: systemd_unit[chef-client.service] (chef-client::systemd_service line 56) had an error: Chef::Exceptions::ValidationFailed: file[/etc/systemd/system/chef-client.service] (dynamically defined) had an error: Chef::Exceptions::ValidationFailed: Proposed content for /etc/systemd/system/chef-client.service failed verification #<Chef::Resource::File::Verification:0x000000028b2200>
[2017-08-18T10:51:51-05:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
From the chef-stacktrace.out:
>>>> Caused by Chef::Exceptions::ValidationFailed: Proposed content for /etc/systemd/system/chef-client.service failed verification #<Chef::Resource::File::Verification:0x000000028b2200>
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/provider/file.rb:352:in `block in do_validate_content'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/provider/file.rb:350:in `each'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/provider/file.rb:350:in `do_validate_content'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/provider/file.rb:151:in `action_create'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/provider.rb:145:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/resource.rb:622:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/provider/systemd_unit.rb:198:in `manage_unit_file'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/provider/systemd_unit.rb:60:in `block in action_create'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/mixin/why_run.rb:52:in `add_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/provider.rb:176:in `converge_by'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/provider/systemd_unit.rb:59:in `action_create'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/provider.rb:145:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/resource.rb:622:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/runner.rb:69:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/runner.rb:97:in `block (2 levels) in converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/runner.rb:97:in `each'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/runner.rb:97:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/resource_collection/resource_list.rb:94:in `block in execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/resource_collection/stepable_iterator.rb:114:in `call_iterator_block'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/resource_collection/stepable_iterator.rb:103:in `iterate'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/resource_collection/resource_list.rb:92:in `execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/runner.rb:96:in `converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/client.rb:670:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/client.rb:665:in `catch'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/client.rb:665:in `converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/client.rb:704:in `converge_and_save'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/client.rb:284:in `run'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/application.rb:295:in `block in fork_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/application.rb:283:in `fork'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/application.rb:283:in `fork_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/application.rb:248:in `block in run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/local_mode.rb:44:in `with_server_connectivity'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/application.rb:236:in `run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/application/client.rb:464:in `sleep_then_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/application/client.rb:451:in `block in interval_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/application/client.rb:450:in `loop'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/application/client.rb:450:in `interval_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/application/client.rb:434:in `run_application'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/lib/chef/application.rb:59:in `run'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.33/bin/chef-client:26:in `<top (required)>'
/usr/bin/chef-client:57:in `load'
Adding in the chef-client -l debug level logs for further analysis:
[2017-08-23T11:11:19-05:00] DEBUG: Initiating POST to https://chef001/organizations/baremetal-sh-npe/reports/nodes/node/runs/61604df2-38ea-4d3d-9be3-3d93ef8187bc
[2017-08-23T11:11:19-05:00] DEBUG: ---- HTTP Request Header Data: ----
[2017-08-23T11:11:19-05:00] DEBUG: X-Ops-Reporting-Protocol-Version: 0.1.0
[2017-08-23T11:11:19-05:00] DEBUG: Content-Encoding: gzip
[2017-08-23T11:11:19-05:00] DEBUG: X-Ops-Server-API-Version: 1
[2017-08-23T11:11:19-05:00] DEBUG: X-OPS-SIGN: algorithm=sha1;version=1.1;
[2017-08-23T11:11:19-05:00] DEBUG: X-OPS-USERID: t9301bmh1001
[2017-08-23T11:11:19-05:00] DEBUG: X-OPS-TIMESTAMP: 2017-08-23T16:11:19Z
[2017-08-23T11:11:19-05:00] DEBUG: X-OPS-CONTENT-HASH: khFZ3GcjogWXzfxs5/c8VQOIK/0=
[2017-08-23T11:11:19-05:00] DEBUG: X-OPS-AUTHORIZATION-1: A3v+4H3vj4baFTprTLXLW9rEUP5qOSu95VC3oXJi8A0GUPqP8D1JK+Fn4C/k
[2017-08-23T11:11:19-05:00] DEBUG: X-OPS-AUTHORIZATION-2: Mksf+LkEIyV+RpNZ/qoRUFeKk90HIViK6+bU31GAkZLArr0aNJHEaBQTOmq7
[2017-08-23T11:11:19-05:00] DEBUG: X-OPS-AUTHORIZATION-3: zEWHWKHPDdw7coQ4VSi/uqQML20K2fSB5e5mRffnqdHu8cYafmzOEXM/dBcR
[2017-08-23T11:11:19-05:00] DEBUG: X-OPS-AUTHORIZATION-4: ozK/ATHfxqK/h3PpEdetfNDRsoLF3z2dsNeBepArtOkpVbi900UkANib1CFD
[2017-08-23T11:11:19-05:00] DEBUG: X-OPS-AUTHORIZATION-5: K4nMY8HbS6SLAjzwZR9+k6WojXwM5Ea5w5o5QKfAvLyAvQJjjOBRh1IIPC+m
[2017-08-23T11:11:19-05:00] DEBUG: X-OPS-AUTHORIZATION-6: mnggHfPZWY3Nat12n6DVR/dFz+xStrPe7P799ZHivA==
[2017-08-23T11:11:19-05:00] DEBUG: HOST: chef001.target.com:443
[2017-08-23T11:11:19-05:00] DEBUG: X-REMOTE-REQUEST-ID: 61604df2-38ea-4d3d-9be3-3d93ef8187bc
[2017-08-23T11:11:19-05:00] DEBUG: Content-Length: 2092
[2017-08-23T11:11:19-05:00] DEBUG: ---- End HTTP Request Header Data ----
[2017-08-23T11:11:19-05:00] DEBUG: ---- HTTP Status and Header Data: ----
[2017-08-23T11:11:19-05:00] DEBUG: HTTP 1.1 200 OK
[2017-08-23T11:11:19-05:00] DEBUG: server: openresty/1.11.2.1
[2017-08-23T11:11:19-05:00] DEBUG: date: Wed, 23 Aug 2017 16:11:19 GMT
[2017-08-23T11:11:19-05:00] DEBUG: content-type: application/json
[2017-08-23T11:11:19-05:00] DEBUG: content-length: 2
[2017-08-23T11:11:19-05:00] DEBUG: connection: close
[2017-08-23T11:11:19-05:00] DEBUG: ---- End HTTP Status/Header Data ----
[2017-08-23T11:11:19-05:00] DEBUG: Audit Reports are disabled. Skipping sending reports.
[2017-08-23T11:11:19-05:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2017-08-23T11:11:19-05:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2017-08-23T11:11:19-05:00] DEBUG: Chef::Exceptions::ValidationFailed: systemd_unit[chef-client.service] (chef-client::systemd_service line 56) had an error: Chef::Exceptions::ValidationFailed: file[/etc/systemd/system/chef-client.service] (dynamically defined) had an error: Chef::Exceptions::Valida tionFailed: Proposed content for /etc/systemd/system/chef-client.service failed verification #<Chef::Resource::File::Verification:0x000000063fa808>
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/file.rb:352:in `block in do_validate_content'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/file.rb:350:in `each'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/file.rb:350:in `do_validate_content'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/file.rb:151:in `action_create'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider.rb:145:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource.rb:622:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/systemd_unit.rb:198:in `manage_unit_file'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/systemd_unit.rb:60:in `block in action_create'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/mixin/why_run.rb:52:in `add_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider.rb:176:in `converge_by'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/systemd_unit.rb:59:in `action_create'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider.rb:145:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource.rb:622:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/runner.rb:69:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/runner.rb:97:in `block (2 levels) in converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/runner.rb:97:in `each'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/runner.rb:97:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource_collection/resource_list.rb:94:in `block in execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource_collection/stepable_iterator.rb:114:in `call_iterator_block'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource_collection/stepable_iterator.rb:103:in `iterate'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource_collection/resource_list.rb:92:in `execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/runner.rb:96:in `converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:670:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:665:in `catch'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:665:in `converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:704:in `converge_and_save'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:284:in `run'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:295:in `block in fork_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:283:in `fork'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:283:in `fork_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:248:in `block in run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/local_mode.rb:44:in `with_server_connectivity'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:236:in `run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:464:in `sleep_then_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:451:in `block in interval_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:450:in `loop'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:450:in `interval_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:434:in `run_application'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:59:in `run'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/bin/chef-client:26:in `<top (required)>'
/usr/bin/chef-client:57:in `load'
/usr/bin/chef-client:57:in `<main>'
>>>> Caused by Chef::Exceptions::ValidationFailed: Proposed content for /etc/systemd/system/chef-client.service failed verification #<Chef::Resource:: File::Verification:0x000000063fa808>
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/file.rb:352:in `block in do_validate_content'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/file.rb:350:in `each'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/file.rb:350:in `do_validate_content'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/file.rb:151:in `action_create'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider.rb:145:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource.rb:622:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/systemd_unit.rb:198:in `manage_unit_file'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/systemd_unit.rb:60:in `block in action_create'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/mixin/why_run.rb:52:in `add_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider.rb:176:in `converge_by'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider/systemd_unit.rb:59:in `action_create'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/provider.rb:145:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource.rb:622:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/runner.rb:69:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/runner.rb:97:in `block (2 levels) in converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/runner.rb:97:in `each'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/runner.rb:97:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource_collection/resource_list.rb:94:in `block in execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource_collection/stepable_iterator.rb:114:in `call_iterator_block'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource_collection/stepable_iterator.rb:103:in `iterate'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/resource_collection/resource_list.rb:92:in `execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/runner.rb:96:in `converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:670:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:665:in `catch'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:665:in `converge'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:704:in `converge_and_save'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:284:in `run'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:295:in `block in fork_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:283:in `fork'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:283:in `fork_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:248:in `block in run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/local_mode.rb:44:in `with_server_connectivity'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:236:in `run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:464:in `sleep_then_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:451:in `block in interval_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:450:in `loop'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:450:in `interval_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:434:in `run_application'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:59:in `run'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/bin/chef-client:26:in `<top (required)>'
/usr/bin/chef-client:57:in `load'
/usr/bin/chef-client:57:in `<main>'
[2017-08-23T11:11:19-05:00] ERROR: systemd_unit[chef-client.service] (chef-client::systemd_service line 56) had an error: Chef::Exceptions::Validation Failed: file[/etc/systemd/system/chef-client.service] (dynamically defined) had an error: Chef::Exceptions::ValidationFailed: Proposed content for /et c/systemd/system/chef-client.service failed verification #<Chef::Resource::File::Verification:0x000000063fa808>
[2017-08-23T11:11:20-05:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Any thoughts on why my first run is going south? My intention is to have the node placed in it's appropriate env and role, and then for the box to run all the appropriate recipes associated with the env/role.
Thanks!

You would need to use the full path, but just use knife bootstrap and it will take care of this.

Related

Unable to use snappy compression with fluentdv1.x

Unable to use snappy compression with fluentdv1.x, getting an exception when the producer pushes messages. In fluent configuration, "snappy" compression is enabled.
Using docker official image v1.15-1,
compression snappy is enabled on fluent configuration file on a topic.
Try pushing messages in that topic.
Expected behavior:
Wants to enable snappy compression on a Kafka topic.
Environment
Fluent version: 1.x docker image
TD Agent version: not sure, how to check.I'm using 1.x image
Operating system: alphine
Kernel version: not sure using official 1.15-1 docker image.
Configuration
Docker image:
FROM fluent/fluentd:v1.15-1
USER root
RUN apk add --no-cache --update --virtual .build-deps \
sudo build-base curl ruby ruby-dev autoconf automake libtool g++ snappy-dev
RUN fluent-gem install snappy
RUN fluent-gem install fluent-plugin-kafka
RUN fluent-gem install fluent-plugin-prometheus
RUN sudo gem sources --clear-all \
&& apk del .build-deps \
&& rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
USER root
Exception
/usr/lib/ruby/gems/3.1.0/gems/ruby-kafka-1.5.0/lib/kafka/snappy_codec.rb:16:in `rescue in load': Using snappy compression requires adding a dependency on the `snappy` gem to your Gemfile. (LoadError)
from /usr/lib/ruby/gems/3.1.0/gems/ruby-kafka-1.5.0/lib/kafka/snappy_codec.rb:13:in `load'
from /usr/lib/ruby/gems/3.1.0/gems/ruby-kafka-1.5.0/lib/kafka/compression.rb:30:in `find_codec'
from /usr/lib/ruby/gems/3.1.0/gems/ruby-kafka-1.5.0/lib/kafka/compressor.rb:28:in `initialize'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/kafka_producer_ext.rb:51:in `new'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/kafka_producer_ext.rb:51:in `custom_producer'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:210:in `create_producer'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:277:in `block in get_producer'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:274:in `synchronize'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:274:in `get_producer'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:323:in `write'
from /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin/output.rb:1180:in `try_flush'
from /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin/output.rb:1501:in `flush_thread_run'
from /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin/output.rb:501:in `block (2 levels) in start'
from /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
<internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require': Error loading shared library libsnappy.so.1: No such file or directory (needed by /usr/lib/ruby/gems/3.1.0/gems/snappy-0.3.0/lib/snappy_ext.so) - /usr/lib/ruby/gems/3.1.0/gems/snappy-0.3.0/lib/snappy_ext.so (LoadError)
from <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
from /usr/lib/ruby/gems/3.1.0/gems/snappy-0.3.0/lib/snappy.rb:5:in `<top (required)>'
from <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:160:in `require'
from <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:160:in `rescue in require'
from <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:149:in `require'
from /usr/lib/ruby/gems/3.1.0/gems/ruby-kafka-1.5.0/lib/kafka/snappy_codec.rb:14:in `load'
from /usr/lib/ruby/gems/3.1.0/gems/ruby-kafka-1.5.0/lib/kafka/compression.rb:30:in `find_codec'
from /usr/lib/ruby/gems/3.1.0/gems/ruby-kafka-1.5.0/lib/kafka/compressor.rb:28:in `initialize'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/kafka_producer_ext.rb:51:in `new'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/kafka_producer_ext.rb:51:in `custom_producer'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:210:in `create_producer'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:277:in `block in get_producer'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:274:in `synchronize'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:274:in `get_producer'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:323:in `write'
from /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin/output.rb:1180:in `try_flush'
from /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin/output.rb:1501:in `flush_thread_run'
from /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin/output.rb:501:in `block (2 levels) in start'
from /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
<internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- snappy (LoadError)
from <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
from /usr/lib/ruby/gems/3.1.0/gems/ruby-kafka-1.5.0/lib/kafka/snappy_codec.rb:14:in `load'
from /usr/lib/ruby/gems/3.1.0/gems/ruby-kafka-1.5.0/lib/kafka/compression.rb:30:in `find_codec'
from /usr/lib/ruby/gems/3.1.0/gems/ruby-kafka-1.5.0/lib/kafka/compressor.rb:28:in `initialize'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/kafka_producer_ext.rb:51:in `new'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/kafka_producer_ext.rb:51:in `custom_producer'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:210:in `create_producer'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:277:in `block in get_producer'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:274:in `synchronize'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:274:in `get_producer'
from /usr/lib/ruby/gems/3.1.0/gems/fluent-plugin-kafka-0.18.1/lib/fluent/plugin/out_kafka2.rb:323:in `write'
from /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin/output.rb:1180:in `try_flush'
from /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin/output.rb:1501:in `flush_thread_run'
from /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin/output.rb:501:in `block (2 levels) in start'
from /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'

Error executing action `install` on resource 'package[pgdg-centos93]'

Environment
Chef
CentOS
PostGres
Rails
Amazon cloud server
$ cat /etc/redhat-release
CentOS release 6.7 (Final)
$ uname -a
Linux ip-10-0-0-250 2.6.32-504.8.1.el6.x86_64 #1 SMP Wed Jan 28 21:11:36 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Put an Amazon CentOS instance to sleep for a couple of months and started it again today. To bring everything up-to-date, the "magic" should happen (figure of speech) with sudo chef-client, at least that was what was expected. And to my current experience for this particular , Chef has brought to my project almost more overhead than gain in time. (NOTE: This is a side comment related to a personal experience.)
Here is my latest headache
Recipe: postgresql::yum_pgdg_postgresql
* remote_file[/var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm] action create (up to date)
* package[pgdg-centos93] action install
================================================================================
Error executing action `install` on resource 'package[pgdg-centos93]'
================================================================================
Chef::Exceptions::Exec
----------------------
rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm returned 2, expected 0
Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/postgresql/recipes/yum_pgdg_postgresql.rb
43: package repo_rpm_package do
44: provider Chef::Provider::Package::Rpm
45: source "#{Chef::Config[:file_cache_path]}/#{repo_rpm_filename}"
46: action :install
47: end
Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/postgresql/recipes/yum_pgdg_postgresql.rb:43:in `from_file'
package("pgdg-centos93") do
provider Chef::Provider::Package::Rpm
action [:install]
retries 0
retry_delay 2
guard_interpreter :default
package_name "pgdg-centos93"
source "/var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm"
version "9.3-1"
cookbook_name "postgresql"
recipe_name "yum_pgdg_postgresql"
end
Running handlers:
[2016-02-02T12:34:02+00:00] ERROR: Running exception handlers
Running handlers complete
[2016-02-02T12:34:02+00:00] ERROR: Exception handlers complete
[2016-02-02T12:34:02+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
Chef Client failed. 8 resources updated in 42.119165396 seconds
[2016-02-02T12:34:03+00:00] ERROR: package[pgdg-centos93] (postgresql::yum_pgdg_postgresql line 43) had an error: Chef::Exceptions::Exec: rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm returned 2, expected 0
[2016-02-02T12:34:03+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Any pointers are most desperately welcomed. I just don't manage to understand the ERROR
UPDATE
As suggested
$ sudo rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm
[sudo] password for ed:
package pgdg-centos93-9.3-2.noarch (which is newer than pgdg-centos93-9.3-1.noarch) is already installed
Running Chef in debug mode
* package[pgdg-centos93] action install[2016-02-02T14:05:16+00:00] INFO: Processing package[pgdg-centos93] action install (postgresql::yum_pgdg_postgresql line 43)
[2016-02-02T14:05:16+00:00] DEBUG: package[pgdg-centos93] checking rpm status
[2016-02-02T14:05:17+00:00] DEBUG: package[pgdg-centos93] checking install state
[2016-02-02T14:05:17+00:00] DEBUG: package[pgdg-centos93] current version is 9.3-2
[2016-02-02T14:05:17+00:00] DEBUG: Executing rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm
[2016-02-02T14:05:17+00:00] DEBUG: ---- Begin output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----
[2016-02-02T14:05:17+00:00] DEBUG: STDOUT:
[2016-02-02T14:05:17+00:00] DEBUG: STDERR: package pgdg-centos93-9.3-2.noarch (which is newer than pgdg-centos93-9.3-1.noarch) is already installed
[2016-02-02T14:05:17+00:00] DEBUG: ---- End output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----
[2016-02-02T14:05:17+00:00] DEBUG: Ran rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm returned 2
================================================================================
Error executing action `install` on resource 'package[pgdg-centos93]'
================================================================================
Chef::Exceptions::Exec
----------------------
rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm returned 2, expected 0
---- Begin output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----
STDOUT: STDERR: package pgdg-centos93-9.3-2.noarch (which is newer than pgdg-centos93-9.3-1.noarch) is already installed
---- End output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----
Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/postgresql/recipes/yum_pgdg_postgresql.rb
43: package repo_rpm_package do
44: provider Chef::Provider::Package::Rpm
45: source "#{Chef::Config[:file_cache_path]}/#{repo_rpm_filename}"
46: action :install
47: end
Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/postgresql/recipes/yum_pgdg_postgresql.rb:43:in `from_file'
package("pgdg-centos93") do
provider Chef::Provider::Package::Rpm
action [:install]
retries 0
retry_delay 2
guard_interpreter :default
package_name "pgdg-centos93"
source "/var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm"
version "9.3-1"
cookbook_name "postgresql"
recipe_name "yum_pgdg_postgresql"
end
[2016-02-02T14:05:17+00:00] INFO: Running queued delayed notifications before re-raising exception
[2016-02-02T14:05:17+00:00] DEBUG: Re-raising exception: Chef::Exceptions::Exec - package[pgdg-centos93] (postgresql::yum_pgdg_postgresql line 43) had an error: Chef::Exceptions::Exec: rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm returned 2, expected 0
---- Begin output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----
STDOUT: STDERR: package pgdg-centos93-9.3-2.noarch (which is newer than pgdg-centos93-9.3-1.noarch) is already installed
---- End output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/command.rb:158:in `handle_command_failures'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/command.rb:104:in `run_command_and_return_stdout_stderr'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/command.rb:79:in `run_command'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/command.rb:171:in `run_command_with_systems_locale'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider/package/rpm.rb:98:in `install_package'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider/package.rb:82:in `block in action_install'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/why_run.rb:52:in `call'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/why_run.rb:52:in `add_action'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider.rb:155:in `converge_by'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider/package.rb:80:in `action_install'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider.rb:120:in `run_action'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource.rb:637:in `run_action'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:49:in `run_action'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:81:in `block (2 levels) in converge'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:81:in `each'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:81:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection.rb:98:in `block in execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection.rb:96:in `execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:80:in `converge'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:345:in `converge'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:431:in `do_run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:213:in `block in run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:207:in `fork'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:207:in `run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application.rb:217:in `run_chef_client'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application/client.rb:328:in `block in run_application'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application/client.rb:317:in `loop'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application/client.rb:317:in `run_application'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application.rb:67:in `run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/bin/chef-client:26:in `<top (required)>'
/usr/bin/chef-client:23:in `load'
/usr/bin/chef-client:23:in `<main>'
Running handlers:
[2016-02-02T14:05:17+00:00] ERROR: Running exception handlers
Running handlers complete
[2016-02-02T14:05:17+00:00] ERROR: Exception handlers complete
[2016-02-02T14:05:17+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2016-02-02T14:05:17+00:00] DEBUG: Chef::Exceptions::Exec: package[pgdg-centos93] (postgresql::yum_pgdg_postgresql line 43) had an error: Chef::Exceptions::Exec: rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm returned 2, expected 0
---- Begin output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----
STDOUT: STDERR: package pgdg-centos93-9.3-2.noarch (which is newer than pgdg-centos93-9.3-1.noarch) is already installed
---- End output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/command.rb:158:in `handle_command_failures'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/command.rb:104:in `run_command_and_return_stdout_stderr'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/command.rb:79:in `run_command'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/command.rb:171:in `run_command_with_systems_locale'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider/package/rpm.rb:98:in `install_package'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider/package.rb:82:in `block in action_install'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/why_run.rb:52:in `call'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/why_run.rb:52:in `add_action'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider.rb:155:in `converge_by'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider/package.rb:80:in `action_install'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider.rb:120:in `run_action'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource.rb:637:in `run_action'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:49:in `run_action'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:81:in `block (2 levels) in converge'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:81:in `each'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:81:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection.rb:98:in `block in execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection.rb:96:in `execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:80:in `converge'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:345:in `converge'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:431:in `do_run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:213:in `block in run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:207:in `fork'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:207:in `run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application.rb:217:in `run_chef_client'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application/client.rb:328:in `block in run_application'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application/client.rb:317:in `loop'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application/client.rb:317:in `run_application'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application.rb:67:in `run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/bin/chef-client:26:in `<top (required)>'
/usr/bin/chef-client:23:in `load'
/usr/bin/chef-client:23:in `<main>'
Chef Client failed. 8 resources updated in 42.825243439 seconds
[2016-02-02T14:05:17+00:00] INFO: Sending resource update report (run-id: ac041d25-eba7-4865-a20a-d16ffa60acef)
[2016-02-02T14:05:17+00:00] DEBUG: {"action"=>"end", "resources"=>[{"type"=>"ruby_block", "name"=>"run-iptables-resources-early", "id"=>"run-iptables-resources-early", "after"=>{}, "before"=>{}, "duration"=>"16183", "delta"=>"", "result"=>"run", "cookbook_name"=>"simple_iptables", "cookbook_version"=>"0.6.0"}, {"type"=>"package", "name"=>"pgdg-centos93", "id"=>"pgdg-centos93", "after"=>{:version=>"9.3-1", :options=>nil}, "before"=>{:version=>"9.3-2", :options=>nil}, "duration"=>"961", "delta"=>"", "result"=>"install", "cookbook_name"=>"postgresql", "cookbook_version"=>"3.4.0"}], "status"=>"failure", "run_list"=>"[\"role[website]\"]", "total_res_count"=>"27", "data"=>{"exception"=>{"class"=>"#<Chef::Exceptions::Exec: package[pgdg-centos93] (postgresql::yum_pgdg_postgresql line 43) had an error: Chef::Exceptions::Exec: rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm returned 2, expected 0\n---- Begin output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----\nSTDOUT: STDERR: \tpackage pgdg-centos93-9.3-2.noarch (which is newer than pgdg-centos93-9.3-1.noarch) is already installed\n---- End output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----\n>", "message"=>"package[pgdg-centos93] (postgresql::yum_pgdg_postgresql line 43) had an error: Chef::Exceptions::Exec: rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm returned 2, expected 0\n---- Begin output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----\nSTDOUT: STDERR: \tpackage pgdg-centos93-9.3-2.noarch (which is newer than pgdg-centos93-9.3-1.noarch) is already installed\n---- End output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----\n", "backtrace"=>"[\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/command.rb:158:in `handle_command_failures'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/command.rb:104:in `run_command_and_return_stdout_stderr'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/command.rb:79:in `run_command'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/command.rb:171:in `run_command_with_systems_locale'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider/package/rpm.rb:98:in `install_package'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider/package.rb:82:in `block in action_install'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/why_run.rb:52:in `call'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/mixin/why_run.rb:52:in `add_action'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider.rb:155:in `converge_by'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider/package.rb:80:in `action_install'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider.rb:120:in `run_action'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource.rb:637:in `run_action'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:49:in `run_action'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:81:in `block (2 levels) in converge'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:81:in `each'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:81:in `block in converge'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection.rb:98:in `block in execute_each_resource'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection.rb:96:in `execute_each_resource'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:80:in `converge'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:345:in `converge'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:431:in `do_run'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:213:in `block in run'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:207:in `fork'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:207:in `run'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application.rb:217:in `run_chef_client'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application/client.rb:328:in `block in run_application'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application/client.rb:317:in `loop'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application/client.rb:317:in `run_application'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application.rb:67:in `run'\",\"/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.4/bin/chef-client:26:in `<top (required)>'\",\"/usr/bin/chef-client:23:in `load'\",\"/usr/bin/chef-client:23:in `<main>'\"]", "description"=>{"title"=>"Error executing action `install` on resource 'package[pgdg-centos93]'", "sections"=>[{"Chef::Exceptions::Exec"=>"rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm returned 2, expected 0\n---- Begin output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----\nSTDOUT: STDERR: \tpackage pgdg-centos93-9.3-2.noarch (which is newer than pgdg-centos93-9.3-1.noarch) is already installed\n---- End output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----\n"}, {"Resource Declaration:"=>"# In /var/chef/cache/cookbooks/postgresql/recipes/yum_pgdg_postgresql.rb\n\n 43: package repo_rpm_package do\n 44: provider Chef::Provider::Package::Rpm\n 45: source \"\#{Chef::Config[:file_cache_path]}/\#{repo_rpm_filename}\"\n 46: action :install\n 47: end\n"}, {"Compiled Resource:"=>"# Declared in /var/chef/cache/cookbooks/postgresql/recipes/yum_pgdg_postgresql.rb:43:in `from_file'\n\npackage(\"pgdg-centos93\") do\n provider Chef::Provider::Package::Rpm\n action [:install]\n retries 0\n retry_delay 2\n guard_interpreter :default\n package_name \"pgdg-centos93\"\n source \"/var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm\"\n version \"9.3-1\"\n cookbook_name \"postgresql\"\n recipe_name \"yum_pgdg_postgresql\"\nend\n"}]}}}, "start_time"=>"2016-02-02 14:04:41 +0000", "end_time"=>"2016-02-02 14:05:17 +0000"}
[2016-02-02T14:05:17+00:00] DEBUG: Sending compressed run data...
[2016-02-02T14:05:17+00:00] DEBUG: Signing the request as ec2-production-web
[2016-02-02T14:05:17+00:00] DEBUG: String to sign: 'Method:POST
Hashed Path:ZNfsXWm0ScG9Y7tct4xU2t8Ruc4=
X-Ops-Content-Hash:JebmaJ0FR9SKdg1Whn6EXf9AjeY=
X-Ops-Timestamp:2016-02-02T14:05:17Z
X-Ops-UserId:ec2-production-web'
Header hash: {"X-Ops-Sign"=>"algorithm=sha1;version=1.0;", "X-Ops-Userid"=>"ec2-production-web", "X-Ops-Timestamp"=>"2016-02-02T14:05:17Z", "X-Ops-Content-Hash"=>"JebmaJ0FR9SKdg1Whn6EXf9AjeY=", "X-Ops-Authorization-1"=>"i25tlVgWl76JsqbNZFU9HKNBWbeIUUdOOIkInmsgw4cPI8zctpzNRZi9e5JF", "X-Ops-Authorization-2"=>"flygf7pnwtIH0UFpUJSukmHAklqGL2pQlMy3oRoSxQhzvYTW+njD4SznjEY/", "X-Ops-Authorization-3"=>"Cf0nkNUkf0mlch5oTiU9PD6md/irpT0IIv+fYCBS1HRoYpbiJVWuYqHchgOo", "X-Ops-Authorization-4"=>"Ir0DnFtZ4SFucPATkHWQMxx/c1tAY1sSoQZv02kceDJVPuMAarRI8gKzOQ6+", "X-Ops-Authorization-5"=>"Z6TBmJWXVqdRU4CAvjihr87hVadAYQQZURuQCEwsnxFUnxDZkrz1jDV+PnhQ", "X-Ops-Authorization-6"=>"uXPPOgdI5DuF45RvJfoFO8AvTkg+QCEwkqwCAY+e3w=="}
[2016-02-02T14:05:17+00:00] DEBUG: Initiating POST to https://api.opscode.com/organizations/[filtered]/reports/nodes/ec2-production-web/runs/ac041d25-eba7-4865-a20a-d16ffa60acef
[2016-02-02T14:05:17+00:00] DEBUG: ---- HTTP Request Header Data: ----
[2016-02-02T14:05:17+00:00] DEBUG: X-Ops-Reporting-Protocol-Version: 0.1.0
[2016-02-02T14:05:17+00:00] DEBUG: Content-Encoding: gzip
[2016-02-02T14:05:17+00:00] DEBUG: X-OPS-SIGN: algorithm=sha1;version=1.0;
[2016-02-02T14:05:17+00:00] DEBUG: X-OPS-USERID: ec2-production-web
[2016-02-02T14:05:17+00:00] DEBUG: X-OPS-TIMESTAMP: 2016-02-02T14:05:17Z
[2016-02-02T14:05:17+00:00] DEBUG: X-OPS-CONTENT-HASH: JebmaJ0FR9SKdg1Whn6EXf9AjeY=
[2016-02-02T14:05:17+00:00] DEBUG: X-OPS-AUTHORIZATION-1: i25tlVgWl76JsqbNZFU9HKNBWbeIUUdOOIkInmsgw4cPI8zctpzNRZi9e5JF
[2016-02-02T14:05:17+00:00] DEBUG: X-OPS-AUTHORIZATION-2: flygf7pnwtIH0UFpUJSukmHAklqGL2pQlMy3oRoSxQhzvYTW+njD4SznjEY/
[2016-02-02T14:05:17+00:00] DEBUG: X-OPS-AUTHORIZATION-3: Cf0nkNUkf0mlch5oTiU9PD6md/irpT0IIv+fYCBS1HRoYpbiJVWuYqHchgOo
[2016-02-02T14:05:17+00:00] DEBUG: X-OPS-AUTHORIZATION-4: Ir0DnFtZ4SFucPATkHWQMxx/c1tAY1sSoQZv02kceDJVPuMAarRI8gKzOQ6+
[2016-02-02T14:05:17+00:00] DEBUG: X-OPS-AUTHORIZATION-5: Z6TBmJWXVqdRU4CAvjihr87hVadAYQQZURuQCEwsnxFUnxDZkrz1jDV+PnhQ
[2016-02-02T14:05:17+00:00] DEBUG: X-OPS-AUTHORIZATION-6: uXPPOgdI5DuF45RvJfoFO8AvTkg+QCEwkqwCAY+e3w==
[2016-02-02T14:05:17+00:00] DEBUG: HOST: api.opscode.com:443
[2016-02-02T14:05:17+00:00] DEBUG: X-REMOTE-REQUEST-ID: ac041d25-eba7-4865-a20a-d16ffa60acef
[2016-02-02T14:05:17+00:00] DEBUG: Content-Length: 1391
[2016-02-02T14:05:17+00:00] DEBUG: ---- End HTTP Request Header Data ----
[2016-02-02T14:05:18+00:00] DEBUG: ---- HTTP Status and Header Data: ----
[2016-02-02T14:05:18+00:00] DEBUG: HTTP 1.1 200 OK
[2016-02-02T14:05:18+00:00] DEBUG: content-type: application/json
[2016-02-02T14:05:18+00:00] DEBUG: date: Tue, 02 Feb 2016 14:05:18 GMT
[2016-02-02T14:05:18+00:00] DEBUG: server: openresty/1.9.3.1
[2016-02-02T14:05:18+00:00] DEBUG: content-length: 2
[2016-02-02T14:05:18+00:00] DEBUG: connection: Close
[2016-02-02T14:05:18+00:00] DEBUG: ---- End HTTP Status/Header Data ----
[2016-02-02T14:05:18+00:00] ERROR: package[pgdg-centos93] (postgresql::yum_pgdg_postgresql line 43) had an error: Chef::Exceptions::Exec: rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm returned 2, expected 0
---- Begin output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----
STDOUT: STDERR: package pgdg-centos93-9.3-2.noarch (which is newer than pgdg-centos93-9.3-1.noarch) is already installed
---- End output of rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm ----
[2016-02-02T14:05:18+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
[ed#ip-10-0-0-250 ~]$
UPDATE 2
Removed package pgdg-centos93-9.3-2.noarch which was newer than the one Chef was trying to install
yum remove pgdg-centos93-9.3-2.noarch
then ran sudo chef-client -l debug again.
Ran into another blocking issue that once resolved takes me to the first problem I had. So I am going in circles. Will validate the one helpful answer because, I believe that there is no easy solution to my problem.
The error is that the command rpm -U /var/chef/cache/pgdg-centos93-9.3-1.noarch.rpm has an exit code of 2, which indicates there was an error. Either run that command by hand to see the error details or run chef-client -l debug to enable Chef's debug-level output which will include the stdout and stderr from the subprocess.
So your broader question: Chef is not magic nor is it a Platform-as-a-Service on its own. Chef itself offers fairly basic primitives like package installs and templated config files, and in general those just do what they say on the tin. When you start either building your own complex cookbooks or using the community cookbooks you can get much more turn-key magic like "add this to your run list and out pops a PostgreSQL server" but with that complexity comes a lot more bugs. Community cookbooks especially are of widely varying quality and you should always check if something is from an author you personally or institutionally trust and if not probably read all the code in it and make sure you like it. Same basic process as with any software development environment when you start consuming external tools/libraries/gems/packages/etc.

Capistrano complains there are too few arguments but it doesn't say what it's referring to

I've been bug solving all morning but I've run into something I'm unable to solve. Capistrano is throwing this message:
cap aborted!
ArgumentError: wrong number of arguments (0 for 1+)
/Users/alexander/.rvm/gems/ruby-2.2.4/gems/capistrano-3.4.0/lib/capistrano/dsl.rb:14:in `invoke'
/Users/alexander/.rvm/gems/ruby-2.2.4/gems/capistrano-3.4.0/lib/capistrano/dsl/task_enhancements.rb:14:in `block in after'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/task.rb:240:in `call'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/task.rb:240:in `block in execute'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/task.rb:235:in `each'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/task.rb:235:in `execute'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/task.rb:179:in `block in invoke_with_call_chain'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/monitor.rb:211:in `mon_synchronize'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/task.rb:172:in `invoke_with_call_chain'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/task.rb:165:in `invoke'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/application.rb:150:in `invoke_task'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/application.rb:106:in `block (2 levels) in top_level'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/application.rb:106:in `each'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/application.rb:106:in `block in top_level'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/application.rb:115:in `run_with_threads'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/application.rb:100:in `top_level'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/application.rb:78:in `block in run'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/application.rb:176:in `standard_exception_handling'
/Users/alexander/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/rake/application.rb:75:in `run'
/Users/alexander/.rvm/gems/ruby-2.2.4/gems/capistrano-3.4.0/lib/capistrano/application.rb:15:in `run'
/Users/alexander/.rvm/gems/ruby-2.2.4/gems/capistrano-3.4.0/bin/cap:3:in `<top (required)>'
/Users/alexander/.rvm/gems/ruby-2.2.2/bin/cap:23:in `load'
/Users/alexander/.rvm/gems/ruby-2.2.2/bin/cap:23:in `<main>'
/Users/alexander/.rvm/gems/ruby-2.2.2/bin/ruby_executable_hooks:15:in `eval'
/Users/alexander/.rvm/gems/ruby-2.2.2/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => deploy
The deploy has failed with an error: wrong number of arguments (0 for 1+)
** Invoke deploy:failed (first_time)
** Execute deploy:failed
Any idea what it's even referring to?
You can see all of the output of running cap production deploy here.
Deploy.rb, Production.rb, and Capfile are here.
I had both Capistrano 2 and 3 installed on my machine. Uninstalling both of those gems then installing only the version I needed allowed a successful deploy!

Peer authentication failed

At my VPS server, I do:
rake db:migrate RAILS_ENV=production
But, I get this error:
/usr/local/rvm/gems/ruby-2.2.1/gems/fog-core-1.24.0/lib/fog/core/collection.rb:148: warning: circular argument reference - filters
/usr/local/rvm/gems/ruby-2.2.1/gems/fog-1.23.0/lib/fog/rackspace/mock_data.rb:42: warning: duplicated key at line 80 ignored: "name"
/usr/local/rvm/gems/ruby-2.2.1/gems/simple_form-3.0.2/lib/simple_form/inputs/boolean_input.rb:38: warning: circular argument reference - unchecked_value
rake aborted!
PG::ConnectionBad: FATAL: Peer authentication failed for user "myuser"
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:888:in `initialize'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:888:in `new'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:888:in `connect'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:568:in `initialize'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:41:in `new'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:41:in `postgresql_connection'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:435:in `new_connection'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:445:in `checkout_new_connection'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:416:in `acquire_connection'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:351:in `block in checkout'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:350:in `checkout'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:265:in `block in connection'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:264:in `connection'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:541:in `retrieve_connection'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_handling.rb:113:in `retrieve_connection'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/connection_handling.rb:87:in `connection'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/migration.rb:909:in `initialize'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/migration.rb:807:in `new'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/migration.rb:807:in `up'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/migration.rb:785:in `migrate'
/usr/local/rvm/gems/ruby-2.2.1/gems/activerecord-4.1.6/lib/active_record/railties/databases.rake:34:in `block (2 levels) in <top (required)>'
Tasks: TOP => db:migrate
database.yml:
production:
adapter: postgresql
encoding: unicode
database: mydatabase
pool: 5
username: myuser
password: mypassword
What possible reason would this be? although myuser is the owner of mydatabase
How solve this?

Section 5.4 Rails Tutorial

I'm tearing my hair out. I'm doing the Rails Tutorial by Hartl, and everything has been working until section 5.4.
It seems to be that I generated a new controller, then generated the integration_test user_pages, then the whole thing went haywire. I can't for the life of me work out what I've done wrong or where to start. I even tried destroying both and the same error comes out. Please help!
My Code
Jordan-Poulton:Projects academy$ cd first_app/
Jordan-Poulton:first_app academy$ subl .
Jordan-Poulton:first_app academy$ rails generate controller Users new --no-test-framework
/Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/bundler-1.2.4/lib/bundler/runtime.rb:197: warning: Insecure world writable dir /usr in PATH, mode 040777
/Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/bundler-1.2.4/lib/bundler/runtime.rb:197: warning: Insecure world writable dir /usr in PATH, mode 040777
create app/controllers/users_controller.rb
route get "users/new"
invoke erb
create app/views/users
create app/views/users/new.html.erb
invoke helper
create app/helpers/users_helper.rb
invoke assets
invoke coffee
identical app/assets/javascripts/users.js.coffee
invoke scss
create app/assets/stylesheets/users.css.scss
Jordan-Poulton:first_app academy$ rails generate integration_test user_pages
/Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/bundler-1.2.4/lib/bundler/runtime.rb:197: warning: Insecure world writable dir /usr in PATH, mode 040777
/Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/bundler-1.2.4/lib/bundler/runtime.rb:197: warning: Insecure world writable dir /usr in PATH, mode 040777
invoke rspec
create spec/requests/user_pages_spec.rb
Jordan-Poulton:first_app academy$ rails s
/Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/bundler-1.2.4/lib/bundler/runtime.rb:197: warning: Insecure world writable dir /usr in PATH, mode 040777
/Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/bundler-1.2.4/lib/bundler/runtime.rb:197: warning: Insecure world writable dir /usr in PATH, mode 040777
=> Booting WEBrick
=> Rails 3.2.12 application starting in development on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
Exiting
/Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/actionpack-3.2.12/lib/action_dispatch/routing/mapper.rb:185:in `default_controller_and_action': missing :action (ArgumentError)
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/actionpack-3.2.12/lib/action_dispatch/routing/mapper.rb:78:in `normalize_options!'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/actionpack-3.2.12/lib/action_dispatch/routing/mapper.rb:61:in `initialize'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/actionpack-3.2.12/lib/action_dispatch/routing/mapper.rb:1304:in `new'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/actionpack-3.2.12/lib/action_dispatch/routing/mapper.rb:1304:in `add_route'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/actionpack-3.2.12/lib/action_dispatch/routing/mapper.rb:1284:in `decomposed_match'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/actionpack-3.2.12/lib/action_dispatch/routing/mapper.rb:1270:in `block in match'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/actionpack-3.2.12/lib/action_dispatch/routing/mapper.rb:1270:in `each'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/actionpack-3.2.12/lib/action_dispatch/routing/mapper.rb:1270:in `match'
from /Users/academy/Projects/first_app/config/routes.rb:6:in `block in <top (required)>'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:282:in `instance_exec'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:282:in `eval_block'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:260:in `draw'
from /Users/academy/Projects/first_app/config/routes.rb:1:in `<top (required)>'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/activesupport-3.2.12/lib/active_support/dependencies.rb:245:in `load'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/activesupport-3.2.12/lib/active_support/dependencies.rb:245:in `block in load'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/activesupport-3.2.12/lib/active_support/dependencies.rb:236:in `load_dependency'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/activesupport-3.2.12/lib/active_support/dependencies.rb:245:in `load'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/application/routes_reloader.rb:40:in `block in load_paths'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/application/routes_reloader.rb:40:in `each'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/application/routes_reloader.rb:40:in `load_paths'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/application/routes_reloader.rb:16:in `reload!'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/application/routes_reloader.rb:26:in `block in updater'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/activesupport-3.2.12/lib/active_support/file_update_checker.rb:78:in `call'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/activesupport-3.2.12/lib/active_support/file_update_checker.rb:78:in `execute'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/application/routes_reloader.rb:27:in `updater'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/application/routes_reloader.rb:7:in `execute_if_updated'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/application/finisher.rb:66:in `block in <module:Finisher>'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/initializable.rb:30:in `instance_exec'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/initializable.rb:30:in `run'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/initializable.rb:55:in `block in run_initializers'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/initializable.rb:54:in `each'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/initializable.rb:54:in `run_initializers'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/application.rb:136:in `initialize!'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/railtie/configurable.rb:30:in `method_missing'
from /Users/academy/Projects/first_app/config/environment.rb:5:in `<top (required)>'
from /Users/academy/Projects/first_app/config.ru:3:in `require'
from /Users/academy/Projects/first_app/config.ru:3:in `block in <main>'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/rack-1.4.5/lib/rack/builder.rb:51:in `instance_eval'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/rack-1.4.5/lib/rack/builder.rb:51:in `initialize'
from /Users/academy/Projects/first_app/config.ru:in `new'
from /Users/academy/Projects/first_app/config.ru:in `<main>'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/rack-1.4.5/lib/rack/builder.rb:40:in `eval'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/rack-1.4.5/lib/rack/builder.rb:40:in `parse_file'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/rack-1.4.5/lib/rack/server.rb:200:in `app'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/commands/server.rb:46:in `app'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/rack-1.4.5/lib/rack/server.rb:304:in `wrapped_app'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/rack-1.4.5/lib/rack/server.rb:254:in `start'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/commands/server.rb:70:in `start'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/commands.rb:55:in `block in <top (required)>'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/commands.rb:50:in `tap'
from /Users/academy/.rvm/gems/ruby-1.9.3-p385/gems/railties-3.2.12/lib/rails/commands.rb:50:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'
Please check your controller. Content should be this
#app/controllers/users_controller.rb
class UsersController < ApplicationController
def new
end
end
and your routes.rb
#config/routes.rb
FirstApp::Application.routes.draw do
get "users/new"
root to: 'pages#home'
Run the following commands before rails s, just in case:
bundle install
bundle update
rake db:migrate
Sometimes a rake db:reset could help, but this will destroy and rebuild your database in your development environment.