Elastic Beanstalk application deployment fails from EBExtension failing to install - postgresql

We're using Elastic Beanstalk, (Postgres, Node.js running on 64bit Amazon Linux/3.2.0) and I woke up today to a Severe Health warning, causing all requests to respond with a 502 Bad Gateway. I haven't manually deployed since 4/9/19, so not sure why this happened all of a sudden.
The original error we got was:
Application deployment failed at 2019-04-18T15:39:51Z with exit status 1 and error: Package listed in EBExtension failed to install.
Yum does not have postgresql96-devel available for installation.
The repo I inherited is a little untidy, and I found instance of postgres96-devel in three different files:
.ebextensions/config.yml
.ebextensions/proxy.config
proxy.config
My config.yml file looks like:
packages:
rpm:
postgresql: https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-6-x86_64/pgdg-ami201503-96-9.6-2.noarch.rpm
yum:
postgresql96-devel: []
perl-CPAN: []
I noticed the rpm link returns a 404, and when looking for a better url, I saw this warning on the Postgres RPM page:
As of 15 April 2019, there is only one repository RPM per distro, and
it includes repository information for all available PostgreSQL
releases
What I've tried:
Redeploying the last successful build from 4/9/19
Changing the config.yml file to look like
// obviously I'm thrashing here
packages:
rpm:
postgresql: https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-6-x86_64/pgdg-redhat-repo-latest.noarch.rpm
yum:
postgresql11-devel: []
perl-CPAN: []
According to this AWS support article, I terminated the instance and let EB bring up a new instance.
With all these trials + redeploying, I still seeing errors like:
Application deployment failed at 2019-04-18T17:40:41Z with exit status 1 and error: Package listed in EBExtension failed to install.
Yum does not have postgresql96-devel available for installation.
Incorrect application version "app-v1_4_1-190418_084747" (deployment
98). Expected version "app-v1_4_1-190409_140626" (deployment 104).
Process default has been unhealthy for 42 minutes
(Target.FailedHealthChecks).
I'm not sure why it's complaining about postgres96-devel since I changed my config file to point to postgres11-devel.
Any ideas how to get things back up and running?

I was able to get everything back up and running. Here's what I think happened.
Our prod instances were running Linux 3.2. This did not contain the correct rpm package, so it relied on the rpm link from config.yml.
That url broke as of 4/15/19, so when EB went to deploy and pull that RPM, it failed, causing the entire deployment to fail.
The fix was to simply downgrade the yum package from postresql96-devel to postgres95-devel. Linux 3.2's yum directory contained postgres95-devel, so the deployment was able to skip going out to the internet to download the rpm (which at this point was broken).

You can install the PostgreSQL 9.6 using the amazon-linux-extra tool:
(if using docker, in a Dockerfile: )
RUN amazon-linux-extras install postgresql9.6

Related

Cannot install pglogical

I cannot install pglogical. I'm fallowing this tutorial: https://docs.aws.amazon.com/dms/latest/sbs/chap-manageddatabases.postgresql-rds-postgresql-full-load-pglogical.html#chap-manageddatabases.postgresql-rds-postgresql-full-load-pglogical-replication
And part about pglogical installation has two steps (I'm using postgres 10.4):
curl https://access.2ndquadrant.com/api/repository/dl/default/release/10/rpm | bash
yum install postgresql10-pglogical
I'm getting an error though:
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
https://dl.2ndquadrant.com/default/release/rpm/packages/amzn/2/x86_64/10/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found
Trying other mirror.
One of the configured repositories failed (2ndQuadrant packages (PG10) for 2 - x86_64),
and yum doesn't have enough cached data to continue. At this point the only
safe thing yum can do is fail. There are a few ways to work "fix" this:
1. Contact the upstream for the repository and get them to fix the problem.
2. Reconfigure the baseurl/etc. for the repository, to point to a working
upstream. This is most often useful if you are using a newer
distribution release than is supported by the repository (and the
packages for the previous distribution release still work).
3. Run the command with the repository temporarily disabled
yum --disablerepo=2ndquadrant-dl-default-release-pg10 ...
4. Disable the repository permanently, so yum won't use it by default. Yum
will then just ignore the repository until you permanently enable it
again or use --enablerepo for temporary usage:
yum-config-manager --disable 2ndquadrant-dl-default-release-pg10
or
subscription-manager repos --disable=2ndquadrant-dl-default-release-pg10
5. Configure the failing repository to be skipped, if it is unavailable.
Note that yum will try to contact the repo. when it runs most commands,
so will have to try and fail each time (and thus. yum will be be much
slower). If it is a very temporary problem though, this is often a nice
compromise:
yum-config-manager --save --setopt=2ndquadrant-dl-default-release-pg10.skip_if_unavailable=true
failure: repodata/repomd.xml from 2ndquadrant-dl-default-release-pg10: [Errno 256] No more mirrors to try.
https://dl.2ndquadrant.com/default/release/rpm/packages/amzn/2/x86_64/10/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found
How can I install it?

error: SERVER does not appear in AM_CONDITIONAL

I am trying to install Lustre on CentOS 7. I followed this link. When I try to run sh ./autogen.sh to generate the configure script I get the above error as illustrated below.
[root#localhost lustre-release]# sh ./autogen.sh
configure.ac:10: installing 'config/config.guess'
configure.ac:10: installing 'config/config.sub'
configure.ac:12: installing 'config/install-sh'
configure.ac:12: installing 'config/missing'
autoMakefile.am:127: error: SERVER does not appear in AM_CONDITIONAL
libcfs/libcfs/autoMakefile.am: installing 'config/depcomp'
Does anyone know how I can resolve this?
To build lustre from git use "autogen.sh" in the top level directory to setup the build environment:
$ bash autogen.sh
libcfs/libcfs/autoMakefile.am: installing 'config/depcomp'
$
Did you try downloading a pre-built package from https://downloads.whamcloud.com/public/lustre/ ? That is usually the easiest compared to building your own.
If you want to build your own Lustre code, which version of the source are you using? The latest code is available at git.whamcloud.com. Most users should use the LTS release (b2_10 branch currently), but if you are doing new development you should use the master branch.

centos yum fails installing anything or updating system

I am trying to install a package on CentOS, but it throws error when I run the "yum" command. The internet connection is working fine.
I try to yum clean all but problem persist.
Error:
[root#dcos-master3 ~]# yum install ntp
Loaded plugins: fastestmirror
One of the configured repositories failed (Unknown),
and yum doesn't have enough cached data to continue. At this point the only
safe thing yum can do is fail. There are a few ways to work "fix" this:
1. Contact the upstream for the repository and get them to fix the problem.
2. Reconfigure the baseurl/etc. for the repository, to point to a working
upstream. This is most often useful if you are using a newer
distribution release than is supported by the repository (and the
packages for the previous distribution release still work).
3. Run the command with the repository temporarily disabled
yum --disablerepo=<repoid> ...
4. Disable the repository permanently, so yum won't use it by default. Yum
will then just ignore the repository until you permanently enable it
again or use --enablerepo for temporary usage:
yum-config-manager --disable <repoid>
or
subscription-manager repos --disable=<repoid>
5. Configure the failing repository to be skipped, if it is unavailable.
Note that yum will try to contact the repo. when it runs most commands,
so will have to try and fail each time (and thus. yum will be be much
slower). If it is a very temporary problem though, this is often a nice
compromise:
yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
Cannot find a valid baseurl for repo: base/$releasever/x86_64
My yum repolist is the next:
[root#dcos-master3 ~]# yum repolist list
Loaded plugins: fastestmirror
https://yum.dockerproject.org/repo/main/centos/%24releasever/repodata/repomd.xml: [Errno 14] HTTPS Error 403 - Forbidden
Trying other mirror.
To address this issue please refer to the below knowledge base article
https://access.redhat.com/solutions/69319
If above article doesn't help to resolve this issue please create a bug on https://bugs.centos.org/
repolist: 0
If I list the repolist:
[root#dcos-master3 ~]# yum repolist
Loaded plugins: fastestmirror
https://yum.dockerproject.org/repo/main/centos/%24releasever/repodata/repomd.xml: [Errno 14] HTTPS Error 403 - Forbidden
Trying other mirror.
To address this issue please refer to the below knowledge base article
https://access.redhat.com/solutions/69319
If above article doesn't help to resolve this issue please create a bug on https://bugs.centos.org/
repo id repo name status
base/$releasever/x86_64 CentOS-$releasever - Base 0
dockerrepo/$releasever Docker Repository 0
extras/$releasever/x86_64 CentOS-$releasever - Extras 0
updates/$releasever/x86_64 CentOS-$releasever - Updates 0
repolist: 0
BEFORE TRYING ANY OF THIS, HAVE A BACKUP OF YOUR MACHINE, YOU COULD DAMAGE YOUR OS MORE/COMPLETELY
It seems that your yum variable $releasever is somehow corrupt,
it usually is caused by missing centos-release package on the machine for some obscure reasons.
You can check if you have the package by:
rpm -qi centos-release
You will probably see:
"package centos-release is not installed"
First find out the exact centos version that you have by executing as root:
cat /etc/redhat-release
You will see something like this:
CentOS Linux release 7.3.1611 (Core)
You can fetch the centos-release package from repo by:
wget http://vault.centos.org/centos/7.3.1611/updates/x86_64/Packages/centos-release-7-3.1611.el7.centos.2.5.x86_64.rpm
Now run reinstall centos-release package via rpm:
sudo rpm -Uvh --replacepkgs centos-release-7-3.1611.el7.centos.2.5.x86_64.rpm
As next you can try to install something with yum and you might get:
[root#dcos-master3 ikerlan]# sudo yum install wget
error: db5 error(-30969) from dbenv->open: BDB0091 DB_VERSION_MISMATCH: Database environment version mismatch
error: cannot open Packages index using db5 - (-30969)
error: cannot open Packages database in /var/lib/rpm
CRITICAL:yum.main:
Error: rpmdb open failed
Now you can try to reboot the machine or try to use the following command to rebuild the rpm db:
rpm --rebuilddb
I came across the same issue while downloading some packages in centos 7. After days of search, I found the solution:
Go to the yum repo directory.
cd /etc/yum.repos.d.
Make a copy of the CentOS-Base.repo file.
cp CentOS-Base.repo CentOS-Base.repo.old
Edit the CentOS-Base.repo file. Comment mirrorlist and uncomment baseurl.
vi CentOS-Base.repo
[base]
name=CentOS-$releasever - Base
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra
baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Now when you use yum, do the following.
sudo yum --disablerepo=* --enablerepo=base install httpd
Likewise for yum update.
sudo yum --disablerepo=* --enablerepo=base install httpd
It should work now.
I ran into this problem when attempting to install MariaDB on CentOS 7. I was super frustrated and after much searching found the answer at this link.
Here is what fixed this problem for me. Run as root.
# yum --disablerepo "*" --enablerepo epel install [package]
# yum clean all
"epel" can be whatever repo you like, but this one worked for me. Place [package] in the command just as written, not what package you are trying to install.
After you run the above, exit root and run whatever install you were attempting before encountering the error.
I noticed that in the failing url, your $releasever is %24releasever, but it should be 7 or 7.14.xx. Please check your yum config file at /etc/yum/var, or search in every .repo file, to see what is its value. It may be corrupted.
See Red Hat documentation for more information about how to set these variables.
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/deployment_guide/sec-using_yum_variables
yum install http://rpms.remirepo.net/enterprise/remi-release-7.rpm -y
yum -y install yum-utils
yum-config-manager --enable remi-php74
Then proceed to your installation/updates/etc
Always make sure NAT is active on your centos7. Specially when there is a VMnet2 for host. Because sometimes VMNet2 can be active instead of NAT and because of that you will not be able to connect to the internet through centos7. This is just a one reason for getting that error.

Cannot retrieve repository metadata for repository: epel (rhel 6.6)

Im trying to install linux nrpe agent on one of the test rhel boxes for nagiosxi to monitor and when i run the fullinstall script im getting this following error
[root#nrpetest2 linux-nrpe-agent]# ./fullinstall
============================
Nagios Linux Agent Installer
============================
This script will install the Nagios Linux Agent by executing all necessary
sub-scripts.
IMPORTANT: This script should only be used on a clean installed system:
RedHat Enterprise, CentOS, Fedora, or Oracle
OpenSUSE or SUSE Enterprise
Ubuntu or Debian
Do NOT use this on a system running any other distro or that
does not allow additional package installation.
Do you want to continue? [Y/n] y
Proceeding with installation...
Running './0-repos'...
Configuring Repos...
Enabling EPEL repo...
Installing EPEL PKG: ./packages/epel-release-6-5.noarch.rpm
warning: ./packages/epel-release-6-5.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Preparing... ##################################################
epel-release ##################################################
Loaded plugins: product-id, rhnplugin, security, subscription-manager
This system is receiving updates from RHN Classic or RHN Satellite.
Error: Cannot retrieve repository metadata (repomd.xml) for repository: epel. Please verify its path and try again
epel-release RPM installed OK
Repos configured OK
RESULT=0
Running './1-prereqs'...
Installing prerequisites...
Loaded plugins: product-id, rhnplugin, security, subscription-manager
This system is receiving updates from RHN Classic or RHN Satellite.
Setting up Install Process
Error: Cannot retrieve repository metadata (repomd.xml) for repository: epel. Please verify its path and try again
RESULT=1
===================
INSTALLATION ERROR!
===================
Installation step failed - exiting.
Check for error messages in the install log (install.log).
If you require assistance in resolving the issue, please include install.log
in your communications with Nagios XI technical support.
The script that failed was: './1-prereqs'
I have it registered with RedHat network as well and i searched everywhere to find a solution to this error but had no luck.
It looks like you are missing the required epel-release for nagios to install.
Have your tried doing a sudo yum install epel-release?
That would be my biggest guess, according to your line:
Error: Cannot retrieve repository metadata (repomd.xml) for repository: epel. Please verify its path and try again RESULT=1The script that failed was: './1-prereqs'
Also, check to make sure that your proxy settings allow for a connection to that repository. Sometimes this causes a problem with downloading certain repo data.
So, in short, update your epel-release and make sure that any other pre-requisities have been updated. It probably wouldn't hurt to do: sudo yum install yum-utils to make sure that yum-utils is fully up-to-date.
UPDATE: From: Access Redhat
Verify that Satellite or Proxy Server has a fully-qualified domain name, and CommonName in SSL cert. used by Apache is set to the fully-qualified domain name.
Verify that the ssl.conf file is pointing to the correct files.grep ^SSLCert /etc/httpd/conf.d/ssl.conf
If system points to a proxy, check to make sure that /etc/sysconfig/rhn/up2date is configured with correct http_proxy information. httpProxy=http_proxy_fqdn or ip_address:port
Update yum to latest version: yum update yum
Clean the cache on client system: rm -rf /var/cache/yum/*

Running orion context broker

Following the tutorial about installing Orion, I installed contextBroker-0.5.0-1.x86_64.rpm and contextBroker-tests-0.5.0-1.x86_64.rpm.
After all, I decided to run Orion, but when I do using # contextBroker I get this notification
"contextBroker: error while loading shared libraries: libmicrohttpd.so.10: cannot open shared object file: No such file or directory"
I've read on the Internet that "libmicrohttpd.so.10" belongs to contextBroker-0.5.0-1.x86_64.rpm, so I don't know what the problem is.
Version 0.5.0 is very old. I would recommend you to install the newest one that, at time of this writting, is 0.14.0.
You can install it in the following way:
Remove the old version (rpm -e contextBroker)
Configure FI-WARE yum respository (explained here)
Install contextBroker running: yum install contextBroker
Ensure you have the newest version: contextBroker --version