How do I recompile HAProxy with ZLIB support? - centos

We are running a high-traffic, load-balanced site on CentOS. When I installed haproxy, I used:
make TARGET=linux26 USE_OPENSSL=1 ADDLIB=-lz
make PREFIX=/usr/local/haproxy install
but now I need to add zlib support.
I know that the command for a fresh install would be:
make TARGET=linux26 USE_OPENSSL=1 USE_ZLIB=1 ADDLIB=-lz
make PREFIX=/usr/local/haproxy install
But how do I recompile it into an existing haproxy install without uninstalling first? The site is too high traffic to take it down for even a minute.
I spent 30 minutes Googling for the answer and while I found something that talks about using make clean to do a recompilation, as somewhat of a Linux noob, I thought I should ask the experts how it's supposed to be done and for the exact syntax.

make clean will indeed clean your compilation directory.
My advice would be to:
clean the compilation directory, NOT the install dir
recompile with zlib support
install in another dir
change the currently installed non zlib haproxy to another path
which translates in bash to
make clean
make TARGET=linux26 USE_OPENSSL=1 USE_ZLIB=1 ADDLIB=-lz
make PREFIX=/usr/local/haproxy-zlibed install
mv /usr/local/haproxy /usr/local/haproxy-not-zlibed
ln -s /usr/local/haproxy-not-zlibed /usr/local/haproxy
At this point you're in the exact same situation as you were before.
then use symbolic links to switch from your current haproxy to the other:
use the current haproxy (without zlib)
rm -fr /usr/local/haproxy
ln -s /usr/local/haproxy-not-zlibed /usr/local/haproxy
and restart haproxy your usual way
or, to use the haproxy with zlib support
rm -fr /usr/local/haproxy
ln -s /usr/local/haproxy-zlibed /usr/local/haproxy
and restart haproxy your usual way
That way you can test your new zlibd haproxy and rollback if necessary

On Linux there's no need to uninstall or even stop a service before you recompile and reinstall.
That's true because of how modern (and even not-so-modern) filesystems work: File contents are attached to inodes and inodes are attached to directory entries (having a 1:0..n relationship). Thus, you can delete the directory entry for a running program, but as long as its inode isn't deallocated (which will never happen so long as it continues running), it still has a file handle on its own executable, and can continue to work.
Now, with HAProxy in particular, there's support for seamless restarts -- where a new process starts up, tells the old process to drop its listen sockets but keep servicing the existing connection, grabs new listen sockets itself, tells the old process that this succeeded (or if it failed, in which case the old process regrabs its own listen sockets), and then allows the old process to shut down when done. See http://www.mgoff.in/2010/04/18/haproxy-reloading-your-config-with-minimal-service-impact/ for a writeup on the process.

Related

Install snap() on wsl2 for flutter [duplicate]

I am attempting to debug some C# / .NET 5 code in WSL 2 with Ubuntu on Windows. I have WSL 2 setup with Windows 10 and want to test out creating a Systemd service. Unfortunately, it appears Systemd is not enabled with WSL 2 by default, even though a standard Ubuntu install does have it enabled by default. Is there any way to get Systemd enabled in WSL 2?
Note: See footnote at bottom of this answer for background on this Community Wiki.
There are several possible paths to enabling Systemd on WSL2 (but not WSL1). These are summarized here, with more detail provided below.
Option 1: Upgrade WSL to the latest application release (if supported by your system) and opt-in to the Systemd feature
Option 2: Run a Systemd-helper script designed for WSL2
Option 3: Manually run Systemd in its own namespace
And while not part of this question, for those simply looking to run certain applications that require Systemd, there are alternatives:
On WSL1 and WSL2:
Alternative 1: SysVInit scripts (e.g. sudo service <service_name> start) where available
Alternative 2: Manually configuring and running the service
On WSL2-only:
Alternative 3: Docker
Should you enable Systemd in WSL?
First, consider whether you should or need to enable Systemd in WSL. Enabling Systemd will automatically start a number of background services and tasks that you really may not need under WSL. As a result, it will also increase WSL startup times, although the impact will be dependent on your system. Check the Alternatives section below to see if there may be a better option that fits your needs. For example, the service command may do what you need without any additional effort.
More detail on each answer:
Option 1: Upgrade WSL to the latest application release (if supported by your system) and opt-in to the Systemd feature
Microsoft has now integrated Systemd support in the WSL2 application release (as opposed to the older "Windows feature" implementation).
Starting with WSL Application Release 1.0.0, this feature is available on both Windows 10 and Windows 11. Windows 10 users do need to be on UBR (update build revision) 2311 or later. The UBR is the last 4 digits of your full Windows build number (e.g. 10.0.19045.2311 for Windows 10 22H2). 2311 is installed with KB5020030, an optional Preview update, although if you are reading this later, it will likely be a later (non-Preview) monthly servicing update.
If you are on a supported Windows release, the WSL application with Systemd support can be installed:
Through the Microsoft Store (as "Windows Subsystem for Linux").
Or from the Releases page in the Github repo. To install a release manually:
Reboot (to make sure that WSL is not in use at all). A simple wsl --shutdown may work, but often will not.
Download the 1.0.0 (or later) release from the link above.
Start an Administrator PowerShell and:
Add-AppxPackage <path.to>/Microsoft.WSL_1.0.0.0_x64_ARM64.msixbundle
wsl --version # to confirm
To enable, start your Ubuntu (or other Systemd) distribution under WSL (typically just wsl ~ will work).
sudo -e /etc/wsl.conf
Add the following:
[boot]
systemd=true
Exit Ubuntu and again:
wsl --shutdown
Then restart Ubuntu.
sudo systemctl status
... should show your Systemd services.
Option 2: Run a Systemd-helper script designed for WSL2
There are a number of Systemd-enablement scripts available from various sources. Given the complexities involved in running Systemd under WSL, it is recommended that you:
Use one that is actively maintained
Attempt to understand, as much as possible, how they operate, and how they may impact other features and applications in your distribution(s) under WSL
When asking questions here or on any other site, disclose in the question which script you are using so that others can attempt to understand and/or reproduce your issue in the proper context
Several of the more popular projects that enable Systemd under WSL2 are:
Genie: 1.8k stars, last commit September, 2022
Distrod: 1.4k stars, last commit July 2022
WSL2-Hacks: 1.1k stars, mostly instructional, with a supporting script example. Last commit January, 2022
At the core, all of them operate on the same principles covered in the next option ...
Option 3: Manually run Systemd in its own namespace
One of the main issues with running Systemd in earlier versions of WSL is that both inits need to be PID 1. To get around this, it is possible to create a new namespace or container where Systemd can run as PID 1.
To see how this is done (at a very basic level):
Run:
sudo -b unshare --pid --fork --mount-proc /lib/systemd/systemd --system-unit=basic.target
This starts Systemd in a new namespace with its own PID mapping. Inside that namespace, Systemd will be PID1 (as it must, to function) and own all other processes. However, the "real" PID mapping still exists outside that namespace.
Note that this is a "bare minimum" command-line for starting Systemd. It will not have support for, at least:
Windows Interop (the ability to run Windows .exe)
The Windows PATH (which isn't necessary without Windows Interop anyway)
WSLg
The scripts and projects listed above do extra work to get these things working as well.
Wait a few seconds for Systemd to start up, then:
sudo -E nsenter --all -t $(pgrep -xo systemd) runuser -P -l $USER -c "exec $SHELL"
This enters the namespace, and you can now use ps -efH to see that systemd is running as PID 1 in that namespace.
At this point, you should be able to run systemctl.
And after proving to yourself that it's possible, it is recommended that you exit all WSL instances completely, then doing wsl --shutdown. Otherwise, some things will be "broken" until you do. They can likely be "fixed", but that's beyond the scope this answer. If you are interested, please refer to the projects listed above to see how they handle these situations.
Alternative 1: SysVInit scripts (e.g. sudo service <service_name> start) where available
In Ubuntu, Debian, and some other distributions on WSL, many of the common system services still have the "old" init.d scripts available to be used in place of systemctl with Systemd units. You can see these by using ls /etc/init.d/.
So, for example, you can start ssh with sudo service ssh start, and it will run the /etc/init.d/ssh script with the start argument.
Even some non-default packages such as MySql/MariaDB will install both the Systemd unit files and the old init.d scripts, so you can still use the service command for them as well.
On the hand, some packages, like Elasticsearch, only install Systemd units. And some distributions only provide Systemd units for most (if not all) packages in their repositories.
Alternative 2: Manually configuring and running the service
For those services that don't have a init-script equivalent, it can be possible to run them "manually".
For simplicity, let's assume that the ssh init.d script wasn't available.
In this case, the "answer" is to figure out what the Systemd unit files are doing and attempt to replicate that manually. This can vary widely in complexity. But I'd start with looking at the Systemd unit file that you are trying to run:
less /lib/systemd/system/ssh.service
# Trimmed
[Service]
EnvironmentFile=-/etc/default/ssh
ExecStartPre=/usr/sbin/sshd -t
ExecStart=/usr/sbin/sshd -D $SSHD_OPTS
RuntimeDirectory=sshd
RuntimeDirectoryMode=0755
Some of the less relevant lines have been trimmed to make it easier to parse, but you can man systemd.exec, man systemd.service, and others to see what most of the options do.
In this case, when you sudo systemctl start ssh, it:
Reads environment variables (the $SSHD_OPTS) from /etc/default/ssh
Tests the config, exits if there is a failure
Makes sure the RuntimeDirectory exists with the specified permissions. This translates to /run/sshd (from man systemd.exec). This also removes the runtime directory when you stop the service.
Runs /usr/sbin/sshd with options
So, if you don't have any environment-based config, you could just set up a script to:
Make sure the runtime directory exists. Note that, since it is in /run, which is a tmpfs mount, it will be deleted after every restart of the WSL instance.
Set the permissions to 0755
Start /usr/sbin/sshd as root
... And you would have done the same thing manually without Systemd.
Again, this is probably the simplest example. You might have much more to work through for more complex tasks.
Alternative 3: Docker
Many packages/services are available as Docker images. Docker typically runs very well under Ubuntu on WSL2 (specifically WSL2; it will not run on WSL1). If there's not a SysVinit "service" script for the service you are trying to start, there may very well be a Docker image available that runs in a containerized environment.
Example: Elasticsearch, as in this question.
Bonus #1: Doesn't interfere with other packages already installed (no dependency issues).
Bonus #2: The Docker images themselves pretty much never use Systemd, so you can often inspect the Dockerfile to see how the service is started without Systemd. For more information see the next option - "The manual way."
Microsoft recommends Docker Desktop for Windows for running Docker containers under WSL2.
Footnote This answer is being posted as a Community Wiki because it can apply to multiple Stack Overflow questions. It is originally based on answers to this Ask Ubuntu question. However, it is hoped that this wiki-answer can be continuously updated by the community as Systemd evolves on WSL.
This question has been chosen since:
It appears to be the most canonical, straightforward, "How do I enable Systemd on WSL?" question.
It is on-topic, as *creating Systemd services is (or at least can-be) unique to programming.

Mongo Procedures Dependencies Cause Neo4j Connection Issues

I am using Neo4j on a remote server (ubuntu 20.4) and would like to stream data from MongoDB to Neo4j. I followed the instructions here. I tried both ways by using the following approaches:
Use the following command:
sudo wget https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/tag/4.3.0.7/apoc-mongodb-dependencies-4.3.0.7.jar -O /mnt/neo4j/plugins/apoc-mongodb-dependencies-4.3.0.7.jar
Note that the plugins directory has a different path due to mounting. I changed the path in the configuration file accordingly. This should not be causing any problems because I had the same problem before mounting.
Also, I tried to match the same release as the apoc-core file (4.4.0.3) in a separate attempt with no better outcome.
Changing the ownership and read permissions as follows didn't help either:
sudo chown neo4j:neo4j apoc-mongodb-dependencies-4.4.0.3.jar
sudo chmod 755 apoc-mongodb-dependencies-4.4.0.3.jar
Use the following commands:
sudo wget https://repo1.maven.org/maven2/org/mongodb/mongo-java-driver/3.12.11/mongo-java-driver-3.12.11.jar -O /mnt/neo4j/plugins/mongo-java-driver-3.12.11.jar
sudo wget https://repo1.maven.org/maven2/org/mongodb/mongodb-driver/3.12.11/mongodb-driver-3.12.11.jar -O /mnt/neo4j/plugins/mongodb-driver-3.12.11.jar
sudo wget https://repo1.maven.org/maven2/org/mongodb/mongodb-driver-core/4.7.1/mongodb-driver-core-4.7.1.jar -O /mnt/neo4j/plugins/mongodb-driver-core-4.7.1.jar
sudo wget https://repo1.maven.org/maven2/org/mongodb/bson/4.7.1/bson-4.7.1.jar -O /mnt/neo4j/plugins/bson-4.7.1.jar
Note that I used the latest versions. I tried the versions available in the instructions as well with no difference in the outcome.
Now when restarting the neo4j.service, I no longer can access the cypher-shell nor the browser. In the first case, I get "connection refused", while I get a blank page in the browser case. When I check the status, the service is active and running. But I noticed that it is missing a line compared to when I don't have the dependencies.
Starting...
This instance is ServerId{#}
======== Neo4j 4.4.5 ======== (This line is missing with the dependencies downloaded!)
When I delete the dependencies from the plugins directory and restart, everything goes back to normal and functions as expected. One more thing to note is that apoc-core procedures work just fine!
I don't know if I'm doing something wrong here or if there is some sort of underlying problem!

acme.sh --renew ignores --stateless

acme.sh --renew-all --stateless seems to ignore the --stateless flag and run stand-alone. I've tried all the command-line finesse I can think of (--stateless ahead of --renew-all, --renew -d mydomain.com, etc.) It always wants to run stand-alone.
What am I missing something? A work-around? I really don't want to dive into script source.
Debian 11.1, acme.sh --upgrade telling me I have the latest, nginx (with /.well-known/ config verified via browser).

Starman and MovableType 5.2

MovableType 5.2 has builtin PSGI support (via a mt.psgi in the main directory) and I've been trying to take advantage of it using starman/plackup.
Starman w/ MT fires up, but I get odd Not Founds and a silent hang & fail when I run mt-upgrade.cgi.
How I am running starman:
cd /home/ec2-user/mysite/perl/components/movabletype
plackup -s Starman --port 8045 --error-log /home/ec2-user/mysite/perl/logs/starman.log --pid /home/ec2-user/mysite/perl/var/starman.pid -a mt.psgi
Mystery 1: My browser returns "Not Found" for index.html, but mt-static and mt.cgi is reachable.
X http://mysite:8045/mt/index.html ... Not Found
✓ http://mysite:8045/mt-static ... listing of static assets
✓ http://mysite:8045/mt/mt.cgi ... redirects to ...
X http://mysite:8045/mt/mt-upgrade.cgi?__mode=install ... fails
Mystery 2: Due to the redirect I believe that MT code is running. But it is mostly silent in the starman log even though I have tried different MT debug options. Worse, the mt-upgrade.cgi seems to be reached then fails with this lone message to the starman log:
[mypid] Bootstrap CGI script in non-buffering mode: /home/ec2-user/mysite/perl/components/movabletype/mt-upgrade.cgi
This non-buffering message seems informational and normal, and is coming from the MT codebase.
I have also run directly under starman and get the same result.
Any ideas or help would be appreciated!
System/Environment:
MT_HOME='/home/ec2-user/mysite/perl/components/movabletype'
linux AMI on an amazon ec2.
perl-5.16.0 under perlbrew.
CPAN modules:
cpanm starman
cpanm CGI::PSGI
cpanm CGI::Parse::PSGI
cpanm CGI::Compile
yum install expat-devel
cpanm XML::Parser
cpanm SOAP::Lite
cpanm SOAP::Transport::HTTP
cpanm XMLRPC::Transport::HTTP::Plack
cpanm DBI
sudo yum install postgresql9-devel
cpanm DBD::Pg
cpanm Task::Plack
MT config:
CGIPath http://mysite:8045/mt
StaticWebPath http://mysite:8045/mt-static
PIDFilePath /home/ec2-user/mysite/perl/var/starman.pid
DebugMode 1
ObjectDriver DBI::postgres
Database db
DBUser dbuser
DBPassword dbpass
DBHost dbhost.mysite
I'm Yuji Takayama, lead engineer of Movable Type. I did try to reproduce your steps but I was not able to reproduce this. (this means I got initial install screen)
So, Can you try with mysql? also can you try "tools/upgrade" script? I think you can get error messages when some errors occurs.
ex.)
cd MT_HOME; perl -Ilib -Iextlib tools/upgrade --username --password --nickname --email --preferred_language --site_name --site_url --site_path --site_theme --site_timezone
And, I have answer about "index.html was not found".
Reason: MT will never mount MT_HOME as static files directory like a mt_static, because we think that MT_HOME must not be possible to list.
If you are trying to run Movable Type under Starman, I would seriously recommend running the upgrade script that comes with Movable Type using the command-line equivalent of your choice. There is no good reason for mt-upgrade.cgi to become persistent in the manner that Starman and Plack permit.
Try to change your paths in mt-config.cgi to relative paths, not URLs. Like so:
CGIPath /mt/
StaticWebPath /mt-static/
Just curious: although it does work now, are you aware that PG isn't supported and that you're taking a risk that a future version of MT might break with it?
P.S. also don't forget the PIDFilePath directive mentioned in the documentation.
BTW, a 404 on http://mysite:8045/mt/index.html looks quite normal. You're not supposed to have your site index at the same level than MT (you could, but that's not the recommended setup and it won't work in a stock MT install that assumes that MT and the produced sites are clearly separated).

Where is the web server root directory on Angstrom Linux (BeagleBoard | BeagleBone)?

I'm trying to find the default web server directory on my BeagleBone with Angstrom Linux. That is, where are the files served when I go to:
http://beaglebone.local:80
Another way would be to answer this question: How do I find out what directory a port number points to on my BeagleBone with Angstrom Linux?
The BeagleBone|BeagleBoard Angstrom Linux distribution ships with a socket server that runs as a service using node.js and bonescript in:
/var/lib/cloud9/bonescript/
and can be accessed at: http://beaglebone.local:80
You can also install lighttpd with
opkg install lighttpd
and will install a config file into
/etc/lighttpd.conf
which can be altered to set the default web directory wherever you like.
I found the following worked, evenutally:
systemctl disable bonescript.service
systemctl disable bonescript-autorun.service
systemctl disable cloud9.service
systemctl disable bonescript.socket
Use 'systemctl list-units' to check they've stopped. Possibly, there's a correct order you have to do these in, I had to fiddle around and repeat these a bit before they were all dead. You could probably just nuke the symlinks in /etc/systemd/system/multi-user.target.wants and reboot.