When I executed sudo apt update I'm getting
Reading package lists... Done
E: List directory /var/lib/apt/lists/partial is missing. - Acquire (20: Not a directory)
Also, I was getting a status error which I solved using
sudo cp /var/lib/dpkg/status-old /var/lib/dpkg/status
I tried sudo mkdir /var/lib/apt/lists/partial as suggested in few other threads
mkdir: cannot create directory ‘/var/lib/apt/lists/partial’: Not a directory
Even I tried sudo mkdir /var/lib/apt/lists/
Any other solution?
The answer may be inappropriate here. But as I came here others may land here too.
If you're using docker and you face the same issue you can do like the following.
USER root
# RUN commands
USER 1001
Reference: Link
You can try adding -u 0 in the command
sudo docker exec -u 0 -it ContainerID bin/bash
According to Docker, the u flag defines what username or UID in the system for the container to run as, setting -u 0 means you run the container as root, use it with caution! Reference here
The same happened to me. I follow as guide this answer:The package lists or status file could not be parsed or opened
I assumed my lists were corrupted. I went to /var/lib/apt/ I saw a file (lists#) instead of a directory. I deleted it (sudo rm lists) and re-created the path (sudo mkdir -p /var/lib/apt/lists/partial). Double-check the path gets created.
I ran into the same issue while trying to build a new container and experimenting with Dockerfile for a while.
What saved me finally was just to delete all containers I've created during this process using docker rm.
I had this same issue when trying to install an Typora on Ubuntu 20.04.
I was running into the error whenever I run the command below:
# add Typora's repository
sudo add-apt-repository 'deb https://typora.io/linux ./'
Here's how I solved it:
I disconnected and reconnected my network connection, and when I ran the command again, it worked fine.
I think it was an issue with my network connectivity.
That's all.
I hope this helps
I had a similar error when using bitnami spark image and docker exec command with arguments -u didn't work for me. I found my answer in the image documentation here.
In case you are using a docker image, it might be that the image is a non root container image. Read the documents of the docker image provider to find the solution to see how you can use the image as a root container image.
this is how it works access as root in docker bash and install your apps
get id container by name
sudo docker ps -aqf "name=name=es01"
access bash as root
sudo docker exec -u 0 -it 3d42134dfd59 bash
Example install:
apt get update
apt-get install nano
You first need to have super user privilege by typing in sudo -i and then inserting your password.
Related
I installed postres using pacman on Manjaro.
When I do:
su postgres
postgres -D ~/data >~/logs/log
I get
FATAL: could not create lock file "/run/postgresql/.s.PGSQL.5432.lock": No such file or directory
And so, every time I boot up I have to
sudo su
mkdir /run/posgresql
chown postgres:postgres /run/posgresql
I tried creating /etc/systemd/system/postgresql.service as described in https://www.postgresql.org/docs/10/server-start.html, but the service doesn't run on startup?
Why isn't the folder persisting after reboot? Is there a better way of starting up postgres? What am I missing?
Using the "service" approach is the proper way. It will ensure the creation of the lockfile for you. If you want it to autostart, you need to run the enable command on your service.
systemctl enable postgresql
I am trying to setup a build server in a Red Hat Enterprise Linux 8 (CentoOS 8) virtual machine.
I installed podman by running sudo dnf install -y #container-tools
I then ran sudo podman pull mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim to pull a container image from docker:
Trying to pull mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim...Getting image source signatures
Copying blob e936bd534ffb done
Copying blob caf64655bcbb done
Copying blob 4156e490f05f done
Copying blob 68ced04f60ab done
Copying blob 7064c3d93b4a done
Copying config e2cd20adb1 done
Writing manifest to image destination
Storing signatures
e2cd20adb1292ef24ca70de7abaddaadd57a5c932d3852b972e43b6f05a03dea
This looks successful to me. And if I run it again, I get told that the layers "already exists". But then I run:
podman image ls
and I get an empty list back:
REPOSITORY TAG IMAGE ID CREATED SIZE
I also tried the following commands to get a list:
podman image ls -a
podman image list
podman image list -a
podman images
podman images ls
podman images ls -a
podman images list
podman images list -a
They all give an empty list.
How can I see the container image that I pulled down?
Update: I ran sudo podman run --rm --name=linuxconfig-test -p 80:80 httpd and (on another machine) browsed to the ip address of my linux machine and got It Works! shown. So podman is working at least in part.
Unlike Docker, Podman stores images in the home directory of the user. The default path is ~/.local/share/containers/storage and it can be verified by running podman info. Since you executed podman pull as root, the pulled image will be stored in the home directory of the root user. This is why no images are listed when you run podman image ls without sudo.
The main idea behind podman is that it can run entirely in user mode without connecting to a priviledged daemon. Ideally, all podman commands should be run without sudo.
Turns out you have to run using sudo. I ran :
sudo podman image ls
and it returned the list of container images.
You can use the --root option to give the path to where the images are stored. That is if you need to run as root.
Though one important part of using podman is that you do not need to run as root or sudo user.
Note - The moment you run this, podman will change the owner of certain folders and files in the overlay location,and later when you run without sudo , you need to chown back. So not recommended
sudo podman images --root /home/xxx/.local/share/containers/storage
I'm new to Postgres.
I updated the Dockerfile I use and successfully installed Postgresql on it. (My image runs Ubuntu 16.04 and I'm using Postgres 9.6.)
Everything worked fine until I tried to move the database to a Volume with docker-compose (that was after making a copy of the container's folder with cp -R /var/lib/postgresql /somevolume/.)
The issue is that Postgres just keeps crashing, as witnessed by supervisord:
2017-07-26 18:55:38,346 INFO exited: postgresql (exit status 1; not expected)
2017-07-26 18:55:39,355 INFO spawned: 'postgresql' with pid 195
2017-07-26 18:55:40,430 INFO success: postgresql entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-07-26 18:55:40,763 INFO exited: postgresql (exit status 1; not expected)
2017-07-26 18:55:41,767 INFO spawned: 'postgresql' with pid 197
2017-07-26 18:55:42,841 INFO success: postgresql entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-07-26 18:55:43,179 INFO exited: postgresql (exit status 1; not expected)
(and so on…)
Logs
It's not clear to me what's happening as /var/log/postgresql remains empty.
chown?
I suspect it has to do with the user. If I compare the data folder inside the container and the copy I made of it to the volume, the only difference is that the original is owned by postgres while the copy is owned by root.
I tried running chown -R postgres:postgres on the copy. The operation was performed successfully, however postmaster.pid remains owned by root and I think that would be the issue.
Questions
How can I get more information about the cause of the crash?
How can I make it so that postmaster.id be owned by postgres ?
Should I consider running postgres with root instead?
Any hint welcome.
EDIT: links to the Dockerfile and the docker-compose.xml.
I'll answer my own question:
Logs & errors
What made matters more complicated was that I was not getting any specific error message.
To change that, I disabled the [program:postgresql] section in supervisord and, instead, started postgres manually from the command-line (thanks to Miguel Marques for setting me on the right track with his comment.)
Then I finally got some useful error messages:
2017-08-02 08:27:09.134 UTC [37] LOG: could not open temporary statistics file "/var/run/postgresql/9.6-main.pg_stat_tmp/global.tmp": No such file or directory
Fixing the configuration
I fixed the error above with this, eventually adding them to my Dockerfile:
mkdir -p /var/run/postgresql/9.6-main.pg_stat_tmp
chown postgres.postgres /var/run/postgresql/9.6-main.pg_stat_tmp -R
(Kudos to this guy for the fix.)
To make the data permanent, I also had to do this, for the volume to be accessible by postgres:
mkdir -p /var/lib/postgresql/9.6/main
chmod 700 /var/lib/postgresql/9.6/main
I also used initdb to initialize the data directory. BEWARE! This will erase any data found in that folder. Like so:
rm -R /var/lib/postgresql/9.6/main/*
ls /var/lib/postgresql/9.6/main/
/usr/lib/postgresql/9.6/bin/initdb -D /var/lib/postgresql/9.6/main
Testing
After the above, I could finally run postgres properly. I used this command to run it and test from the command-line:
su postgres
/usr/lib/postgresql/9.6/bin/postgres -D /var/lib/postgresql/9.6/main -c config_file=/etc/postgresql/9.6/main/postgresql.conf # as per the Docker docs
To test, I kept it running and then, from another prompt, checked everything ran fine with this:
su postgres
psql
CREATE TABLE cities ( name varchar(80), location point );
INSERT INTO cities VALUES ('San Francisco', '(-194.0, 53.0)');
select * from cities; # repeat this command after restarting the container to check that the data does persist
…making sure to restart the container and test again to check the data did persist.
And then finally restored the [program:postgresql] section in supervisord, rebuilt the image and restarted the container, making sure everything ran fine (in particular supervisord: tail /var/log/supervisor/supervisord.log), which it did.
(The command I used inside of supervisord.conf is also /usr/lib/postgresql/9.6/bin/postgres -D /var/lib/postgresql/9.6/main -c config_file=/etc/postgresql/9.6/main/postgresql.conf, as per this Docker article and other postgres+supervisord examples. Other options would have been using pg_ctl or an init.d script, but it's not clear to me why/when one would use those.)
I spent a lot of time on this. Hopefully the detailed answer will help someone down the line.
P.S.: I did end up producing a minimal example of my issue. If that can help anyone, here they are: Dockerfile, supervisord.conf and docker-compose.yml.
I do not know if this would be another way to achieve the same result (I'm new on Docker and Postgres too), but have you try the oficial repository image for Postgres (https://hub.docker.com/_/postgres/)?
I'm getting the data out of the container setting the environment variable PGDATA to '/var/lib/postgresql/data/pgdata' and binding this to an external volume on the run command:
docker run --name bd_TEST --network=my_network --restart=always -e POSTGRES_USER="superuser" -e POSTGRES_PASSWORD="myawesomepass" -e PGDATA="/var/lib/postgresql/data/pgdata" -v /var/local/db_data:/var/lib/postgresql/data/pgdata -itd -p 5432:5432 postgres:9.6
When the volume is empty, all the files are created by the image startup script, and if they already exist, the database start to used it.
From past experience I can see what may be a problem. I can't say if this will help but it is worth a try.
I would have added this as a comment, but I can't because my rep isn't hight enough.
I've spied a couple problems with how you have structured your statements in your Dockerfile. You have installed various things multiple times and also updated sporadically through the code. In my own files i've noticed that this can lead to somewhat random behaviour of my services and installation because of the different layers.
This may not seem to solve your problem directly, but cleaning up your file as is outlined in the best practices has solved many Dockerfile problems for me in the past.
One of the first places upon finding such problems is to start here at the best practices for RUN. This has helped me solve tricky problems in the past and I hope it'll solve or at least make it easier.
Pay special attention to this part:
After building the image, all layers are in the Docker cache. Suppose you later modify apt-get install by adding extra package:
FROM ubuntu:14.04
RUN apt-get update
RUN apt-get install -y curl nginx
Docker sees the initial and modified instructions as identical and reuses the cache from previous
steps. As a result the apt-get update is NOT executed because the
build uses the cached version. Because the apt-get update is not run,
your build can potentially get an outdated version of the curl and
nginx packages.
After reading this I would start by consolidating all your dependencies.
In my case, having the same error, I debugged it until I found out:
the disk was full and I increased the diskspace to solve this.
(stupid error, easy fix - maybe reading this here helps someone not wasting time)
also linking this questiong for other options:
Supervisord "exit status 1 not expected" running php script
https://serverfault.com/questions/537773/supervisor-process-exits-with-exit-status-1-not-expected/1076115#1076115
I was trying to install Apache MADLib on Postgres. Having difficulty with YUM approach I moved to Docker approach as suggested by this website https://pgxn.org/dist/madlib/
I was able to pull docker image as suggested at para 1. Now at para 2 I am stuck with comment "Path to incubator-madlib directory". I am not able to understand whether it should be the URL to MADLib Incubator such as "https://github.com/apache/incubator-madlib" or it should refer to local disk area. It would be great by giving an example of how to run this command.
2) Launch a container corresponding to the MADlib image, mounting the
source code folder to the container:
docker run -d -it --name madlib \ -v (path to incubator-madlib directory):/incubator-madlib/ madlib/postgres_9.6
The (path to incubator-madlib directory) refers to wherever you have git cloned the MADlib code base to on your machine. Say for example, your home directory in your machine is /home/xyz/ and you have cloned the MADlib code base there, you should have a directory called /home/xyz/incubator-madlib. You can now run the docker command documented in the MADlib repo as follows:
docker run -d -it --name madlib -v /home/xyz/incubator-madlib/:/incubator-madlib/ madlib/postgres_9.6
You were probably getting the Permission denied docker:... error after trying Robert's suggestion because the $(pwd) was not referring to your incubator-madlib source code folder, but was referring to /var/lib/docker/devicemapper/devicemapper/data, which should not be the case. In any case, it might be a better idea to provide the incubator-madlib directory's absolute path in the docker command, as specified above.
As is documented, that is the directory on your computer where the source code resides:
where incubator-madlib is the directory where the MADlib source code resides.
So, supossing that you have downloaded the source code in ./incubator-madlib, run as this:
docker run -d -it --name madlib -v $(pwd)/incubator-madlib:/incubator-madlib/ madlib/postgres_9.6
Then see what the container logs:
docker logs -f madlib
I find I have the wreckage of two old PostgreSQL installations on Ubuntu 10.04:
$ pg_lsclustersVersion Cluster Port Status Owner Data directory Log file
Use of uninitialized value in printf at /usr/bin/pg_lsclusters line 38.
8.4 main 5432 down /var/lib/postgresql/8.4/main /var/log/postgresql/postgresql-8.4-main.log
Use of uninitialized value in printf at /usr/bin/pg_lsclusters line 38.
9.1 main 5433 down /var/lib/postgresql/9.1/main /var/log/postgresql/postgresql-9.1-main.log
$
Attempts to perform basic functions return errors, for instance:
createuser: could not connect to database postgres: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
More information comes when I try to start the database server:
$ sudo /etc/init.d/postgresql start
* Starting PostgreSQL 9.1 database server
* Error: The cluster is owned by user id 109 which does not exist any more
...fail!
$
My question: how do I completely remove both clusters and set up a new one? I've tried removing, purging, and reinstalling postgresql, following the advice here: https://stackoverflow.com/a/2748644/621762. Now pg_lsclusters shows no clusters in existence, but the No such file or directory error persists when I try to createuser, createdb or run psql. What have I failed to do?
First, that answer you linked to was pretty unsafe - hand-editing /etc/passwd ?!? dselect where an apt wildcard would do? Crazy stuff. I'm not surprised you're having issues.
As for the no such file or directory messages: You need to make sure you have a running PostgreSQL server ("cluster") before you can use admin commands like createdb, because they make a connection to the server. The No such file or directory message is telling you that the server doesn't exist or isn't running.
Here's what's happening:
Ubuntu uses pg_wrapper to manage multiple concurrent PostgreSQL instances. The issues you're having are really with pg_wrapper.
Ideally you would've just used pg_dropcluster to get rid of the unwanted clusters. Unfortunately, by following bad advice it sounds like you've got your system into a bit of a messed-up state where the PostgreSQL packages are half-installed and kind of mangled. You need to either repair the install, or totally clean it out.
I'd clean it out. I'd recommend:
Verify that pg_lsclusters lists no database clusters
apt-get --purge remove postgresql\* - this is important
Remove /etc/postgresql/
Remove /etc/postgresql-common
Remove /var/lib/postgresql
userdel -r postgres
groupdel postgres
apt-get install postgresql-common postgresql-9.1 postgresql-contrib-9.1 postgresql-doc-9.1
It's possible that the apt-get --purge step will fail because you've removed the user IDs, etc. Re-creating the postgres user ID with useradd -r -u 109 postgres should allow you to re-run the purge successfully then delete the user afterwards.
This answer is not directly about removing a postgres instance, rather, about resoliving the issue,
Error: The cluster is owned by user ...
I got this error while trying to spin up a docker container pointed to a postgres data directory that was produced via a different container (on a different host machine).
The error is directly related to directory ownership. In my case, the system was unable to find the user that certain postgres directories was owned by in the current environment. By re-owning those directories to the right user resolves the issue. Following is an example mapping (that worked for me):
chown -R postgres:postgres /var/lib/postgresql
chown -R postgres:postgres /etc/postgresql
chown -R postgres:postgres /var/log/postgresql
chown -R postgres:postgres /var/run/postgresql