Facts:
Rootless podman works perfectly for uid 1480
Rootless podman fails for uid 2088
CentOS 7
Kernel 3.10.0-1062.1.2.el7.x86_64
podman version 1.4.4
Almost the entire environment has been removed between the two
The filesystem for /tmp is xfs
The capsh output of the two users is identical but for uid / username
Both UIDs have identical entries in /etc/sub{u,g}id files
The $HOME/.config/containers/storage.conf is the default and is identical between the two with the exception of the uids. The storage.conf is below for reference.
I wrote the following shell script to demonstrate just how similar an environment the two are operating in:
#!/bin/sh
for i in 1480 2088; do
sudo chroot --userspec "$i":10 / env -i /bin/sh <<EOF
echo -------------- $i ----------------
/usr/sbin/capsh --print
grep "$i" /etc/subuid /etc/subgid
mkdir /tmp/"$i"
HOME=/tmp/"$i"
export HOME
podman --root=/tmp/"$i" info > /tmp/podman."$i"
podman run --rm --root=/tmp/"$i" docker.io/library/busybox printf "\tCOMPLETE\n"
echo -----------END $i END-------------
EOF
sudo rm -rf /tmp/"$i"
done
Here's the output of the script:
$ sh /tmp/podman-fail.sh
[sudo] password for functional:
-------------- 1480 ----------------
Current: =
Bounding set =cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,35,36
Securebits: 00/0x0/1'b0
secure-noroot: no (unlocked)
secure-no-suid-fixup: no (unlocked)
secure-keep-caps: no (unlocked)
uid=1480(functional)
gid=10(wheel)
groups=0(root)
/etc/subuid:1480:100000:65536
/etc/subgid:1480:100000:65536
Trying to pull docker.io/library/busybox...Getting image source signatures
Copying blob 7c9d20b9b6cd done
Copying config 19485c79a9 done
Writing manifest to image destination
Storing signatures
ERRO[0003] could not find slirp4netns, the network namespace won't be configured: exec: "slirp4netns": executable file not found in $PATH
COMPLETE
-----------END 1480 END-------------
-------------- 2088 ----------------
Current: =
Bounding set =cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,35,36
Securebits: 00/0x0/1'b0
secure-noroot: no (unlocked)
secure-no-suid-fixup: no (unlocked)
secure-keep-caps: no (unlocked)
uid=2088(broken)
gid=10(wheel)
groups=0(root)
/etc/subuid:2088:100000:65536
/etc/subgid:2088:100000:65536
Trying to pull docker.io/library/busybox...Getting image source signatures
Copying blob 7c9d20b9b6cd done
Copying config 19485c79a9 done
Writing manifest to image destination
Storing signatures
ERRO[0003] Error while applying layer: ApplyLayer exit status 1 stdout: stderr: there might not be enough IDs available in the namespace (requested 65534:65534 for /home): lchown /home: invalid argument
ERRO[0003] Error pulling image ref //busybox:latest: Error committing the finished image: error adding layer with blob "sha256:7c9d20b9b6cda1c58bc4f9d6c401386786f584437abbe87e58910f8a9a15386b": ApplyLayer exit status 1 stdout: stderr: there might not be enough IDs available in the namespace (requested 65534:65534 for /home): lchown /home: invalid argument
Failed
Error: unable to pull docker.io/library/busybox: unable to pull image: Error committing the finished image: error adding layer with blob "sha256:7c9d20b9b6cda1c58bc4f9d6c401386786f584437abbe87e58910f8a9a15386b": ApplyLayer exit status 1 stdout: stderr: there might not be enough IDs available in the namespace (requested 65534:65534 for /home): lchown /home: invalid argument
Here's the storage.conf for the 1480 uid. It's identical except s/1480/2088/:
[storage]
driver = "vfs"
runroot = "/run/user/1480"
graphroot = "/tmp/1480/.local/share/containers/storage"
[storage.options]
size = ""
remap-uids = ""
remap-gids = ""
remap-user = ""
remap-group = ""
ostree_repo = ""
skip_mount_home = ""
mount_program = ""
mountopt = ""
[storage.options.thinpool]
autoextend_percent = ""
autoextend_threshold = ""
basesize = ""
blocksize = ""
directlvm_device = ""
directlvm_device_force = ""
fs = ""
log_level = ""
min_free_space = ""
mkfsarg = ""
mountopt = ""
use_deferred_deletion = ""
use_deferred_removal = ""
xfs_nospace_max_retries = ""
You can see there's basically no difference between the two podman info outputs for the users:
$ diff -u /tmp/podman.1480 /tmp/podman.2088
--- /tmp/podman.1480 2019-10-17 22:41:21.991573733 -0400
+++ /tmp/podman.2088 2019-10-17 22:41:26.182584536 -0400
## -7,7 +7,7 ##
Distribution:
distribution: '"centos"'
version: "7"
- MemFree: 45654056960
+ MemFree: 45652697088
MemTotal: 67306323968
OCIRuntime:
package: containerd.io-1.2.6-3.3.el7.x86_64
## -24,7 +24,7 ##
kernel: 3.10.0-1062.1.2.el7.x86_64
os: linux
rootless: true
- uptime: 30h 17m 50.23s (Approximately 1.25 days)
+ uptime: 30h 17m 54.42s (Approximately 1.25 days)
registries:
blocked: null
insecure: null
## -35,14 +35,14 ##
- quay.io
- registry.centos.org
store:
- ConfigFile: /tmp/1480/.config/containers/storage.conf
+ ConfigFile: /tmp/2088/.config/containers/storage.conf
ContainerStore:
number: 0
GraphDriverName: vfs
GraphOptions: null
- GraphRoot: /tmp/1480
+ GraphRoot: /tmp/2088
GraphStatus: {}
ImageStore:
number: 0
- RunRoot: /run/user/1480
- VolumePath: /tmp/1480/volumes
+ RunRoot: /run/user/2088
+ VolumePath: /tmp/2088/volumes
I refuse to believe there's an if (2088 == uid) { abort(); } or similar nonsense somewhere in podman's source code. What am I missing?
Does podman system migrate fix there might not be enough IDs available in the namespace for you?
It did for me and others:
https://github.com/containers/libpod/issues/3421
AFAICT, sub-UID and GID ranges should not overlap between users. For reference, here is what the useradd manpage has to say about the matter:
SUB_GID_MIN (number), SUB_GID_MAX (number), SUB_GID_COUNT
(number)
If /etc/subuid exists, the commands useradd and newusers
(unless the user already have subordinate group IDs)
allocate SUB_GID_COUNT unused group IDs from the range
SUB_GID_MIN to SUB_GID_MAX for each new user.
The default values for SUB_GID_MIN, SUB_GID_MAX,
SUB_GID_COUNT are respectively 100000, 600100000 and 65536.
SUB_UID_MIN (number), SUB_UID_MAX (number), SUB_UID_COUNT
(number)
If /etc/subuid exists, the commands useradd and newusers
(unless the user already have subordinate user IDs) allocate
SUB_UID_COUNT unused user IDs from the range SUB_UID_MIN to
SUB_UID_MAX for each new user.
The default values for SUB_UID_MIN, SUB_UID_MAX,
SUB_UID_COUNT are respectively 100000, 600100000 and 65536.
The key word is unused.
CentOS 7.6 does not suport rootless buildah by default - see https://github.com/containers/buildah/pull/1166 and https://www.redhat.com/en/blog/preview-running-containers-without-root-rhel-76
I am unable to add rules to audit daemon using /etc/audit/audit.rules
Every time i add the rules using auditctl it gets removed on reboot or audit daemon restart I have attached the /etc/audit/audit.rules and /etc/audit/auditd.conf
cat /etc/audit/auditd.conf
$ cat /etc/audit/auditd.conf
#
# This file controls the configuration of the audit daemon
#
local_events = yes
write_logs = yes
log_file = /NU_Application/audit.log
log_group = root
log_format = RAW
flush = INCREMENTAL_ASYNC
freq = 50
max_log_file = 8
num_logs = 5
priority_boost = 4
disp_qos = lossy
dispatcher = /sbin/audispd
name_format = NONE
##name = mydomain
max_log_file_action = ROTATE
space_left = 75
space_left_action = SYSLOG
verify_email = yes
action_mail_acct = root
admin_space_left = 50
admin_space_left_action = SUSPEND
disk_full_action = SUSPEND
disk_error_action = SUSPEND
use_libwrap = yes
##tcp_listen_port = 22
tcp_listen_queue = 5
tcp_max_per_addr = 1
##tcp_client_ports = 1024-65535
tcp_client_max_idle = 0
enable_krb5 = no
krb5_principal = auditd
##krb5_key_file = /etc/audit/audit.key
distribute_network = no
cat /etc/audit/audit.rules
$ cat /etc/audit/audit.rules
## First rule - delete all
## Increase the buffers to survive stress events.
## Make this bigger for busy systems
-b 8192
## This determine how long to wait in burst of events
--backlog_wait_time 0
## Set failure mode to syslog
-f 1
-w /var/log/lastlog -p wa
root#iWave-G22M:~# auditctl
When i restart the audit daemon ( i.e /etc/init.d/auditd restart ) and try to list the rules i get the message No rules
$ /etc/init.d/auditd restart
Restarting audit daemon auditd
type=1305 audit(1558188111.980:3): audit_pid=0 old=1148 auid=4294967295 ses=4294967295
res=1
type=1305 audit(1558188112.010:4): audit_enabled=1 old=1 auid=4294967295 ses=4294967295
res=1
type=1305 audit(1558188112.020:5): audit_pid=30342 old=0 auid=4294967295 ses=4294967295
res=1
1
$ auditctl -l
No rules
OS INFO
$ uname -a
Linux iWave-G22M 3.10.31-ltsi-svn743 #5 SMP PREEMPT Mon May 27 18:28:01 IST 2019 armv7l GNU/Linux
audit_2.8.4.bb file was used to install auditd daemon via yocto
path of audit_2.8.4.bb -- http://git.yoctoproject.org/cgit/cgit.cgi/meta-selinux/tree/recipes-security/audit/audit_2.8.4.bb?h=master
audit rules add via /etc/audit/audit.rules and auditctl command are not permanent. to make them permanent across reboot you have to add them /etc/audit/rules.d/audit.rules file.
after adding the rule, restart auditd service and run command auditctl -l, it will list all the rules and also reflect in /etc/audit/audit.rules file.
My system is
% uname -or
FreeBSD 11.0-RELEASE-p2
Sphinx version is
% searchd --help
Sphinx 2.2.11-id64-release (95ae9a6)
Sphinx configuration:
index content_rt_template : common_template
{
type = rt
rt_mem_limit = 128M # 128M only...
rt_field = text
rt_attr_string = text
rt_field = title
rt_attr_string = title
rt_field = url
rt_attr_string = url
rt_attr_bigint = item_id
rt_attr_uint = source_id
rt_attr_timestamp = published_date
rt_attr_timestamp = created_date
}
common {
lemmatizer_base = /path/to/sphinx/
}
indexer
{
mem_limit = 128M # 128M only...
}
index content_rt_from_20170501_to_20170601 : content_rt_template
{
path = /path/to/sphinx/data/2017/content_rt_from_20170501_to_20170601
}
index content_rt_from_20170601_to_20170701 : content_rt_template
{
path = /path/to/sphinx/data/2017/content_rt_from_20170601_to_20170701
}
index content_rt_from_20171201_to_20180101 : content_rt_template
{
path = /path/to/sphinx/data/2017/content_rt_from_20171201_to_20180101
}
index content2017
{
type = distributed
local = content_rt_from_20170501_to_20170601
local = content_rt_from_20170601_to_20170701
local = content_rt_from_20171201_to_20180101
}
searchd
{
listen = 127.0.0.1:9417
listen = 9317:mysql41
log = /path/to/sphinx/log/searchd_2017.log
query_log = /path/to/sphinx/log/query_2017.log
read_timeout = 60
max_children = 30
pid_file = /path/to/sphinx/pid/searchd2017.pid
seamless_rotate = 0
preopen_indexes = 0
unlink_old = 1
workers = threads # for RT to work
binlog_path = /path/to/sphinx/data/2017/
}
Used memory before starting Sphinx:
Mem[||||||||| 5.33G/40.0G]
Swp[||||||||||||||||||||||||||||||3.35G/4.00G]
Log on Sphinx start:
% ./start.sh
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)
using config file '/path/to/sphinx/conf/content2017.conf'...
listening on 127.0.0.1:9417
listening on all interfaces, port=9317
WARNING: index 'common_template': key 'path' not found - NOT SERVING
WARNING: index 'content_rt_template': path must be specified - NOT SERVING
WARNING: failed to init process shared rwlock: process shared rwlock is not supported by FreeBSD; ALTER disabled
precaching index 'content_rt_from_20170501_to_20170601'
WARNING: failed to init process shared rwlock: process shared rwlock is not supported by FreeBSD; ALTER disabled
precaching index 'content_rt_from_20170601_to_20170701'
WARNING: failed to init process shared rwlock: process shared rwlock is not supported by FreeBSD; ALTER disabled
precaching index 'content_rt_from_20171201_to_20180101'
precached 3 indexes in 6.520 sec
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)
using config file '/path/to/sphinx/conf/content_dist.conf'...
listening on 127.0.0.1:9312
listening on all interfaces, port=9306
WARNING: index 'common_template': key 'path' not found - NOT SERVING
WARNING: index 'content_rt_template': path must be specified - NOT SERVING
WARNING: failed to init process shared rwlock: process shared rwlock is not supported by FreeBSD; ALTER disabled
precaching index 'content_snippet'
precached 1 indexes in 0.064 sec
Used memory after Sphinx started:
Mem[||||||||||||||| 11.6G/40.0G]
Swp[||||||||||||||||||||||||||||||3.34G/4.00G]
Sphinx use about 6G. But according to mem_limit and rt_mem_limit must use not more than 128 * 3 = 384M.
What may be reason of using lot of memory?
Maybe reason in this warning?
WARNING: failed to init process shared rwlock: process shared rwlock
UPD
I has tryed it on Ubuntu 16.04 and situation is same.
The rt_mem_limit only limits the size of the ram chunk itself. Any disk chunks will use their own memory.
Typically its attributes that compose the biggest part , as by default, all are held in memory. Can cut down memory with http://sphinxsearch.com/docs/current.html#conf-ondisk-attrs
With option
ondisk_attrs = pool
RT index use not many RAM.
I have been working on trying to get a value with a custom MIB through SNMP following this tutorial. I'm using net-snmp 5.7.3. I have used this MIB along with the corresponding codes with this.
My setup: I have two VM's, both Ubuntu 16, one is snmp-server with IP:192.168.5.20 and the other snmp-agent with IP:192.168.5.21.
I did MIBS=ALL and export MIBS. On the agent, I have successfully compiled my MIB and got the expected output. My code files are in the path /home/snmp-agent/Desktop/proj/net-snmp-5.7.3/agent/mibgroup/. I reconfigure mthe net-snmp like this:
./configure --with-mib-modules="pingCtlTable pingResultsTable pingProbeHistoryTable"
make
make install
Everything worked fine. And when I do snmptranslate on the agent I get:
root#snmpagent:~# snmptranslate -On DISMAN-PING-MIB::pingResultsTable
.1.3.6.1.2.1.80.1.3
root#snmpagent:~# snmptranslate -On DISMAN-PING-MIB::pingCtlTable
.1.3.6.1.2.1.80.1.2
root#snmpagent:~# snmptranslate -On DISMAN-PING-MIB::pingProbeHistoryTable
.1.3.6.1.2.1.80.1.4
Output on the server, when I do this:
snmpget -v2c -c public 192.168.5.21 sysDescr.0
SNMPv2-MIB::sysDescr.0 = STRING: Linux snmp-agent 4.10.0-33-generic #37~16.04.1-Ubuntu SMP Fri Aug 11 14:07:24 UTC 2017 x86_64
But when I run snmpget, from the server, I get this error for all the three:
snmpget -v2c -c public 192.168.5.21 DISMAN-PING-MIB::pingCtlTable.0
DISMAN-PING-MIB::pingCtlTable = No Such Object available on this agent at this OID
Output when specified the OID's:
snmpget -v2c -c public 192.168.5.21 .1.3.6.1.2.1.80.1.2
DISMAN-PING-MIB::pingCtlTable = No Such Object available on this agent at this OID
NOTE: I have already seen these posts with no luck for me.
add new mib master agent
snmpget-no-such-object... and snmpget-returns-no-such-object...
UPDATE 2:
When I do snmpwalk from the server, I get:
root#snmp-server:# snmp-server:# snmpwalk -v 2c -c ncs -m DISMAN-PING-MIB 192.168.5.22 .1.3.6.1.2.1.80
DISMAN-PING-MIB::pingObjects.0 = INTEGER: 1
DISMAN-PING-MIB::pingFullCompliance.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 = STRING: "/bin/echo"
DISMAN-PING-MIB::pingMinimumCompliance.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 = ""
DISMAN-PING-MIB::pingCompliances.4.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 = ""
DISMAN-PING-MIB::pingCompliances.5.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 = INTEGER: 5
DISMAN-PING-MIB::pingCompliances.6.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 = INTEGER: 1
DISMAN-PING-MIB::pingCompliances.7.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 = INTEGER: 1
DISMAN-PING-MIB::pingCompliances.20.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 = INTEGER: 4
DISMAN-PING-MIB::pingCompliances.21.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 = INTEGER: 1
DISMAN-PING-MIB::pingIcmpEcho.1.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 = ""
DISMAN-PING-MIB::pingIcmpEcho.2.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 = ""
DISMAN-PING-MIB::pingIcmpEcho.3.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 = INTEGER: 1
DISMAN-PING-MIB::pingIcmpEcho.4.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 = INTEGER: 0
DISMAN-PING-MIB::pingMIB.4.1.2.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48.1 = ""
So accordingly when I follow the output of snmpwalk, I get:
root#snmp-server:# snmpget 192.168.5.22 DISMAN-PING-MIB::pingFullCompliance.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48
DISMAN-PING-MIB::pingFullCompliance.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 = Wrong Type (should be INTEGER): STRING: "/bin/echo"
I don't understand why I have such a long OID? What is pingFullCompliance.15.46.49.46.51.46.54.46.49.46.50.46.49.46.56.48 ? Where am I going wrong? Can anyone please point me in the right direction... Thank you.
Is it possible to read a znode with a space in it through Zookeeper CLI?
I've 2 values under regions ('us-west 1' and 'us-east')
[zk: localhost:2181(CONNECTED) 11] get /regions/
us-west 1 us-east
I can read 'us-east'.
[zk: localhost:2181(CONNECTED) 11] get /regions/us-east
null
cZxid = 0xa
ctime = Tue Jul 10 12:41:49 IST 2012
mZxid = 0xa
mtime = Tue Jul 10 12:41:49 IST 2012
pZxid = 0x1b
cversion = 9
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 9
But not 'us-west 1'
[zk: localhost:2181(CONNECTED) 11] get /regions/us-west 1
Node does not exist: /regions/us-west
I tried options like '%20', '\ ' , '+' etc.. for the space, but nothing worked.
Please try zookeepercli, as follows:
$ zookeepercli --servers srv1,srv2,srv3 -c create "/demo_only 1" "the value"
$ zookeepercli --servers srv1,srv2,srv3 -c get "/demo_only 1"
the value
zookeepercli is free and open source.
Disclaimer: I'm author of this tool.
It looks like you will not be able to do that from the ZK command-line client. The Zookeeper Java client (which is the one you are using, probably) separates commands (e.g. get) from their parameters (e.g. /regions/us-west 1 in your case) by parsing whitespace characters, as you can see in the code provided with the client (e.g. zookeeper-3.3.5\src\java\main\org\apache\zookeeper\ZooKeeperMain.java):
public boolean parseCommand( String cmdstring ) {
String[] args = cmdstring.split(" ");
if (args.length == 0){
return false;
}
command = args[0];
cmdArgs = Arrays.asList(args);
return true;
}
Since they are splitting by a " ", unless you discover a way to overcome that split call by some sort of escaping when calling the get command, you won't be able to retrieve those nodes using this client. The code above interprets your 1 in a get call as if it were the watch parameter, as you can see in the get command syntax:
get path [watch]
I recommend you to use a different character, like '_' for example, instead of the whitespace on the znodes naming. If that is not an option, you will either need to modify the ZK Java client yourself, or use another client.
There's an open issue in JIRA for this feature, but a workaround is to pass a command on the command-line instead of using the interactive console:
$ zkCli.sh -c get "/regions/us-west 1"