Understanding LXC user/group mapping
I’ve been moving some docker containers back from the VM I put them back to Linux Containers (LXC) because of I/O performance and cpu consumption reasons (caused by that I/O).
LXC is a light-weight container runtime, enabling you to containerize an entire OS. Docker containers, on the other hand, are application level containers. The LXC container runs on the same kernel as the host OS, and basically uses the same underlying filesystem (in this case, ZFS!)
The docker containers I wanted to move back were influxdb, prometheus and several other containers causing serious levels of I/O.
One caveat: I also wanted to access some of my host filesystems inside LXC, and preferrably not doing it via NFS. Bind mounts in LXC to the rescue!
Binding a local filesystem is easy: shutdown the container, then edit
/etc/pve/lxc/XXX.conf, with XXX being your container identifier in Proxmox.
To this file, you add:
This will make the host filesystem under
/local/data show up as
/mount/data/ on the guest LXC.
Problem: the data is written by different processes, with users that do not map with LXC. You’ll notice that the directory is not accessible - the onmious Access Denied!
By default LXC starts with different UID/GID’s, starting (on Proxmox) with UID 100000 and GID 100000, to properly segregate the different containers/processes.
You could open up the files on the host OS, going
chmod -R 777 /local/data everywhere, but that’s a Really Bad IdeaTM.
Luckely LXC comes with support for id mapping! This allows us to tell LXC that it should map specific host UID/GID’s into the guest container. It took me a bit to wrap my head around how it actually works, though.
Telling LXC which ID’s need to be mapped
Configuring the host to allow mapping
On the host OS, there are two files which play a major role:
/etc/subgid. These contain the mappings of the UID’s that can be remapped.
By default they contain:
which means that the user/group root can map from UID/GID
100000, and for
65536 consequitive ID’s.
Keeping in mind that LXC is started as
root on Proxmox, this will mean that inside LXC a process started with UID 0 will be remapped to UID 100000 on the host, UID 1 will be 100001, UID 65536 will be 165536. Same with groups.
Now, if we want to allow a container to map to a UID/GID on the host, we’ll have to specifically allow it. Say, you want to actually use UID 1002, and make it match the host UID. In that case, we add this to
This means that user root can map UID 1002, and it can do that for 1 sequential UID. So only 1002.
Now, say that you want to map UID 1002-1005.
Read: root can map from 1002 onwards, and for a max of 4 UID’s. This is 1002, 1003, 1004 and 1005.
Configuring the container
/etc/lxc/pve/XXX.conf, you need to add a line telling LXC to map the UID/GID. For example, to map UID 1002:
lxc.idmap: u 0 100000 1002 lxc.idmap: u 1002 1002 1 lxc.idmap: u 1003 101003 64533
Let’s unwrap this:
umeans user id mapping. You also have
gfor group id mapping.
0is the start UID to start mapping.
100000is the UID to map to
1002is the amount of UID’s to remap.
UID 0 to 1001 is mapped on UID 100000 to 101001.
1002is the start UID to start mapping.
1002is the UID to map to
1is the amount of UID’s to remap.
UID 1002 is mapped to 1002, and for 1 UIDs.
1003is the start UID to start mapping.
101003is the UID to map to
64533is the amount of UID’s to remap.
UID 1003 to 65536 is mapped to 101003 and on, to 165536.
This now matches with what’s in
/etc/subuid, and should work ;)
A restart of the container later the UID’s should be mapped correctly.