LXD FIRST IMPRESSIONS:
Creating LXD Container with Static IP (and Docker Profile)We want to create a docker capable LXD container using an existing bridge with a static ip and zfs. Then we want to install docker and test it. We will make a copy of this container once the admin users have been added so that we wont have to replicate these tasks. Our security model requires ssh keys to log in AND passwords to escalate privileges.
The first thing we learned is that LXC and LXD are pretty different beasts and that while lxc with lxc-templates is a straightforward way to create containers that act a lot like regular old hardware LXD brings on all of its we love the mother fucking cloud baggage. Major differences had to do with user mapping on the containers files created by root on the host were mapped to nobody on the container, making it really difficult to set up home directories etc. (for a workaround to this see https://stackoverflow.com/questions/33377916/migrating-lxc-to-lxd) The second was the way that the network is initialized with the assumption that LXD would be providing the bridge and the context.
First Attempt and zfs/bridge setup
Create zfs container and bridge as before
```root@bs2020:~# lxd init
... create new zfs pool and use all of /dev/sdd1 do not configure bridge ...
root@bs2020:~# dpkg-reconfigure -p medium lxd
... no yes br1 ... use existing bridge...
root@bs2020:~# lxc launch ubuntu:16.04 franklin -p default -p docker
root@bs2020:~# lxc stop franklin
root@bs2020:~# passwd -l ubuntu -R /var/lib/lxd/containers/franklin.zfs/rootfs
passwd: user 'ubuntu' does not exist
root@bs2020:~# cd /var/lib/lxd/containers/franklin.zfs/rootfs/
root@bs2020:/var/lib/lxd/containers/franklin.zfs/rootfs# cat ~feurig/passwd.add>>etc/passwd
root@bs2020:/var/lib/lxd/containers/franklin.zfs/rootfs# cat ~feurig/shadow.add>>etc/shadow
root@bs2020:/var/lib/lxd/containers/franklin.zfs/rootfs# tar -xzvf ~feurig/fnj.tgz
home/feurig
...
home/joe/.ssh/authorized_keys
root@bs2020:/var/lib/lxd/containers/franklin.zfs/rootfs# usermod -R /var/lib/lxd/containers/franklin.zfs/rootfs -G sudo,root joe
root@bs2020:/var/lib/lxd/containers/franklin.zfs/rootfs# usermod -R /var/lib/lxd/containers/franklin.zfs/rootfs -G sudo,root feurig
root@bs2020:/var/lib/lxd/containers/franklin.zfs/rootfs# groupadd -R /var/lib/lxd/containers/franklin.zfs/rootfs -g 1001 feurig
root@bs2020:/var/lib/lxd/containers/franklin.zfs/rootfs# groupadd -R /var/lib/lxd/containers/franklin.zfs/rootfs -g 1002 joe
root@bs2020:/var/lib/lxd/containers/franklin.zfs/rootfs# cat <<eod >>/var/lib/lxd/containers/franklin.zfs/rootfs/etc/resolvconf/resolv.conf.d/base
> dns-nameserver 198.202.31.132 198.202.31.141 8.8.8.8
> nameserver 198.202.31.132 198.202.31.141 8.8.8.8
> eod
root@bs2020:/var/lib/lxd/containers/franklin.zfs/rootfs# sed -i 's/^iface eth0/#iface eth0/' /var/lib/lxd/containers/franklin.zfs/rootfs/etc/network/interfaces
root@bs2020:/var/lib/lxd/containers/franklin.zfs/rootfs# cat <<eod2 >>/var/lib/lxd/containers/franklin.zfs/rootfs/etc/network/interfaces
> iface eth0 inet static
> address 198.202.31.201/25
> gateway 198.202.31.129
> dns-nameservers 198.202.31.132 198.202.31.141 8.8.8.8
> dns-search suspectdevices.com digithink.com
> eod2
root@bs2020:/var/lib/lxd/containers/franklin.zfs/rootfs# lxc start franklin
Second thought: Fuck that! Make it work!
Second attempt
(create LXD profile for suspect devices development).
root@bs2020:~# lxc stop franklin
root@bs2020:~# lxc delete franklin
root@bs2020:~# lxc profile create susdev
root@bs2020:~# lxc profile edit susdev
...
Repeat until you have a working system that can be logged into remotely
Create docker container container
root@bs2020:~# lxc profile show susdev
config:
user.network_mode: link-local
user.user-data: |
#cloud-config
timezone: America/Vancouver
users:
- name: feurig
passwd: "... SUBSTITUTE REAL PASSWORD HASH HERE ...."
gecos: Donald Delmar Davis
ssh-authorized-keys:
- ssh-rss ... SUBSTITUTE REAL KEY HERE ... don@viscious
groups: sudo,root
shell: /bin/bash
- name: joe
passwd: "... SUBSTITUTE REALPASSWORD HASH HERE ...."
gecos: Joseph Wayne Dumoulin
ssh-authorized-keys:
- ssh-rss ...SUBSTITUTE REAL KEY HERE... jdumoulin@joeslaptop
groups: sudo,root
shell: /bin/bash
manage_resolv_conf: true
resolv_conf:
nameservers: ['198.202.31.141', '198.202.31.132', '8.8.8.8']
searchdomains:
- suspectdevices.com
- digithink.com
domain: suspectdevices.com
options:
rotate: true
timeout: 1
write_files:
# Set static IP address could not get this to work the "right" way
- path: /etc/network/interfaces
permissions: '0644'
owner: root:root
content: |
auto lo
iface lo inet loopback
auto eth0
# change this after first instantiation
iface eth0 inet static
address 198.202.31.200
broadcast 198.202.31.255
netmask 255.255.255.128
gateway 198.202.31.129
dns-nameservers 198.202.31.141 198.202.31.132 8.8.8.8
runcmd:
# sudo needs to be able to resolve itself to authenticate users
# and the users are locked by default
- sed -i "s/^127.0.0.1/#127.0.0.1/" /etc/hosts
- echo 127.0.0.1 `hostname` localhost >>/etc/hosts
- passwd joe -u
- passwd feurig -u
description: Try to create a sane environment for cloud-init based operating systems
devices:
eth0:
name: eth0
nictype: bridged
parent: br1
type: nic
name: susdev
root@bs2020:~#
root@bs2020:~# lxc list
+--------+---------+-----------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+--------+---------+-----------------------+------+------------+-----------+
| test13 | RUNNING | 198.202.31.200 (eth0) | | PERSISTENT | 0 |
+--------+---------+-----------------------+------+------------+-----------+
root@bs2020:~# lxc init ubuntults16 franklin -p susdev -p docker
Creating franklin
root@bs2020:~# lxc start franklin
root@bs2020:~# lxc exec franklin bash
root@franklin:~# tail -2 /etc/shadow
feurig:<<HASHED PASSWORD>>:17453:0:99999:7:::
joe:<<HASHED PASSWORD>>:17453:0:99999:7:::
root@franklin:~# nano /etc/network/interfaces
root@franklin:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback
auto eth0
# change this after first instantiation
iface eth0 inet static
address 198.202.31.201
broadcast 198.202.31.255
netmask 255.255.255.128
gateway 198.202.31.129
dns-nameservers 198.202.31.141 198.202.31.132 8.8.8.8
root@franklin:~# cat /etc/hosts
#127.0.0.1 localhost
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
127.0.0.1 franklin localhost
root@franklin:~# reboot
root@bs2020:~# lxc list
+----------+---------+-----------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+----------+---------+-----------------------+------+------------+-----------+
| franklin | RUNNING | 198.202.31.201 (eth0) | | PERSISTENT | 0 |
+----------+---------+-----------------------+------+------------+-----------+
| test13 | RUNNING | 198.202.31.200 (eth0) | | PERSISTENT | 0 |
+----------+---------+-----------------------+------+------------+-----------+
root@bs2020:~#
References
- http://www.whiteboardcoder.com/2016/04/cloud-init-nocloud-with-url-for-meta.html
- https://stgraber.org/2016/03/11/lxd-2-0-blog-post-series-012/
- https://github.com/lxc/lxd/blob/master/doc/cloud-init.md
- http://www.mattjarvis.org.uk/post/lxd-openstack-cloudinit-pt1/
- https://sdgsystems.com/blog/understanding-and-using-lxc-and-lxd
- http://cloudinit.readthedocs.io/en/latest/topics/examples.html
- http://cloudinit.readthedocs.io/en/latest/topics/debugging.html