diff --git a/content/blog/20240817-samba-adventures-in-guix.md b/content/blog/20240817-samba-adventures-in-guix.md new file mode 100644 index 0000000..b66a71e --- /dev/null +++ b/content/blog/20240817-samba-adventures-in-guix.md @@ -0,0 +1,584 @@ ++++ +title = "Samba Adventures with Guix" +author = ["Peter Tillemans"] +date = 2024-08-17T00:00:00+02:00 +draft = false +[taxonomies] + tags = ["linux", "guix"] + categories = ["guix", "os", "linux"] ++++ + +Samba or CIFS file sharing is a finicky area at best, but widely used, +especially since it was heavily pushed by Microsoft in the Windows +ecosystem, This makes it widely used in corporate and NAS environments +and even for Linux file sharing. + +In this post I will look at some ways to use RF'S file sharing as a +client. Personally I find hardly any uses for Samba file serving +nowadays from my compute environments, with **git**, **ssh**, **http(s)**, +databases, MQTT, file synchronization services ... it hardly ever happens that +I face a situation where I think : "Hey, I wish I could serve my files +from my PC/VPS/VM/... with Samba". It is such a generic non-specific +service I usually find a more opinionated data sharing +service. Actually I find the same for NFS. + +That being said, it is still ubiquitous to get enterprise data and to +connect to NAS or remote hard disks. + + +## CIFS Server Simulator {#cifs-server-simulator} + +In practice setting up and connecting to a CIFS server is a +frustrating experience due to a bewildering array of different +protocols, security systems, credentials, ... . + +To test out accessing a Samba server without having to deal with too +many moving pieces at the same time I like to use a known fixed local +basic samba server. This enables me to get working client - server +configurations which I find easier to tweak to real life servers than +starting from scratch. + +Docker hub provides preconfigured images for Samba servers which are +easy to use : + +```shell +$ docker run --name test-smb -p 4139:139 --rm -p 4445:445 -v `pwd`/samples/:/mnt/export --rm -d dperson/samba -p -u "joe;schmoe" -s "export;/mnt/export/;yes;no;no;joe;;;Test Share" +``` + +This will start a samba server and listen on ports 4139 and 4445 so it +does not clash if a server is running on the machine we're using and +we do not need special privileges other than being in the `docker` group +to run **docker** as a regular user. The `-s ...` option configures a share +name **export** which is browsable, not readonly, not accessible by guest +users and can only be accessed with the **joe** user account. This is to +make the experience a bit more in line with usual real world +configuration which are seldom as open as the default settings for +shares. For more details, see [the github repo for the image](https://hub.docker.com/r/dperson/samba). + +Because this is not a fun command line to type I like to put them in a +**Makefile** in a folder with some support files + +```shell +$ cd ... +$ mkdir test-smb +$ cd test-smb +$ mkdir samples +$ echo "Hello, Samba" >samples/hello.txt +``` + +and then add the Makefile + +```makefile +servers-start: + docker run --name test-smb -p 4139:139 --rm -p 4445:445 -v `pwd`/samples/:/mnt/export --rm -d dperson/samba -p -u "joe;schmoe" -s "export;/mnt/export/;yes;no;no;joe;;;Test Share" + +servers-stop: + docker stop test-smb +``` + + +### Guix Notes for Docker {#guix-notes-for-docker} + +Docker (and podman too) need some OS support to talk to the kernel to +create the namespaces to make the containers work. Even though podman +does not need a daemon to run, it still needs some **setuid** helpers. I +tend to mostly use **docker** because of habit and everyone else uses it +and tends to be better supported and documented for my use cases. In +any case I did not have any luck getting either to work in a local +`guix shell`. + +To enable **docker** on Guix-SD I have the following in my system configuration. + +```scheme + ... + (use-service-modules cups desktop docker networking ssh xorg) + ... + +(operating-system + ... + (users + (cons* + (user-account + ... + (supplementary-groups + '( ... "docker"))) ;; enable access to docker daemon + %base-user-accounts)) +... +(packages + (append + (list + ... + ;; add docker command line tools + docker + ... + )))) +... +(services + (append + (list + ... + ;; enable docker + (service docker-service-type) +``` + +When using Guix on a host os, the native **docker** package should work +fine. + + +## Connecting using smbclient {#connecting-using-smbclient} + +Before mounting drives we need to ascertain our credentials are +accepted by the Samba server. It makes no sense proceeding trying to +mount shares if we cannot get past authentication and the overhead of +configuring those shares and the Guix configuration rebuilding really +impacts iteration speed when trying things out. + +The most straightforward way to connect to a CIFS server is with the +**smbclient** tool which is part of the **samba** package: + +```shell +➜ guix shell samba + +test-smb on  main via 🐃 +➜ smbclient -L //localhost -p 4445 -U joe +Password for [WORKGROUP\joe]: + + Sharename Type Comment + --------- ---- ------- + export Disk Test Share + IPC$ IPC IPC Service (Samba Server) +SMB1 disabled -- no workgroup available + +test-smb on  main [!] via 🐃 took 4s +➜ smbclient //localhost/export -p 4445 -U joe +Password for [WORKGROUP\joe]: +Try "help" to get a list of possible commands. +smb: \> ls + . D 0 Sat Aug 17 03:49:32 2024 + .. D 0 Sat Aug 17 05:22:38 2024 + hello.txt N 13 Sat Aug 17 03:50:27 2024 + + 1912951596 blocks of size 1024. 998957000 blocks available +smb: \> +``` + +The `guix shell samba` creates a profile with the **samba** package +installed which places \`smbclient\` on the path. + +`smbclient -L //servername` lists the services offered by the server, in +our case **localhost** to connect to our docker instance. The `-p 4445` +option selects the custom port we specified on our docker +container. By default **smbclient** will use your login as uid, so we need +to override it to the user we created when starting the docker +container with `-U joe` . + +The password is configured as `schmoe`. + +We see the `export` folder is shared so we connect to it using +\`smbclient //localhost/export -p 4445 -U joe\` and can list the +contents. And of course we can upload, download and all the other +terminal goodness offered by **smbclient**. + +In practice it can be very fiddly to get the right username, password, +Workgroup or Domain, port number, SMB protocol, etc dialed in. The +obtuse messages often do not really help. The other methods make +interpreting errors even harder, with the exception of programmatic +access which is sometime surprisingly helpful in getting a connection +going. + + +### Save the credentials {#save-the-credentials} + +The samba tools have a convention to save the credentials in a +**authentication file** which is supported by most tools AFAICT. + +To use this create a file `.smbcredentials` in your home folder. Well, +it can be anything but I like that place as I typically only have my +NAS to connect to using samba and it is for my home folder, backup +folder, my Music and Movies folder and the like, i.e. stuff related to +my user account, so I find it in its place in my home folder. + +In it place the `username`, `password` and `domain` which worked with +smbclient so they no longer need to be provided : + +```text +username=joe +password=schmoe +domain=WORKGROUP +``` + +Then we can use it: + +```shell +$ smbclient //localhost/export -p 4445 -A ~/.smbcredentials + +Try "help" to get a list of possible commands. +smb: \> ls + . D 0 Sat Aug 17 03:49:32 2024 + .. D 0 Sat Aug 17 12:02:13 2024 + hello.txt N 13 Sat Aug 17 03:50:27 2024 + + 1912951596 blocks of size 1024. 998918312 blocks available +smb: \> +``` + +This save a lot of typing and we can use this file in the next stages +and avoid spreading the credentials all over the disk. I am going to +gloss over encrypting this info any further because I do not keep +nuclear (or any other for that matter) secrets on my nas. + + +## Mounting Shares with `mount.cifs` {#mounting-shares-with-mount-dot-cifs} + +Let's create a mount point in our test folder + +```shell +$ mkdir mnt +``` + +and then mount the share with `mount.cifs`. This is part of the `cifs-utils` package. + +```shell +ttest-smb on  main [?] via 🐃 +➜ guix shell cifs-utils +The following derivation will be built: + /gnu/store/2x7mmyrsnsf21aing02ass82899gm2yh-profile.drv + +building CA certificate bundle... +listing Emacs sub-directories... +building fonts directory... +building directory of Info manuals... +building profile with 1 package... +est-smb on  main via 🐃 +❮ sudo mount.cifs //localhost/export mnt -o credentials=/home/pti/.smbcredentials,port=4445 + +test-smb on  main [?] via 🐃 +➜ ls mnt +hello.txt + +test-smb on  main [?] via 🐃 +➜ +``` + +This mounts the share to the folder `mnt`. The `-o` option is used to +point to the credentials file and the port number of our docker NAS +simulator. + +Checking with the regular `mount` command to see if it agrees we mounted +the share: + +```shell + +❯ sudo mount -t cifs +//localhost/export on /home/pti/src/test-smb/mnt type cifs (rw,relatime,vers=3.1.1,cache=strict,username=joe,domain=WORKGROUP,uid=0,noforceuid,gid=0,noforcegid,addr=0000:0000:0000:0000:0000:0000:0000:0001,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=1) + +test-smb on  main [?] via 🐃 +➜ sudo mount | grep cifs +/etc/auto.cifs on /nas type autofs (rw,relatime,fd=6,pgrp=1732,timeout=600,minproto=5,maxproto=5,indirect,pipe_ino=27049) +//localhost/export on /home/pti/src/test-smb/mnt type cifs (rw,relatime,vers=3.1.1,cache=strict,username=joe,domain=WORKGROUP,uid=0,noforceuid,gid=0,noforcegid,addr=0000:0000:0000:0000:0000:0000:0000:0001,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=1) +``` + +Calling `mount` without arguments lists all mounted filesystems, however +nowadays the output is so overwhelming that I prefer to use `grep` to +narrow down the output to the filesystem type I am interested in. The +same thing can be achieved by specifying the filesystem type with the +`-t` option. + + +## Configuring the mounts in the operating system. {#configuring-the-mounts-in-the-operating-system-dot} + +The `mount.cifs` command is fine for testing and ad-hoc use but in +practice I mostly (read 99+% of the time) want the same shares of the +NAS mounted in the same place on my system. This is where the `fstab` +file comes in. This file is read by the mount command at boot time and +mounts the configured filesystems. + +I do not recommend mounting CIFS shares at boot time, but configuring +the shares in `fstab` is a good way to be able to mount them when needed +with a quick `mount -a` command. + +On my Tuxedo system I have in the `/etc/fstab` file + +```text +... +//nas.snamellit.com/home /home/pti/nas cifs rw,uid=1000,gid=100,credentials=/home/pti/.smbcredentials 0 0 +//nas.snamellit.com/public /mnt/public cifs rw,uid=1000,gid=100,credentials=/home/pti/.smbcredentials 0 0 +//nas.snamellit.com/multimedia /mnt/multimedia cifs rw,uid=1000,gid=100,credentials=/home/pti/.smbcredentials 0 0 +... +``` + +The first column is the reference to the share as we've seen +above. Then the mount point. The 3rd column indicates it is a `cifs` +share, Then the options we've seen with smbmount. Here I have to play +with the `uid` and `gid` to align my local user and group with the +configuration on the nas. The last 2 columns are whether to include +the mounts when dumping the filesystem and doing a filesystem check +which will probably always be 0 as the CIFS server is responsible for +that. + +In theory this will mount the shares at boot time, and it probably +does, however in my experience it seems they are unmounted on suspend +and not remounted after resume. Or there is some timeout. I never +investigated I must admit. I just do a quick `mount -a` before I need +them and this works wonders. + +On Guix-SD the `file-systems` are specified in the `operating-system` +section of the system configuration : + +```scheme +(operating-system + ... + +(file-systems + (cons* + ... + (file-system + (device "//nas.snamellit.com/public") + (options "uid=1000,gid=1000,credentials=/home/pti/.smbcredentials") + (mount-point "/mnt/public") + (type "cifs") + (mount? #f) + (create-mount-point? #t)) + (file-system + (device "//nas.snamellit.com/multimedia") + (options "uid=1000,gid=1000,credentials=/home/pti/.smbcredentials") + (mount-point "/mnt/multimedia") + (type "cifs") + (mount? #f) + (create-mount-point? #t)) + (file-system + (device "//nas.snamellit.com/home") + (options "uid=1000,gid=1000,credentials=/home/pti/.smbcredentials") + (mount-point "/home/pti/nas") + (type "cifs") + (mount? #f) + (create-mount-point? #t)) + %base-file-systems))) + ) +``` + +Which is just a straightforward translation of the **fstab** columns in +[Guix-ese.](https://guix.gnu.org/manual/devel/en/html_node/File-Systems.html) + + +## Automatic Mounting with AutoFs {#automatic-mounting-with-autofs} + +Of course the inefficiency of having to type `mount -a` almost on a +weekly basis is unbearable and this inefficiency has to be addressed +even if this means we cannot expect net positive effect in the coming +millenia. + +Enter `autofs` which is an awesome system to dynamically mount and +unmount filesystems on an as needed basis. The kernel will notice when +a mounted folder is accessed and ask the **autofs daemon** to mount the +configured mount and unmount it after some timeout occurs without any +activity. + +This is transparant for the user other than a slight delay when +accessing the folder the first time when it is unmounted. + +This system is very flexible at it was clearly intended for far more +ambitious use-cases than accessing your personal music library from +your nas, but that does not mean we cannot strip it down and use it +for our purposes. + +What is a bit confusing is the configuration. It uses 3 different +file types to configure the system. + +- the `/etc/autofs.conf` file which configures the daemon operating + options, like the timeouts and the location of the main **master file**. +- the master file(s), there is always a main file, but this can + delegate to directory with additional parts of the file which can be + assumed to be imported in the main file. Its main purpose is to map + a parent folder of mount points to a **map file**. +- the map files which map a subfolder in the folder from the line in + the **master file** to a mount specification which will be used when + that folder is accessed. + + +### Example on the Tuxedo: {#example-on-the-tuxedo} + +On my Tuxedo (running Tuxedo OS which is an Ubuntu 22.04 derivate) I have the following configuration: + +- `/etc/autofs.conf`: + ```text + ... + [ autofs ] + # + # master_map_name - default map name for the master map. + # + master_map_name = /etc/auto.master + ... + ``` + so the **master map** is in the file `/etc/auto.master`. + +- `/etc/auto.master`: + ```text + ... + /- /etc/autofs.direct -ro + ``` + +The "/-" is a special case which means "any" directory. In this case +the folder name in the **map file** is assumed to be a fully qualified +directory name instead of a subfolder in the folder mentioned in the +first column. In this case the **map file** is `/etc/autofs.direct` and the +default options are `-ro` . + +- `/etc/autofs.direct`: + ```text + /home/pti/nas -fstype=cifs,rw,noperm,vers=3.0,credentials=/home/pti/.smbcredentials ://nas.snamellit.com/home + ``` + This shows that the mount point `/home/pti/nas` will mount the share + `//nas.snamellit.com/home` of file system type cifs with the options + (the rest of the 2nd column) as mount options. + + +### Example on the Guix-SD desktop {#example-on-the-guix-sd-desktop} + +Unfortunately there is not (yet) packaged support for an **autofs** +service-type, so we have to make it ourselves. + +First we define a configuration record type for our new service: + +```scheme +(define-record-type* + autofs-configuration make-autofs-configuration + autofs-configuration? + (pid-file autofs-configuration-pid-file + (default "/var/run/autofs.pid")) + (autofs-direct autofs-configuration-autofs-direct + (default (plain-file "autofs.direct" "/home/pti/autonas -fstype=cifs,rw,noperm,vers=3.0,credentials=/home/pti/.smbcredentials ://nas.snamellit.com/home")))) +``` + +I should probably not put my actual configuration in the default, but +it makes my life easier at the moment and I'll refactor it +later. (Yeah, sure...) + +Since the autofs service needs some boilerplate configuration files I +generate them with an activation function: + +```scheme +(define (autofs-activation config) + "Return the activation GEXP to create the config files for autofs" + (with-imported-modules '((guix build utils)) + #~(begin + (use-modules (guix build utils) + (ice-9 textual-ports)) + + (define (touch file-name) + (call-with-output-file file-name (const #t))) + + ;; 'sshd' complains if the authorized-key directory and its parents + ;; are group-writable, which rules out /gnu/store. Thus we copy the + ;; authorized-key directory to /etc. + (call-with-output-file "/etc/autofs.conf" + (lambda (f) (put-string f " +[ autofs ] +master_map_name = /etc/auto.master +timeout = 300 +"))) + (call-with-output-file "/etc/auto.master" + (lambda (f) (put-string f (format #f "/- ~a -ro\n" #$(autofs-configuration-autofs-direct config))))) + + ))) +``` + +You'll recognize the absolutely stripped down versions of the files +mentioned above being generated. The `/etc/auto.master` file maps the +**directfs** map file to the file generated in the configuration record. + +This should tie up nicely the links between the 3 files. + +Then I need a **shepherd service** to start the **autofs** daemon: + +```scheme +(define (autofs-shepherd-service config) + (define pid-file (autofs-configuration-pid-file config)) + + (list + (shepherd-service + (provision '(autofs)) + (documentation "AutoFS Service.") + (requirement '(networking)) + (start #~(make-forkexec-constructor + (list + #$(file-append autofs "/sbin/automount") + "-f" ;; run in foreground to give shepherd more control + "-p" #$(autofs-configuration-pid-file config) + "/etc/auto.master") + #:pid-file #$(autofs-configuration-pid-file config))) + (stop #~(make-kill-destructor)) + ))) +``` + +We point it to the generated **auto.master** file from the activation +function. (I could probably refactor this to use guix instantiated +files but at this stage that extra level of indirection would only +confuse me and make debugging harder. I find jumping through 3 files +already hard enough to follow). + +Then we need to put a bow around it and define an `autofs-service-type`. + +```scheme +(define autofs-service-type + (service-type (name 'autofs) + (description "Run the autofs daemon to automount folders on access.") + (extensions + (list + (service-extension activation-service-type autofs-activation) + (service-extension shepherd-root-service-type autofs-shepherd-service))) + (compose concatenate) + (default-value (autofs-configuration))) + ) +``` + +This just extends the `activation-service-type` and the +`shepherd-root-service-type` to run our initialisation code and ensure +the **autofs** daemon gets started. + +Then we can add this to our system configuration: + +```scheme +(operating-system + ... + (packages + (append + (list + ... + autofs))) + (services + (append + (list + ... + (service autofs-service-type)) + ...))) +``` + +i.e. add the `autofs` package to install the daemon and support stuff +and enable the service of type `autofs-service-type`. + + +## Conclusion {#conclusion} + +I can now use files on my nas transparantly. I was a bit flippant on +my reasons to enable **autofs**. The real reason was that I want to keep +automatic backup copies of my forge running on an VPS somewhere on my +NAS with a cron job, which means I would not be there to run a `mount +-a` at the time. (I now realize I could do that as part of the cron +job : 20/20 hindsight). In any case this is a major quality of life +improvement. As a side effect, my music library now gets properly +indexed and is made available on the default music player. Apparently +I still use CIFS more than I care to admit. + +The big problem with SMB/CIFS is getting the initial connection +going. Once that is achieved and the credentials are safely stored +away in **credentials** file, they can be easily reused with a lot less +surprising things along the way. + +It also pays to test things out in the smallest possible meaningful +scope, in this case **smbclient** before adding more obfuscation layers on +top of it, as that just adds more complexity, pitfalls and rabbit +holes to get lost in. By going step by step, building on previous +result I often get results faster (or at all) even if I have to do +additional steps which turn out to no longer be needed in the final +solution.