website/content/blog/20240817-samba-adventures-in-guix.md

+++ title = "Samba Adventures with Guix" author = ["Peter Tillemans"] date = 2024-08-17T00:00:00+02:00 tags = ["linux", "guix"] categories = ["os", "linux"] draft = false [taxonomies] tags = ["linux", "guix"] categories = ["os", "linux"] +++

Samba or CIFS file sharing is a finicky area at best, but widely used, especially since it was heavily pushed by Microsoft in the Windows ecosystem, This makes it widely used in corporate and NAS environments and even for Linux file sharing.

In this post I will look at some ways to use CIFS file sharing as a client. Personally I find hardly any uses for Samba file serving nowadays from my compute environments, with git, ssh, http(s), databases, MQTT, file synchronization services ... it hardly ever happens that I face a situation where I think : "Hey, I wish I could serve my files from my PC/VPS/VM/... with Samba". It is such a generic non-specific service I usually find a more opinionated data sharing service. Actually I find the same for NFS.

That being said, it is still ubiquitous to get enterprise data and to connect to NAS or remote hard disks.

CIFS Server Simulator

In practice setting up and connecting to a CIFS server is a frustrating experience due to a bewildering array of different protocols, security systems, credentials, ... .

To test out accessing a Samba server without having to deal with too many moving pieces at the same time I like to use a known fixed local basic samba server. This enables me to get working client - server configurations which I find easier to tweak to real life servers than starting from scratch.

Docker hub provides preconfigured images for Samba servers which are easy to use :

$ docker run --name test-smb -p 4139:139 --rm -p 4445:445 -v `pwd`/samples/:/mnt/export --rm -d dperson/samba -p -u "joe;schmoe" -s "export;/mnt/export/;yes;no;no;joe;;;Test Share"

This will start a samba server and listen on ports 4139 and 4445 so it does not clash if a server is running on the machine we're using and we do not need special privileges other than being in the docker group to run docker as a regular user. The -s ... option configures a share name export which is browsable, not readonly, not accessible by guest users and can only be accessed with the joe user account. This is to make the experience a bit more in line with usual real world configuration which are seldom as open as the default settings for shares. For more details, see the github repo for the image.

Because this is not a fun command line to type I like to put them in a Makefile in a folder with some support files

$ cd ...
$ mkdir test-smb
$ cd test-smb
$ mkdir samples
$ echo "Hello, Samba" >samples/hello.txt

and then add the Makefile

servers-start:
        docker run --name test-smb -p 4139:139 --rm -p 4445:445 -v `pwd`/samples/:/mnt/export --rm -d dperson/samba -p -u "joe;schmoe" -s "export;/mnt/export/;yes;no;no;joe;;;Test Share"

servers-stop:
        docker stop test-smb

Guix Notes for Docker

Docker (and podman too) need some OS support to talk to the kernel to create the namespaces to make the containers work. Even though podman does not need a daemon to run, it still needs some setuid helpers. I tend to mostly use docker because of habit and everyone else uses it and tends to be better supported and documented for my use cases. In any case I did not have any luck getting either to work in a local guix shell.

To enable docker on Guix-SD I have the following in my system configuration.

  ...
  (use-service-modules cups desktop docker networking ssh xorg)
  ...

(operating-system
  ...
  (users
   (cons*
    (user-account
     ...
     (supplementary-groups
      '( ... "docker")))  ;; enable access to docker daemon
    %base-user-accounts))
...
(packages
 (append
  (list
   ...
   ;; add docker command line tools
   docker
   ...
   ))))
...
(services
 (append
  (list
   ...
   ;; enable docker
   (service docker-service-type)

When using Guix on a host os, the native docker package should work fine.

Connecting using smbclient

Before mounting drives we need to ascertain our credentials are accepted by the Samba server. It makes no sense proceeding trying to mount shares if we cannot get past authentication and the overhead of configuring those shares and the Guix configuration rebuilding really impacts iteration speed when trying things out.

The most straightforward way to connect to a CIFS server is with the smbclient tool which is part of the samba package:

➜ guix shell samba

test-smb on  main via 🐃
➜ smbclient -L //localhost -p 4445 -U joe
Password for [WORKGROUP\joe]:

    Sharename       Type      Comment
    ---------       ----      -------
    export          Disk      Test Share
    IPC$            IPC       IPC Service (Samba Server)
SMB1 disabled -- no workgroup available

test-smb on  main [!] via 🐃  took 4s
➜ smbclient //localhost/export -p 4445 -U joe
Password for [WORKGROUP\joe]:
Try "help" to get a list of possible commands.
smb: \> ls
  .                                   D        0  Sat Aug 17 03:49:32 2024
  ..                                  D        0  Sat Aug 17 05:22:38 2024
  hello.txt                           N       13  Sat Aug 17 03:50:27 2024

                1912951596 blocks of size 1024. 998957000 blocks available
smb: \>

The guix shell samba creates a profile with the samba package installed which places `smbclient` on the path.

smbclient -L //servername lists the services offered by the server, in our case localhost to connect to our docker instance. The -p 4445 option selects the custom port we specified on our docker container. By default smbclient will use your login as uid, so we need to override it to the user we created when starting the docker container with -U joe .

The password is configured as schmoe.

We see the export folder is shared so we connect to it using `smbclient //localhost/export -p 4445 -U joe` and can list the contents. And of course we can upload, download and all the other terminal goodness offered by smbclient.

In practice it can be very fiddly to get the right username, password, Workgroup or Domain, port number, SMB protocol, etc dialed in. The obtuse messages often do not really help. The other methods make interpreting errors even harder, with the exception of programmatic access which is sometime surprisingly helpful in getting a connection going.

Save the credentials

The samba tools have a convention to save the credentials in a authentication file which is supported by most tools AFAICT.

To use this create a file .smbcredentials in your home folder. Well, it can be anything but I like that place as I typically only have my NAS to connect to using samba and it is for my home folder, backup folder, my Music and Movies folder and the like, i.e. stuff related to my user account, so I find it in its place in my home folder.

In it place the username, password and domain which worked with smbclient so they no longer need to be provided :

username=joe
password=schmoe
domain=WORKGROUP

Then we can use it:

$ smbclient //localhost/export -p 4445 -A ~/.smbcredentials

Try "help" to get a list of possible commands.
smb: \> ls
  .                                   D        0  Sat Aug 17 03:49:32 2024
  ..                                  D        0  Sat Aug 17 12:02:13 2024
  hello.txt                           N       13  Sat Aug 17 03:50:27 2024

                1912951596 blocks of size 1024. 998918312 blocks available
smb: \>

This save a lot of typing and we can use this file in the next stages and avoid spreading the credentials all over the disk. I am going to gloss over encrypting this info any further because I do not keep nuclear (or any other for that matter) secrets on my nas.

Mounting Shares with mount.cifs

Let's create a mount point in our test folder

$ mkdir mnt

and then mount the share with mount.cifs. This is part of the cifs-utils package.

ttest-smb on  main [?] via 🐃
➜ guix shell cifs-utils
The following derivation will be built:
  /gnu/store/2x7mmyrsnsf21aing02ass82899gm2yh-profile.drv

building CA certificate bundle...
listing Emacs sub-directories...
building fonts directory...
building directory of Info manuals...
building profile with 1 package...
est-smb on  main via 🐃
❮ sudo mount.cifs //localhost/export mnt -o credentials=/home/pti/.smbcredentials,port=4445

test-smb on  main [?] via 🐃
➜ ls mnt
hello.txt

test-smb on  main [?] via 🐃
➜

This mounts the share to the folder mnt. The -o option is used to point to the credentials file and the port number of our docker NAS simulator.

Checking with the regular mount command to see if it agrees we mounted the share:

❯ sudo mount -t cifs
//localhost/export on /home/pti/src/test-smb/mnt type cifs (rw,relatime,vers=3.1.1,cache=strict,username=joe,domain=WORKGROUP,uid=0,noforceuid,gid=0,noforcegid,addr=0000:0000:0000:0000:0000:0000:0000:0001,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=1)

test-smb on  main [?] via 🐃
➜ sudo mount | grep cifs
/etc/auto.cifs on /nas type autofs (rw,relatime,fd=6,pgrp=1732,timeout=600,minproto=5,maxproto=5,indirect,pipe_ino=27049)
//localhost/export on /home/pti/src/test-smb/mnt type cifs (rw,relatime,vers=3.1.1,cache=strict,username=joe,domain=WORKGROUP,uid=0,noforceuid,gid=0,noforcegid,addr=0000:0000:0000:0000:0000:0000:0000:0001,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=1)

Calling mount without arguments lists all mounted filesystems, however nowadays the output is so overwhelming that I prefer to use grep to narrow down the output to the filesystem type I am interested in. The same thing can be achieved by specifying the filesystem type with the -t option.

Configuring the mounts in the operating system.

The mount.cifs command is fine for testing and ad-hoc use but in practice I mostly (read 99+% of the time) want the same shares of the NAS mounted in the same place on my system. This is where the fstab file comes in. This file is read by the mount command at boot time and mounts the configured filesystems.

I do not recommend mounting CIFS shares at boot time, but configuring the shares in fstab is a good way to be able to mount them when needed with a quick mount -a command.

On my Tuxedo system I have in the /etc/fstab file

...
//nas.snamellit.com/home /home/pti/nas   cifs   rw,uid=1000,gid=100,credentials=/home/pti/.smbcredentials   0   0
//nas.snamellit.com/public /mnt/public   cifs   rw,uid=1000,gid=100,credentials=/home/pti/.smbcredentials   0   0
//nas.snamellit.com/multimedia /mnt/multimedia   cifs   rw,uid=1000,gid=100,credentials=/home/pti/.smbcredentials   0   0
...

The first column is the reference to the share as we've seen above. Then the mount point. The 3rd column indicates it is a cifs share, Then the options we've seen with smbmount. Here I have to play with the uid and gid to align my local user and group with the configuration on the nas. The last 2 columns are whether to include the mounts when dumping the filesystem and doing a filesystem check which will probably always be 0 as the CIFS server is responsible for that.

In theory this will mount the shares at boot time, and it probably does, however in my experience it seems they are unmounted on suspend and not remounted after resume. Or there is some timeout. I never investigated I must admit. I just do a quick mount -a before I need them and this works wonders.

On Guix-SD the file-systems are specified in the operating-system section of the system configuration :

(operating-system
 ...

(file-systems
 (cons*
  ...
  (file-system
    (device "//nas.snamellit.com/public")
    (options "uid=1000,gid=1000,credentials=/home/pti/.smbcredentials")
    (mount-point "/mnt/public")
    (type "cifs")
    (mount? #f)
    (create-mount-point? #t))
  (file-system
    (device "//nas.snamellit.com/multimedia")
    (options "uid=1000,gid=1000,credentials=/home/pti/.smbcredentials")
    (mount-point "/mnt/multimedia")
    (type "cifs")
    (mount? #f)
    (create-mount-point? #t))
  (file-system
    (device "//nas.snamellit.com/home")
    (options "uid=1000,gid=1000,credentials=/home/pti/.smbcredentials")
    (mount-point "/home/pti/nas")
    (type "cifs")
    (mount? #f)
    (create-mount-point? #t))
  %base-file-systems)))
 )

Which is just a straightforward translation of the fstab columns in Guix-ese.

Automatic Mounting with AutoFs

Of course the inefficiency of having to type mount -a almost on a weekly basis is unbearable and this inefficiency has to be addressed even if this means we cannot expect net positive effect in the coming millenia.

Enter autofs which is an awesome system to dynamically mount and unmount filesystems on an as needed basis. The kernel will notice when a mounted folder is accessed and ask the autofs daemon to mount the configured mount and unmount it after some timeout occurs without any activity.

This is transparant for the user other than a slight delay when accessing the folder the first time when it is unmounted.

This system is very flexible at it was clearly intended for far more ambitious use-cases than accessing your personal music library from your nas, but that does not mean we cannot strip it down and use it for our purposes.

What is a bit confusing is the configuration. It uses 3 different file types to configure the system.

• the /etc/autofs.conf file which configures the daemon operating options, like the timeouts and the location of the main master file.
• the master file(s), there is always a main file, but this can delegate to directory with additional parts of the file which can be assumed to be imported in the main file. Its main purpose is to map a parent folder of mount points to a map file.
• the map files which map a subfolder in the folder from the line in the master file to a mount specification which will be used when that folder is accessed.

Example on the Tuxedo:

On my Tuxedo (running Tuxedo OS which is an Ubuntu 22.04 derivate) I have the following configuration:

• /etc/autofs.conf:

  ...
  [ autofs ]
  #
  # master_map_name - default map name for the master map.
  #
  master_map_name = /etc/auto.master
  ...
  

  so the master map is in the file /etc/auto.master.

• /etc/auto.master:

  ...
  /-  /etc/autofs.direct  -ro
  

The "/-" is a special case which means "any" directory. In this case the folder name in the map file is assumed to be a fully qualified directory name instead of a subfolder in the folder mentioned in the first column. In this case the map file is /etc/autofs.direct and the default options are -ro .

• /etc/autofs.direct:

  /home/pti/nas -fstype=cifs,rw,noperm,vers=3.0,credentials=/home/pti/.smbcredentials    ://nas.snamellit.com/home
  

  This shows that the mount point /home/pti/nas will mount the share //nas.snamellit.com/home of file system type cifs with the options (the rest of the 2nd column) as mount options.

Example on the Guix-SD desktop

Unfortunately there is not (yet) packaged support for an autofs service-type, so we have to make it ourselves.

First we define a configuration record type for our new service:

(define-record-type* <autofs-configuration>
  autofs-configuration make-autofs-configuration
  autofs-configuration?
  (pid-file autofs-configuration-pid-file
            (default "/var/run/autofs.pid"))
  (autofs-direct autofs-configuration-autofs-direct
                 (default (plain-file "autofs.direct" "/home/pti/autonas -fstype=cifs,rw,noperm,vers=3.0,credentials=/home/pti/.smbcredentials  ://nas.snamellit.com/home"))))

I should probably not put my actual configuration in the default, but it makes my life easier at the moment and I'll refactor it later. (Yeah, sure...)

Since the autofs service needs some boilerplate configuration files I generate them with an activation function:

(define (autofs-activation config)
  "Return the activation GEXP to create the config files for autofs"
  (with-imported-modules '((guix build utils))
                         #~(begin
                             (use-modules (guix build utils)
                                          (ice-9 textual-ports))

                             (define (touch file-name)
                               (call-with-output-file file-name (const #t)))

                             ;; 'sshd' complains if the authorized-key directory and its parents
                             ;; are group-writable, which rules out /gnu/store.  Thus we copy the
                             ;; authorized-key directory to /etc.
                             (call-with-output-file "/etc/autofs.conf"
                               (lambda (f) (put-string f "
[ autofs ]
master_map_name = /etc/auto.master
timeout = 300
")))
                             (call-with-output-file "/etc/auto.master"
                               (lambda (f) (put-string f (format #f "/-  ~a -ro\n" #$(autofs-configuration-autofs-direct config)))))

                             )))

You'll recognize the absolutely stripped down versions of the files mentioned above being generated. The /etc/auto.master file maps the directfs map file to the file generated in the configuration record.

This should tie up nicely the links between the 3 files.

Then I need a shepherd service to start the autofs daemon:

(define (autofs-shepherd-service config)
  (define pid-file (autofs-configuration-pid-file config))

  (list
   (shepherd-service
    (provision '(autofs))
    (documentation "AutoFS Service.")
    (requirement '(networking))
    (start #~(make-forkexec-constructor
              (list
               #$(file-append autofs "/sbin/automount")
               "-f" ;; run in foreground to give shepherd more control
               "-p" #$(autofs-configuration-pid-file config)
               "/etc/auto.master")
              #:pid-file #$(autofs-configuration-pid-file config)))
    (stop #~(make-kill-destructor))
    )))

We point it to the generated auto.master file from the activation function. (I could probably refactor this to use guix instantiated files but at this stage that extra level of indirection would only confuse me and make debugging harder. I find jumping through 3 files already hard enough to follow).

Then we need to put a bow around it and define an autofs-service-type.

(define autofs-service-type
  (service-type (name 'autofs)
                (description "Run the autofs daemon to automount folders on access.")
                (extensions
                 (list
                  (service-extension activation-service-type autofs-activation)
                  (service-extension shepherd-root-service-type autofs-shepherd-service)))
                (compose concatenate)
                (default-value (autofs-configuration)))
  )

This just extends the activation-service-type and the shepherd-root-service-type to run our initialisation code and ensure the autofs daemon gets started.

Then we can add this to our system configuration:

(operating-system
 ...
 (packages
  (append
   (list
    ...
    autofs)))
 (services
  (append
   (list
    ...
    (service autofs-service-type))
   ...)))

i.e. add the autofs package to install the daemon and support stuff and enable the service of type autofs-service-type.

Conclusion

I can now use files on my nas transparantly. I was a bit flippant on my reasons to enable autofs. The real reason was that I want to keep automatic backup copies of my forge running on an VPS somewhere on my NAS with a cron job, which means I would not be there to run a mount -a at the time. (I now realize I could do that as part of the cron job : 20/20 hindsight). In any case this is a major quality of life improvement. As a side effect, my music library now gets properly indexed and is made available on the default music player. Apparently I still use CIFS more than I care to admit.

The big problem with SMB/CIFS is getting the initial connection going. Once that is achieved and the credentials are safely stored away in credentials file, they can be easily reused with a lot less surprising things along the way.

It also pays to test things out in the smallest possible meaningful scope, in this case smbclient before adding more obfuscation layers on top of it, as that just adds more complexity, pitfalls and rabbit holes to get lost in. By going step by step, building on previous result I often get results faster (or at all) even if I have to do additional steps which turn out to no longer be needed in the final solution.