website/content/blog/20240624T104859--secrets-management-using-unix-password-store-pass__linux_osx_sysadmin.md

+++
title = "Workflows for Unix Password Store with Emacs and Shell"
date = 2024-06-25
[taxonomies]
tags = ["programming", "guix", "debian", "linux", "mac", "ubuntu"]
categories = ["lisp", "shell", "apps"]
+++

# Table of Contents

1.  [Managing Secrets using Pass](#orgeea54c9)
2.  [Installation](#org44c3160)
    1.  [GUIX](#org7920015)
    2.  [Debian, Ubuntu et al](#orgbee77ce)
3.  [Emacs Integration](#orgad61fe8)
    1.  [Enable the Unix Password Store.](#orgf739a63)
    2.  [Helper Function](#org1c798ff)
    3.  [Using in Emacs Configuration](#org25108ee)
4.  [Shell integration](#org849f72f)
    1.  [DirEnv integration](#org299dd7c)
5.  [Tips](#org158dbf2)
    1.  [Entering passwords on the terminal](#org00fffd0)
    2.  [Getting fields from multi field secrets](#org6ab1e11)



<a id="orgeea54c9"></a>

# Managing Secrets using Pass

On UNIX there is a well known tool to manage secrets called
**password-store** or **pass** for short.

It is a very minimal tool, more to facilitate workflows that to do
real work, very much in the UNIX philosophy. It stores all secrets in
plain files in a folder structure. It does not care about what is in
the files and encrypts them using a GPG public key so only the owner
of the private key can decrypt them. It does offer special access to
the first line so a password can be quickly fetched and copied to the
clipboard or stdout or wherever some **pass** aware integration needs it.

Certain folders can be configured to use a different public keys to
allow pragmatic secret delegation to different systems without
exposing all secrets.

It leverages the **gpg** infrastructure for key management, distribution,
unlocking with **pinentry**, caching with **gpg-agent**.


<a id="org44c3160"></a>

# Installation


<a id="org7920015"></a>

## GUIX

Add **password-store**, **gpg** and **pinentry** to your package list.

The **pinentry** program provides a client to securely unlock your keys in
a GUI and terminal environment. There are other options to make it
better fit your environment, but this is fine for me.


<a id="orgbee77ce"></a>

## Debian, Ubuntu et al

Add **pass**, **gpg**, **gpg-agent** and **pinentry-gnome** of **pinentry-qt** using

    $ apt-get install pass gpg gpg-agent pinentry-qt

Both the gnome and qt versions at least fall back gracefully if no
graphical environment are available.

The current selected pinentry program is provided as
**/usr/bin/pinentry** and this link is managed by the usual alternatives
machinery in debian based distros.


<a id="orgad61fe8"></a>

# Emacs Integration

Emacs **auth-source** infrastructure supports **pass** out of the box. 


<a id="orgf739a63"></a>

## Enable the Unix Password Store.

We have to make sure the password-store is added to the
auth-sources. There is a handy function for that:

    ;; enable unix password-store
    (auth-source-pass-enable)

We add that somewhere in the init.el before any secrets are needed.


<a id="org1c798ff"></a>

## Helper Function

Auth-sources is a flexible system and works like a database, i.e. you
can query, browse through results and have multiple fields per secret.

Example:

    (defun snam-password (host user)
      "Get password from the unix password store.
    
    Searches the password file for a secret in the folder corresponding to
    the HOST name given, which is the folder with the '/' replaced by a '.'.
    The filename in the folder corresponds to the USER argument with a
    '.pgp' extension."
      (auth-info-password
       (car (auth-source-search
             :max 1
             :host host
             :user user))))

\`auth-source-search\` returns a list of results, so we have to get the
first entry with \`car\`. The secret is lightly obfuscated, hence the
need to decode it with the \`auth-info-password\` function.

Luckily there is a function [auth-source-pass-get](help:auth-source-pass-get) to get a password
from the password store which also follows the recommended conventions
for multiple fields in a pass file.

    (auth-source-pass-get 'secret "snamellit/znc")

The pseudo key \`'secret\` returns the first line of the password store
entry which contains the password, per **pass** conventions.


<a id="org25108ee"></a>

## Using in Emacs Configuration

For single secrets, like connecting to my **znc** IRC bouncer:

    (erc-tls :id 'znc :server "********"
             :port "****" :user "xyz" :nick
             "foobar" :password (auth-source-pass-get 'secret "foobar/znc")))

For multifield secrets, like for google authentication, we can
leverage the multiple fields in the password store. Here is my
**org-gcal** configuration:

    (setq org-gcal-client-id (auth-source-pass-get 'secret "snamellit/org-gcal-client")
          org-gcal-client-secret (auth-source-pass-get "id" "snamellit/org-gcal-client")
          org-gcal-fetch-file-alist '(("xyz@foobar.com" .  "~/org/schedule.org")))


<a id="org849f72f"></a>

# Shell integration


<a id="org299dd7c"></a>

## DirEnv integration

When developping 12-factor or similar inspired apps, the configuration
is passed using environment variables. Using **.envrc** files with
**direnv** integration in the shell is a very smooth way to work in
multiple projects.

It is of course less than ideal to have the secrets exposed in the
**.envrc** files in your project tree even if it is in the **.gitignore**
file, although that is infinitely better than having secrets end up in
the git repository.

Secrets in the **.envrc** files can be easily moved to the password store
by entering the folder. The following snippet prints the current value
to the screen and then inserts it in the password store, and verifies
it actually is entered correctly.

    $ echo $FOO_BAR_PASSWORD
    $ echo $FOO_BAR_PASSWORD | pass add -e foo/bar
    $ pass foo/bar

and then editing the **.envrc** file from

    ...
    FOO_BAR_PASSWORD=<some secret>
    ...

to

    ...
    FOO_BAR_PASSWORD=$(pass foo/bar)
    ...

After modification you'll be asked to allow to read the new **.envrc**
file content and you can check if it still works by comparing the
password with the one printed previously.

Printing the passwords allows to fix any typos. If this are the only
copies you have of them you might rug-pull yourself. Ideally it should
be possible to quickly recreate secrets if you lose any, but reality
is often far from ideal. Echoing all these passwords is also not
ideal, but preferable over keeping a little black book of secrets. If
these passwords are coming from another password manager it is not
needed of course.

Do not forget to clear your terminal scroll back history with

    $ clear

To help migration of **.envrc** files I created a bash script I stored in
**~/.local/bin/envrc-to-pass**:

    #!/bin/bash
    
    if [ "$#" -ne 2 ]; then
        echo "Usage: $0 <variable_name> <namespace>" >&2
        exit 1
    fi
    
    VAR_NAME=$1
    SLUG=$(echo $VAR_NAME | sed 's/_/-/g' | tr '[:upper:]' '[:lower:]')
    NAMESPACE=$2
    
    
    echo "Moving variable $VAR_NAME to $NAMESPACE/$SLUG in password store"
    
    SECRET=$(grep "$VAR_NAME=" .envrc | cut -d'=' -f2)
    if (echo $SECRET | grep '^\$(pass.*)'); then
        echo "secret already migrated"
    else
        echo $SECRET | pass insert -e $NAMESPACE/$SLUG
    fi
    
    sed -i "s#$VAR_NAME=.*#$VAR_NAME=\\\$(pass $NAMESPACE\/$SLUG)#" .envrc

This makes short work of migrating projects to use the password-store.


<a id="org158dbf2"></a>

# Tips


<a id="org00fffd0"></a>

## Entering passwords on the terminal

Usually invoking \`pass add foobar/baz\` will ask to enter the password
and confirm it in the shell.

However when piping a secret into the password-store
with

    echo FOO_BAR_PASSWORD | pass add foo/bar

will silently fail although the man pages tell that \`pass add\` will
read from **stdin**. It is not clear IMO that you have to specify the **-e**
flag like :

    echo FOO_BAR_PASSWORD | pass -e add foo/bar

to suppress the confirmation and make it work as expected.


<a id="org6ab1e11"></a>

## Getting fields from multi field secrets

Often secrets come in multiple parts which are nice to be stored in a
single entry in order not to complicate the tree. The **pass**
documentation suggests:

    <password or main secret>
    field1: <some data>
    field2: <more data>

the main secret is just the first line, and can have the same
structure as the other lines, if that makes more sense.

e.g. for a google integration:

    service-account: 1234567-abcdefghijklm@developer.gserviceadmin.com(some-project)
    email: xyz@foobar.com
    private-key: ...

In this case it is useful to <span class="underline">document</span> which kind of secret it is as it
could also be an api-key, or a refresh-token, or whatever part of the
authentication menagerie that Google offers.

To get these fields individually I use:

    GOOGLE_EMAIL=$(pass foo/bar | awk '/^email:/ {print $2}) 

in emacs this syntax is supported directly and the same info can be
fetched with

    (let
        ((google-email (auth-source-pass-get "email" "foo/bar")))
      ... )