content/lessons/PodmanTestinfo.rst

   1 #####################################
   2 Podman/Testinfra/Salt Lessons Learned
   3 #####################################
   4
   5 :date: 2022-11-13
   6 :tags: lessons,podman,testinfra,salt
   7 :category: lessons
   8 :author: Jude N
   9
  10 Oh my, less than two years between posts.  I'm on a roll !
  11
  12 I've been looking into using `Podman`_ and `Testinfra`_ to test `Salt`_ states.
  13
  14 I'd like to add some unit tests and a Containerfile to an existing repo of salt states, where running 'pytest' in the repo's workspace would spin up the container and run the tests against it, and then tear down the container.
  15
  16 The tests would run 'salt state.apply' commands against the container, applying different sets of pillar data depending on the test.
  17
  18 Project Directory Structure
  19 ===========================
  20
  21 First let's set up a directory structure for the project that includes the states, their tests, and any needed test data.  In the case of salt states, the test dtaa will be pillar files and files served by ext_pillar.  The directory structure below is what I ended up using:
  22
  23 ::
  24
  25    ├─ test_project repo
  26    ├─── .git
  27    ├─── env
  28    ├────── ... testinfra egg
  29    ├─── Containerfile
  30    ├─── setup.cfg
  31    ├─── tests
  32    ├───── test_*.py
  33    ├───── data
  34    ├──────── ext_pillar
  35    ├──────── pillar
  36    ├────────── top.sls
  37    ├────────── test_zero.sls
  38    ├────────── test_one.sls
  39    ├────────── ...
  40    ├──────── top.sls
  41    ├─── test_project
  42    ├───── *.sls
  43    ├───── *.jinja
  44    ├───── templates
  45    ├──────── *.jinja
  46    ├───── files
  47    ├───── ...
  48
  49
  50 Assuming all these files are stored in git, there's a .git directory from when you cloned the repo
  51
  52 The 'env' directory is a python virtualenv under 'env', where the testinfra egg has been installed.  You can skip the virtualenv if you're pulling in testinfra from a global package.
  53
  54 Containerfile is, well a Podman Containerfile, and setup.cfg contains some pytest-specific settings.
  55
  56 The tests directory is where the testinfra test\_\*.py files are stored.
  57
  58 The tests/data/pillar directory will end up be mapped to the /srv/pillar directory in the test container.  Similarly tests/data/ext_pillar will be mapped to /srv/ext_pillar.
  59
  60 The salt-states directory includes the \*.sls and \*.jinja files, and any other salt-related subdirectories like 'templates', 'files', 'macros', etc.  This directory will be mapped to /srv/salt/project in the container.
  61
  62 Containerfile
  63 -------------
  64
  65 The Containerfile I'm using for this project is below.
  66
  67 .. code-block:: docker
  68
  69   # I'm using Ubuntu 20.4 for this project-under-test so pull in the stock Ubuntu image for that version
  70   FROM ubuntu:focal
  71   RUN apt-get update
  72
  73   # The stock image doesn't include curl, so install it and bootstrap salt
  74   # Longterm, I would host the bootstrapping script internally in case that site disappeared.
  75   RUN apt-get install -y curl
  76   RUN curl -L https://bootstrap.saltproject.io | sh -s --
  77
  78   # Configure salt run as a masterless minion
  79   RUN echo "file_client: local" > /etc/salt/minion.d/masterless.conf
  80   RUN printf "local" > /etc/salt/minion_id
  81
  82   # Set up the /srv/salt environment
  83   RUN mkdir -p /srv/salt
  84   RUN mkdir -p /srv/ext_pillar/hosts/local/files
  85   RUN printf "ext_pillar:\n  - file_tree:\n      root_dir: /srv/ext_pillar\n" >>  /etc/salt/minion.d/masterless.conf
  86
  87   # Delay setting up /srv/salt/top.sls until the container starts. so PROJECT can be sent in as a ENV
  88   RUN printf "printf \"base:\\n    '*':\\n      - \${PROJECT}\\n\" > /srv/salt/top.sls" >> /root/.bashrc
  89
  90   # Create a local user
  91   RUN useradd local_user
  92
  93   # The Salt git states apparently assume git is already installed on the host, so install it.
  94   RUN apt-get install -y git
  95
  96 Building and verifying the saltmasterless:latest image
  97 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  98
  99 Using this Containerfile, I built a saltmasterless:latest image:
 100
 101 .. code-block:: shell-session
 102
 103   workstation:~/projects/test_project.git# podman build -t saltmasterless:latest .
 104
 105 Then with this image, I can start a container that includes volumes mapping the tests/data/pillar ro /srv/pillar, tests/data/ext_pillar to /srv/ext_pillar, and test_project to /srv/salt:
 106
 107 .. code-block:: shell-session
 108
 109   workstation:~/projects/test_project.git# podman run -it --env "PROJECT=test_project" -v ${PWD}/test_project:/srv/salt/test_project -v ${PWD}/tests/data/pillar:/srv/pillar -v ${PWD}/tests/data/ext_pillar:/srv/ext_pillar/hosts/local/files --name test_box --hostname local saltmasterless:latest
 110   root@local:/#
 111   root@local:/# find /srv
 112   root@local:/# exit
 113   workstation:~/projects/test_project.git# podman rm -f test_box
 114
 115 setup.cfg
 116 ---------
 117 The setup.cfg file is mostly used to tell pytest to ignore the salt states directory:
 118
 119 .. code-block
 120
 121   [tool:pytest]
 122   norecursedirs = test_project/files/*
 123   addopts = -s
 124   log_cli=true
 125   log_level=NOTSET
 126
 127 tests/data/pillar/top.sls
 128 -------------------------
 129 As mentioned above the tests/data/pillar directory will be mapped to /srv/pillar in the container, but let's look at the top.sls a little closer.  From the Containerfile, /etc/salt/minion_id was set to 'local', so normally the top.sls file will end up using /srv/pillar/test_zero.sls for it's pillar data.
 130
 131 But lets say we want to run a test with some other pillar data.  In that case , in the test we'll use the salt-call '-id' argument to run the command as a different minion id.  So with the top.sls file below, running 'salt-call --local -id=test_one  state.apply' will use the test_one.sls pillar data instead of test_zero.sls
 132
 133 .. code-block:: yaml+jinja
 134
 135   {{ saltenv }}:
 136
 137     '*':
 138     - match: glob
 139     - ignore_missing: True
 140
 141     'local':
 142     - test_zero
 143
 144     'test_one':
 145     - test_one
 146
 147     'missing_mandatory_pillar_item':
 148     - missing_mandatory_pillar_item
 149
 150
 151 tests/test_project.py host fixture
 152 ----------------------------------
 153
 154 The tests/test_project.py file includes a host fixture based on https://testinfra.readthedocs.io/en/latest/examples.html#test-docker-images.  Note that the podman_cmd is pretty much the same as the command used above when testing the container.  The cwd-related logic is because the -v args required full path names.
 155
 156 .. code-block:: python
 157
 158   # scope='session' uses the same container for all the tests;
 159   # scope='function' uses a new container per test function.
 160   @pytest.fixture(scope='session')
 161   def host(request):
 162
 163       cwd = os.getcwd()
 164
 165       podman_cmd = "podman run -d -it --env PROJECT=test_project -v ${PWD}/test_project:/srv/salt/test_project -v ${PWD}/tests/data/pillar:/srv/pillar -v ${PWD}/tests/data/ext_pillar:/srv/ext_pillar/hosts/local/files --name test_box --hostname local saltmasterless:latest bash"
 166       podman_cmd = podman_cmd.replace("${PWD}",cwd)
 167       podman_cmd_list = podman_cmd.split(' ')
 168
 169       # run a container
 170       podman_id = subprocess.check_output(podman_cmd_list).decode().strip()
 171       # return a testinfra connection to the container
 172       yield testinfra.get_host("podman://" + podman_id)
 173
 174       # at the end of the test suite, destroy the container
 175       subprocess.check_call(['podman', 'rm', '-f', podman_id])
 176
 177 tests/test_project.py full salt run test
 178 ----------------------------------------
 179
 180 Here's a test that does a full salt state.apply on the container.  This test is slow, since the container starts are with just salt and git installed, and the project-under-test is making a lot of changes. Note theuse of the '--local' argument to tell salt to try to pull data from a saltmaster.
 181
 182 .. code-block:: python
 183
 184   def test_full_salt_run(host):
 185       print('running salt-call state.apply.  This will take a few minutes')
 186       cmd_output = host.run('salt-call --state-output=terse --local state.apply')
 187
 188       print('cmd.stdout: ' + cmd_output.stdout)
 189
 190       assert cmd_output.rc == 0
 191       assert cmd_output.stderr == ''
 192
 193 tests/test_project.py alternative pillar data test
 194 ---------------------------------------------------
 195
 196 In this example, suppose ./test_project/map.jinja included a check like below:
 197
 198 .. code-block:: jinja
 199
 200    {% if not salt['pillar.get']('mandatory_pillar_item') %}
 201      {{ raise('mandatory_pillar_item is mandatory') }}
 202    {% endif %}
 203
 204 And then there's a 'missing_mandatory_pillar_item' in the ./test/data/pillar/top.sls as per above, and a ./test/data/pillar/missing_mandatory_pillar_item.sls file exists that's missing the mandatory pillar item.
 205
 206 Then a test like below could force a salt run that uses this pillar data by using the '--id' argument as per below, and an assertion could check the error was raised.
 207
 208 .. code-block:: python
 209
 210   def test_missing_mandatory_pillar_itemn(host):
 211       print('running another salt-call state.apply with bad pillar data.')
 212       cmd_output = host.run('salt-call --state-output=terse --local --id=missing_mandatory_pillar_item state.apply')
 213       assert "mandatory_pillar_item is mandatory" in cmd_output.stderr
 214       assert cmd_output.rc != 0
 215
 216 Running the tests
 217 =================
 218
 219 The tests are kicked off via 'pytest' like any other python project using pytest.`
 220
 221 .. code-block:: shell-session
 222
 223   workstation:~/projects/test_project.git# source ./env/bin/activate
 224   (env) workstation:~/projects/test_project.git# pytest
 225   ...
 226   ================================================================================ 3 passed in 333.88s (0:05:33) ================================================================================
 227
 228
 229 What's Next
 230 ===========
 231
 232 - Set up the salt bootstrapping so it'll work without having to reach out to bootstrap.saltproject.io
 233 - Move the host fixture out of /tests/test_project.py to ./tests/conftest.py
 234 - Speed up the tests.  As mentioned above, a full 'salt state.apply' for a project can take a few minutes on my workstation
 235
 236 .. _Podman: https://podman.io/
 237 .. _Testinfra: https://github.com/pytest-dev/pytest-testinfra
 238 .. _Salt: https://saltproject.io/
 239