1 #####################################
2 Podman/Testinfra/Salt Lessons Learned
3 #####################################
6 :tags: lessons,podman,testinfra,salt
10 Oh my, less than two years between posts. I'm on a roll !
12 I've been looking into using `Podman`_ and `Testinfra`_ to test `Salt`_ states.
14 I'd like to add some unit tests and a Containerfile to an existing repo of salt states, where running 'pytest' in the repo's workspace would spin up the container and run the tests against it, and then tear down the container.
16 The tests would run 'salt state.apply' commands against the container, applying different sets of pillar data depending on the test.
18 Project Directory Structure
19 ===========================
21 First let's set up a directory structure for the project that includes the states, their tests, and any needed test data. In the case of salt states, the test dtaa will be pillar files and files served by ext_pillar. The directory structure below is what I ended up using:
28 ├────── ... testinfra egg
37 ├────────── test_zero.sls
38 ├────────── test_one.sls
50 Assuming all these files are stored in git, there's a .git directory from when you cloned the repo
52 The 'env' directory is a python virtualenv under 'env', where the testinfra egg has been installed. You can skip the virtualenv if you're pulling in testinfra from a global package.
54 Containerfile is, well a Podman Containerfile, and setup.cfg contains some pytest-specific settings.
56 The tests directory is where the testinfra test\_\*.py files are stored.
58 The tests/data/pillar directory will end up be mapped to the /srv/pillar directory in the test container. Similarly tests/data/ext_pillar will be mapped to /srv/ext_pillar.
60 The salt-states directory includes the \*.sls and \*.jinja files, and any other salt-related subdirectories like 'templates', 'files', 'macros', etc. This directory will be mapped to /srv/salt/project in the container.
65 The Containerfile I'm using for this project is below.
67 .. code-block:: docker
69 # I'm using Ubuntu 20.4 for this project-under-test so pull in the stock Ubuntu image for that version
73 # The stock image doesn't include curl, so install it and bootstrap salt
74 # Longterm, I would host the bootstrapping script internally in case that site disappeared.
75 RUN apt-get install -y curl
76 RUN curl -L https://bootstrap.saltproject.io | sh -s --
78 # Configure salt run as a masterless minion
79 RUN echo "file_client: local" > /etc/salt/minion.d/masterless.conf
80 RUN printf "local" > /etc/salt/minion_id
82 # Set up the /srv/salt environment
83 RUN mkdir -p /srv/salt
84 RUN mkdir -p /srv/ext_pillar/hosts/local/files
85 RUN printf "ext_pillar:\n - file_tree:\n root_dir: /srv/ext_pillar\n" >> /etc/salt/minion.d/masterless.conf
87 # Delay setting up /srv/salt/top.sls until the container starts. so PROJECT can be sent in as a ENV
88 RUN printf "printf \"base:\\n '*':\\n - \${PROJECT}\\n\" > /srv/salt/top.sls" >> /root/.bashrc
91 RUN useradd local_user
93 # The Salt git states apparently assume git is already installed on the host, so install it.
94 RUN apt-get install -y git
96 Building and verifying the saltmasterless:latest image
97 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
99 Using this Containerfile, I built a saltmasterless:latest image:
101 .. code-block:: shell-session
103 workstation:~/projects/test_project.git# podman build -t saltmasterless:latest .
105 Then with this image, I can start a container that includes volumes mapping the tests/data/pillar ro /srv/pillar, tests/data/ext_pillar to /srv/ext_pillar, and test_project to /srv/salt:
107 .. code-block:: shell-session
109 workstation:~/projects/test_project.git# podman run -it --env "PROJECT=test_project" -v ${PWD}/test_project:/srv/salt/test_project -v ${PWD}/tests/data/pillar:/srv/pillar -v ${PWD}/tests/data/ext_pillar:/srv/ext_pillar/hosts/local/files --name test_box --hostname local saltmasterless:latest
111 root@local:/# find /srv
113 workstation:~/projects/test_project.git# podman rm -f test_box
117 The setup.cfg file is mostly used to tell pytest to ignore the salt states directory:
122 norecursedirs = test_project/files/*
127 tests/data/pillar/top.sls
128 -------------------------
129 As mentioned above the tests/data/pillar directory will be mapped to /srv/pillar in the container, but let's look at the top.sls a little closer. From the Containerfile, /etc/salt/minion_id was set to 'local', so normally the top.sls file will end up using /srv/pillar/test_zero.sls for it's pillar data.
131 But lets say we want to run a test with some other pillar data. In that case , in the test we'll use the salt-call '-id' argument to run the command as a different minion id. So with the top.sls file below, running 'salt-call --local -id=test_one state.apply' will use the test_one.sls pillar data instead of test_zero.sls
133 .. code-block:: yaml+jinja
139 - ignore_missing: True
147 'missing_mandatory_pillar_item':
148 - missing_mandatory_pillar_item
151 tests/test_project.py host fixture
152 ----------------------------------
154 The tests/test_project.py file includes a host fixture based on https://testinfra.readthedocs.io/en/latest/examples.html#test-docker-images. Note that the podman_cmd is pretty much the same as the command used above when testing the container. The cwd-related logic is because the -v args required full path names.
156 .. code-block:: python
158 # scope='session' uses the same container for all the tests;
159 # scope='function' uses a new container per test function.
160 @pytest.fixture(scope='session')
165 podman_cmd = "podman run -d -it --env PROJECT=test_project -v ${PWD}/test_project:/srv/salt/test_project -v ${PWD}/tests/data/pillar:/srv/pillar -v ${PWD}/tests/data/ext_pillar:/srv/ext_pillar/hosts/local/files --name test_box --hostname local saltmasterless:latest bash"
166 podman_cmd = podman_cmd.replace("${PWD}",cwd)
167 podman_cmd_list = podman_cmd.split(' ')
170 podman_id = subprocess.check_output(podman_cmd_list).decode().strip()
171 # return a testinfra connection to the container
172 yield testinfra.get_host("podman://" + podman_id)
174 # at the end of the test suite, destroy the container
175 subprocess.check_call(['podman', 'rm', '-f', podman_id])
177 tests/test_project.py full salt run test
178 ----------------------------------------
180 Here's a test that does a full salt state.apply on the container. This test is slow, since the container starts are with just salt and git installed, and the project-under-test is making a lot of changes. Note theuse of the '--local' argument to tell salt to try to pull data from a saltmaster.
182 .. code-block:: python
184 def test_full_salt_run(host):
185 print('running salt-call state.apply. This will take a few minutes')
186 cmd_output = host.run('salt-call --state-output=terse --local state.apply')
188 print('cmd.stdout: ' + cmd_output.stdout)
190 assert cmd_output.rc == 0
191 assert cmd_output.stderr == ''
193 tests/test_project.py alternative pillar data test
194 ---------------------------------------------------
196 In this example, suppose ./test_project/map.jinja included a check like below:
198 .. code-block:: jinja
200 {% if not salt['pillar.get']('mandatory_pillar_item') %}
201 {{ raise('mandatory_pillar_item is mandatory') }}
204 And then there's a 'missing_mandatory_pillar_item' in the ./test/data/pillar/top.sls as per above, and a ./test/data/pillar/missing_mandatory_pillar_item.sls file exists that's missing the mandatory pillar item.
206 Then a test like below could force a salt run that uses this pillar data by using the '--id' argument as per below, and an assertion could check the error was raised.
208 .. code-block:: python
210 def test_missing_mandatory_pillar_itemn(host):
211 print('running another salt-call state.apply with bad pillar data.')
212 cmd_output = host.run('salt-call --state-output=terse --local --id=missing_mandatory_pillar_item state.apply')
213 assert "mandatory_pillar_item is mandatory" in cmd_output.stderr
214 assert cmd_output.rc != 0
219 The tests are kicked off via 'pytest' like any other python project using pytest.`
221 .. code-block:: shell-session
223 workstation:~/projects/test_project.git# source ./env/bin/activate
224 (env) workstation:~/projects/test_project.git# pytest
226 ================================================================================ 3 passed in 333.88s (0:05:33) ================================================================================
232 - Set up the salt bootstrapping so it'll work without having to reach out to bootstrap.saltproject.io
233 - Move the host fixture out of /tests/test_project.py to ./tests/conftest.py
234 - Speed up the tests. As mentioned above, a full 'salt state.apply' for a project can take a few minutes on my workstation
236 .. _Podman: https://podman.io/
237 .. _Testinfra: https://github.com/pytest-dev/pytest-testinfra
238 .. _Salt: https://saltproject.io/