--- /dev/null
+################################################
+An Exploration into Jinja2 Unit Tests Wth Pytest
+################################################
+
+:date: 2023-11-20
+:tags: lessons,jinja2,pytest,salt
+:category: lessons
+:author: Jude N
+
+This post covers an approach I've used for add pytest-style unit tests to my Salt-related projects.
+
+The content below may be specific to Salt, but the testing techniques should work on any project using Jinja2.
+
+Project Directory Structure
+===========================
+
+In my existing salt project repo, I added a tests subdirectory, a setup.cfg with the contents below, and added a test target to the project's Makefile.
+
+I also installed the pytest and jinja2 eggs in a virtualenv in my working directory.
+
+::
+
+ ├─ test_project repo
+ ├─── .git
+ ├─── init.sls
+ ├─── map.jinja
+ ├─── templates
+ ├─────── some_template.cfg
+ ├─── tests
+ ├─────── conftest.py
+ ├─────── test_init.py
+ ├─────── test_map.py
+ ├─────── test_some_template.py
+ ├─── setup.cfg
+ ├─── Makefile
+ ├─── env
+ ├─────── ... pytest
+ ├─────── ... jinja2
+
+Here's a snippet of the Makefile for kicking off the tests:
+
+Note that since the tests are python code, you should run them through whatever linters and style-checkers you're using on the rest of your python code.
+
+::
+
+ test:
+ py.test-3
+ pycoderstyle ./tests/*.py
+
+
+And here's the setup.cfg:
+
+Without the extra '--tb=native' argument, pytest would sometimes through an internal error when jinja ended up throwing exception, as we'll see later below.
+
+::
+
+ [tool:pytest]
+ python_files = test_*.py tests/__init__.py tests/*/__init__.py
+ #uncomment the line below for full unittest diffs
+ addopts =
+ # Any --tb option except native (including no --tb option) throws an internal pytest exception
+ # jinja exceptions are thrown
+ --tb=native
+ # Uncomment the next line for verbose output
+ # -vv
+
+ [pycodestyle]
+ max-line-length=999
+ ignore=E121,E123,E126,E226,E24,E704,E221,E127,E128,W503,E731,E131,E402
+
+
+Note there is a test_*.py file for each file that includes Jinja2 markup.
+
+tests/conftest.py
+=================
+
+The conftest.py contains the common fixtures used by the tests. I've tried adding docstring
+comments to explain how to use the fixtures, but also see the examples.
+
+::
+
+ import pytest
+ from unittest.mock import Mock
+
+ import jinja2
+ from jinja2 import Environment, FileSystemLoader, ChoiceLoader, DictLoader, StrictUndefined
+
+
+ class RaiseException(Exception):
+ """ Exception raised when using raise() in the mocked Jinja2 context"""
+ pass
+
+ @pytest.fixture()
+ def mocked_templates(request):
+ """ A dict of template names to template content.
+ Use this to mock out Jinja 'import "template" as tmpl" lines.
+ """
+ mocked_templates = {}
+ return mocked_templates
+
+
+ @pytest.fixture(autouse=True)
+ def jinja_env(request, mocked_templates):
+ """ Provide a Jinja2 environment for loading templates.
+ The ChoiceLoader will first check when mocking any 'import' style templates,
+ Then the FileSystemLoader will check the local file system for templates.
+
+ The DictLoader is first so the Jinja won't end up using the FileSystemLoader for
+ templates pulled in with an import statement which doesn't include the 'with context'
+ modifier.
+
+ Setting undefined to StrictUndefined throws exceptions when the templates use undefined variables.
+ """
+
+ test_loader=ChoiceLoader([
+ DictLoader(mocked_templates),
+ FileSystemLoader('.'),
+ ])
+
+ env = Environment(loader=test_loader,
+ undefined=StrictUndefined,
+ extensions=['jinja2.ext.do', 'jinja2.ext.with_', 'jinja2.ext.loopcontrols'])
+
+ return env
+
+
+ @pytest.fixture(scope='session', autouse=True)
+ def salt_context():
+ """ Provide a set of common mocked keys.
+ Currently this is only the 'raise' key for mocking out the raise() calls in the templates,
+ and an empty 'salt' dict for adding salt-specific mocks.
+ """
+
+ def mocked_raise(err):
+ raise RaiseException(err)
+
+ context = {
+ 'raise': mocked_raise,
+ 'salt': {}
+ }
+
+ return context
+
+init.sls
+========
+For purposes of the sections below, here's what the init.sls looks like:
+
+::
+
+ #!jinja|yaml
+ # {% set version = salt['pillar.get']('version', 'latest') %}
+ # version: {{ version }}
+
+ # {% if version == 'nope' %}
+ # {{ raise("OH NO YOU DIDN'T") }}
+ # {% endif %}
+
+
+
+Mock out the Jinja Context
+==========================
+
+Let's test out that rendering init.sls should return a version key with some value.
+
+Being able to mock out the salt pillar.get() function was a big breakthrough with respect to being able to write any sort of unit tests for the Salt states.
+
+::
+
+ @pytest.fixture
+ def poc_context(self, salt_context):
+ """ Provide a proof-of-concept context for mocking out salt[function](args) calls """
+ poc_context = salt_context.copy()
+
+ def mocked_pillar_get(key,default):
+ """ Mocked salt['pillar.get'] function """
+ pillar_data = {
+ 'version' : '1234'
+ }
+ return pillar_data.get(key, default)
+
+ # This is the super sauce:
+ # We can mock out the ``salt['function'](args)`` calls in the salt states by
+ # defining a 'salt' dict in the context, who's keys are the functions, and the values of mocked functions
+ poc_context['salt']['pillar.get'] = mocked_pillar_get
+
+ return poc_context
+
+
+ def test_jinja_template_poc(self, jinja_env, poc_context):
+ """ Render a template and check it has the expected content """
+
+ # This assumes the tests are run from the root of the project.
+ # The conftest.py file is setting the jinja_env to look for files under the 'latest' directory
+ template = jinja_env.get_template('init.sls')
+
+ # Return a string of the rendered template.
+ result = template.render(poc_context)
+
+ # Now we can run assertions on the returned rendered template.
+ assert "version: 1234" in result
+
+
+Mocking a raise() error
+=======================
+
+Now, let's see how we can test triggering the raise() error based on the pillar data:
+
+::
+
+ @pytest.fixture
+ def bad_context(self, salt_context):
+ """ Lets see what happens if the template triggers a raise() """
+
+ # The base salt_context from conftest.py includes a 'raise' entry that raises a RaiseException
+ bad_context = salt_context.copy()
+ bad_context['salt']['pillar.get'] = lambda k, d: 'nope'
+ return bad_context
+
+ def test_raise_poc(self, jinja_env, bad_context):
+ """ Try rendering a template that should fail with some raise() exception """
+
+ with pytest.raises(RaiseException) as exc_info:
+ template = jinja_env.get_template('init.sls')
+ result = template.render(bad_context)
+
+ raised_exception = exc_info.value
+ assert str(raised_exception) == "OH NO YOU DIDN'T"
+
+
+Mocking imported templates
+==========================
+
+Sometimes the Jinja templates may try import other templates which are either out of scope with respect to the current project, or the import doesn't include the 'with context' modifier, so the Jinja context isn't available when rendering the template.
+
+In this case we can used with DictLoader portion of the jinja_env to mock out importing the template.
+
+In this example, lets assume the following template file exists in the templates directory:
+
+::
+
+ {%- import 'missing.tmpl' as missing -%}
+ Can we mock out missing/out of scope imports ?
+
+ Mocked: {{ missing.perhaps }}
+ Macro Call: {{ missing.lost('forever' }}
+
+Now here is a test that can mock out the missing.tmpl contents, including the lost() macro call:
+
+::
+
+ def test_missing_template(self, jinja_env, mocked_templates, salt_context):
+ """
+ In this example, templates/missing-import.tmpl tries to import a non-available 'missing.tmpl' template.
+ The ChoiceLoader checks DictLoader loader, which checks mocked_templates and finds a match
+ """
+
+ mocked_templates['missing.tmpl'] = """
+ {% set perhaps="YES" %}
+ {% macro lost(input) %}MOCKED_LOST{% endmacro %}
+ """
+ missing_template = jinja_env.get_template('templates/missing-import.tmpl')
+ missing_result = missing_template.render(salt_context)
+ assert "Mocked: YES" in missing_result
+ assert "Macro Call: MOCKED_LOST" in missing_result
+
+
+Mocking a macro call
+====================
+
+Let's say I have a Jinja2 macro defined below:
+
+::
+
+ #!jinja|yaml
+
+ # {% macro test_macro(input) %}
+ # {% if input == 'nope' %}
+ # {{ raise("UNACCEPTABLE") }}
+ # {% endif %}
+ # {% set version = salt['pillar.get']('version', 'latest') %}
+ "macro sez {{ input }}":
+ test.show_notification:
+ - text: "{{ input }}"
+
+ "version sez {{ version }}":
+ test.show_notifications:
+ - text: "{{ version }}"
+
+ # {% endmacro %}
+
+Testing out this macro is a little more involved, since first we have to include calls to the macro after rendering the template. Note we're reusing the poc_context fixture defined earlier so the pillar.get() call is still mocked out to return 1234 for the version"
+
+::
+
+ def test_get_pillar_from_macro(self, jinja_env, poc_context):
+ """
+ If we want to reference the mocked context in the macros, we need
+ to render the source + macro call within a context.
+ """
+
+ # The '[0]' is because get source returns a (source,filename,up-to-date) tuple.
+ template_source = jinja_env.loader.get_source(jinja_env, 'macro.sls')[0]
+ new_template = jinja_env.from_string(template_source + "{{ test_macro('hello') }}")
+ result = new_template.render(poc_context)
+
+ assert "macro sez hello" in result
+ assert "version sez 1234" in result
+
+It's also possible to check that the macro raises an error based on the input:
+
+::
+
+ def test_raise_from_macro(self, jinja_env, salt_context):
+ """
+ In this test, try forcing a raise() from within a macro
+ """
+
+ with pytest.raises(RaiseException) as exc_info:
+ template_source = jinja_env.loader.get_source(jinja_env, 'macro.sls')[0]
+ new_template = jinja_env.from_string(template_source + "{{ test_macro('nope') }}")
+ result = new_template.render(salt_context)
+
+ raised_exception = exc_info.value
+ assert str(raised_exception) == "UNACCEPTABLE"
+
+FECUNDITY: Checking for undefined variables during template rendering
+=====================================================================
+Back in the day I learned that one of the virtues of a scientific theory was 'fecundity', or the ability for the theory to predict new behavior the original theory hadn't considered.
+
+It looks like this may be called `fruitfulness`_ now, but still whenever I stumble across something like this, I shout out 'FECUNDITY' internally to myself. :shrug:
+
+While I was working on this project, I noticed the jinja Environment constructor has an `undefined`_ argument that defaulted to Undefined. I also noticed `StrictUndefined`_ was another value that the undefined argument could use.
+
+It would be useful if the tests could throw exceptions when they ran into undefined variables. This could happen from typos in the templates, or possibly not mocking out all the globals variables used in a template.
+
+So I tried making an jinja Environment with undefined=StrictUndefined, and wrote a test with template with a typo in a variable name to see if the test would raise an exception, and it did !
+
+This is much more useful than the default jinja behavior where Jinja would give the typo a value of None and would likely surface in the output as a empty string.
+
+It's also more useful than setting undefined to `DebugUndefined`_, which sometimes raised an exception, but sometimes left the un-modified '{{ whatever }}' strings in the rendered templates. Bleh.
+
+Here's the sample template I used, called unexpected_variable.sls. It's the same as the original init.sls, but with a 'verion' typo:
+
+::
+
+ #!jinja|yaml
+
+ # {% set version = salt['pillar.get']('role:echo-polycom:version', 'latest') %}
+ # version: {{ version }}
+
+ # {% if verion == 'nope' %}
+ # {{ raise("OH NO YOU DIDN'T") }}
+ # {% endif %}
+
+And let's try adding this test, which is the same as the earlier test_jinja_template_poc() test, but with the buggy template:
+
+::
+
+ def test_unexpected_variable(self, jinja_env, poc_context):
+ """ Render a template and check it has the expected content """
+
+ # This assumes the tests are run from the root of the project.
+ # The conftest.py file is setting the jinja_env to look for files under the 'latest' directory
+ template = jinja_env.get_template('unexpected_variable.sls')
+
+ # Return a string of the rendered template.
+ result = template.render(poc_context)
+
+ # Now we can run assertions on the returned rendered template.
+ assert "version: 1234" in result
+
+This test will fail with an undefined error exception below !
+Cool. I can fix the typo, and rerun the test to get it passing again ! FECUNDITY !
+
+::
+
+ ==================================================== FAILURES =======================================================
+ _________________________________________ TestJinja.test_unexpected_variable __________________________________________
+ Traceback (most recent call last):
+ File "/my/working/dir/test_jinja_template_poc.py", line 150, in test_unexpected_variable
+ result = template.render(poc_context)
+ File "/usr/lib/python3.6/site-packages/jinja2/asyncsupport.py", line 76, in render
+ return original_render(self, *args, **kwargs)
+ File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 1008, in render
+ return self.environment.handle_exception(exc_info, True)
+ File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 780, in handle_exception
+ reraise(exc_type, exc_value, tb)
+ File "/usr/lib/python3.6/site-packages/jinja2/_compat.py", line 37, in reraise
+ raise value.with_traceback(tb)
+ File "unexpected_variable.sls", line 6, in top-level template code
+ # {% if verion == 'nope' %}
+ jinja2.exceptions.UndefinedError: 'verion' is undefined
+ ========================================= 1 failed, 5 passed in 0.89 seconds ==========================================
+
+
+Running the tests
+=================
+
+The tests are kicked off via 'pytest' like any other python project using pytest.
+
+.. code-block:: shell-session
+
+ workstation:~/projects/test_project.git# source ./env/bin/activate
+ (env) workstation:~/projects/test_project.git# pytest
+ ===================================================================== test session starts =====================================================================
+ platform linux -- Python 3.6.8, pytest-2.9.2, py-1.4.32, pluggy-0.3.1
+ rootdir: /vagrant, inifile:
+ plugins: catchlog-1.2.2
+ collected 5 items
+
+ latest/tests/test_jinja_template_poc.py .....
+
+Credit
+======
+
+I based this work on some ideas from the blog post `A method of unit testing Jinja2 templates`_ by `alexharv074`_.
+
+
+.. _`A method of unit testing Jinja2 templates` : https://alexharv074.github.io/2020/01/18/a-method-of-unit-testing-jinja2-templates.html
+.. _`alexharv074` : https://alexharv074.github.io/
+.. _`fruitfulness` : https://link.springer.com/article/10.1007/s11229-017-1355-6
+.. _`undefined`: https://jinja.palletsprojects.com/en/3.1.x/api/#undefined-types
+.. _`StrictUndefined`: https://jinja.palletsprojects.com/en/3.1.x/api/#jinja2.StrictUndefined
+.. _`DebugUndefined`: https://jinja.palletsprojects.com/en/3.1.x/api/#jinja2.DebugUndefined
Jude Nagurney
#############
-.. :date: 2023-08-12
+.. :date: 2024-05-02
.. :tags: resume
.. :category: resume
.. :author: Jude N
Technical Skills:
=================
-- **Languages** : Python, C/C++, Ruby, SQL, Bash, lua
-- **Tools** : Salt, SELinux, Jenkins, emacs, vi, Jira, git, Gitlab, Docker, podman, Automate Android app, Nagios
-- **Frameworks** : Django, Angular, Rails, SqlAlchemy
+- **Languages** : Python, C/C++, Ruby, SQL, Bash, lua, Javascript, Typescript
+- **Tools** : Salt, SELinux, Jenkins, emacs, vi, Jira, git, Gitlab, Docker, podman, Automate Android app, Nagios, Zabbix
+- **Frameworks** : Django, Angular, Rails, SqlAlchemy, AngularJS
- **Operating Systems** : Linux (Ubuntu, Debian, RedHat, CentOS, Rocky, Raspbian), Microsoft Windows
- **Databases** : PostgresSQL, MySQL, Oracle, sqlite
===============
------------------
-Layer 2 Technology
+L2T, LLC
------------------
-| **Reston Virginia**
+| **Herndon Virginia**
| **October 2016 - Present**
--------------
- Created a series of project-based orientation presentations to facilitate onboarding of new employees.
-- Assumed responsibility for a mission-critical dormant project, ensuring users had a dedicated point of contact.
+- Assumed responsibility for a mission-critical, dormant project, ensuring users had a dedicated point of contact.
-- Assumed responsibility for tagging and releasing unreleased software, ensuring its deployability.
+- Assumed responsibility for tagging new software releases and pushing the releases into production.
-- Played an instrumental role in defining the staff engineer position within my current company.
+- Worked with the leadership at my current company to recognize the staff engineer role.
-- Reviewed software changes to ensure code quality standards were upheld, and to distribute knowledge across the teams.
+- Reviewed software changes to ensure code quality standards were upheld, and that knowledge was shared across the teams.
------------------------
Senior Software Engineer
- Developed and maintained Salt states and Puppet manifests for various projects.
-- Developed iflows with the Automate app for controlling and monitoring Android phones.
+- Developed flows with the Automate app for controlling and monitoring Android phones.
- Wrote a Errbot plugin for reporting open merge requests that were waiting for peer reviews.
- Developed and maintained Jenkins continuous integration jobs for multiple projects.
Also proactively tracked down the root causes of build failures when the jobs failed.
-- Developd and monitored Nagious monitoring for the development network.
+- Developd and monitored Nagios monitoring for the development network.
- Provided Production support and troubleshooting.
projects / pwan.org.git / commitdiff