pytest-tricksurn:uuid:7f0b50d2-7dc3-329b-b27b-5001002cb5812017-08-13T00:00:00ZRaphael Pierzina, pytest-dev team and contributorsTips and Tricks for the Python Testing ToolLektor Atom PluginFixtures as class attributesurn:uuid:66713ffe-c32d-33ab-a198-202ac65333bb2017-08-13T00:00:00ZSergey Kraynev<h1>Use Case</h1>
<p>Migration from unittest-style tests with <code>setUp</code> methods to pytest fixtures can
be laborious, because users have to specify fixtures parameters in each
test method in class. Also <code>flake8</code> checks will complain about
unknown methods in parameters (it's minor issue, but it's still exists).
This article demonstrates alternatives way for an easier migration, with the
following benefits:</p>
<ul>
<li>No need to specify fixture names in tests signatures. (No changes to tests required).</li>
<li>Accessing fixtures as instance attributes. (No changes to tests required).</li>
</ul>
<h1>Solutions</h1>
<p>The main feature used to solve this issue is <em>autouse</em> fixtures.
The detailed explanation is available in
<a href="https://docs.pytest.org/en/latest/fixture.html?highlight=autouse#autouse-fixtures-xunit-setup-on-steroids">official documentation</a>,
but shortly: it allows to execute fixture before each test or test method in the current case.</p>
<h2>Approach one (with decorator)</h2>
<p>The first approach uses a new decorator for test classes.
This decorator injects an autouse fixture in the decorated class; this fixture
will then be called automatically before each test.
Thanks to <code>Bruno Oliveira</code>(nicoddemus) for the help with creation this
solution.</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">functools</span> <span class="kn">import</span> <span class="n">partial</span>
<span class="kn">import</span> <span class="nn">pytest</span>
<span class="k">def</span> <span class="nf">_inject</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">names</span><span class="p">):</span>
<span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">autouse</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">auto_injector_fixture</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">request</span><span class="p">):</span>
<span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">names</span><span class="p">:</span>
<span class="nb">setattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">request</span><span class="o">.</span><span class="n">getfixturevalue</span><span class="p">(</span><span class="n">name</span><span class="p">))</span>
<span class="bp">cls</span><span class="o">.</span><span class="n">__auto_injector_fixture</span> <span class="o">=</span> <span class="n">auto_injector_fixture</span>
<span class="k">return</span> <span class="bp">cls</span>
<span class="k">def</span> <span class="nf">auto_inject_fixtures</span><span class="p">(</span><span class="o">*</span><span class="n">names</span><span class="p">):</span>
<span class="k">return</span> <span class="n">partial</span><span class="p">(</span><span class="n">_inject</span><span class="p">,</span> <span class="n">names</span><span class="o">=</span><span class="n">names</span><span class="p">)</span>
<span class="nd">@auto_inject_fixtures</span><span class="p">(</span><span class="s1">'tmpdir'</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">Test</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">test_foo</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">assert</span> <span class="bp">self</span><span class="o">.</span><span class="n">tmpdir</span><span class="o">.</span><span class="n">isdir</span><span class="p">()</span>
</pre></div>
<p>More importantly, this also works for unittest subclasses:</p>
<div class="highlight"><pre><span></span><span class="nd">@auto_inject_fixtures</span><span class="p">(</span><span class="s1">'tmpdir'</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">Test2</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">test_foo</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertTrue</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">tmpdir</span><span class="o">.</span><span class="n">isdir</span><span class="p">())</span>
</pre></div>
<p>As it was demonstrated fixtures names passed to the decorator are now available to test methods as instance attributes.</p>
<h2>Approach two (define as attributes)</h2>
<p>Consider the scenario where user has more then 5 or 10 classes and they all inherited from a main <code>TestBase</code> class?
The answer is to take the first approach and modify it to do all <em>hard</em>
work with decorators automatically.</p>
<p>First put the autouse fixture in the <code>TestBase</code> class, which
will make it available to all child classes automatically.
A second step is to declare a list of fixtures that will be injected in each
test class. In the ideal world they
will be the same for all classes and tests, but in reality it will be
different for each class. Define a tuple <code>fixture_names</code> in each test class with the names of the fixtures that should be injected for each test.
The example below demostrates ideas mentioned before.</p>
<div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">TestBase</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
<span class="n">fixture_names</span> <span class="o">=</span> <span class="p">()</span>
<span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">autouse</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">auto_injector_fixture</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">request</span><span class="p">):</span>
<span class="n">names</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">fixture_names</span>
<span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">names</span><span class="p">:</span>
<span class="nb">setattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">request</span><span class="o">.</span><span class="n">getfixturevalue</span><span class="p">(</span><span class="n">name</span><span class="p">))</span>
<span class="k">class</span> <span class="nc">MyTest</span><span class="p">(</span><span class="n">TestBase</span><span class="p">):</span>
<span class="n">fixture_names</span> <span class="o">=</span> <span class="p">(</span><span class="s1">'tmpdir'</span><span class="p">,</span> <span class="s1">'random'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">test_bar</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertTrue</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">tmpdir</span><span class="o">.</span><span class="n">isdir</span><span class="p">())</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">string</span><span class="p">(</span><span class="mi">5</span><span class="p">)),</span> <span class="mi">5</span><span class="p">)</span>
</pre></div>
<p>Important to mention that the approach above also work for pytest-style classes (subclassing only <code>object</code>).</p>
<p>Last example can be improved for scenario tests. However the guide mentioned
in the <a href="https://docs.pytest.org/en/latest/example/parametrize.html?highlight=testscenario#a-quick-port-of-testscenarios">official documentation</a> is not compatible with unittests subclasses.</p>
<div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">TestBase</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="n">fixture_names</span> <span class="o">=</span> <span class="p">()</span>
<span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">autouse</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">auto_injector_fixture</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">request</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="s1">'scenarios'</span><span class="p">):</span>
<span class="n">names</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">scenarios</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">names</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">fixture_names</span>
<span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">names</span><span class="p">:</span>
<span class="nb">setattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">request</span><span class="o">.</span><span class="n">getfixturevalue</span><span class="p">(</span><span class="n">name</span><span class="p">))</span>
<span class="k">class</span> <span class="nc">MyTestScenario</span><span class="p">(</span><span class="n">TestBase</span><span class="p">):</span>
<span class="n">scenarios</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">(</span><span class="n">test_one</span><span class="p">,</span> <span class="nb">dict</span><span class="p">(</span><span class="n">val</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">res</span><span class="o">=</span><span class="mi">5</span><span class="p">)),</span>
<span class="p">(</span><span class="n">test_two</span><span class="p">,</span> <span class="nb">dict</span><span class="p">(</span><span class="n">val</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">res</span><span class="o">=</span><span class="mi">10</span><span class="p">))</span>
<span class="p">]</span>
<span class="k">def</span> <span class="nf">test_baz</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">assert</span> <span class="bp">self</span><span class="o">.</span><span class="n">res</span> <span class="o">==</span> <span class="bp">self</span><span class="o">.</span><span class="n">val</span> <span class="o">*</span> <span class="mi">5</span>
</pre></div>
Load pytest plugins dynamicallyurn:uuid:aaf81e61-98b7-3a01-b2ca-40d470d59d2d2016-12-09T00:00:00ZGeorge Shuklin<p>There is a way to pass fixtures from python code to tests via plugins argument of <code>pytest.main()</code>.</p>
<h1>Creating fixture</h1>
<p>To create fixture we can use decorator <code>@pytest.fixture</code>:</p>
<div class="highlight"><pre><span></span><span class="nd">@pytest.fixture</span>
<span class="k">def</span> <span class="nf">info</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="s2">"Information!"</span>
</pre></div>
<h1>Passing plugin to pytest</h1>
<p>To pass something to pytest-executed tests we can use plugin mechanism of pytest. (For not let says that <code>myplugin</code> contains actual plugin, I’ll show later how to create it).</p>
<div class="highlight"><pre><span></span><span class="n">pytest</span><span class="o">.</span><span class="n">main</span><span class="p">([</span><span class="n">path_to_test</span><span class="p">,</span> <span class="s1">'-v'</span><span class="p">],</span> <span class="n">plugins</span><span class="o">=</span><span class="p">[</span><span class="n">myplugin</span><span class="p">])</span>
</pre></div>
<p>Please not that <code>plugins</code> takes a list of plugins, not plugin itself</p>
<h1>Creating plugin</h1>
<p>There are two ways to create plugin:</p>
<ul>
<li>Use stand-alone module</li>
<li>Create class or other <em>dotted</em> object (object with members accessible by dot: plugin.something).</li>
</ul>
<h2>Module way</h2>
<p>Use a separate file to contains plugin code:</p>
<p>myplugin.py:</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pytest</span>
<span class="nd">@pytest.fixture</span>
<span class="k">def</span> <span class="nf">info</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="s2">"Information"</span>
</pre></div>
<p>actual call of pytest:</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">myplugin</span>
<span class="n">pytest</span><span class="o">.</span><span class="n">main</span><span class="p">([</span><span class="s1">'mytest.py'</span><span class="p">,</span> <span class="s1">'-v'</span><span class="p">],</span> <span class="n">plugins</span><span class="o">=</span><span class="p">[</span><span class="n">myplugin</span><span class="p">])</span>
</pre></div>
<p>Fixture usage (mytest.py):</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">test_foo</span><span class="p">(</span><span class="n">info</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">info</span><span class="p">()</span> <span class="o">==</span> <span class="s2">"Information"</span>
</pre></div>
<h2>Class way</h2>
<p>This way I found more preferable, because I have higher degree of freedom during class initialization. I can put all information I want inside class in very natural way.</p>
<p>Plugin and fixture constuction and test call:</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pytest</span>
<span class="k">class</span> <span class="nc">MyPlugin</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">data</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span> <span class="o">=</span> <span class="n">data</span>
<span class="nd">@pytest.fixture</span>
<span class="k">def</span> <span class="nf">info</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">data</span>
<span class="n">myplugin</span> <span class="o">=</span> <span class="n">MyPlugin</span><span class="p">(</span><span class="s1">'information'</span><span class="p">)</span>
<span class="n">pytest</span><span class="o">.</span><span class="n">main</span><span class="p">(</span><span class="s2">"mytest.py"</span><span class="p">,</span> <span class="n">plugins</span><span class="o">=</span><span class="p">[</span><span class="n">myplugin</span><span class="p">])</span>
</pre></div>
<p>Test is exactly the same as before (mytest.py):</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">test_foo</span><span class="p">(</span><span class="n">info</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">info</span><span class="p">()</span> <span class="o">==</span> <span class="s2">"Information"</span>
</pre></div>
<h1>More than one fixture</h1>
<p>It is very easy to put any number of fixtures into a class:</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pytest</span>
<span class="k">class</span> <span class="nc">MyPlugin</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">data</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span> <span class="o">=</span> <span class="n">data</span>
<span class="nd">@pytest.fixture</span>
<span class="k">def</span> <span class="nf">info</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">data</span>
<span class="nd">@pytest.fixture</span>
<span class="k">def</span> <span class="nf">length</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">)</span>
<span class="n">myplugin</span> <span class="o">=</span> <span class="n">MyPlugin</span><span class="p">(</span><span class="s1">'information'</span><span class="p">)</span>
<span class="n">pytest</span><span class="o">.</span><span class="n">main</span><span class="p">(</span><span class="s2">"mytest.py"</span><span class="p">,</span> <span class="n">plugins</span><span class="o">=</span><span class="p">[</span><span class="n">myplugin</span><span class="p">])</span>
</pre></div>
<p>A bit upgraded version of the test accepts two fixtures (mytest.py):</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">test_foo</span><span class="p">(</span><span class="n">info</span><span class="p">,</span> <span class="n">length</span><span class="p">):</span>
<span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">info</span><span class="p">())</span> <span class="o">==</span> <span class="n">length</span><span class="p">()</span>
</pre></div>
ids for fixtures and parametrizeurn:uuid:1bd554e3-af75-3403-991f-0beeb0c766722016-11-12T00:00:00ZRaphael Pierzina<h1>parameters for tests</h1>
<p><strong>pytest</strong> comes with a handful of powerful tools to generate parameters for a
test, so you can run various scenarios against the same test implementation.</p>
<ul>
<li><code>params</code> on a <code>@pytest.fixture</code></li>
<li><code>parametrize</code> marker</li>
<li><code>pytest_generate_tests</code> hook with <code>metafunc.parametrize</code></li>
</ul>
<p>All of the above have their individual strengths and weaknessses. In this post
I'd like to cover ids for tests and why I think it's a good idea to use them.</p>
<p>Please check out older posts on this blog to find out more about
parametrization in pytest, if you haven't already.</p>
<ul>
<li><a href="http://hackebrot.github.io/pytest-tricks/create_tests_via_parametrization/">create tests via parametrization</a></li>
<li><a href="http://hackebrot.github.io/pytest-tricks/mark_parametrize_with_indirect/">mark.parametrize with indirect</a></li>
</ul>
<h1>test items</h1>
<p>To understand how <strong>pytest</strong> generates ids for tests, we first need to know
what test items are.</p>
<p>For <strong>pytest</strong> a test item is a single test, defined by an underlying test
function along with the setup and teardown code for the fixtures it uses.
Essentially, everything we need to run a test. Suffice to say that this is a
slightly simplified explanation, but that's all we need to know for now.</p>
<p>Here's a code example for clarification:</p>
<div class="highlight"><pre><span></span><span class="nd">@pytest.mark.parametrize</span><span class="p">(</span>
<span class="s1">'number, word'</span><span class="p">,</span> <span class="p">[</span>
<span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s1">'1'</span><span class="p">),</span>
<span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="s1">'fizz'</span><span class="p">),</span>
<span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="s1">'buzz'</span><span class="p">),</span>
<span class="p">(</span><span class="mi">8</span><span class="p">,</span> <span class="s1">'8'</span><span class="p">),</span>
<span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="s1">'buzz'</span><span class="p">),</span>
<span class="p">(</span><span class="mi">15</span><span class="p">,</span> <span class="s1">'fizzbuzz'</span><span class="p">),</span>
<span class="p">]</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">test_fizzbuzz</span><span class="p">(</span><span class="n">number</span><span class="p">,</span> <span class="n">word</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">fizzbuzz</span><span class="p">(</span><span class="n">number</span><span class="p">)</span> <span class="o">==</span> <span class="n">word</span>
</pre></div>
<p>There is a single test function in above code example. It calls a function
called <code>fizzbuzz</code> with an integer and checks that its return value matches a
certain string.</p>
<p>Now when we run this, <strong>pytest</strong> creates a number of test items under the hood.</p>
<ul>
<li>Item with function: <code>test_fizzbuzz</code>, number: <code>1</code>, word: <code>'1'</code></li>
<li>Item with function: <code>test_fizzbuzz</code>, number: <code>3</code>, word: <code>'fizz'</code></li>
<li>Item with function: <code>test_fizzbuzz</code>, number: <code>5</code>, word: <code>'buzz'</code></li>
<li>Item with function: <code>test_fizzbuzz</code>, number: <code>8</code>, word: <code>'8'</code></li>
<li>Item with function: <code>test_fizzbuzz</code>, number: <code>10</code>, word: <code>'buzz'</code></li>
<li>Item with function: <code>test_fizzbuzz</code>, number: <code>15</code>, word: <code>'fizzbuzz'</code></li>
</ul>
<p>This is exactly what we can see in the test result log when running <strong>pytest</strong>
in normal mode (not quiet or verbose).</p>
<h1>what are ids?</h1>
<p>Now <strong>pytest</strong> lets us add <em>ids</em> to test items to make them easier to
distinguish in the log, especially in verbose mode.</p>
<h2>auto-generated ids</h2>
<p>If you leave it up to <strong>pytest</strong> to generate ids, you will see that most of
them are pretty good, but others feel somewhat random. The reason being that
<em>non-primitive</em> types such as <code>dict</code>, <code>list</code>, <code>tuple</code> or instances of
<code>classes</code> are non trivial to translate into ids.</p>
<p>The following test is parametrized and the id for a test item will be a
representation for <strong>each individual parameter</strong> - joined with a <code>-</code>.</p>
<div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">CookiecutterTemplate</span><span class="p">:</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">url</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="n">name</span>
<span class="bp">self</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="n">url</span>
<span class="n">PYTEST_PLUGIN</span> <span class="o">=</span> <span class="n">CookiecutterTemplate</span><span class="p">(</span>
<span class="s1">'pytest-plugin'</span><span class="p">,</span>
<span class="s1">'https://github.com/pytest-dev/cookiecutter-pytest-plugin'</span><span class="p">,</span>
<span class="p">)</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span>
<span class="s1">'a, b'</span><span class="p">,</span>
<span class="p">[</span>
<span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="p">{</span><span class="s1">'Two Scoops of Django'</span><span class="p">:</span> <span class="s1">'1.8'</span><span class="p">}),</span>
<span class="p">(</span><span class="bp">True</span><span class="p">,</span> <span class="s1">'Into the Brambles'</span><span class="p">),</span>
<span class="p">(</span><span class="s1">'Jason likes cookies'</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]),</span>
<span class="p">(</span><span class="n">PYTEST_PLUGIN</span><span class="p">,</span> <span class="s1">'plugin_template'</span><span class="p">),</span>
<span class="p">]</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">test_foobar</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="k">assert</span> <span class="bp">True</span>
</pre></div>
<p><code>$ pytest -v</code> produces the following report:</p>
<div class="highlight"><pre><span></span>============================ test session starts =============================
collecting ... collected 4 items
test_ids.py::test_foobar[1-b0] PASSED
test_ids.py::test_foobar[True-Into the Brambles] PASSED
test_ids.py::test_foobar[Jason likes cookies-b2] PASSED
test_ids.py::test_foobar[a3-plugin_template] PASSED
========================== 4 passed in 0.03 seconds ==========================
</pre></div>
<p>As you can see whenever <strong>pytest</strong> encounters one of the <em>non-primitives</em> it
uses the parametrized argument name with a suffix instead.</p>
<p>For instance a tuple of str and list, such as:</p>
<p><code>('Jason likes cookies', [1, 2, 3])</code></p>
<p>is translated to</p>
<p><code>Jason likes cookies-b2</code></p>
<h2>explicit ids</h2>
<p>The good news is that you can define ids yourself rather than leaving it up
to <strong>pytest</strong> to somehow figure them out.</p>
<div class="highlight"><pre><span></span><span class="n">PYTEST_PLUGIN</span> <span class="o">=</span> <span class="n">CookiecutterTemplate</span><span class="p">(</span>
<span class="s1">'pytest-plugin'</span><span class="p">,</span>
<span class="s1">'https://github.com/pytest-dev/cookiecutter-pytest-plugin'</span><span class="p">,</span>
<span class="p">)</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span>
<span class="s1">'a, b'</span><span class="p">,</span>
<span class="p">[</span>
<span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="p">{</span><span class="s1">'Two Scoops of Django'</span><span class="p">:</span> <span class="s1">'1.8'</span><span class="p">}),</span>
<span class="p">(</span><span class="bp">True</span><span class="p">,</span> <span class="s1">'Into the Brambles'</span><span class="p">),</span>
<span class="p">(</span><span class="s1">'Jason likes cookies'</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]),</span>
<span class="p">(</span><span class="n">PYTEST_PLUGIN</span><span class="p">,</span> <span class="s1">'plugin_template'</span><span class="p">),</span>
<span class="p">],</span> <span class="n">ids</span><span class="o">=</span><span class="p">[</span>
<span class="s1">'int and dict'</span><span class="p">,</span>
<span class="s1">'bool and str'</span><span class="p">,</span>
<span class="s1">'str and list'</span><span class="p">,</span>
<span class="s1">'CookiecutterTemplate and str'</span><span class="p">,</span>
<span class="p">]</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">test_foobar</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="k">assert</span> <span class="bp">True</span>
</pre></div>
<div class="highlight"><pre><span></span>============================ test session starts =============================
collecting ... collected 4 items
test_ids.py::test_foobar[int and dict] PASSED
test_ids.py::test_foobar[bool and str] PASSED
test_ids.py::test_foobar[str and list] PASSED
test_ids.py::test_foobar[CookiecutterTemplate and str] PASSED
========================== 4 passed in 0.01 seconds ==========================
</pre></div>
<p>Note that passing a list of <code>str</code> values to the <code>ids</code> keyword overwrites
ids <strong>per parameter combination in a marker</strong> and <strong>not for individual
parameters</strong>. See how there is no <code>-</code> in the logged ids?</p>
<p>"Hey, you've talked about tests, items, ids, and now markers?!"</p>
<p>I know...it can be confusing. I hope to make this as clear as possible as we
go on. Hopefully in the end of this post, when reading the TL;DR, you know how
ids work and how you can use them to make your test suite more maintainable.</p>
<h2>markers</h2>
<p>As I've mentioned earlier <code>str</code> id values are applied to a specific parameter
combination of markers rather than test items or individual parameters. To
illustrate this, let's have a look at the following code example.</p>
<div class="highlight"><pre><span></span><span class="n">PYTEST_PLUGIN</span> <span class="o">=</span> <span class="n">CookiecutterTemplate</span><span class="p">(</span>
<span class="s1">'pytest-plugin'</span><span class="p">,</span>
<span class="s1">'https://github.com/pytest-dev/cookiecutter-pytest-plugin'</span><span class="p">,</span>
<span class="p">)</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span>
<span class="s1">'a, b'</span><span class="p">,</span>
<span class="p">[</span>
<span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="p">{</span><span class="s1">'Two Scoops of Django'</span><span class="p">:</span> <span class="s1">'1.8'</span><span class="p">}),</span>
<span class="p">(</span><span class="bp">True</span><span class="p">,</span> <span class="s1">'Into the Brambles'</span><span class="p">),</span>
<span class="p">(</span><span class="s1">'Jason likes cookies'</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]),</span>
<span class="p">(</span><span class="n">PYTEST_PLUGIN</span><span class="p">,</span> <span class="s1">'plugin_template'</span><span class="p">),</span>
<span class="p">],</span> <span class="n">ids</span><span class="o">=</span><span class="p">[</span>
<span class="s1">'int and dict'</span><span class="p">,</span>
<span class="s1">'bool and str'</span><span class="p">,</span>
<span class="s1">'str and list'</span><span class="p">,</span>
<span class="s1">'CookiecutterTemplate and str'</span><span class="p">,</span>
<span class="p">]</span>
<span class="p">)</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span>
<span class="s1">'c'</span><span class="p">,</span>
<span class="p">[</span>
<span class="s1">'hello world'</span><span class="p">,</span>
<span class="mi">123</span><span class="p">,</span>
<span class="p">],</span>
<span class="n">ids</span><span class="o">=</span><span class="p">[</span>
<span class="s1">'str'</span><span class="p">,</span>
<span class="s1">'int'</span><span class="p">,</span>
<span class="p">],</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">test_foobar</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">):</span>
<span class="k">assert</span> <span class="bp">True</span>
</pre></div>
<p>Above is the same test from the previous section, but uses an additional test
parameter <code>c</code>, which is set up with another <code>parametrize</code> marker.</p>
<p>So here's what you see when you would run this in <code>--verbose</code> mode:</p>
<div class="highlight"><pre><span></span>============================ test session starts =============================
collecting ... collected 8 items
test_multiple_markers.py::test_foobar[str-int and dict] PASSED
test_multiple_markers.py::test_foobar[str-bool and str] PASSED
test_multiple_markers.py::test_foobar[str-str and list] PASSED
test_multiple_markers.py::test_foobar[str-CookiecutterTemplate and str] PASSED
test_multiple_markers.py::test_foobar[int-int and dict] PASSED
test_multiple_markers.py::test_foobar[int-bool and str] PASSED
test_multiple_markers.py::test_foobar[int-str and list] PASSED
test_multiple_markers.py::test_foobar[int-CookiecutterTemplate and str] PASSED
========================== 8 passed in 0.01 seconds ==========================
</pre></div>
<p>As you can see here from the printed ids, for instance <code>[int-bool and str]</code>,
a string value is taken from each marker and joined with <code>-</code> as it would for
the automatically generated ids.</p>
<h2>ids callables</h2>
<p>Instead of providing <code>str</code> values for test items, you can also pass in a
function or method, that will be called for <strong>every single parameter</strong> and is
expected to return a <code>str</code> id. Unlike <code>str</code> ids the function is called for
every parameter!</p>
<p>If your callable returns <code>None</code> pytest falls back to the autogenerated id for
that particular parameter.</p>
<div class="highlight"><pre><span></span><span class="n">PYTEST_PLUGIN</span> <span class="o">=</span> <span class="n">CookiecutterTemplate</span><span class="p">(</span>
<span class="s1">'pytest-plugin'</span><span class="p">,</span>
<span class="s1">'https://github.com/pytest-dev/cookiecutter-pytest-plugin'</span><span class="p">,</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">id_func</span><span class="p">(</span><span class="n">param</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">param</span><span class="p">,</span> <span class="n">CookiecutterTemplate</span><span class="p">):</span>
<span class="k">return</span> <span class="s1">'template {.name}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">param</span><span class="p">)</span>
<span class="k">return</span> <span class="nb">repr</span><span class="p">(</span><span class="n">param</span><span class="p">)</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span>
<span class="s1">'a, b'</span><span class="p">,</span>
<span class="p">[</span>
<span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="p">{</span><span class="s1">'Two Scoops of Django'</span><span class="p">:</span> <span class="s1">'1.8'</span><span class="p">}),</span>
<span class="p">(</span><span class="bp">True</span><span class="p">,</span> <span class="s1">'Into the Brambles'</span><span class="p">),</span>
<span class="p">(</span><span class="s1">'Jason likes cookies'</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]),</span>
<span class="p">(</span><span class="n">PYTEST_PLUGIN</span><span class="p">,</span> <span class="s1">'plugin_template'</span><span class="p">),</span>
<span class="p">],</span>
<span class="n">ids</span><span class="o">=</span><span class="n">id_func</span><span class="p">,</span>
<span class="p">)</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span>
<span class="s1">'c'</span><span class="p">,</span>
<span class="p">[</span>
<span class="s1">'hello world'</span><span class="p">,</span>
<span class="mi">123</span><span class="p">,</span>
<span class="p">],</span>
<span class="n">ids</span><span class="o">=</span><span class="n">id_func</span><span class="p">,</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">test_foobar</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">):</span>
<span class="k">assert</span> <span class="bp">True</span>
</pre></div>
<div class="highlight"><pre><span></span>============================ test session starts =============================
collecting ... collected 8 items
test_markers.py::test_foobar['hello world'-1-{'Two Scoops of Django': '1.8'}] PASSED
test_markers.py::test_foobar['hello world'-True-'Into the Brambles'] PASSED
test_markers.py::test_foobar['hello world'-'Jason likes cookies'-[1, 2, 3]] PASSED
test_markers.py::test_foobar['hello world'-template pytest-plugin-'plugin_template'] PASSED
test_markers.py::test_foobar[123-1-{'Two Scoops of Django': '1.8'}] PASSED
test_markers.py::test_foobar[123-True-'Into the Brambles'] PASSED
test_markers.py::test_foobar[123-'Jason likes cookies'-[1, 2, 3]] PASSED
test_markers.py::test_foobar[123-template pytest-plugin-'plugin_template'] PASSED
========================== 8 passed in 0.02 seconds ==========================
</pre></div>
<h2>id hook</h2>
<p>A new hook called <code>pytest_make_parametrize_id</code> was added in pytest 3.0 that
makes it easy to centralize id generation. One of the reasons why you might
want to do this, is when you use instances of your custom classes for
parametrized tests. I personally don't think it's a good idea to change the
classes, by modifying the <code>__repr__</code> or <code>__str__</code> magic methods, just so
that you get this extra convenience in your tests. Instead I would encourage
you to try out this hook.</p>
<p>Let's have a look at how this hook works:</p>
<div class="highlight"><pre><span></span><span class="n">PYTEST_PLUGIN</span> <span class="o">=</span> <span class="n">CookiecutterTemplate</span><span class="p">(</span>
<span class="s1">'pytest-plugin'</span><span class="p">,</span>
<span class="s1">'https://github.com/pytest-dev/cookiecutter-pytest-plugin'</span><span class="p">,</span>
<span class="p">)</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span>
<span class="s1">'a, b'</span><span class="p">,</span>
<span class="p">[</span>
<span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="p">{</span><span class="s1">'Two Scoops of Django'</span><span class="p">:</span> <span class="s1">'1.8'</span><span class="p">}),</span>
<span class="p">(</span><span class="bp">True</span><span class="p">,</span> <span class="s1">'Into the Brambles'</span><span class="p">),</span>
<span class="p">(</span><span class="s1">'Jason likes cookies'</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]),</span>
<span class="p">(</span><span class="n">PYTEST_PLUGIN</span><span class="p">,</span> <span class="s1">'plugin_template'</span><span class="p">),</span>
<span class="p">],</span>
<span class="p">)</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span>
<span class="s1">'c'</span><span class="p">,</span>
<span class="p">[</span>
<span class="s1">'hello world'</span><span class="p">,</span>
<span class="mi">123</span><span class="p">,</span>
<span class="p">],</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">test_foobar</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">):</span>
<span class="k">assert</span> <span class="bp">True</span>
</pre></div>
<p>The test and the markers are identical to the example from the previous
section, except for the <code>ids</code> keyword argument in the markers.</p>
<p>Now what we do instead is implementing the hook in a <strong>conftest.py</strong> file:</p>
<div class="highlight"><pre><span></span><span class="c1"># -*- coding: utf-8 -*-</span>
<span class="kn">from</span> <span class="nn">templates</span> <span class="kn">import</span> <span class="n">CookiecutterTemplate</span>
<span class="k">def</span> <span class="nf">pytest_make_parametrize_id</span><span class="p">(</span><span class="n">config</span><span class="p">,</span> <span class="n">val</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">CookiecutterTemplate</span><span class="p">):</span>
<span class="k">return</span> <span class="s1">'template {.name}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">val</span><span class="p">)</span>
<span class="k">return</span> <span class="nb">repr</span><span class="p">(</span><span class="n">val</span><span class="p">)</span>
</pre></div>
<p><code>val</code> is the value that will be passed into the test a particular parameter
and <code>config</code> is the test run config, so you could check for command-line
flags etc. if you want to.</p>
<p>We effectively moved the implementation of our <code>id_func</code> to this hook, which
means we don't need to set <code>ids</code> in all of the <code>@pytest.mark.parametrize</code>
as long as we are happy with the way it generates ids. We can still overwrite
them by explictly by setting a <code>str</code> id or passing in a callable.</p>
<h1>fixtures</h1>
<p>Ids for fixtures are in fact easier to understand as for <code>parametrize</code>
markers, as they don't have any edge cases that you need to be aware of.
Fixtures always return a single value, which means regardless of whether you
use <code>str</code> ids, an id callable or the <code>pytest_make_parametrize_id</code> it will
always be applied to this very parameter that the fixture returns.</p>
<div class="highlight"><pre><span></span><span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">params</span><span class="o">=</span><span class="p">[</span>
<span class="n">CookiecutterTemplate</span><span class="p">(</span>
<span class="n">name</span><span class="o">=</span><span class="s1">'cookiecutter-pytest-plugin'</span><span class="p">,</span>
<span class="n">url</span><span class="o">=</span><span class="s1">'https://github.com/pytest-dev/cookiecutter-pytest-plugin'</span><span class="p">,</span>
<span class="p">),</span>
<span class="n">CookiecutterTemplate</span><span class="p">(</span>
<span class="n">name</span><span class="o">=</span><span class="s1">'cookiecutter-django'</span><span class="p">,</span>
<span class="n">url</span><span class="o">=</span><span class="s1">'https://github.com/pydanny/cookiecutter-django'</span><span class="p">,</span>
<span class="p">),</span>
<span class="p">],</span> <span class="n">ids</span><span class="o">=</span><span class="p">[</span>
<span class="s1">'cookiecutter-pytest-plugin'</span><span class="p">,</span>
<span class="s1">'cookiecutter-django'</span>
<span class="p">])</span>
<span class="k">def</span> <span class="nf">template</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="n">request</span><span class="o">.</span><span class="n">param</span>
<span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">params</span><span class="o">=</span><span class="p">[</span>
<span class="s1">'pydanny'</span><span class="p">,</span>
<span class="s1">'audreyr'</span><span class="p">,</span>
<span class="s1">'michaeljoseph'</span><span class="p">,</span>
<span class="p">])</span>
<span class="k">def</span> <span class="nf">github_user</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="n">request</span><span class="o">.</span><span class="n">param</span>
<span class="k">def</span> <span class="nf">test_template</span><span class="p">(</span><span class="n">template</span><span class="p">,</span> <span class="n">github_user</span><span class="p">):</span>
<span class="k">assert</span> <span class="bp">True</span>
</pre></div>
<div class="highlight"><pre><span></span>============================ test session starts =============================
collecting ... collected 6 items
test_markers.py::test_template[cookiecutter-pytest-plugin-pydanny] PASSED
test_markers.py::test_template[cookiecutter-pytest-plugin-audreyr] PASSED
test_markers.py::test_template[cookiecutter-pytest-plugin-michaeljoseph] PASSED
test_markers.py::test_template[cookiecutter-django-pydanny] PASSED
test_markers.py::test_template[cookiecutter-django-audreyr] PASSED
test_markers.py::test_template[cookiecutter-django-michaeljoseph] PASSED
========================== 6 passed in 0.04 seconds ==========================
</pre></div>
<h1>Conclusion</h1>
<ol>
<li><p>If you don't set ids, pytest will generate them for you</p>
</li>
<li><p>If you set them explicitly with <code>str</code> values</p>
<p>a. they are set for a <strong>parameter combination</strong> in case of
<code>@pytest.mark.parametrize</code></p>
<p>b. or a value returned by <code>@pytest.fixture</code></p>
</li>
<li><p>If you use a callable</p>
<p>a. it will be invoked for <strong>each parameter</strong> in case of
<code>@pytest.mark.parametrize</code></p>
<p>b. or a value returned by <code>@pytest.fixture</code></p>
</li>
<li><p>If you use the <code>pytest_make_parametrize_id</code> hook</p>
<p>a. it will be invoked for <strong>each parameter</strong> in case of
<code>@pytest.mark.parametrize</code></p>
<p>b. or a value returned by <code>@pytest.fixture</code></p>
</li>
</ol>
<p>So the only thing to keep in mind really is that <code>str</code> ids for
<code>pytest.mark.parametrize</code>.</p>
<p>Hope this helps!</p>
mark.parametrize with indirecturn:uuid:22735055-1984-3a4c-a653-a1c7fb3b73a12016-07-10T00:00:00ZRaphael Pierzina<h1>mark.parametrize</h1>
<p>In <a href="http://hackebrot.github.io/pytest-tricks/create_tests_via_parametrization/">Create Tests via Parametrization</a> we've learned how to
use <code>@pytest.mark.parametrize</code> to generate a number of test cases for
one test implementation.</p>
<h1>indirect</h1>
<p>You can pass a keyword argument named <code>indirect</code> to <code>parametrize</code> to change
how its parameters are being passed to the underlying test function. It accepts
either a <strong>boolean value</strong> or a <strong>list of strings</strong> that refer to
<code>pytest.fixure</code> functions.</p>
<h2>False</h2>
<p>If you set <code>indirect</code> to <code>False</code> or omit the parameter altogether, pytest
will treat the given parameters as is w/o any specialties.</p>
<h2>True</h2>
<p>All of the parameters are stored on the special <code>request</code> fixture. Other
fixtures can then access it via <code>request.param</code> and modify the test
parameter.</p>
<h2>List of fixture names</h2>
<p>You can choose to only set <code>indirect</code> for a subset of arguments, by passing a
list to the keyword argument: <code>indirect=['foo', 'bar']</code>.</p>
<h1>Example code</h1>
<p>We define two classes that have a few fields to hold information. The tests
will access the attributes to check for correctness of our code. For that we
create a number of instances of the <code>Sushi</code> class with varying parameters,
before we call its property <code>is_vegetarian</code> in the test to see if the <em>Fooshi
Bar</em> offers a few vegetarian dishes.</p>
<p><code>sushi.py</code></p>
<div class="highlight"><pre><span></span><span class="c1"># -*- coding: utf-8 -*-</span>
<span class="k">class</span> <span class="nc">Restaurant</span><span class="p">:</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">location</span><span class="p">,</span> <span class="n">menu</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">menu</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">ValueError</span>
<span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="n">name</span>
<span class="bp">self</span><span class="o">.</span><span class="n">location</span> <span class="o">=</span> <span class="n">location</span>
<span class="bp">self</span><span class="o">.</span><span class="n">menu</span> <span class="o">=</span> <span class="n">menu</span>
<span class="k">class</span> <span class="nc">Sushi</span><span class="p">:</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">ingredients</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">ingredients</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">ValueError</span>
<span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="n">name</span>
<span class="bp">self</span><span class="o">.</span><span class="n">ingredients</span> <span class="o">=</span> <span class="n">ingredients</span>
<span class="k">def</span> <span class="fm">__contains__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">ingredient</span><span class="p">):</span>
<span class="k">return</span> <span class="n">ingredient</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">ingredients</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">is_vegetarian</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">for</span> <span class="n">ingredient</span> <span class="ow">in</span> <span class="p">[</span><span class="s1">'Crab'</span><span class="p">,</span> <span class="s1">'Salmon'</span><span class="p">,</span> <span class="s1">'Shrimp'</span><span class="p">,</span> <span class="s1">'Tuna'</span><span class="p">]:</span>
<span class="k">if</span> <span class="n">ingredient</span> <span class="ow">in</span> <span class="bp">self</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">False</span>
<span class="k">return</span> <span class="bp">True</span>
</pre></div>
<h2>Fixtures</h2>
<p>A fixture named <code>fooshi_bar</code> creates a <code>Restaurant</code> with a variety of
dishes on the menu.</p>
<p>The fixture <code>sushi</code> creates instances based on a name and looking up
ingredients from the <code>session</code> scoped <code>recipes</code> fixture when the test is
being run. Since it is created with <code>params</code> it will not only yield one but
many instances. pytest will then create a number of test items for each of the
test function that uses it.</p>
<p><code>conftest.py</code></p>
<div class="highlight"><pre><span></span><span class="c1"># -*- coding: utf-8 -*-</span>
<span class="kn">import</span> <span class="nn">pytest</span>
<span class="kn">from</span> <span class="nn">sushi</span> <span class="kn">import</span> <span class="n">Restaurant</span><span class="p">,</span> <span class="n">Sushi</span>
<span class="nd">@pytest.fixture</span>
<span class="k">def</span> <span class="nf">fooshi_bar</span><span class="p">():</span>
<span class="sd">"""Returns a Restaurant instance with a great menu."""</span>
<span class="k">return</span> <span class="n">Restaurant</span><span class="p">(</span>
<span class="s1">'Fooshi Bar'</span><span class="p">,</span>
<span class="n">location</span><span class="o">=</span><span class="s1">'Buenos Aires'</span><span class="p">,</span>
<span class="n">menu</span><span class="o">=</span><span class="p">[</span>
<span class="s1">'Ebi Nigiri'</span><span class="p">,</span>
<span class="s1">'Edamame'</span><span class="p">,</span>
<span class="s1">'Inarizushi'</span><span class="p">,</span>
<span class="s1">'Kappa Maki'</span><span class="p">,</span>
<span class="s1">'Miso Soup'</span><span class="p">,</span>
<span class="s1">'Sake Nigiri'</span><span class="p">,</span>
<span class="s1">'Tamagoyaki'</span><span class="p">,</span>
<span class="p">],</span>
<span class="p">)</span>
<span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">scope</span><span class="o">=</span><span class="s1">'session'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">recipes</span><span class="p">():</span>
<span class="sd">"""Return a map from types of sushi to ingredients."""</span>
<span class="k">return</span> <span class="p">{</span>
<span class="s1">'California Roll'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'Rice'</span><span class="p">,</span> <span class="s1">'Cucumber'</span><span class="p">,</span> <span class="s1">'Avocado'</span><span class="p">,</span> <span class="s1">'Crab'</span><span class="p">],</span>
<span class="s1">'Ebi Nigiri'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'Shrimp'</span><span class="p">,</span> <span class="s1">'Rice'</span><span class="p">],</span>
<span class="s1">'Inarizushi'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'Fried tofu'</span><span class="p">,</span> <span class="s1">'Rice'</span><span class="p">],</span>
<span class="s1">'Kappa Maki'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'Cucumber'</span><span class="p">,</span> <span class="s1">'Rice'</span><span class="p">,</span> <span class="s1">'Nori'</span><span class="p">],</span>
<span class="s1">'Maguro Nigiri'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'Tuna'</span><span class="p">,</span> <span class="s1">'Rice'</span><span class="p">,</span> <span class="s1">'Nori'</span><span class="p">],</span>
<span class="s1">'Sake Nigiri'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'Salmon'</span><span class="p">,</span> <span class="s1">'Rice'</span><span class="p">,</span> <span class="s1">'Nori'</span><span class="p">],</span>
<span class="s1">'Tamagoyaki'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'Fried egg'</span><span class="p">,</span> <span class="s1">'Rice'</span><span class="p">,</span> <span class="s1">'Nori'</span><span class="p">],</span>
<span class="s1">'Tsunamayo Maki'</span><span class="p">:</span> <span class="p">[</span><span class="s1">'Tuna'</span><span class="p">,</span> <span class="s1">'Mayonnaise'</span><span class="p">],</span>
<span class="p">}</span>
<span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">params</span><span class="o">=</span><span class="p">[</span>
<span class="s1">'California Roll'</span><span class="p">,</span>
<span class="s1">'Ebi Nigiri'</span><span class="p">,</span>
<span class="s1">'Inarizushi'</span><span class="p">,</span>
<span class="s1">'Kappa Maki'</span><span class="p">,</span>
<span class="s1">'Maguro Nigiri'</span><span class="p">,</span>
<span class="s1">'Sake Nigiri'</span><span class="p">,</span>
<span class="s1">'Tamagoyaki'</span><span class="p">,</span>
<span class="s1">'Tsunamayo Maki'</span><span class="p">,</span>
<span class="p">])</span>
<span class="k">def</span> <span class="nf">sushi</span><span class="p">(</span><span class="n">recipes</span><span class="p">,</span> <span class="n">request</span><span class="p">):</span>
<span class="sd">"""Create a Sushi instance based on recipes."""</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">param</span>
<span class="k">return</span> <span class="n">Sushi</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">ingredients</span><span class="o">=</span><span class="n">recipes</span><span class="p">[</span><span class="n">name</span><span class="p">])</span>
</pre></div>
<h2>Tests</h2>
<p>We define two test functions that both use the <code>sushi</code> fixture.
<code>test_fooshi_serves_vegetarian_sushi</code> also uses <code>fooshi_bar</code> as well as
<code>side_dish</code>, which is dynamically created during collection phase via
<code>mark.parametrize</code>.</p>
<p>Note how <code>sushi</code> is created with <code>indirect=True</code>. Unlike <code>side_dish</code> it
will be passed on to the according fixture function which turns the name of a
type of sushi to an actual instance as explained above.</p>
<p><code>test_sushi.py</code></p>
<div class="highlight"><pre><span></span><span class="c1"># -*- coding: utf-8 -*-</span>
<span class="kn">import</span> <span class="nn">pytest</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span>
<span class="s1">'sushi'</span><span class="p">,</span>
<span class="p">[</span><span class="s1">'Kappa Maki'</span><span class="p">,</span> <span class="s1">'Tamagoyaki'</span><span class="p">,</span> <span class="s1">'Inarizushi'</span><span class="p">],</span>
<span class="n">indirect</span><span class="o">=</span><span class="bp">True</span><span class="p">,</span>
<span class="p">)</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span>
<span class="s1">'side_dish'</span><span class="p">,</span>
<span class="p">[</span><span class="s1">'Edamame'</span><span class="p">,</span> <span class="s1">'Miso Soup'</span><span class="p">],</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">test_fooshi_serves_vegetarian_sushi</span><span class="p">(</span><span class="n">fooshi_bar</span><span class="p">,</span> <span class="n">sushi</span><span class="p">,</span> <span class="n">side_dish</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">sushi</span><span class="o">.</span><span class="n">is_vegetarian</span>
<span class="k">assert</span> <span class="n">sushi</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">fooshi_bar</span><span class="o">.</span><span class="n">menu</span>
<span class="k">assert</span> <span class="n">side_dish</span> <span class="ow">in</span> <span class="n">fooshi_bar</span><span class="o">.</span><span class="n">menu</span>
<span class="k">def</span> <span class="nf">test_sushi</span><span class="p">(</span><span class="n">sushi</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">sushi</span><span class="o">.</span><span class="n">name</span>
<span class="k">assert</span> <span class="n">sushi</span><span class="o">.</span><span class="n">ingredients</span>
</pre></div>
<h1>Test run</h1>
<p>When running these tests we can see that <code>test_sushi</code> is run eight times as
expected as its only fixture <code>sushi</code> is created with eight parameters.</p>
<p>On the other hand <code>test_fooshi_serves_vegetarian_sushi</code> is run six times combining
one value for <code>fooshi_bar</code>, two values for <code>side_dish</code> and three values for
<code>sushi</code>!</p>
<p><code>$ py.test test_sushi.py</code></p>
<pre><code>test_sushi.py::test_fooshi_serves_vegetarian_sushi[Edamame-Kappa Maki] PASSED
test_sushi.py::test_fooshi_serves_vegetarian_sushi[Edamame-Tamagoyaki] PASSED
test_sushi.py::test_fooshi_serves_vegetarian_sushi[Edamame-Inarizushi] PASSED
test_sushi.py::test_fooshi_serves_vegetarian_sushi[Miso Soup-Kappa Maki] PASSED
test_sushi.py::test_fooshi_serves_vegetarian_sushi[Miso Soup-Tamagoyaki] PASSED
test_sushi.py::test_fooshi_serves_vegetarian_sushi[Miso Soup-Inarizushi] PASSED
test_sushi.py::test_sushi[California Roll] PASSED
test_sushi.py::test_sushi[Ebi Nigiri] PASSED
test_sushi.py::test_sushi[Inarizushi] PASSED
test_sushi.py::test_sushi[Kappa Maki] PASSED
test_sushi.py::test_sushi[Maguro Nigiri] PASSED
test_sushi.py::test_sushi[Sake Nigiri] PASSED
test_sushi.py::test_sushi[Tamagoyaki] PASSED
test_sushi.py::test_sushi[Tsunamayo Maki] PASSED
========================== 14 passed in 0.02 seconds ==========================
</code></pre>
<h1>Deferred loading of resources with hooks</h1>
<p>Since test parametrization is performed at collection time, you might want to
set up expensive resources only when the tests that use it are being run.</p>
<p>You can achieve this by using <code>indirect</code> and doing the hard work in fixtures
rather than helper functions directly. This is also available from the
<code>pytest_generate_tests</code> hook:</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">pytest_generate_tests</span><span class="p">(</span><span class="n">metafunc</span><span class="p">):</span>
<span class="k">if</span> <span class="s1">'sushi'</span> <span class="ow">in</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">fixturenames</span><span class="p">:</span>
<span class="n">metafunc</span><span class="o">.</span><span class="n">parametrize</span><span class="p">(</span>
<span class="s1">'sushi'</span><span class="p">,</span>
<span class="p">[</span><span class="s1">'Kappa Maki'</span><span class="p">,</span> <span class="s1">'Tamagoyaki'</span><span class="p">,</span> <span class="s1">'Inarizushi'</span><span class="p">],</span>
<span class="n">indirect</span><span class="o">=</span><span class="bp">True</span><span class="p">,</span>
<span class="p">)</span>
</pre></div>
<p>For more information see the <a href="http://pytest.org/latest/example/parametrize.html#deferring-the-setup-of-parametrized-resources">parametrize pytest docs</a>.</p>
Customize How py.test Collects Tests From Classesurn:uuid:b9997068-d3b9-3aea-bc05-4f8d47cfcbdd2016-04-20T00:00:00ZBruno Oliveira<p>By default py.test collects all classes which start with <code>Test</code> that are found
in <code>test*.py</code> or <code>*test.py</code> modules. This behavior can be configured using the
<code>python_classes</code> and <code>python_files</code> <a href="http://pytest.org/latest/customize.html#confval-python_files">configuration
options</a>.</p>
<p>Also, it will out-of-the box collect classes which subclass <code>unittest.TestCase</code>
(regardless of their name), making it easy to run existing test suites.</p>
<p>But thanks to py.test's flexible plugin architecture, one can customize how
tests are collected from classes to suit almost any requirement.</p>
<h1>Collect All Subclasses of a Certain Class</h1>
<p>Recently someone asked in the
<a href="https://mail.python.org/pipermail/pytest-dev/2016-April/003556.html">pytest-dev</a>
mailing list how to make py.test collect all classes, that subclass from some
specific base class, independently of their name.</p>
<p>Suppose all your test classes subclass from a certain utility class:</p>
<div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">TestingUtils</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">connect_to_database</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="o">...</span>
<span class="k">def</span> <span class="nf">validate_ui</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="o">...</span>
</pre></div>
<p>And all your tests are written in functions in subclasses of <code>TestingUtils</code>,
but that don't follow any particular naming convention:</p>
<div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">BorgTests</span><span class="p">(</span><span class="n">TestingUtils</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">test_borg_creation</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">db</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">connect_to_database</span><span class="p">()</span>
<span class="o">...</span>
<span class="k">class</span> <span class="nc">ValidationRules</span><span class="p">(</span><span class="n">TestingUtils</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">test_rule_1</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="o">...</span>
</pre></div>
<p>You can implement your own collection rules by implementing the
<a href="http://pytest.org/latest/writing_plugins.html#_pytest.hookspec.pytest_pycollect_makeitem">pytest_pycollect_makeitem</a> hook.</p>
<p>Simply add this code to a <code>conftest.py</code> file at the root of your tests'
directory and that's it:</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">inspect</span>
<span class="k">def</span> <span class="nf">pytest_pycollect_makeitem</span><span class="p">(</span><span class="n">collector</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span>
<span class="k">if</span> <span class="n">inspect</span><span class="o">.</span><span class="n">isclass</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">issubclass</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="n">TestingUtils</span><span class="p">):</span>
<span class="n">Class</span> <span class="o">=</span> <span class="n">collector</span><span class="o">.</span><span class="n">_getcustomclass</span><span class="p">(</span><span class="s2">"Class"</span><span class="p">)</span>
<span class="k">return</span> <span class="n">Class</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">parent</span><span class="o">=</span><span class="n">collector</span><span class="p">)</span>
</pre></div>
<p>This won't interfere with the normal test collection mechanism, only add to it,
so classes prefixed with <code>Test</code> will also be collected as usual.</p>
Show Pytest Warningsurn:uuid:fb23d039-764c-3939-9728-b9a78c865aae2016-04-08T00:00:00ZRaphael Pierzina<p>If you are migrating from unittest to pytest, you might encounter warnings when
running your tests. No failures, no errors, but <em>pytest-warnings</em>. This may be
confusing to you, regardless of whether you are new to pytest or an experienced
user.</p>
<h1>Pytest</h1>
<p>In the following example, pytest displays <em>pytest-warnings</em> at the very end of
the test run in the session summary.</p>
<div class="highlight"><pre><span></span>$ py.test
=========================== test session starts ============================
platform darwin -- Python 3.5.0, pytest-2.8.7, py-1.4.31, pluggy-0.3.1
collected 2 items
test_foobar.py ..
=============== 2 passed, 1 pytest-warnings in 0.02 seconds ================
</pre></div>
<h1>Unittest</h1>
<p>Running the same tests under <em>unittest</em> does not show any warnings.</p>
<div class="highlight"><pre><span></span>$ python -m unittest
..
---------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
</pre></div>
<h1>Tests</h1>
<p>As you can see there are two tests that are collected and both pass without any
failures or errors.</p>
<p>Let's have a look at the code:</p>
<div class="highlight"><pre><span></span><span class="c1"># -*- coding: utf-8 -*-</span>
<span class="kn">import</span> <span class="nn">unittest</span>
<span class="k">class</span> <span class="nc">Client</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">get</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">url</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="c1"># Send a real request based on the given parameters</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">TestResponse</span><span class="p">:</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">method</span><span class="p">,</span> <span class="n">url</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">if</span> <span class="s1">'foobar'</span> <span class="ow">in</span> <span class="n">url</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">status</span> <span class="o">=</span> <span class="mi">404</span>
<span class="bp">self</span><span class="o">.</span><span class="n">reason</span> <span class="o">=</span> <span class="s1">'foobar'</span>
<span class="k">else</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">status</span> <span class="o">=</span> <span class="mi">200</span>
<span class="bp">self</span><span class="o">.</span><span class="n">reason</span> <span class="o">=</span> <span class="bp">None</span>
<span class="k">class</span> <span class="nc">TestClient</span><span class="p">(</span><span class="n">Client</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">get</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">url</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">return</span> <span class="n">TestResponse</span><span class="p">(</span><span class="s1">'get'</span><span class="p">,</span> <span class="n">url</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">TestScrapingTool</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">setUp</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">client</span> <span class="o">=</span> <span class="n">TestClient</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">test_success</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">response</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'https://github.com/pytest-dev'</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">status</span><span class="p">,</span> <span class="mi">200</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">reason</span><span class="p">,</span> <span class="bp">None</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">test_failure</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">response</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'foobar'</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">status</span><span class="p">,</span> <span class="mi">404</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">reason</span><span class="p">,</span> <span class="s1">'foobar'</span><span class="p">)</span>
</pre></div>
<p>At first glance, the implementation may look just fine (albeit admittedly not
particularly meaningful). It implements a stub client that inherits from a
<em>real</em> client and returns a <code>TestResponse</code> instance as opposed to sending a
request over a network connection.</p>
<p>Then there are two unittest tests, one for a valid url and another one for an
invalid url respectively. Pytest is perfectly fine with it being
<code>unittest.TestCase</code> methods, so that is unlikely to cause the issue.</p>
<h1>Warnings</h1>
<p>Pytest comes with a <code>-rw</code> command line flag to display internal warnings,
such as the one that is reported for our test session:</p>
<div class="highlight"><pre><span></span>$ py.test -rw
=========================== test session starts ============================
platform darwin -- Python 3.5.0, pytest-2.8.7, py-1.4.31, pluggy-0.3.1
collected 2 items
test_foobar.py ..
========================== pytest-warning summary ==========================
WC1 /show_pytest_warnings/test_foobar.py cannot collect test class
'TestResponse' because it has a __init__ constructor
=============== 2 passed, 1 pytest-warnings in 0.02 seconds ================
</pre></div>
<p>Running the suite with this flag now points us to the source of the problem.
Pytest tries to collect <code>TestResponse</code> as its name matches the naming
conventions for test classes. However it finds a <code>__init__</code> method, which it
cannot understand.</p>
<h1>Solution</h1>
<p>In this particular case, there is an simple solution to what one could consider
a code smell: Rename the classes which are used in your tests but not actual
test cases.</p>
<div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">StubResponse</span><span class="p">:</span>
<span class="o">...</span>
<span class="k">class</span> <span class="nc">StubClient</span><span class="p">(</span><span class="n">Client</span><span class="p">):</span>
<span class="o">...</span>
</pre></div>
<h1>Reporting Options</h1>
<p>For more information about the various reporting options, please consult the
help via <code>$ py.test --help</code> and see the according section:</p>
<div class="highlight"><pre><span></span>-r chars show extra test summary info as specified by chars
(f)ailed, (E)error, (s)skipped, (x)failed, (X)passed
(w)pytest-warnings (a)all.
</pre></div>
Debug Test Failures With Pdburn:uuid:0040be3f-ecb0-36e5-937b-4d4af50b97e62016-02-28T00:00:00ZRaphael Pierzina<h1>Tracebacks</h1>
<p>Pytest excels at helping you with test failures. After running your tests via
py.test, you will see an error report with a detailed traceback for each of the
failures, if any. You can change the mode in which output is presented to you
via the <code>--tb</code> cli option:</p>
<div class="highlight"><pre><span></span>py.test --tb=auto # (default) 'long' tracebacks for the first and last
# entry, but 'short' style for the other entries
py.test --tb=long # exhaustive, informative traceback formatting
py.test --tb=short # shorter traceback format
py.test --tb=line # only one line per failure
py.test --tb=native # Python standard library formatting
py.test --tb=no # no traceback at all
</pre></div>
<h1>Debugger</h1>
<p>Have you ever heard of <a href="https://docs.python.org/3.5/library/pdb.html">pdb - the Python Debugger</a>?
I strongly recommend checking it out, if you haven't used it yet. The standard
library's built-in debugger will prove useful for debugging your Python code
without installing any external dependencies or setting up some fancy IDE.</p>
<p>If invoked with the <code>--pdb</code> option, pytest will place a debugger breakpoint
whenever an error occurs in your tests.</p>
<div class="highlight"><pre><span></span>$ py.test --pdb
</pre></div>
<p>You can also set a debugger breakpoint yourself with <code>import pdb;
pdb.set_trace()</code>.</p>
<p>Use commands <code>help</code>, <code>list</code> and <code>pp</code> to inspect the test code.</p>
<p>If you want to get a list of all local variables and their value, it's best to
run pytest with the <code>-l</code> option enabled, as <code>locals()</code> may contain internal
pytest items if you print it from the breakpoint.</p>
<h1>Running Broken Tests Only</h1>
<p>An underrated feature of pytest is the <code>--lf</code> option, which tells pytest to
only run the tests that failed the last time it was executed.</p>
<p>Once you've encountered any errors in your tests, you want to focus on the
failures and get a better understanding of what's causing the problems as
opposed to spending time on running tests that are perfectly fine.</p>
<h1>Re-order Tests. Failures First</h1>
<p>Keep in mind though, whenever you modify your code other tests may break.</p>
<p>If you run <code>$ py.test --ff</code> all tests will be executed, but re-ordered based
on whether they've failed in the previous run or not. Failures first,
successful tests after.</p>
<h1>Stop Tests After N Failures</h1>
<p>You can end the test session early by pressing <code>CTRL-C</code>. Although pytest
catches the <code>KeyboardInterrupt</code> error and runs the teardown code before
closing the session gracefully, it is definitely not the best way to stop the
test suite.</p>
<p>Instead use <code>-x</code> to stop after the first failure, or <code>--maxfail=N</code> to stop
after the first N failures.</p>
Create Tests via Parametrizationurn:uuid:2a28aae5-cb71-3cb3-8195-f6c32fb094112016-02-23T00:00:00ZRaphael Pierzina<h1>Use Case</h1>
<p>Imagine you want to write a test for a particular function, but for multiple input values.
Writing a for-loop is a bad idea as the test will fail as soon as it hits the first <code>AssertionError</code>.
Subsequent input values will not be tested and you have no idea which part of your code is actually broken.
At the same time you want to stick to DRY and not implement the same <code>unittest.Testcase</code>
method over and over again with slightly different input values.</p>
<p>Keep in mind <strong>why</strong> we write unit tests:</p>
<p><strong>We want to know when we break stuff, but also at the same time get as many hints as possible on why the error occurs!</strong></p>
<p>Pytest provides various ways of creating individual test items. There are parametrized fixtures and mark.parametrize (and hooks).</p>
<h1>Marker</h1>
<p>Using this built-in marker you do not need to implement any fixtures. Instead you define your scenarios in a decorator and the only
thing you really need to look out for is to match the number of positional test arguments with your iterable.</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pytest</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span>
<span class="s1">'number, word'</span><span class="p">,</span> <span class="p">[</span>
<span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s1">'1'</span><span class="p">),</span>
<span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="s1">'Fizz'</span><span class="p">),</span>
<span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="s1">'Buzz'</span><span class="p">),</span>
<span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="s1">'Buzz'</span><span class="p">),</span>
<span class="p">(</span><span class="mi">15</span><span class="p">,</span> <span class="s1">'FizzBuzz'</span><span class="p">),</span>
<span class="p">(</span><span class="mi">16</span><span class="p">,</span> <span class="s1">'16'</span><span class="p">)</span>
<span class="p">]</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">test_fizzbuzz</span><span class="p">(</span><span class="n">number</span><span class="p">,</span> <span class="n">word</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">fizzbuzz</span><span class="p">(</span><span class="n">number</span><span class="p">)</span> <span class="o">==</span> <span class="n">word</span>
</pre></div>
<h1>Fixture</h1>
<p>To parametrize a fixture you need pass an interable to the <code>params</code> keyword argument.
The built-in fixture <code>request</code> knows about the current parameter and if you don't want to do anything fancy,
you can pass it right to the test via the <code>return</code> statement.</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pytest</span>
<span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">params</span><span class="o">=</span><span class="p">[</span><span class="s1">'apple'</span><span class="p">,</span> <span class="s1">'banana'</span><span class="p">,</span> <span class="s1">'plum'</span><span class="p">])</span>
<span class="k">def</span> <span class="nf">fruit</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="n">request</span><span class="o">.</span><span class="n">param</span>
<span class="k">def</span> <span class="nf">test_is_healthy</span><span class="p">(</span><span class="n">fruit</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">is_healthy</span><span class="p">(</span><span class="n">fruit</span><span class="p">)</span>
</pre></div>
<h1>Example Implementation</h1>
<p><em>Please note that the examples are written in Python3</em></p>
<p>Sometimes you may find yourself struggling to chose which is the best way to parametrize your tests.
At the end of the day it really depends on what you want to test. But...
Good news! Pytest lets you combine both methods to get the most out of both worlds.</p>
<h2>Some Classes in a Module</h2>
<p>Imagine this Python module (<code>foobar.py</code>) which contains a few class definitions with a bit of logic:</p>
<div class="highlight"><pre><span></span><span class="c1"># -*- coding: utf-8 -*-</span>
<span class="n">FOSS_LICENSES</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'Apache 2.0'</span><span class="p">,</span> <span class="s1">'MIT'</span><span class="p">,</span> <span class="s1">'GPL'</span><span class="p">,</span> <span class="s1">'BSD'</span><span class="p">]</span>
<span class="n">PYTHON_PKGS</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'pytest'</span><span class="p">,</span> <span class="s1">'requests'</span><span class="p">,</span> <span class="s1">'django'</span><span class="p">,</span> <span class="s1">'cookiecutter'</span><span class="p">]</span>
<span class="k">class</span> <span class="nc">Package</span><span class="p">:</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">license</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="n">name</span>
<span class="bp">self</span><span class="o">.</span><span class="n">license</span> <span class="o">=</span> <span class="n">license</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">is_open_source</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">license</span> <span class="ow">in</span> <span class="n">FOSS_LICENSES</span>
<span class="k">class</span> <span class="nc">Person</span><span class="p">:</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">gender</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="n">name</span>
<span class="bp">self</span><span class="o">.</span><span class="n">gender</span> <span class="o">=</span> <span class="n">gender</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_skills</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'eating'</span><span class="p">,</span> <span class="s1">'sleeping'</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">learn</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">skill</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_skills</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">skill</span><span class="p">)</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">looks_like_a_programmer</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">any</span><span class="p">(</span>
<span class="n">package</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_skills</span>
<span class="k">for</span> <span class="n">package</span> <span class="ow">in</span> <span class="n">PYTHON_PKGS</span>
<span class="p">)</span>
<span class="k">class</span> <span class="nc">Woman</span><span class="p">(</span><span class="n">Person</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="s1">'female'</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">Man</span><span class="p">(</span><span class="n">Person</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="s1">'male'</span><span class="p">)</span>
</pre></div>
<h2>Tests in a Separate Module</h2>
<p>With only two few lines of pytest code, we can create loads of different scenarios that we would like to test.
By re-using parametrized fixtures and applying the aforementioned markers to your tests, you can focus on the actual test implementation, as opposed to
writing the same boilerplate code for each of the methods that you would have to write with <code>unittest.TestCase</code>.</p>
<div class="highlight"><pre><span></span><span class="c1"># -*- coding: utf-8 -*-</span>
<span class="kn">import</span> <span class="nn">operator</span>
<span class="kn">import</span> <span class="nn">pytest</span>
<span class="kn">from</span> <span class="nn">foobar</span> <span class="kn">import</span> <span class="n">Package</span><span class="p">,</span> <span class="n">Woman</span><span class="p">,</span> <span class="n">Man</span>
<span class="n">PACKAGES</span> <span class="o">=</span> <span class="p">[</span>
<span class="n">Package</span><span class="p">(</span><span class="s1">'requests'</span><span class="p">,</span> <span class="s1">'Apache 2.0'</span><span class="p">),</span>
<span class="n">Package</span><span class="p">(</span><span class="s1">'django'</span><span class="p">,</span> <span class="s1">'BSD'</span><span class="p">),</span>
<span class="n">Package</span><span class="p">(</span><span class="s1">'pytest'</span><span class="p">,</span> <span class="s1">'MIT'</span><span class="p">),</span>
<span class="p">]</span>
<span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">params</span><span class="o">=</span><span class="n">PACKAGES</span><span class="p">,</span> <span class="n">ids</span><span class="o">=</span><span class="n">operator</span><span class="o">.</span><span class="n">attrgetter</span><span class="p">(</span><span class="s1">'name'</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">python_package</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="n">request</span><span class="o">.</span><span class="n">param</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span><span class="s1">'person'</span><span class="p">,</span> <span class="p">[</span>
<span class="n">Woman</span><span class="p">(</span><span class="s1">'Audrey'</span><span class="p">),</span> <span class="n">Woman</span><span class="p">(</span><span class="s1">'Brianna'</span><span class="p">),</span>
<span class="n">Man</span><span class="p">(</span><span class="s1">'Daniel'</span><span class="p">),</span> <span class="n">Woman</span><span class="p">(</span><span class="s1">'Ola'</span><span class="p">),</span> <span class="n">Man</span><span class="p">(</span><span class="s1">'Kenneth'</span><span class="p">)</span>
<span class="p">])</span>
<span class="k">def</span> <span class="nf">test_become_a_programmer</span><span class="p">(</span><span class="n">person</span><span class="p">,</span> <span class="n">python_package</span><span class="p">):</span>
<span class="n">person</span><span class="o">.</span><span class="n">learn</span><span class="p">(</span><span class="n">python_package</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
<span class="k">assert</span> <span class="n">person</span><span class="o">.</span><span class="n">looks_like_a_programmer</span>
<span class="k">def</span> <span class="nf">test_is_open_source</span><span class="p">(</span><span class="n">python_package</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">python_package</span><span class="o">.</span><span class="n">is_open_source</span>
</pre></div>
<h1>Test Report</h1>
<p>Going the extra mile and setting up <code>ids</code> for your test scenarios greatly increases
the comprehensibilty of your test report. In this case we would like to display the name
of each <code>Package</code> rather than the fixture name with a numbered suffix such as <code>python_package2</code>.</p>
<p>If you run the tests now, you will see that pytest created 18 <strong>individual tests</strong> for us (Yes, yes indeed. 18 = 3 * 5 + 3).</p>
<div class="highlight"><pre><span></span>$ py.test -v
================================== test session starts ==================================
platform darwin -- Python 3.5.0, pytest-2.8.7, py-1.4.31, pluggy-0.3.1
collected 18 items
test_foobar.py::test_become_a_programmer[requests-person0] PASSED
test_foobar.py::test_become_a_programmer[requests-person1] PASSED
test_foobar.py::test_become_a_programmer[requests-person2] PASSED
test_foobar.py::test_become_a_programmer[requests-person3] PASSED
test_foobar.py::test_become_a_programmer[requests-person4] PASSED
test_foobar.py::test_become_a_programmer[django-person0] PASSED
test_foobar.py::test_become_a_programmer[django-person1] PASSED
test_foobar.py::test_become_a_programmer[django-person2] PASSED
test_foobar.py::test_become_a_programmer[django-person3] PASSED
test_foobar.py::test_become_a_programmer[django-person4] PASSED
test_foobar.py::test_become_a_programmer[pytest-person0] PASSED
test_foobar.py::test_become_a_programmer[pytest-person1] PASSED
test_foobar.py::test_become_a_programmer[pytest-person2] PASSED
test_foobar.py::test_become_a_programmer[pytest-person3] PASSED
test_foobar.py::test_become_a_programmer[pytest-person4] PASSED
test_foobar.py::test_is_open_source[requests] PASSED
test_foobar.py::test_is_open_source[django] PASSED
test_foobar.py::test_is_open_source[pytest] PASSED
=============================== 18 passed in 0.02 seconds ===============================
</pre></div>
Shared Directory Between Master and Workers With pytest-xdisturn:uuid:f47bb838-047d-3a2b-8997-e65bf1230ce42016-02-22T00:00:00ZBruno Oliveira<h1>pytest-xdist</h1>
<p><a href="https://github.com/pytest-dev/pytest-xdist">pytest-xdist</a> is a plugin for
distributed testing.</p>
<p>A somewhat common need is how one can make a plugin share data between
<code>master</code> and <code>worker</code> nodes?</p>
<h1>Shared directory</h1>
<p>Here's a simple recipe that provides a <code>shared_directory</code> fixture which
points to a temporary directory accessible by both <code>masters</code> and <code>worker</code> nodes.</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">shutil</span>
<span class="kn">import</span> <span class="nn">tempdir</span>
<span class="k">def</span> <span class="nf">pytest_configure</span><span class="p">(</span><span class="n">config</span><span class="p">):</span>
<span class="k">if</span> <span class="n">is_master</span><span class="p">(</span><span class="n">config</span><span class="p">):</span>
<span class="n">config</span><span class="o">.</span><span class="n">shared_directory</span> <span class="o">=</span> <span class="n">tempdir</span><span class="o">.</span><span class="n">mktemp</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">pytest_unconfigure</span><span class="p">(</span><span class="n">config</span><span class="p">):</span>
<span class="k">if</span> <span class="n">is_master</span><span class="p">(</span><span class="n">config</span><span class="p">):</span>
<span class="n">shutil</span><span class="o">.</span><span class="n">rmtree</span><span class="p">(</span><span class="n">config</span><span class="o">.</span><span class="n">shared_directory</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">pytest_configure_node</span><span class="p">(</span><span class="n">node</span><span class="p">):</span>
<span class="sd">"""xdist hook"""</span>
<span class="n">node</span><span class="o">.</span><span class="n">slaveinput</span><span class="p">[</span><span class="s1">'shared_dir'</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">shared_directory</span>
<span class="k">def</span> <span class="nf">is_master</span><span class="p">(</span><span class="n">config</span><span class="p">):</span>
<span class="sd">"""True if the code running the given pytest.config object is running in a xdist master</span>
<span class="sd"> node or not running xdist at all.</span>
<span class="sd"> """</span>
<span class="k">return</span> <span class="ow">not</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">config</span><span class="p">,</span> <span class="s1">'slaveinput'</span><span class="p">)</span>
<span class="nd">@pytest.fixture</span>
<span class="k">def</span> <span class="nf">shared_directory</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="sd">"""Returns a unique and temporary directory which can be shared by</span>
<span class="sd"> master or worker nodes in xdist runs.</span>
<span class="sd"> """</span>
<span class="k">if</span> <span class="n">is_master</span><span class="p">(</span><span class="n">request</span><span class="o">.</span><span class="n">config</span><span class="p">):</span>
<span class="k">return</span> <span class="n">request</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">shared_directory</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">request</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">slaveinput</span><span class="p">[</span><span class="s1">'shared_dir'</span><span class="p">]</span>
</pre></div>
<p>Anything put in <code>node.slaveinput</code> dictionary during the <code>pytest_configure_node</code>
xdist hook can be accessed by the worker nodes later. You can put any
simple builtin type, like lists, strings, tuples, etc.</p>
<p>The <code>shared_directory</code> fixture can then be used by a plugin or even tests
to have a common directory where information can be exchanged. This
recipe also shows how one can find out if the code is running in the master
or a worker node.</p>
<p>(this recipe appeared originally in
<a href="https://github.com/pytest-dev/pytest/issues/1402#issuecomment-186299177">this pytest issue</a>)</p>
Run Tests Using a Certain Fixtureurn:uuid:c183eb24-5344-3bf5-b4d2-5285208d5f142016-02-21T00:00:00ZRaphael Pierzina<h1>Hooks</h1>
<p>The hook-based plugin architecture of pytest empowers you to heavily customize the test runners behavior. In this particular case we'll make use of the <code>pytest_collection_modifyitems</code> hook. It is called after collection has been performed and hence provides access to the actual test items - objects which represent a single test.</p>
<h1>Fixtures</h1>
<p>Hooks have access to pytest's builtin fixtures such as <code>config</code>. We case use it to explicitly run a hook for deselecting test items, namely <code>config.hook.pytest_deselected</code>.</p>
<h1>Items</h1>
<p>Watch out! This hook does not return the test items. Instead you want to populate the given list as needed: <code>items[:] = selected_items</code></p>
<h1>Implicit Fixture Usage</h1>
<p>Pytests modular design allows you to re-use fixtures. The <code>fixturenames</code> attribute of a test item not only holds the fixtures which are specified in its signature but also the ones that are used implicitly.</p>
<h1>Example Implementation</h1>
<div class="highlight"><pre><span></span><span class="c1"># conftest.py</span>
<span class="k">def</span> <span class="nf">pytest_collection_modifyitems</span><span class="p">(</span><span class="n">items</span><span class="p">,</span> <span class="n">config</span><span class="p">):</span>
<span class="n">selected_items</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">deselected_items</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">items</span><span class="p">:</span>
<span class="k">if</span> <span class="s1">'new_fixture'</span> <span class="ow">in</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s1">'fixturenames'</span><span class="p">,</span> <span class="p">()):</span>
<span class="n">selected_items</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">deselected_items</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="n">config</span><span class="o">.</span><span class="n">hook</span><span class="o">.</span><span class="n">pytest_deselected</span><span class="p">(</span><span class="n">items</span><span class="o">=</span><span class="n">deselected_items</span><span class="p">)</span>
<span class="n">items</span><span class="p">[:]</span> <span class="o">=</span> <span class="n">selected_items</span>
</pre></div>
<div class="highlight"><pre><span></span><span class="c1"># test_foobar.py</span>
<span class="kn">import</span> <span class="nn">pytest</span>
<span class="nd">@pytest.fixture</span>
<span class="k">def</span> <span class="nf">new_fixture</span><span class="p">():</span>
<span class="k">return</span> <span class="s1">'foobar'</span>
<span class="nd">@pytest.fixture</span>
<span class="k">def</span> <span class="nf">another_fixture</span><span class="p">(</span><span class="n">new_fixture</span><span class="p">):</span>
<span class="k">return</span> <span class="s1">'foobar'</span>
<span class="k">def</span> <span class="nf">test_abc</span><span class="p">(</span><span class="n">new_fixture</span><span class="p">):</span>
<span class="k">assert</span> <span class="bp">True</span>
<span class="k">def</span> <span class="nf">test_def</span><span class="p">(</span><span class="n">another_fixture</span><span class="p">):</span>
<span class="k">assert</span> <span class="bp">True</span>
<span class="k">def</span> <span class="nf">test_that_does_not_use_fixture</span><span class="p">():</span>
<span class="k">assert</span> <span class="bp">False</span>
</pre></div>
<h1>Test Report</h1>
<div class="highlight"><pre><span></span>$ py.test -v
============================= test session starts ==============================
platform darwin -- Python 3.5.0, pytest-2.8.7, py-1.4.31, pluggy-0.3.1
collected 3 items
test_foobar.py::test_abc PASSED
test_foobar.py::test_def PASSED
==================== 2 passed, 1 deselected in 0.01 seconds ====================
</pre></div>