Shared Directory Between Master and Workers With pytest-xdist (2016)

Post

Date: Feb 22, 2016
Author: Bruno Oliveira
Tags: fixture, hook, xdist

TL;DR

Use request.config.slaveinput to send information from master to worker nodes in pytest-xdist

Follow @nicoddemus Star Fork

pytest-xdist

A somewhat common need is how one can make a plugin share data between master and worker nodes?

Shared directory

Here's a simple recipe that provides a shared_directory fixture which points to a temporary directory accessible by both masters and worker nodes.

import shutil
import tempdir


def pytest_configure(config):
    if is_master(config):
        config.shared_directory = tempdir.mktemp()


def pytest_unconfigure(config):
    if is_master(config):
        shutil.rmtree(config.shared_directory)


def pytest_configure_node(node):
    """xdist hook"""
    node.slaveinput['shared_dir'] = node.config.shared_directory


def is_master(config):
    """True if the code running the given pytest.config object is running in a xdist master
    node or not running xdist at all.
    """
    return not hasattr(config, 'slaveinput')


@pytest.fixture
def shared_directory(request):
    """Returns a unique and temporary directory which can be shared by
    master or worker nodes in xdist runs.
    """
    if is_master(request.config):
        return request.config.shared_directory
    else:
        return request.config.slaveinput['shared_dir']

Anything put in node.slaveinput dictionary during the pytest_configure_node xdist hook can be accessed by the worker nodes later. You can put any simple builtin type, like lists, strings, tuples, etc.

The shared_directory fixture can then be used by a plugin or even tests to have a common directory where information can be exchanged. This recipe also shows how one can find out if the code is running in the master or a worker node.

pytest-tricks