We recently achieved 1000+ active installs of our Bootstrap Blocks plugin. By reaching this milestone the pressure to keep the plugin backward compatible while changing existing features or implementing new ones increased. At this point during development we had to manually check if older block content still works with the latest version of the plugin. This of course took more and more time by the growing number of plugin versions.

That‘s why we had to start implement automated tests which supported us doing this.

In this tutorial we will implement End-to-End (E2E) tests for a modified version of the wrapper block from this tutorial. Our enhanced wrapper block makes it possible to set a background color, enable a margin after the block as well as setting the alignment of the block content.

Wrapper Block in the editor

This is the implementation of our wrapper block:

/**
 * BLOCK: e2e-tests-example/wrapper-block
 */

const { __ } = wp.i18n; // Translate function
const { registerBlockType } = wp.blocks; // Function to register our block
const {
    Fragment, // Used to wrap our edit component and only have one root element
} = wp.element;
const {
    InnerBlocks, // Allows it to place child blocks inside our block
    InspectorControls, // We place our select control inside the inspector controls which show up at the right side of the editor
    AlignmentToolbar, // This is the alignment toolbar to select the alignment of the block content
    BlockControls, // We place our alignment toolbar inside the block controls which show up above the selected block
} = wp.blockEditor;
const {
    PanelBody, // A panel where we place our select control in (creates a collapsible element)
    SelectControl, // Our select control to choose the background color
    CheckboxControl, // We use the checkbox control to enable the margin bottom
} = wp.components;
const {
    applyFilters, // We use this function to make the background color options filterable
} = wp.hooks;

let bgColorOptions = [
    {
        value: '',
        label: __( 'No Background Color', 'e2e-tests-example' ),
    },
    {
        value: 'paleturquoise',
        label: __( 'Light Blue', 'e2e-tests-example' ),
    },
    {
        value: 'orange',
        label: __( 'Orange', 'e2e-tests-example' ),
    },
];
bgColorOptions = applyFilters( 'e2eTestsExample.wrapperBlock.bgColorOptions', bgColorOptions );

const prepareStyles = ( bgColor, alignment, marginBottom ) => {
    return {
        backgroundColor: bgColor && bgColor !== '' ? bgColor : null,
        marginBottom: marginBottom ? '60px' : null,
        textAlign: alignment && alignment !== '' ? alignment : null,
    };
};

registerBlockType( 'e2e-tests-example/wrapper-block', {
    title: __( 'Wrapper Block', 'e2e-tests-example' ), // Block title.
    icon: 'editor-table', // Block icon from Dashicons
    category: 'layout', // Block category
    keywords: [
        __( 'Wrapper Block', 'e2e-tests-example' ),
    ],

    attributes: {
        bgColor: {
            type: 'string',
        },
        alignment: {
            type: 'string',
        },
        marginBottom: {
            type: 'boolean',
        },
    },

    edit( { attributes, setAttributes, className } ) {
        const {
            bgColor = '',
            alignment = '',
            marginBottom = false,
        } = attributes;

        return (
            <Fragment>
                <InspectorControls>
                    <PanelBody
                        title={ __( 'Background Color', 'e2e-tests-example' ) }
                        initialOpen={ false }
                    >
                        <SelectControl
                            label={ __( 'Background Color', 'e2e-tests-example' ) }
                            value={ bgColor }
                            options={ bgColorOptions }
                            onChange={ ( selectedOption ) => setAttributes( { bgColor: selectedOption } ) }
                        />
                    </PanelBody>
                    <PanelBody
                        title={ __( 'Margin bottom', 'e2e-tests-example' ) }
                        initialOpen={ false }
                    >
                        <CheckboxControl
                            label={ __( 'Add margin bottom', 'e2e-tests-example' ) }
                            checked={ marginBottom }
                            onChange={ ( isChecked ) => setAttributes( { marginBottom: isChecked } ) }
                        />
                    </PanelBody>
                </InspectorControls>
                <BlockControls>
                    <AlignmentToolbar
                        value={ alignment }
                        label={ __( 'Change wrapper block alignment', 'e2e-tests-example' ) }
                        onChange={ ( newAlignment ) => ( setAttributes( { alignment: newAlignment } ) ) }
                    />
                </BlockControls>
                <div
                    className={ className }
                    style={ prepareStyles( bgColor, alignment, marginBottom ) }
                >
                    <InnerBlocks />
                </div>
            </Fragment>
        );
    },

    save( { attributes } ) {
        const {
            bgColor = '',
            alignment = '',
            marginBottom = false,
        } = attributes;

        return (
            <div
                style={ prepareStyles( bgColor, alignment, marginBottom ) }
            >
                <InnerBlocks.Content />
            </div>
        );
    },
} );

You'll also find the block implementation (and all the described tests) in this fully working WordPress plugin: https://github.com/liip/e2e-tests-example-wp-plugin.

Setting up the test framework

We will use the @wordpress/scripts package as a base for building our E2E tests. For this you need to have a working WordPress development environment running on your machine. If you didn't already set it up please do this by following this tutorial: Setting up local dev environment for WordPress with @wordpress/env.

The @wordpress/scripts package comes with a set of commands for running tests. One of it is wp-scripts test-e2e which launches the End-To-End (E2E) test runner based on Jest in combination with Puppeteer.

To use the tests we have to have to add an additional script to our package.json file:

// package.json
"scripts": {
  "test:e2e": "wp-scripts test-e2e"
}

With this we can later execute the tests by running this command:

npm run test:e2e

To make writing tests based on Puppeteer a little easier the team behind Gutenberg released the @wordpress/e2e-test-utils package which provides some utility functions. Let's add it to our setup:

npm install @wordpress/e2e-test-utils --save-dev

Our setup should now be ready to write the first tests.

Base structure of the tests

The test:e2e command looks for test files which have the following name pattern and runs them:

  • Files with .js (or .ts) suffix at any level of depth in spec/ folders.
  • Files with .spec.js (or .spec.ts) suffix.

I usually create a folder called e2e-tests/ in my project root and create my test files (with a .spec.js suffix) there but it's up to you how you call this folder.

Let's create a file called wrapper-block.spec.js inside our test folder which should contain all basic tests for our wrapper block.

The base structure for each test file looks like this:

// wrapper-block.spec.js
import {
    createNewPost,
    enablePageDialogAccept,
} from '@wordpress/e2e-test-utils';

describe( 'Wrapper block', () => {
    beforeAll( async () => {
        await enablePageDialogAccept();
    } );
    beforeEach( async () => {
        await createNewPost();
    } );

    // Tests can be added here by using the it() function
    it( 'Should test something', () => {} )
} );

Each test file groups tests by wrapping them in a describe() function call. Inside this function we can define our tests as well as some lifecycle methods which run at a specific time during the test run.

We first need to call enablePageDialogAccept() before running the tests (inside the beforeAll() lifecycle method). This ensures that page dialogs which might appear while navigating away from a Gutenberg page are automatically accepted and the test runner can continue.

Additionally we need to create a new post by calling the createNewPost() function before each test (inside the beforeEach() lifecycle function). By doing this we don't run into any side effects when running different tests with the same post.

Let's write our first test now.

Implement the first test

One of the easiest but also most effective test you can write is the one that simply adds your custom block to the content and checks if no error appeared. With this test you are already save from syntax errors, missing imports or anything else which might break the JavaScript of your block.

We do this with the following test:

import {
    getEditedPostContent,
    insertBlock,
} from '@wordpress/e2e-test-utils';

it( 'Wrapper block should be available', async () => {
    await insertBlock( 'Wrapper Block' );

    // Check if block was inserted
    expect( await page.$( '[data-type="e2e-tests-example/wrapper-block"]' ) ).not.toBeNull();

    expect( await getEditedPostContent() ).toMatchSnapshot();
} );

Let me explain what this test does:

We start by inserting a Wrapper Block. This is done by using the utility function insertBlock() from the @wordpress/e2e-test-utils package.

await insertBlock( 'Wrapper Block' );

There are some things you should know about this function which might occur when testing your own blocks. The function opens the block inserter panel, types the given block name into the search bar and inserts the first block in the search results by clicking on it. If there are other blocks which match the given name a wrong block might be inserted and your tests will fail. So always try to search for your block manually and check the results before using this function. If there are other blocks above your own you can also just add a more specifc keyword which you can search for when registring your block (in registerBlockType()).

After the block was inserted we try to find the block with a CSS selector ([data-type="e2e-tests-example/wrapper-block"]) in the content. We expect the result not to be null which would be returned by page.$ if no element was found.

expect( await page.$( '[data-type="e2e-tests-example/wrapper-block"]' ) ).not.toBeNull();

We create a snapshot of the whole content and check if it matches a reference snapshot.

expect( await getEditedPostContent() ).toMatchSnapshot();

It there isn't a snapshot yet the test runner will automatically generate one during the first run. With this we can always be sure that our block attributes are correctly set and the block content matches our expectations.

As you might guess the snapshots generated in the last step need to be updated once our block attributes or content changes. This can be done by running the tests with the -u option.

npm run test:e2e -- -u

By setting this option all snapshots will be updated.

Tests for block attributes

Next to the basic test if the block can be inserted correctly we should always implement tests for everything which modifies the block output. These output variations are normally a result of different settings in the block attributes. That's why we should implement tests for every attribute.

There are 3 attributes in our wrapper block:

  • Background Color (bgColor) which can be chosen using a SelectControl.
  • Margin bottom (marginBottom) which can be enabled using a CheckboxControl.
  • Alignment (alignment) which can be set using an AlignmentToolbar.

Let's write a test for each of them.

Background Color (SelectControl)

The background color can be selected with a SelectControl component inside the InspectorControls of the block. Here's our test for that:

it( 'Background color should be applied', async () => {
    await insertBlock( 'Wrapper Block' );
    await selectBlockByName( 'e2e-tests-example/wrapper-block' );

    // Change background color
    await openSidebarPanelWithTitle( 'Background Color' );
    await selectOption( 'Background Color', 'orange' );

    expect( await getEditedPostContent() ).toMatchSnapshot();
} );

Let's go through the test line by line:

We again insert a Wrapper Block as we did in the first test.

await insertBlock( 'Wrapper Block' );

After that we select the block in the editor so that the InspectorControls of the block are visible.

await selectBlockByName( 'e2e-tests-example/wrapper-block' );

The selectBlockByName() function is a custom function which isn't included in the @wordpress/e2e-test-utils package. Let's have a look at it:

import {
    getAllBlocks,
    selectBlockByClientId,
} from '@wordpress/e2e-test-utils';

export const selectBlockByName = async ( name ) => {
    await selectBlockByClientId(
        ( await getAllBlocks() ).find( ( block ) => block.name === name ).clientId
    );
};

This function first retrieves all blocks in the content by calling getAllBlocks(). It iterates through all of them and looks for the block which has the given name. When the block was found we use its clientId to select it in the editor by passing it to the selectBlockByClientId() utility function.

It's a good practice to add missing utility functions to an own helper file and export it for the usage in our tests. This helps keeping the readability of the tests itself as good as possible. The helper functions can quickly get pretty complex as you will see later on. That's why I recommend you to add the function above into a new file called helper.js. After that import the function in our test file (wrapper-block.spec.js):

import {
    selectBlockByName,
} from './helper';

After selecting the block in the editor we have to open the background color panel in the InspectorControl.

await openSidebarPanelWithTitle( 'Background Color' );

We're using again a custom helper function (openSidebarPanelWithTitle()) for that:

export const openSidebarPanelWithTitle = async ( title ) => {
    // Check if sidebar panel exists
    await page.waitForXPath( `//div[contains(@class,"edit-post-sidebar")]//button[@class="components-button components-panel__body-toggle"][contains(text(),"${ title }")]` );

    // Only open panel if it's not expanded already (aria-expanded check)
    const [ panel ] = await page.$x(
        `//div[contains(@class,"edit-post-sidebar")]//button[@class="components-button components-panel__body-toggle"][@aria-expanded="false"][contains(text(),"${ title }")]`
    );
    if ( panel ) {
        await panel.click();
    }
};

The function looks for the sidebar panel with the given title. When it's found it checks if the panel is already opened by checking the aria-expandend attribute. If not it clicks on the panel to open it. I won't go too much into detail of the implementation to keep the focus on writing the test itself.

We again import the function into our test file:

import {
    openSidebarPanelWithTitle,
} from './helper';

Now that the sidebar panel is opened we can select the background color by chosing it in the SelectControl.

await selectOption( 'Background Color', 'orange' );

As you might have guessed the selectOption() function is again a custom helper function:

export const selectOption = async ( label, value ) => {
    const [ selectEl ] = await page.$x( `//label[@class="components-base-control__label"][contains(text(),"${ label }")]/following-sibling::select[@class="components-select-control__input"]` );
    const selectId = await page.evaluate(
        ( el ) => el.id,
        selectEl
    );
    await page.select( `#${ selectId }`, value );
};

The function searches for a select element with the given label text. If it's found it gets the id of the element and passes it and the given value to the official page.select() function of Puppeteer which takes an id and a value as parameters.

The background color should now be selected and we can again check if the editor content matches our expected content by comparing it with a snapshot.

expect( await getEditedPostContent() ).toMatchSnapshot();

Margin Bottom (CheckboxControl)

Our next test will validate if the margin bottom gets correctly added to the block if we check the associated CheckboxControl in the InspectorControls.

it( 'Margin bottom should be applied', async () => {
    await insertBlock( 'Wrapper Block' );
    await selectBlockByName( 'e2e-tests-example/wrapper-block' );

    // Apply margin bottom
    await openSidebarPanelWithTitle( 'Margin bottom' );
    await clickElementByText( 'label', 'Add margin bottom' );

    expect( await getEditedPostContent() ).toMatchSnapshot();
} );

Let's again go through the test line by line. As in our previous tests we insert a Wrapper Block and select it.

await insertBlock( 'Wrapper Block' );
await selectBlockByName( 'e2e-tests-example/wrapper-block' );

After that we open the Margin bottom panel in the InspectorControls.

await openSidebarPanelWithTitle( 'Margin bottom' );

Now we enable the Margin bottom checkbox by clicking on its label.

await clickElementByText( 'label', 'Add margin bottom' );

We again use a custom helper function clickElementByText(). Here's the implementation of it:

export const clickElementByText = async ( elementExpression, text ) => {
    const [ element ] = await page.$x( `//${ elementExpression }[contains(text(),"${ text }")]` );
    await element.click();
};

It first gets the label element with the given text. Afterwards it triggers a click event on the label.

Remember to import this function in our test file:

import {
    clickElementByText,
} from './helper';

At last we again check if the editor content matches our expected content by comparing it with a snapshot.

expect( await getEditedPostContent() ).toMatchSnapshot();

Alignment

The last attribute we need to test is the alignment attribute. It is set by selecting a block alignment in the BlockControls. Here's the test for that:

import {
    clickBlockToolbarButton,
    clickButton,
} from '@wordpress/e2e-test-utils';

it( 'Alignment should be set', async () => {
    await insertBlock( 'Wrapper Block' );
    await selectBlockByName( 'e2e-tests-example/wrapper-block' );

    // Change alignment
    await clickBlockToolbarButton( 'Change wrapper block alignment' );
    await clickButton( 'Align Text Center' );

    expect( await getEditedPostContent() ).toMatchSnapshot();
} );

The test again inserts a Wrapper Block and selects it.

await insertBlock( 'Wrapper Block' );
await selectBlockByName( 'e2e-tests-example/wrapper-block' );

Afterwards it clicks on the toolbar button with the given label Change wrapper block alignment to open the alignment options.

await clickBlockToolbarButton( 'Change wrapper block alignment' );

At last it clicks the Align Text Center button.

await clickButton( 'Align Text Center' );

These two functions (clickBlockToolbarButton and clickButton) are official utility functions form the @wordpress/e2e-test-utils package. That's why we need to import them from it.

import {
    clickBlockToolbarButton,
    clickButton,
} from '@wordpress/e2e-test-utils';

The last step is again to compare the editor content with a snapshot of it to check if the attribute and the block content gets correctly set.

expect( await getEditedPostContent() ).toMatchSnapshot();

Wrapping up

Let's have a look at the full implementation of our tests (including the helper methods):

// helper.js

import {
    getAllBlocks,
    selectBlockByClientId,
} from '@wordpress/e2e-test-utils';

export const selectBlockByName = async ( name ) => {
    await selectBlockByClientId(
        ( await getAllBlocks() ).find( ( block ) => block.name === name ).clientId
    );
};

export const clickElementByText = async ( elementExpression, text ) => {
    const [ element ] = await page.$x( `//${ elementExpression }[contains(text(),"${ text }")]` );
    await element.click();
};

export const selectOption = async ( label, value ) => {
    const [ selectEl ] = await page.$x( `//label[@class="components-base-control__label"][contains(text(),"${ label }")]/following-sibling::select[@class="components-select-control__input"]` );
    const selectId = await page.evaluate(
        ( el ) => el.id,
        selectEl
    );
    await page.select( `#${ selectId }`, value );
};

export const openSidebarPanelWithTitle = async ( title ) => {
    // Check if sidebar panel exists
    await page.waitForXPath( `//div[contains(@class,"edit-post-sidebar")]//button[@class="components-button components-panel__body-toggle"][contains(text(),"${ title }")]` );

    // Only open panel if it's not expanded already (aria-expanded check)
    const [ panel ] = await page.$x(
        `//div[contains(@class,"edit-post-sidebar")]//button[@class="components-button components-panel__body-toggle"][@aria-expanded="false"][contains(text(),"${ title }")]`
    );
    if ( panel ) {
        await panel.click();
    }
};
// wrapper-block.spec.js

import {
    clickBlockToolbarButton,
    clickButton,
    createNewPost,
    enablePageDialogAccept,
    getEditedPostContent,
    insertBlock,
} from '@wordpress/e2e-test-utils';
import {
    clickElementByText,
    openSidebarPanelWithTitle,
    selectBlockByName,
    selectOption,
} from './helper';

describe( 'Wrapper block', () => {
    beforeAll( async () => {
        enablePageDialogAccept();
    } );
    beforeEach( async () => {
        await createNewPost();
    } );

    it( 'Wrapper block should be available', async () => {
        await insertBlock( 'Wrapper Block' );

        // Check if block was inserted
        expect( await page.$( '[data-type="e2e-tests-example/wrapper-block"]' ) ).not.toBeNull();

        expect( await getEditedPostContent() ).toMatchSnapshot();
    } );

    it( 'Background color should be applied', async () => {
        await insertBlock( 'Wrapper Block' );
        await selectBlockByName( 'e2e-tests-example/wrapper-block' );

        // Change background color
        await openSidebarPanelWithTitle( 'Background Color' );
        await selectOption( 'Background Color', 'orange' );

        expect( await getEditedPostContent() ).toMatchSnapshot();
    } );

    it( 'Margin bottom should be applied', async () => {
        await insertBlock( 'Wrapper Block' );
        await selectBlockByName( 'e2e-tests-example/wrapper-block' );

        // Apply margin bottom
        await openSidebarPanelWithTitle( 'Margin bottom' );
        await clickElementByText( 'label', 'Add margin bottom' );

        expect( await getEditedPostContent() ).toMatchSnapshot();
    } );

    it( 'Alignment should be set', async () => {
        await insertBlock( 'Wrapper Block' );
        await selectBlockByName( 'e2e-tests-example/wrapper-block' );

        // Change alignment
        await clickBlockToolbarButton( 'Change wrapper block alignment' );
        await clickButton( 'Align Text Center' );

        expect( await getEditedPostContent() ).toMatchSnapshot();
    } );
} );

Our block is now tested with End-to-End tests. They help a lot when implementing new features or modifying existing ones.

We are testing two important things with this tests:

  1. If the block itself works and can be successfully inserted in the editor.
  2. All of our block attributes. They normally define the different block states and its output.

What's missing

Another thing which should be tested are all block filters if there are any. Since this is a bit more complex I will explain it in another tutorial.

Additional help

Selecting elements with Puppeteer

One important thing when writing tests is the selection of elements in the content. In Puppeteer this can be done with various functions. Mainly we're using the following ones:

page.$

The page.$ function returns a single element of the DOM tree which matches the given CSS selector. If no element matches the selector, the return value resolves to null.

page.$$

The page.$$ function does the same as the page.$ but it returns all matching elements in an array. If no element was found it returns an empty array.

page.$x

The page.$x function returns all elements which matches the given XPath expression. With XPath it is possible to build more complex selectors as with CSS selectors (e.g. selecting the preceding sibling on an element or an element which contains a given text). Here are some helpful links when writing XPath expressions:

Another really helpful thing is to use the Chrome developer tools to check if an XPath expression can be found in the current DOM tree. You can do this by accessing the same page which your test does and open the Chrome developer tools. Go to the "Elements" tab and press CMD + F (with macOS) or CTRL + F (with Windows). Now paste your XPath expression into the search field and see if it can be found in the DOM tree.

Further ressources

The whole block implementation and tests of this tutorial are also available as a WordPress plugin on GitHub: https://github.com/liip/e2e-tests-example-wp-plugin. Feel free to use it as a starting point to test your own Gutenberg block.