Behat hackday

A few days ago, we held an open hackday on doing Behaviour Driven Development (BDD)  with Behat at Liip Fribourg. It was the opportunity for the handful of participants to get a first grip on Behat and explore some aspects of BDD through it.

Introduction

BDD is an interesting practice for agile web developers, it ideally allows the formalization of the acceptance criteria of a story through scenarios written in Gherkin, a domain specific language that is human writable and machine readable. An archetypal Gherkin feature with one scenario (from Behat documentation):

Feature: My feature
      As an explicit system actor
      I want to gain some beneficial outcome which furthers the goal
      So that I realize a named business value
      @some_tag @some_other_tag
      Scenario: Some determinable business situation
          Given some precondition
              And some other precondition
          When some action by the actor
              And some other action
              And yet another action
          Then some testable outcome is achieved
              And something else we can check happens too

Gherkin Scenarios are composed of steps, and subordinated to features. Scenarios can be tagged, to later specify which subset to test, or to put constraints on their testing. The actual semantic of steps is implemented in Contexts (PHP Classes).

Explorations

Sylvain and Thomas started integrating Behat in one of their existing Symfony2 project. They appreciated the effectiveness and the ease of use of Behat+Mink to test form interaction, and the BehatBundle for Symfony2 which made integrating Behat into Symfony2 a breeze. They outlined that the first steps to implement are the ones specifying the role of the user: “As an editor …”. At the moment this is not used during the test but it could be interesting to use this feature to authenticate the user before running the test.

Donato mainly tested the Mink SahiDriver against liip.ch's project bundle and, outside Symfony, on a simple FuelPHP based application. In both cases the Behat+Mink+Sahi combo proved to be very handy and ran smoothly. The tests with Sahi were run successfully with both Firefox and Chrome.

Adi played with remoting sahi in order to start tests inside a virtual machine but have them executed in the browser on a remote system for testing with different browsers on different OS.

Lukas aimed at testing Ajax requests. He outlined the limits of testing asynchronous process through Behat.

Timo and Benoît focused on the product owner interests in the system. They went through some existing sprint review protocols (made of demo scenarios) and analyzed how much of them could actually be transformed into Behat features and scenarios. They found out that currently Behat, through Mink and the various browser drivers, allows to test only for content presence, and some browsing and navigation interactions. Still a lot more standard steps might need to be provided to test for the presence of document header elements, element attributes, element styling, element metrics, document validation, mouse hovering, scrolling, element visibility …

Conclusions, open questions

BBD and Behat are certainly the easiest way to introduce test-driven development on a project.

One of true value of BDD lies in the ecosystem of scenarios steps provided out-of-the-box: only with a broad spectrum of steps covering most aspects of browsing will Behat be effectively usable for a PO.

BDD empowers the product owner, but he still has to closely collaborate with developers when writing acceptance criteria: the syntax has to be respected, the PO may not know how to test for an element presence, etc.

It is yet undecided how BDD concepts match agile concepts. A user story will be covered by many scenarios, but should a Gherkin feature represent only one user story? or should we rather use tags and match features to epic stories or even code parts (Symfony2 bundles)?

Despite the shortcomings or difficulties in the Javascript and XHR area we're planning to get started with BDD in an existing project as of mid-December. It will be a slow start and we will grant ourselves some learning time. Depending on how practical the approach will prove in our context, we will not limit ourselves to new user stories, but also describe a number of existing features and get them tested this way.