Sunday, November 3, 2013

Customizing fitSharp with plug-in behavior

When I started working on FitNesse.NET several years ago, it already included a plug-in mechanism, known as 'cell handlers'.  The purpose of cell handlers was to customize the way the cell contents were handled.  For example, the standard behavior for checking a cell with an expected value is to parse the cell contents based on the actual value type, compare the actual and expected values, and mark the cell as right or wrong.  But expected value cells with keywords, like 'null', or symbol operators, like '>>mysymbol', require different behavior.

FitNesse.NET contained a list of classes that implemented a cell handler interface.  A cell handler could tell if it was able to handle a cell, usually based on the cell content.  If so, it could perform the appropriate behavior for basic cell operations like parsing and checking.  The list of cell handlers could be modified by a 'cell handler loader' fixture.  So individual tests or suites could have customized behavior by loading new cell handlers.

When I re-designed and re-branded FitNesse.NET as fitSharp, I kept the cell handler concept, expanding its use and making it into a more generic plug-in mechanism.  Rather than a single cell handler interface, fitSharp uses a separate interface for each operation that can be customized.  For example, there is a ParseOperator interface for parsing cell contents and a CheckOperator interface for checking an expected value cell. A class can implement one or more of these interfaces to provide customized behavior.

Every operator interface has two methods: one to indicate if the implementing class can provide behavior for a cell, and one to implement the behavior for the cell.  So the CheckOperator interface has methods CanCheck and Check.

fitSharp contains a list of operator classes, and whenever it needs to perform an operation, it scans the list looking for the first class that returns true from CanXxxx.  The list includes a priority number so, if needed, one operator class can always override another operator class.  Operator classes can be added to the list in a configuration file, in fixture code or in a test.

fitSharp currently contains these operator interfaces:
  • CheckOperator - checks an expected value cell
  • CompareOperator - compares a cell value to an actual value
  • ComposeOperator - constructs a cell with a given value
  • CreateOperator - creates an instance of a type
  • ExecuteOperator - executes a method described by member name and parameter cells
  • FindMemberOperator - finds a member of a type
  • InvokeOperator - invokes a method on the system under test
  • InvokeSpecialOperator - invokes a keyword method
  • ParseOperator - parses a cell value
  • RunTestOperator - runs a story test
  • WrapOperator - wraps a value in a fixture that can evaluate it
As you can see from the list, this plug-in mechanism is now used to customize a variety of operations, beyond basic cell behavior.

Every standard fitSharp fixture, and any other fixtures that need to perform any of these operations, must use the operator list to find the appropriate class to perform the operation.  Every fixture is passed a Processor interface that provides a method to do this, e.g:
processor.Operate(...);
 There is also a number of helper methods for common operations, like processor.Parse(), processor.Invoke() and others.

Sunday, July 24, 2011

Command line execution in storytests with fitSharp

A recent question on Stack Overflow asked if there is a command line execution fixture available for fitSharp. Something like:
|command line fixture|
|mypath\myprogram.exe|my arguments|
I answered that I didn't know of any but it would be easy to write one. (I put aside any thoughts of why this would be necessary. People use fitSharp for all kinds of things that I've never imagined.)

When I started to think more carefully, I realized that fitSharp provides the capability to do this without writing any fixtures at all. In classic Fit, the first cell in a table is the name of a fixture class to be invoked. In fitSharp, if this cell does not contain the name of a fixture class, a DoFixture is created which lets us execute methods on a class without writing a fixture. So we can write a command line class:
using System.Diagnostics;
namespace myStuff {
    public class CommandLine {
        public void Run(string program, string arguments) {
            var process = Process.Start(program, arguments);
            process.WaitForExit();
        }
    }
}
Then our test looks like:
|myStuff.CommandLine|
|run|mypath\myprogram.exe||my arguments|
This is a common pattern: we write a 'domain adapter' class to make it easier to invoke methods on our system under test from standard fixtures like DoFixture. These classes tend to be more flexible and reusable than if we write custom fixtures for each of our test scenarios.

But we can go further than this. We can execute methods on the System.Diagnostics classes directly. The 'with' keyword tells DoFixture which object to execute methods on. So our test can be:
|with|type|System.Diagnostics.Process|
|with|start|mypath\myprogram.exe||my arguments|
|wait for exit|
Of course, it would be nice to not duplicate this throughout our tests, so we can define a procedure:
|define|run|@program|with|@arguments|
|with|type|System.Diagnostics.Process|
|with|start|@program||@arguments|
|wait for exit|
Then we can use this whenever we need to run a program:
|run|mypath\myprogram.exe|with|my arguments|
This is a simple example of how we can start building a testing DSL that suits the kind of test scenarios we need to execute.

Saturday, July 9, 2011

Starting a FitNesse project with fitSharp

FitNesse and fitSharp offer so much flexibility that there's many different ways to configure a project, but for the beginner, a common question is: OK, but where do I start? So here's how I set up a sample project.

I got fitnesse.jar from http://fitnesse.org and put it in C:\apps\fitnesse. I got the latest fitSharp release from http://github.com/jediwhale/fitsharp and unzipped it into c:\apps\fitnesse\dotnet.

I created a Visual Studio solution in c:\projects\fitsharpsample.  I added a class library  project called Fixtures.  I added references to fit.dll and fitsharp.dll from c:\apps\fitnesse\dotnet.  I added a class called SampleDo:
using fitlibrary;

namespace Fixtures {
    public class SampleDo: DoFixture {
        public string Greeting { get { return "hi"; } }
    }
}
In my solution folder, I created a fitnesseRoot folder. I created a file fitnesse.cmd to run FitNesse:
java -jar \apps\fitnesse\fitnesse.jar -p 8080 -d "c:\projects\fitsharpsample" -r fitNesseRoot
pause
I created a configuration file storytest.config.xml:
<suiteConfig>
    <Settings>
        <Runner>fitnesse.fitserver.FitServer</Runner>
    </Settings>

    <ApplicationUnderTest>
        <addNamespace>Fixtures</addNamespace>
        <addAssembly>c:\projects\fitsharpsample\build\debug\Fixtures.dll</addAssembly>
    </ApplicationUnderTest>
</suiteConfig>
In my browser, I went to http://localhost:8080 and clicked the root link and edited the root page to contain:
!define TEST_RUNNER {c:\apps\fitnesse\dotnet\Runner.exe}
!define COMMAND_PATTERN {%m -c c:\projects\fitsharpsample\storytest.config.xml %p}
 I added a SampleTest page in FitNesse:
|sample do|
|check|greeting|hi|
I ran the test and all green!

Assertions: 1 right, 0 wrong, 0 ignored, 0 exceptions (0.000 seconds)
sample do
check greeting hi

Wednesday, June 1, 2011

Working remotely: a success story

I just finished a great project where the entire team was geographically distributed.  I'm going to jot down some retrospective thoughts about what made this project a success, while it's fresh in my mind.
  • Being all distributed probably worked better than having a centralized core team with remote members.  There were no second class citizens and we were all more aware of keeping everyone else connected to what we were doing.
  • The technology is there - just make sure you use it.  We did a daily group video stand-up and iteration planning and retrospectives with Skype and Mikogo, a screen-sharing tool.
  • Video is important. Occasionally, a team member would not have access to video and you could feel the collaboration start to slip almost immediately.
  • Distributed source control, like git or mercurial, is a must.
  • We had a small, highly competent team with diverse and complementary skills. The old rule about each new team member adding 'n' communication paths is amplified when those paths are remote.
  • There was a strong team culture of collaboration.  We had 3 developers, a product owner and an analyst/tester, but everyone pitched in wherever they were needed.
  • We used pair-programming from time to time to share knowledge and developers worked on tasks in different areas of the code base from iteration to iteration.  There was a team culture of learning that reinforced collective code ownership.
  • Automated unit tests and executable specifications are a must. (We used NUnit and FitNesse.)
Your mileage may vary, but I see the distributed team as an effective way to get the very best people assembled for your next project.

Tuesday, January 4, 2011

Customizing the new FitNesse parser

FitNesse began its life using the 'simplest thing that works' to parse wiki markup and render it as HTML: a set of regular expressions. Over the years, FitNesse functionality has grown, and the regex-based parser has had to support more and more complex tasks.  More and more hacks have been added to work around regex limitations and performance has suffered as bigger and more complex wiki pages have been thrown at it. So last spring, I started on a project to re-write the entire FitNesse parser, using classic grammar theory approaches.

Although the realities of earning a living slowed progress at times, and the challenges of replicating all the quirks of the original parser tested my resolve, we have finally merged the new parser into the main code base.  Thanks to a number of beta users, it has been tested on some major FitNesse test suites and the next FitNesse release, coming soon, will use the new parser.

One of the features of the original parser was the ability to extend the wiki syntax by plugging in your own custom wiki 'widgets'. This is described here: you write a class that extends WikiWidget and you add a line to a plugins.properties file.

WikiWidgets=className

James Carr wrote a nice post describing a detailed example of this.

The new parser also has this feature, but the plug-in class that you write is different. I'm going to show a very simple example here. Let's imagine we want to write !pi in our wiki pages and have it rendered with the value of pi in the HTML.


The plug-in class must extend SymbolType. A plug-in class can specify up to four pieces of information for the parser. Our simple example only needs to supply three of these.

The first is a name, specified in the super constructor. The name is just used for error reporting and debugging and so it can be any descriptive string.

The second is the wikiMatcher. This is an object that knows how to identify the symbol type in the source string. The Matcher class provides a lot of common matching behavior, so we can just tell it that our symbol type is recognized by the string "!pi". You can look at the Matcher source to find other matching behavior.

The third is the wikiRule, which our symbol type doesn't require. This is an object that implements a grammar production rule if our symbol type is composed of other symbol types (a non-terminal, in grammar-speak). Our symbol type is a terminal so we don't need a production rule. Look at the fitnesse.wikitext.parser package to see examples of how production rule classes are written.

The fourth is the htmlTranslation. This is an object that renders the symbol type as a string in the HTML output. We can implement the Translation interface and specify this as our translation object. The toTarget method renders our output, a string containing the value of pi.

We add a line to the plugins.properties file.

SymbolTypes=PiSymbolType


That's it!

Thursday, November 11, 2010

Fixture-free gets a quick answer

Someone on stack overflow asked what the current directory is when a fixture is executing. I didn't remember so I ran a quick test to find out:

|with|type|System.Environment|
|show|current directory|

No fixture, no IDE, no build. Just the answer I wanted.

Thursday, August 26, 2010

Fixture-free in the rain

I'll be presenting at Agile Vancouver in November on "Fixture-free Automated Acceptance Testing". This is a brief excerpt from a discussion that sparked the idea for the presentation:

Fixture-free is about exercising the System Under Test directly with a testing DSL like Fit or Slim (or whatever). It does require developer-tester collaboration because the System Under Test needs to expose classes and methods that are (a) testable and (b) named with ubiquitous language. (Otherwise it degenerates into trying to write code in a poor excuse for a programming language!). But once we have this, testers can do lots of stuff without additional developer help. For (a trivial) example, if we have a class Account with methods deposit, withdraw and balance, with fitSharp, with no fixture code, we can write:

|with|new|Account|

|deposit|100|

|withdraw|25|

|check|balance|75|

In reality, 100% fixture-free is unlikely, but we can drastically reduce the need for fixtures and for developers to write fixture code for every test.