After the Confoo.ca conference, Yannick Gringas from the Montreal Python user group organized two

Sprinting on TurboGears 2

small sprints at the Brasseurs Numériques headquarters (Digital Brewmasters). Yannick’s company is called like that because he brews his own beers ! As a matter of fact, he prepared a beer for the sprints, that was really good (and quite strong.). His beer would probably beat some good Belgium beers on a blind test.

Sprint #1 : Turbogears

Chris Perkins led a Turbogears sprint Saturday. I am not very familiar with this framework but I am using Pylons a lot, and its the basis of Turbogears. Alice Bevan-McGregor was present, and could confront his ideas on web frameworkery with Chris since he created his own tool : WebCore. I’ve heard that they worked on making the Turbogears dispatcher a standalone library so it could be used by both frameworks.

I worked on my side on small packaging issues TG has. We fixed the latest TG 2.x beta custom package index that was broken (it was generating incomprehensible errors when installing TG, and I found out that the index pages in the TG PyPI were broken)

Next,  I’ve added to easy_install a –no-find-links option to prevent links added by projects in their setup.cfg. This will prevent projects like Pylons to implicitely add links that easy_install reads. The effect is that some old version of some packages like nose were installed. This is not released yet.

If you were present to the sprint, please comment to tell us what you’ve done !

Sprint #2 : Packaging

Monday evening we worked on packaging issues. I started the sprint by presenting the current state of packaging on a board. That took quite a while because it is not obvious to understand the packaging eco-system (distutils, setuptools, distutils2 and pip.).

Then I’ve listed possible tasks and people started to work.

• Yannick Gringas worked a bit on the Hitchicker’s Guide to Packaging then worked on Distribute on Issue #133.
• Ahmed Al-Saadi was pretty new to packaging so he worked on the guide and tried to catch up with the state of packaging (that’s a real work )
• Alexandre Vassaloti worked on porting distutils2 into Python3. So basically, like Distribute, Distutils2 will be installable on Python 2 and 3, using the same source tree, and a 2to3 call upon installation.
• Nicolas Cadou worked on PEP 345 support. He created a sample project that will be used in a functional test to validate that everything works. He eventually fixed some code in Distutils2 so it works with the PEP 345 DistributionMetadata class I’ve built during Pycon. I need to merge his work asap.
• Matthieu Leduc-Hammel worked with me on PEP 345 support for PyPI. I’ve changed the postgres database to add the new fields, and Matthieu worked on PyPI UI. For instance, you will have a nice box on the project pages now that displays links from the Project-URL metadata field. I need to merge his work asap.

As a global note: Mercurial was the perfect tool for this sprint. I am able to merge people work in Distutils2 and other projects without all the repository access issues we usually get when we start a sprint. I am looking forward for a full Mercurial switch of Python, because this will boost contributions.

Thanks Yannick, Nicolas for the Sprint and the Beer ! Thanks Ubity for sponsoring the Packaging sprint with pizzas ! (they are looking for developers btw)

One of the posts I’ve been meaning to do is this one – a more extended version outlining what I spoke about in my five minute lightning talk I did at PyCon 2010 (more general thoughts on the conference later). Of course, meaning to post it – and actually getting a chance to are two different things.

So – my goal of the lightning talk was to say thank you to the python community, not only from me, but from the point of view of the company I work for (Nasuni) which recently launched. Thanks can be given back to the community in a variety of ways – money, code, sponsorship, programs, etc.

Sometimes, a simple Thank You can be a good start, and that was my goal. Unfortunately, I also completely deleted my slides (intentionally, what was I thinking?) after my talk was done.

What is Nasuni? We’re pretty small – obviously much smaller when starting (5 core engineers). The goal was to build a simple to manage, secure, reliable cloud-storage backed virtual storage appliance. Everyone helped code and design the product. We had to build it fast – when you’re a startup on VC money, you’ve got to be fast – you have finite time. Limited time, money and people.

From a technology standpoint – it simply made sense to put the core data path in C and C++ – you need speed, security, and the compactness C offers you when you’re talking speed. Not to mention, this is a device which has a finite amount of space, so keeping it relatively compact is a Good Thing. So the brain and central nervous system are in C – but we’re lacking something:

The rest of the person. D’oh. You know, arms and legs and stuff.

Let’s look at all of the other stuff we needed:

• tests
• tools/glue scripts
• prototypes
• user interface(s)
• other system daemons
• infrastructure
• web site
• deployment tools

And that’s just the stuff I could think of off the top of my head right now. So, we needed to compliment our super strong core with something pretty flexible, something that could also easily mesh with the core system as well.

When picking the technology; we needed speed of development, a broad ecosystem to build from, and it needed to ramp up and down. It had to be flexible enough that anyone involved in the project could become immediately effective with it.

Enter Python.

“Duct tape is like the force. It has a light side, a dark side, and it holds the universe together.”

Python Ramps “up”, and it ramps “down” – it’s good for “hard” programmers, and people just trying to get things done. It’s simple enough that anyone who has scripted in the past can get something done, and quickly. My motto has slowly become “there is no such thing as bad code – there’s just code that works, and code that doesn’t”. Code that comes out of someone who is just learning, and gets the job done is just as valuable as code that is well thought out and designed.

Python, and it’s broad ecosystem. It’s the glue, the face, the hands, the legs. It, plus insanely smart engineers let us move quickly, adapt to anything that popped up and put together a kick ass product.

8 months.

With Python’s help, we put together a ready-to-ship simple to manage, secure, reliable cloud-storage backed virtual storage appliance. 8 months! This isn’t a web app, or a another web service. This is something just shy of putting actual hardware into a datacenter! We’ve got python deep in our DNA. (And heck – the president of the company wrote a good chunk of it! How awesome is that?)

The moral of the story, however brief it is – when you need to sell Python to your boss, or that client – or if you’re just starting out. Don’t sell it with words. Sell it by showing it’s effectiveness, it’s speed. Just do it – that’s what Python is here for. helping you get your job done and succeed.

Python is not meant to be the prettiest, or the fastest-to-run, or the one with the latest feature du-jour. It’s there to let you get your job done. That’s why I love it, and much of the community around it – it’s the pragmatism, how will this let people be effective.

I wanted to take a moment not just to cheerlead – but to also specifically thank the projects and tools out there in the community that helped us get the job done.

• Python (duh)
• The Python standard library: This can not be stressed enough! Despite anything else we use from the ecosystem, the standard library is rich enough that it’s a massive time, and life saver. This is why I don’t believe that “getting rid of it” makes any sense at all. Python’s success is not just a by product of the language itself – the standard library is such a big selling point it can’t be stated enough.
• Django
• Virtualenv / Virtualenvwrapper
• Nose
• pip
• fabric
• paramiko
• lxml
• pycurl

And many, many others (including obviously non-python projects). Thank you, thank you, thank you!

In closing – Thank You!. Without Python, as a language – Python as a community, Django, and so many others – we would not have been able to achieve our goals. There’s so much to be proud of as a community, and sometimes we loose sight of it while debating frameworks, this, that or the next thing. Python is the perfect secret weapon for people and companies just trying to get things done

Thank you all – hopefully we can give back as much as we’ve gotten.

One classic computation problem, the travelling salesman. Even if it’s quite simple to describe, it’s one of those hideous NP complete problems, the ones that are quite difficult to resolve, as the computation time needed grows greatly with the length of the data. On Programming Praxis they have proposed to resolve the problem using brute force, and using the closest neighbor (a simplification of the problem).

I’ve created this program, in Python.

import random
import itertools
import operator
import datetime

MAX_X = 100
MAX_Y = 100

def random_cities(number):
    ''' Generate a number of cities located on random places '''

    cities = [ (random.randrange(0, MAX_X),
                random.randrange(0, MAX_Y))
                for i in range(number) ]

    return cities

def path_lenght(path):
    ''' Get the lenght of a path '''
    lenght = 0
    for i in xrange(len(path) - 1):
        # Add the distance between two cities
        lenght += abs(complex(path[i][0], path[i][1])
                       - complex(path[i + 1][0], path[i + 1][1]))

    return lenght

def find_path_bruteforce(cities):
    ''' Find the smallest path using brute force '''

    lenghts = []

    for path in itertools.permutations(cities, len(cities)):
        # Get the length of the path, adding the returning point
        total_path = path + (path[0],)
        lenght = path_lenght(total_path)
        lenghts.append((total_path, lenght))

    # Get minimum
    lenghts.sort(key=operator.itemgetter(1))
    return lenghts[0]

def find_path_nearest(cities):
    ''' Find the closest neibour '''

    lenghts = []
    for city in cities:
        lenght = 0
        actual_cities = cities[:]
        actual_city = actual_cities.pop(actual_cities.index(city))
        path = [actual_city, ]
        # Find nearest neibour
        while actual_cities:
            min_lenght = []
            for next_city in actual_cities:
                min_lenght.append((next_city, abs(complex(city[0], city[1])
                                                 - complex(next_city[0], next_city[1]))))
            # Get closest neibor
            min_lenght.sort(key=operator.itemgetter(1))

            actual_city = min_lenght[0][0]
            lenght += min_lenght[0][1]
            actual_cities.pop(actual_cities.index(actual_city))
            path.append(actual_city)

        # Complete the trip with the first city
        path.append(city)

        lenghts.append((tuple(path), path_lenght(path)))

    # Get minimum
    lenghts.sort(key=operator.itemgetter(1))
    return lenghts[0]

if __name__ == '__main__':
    for i in range(3, 10):
        print 'Number of cities: ', i
        cities = random_cities(i)

        time1 = datetime.datetime.now()
        path2, lenght_neighbor = find_path_nearest(cities)
        time2 = datetime.datetime.now()
        print path2, lenght_neighbor
        time_neighbor = time2 - time1
        print 'Time neighbor: ', time_neighbor

        time1 = datetime.datetime.now()
        path1, lenght_brute = find_path_bruteforce(cities)
        time2 = datetime.datetime.now()
        print path1, lenght_brute
        time_brute = time2 - time1
        print 'Time brute force: ', time_brute
        print 'Diff: ', float(lenght_neighbor - lenght_brute) / lenght_brute * 100, '%'

The time spend on each is brutally different, specially as the number of cities grow. With more than 9 cities, the time grows at some point when my computer has to take more than minutes to compute the brute force approach, while the neighbor approach will still be below the 1/100 of a second. The error  also grows, being the same on small number of cities, but getting afar fast when the number of cities grow. At 9 cities, the error is about a 20-30%

import smtplib

server = smtplib.SMTP('smtp.gmail.com', 587)
server.ehlo()
server.starttls()
server.ehlo()
server.login('fromuser@gmail.com', 'mypassword')
server.sendmail('fromuser@gmail.com', 'touser@example.com', 'HELLO')
server.rset()
server.quit()

At the time of our last blog post, there were 120 tickets open. Over the last week, we have purged a bunch of tickets that weren't critical to the release of 1.2, and we have made 50 Subversion commits. There have also been a couple of new tickets added.

As a result of this activity, 84 tickets remain. Of those tickets, 21 are documentation and translation updates. This leaves 63 substantive tickets that need to be addressed before we have a release candidate.

There are three areas in particular that have large ticket counts. Not surprisingly, these areas correspond to the three areas of biggest change in 1.2:

1. Regressions in query behavior caused by the multi-db refactoring,
2. Changes in admin behavior caused by the admin javascript improvements, and
3. Edge cases in localization handling.

Many of these issues are small oversights or minor corrections. However, there are a couple of tickets (for example #13023) that aren't trivial, and will require some significant design work.

As a result, we're going to push back the expected release date by another 2 weeks. This would put an RC1 release around April 5, with a final release around April 12.

As always -- any and all assistance is most welcome; the more assistance we get, the faster 1.2 will land.

• Acceptance Testing .NET Applications using IronPython

The following was originally published in the excellent Python Magazine. Thier contractual exclusivity period has now long expired, so here it comes again. Many thanks to my technical reviewer Michael Foord, and to the editors Brandon Craig Rhodes and Doug Hellmann, who patiently gave excellent and much needed guidance through its protracted gestation, and especially to all my co-workers at Resolver Systems, from whom I’ve learned so much.

Acceptance Testing .NET Applications using IronPython

Unit tests demonstrate to developers that individual functions and classes work as expected. Acceptance tests are an orthogonal complement to this. They verify to everybody, including managers and clients, that features they understand and care about are completed and working correctly. They also prove that the system as a whole is correctly integrated and that no regressions have occurred.

Resolver Systems is developing a .NET desktop spreadsheet application, Resolver One, for which we have accumulated an acceptance testing framework. This framework uses Python’s standard unittest module, and is executed using IronPython.

While Resolver One is written in IronPython, this technique allows IronPython tests to interact with product code written in any .NET language.

This article describes the principles of this IronPython acceptance testing framework, and demonstrates them by creating an acceptance test for a small sample C# GUI application.

Caveats

When testing products written in static .NET languages such as C#, some common testing practices like monkey-patching will be unavailable. Static language classes are not modifiable at runtime, not even from tests written in IronPython. Fortunately, this is less of a concern for acceptance testing than it is for unit testing – we want the tests to operate on the unmodified whole end-to-end system.

Resolver One is currently 40k lines of IronPython. I would guess this is maybe equivalent to 60-80k lines of C#, demonstrating the viability of this approach for desktop applications of this size.

Our approach requires source code modifications to the system under test (SUT.) In particular, the SUT must provide methods for the test to start and stop the application, and must provide public access to its forms and other GUI objects. This means that this methodology cannot be used to black-box test arbitrary compiled programs – it requires the SUT to be written with testing in mind.

Why Acceptance Test?

Unit tests call individual methods and functions of the SUT, and have a close correspondence with the internal design of the product. Acceptance tests, in contrast, invoke the program as a whole, just like a user would, and have a close correspondence with the product specification.

Figure 1. Unit tests correspond to the SUT's internal design, but acceptance tests correspond to the specifications.

Acceptance testing automates the expensive, time consuming, error-prone and soul-destroying process of using a team of human testers to fire up the application under test, and exhaustively interact with the user interface to verify the program behaves correctly. Traditionally, a single iteration of this process can take days or weeks for substantial applications. Automating the process can yield the same feedback – or better – in minutes or hours. This reduces costs and provides valuable, rapid feedback to both developers and project stakeholders.

This is useful for assessing whether user-visible features are correctly implemented, for doing quick smoke tests to make sure recent changes haven’t accidentally broken other features, or for systematic checking that new functionality works under various conditions, such as on different operating systems, or in various browsers. Acceptance tests can include stress testing, and continually running acceptance tests on an integration server can detect infrequent, intermittent bugs.

Best of all, acceptance tests that are derived directly from the specification can prove to clients that the system does what the requirements ask. This can be invaluable when it comes to client sign-off on deliverables, especially if the client trusts this process due to having participated in the creation of the user-stories or acceptance tests themselves.

Acceptance tests do not yield the same incidental benefits in terms of good code design as unit tests do. However, creating acceptance tests before the product code is implemented does allow developers to focus exclusively on the requirements from a user’s point of view. In practice, this turns out to help immeasurably in defining the specifications, and in giving developers a solid understanding of them.

Acceptance Tests Should Derive From User Stories

User Stories are a human-readable specification document that describes a short scenario, using the SUT to perform some actions that a real user cares about. User stories usually form the entire specification. Such documents should be informal yet precise, succinct and easy to understand. In ideal circumstances, your customer would collaborate with you in creating these documents.

An example user story might look like this:

1. Alice starts WizBang. The window appears.
2. She sees the three default list entries: ‘one’, ‘two’, ‘three’. Nothing is selected.
3. She clicks the ‘AddButton’
4. The ‘Add Item’ dialog appears
5. She types an item name into the dialog and clicks OK
6. The new item is at the end of the list, selected.
7. She clicks the ‘CloseButton’
8. The application closes

An acceptance test is an executable script that performs the actions described in the user story, and verifies the application responds as expected. To create an acceptance test, at Resolver Systems we paste the entire user story, as comments, into a new test method, on a class derived from Python’s unittest.TestCase.

from unittest import main as run_test, TestCase

class AT001_AddItems(TestCase):

    def test_additems(self):
        # 1. Alice starts WizBang. The window appears.
        # 2. She sees the three default list entries:
        #    'one', 'two', 'three'. Nothing is selected.
        # 3. She clicks the 'AddButton'
        # 4. The 'Add Item' dialog appears
        # 5. She types an item name into the dialog and clicks OK
        # 6. The new item is at the end of the list, selected.
        # 7. She clicks the 'CloseButton'
        # 8. The application closes
        self.fail("test not finished")

if __name__ == '__main__':
    run_test()

The call to fail() at the end of this test is a good habit to develop. It stops unfinished tests like this one from passing, making them easy to accidentally overlook and be forgotten, invisible amongst a large collection of passing tests. When the test is complete, this fail can be removed.

Setting up IronPython

In order for IronPython to be able to import from unittest like this, a copy of the CPython standard library must be on sys.path. If you installed version 2+ of IronPython from the MSI installer, this is all taken care of automatically, using a copy of the standard library that is included with the install. Otherwise, you need to set this up manually, either by setting an environment variable:

set IRONPYTHONPATH=C:\Python25\Lib

or by appending this directory to sys.path inside your IronPython install’s Lib\site.py file:

import sys
sys.path.append(r'C:\Python25\Lib')

Note that different versions of IronPython require different versions of the CPython standard library:

• IronPython 1.1: Python 2.4
• IronPython 2.0: Python 2.5
• IronPython 2.6: Python 2.6

Once this is done, test it out by starting ipy.exe, and typing:

>>> import unittest

If this works without complaint, exit ipy.exe, and run the nascent acceptance test above, using the DOS command-line:

>ipy.exe AT001_AddItems.py
F
======================================================================
FAIL: test_add_address (__main__.AT001_AddItems)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "AT001_AddItems.py", line 11, in test_add_address
    self.fail("test not finished")
AssertionError: test not finished

----------------------------------------------------------------------
Ran 1 test in 0.141s

FAILED (failures=1)

About IronPython

IronPython is a .NET based reimplementation of the Python language. It combines the strength and elegance of Python as a language with the ability to directly call and interoperate with other .NET code. In daily use, I am continually surprised by how well this works. IronPython faithfully mimics CPython – there are almost no surprising differences between the two. Native Python types are mapped seamlessly and intuitively to equivalent .NET data types with an absolute minimum of fuss. For example, IronPython code can generally pass native Python types, like lists or dictionaries, to .NET functions or methods, instead of having to instantiate and populate .NET collection classes.

To use a .NET library, your IronPython code first has to add a reference to the containing .NET assembly. An assembly is a physical chunk of code, usually contained in a DLL file. To add a reference, use the clr module, which is built-in to IronPython:

import clr
clr.AddReference('System.Windows.Forms')

This AddReference() function behaves just the same regardless of whether you are referencing assemblies from the .NET standard library (as shown here), 3rd party DLLs, or your own .NET projects.

Code within an assembly is contained within namespaces. For the .NET standard library, the assemblies are usually given the same name as the namespace they implement. This is the case here, so once the above assembly is referenced, we can import code from the System.Windows.Forms namespace just as if it was a Python module:

from System.Windows.Forms import Form
form = Form()
form.Show()

This will display an instance of the .NET form class on screen. Note that the form does not yet respond to events. For that, we will add a call to Application.Run(), discussed below.

Note that our IronPython projects always contain an automatic reference to the System assembly, so anything implemented in there, such as the System.Threading namespace, can always be imported without having to explicitly add any references.

Ostensibly, using .NET and IronPython limits the operations described in this whole article to Windows only. The Mono project should allow this acceptance testing technique to be used directly on other operating systems, but that has not been tried.

Implementing the Test

To implement the comments pasted into our acceptance test, three things must be done:

Firstly, the test must invoke the SUT, once for every test method, in such a way that the test and the SUT then both run simultaneously.

Secondly, while the test runs, it must be able to make assertions about the behaviour of the SUT. On the GUI, for example, the test must be able to read the state of the form and its controls, in order to assert that the correct text is displayed.

Thirdly, the test must provide simulated input on each of the SUT’s external interfaces, to stimulate the program into action. For example, then the test must drive the SUT by providing simulated button clicks or keyboard input.

The method chosen to fulfil all three of these requirements is for the test to invoke the SUT in the same process, but on a new thread. Since IronPython is a .NET language, it can directly access the SUT’s form and control objects, making assertions about the state of the controls that are visible to the user. It can also simulate user actions by calling methods and firing events on the SUT’s GUI controls.

For this acceptance testing technique to work, the start-up of the SUT must be structured so as to expose a few public members and objects that are monitored and manipulated by the test. For example, elements of the GUI must be public. This obviously makes the tests fairly invasive.

We’ve justified this to ourselves at Resolver Systems by taking a pragmatic philosophy: These are the smallest set of changes to our application that we could find in order to make it testable. This technique has allowed us to create a set of working acceptance tests that wouldn’t otherwise have existed.

The System Under Test

The public methods and objects required from our SUT are shown in the following minimal C# GUI application, called WizBang. This was created using the free Visual Studio Express Edition, but could easily be created using your own development tools of choice. At Resolver Systems, we prefer to create form layouts using Visual Studio’s excellent GUI designing tool, and then inherit from these generated classes in other editors, such as Wing, Emacs or Vi.

WizBang defines a couple of simple forms, MainForm and AddItemForm:

Figure 2. WizBang's main form

Figure 3. WizBang's add item form

WizBang has a public class called Program, which provides public access to the application’s forms, and handles the startup and shutdown of the application. During startup, it creates and shows an instance of the main form.

// C# scaffolding for the WizBang application.
// Exposes public functions and GUI forms, for use by tests.
using System;
using System.Threading;
using System.Windows.Forms;

namespace WizBang
{
    public class AllForms
    {
        public static MainForm mainForm;
        public static AddItemForm addItemForm;
    }

    public class Program
    {
        public ManualResetEvent eventloopReady =
            new ManualResetEvent(false);

        public void Start()
        {
            AllForms.mainForm = new MainForm();
            AllForms.mainForm.Show();
            eventloopReady.Set();
            Application.Run();
        }

        public static void Stop()
        {
            Application.Exit();
        }
    }
}

Program.cs and its respective forms can be compiled to a .NET assembly, WizBang.dll. When the application is run normally by users, Program.Start() is called by a second Visual Studio project, RunWizBang, which exists simply to create a minimal Windows executable.

To run WizBang during testing, the test code references the WizBang.dll assembly, and calls Program.Start() itself.

Importantly, a new instance of the main form is created every time Program.Start() is called. This allows many successive tests to run, each with their own instance of the main form, so that state changes in one test do not affect subsequent tests. This is important – the application’s state should be completely reset before the start of each test. Take particular care to do this right if your application has global state, such as class-level variables, singletons, or relies on external systems such as the file system, the registry or databases.

After the main form is created and shown, Application.Run() is called. This is a .NET method which starts the main form’s event loop, making the form responsive to events such as form moves and resizes, control clicks and keyboard presses.

When the test is completed, it can call the public Program.Stop() method, which safely disposes of resources and unconditionally closes the application by calling Application.Exit(). This is a .NET method which closes all our forms and ends their event loops.

Program.Stop() should be the same method that your application calls when quitting, after any user confirmations have happened. The following handler on the main form’s Closed event ensures this:

// C# event hander for the main form of the WizBang application
private void MainForm_Closed(object sender, EventArgs e)
{
    Program.Stop();
}

Invoking the SUT on a New Thread

If the test calls Program.Start() directly, as described above, then the program will run, and the main form will be displayed and responsive, but the test will not work. The reason is that the call to Application.Run() is synchronous – it does not return until the program exits. The test will be blocked, waiting for the application’s event loop to end.

The test must invoke the SUT in such a way that the program and the test can run together in parallel. Invoking the SUT in a new process would be nice for the sake of decoupling one test from the next. For our approach, however, the SUT has to run in the same process, to give the test access to the SUT’s public methods and objects.

Program.Start() therefore needs to be called on a new thread, created by the test. The program’s event loop can proceed on the new thread, handling events to let the application run normally. Meanwhile, our test can proceed on the original thread, stimulating the GUI and watching the application’s behaviour.

This threading is the reason for the ManualResetEvent instance in Program.cs. Manual reset events are a .NET construct to facilitate synchronisation between threads. In this case, the SUT calls Set() in Program.Start(), to tell the test thread that the main form has been created and shown. The test can then start to make assertions about the state of the main form’s controls, and trigger events to manipulate those controls.

For many activities in IronPython, there is a choice of using the familiar Python libraries, or the .NET equivalents. Threads are no exception to this – we may use the Python thread or threading modules, or we may use .NET’s Threading library.

Incidentally, these threads differ from those in CPython in one important respect – there is no Global Interpreter Lock (GIL). The GIL is an implementation detail of the CPython interpreter. The threads created by .NET will run concurrently on multiple cores, no matter which library we use.

We implement this in a new class AcceptanceTest, which sits between AT001_AddItems and unittest.TestCase in the inheritance hierarchy:

Figure 4. Our test class inheritance hierarchy.

On a real project, many ATxxx test classes would inherit from AcceptanceTest, which looks like this:

# reference .NET assemblies - requires IronPython
import clr
clr.AddReference('WizBang')

# import from .NET namespaces - requires IronPython
from System.Threading import ApartmentState, Thread, ThreadStart

# import from the Python standard library
from unittest import TestCase

# import from the .NET namespace of the system under test
from WizBang import Program

class AcceptanceTest(TestCase):
    """Starts up the program under test (PUT) on a new thread at the start
    of each test, and shut it down again after each test has run"""

    def __init__(self, *args):
        TestCase.__init__(self, *args)
        self.program = None
        self.eventloop = None

    def setUp(self):
        TestCase.setUp(self)
        self.program = Program()
        self.eventloop = Thread(ThreadStart(self.program.Start))
        self.eventloop.Name = "eventloop"
        self.eventloop.SetApartmentState(ApartmentState.STA)
        self.eventloop.Start()
        self.program.eventloopReady.WaitOne()

    def tearDown(self):
        self.program.Stop()
        self.eventloop.Join()
        TestCase.tearDown(self)

Python’s unittest module will call AcceptanceTest’s setUp() method before each test method is run, so this is an ideal place to create a new thread and invoke Program.Start() on it. The new thread is given a name, for ease of hypothetical future debugging. It must be set to a single threaded apartment state to prevent possible errors if your code calls any COM components, even indirectly.

Once the new thread has been started, starting the WizBang application, setUp() waits for a signal from the application on the ManualResetEvent eventloopReady. This is used by the application to signal to the test that the main form is shown and its event loop is started. This prevents over-eager tests from attempting to access the main form before it is visible and responding to events.

Similarly, tearDown() will be called by unittest after every test has completed. In this method, Program.Stop() is called to exit this instance of WizBang. tearDown() then waits for the SUT’s event loop thread to end, by joining it. This is to ensure the next test is not affected in some way, by allowing it to start before this test has ended.

This can all be tried out, by modifying AT001_AddItems to inherit from AcceptanceTest instead of TestCase, and adding a sleep in the body of the test method, before the fail:

from System.Threading import Thread
from unittest import main as run_test
from AcceptanceTest import AcceptanceTest

class AT001_AddItems(AcceptanceTest):

    def test_add_address(self):
        # 1. Alice starts WizBang. The window appears.
        # etc (not yet implimented)

        Thread.Sleep(5000)
        self.fail("test not finished")

if __name__ == '__main__':
    run_test()

Running this test now displays the SUT’s form for the duration of the sleep, and during that time it is responsive to move and resize events. Unlike the last version, this test proceeds to execute while the form is visible, so at the end of the sleep, the test fails, with ‘test not finished’, and the SUT is ended, closing its form.

Note that the sleep triggers a runtime warning from ipy.exe on stdout, about sleeping threads not pumping GUI events. This makes no difference here, but from now on, we’ll do as this warning suggests and replace sleeps with calls to Thread.CurrentThread.Join(), which behaves the same as sleep, but continues to process any events that arrive while sleeping.

Now that our acceptance test is properly starting and stopping the SUT, we are in a position to start making assertions about the state and behaviour of the main form.

Asserting Correct Behaviour

We can now start coding the requirements that have been pasted into our acceptance test as comments. We might be tempted to implement the first requirement of the acceptance test as follows:

import clr
clr.AddReference("WizBang")
from WizBang import AllForms

# 1. Alice starts WizBang. The window appears.
mainform = AllForms.mainform
self.assertEquals(mainform.Visible, "form should be visible")

On first glance, this appears to work – you can run this and this assert will pass. However, there is an insidious problem here, because it accesses properties of a .NET control from a thread other than the one it was created on. Such access can sometimes result in an InvalidOperationException, with a message along the lines of “Cross-thread operation not valid: Control ‘mainform’ accessed from a thread other than the thread it was created on.” Worse, depending on circumstances, sometimes no exception is raised, but values are returned which may not be correctly synced to the current value of mainform.Visible.

The proper way to access properties like this is to invoke such code on the control’s own thread. All .NET controls, of which forms are a subclass, have an Invoke() method for just this purpose.

Conceptually, Invoke() takes a callable, which is executed synchronously by passing it as an event to the control’s event loop (or the event loop of its parent form). When the event loop processes this event, the passed callable is invoked on the event loop’s thread – which can safely access the properties of its own controls. The return value from the callable is passed back by the event-handler, and then safely marshalled back to the invoking thread as the return value from Invoke().

In practice, the callable passed to Invoke needs to be wrapped in a delegate. Delegates are .NET’s type-safe function pointers. An appropriate delegate can be constructed using the IronPython construct CallTarget0, which denotes a delegate taking zero arguments.

The above sounds like quite a mouthful, and the code is correspondingly verbose:

import clr
clr.AddReference('IronPython')
from IronPython.Runtime.Calls import CallTarget0

# 1. Alice starts WizBang. The window appears.
getVisible = CallTarget0(lambda: mainform.Visible)
self.assertTrue(mainform.Invoke(getVisible), "form not visible")

Note that the location of CallTarget0 was changed between IronPython versions 1 and 2. The location of the import will need to change for the above code to work on IronPython 1.

Invoking on the control’s own thread like this means that our callable (the lambda: mainform.Visible) can safely access any of mainform’s properties and methods.

More Concise Test Code

The cross-thread invoking described above will be used frequently throughout our acceptance tests, whenever the properties or methods of a control are accessed. Such code can be abbreviated slightly, by defining a method on AcceptanceTest, to help us invoke on the main form’s thread:

def on_gui(self, target):
    return self.program.mainform.Invoke(CallTarget0(target))

Which can be used to reduce the length of our assertion to:

# 1. Alice starts WizBang. The window appears.
self.assertTrue(self.on_gui(lambda: mainform.Visible), "form not visible")

Even so, wrapping access to each attribute individually like is still a little fiddly, especially if it is happening many times. To improve this, there is nothing to stop us wrapping larger callables instead. For example, consider the second user story requirement:

# 2. She sees the three default list entries:
#    'one', 'two', 'three'. Nothing is selected.
self.on_gui(self.assert_list_at_startup)

The function assert_list_at_startup(), shown below, can now have access to properties on all controls without using Invoke(), since it runs entirely on the GUI thread:

def assert_list_at_startup(self):
    wizList = self.program.mainform.Controls["WizList"]
    self.assertEquals(wizList.SelectedIndex, -1,
        "should be nothing selected")
    self.assertEquals(wizList.Items[0], 'one', "list[0] wrong")
    self.assertEquals(wizList.Items[1], 'two', "list[1] wrong")
    self.assertEquals(wizList.Items[2], 'three', "list[2] wrong")

If any of these assertions should fail, raised exceptions are correctly propagated back to our test thread. The only problem is that the stack trace displayed in the test output ends at on_gui()’s cross-thread invoke – i.e. it does not display the line within assert_list_at_startup() which failed. However such stack traces are accompanied by the error message from the failing assertion, so this is not usually a problem. If this turns out to be critical for your situation, it is possible to create your own cross-thread exception handler which fixes this, reconstituting the entire stack trace even across thread boundaries.

Wrapping callables, as on_gui() does, is often usefully implemented as a decorator. This can be provided by our AcceptanceTest module:

def guithread(target):
    def wrapper(*args, **kwargs):
        test = args[0]
        return test.on_gui(lambda: target(*args, **kwargs))
    return wrapper

This decorator expects to be applied to methods on AcceptanceTest, so that it can call .on_gui() on this method’s first parameter. if the decorated function is not a method on AcceptanceTest, the decorator will not work.

Functions like assert_list_at_startup(), above, which make frequent access to properties of controls, can now be decorated:

@guithread
def assert_list_at_startup(self):
    wizList = self.program.mainform.Controls["WizList"]
    self.assertEquals(wizList.SelectedIndex, -1,
    # etc

Such a decorated method can then be conveniently called by the acceptance test:

# 2. She sees the three default list entries:
#    'one', 'two', 'three'. Nothing is selected.
self.assert_list_at_startup()

It is tempting at this point to simply decorate our entire test method with @guithread, so that the whole thing can execute on the GUI thread and have unfettered access to the form’s attributes and controls. However, this would not work since the test needs to surrender its use of the GUI thread from time to time, to allow the form to process all the events on its event loop. Without this, the form would be blocked, waiting for the test to finish, and would be unable to handle button clicks and other input. Amongst other things, this would prevent the form from reacting to the simulated user input that our test is about to provide.

Simulating User Button Clicks

The next part of our acceptance test requires that the test provides some input to the SUT, simulating the actions of a user:

# 3. She clicks the 'AddButton'

Buttons provide a method specifically to simulate being clicked, which our test can use. A small utility method on AcceptanceTest calls this on the GUI thread:

@guithread
def click_button(self, form, buttonName):
    form.Controls[buttonName].PerformClick()

This can be used in our acceptance test:

# 3. She clicks the 'AddButton'
self.click_button(mainform, 'addButton')

Running the test will now correctly click the add item button. However, the Wizbang application does not yet have any button click handlers, so the button has no effect.

Up until this point, all the assertions in AT001_AddItems have passed without us having to modify WizBang the application. This is because the conditions being tested were already set up correctly by the design-time properties of the main form, or by the application’s scaffolding code in WizBang.Program.

When using test-driven development (TDD), this is generally not the case, and in fact, this is not true for our next requirement, which tests that the add item button click caused the add item form to appear. Since this is not yet implemented, when the test is run, this assertion will fail:

# 4. The 'Add Item' dialog appears
addItemForm = AllForms.addItemForm
self.assertTrue(self.on_gui(lambda: addItemForm.Visible),
    "additem form should be visible")

To make this requirement pass, the following click handler is attached to the Add Item button on WizBang’s main form. We’re going to skip unit tests for this article, but on a real project, this is the perfect time to create them – after the acceptance test, but before the implimentation. Once they are done, the handler to make them and the acceptance test both pass looks like this:

// C# click handler for the AddItem button on WizBang's main form
private void AddButton_Click(object sender, EventArgs e)
{
    if (AllForms.addItemForm == null)
        AllForms.addItemForm = new AddItemForm();
    AllForms.addItemForm.Show();
}

In a real application, this handler would need to be a little more robust, able to handle multiple clicks of the button, and forms which are disposed or already visible. However, this behaviour is not yet tested by our acceptance test, and therefore TDD conveniently suggests that for the moment, it should remain unimplemented.

Simulating User Keyboard Input

The next requirement asks that our test simulate the user typing into a TextBox control on the add item form. This can be trivially implemented using the techniques discussed thus far:

# 5. She types an item name into the dialog and clicks OK
self.set_text(addItemForm.addItem, 'hello')
self.click_button(addItemForm, 'okButton')
self.assertFalse(self.on_gui(lambda: addItemForm.Visible),
    "additem form should close")

Where set_text() is a small helper function provided by AcceptanceTest:

@guithread
def set_text(self, textbox, text):
    textbox.Text = text

Running this test will successfully populate the TextBox with the word “hello”. However, this is a fairly poor acceptance test. If the TextBox control did not have focus, then a real user would have to perform extra steps before being able to type into it. If the TextBox was not visible or enabled, then a user would not be able to type into it at all. Our test implementation, by simply setting the Text attribute, performs an end-run around many of the restrictions that real users would face, and hence is not a good test of the application’s behaviour as a user would actually experience it.

The simplest way to improve on this for the moment is to explicitly test for these conditions:

@guithread
def set_text(self, textbox, text):
    self.assertTrue(textbox.Visible, 'textbox should be visible')
    self.assertTrue(textbox.Focused, 'textbox should have focus')
    self.assertTrue(textbox.Enabled, 'textbox should be enabled')
    textbox.Text = text

Running this test does actually reveal a genuine error – the add item form’s textbox does not have focus. A real user, on running WizBang, would have to click or use the tab key to give the textbox focus, a step which our test has previously been able to obliviously skip. We would prefer that our user didn’t have to do this either, so we add an Activated handler on the AddItemForm:

// C# handler for the add item form's Activated event
private void AddItemForm_Activated(object sender, EventArgs e)
{
    addItem.Focus();
}

This makes the acceptance test pass. Implementing the next test requirement is straightforward:

# 6. The new item is at the end of the list, selected.
self.assert_list_after_add()

where:

@guithread
def assert_list_after_add(self):
    self.assertEquals(self.item_count(), 4, "should be 4 items")
    wizList = AllForms.mainForm.Controls["WizList"]
    self.assertEquals(wizList.Items[3], 'hello', "list[3] wrong")
    self.assertEquals(wizList.SelectedIndex, 3, "4th should be selected")

This is asserting that the text we typed into the add item form (“hello”) is added to the end of the main form’s list, and is selected. In order to make this pass, we need a click handler for the OK button on the add item form:

// C# handler for add item form's ok button
private void okButton_Click(object sender, EventArgs e)
{
    ListBox wizList = AllForms.mainForm.WizList;
    wizList.Items.Add(addItem.Text);
    wizList.SelectedIndex = wizList.Items.Count - 1;
    this.Close();
}

Finally, the test closes down the application and ends:

# 7. She clicks the 'CloseButton'
self.click_button(mainForm, 'closeButton')

# 8. The application closes
self.assertTrue(mainForm.IsDisposed, 'mainform should close')

Notice how access to mainForm.IsDisposed is not invoked on the GUI thread. By this point in the test, all being well, the main form has been closed, and its thread will be ended. Attempting to invoke on it will fail. Instead, we read this property directly.

To make this pass, we add a simple button click hander to the main form’s Close button:

// C# handler for click event on main form's close button
private void CloseButton_Click(object sender, EventArgs e)
{
    this.Close();
}

Simulating Mouse and Keyboard Events

Directly calling controls’ methods and setting their properties, as described above, can be problematic, since the controls will not behave precisely as they would when in use by a real user. Our assertions above that controls being manipulated are currently visible, enabled and focussed are a partial solution to this problem.

However, there are many other possible ways in which our tests could unintentionally get the SUT to behave in ways that are different from the behaviour a real user would see. For example, a textbox could have a custom keypress handler, which performs input validation of some kind. Such a handler would not be invoked when our test simply sets the .Text property of the control, as we do above.

In a worst case scenario, acceptance tests could pass even though the application was completely unusable by a real user.

To combat this, the approach taken at Resolver Systems is to stimulate controls by generating fake Win32 mouse and keyboard events. This drives the application by, for example, actually moving the mouse cursor over a button, then issuing a mouse click event. Windows itself then fires the button click event, which ends up calling the click event handlers.

This approach guarantees that our acceptance tests are only able to perform the same actions that a human user would be able to, and is more of a true ‘end-to-end’ test – which seems like a good principle for acceptance tests to aim for in general.

Simulating user input in this way, however, is not without drawbacks.

The acceptance test framework sends low level mouse events, using the win32 function SendInput(), from the Windows user32.dll. Calling win32 functions from IronPython like this requires creating a C# assembly, in which we expose SendInput(), and more than a dozen similar functions, by declaring them as follows:

using System.Runtime.InteropServices;

namespace UnmanagedCode
{
    // extensive type declarations for type 'Input' go here

    public class User32
    {
        [DllImport("user32.dll", CharSet = CharSet.Auto)]
        public static extern int SendInput(
            int nInputs, ref Input pInputs, int cbSize);
    }
}

It turns out that populating the large Input data structure to be passed to this function takes another 50 lines of code. Similar scaffolding is required to support keyboard events. This is a substantial extra layer of complexity – albeit one that has proven reliable and stable since it was created.

Having said that, the acceptance tests written this way are not completely reliable in operation – developers (or office cleaners) moving the mouse or leaning on the keyboard can interfere with the running tests. Sometimes, due simply to race conditions, the GUI does not respond as quickly as the test script expects, causing spurious failures. While issues such as these can be accounted for, for example by waiting for events or for conditions before allowing tests to proceed, this still causes a significant level of false positives in our test failures, as new tests ‘bed down’ and we gradually discover and compensate for their intermittent failure modes.

Finally, it is probable that our tests run more slowly than they would if they invoked event handlers directly. To some extent this is mitigated by a distributed test runner, which splits any given test suite across all idle desktop machines in the office. However, there are many tests, some of which are stress tests, and running our full acceptance test suite still takes at least a couple of hours. Having it run faster would always be beneficial.

Because both speed and reliability might be improved to some extent by reducing our use of keyboard and mouse events, it would be advisable to at least consider trying the simpler techniques described earlier in this article, before attempting to generate mouse and keyboard events like this.

Conclusion

We’ve seen how to construct an IronPython acceptance test for an application written in C#.

Despite the pitfalls described in the last section, we’re very happy with acceptance testing of our .NET desktop application at Resolver Systems. It has proven to be eminently feasible, and of great value to us. In addition to the well-known benefits of unit-testing, acceptance testing provides an orthogonal layer of information about the system-under-test.

Most directly, it gives concrete, rapid feedback on the completion and robustness of features at a level that managers care about and users understand. Also, it provides thorough smoke tests to check that recent changes have not broken existing functionality. Perhaps most important of all, however, acceptance tests provide a method to precisely specify detailed requirements which can be easily created by users, are intimately understood by developers, are provably met, and are trivially traceable through to the final phases of the project.

Given acceptance tests such as those described in this article, test-driven development then provides developers with the ability to navigate from requirements through to working code that provably fulfils the specification, using a series of small, discrete and well-understood steps. This gives projects practical tools with which to make rapid progress, to avoid major risks, and to meaningfully measure the project’s current status and velocity.

Readers who are interested in using IronPython, for testing or for other purposes, should check out IronPython in Action, a pragmatic and densely informative new book by Michael Foord, which caters to both Python and .NET developers.

This little factoid is only useful for people living in Toowoomba. We bought a trampoline for the kids today, and it was too big to fit in the car and we got a third party courier to deliver it.

Well, the delivery guy said I would have saved $10 if I had contacted him directly, instead of doing it through Target. So here’s a plug for his business – 0417 708 362.

Since I live in Australia, shipping books from Amazon can be a significant cost in addition to the book price. Therefore, I was attracted to a New Zealand e-tailer called Fishpond who offers an Amazon lowest price guarantee.

However, there was some fine print where it can end up being more expensive than purchasing from Amazon. The catch is that shipping charges from Amazon are calculated as the sum of the base shipping charge and then the per item shipping charge, per book.

I’ve drawn up a spreadsheet (using numsum) to work this out. If you’re living in Australia, hope it helps you.

Comparison of Amazon vs Fishpond competitive pricing

It’s probably a cliché by now, but one of the most important things about Python — and dynamic programming languages in general — is the how quickly you can go from idea to implementation. Such was the case with Vulcan, a little Python module that generates random test data for Redis. It was probably 30 minutes to think up and write (I was also eating at the time).

Vulcan is dead-simple; it uses the random module of Python’s standard library to generate random integers, and then associates those random integers with Redis keys. The end-user interaction is direct — you can set how large a collection of test data you want as easily as this:

from vulcan import Vulcan
random_strings = Vulcan(100000, 'strings')

Now you just call populate():

random_strings.populate()

And Redis is now populated with a 100,000 keys on DB 6, with random integer values. You can do the same with lists, sets, and sorted sets; each gets their own test database.

It’s the kind of little thing that might save a few minutes here and there. But minutes add up — and the quicker you can take care of generic issues like this, the more time you can spend on your unique problems.

* * * * *

In other Redis-related news, I’ve added a bunch of new features to Redweb, a web interface for Redis. The new features include:

• additional sorted set functions
• more detailed dashboard information
• improvements to the UI.

I plan to spend some more time beautifying the UI this week.

* * * * *

In more general Redis news, antirez announced today that he’s joining VMWare. This is wonderful news — it means Salvatore can work full-time on Redis. Redis remains fully open-source and under Salvatore’s leadership. It’s great news for the community (and a testament to antirez’ hard work). It’s also an example of (IMHO) the most vital, forward-thinking open-source model:

Permissive license (BSD, MIT, Apache) + corporate support + developer freedom

I think we’ll see a lot more of this model going forward.

Everyone knows languages don't have speeds, implementations do. Python isn't slow; CPython is. Javascript isn't fast; V8, Squirrelfish, and Tracemonkey are. But what if a language was designed in such a way that it appeared that it was impossible to be implemented efficiently? Would it be fair to say that language is slow, or would we still have to speak in terms of implementations? For a long time I followed the conventional wisdom, that languages didn't have speeds, but lately I've come to believe that we can learn something by thinking about what the limits on how fast a language could possibly be, given a perfect implementation.

For example consider the following Python function:

def f(n):
    i = 0
    while i < n:
        i += 1
        n += i
    return n

And the equivilant C function:

int f(int n) {
    int i = 0;
    while (i < n) {
        i += 1;
        n += i;
    }
    return n;
}

CPython probably runs this code 100 times slower than the GCC compiled version of the C code. But we all know CPython is slow right? PyPy or Psyco probably runs this code 2.5 times slower than the C version (I'm just spitballing here). Psyco and PyPy are, and contain, really good just in time compilers that can profile this code, see that f is always called with an integer, and therefore a much more optimized version can be generated in assembly. For example the optimized version could generate just a few add instructions in the inner loop (plus a few more instructions to check for overflow), this would skip all the indirection of calling the __add__ function on integers, allocating the result on the heap, and the indirection of calling the __lt__ function on integers, and maybe even some other things I missed.

But there's one thing no JIT for Python can do, no matter how brilliant. It can't skip the check if n is an integer, because it can't prove it always will be an integer, someone else could import this function and call it with strings, or some custom type, or anything they felt like, so the JIT must verify that n is an integer before running the optimized version. C doesn't have to do that. It knows that n will always be an integer, even if you do nothing but call f until the end of the earth, GCC can be 100% positive that n is an integer.

The absolute need for at least a few guards in the resulting assembly guarantees that even the most perfect JIT compiler ever, could not generate code that was strictly faster than the GCC version. Of course, a JIT has some advantages over a static compiler, for example, it can inline at dynamic call sites. However, in practice I don't believe this ability is ever likely to beat a static compiler for a real world program. On the other hand I'm not going to stop using Python any time soon, and it's going to continue to get faster, a lot faster.

I gave a presentation at OpenEye's CUP last week. More precisely, I was assigned a talk with the title "Evils of KNIME." I don't chose that sort of name, but the CUP organizers like to be a bit confrontational with presentation titles. I used my speaking slot as a platform for expressing my views on dataflow/visual languages. I don't like them, and think their effectivity is limited compared to a text language, so I explained why. Other people do like them and enjoy them. I've asked them why, and they have some good reasons. My presentation outlined those responses with some observations of my own, including suggestions for ways to improve the text-based toolkits so they are more accessible to "non-programmers."

The next few posts will be based on parts of that talk. Feel free to leave comments.

Upcoming training classes (pre-announcement)

I ended by pointing out that these are technological solutions. Why not spend some time training computational chemists to be more effective at writing software? I provide that sort of training. If you are interested, email me. I'm pinning down the dates for a course in Leipzig in mid-May (likely 18-20 May), and another in Boston in late July. I'll announce them when the dates are determined. if you want to influence those dates or schedule a course at your site, let me know.

Sample test case for KNIME

I haven't used KNIME for about two years. That experience was with KNIME 1.x. People told me that it's gotten better, so I decided it was well time to take a fresh look. Last time I couldn't get it to work on my Mac. I'm happy to report that things have changed, although there are still some difficulties with it regarding updates.

My test case was the first example from the Chemistry Toolkit Rosetta, specifically, to compute the heavy atom counts from an SD file. The pybel solution is:

import pybel
 
for mol in pybel.readfile("sdf", "benzodiazepine.sdf.gz"):
    print mol.OBMol.NumHvyAtoms()

What molecular properties can I compute? And how do I do it?

The first step was to find out if KNIME could compute the number of heavy atoms. When I say "KNIME" I mean "the CDK nodes which come with KNIME" since KNIME is a dataflow-based visual programming language with support for a number of extension packages, including chemistry nodes based on the CDK. Schrodinger, Tripos, ChemAxon and likely other companies provide nodes based on their respective toolkits, but I don't have a license to those tools. In any case the Mac version of KNIME doesn't yet support adding new nodes.

The most likely candidate was "Molecular Properties." The help says:

Create new columns holding molecular properties, computed for each structure. The computations are based on the CDK toolkit and include logP, molecular weight, number of aromatic bonds, and many others.

The dialog cannot be opened for the following reason:
No column in spec compatible to "CDKValue".

A Google search for that error message found the same question from 9 September 2009 although concerning a different node. Bernd Wiswedel answered:

We obviously need to improve on the error messages. You need to process the output of the SD reader with the "Molecule to CDK" node, which will parse the structures into an appropriate format for the Lipinski node. Reason is that the Lipinski node is contributed from the CDK plugin, so it needs its desired input format.

The dialog cannot be opened for the following reason:
No column in spec compatible to "SdfValue" "SmilesValue" "MolValue" "Mol2Value" or "CMLValue".

How accessible is KNIME to first-time users?

Is that really friendly for first-time users? That is, how is a first-time user supposed to: 1) know which options are available if they can't open an unconnected node, 2) know which inputs are required for a node, or for that matter see what outputs are available, 3) know that the "SDF Reader" needs to be converted from "Molecule to CDK" before it can be used by the CDK nodes?

Of course all those can be explained in the documentation, and perhaps they are explained. I admit I haven't read it, but then again the knime.org documentation doesn't show how to use the CDK nodes. And should someone have to read the documentation in order to do something basic like this task? If so, are dataflow systems really any easier than working with a text-based programming language?

Can't compute the number of heavy atoms?

I looked through the list of properties which could be computed:

• Atomic Polarizabilities
• Aromatic Atoms Count
• Aromatic Bonds Count
• Element Count
• Bond Polarizabilities
• Bond Count
• Carbon connectivity index (order 1)
• Carbon connectivity index (order 0)
• Eccentric Connectivity Index
• Fragment Complexity
• Hydrogen Bond Acceptors
• Hydrogen Bond Donors
• Largest Chain
• Largest Pi Chain
• Petitjean Number
• Rotatable Bonds Count
• Topological Polar Surface Area
• Vertex adjacency information magnitude
• Molecular Weight
• Zagreb Index

No "heavy atom count." Next option is to see if there's a way to specify the counts based on a SMARTS pattern. Nope, didn't find anything.

As far as I can tell, there's no way with the default nodes to do much of anything with KNIME. I assume there are additional packages which I can install, but why aren't there more useful CDK nodes as part of the standard installation? An obvious one to me would be a SMARTS count pattern matcher, where I could specify the SMARTS pattern, the option for unique or non-unique matche counts, and the output column name.

Is my problem because I'm on a Mac? Do Linux users get more nodes? Or is there something else I'm missing? How would you find the number of heavy atoms using KNIME? Is there a solution using the default CDK nodes or do I have to use one of the commercial toolkits?

Leave answers and comments here.

For the familiar and impatient: Loghetti has moved to github and has been updated. An official release hasn’t been made yet, but cloning the repository and installing argparse will result in perfectly usable code. More on the way.

For the uninitiated, Loghetti is a command line log sifting/reporting tool written in Python to parse Apache Combined Format log files. It was initially released in late 2008 on Google Code. I used loghetti for my own work, which involved sifting log files with tens of millions of lines. Needless to say, it needed to be reasonably fast, and give me a decent amount of control over the data returned. It also had to be easy to use; just because it’s fast doesn’t mean I want to retype my command because of confusing options or the like.

So, loghetti is reasonably fast, and reasonably easy, and gives a reasonable amount of control to the end user. It’s certainly a heckuva lot easier than writing regular expressions into ‘grep’ and doing the ol’ ‘press & pray’.

Loghetti suffered a bit over the last several months because one of its dependencies broke backward compatibility with earlier releases. Such is the nature of development. Last night I finally got to crack open the code for loghetti again, and was able to put a solution together in an hour or so, which surprised me.

I was able to completely replace Doug Hellmann’s CommandLineApp with argparse very, very quickly. Of course, CommandLineApp was taking on responsibility for actually running the app itself (the main loghetti class was a subclass of CommandLineApp), and was dealing with the options, error handling, and all that jazz. It’s also wonderfully generic, and is written so that pretty much any app, regardless of the type of options it takes, could run as a CommandLineApp.

argparse was not a fast friend of mine. I stumbled a little over whether I should just update the namespace of my main class via argparse, or if I should pass in the Namespace object, or… something else. Eventually, I got what I needed, and not much more.

So loghetti now requires argparse, which is not part of the standard library, so why replace what I knew with some other (foreign) library? Because argparse is, as I understand it, slated for inclusion in Python 3, at which point optparse will be deprecated.

So, head on over to the GitHub repo, give it a spin, and send your pull requests and patches. Let the games begin!

So, I happened across this post about hiring programmers, which references two other posts about hiring programmers. There seems to be a demand for blog posts about hiring programmers, but that’s not why I’m writing this. I’m writing because there was this sort of nagging irony that I couldn’t help but stumble upon.

In a blog post, Joel Spolsky talks about the mathematical inaccuracies associated with claims of “only hiring the top 1%”. It seemed pretty obvious to me that whether or not you’re hiring the top 1% of all programmers is pretty much unknowable, and when managers say they hire “the top 1%”, I assume they’re talking about the top 1% of their applicants. Note too that I always thought it was idiotic to point this out, because, well, isn’t that what you’re SUPPOSED to do? You’re not very well going to aim for the middle & hope for the best are you?

Apparently I’ve been giving too much credit to management. There I go giving people with ties on the benefit of the doubt again.

Then, in another blog post, Jeff Atwood talks about how it’s very difficult to even get interviews with programmers who can actually program. The problem is real.

The original blog post that pointed me at the two others is one by Roberto Alsina where he talks about his own methods for weeding out the non-programmers. He’s clearly seen the issue as well.

But if you open all three of these posts in separate tabs and read them, you’re likely to come away with the same basic problem I did:

• Who the hell are these managers who can’t figure out a dead simple statistics problem?
• How can a person fairly inept at simple math be qualified to make a hiring decision for anything but a summer intern?

That sorta blew my mind a little. But it blew my mind a lot when Atwood started describing the problems that interviewees *couldn’t* perform in an interview! One task described by Imran was called a ‘FizzBuzz’ question. Here’s one such question:

Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.

Here’s the part that blew my mind: He says, and I quote:

Most good programmers should be able to write out on paper a program which does this in a under a couple of minutes.

Want to know something scary ? – the majority of comp sci graduates can’t. I’ve also seen self-proclaimed senior programmers take more than 10-15 minutes to write a solution.

That’s amazing to me. I decided to quickly pop open a Python prompt and see if I could do it:

>>> for i in range(1,101):
...     if (i % 3 == 0) and (i % 5 == 0):
...             print i,'FizzBuzz'
...     elif i % 3 == 0:
...             print i, 'Fizz'
...     elif i % 5 == 0:
...             print i, 'Buzz'
...     else:
...             print i
...

Note that I’ve taken the liberty of printing out the numbers in addition to the required words. I’m playing the role of interviewer and interviewee here, and wanted to be able to easily verify that things were correct, since there was no time for unit testing

Turns out it worked on the first try! That was pasted directly from my terminal screen. I didn’t time myself, but it took far less than 5 minutes. This leads to my other question, of course, which is “if you’re going to complain about CS degree holders not writing good code, maybe it’s time to open the doors to non-CS degree holders?”

Creating a framework takes an incredible amount of hubris. When you create your own framework, you appear to be saying these things:

• The set of contracts offered to you by the other 200 frameworks of the same kind are wrong.
• I know what the right set of contracts is.

This assessment of the pigheadness of creating a new framework isn't quite right though. What you're really saying when you create a new framework is:

• The set of contracts offered to me by the other 200 frameworks don't fit my brain, or I can't figure them out.
• I know what the right set of contracts is for me.

Thus, it's never a-priori pointless to create a framework. It's always an option. In a world of poorly-documented, poorly-tested, poorly-designed frameworks, sometimes it's just required. But it is pointless to create a framework if you advertise it as something that other people should use and any of the following conditions are true:

• It doesn't need to be a framework.
• You write no documentation for it.
• You write some good, incomplete documentation for it.
• You write bad documentation for all of it.
• You write no tests for it.
• You write some good, incomplete tests for it.
• You write bad tests for all of it.
• Your API design skill are poor.
• You have no intention of maintaining the software in the future or you have a record of abandoning projects quickly.
• You can't stand criticism.
• You're relying on directly making money from its sale or adoption.
• You completely reinvent the wheel; your framework is largely indistinguishable from some other non-pointless framework.

By this set of criterion, some percentage of the web frameworks listed on the Python wiki are indeed pointless. But although the very last bullet point condition in the above list ("you completely reinvent the wheel...") is often cited as the primary reason that writing another web framework is a bad idea, most of the web frameworks on that page can't be discounted as pointless for this reason at all. Most are quite novel, and contain new ideas, and many solve some web development subproblem better than any another. Instead, web frameworks can more often be categorized as pointless for some combination of the other reasons in the above list: didn't need to be a framework, no docs, bad docs, no tests, bad tests, bad API, bad maintenance record, or inappropriate expectations.

As someone who has written a good number of frameworks (many of which have utterly failed), I'll try to offer some tips for people thinking of writing their own framework. Using these tips doesn't guarantee success, but failing to heed them seems to almost certainly guarantee failure.

Tips for checking that your idea even rates being a framework at all

• You've written the same code at least 20 times in your programming career.
• It doesn't make sense as a library.
• No, really. It really doesn't make sense as a library.
• The problem that your framework solves should largely represent a career choice.
• If the framework doesn't represent a career choice, it better be damn small.
• If someone tells you this shouldn't be a framework, they might be right.
• If someone tells you this shouldn't be a framework, they might be wrong.

Some tips for writing docs

• Write them.
• Write them in English (even if it's not your mother tongue, sorry).
• Write them in clear English.
• Write them in clear, engaging English.
• You need to document everything.
• You need to document everything, even the stuff you don't think you need to document.
• When someone complains about something not being documented: remember, you need to document everything, even the stuff you don't think you need to document; write docs in clear, engaging English.
• Keep the docs up to date.
• When the docs are out of date, fix them so that they're back in date.
• Keep around versions of docs that relate to versions of your framework; not everybody uses the newest one.
• Doctests aren't docs.

Tips for writing tests

• Write them.
• Write them in Python.
• Write them in clear Python.
• You need to test everything.
• You need to test everything, even the stuff you don't think you need to test.
• When someone complains about something not being tested: remember, you need to test everything, even the stuff you don't think you need to test.
• Keep the tests up to date.
• When the tests are out of date, fix them so that they're back in date.
• Doctests aren't tests.
• Run the tests often.

Tips for writing APIs

• More API is never good. Adding an API should be a momentous occasion.
• Less API is usually good, but only if it's not too little.
• APIs are contracts: break them at your peril.
• Coddling a bad API forever is worse than breaking a contract.
• APIs that take a lot of arguments are usually bogus.
• Sometimes big, bogus APIs are just what the doctor ordered, though, if it means that people need to remember less.
• When your fingers type the wrong thing, they're usually righter than your brain. Fix your API instead of retraining your fingers.
• Features are the enemy.
• Libraries are better than frameworks.
• Frameworks are better than failure.
• Requiring that a user subclass a framework-defined class is almost always the wrong answer.
• Requiring use of a framework-defined decorator is almost always the wrong answer.

Tips for maintenance

• If you usually start things and don't finish them: don't do it.
• Maintainance sucks.
• You have to do maintenance, even though it sucks.

Tips for expectation-related things

• You won't make any money from this.
• If you make some money from this, consider it a freak accident.
• Criticism is healthy. It means someone cares enough to even try your code. Treat criticism as massive success.

Have fun with your new framework!