Witter – a basic python twitter client for Maemo

So I wrote last week about developing a basic twitter client. And this week I got the main stuff done, and wanted to share the code example here.

In my looking for help developing apps for Maemo from a start of basically no GTK knowledge or python knowledge I found the examples either too trivial, or way over engineered. So I wrote this intending it to be useful (to me), contain only enough capability to basically read my timeline and tweet. I’ve intentionally not added bells and whistles (yet) and it’s a single ‘monolithic’ app. By which I mean it’s all in a single python file, there is no separation of gui and logic, no nice engineered constructs etc etc.

I hope that it does show an intermediate level example of writing an application for Maemo using Python. This is what it looks like in action

This was taken full screen. The app supports switching in and our of full screen. And it adjusts the width of the displayed text to fit. As you can see the shot was taken not long after completing the application.

It also sorts the tweets using the ListStore ability to just tell it which column to sort on. This is very useful as it means I don’t have to mess around myself. Originally I had it sorting on created_at, but since I was loading that as a String it would order Thursday below Wednesday. Rather than cast the string into a meaningful date object of some form, I just used ID instead, which is a Long. Still comparing as a string, but the number always increments so newer tweets always appear t the top.  Obviously if you prefer newer tweets at the bottom, just flip the sort order to Ascending.

So here is the code, it’s a little under 300 lines, but I’ve commented it pretty well (I think) to explain what it’s all doing.

# ============================================================================
# Name        : witter.py
# Author      : Daniel Would
# Version     : 0.1
# Description : Witter
# ============================================================================

#This is the bunch of things I wound up importing
#I think I need them all..
import gtk
import pygtk
import hildon
import urllib2
import urllib
import base64
import urlparse
import simplejson
import socket

#Initially I found I'd hang the whole interface if I was having network probs
#because by default there is an unlimited wait on connect so I set
#the timeout to 10 seconds afterwhich you get back a timeout error
# timeout in seconds
timeout = 10
socket.setdefaulttimeout(timeout)

#the main witter application
class Witter(hildon.Program):
    #first an init method to set everything up
    def __init__(self):
        hildon.Program.__init__(self)
        #being lazy this just uses basic auth and I am not doing anything
        #yet to store uid/pwd so for the moment just put info here
        self.username = "YOUR_USERNAME"
        self.password = "YOUR_PASSWORD"
        #This being a hildon app we start with a hildon.Window
        self.window = hildon.Window()
        #connect the delete event for closing the window
        self.window.connect("delete_event", self.quit)
        #add window to self
        self.add_window(self.window)
        #For this app I wanted a scrollable area for the tweets to show up
        #so I create a gtk ScrolledWindow
        self.scrolled_window = gtk.ScrolledWindow(hadjustment=None, vadjustment=None)
        # as well as somewhere to show the tweets we need somewhere to write a tweet
        # this being twitter we cap the input at 140 chars
        self.tweetInput = gtk.Entry(max=140)
        # we also want a couple of control buttons to load up tweets and submit a tweet
        self.buttonloadTweets = gtk.Button(label="Load Tweets",stock=None, use_underline=None );
        # we connect out load tweets button to the getTweets method
        self.buttonloadTweets.connect("clicked", self.getTweets)
        self.buttonnewTweet = gtk.Button(label="Tweet",stock=None, use_underline=None );
        #we connect the Tweet button to the newTweet method
        self.buttonnewTweet.connect("clicked", self.newTweet, self.tweetInput)
        # a vertical box to set the scrollable window and the button box
        # in the display
        self.box1 = gtk.VBox(False, 0)
        #a horizontal box to put our tweet input box and two control buttons in
        self.buttonBox = gtk.HBox()
        # add the Vbox to the window
        self.window.add(self.box1)
        # create a menu object by calling a method to deine it
        menu = self.create_menu(self.scrolled_window)
        # add the menu to the window
        self.window.set_menu(menu)
        # define a liststore we use this to store our tweets and some associated data
        # the fields are : Name,nameColour,Tweet+timestamp,TweetColour,Id
        self.liststore = gtk.ListStore(str, str, str, str, str)
        # create the TreeView using treestore this is the object which displays the
        # info stored in the liststore
        self.treeview = gtk.TreeView(self.liststore)
        # create the TreeViewColumn to display the data, I decided on two colums
        # one for name and the other for the tweet
        self.tvcname = gtk.TreeViewColumn('Name')
        self.tvctweet = gtk.TreeViewColumn('Tweet')
        # add the two tree view columns to the treeview
        self.treeview.append_column(self.tvcname)
        self.treeview.append_column(self.tvctweet)
        # we need a CellRendererText to render the data
        self.cell = gtk.CellRendererText()
        # add the cell renderer to the columns
        self.tvcname.pack_start(self.cell, True)
        self.tvctweet.pack_start(self.cell,True)
        # set the cell "text" attribute to column 0 - retrieve text
        # from that column in liststore and treat it as the text to render
        # in this case it's the name of a tweeter
        self.tvcname.add_attribute(self.cell, 'text', 0)
        # we then use the second field of our liststore to hold the colour for
        # the 'name' text
        self.tvcname.add_attribute(self.cell, 'foreground', 1)
        # next we add a mapping to the tweet column, again the third field
        # in our list store is the tweet text
        self.tvctweet.add_attribute(self.cell, 'text',2)
        # and the fourth is the colour of the tweet text
        self.tvctweet.add_attribute(self.cell, 'foreground', 3)
        # we start up non-fullscreen, and we want the tweets to appear without
        # scrolling left-right (well I wanted that) so I set a wrap width for
        # the text being rendered
        self.cell.set_property('wrap-width', 500)
        # make it searchable (I found this in an example and thought I might use it
        # but currently I make no use of this setting
        self.treeview.set_search_column(0)
        # Allow sorting on the column. This is cool because no matter what order
        # we load tweets in, we always get a view which is sorted by the tweet id which
        # always increments, so we get them in order
        self.liststore.set_sort_column_id(4,gtk.SORT_DESCENDING)
        # I don't want to accidentally be dragging and dropping rows out of order
        self.treeview.set_reorderable(False)
        #with all that done I add the treeview to the scrolled window
        self.scrolled_window.add(self.treeview)
        # Then just 'pack# the scrolled window and a Hbox into the
        # V box
        self.box1.pack_start(self.scrolled_window, True, True, 0)
        self.box1.pack_start(self.buttonBox, False, True,0)
        #and pack the hbox with input field and buttons
        self.buttonBox.pack_start(self.tweetInput, True,True,0)
        self.buttonBox.pack_start(self.buttonnewTweet, False, False,0)
        self.buttonBox.pack_start(self.buttonloadTweets, False, False,0)
        #setup some urllib things to use to fetch twitter feeds
        self.last_id=None

    def quit(self, *args):
        #this is our end method called when window is closed
        print "Stop Wittering"
        gtk.main_quit()

    def create_menu(self, widget):
        #a fairly standard menu create
        #I put in the same options as I have buttons
        # and linked to the same methods
        menu = gtk.Menu()

        menuItemGetTweets = gtk.MenuItem("Get Tweets")
        menuItemGetTweets.connect("activate", self.getTweets )
        menuItemTweet = gtk.MenuItem("Tweet")
        menuItemTweet.connect("activate",self.newTweet)
        menuItemSeparator = gtk.SeparatorMenuItem()
        menuItemExit = gtk.MenuItem("Exit")
        menuItemExit.connect("activate", self.quit);
        menu.append(menuItemGetTweets)
        menu.append(menuItemTweet)
        menu.append(menuItemSeparator)
        menu.append(menuItemExit)
        menuItemFile = gtk.MenuItem("File")
        menuItemFile.set_submenu(menu)
        return menu

    def run(self):
        #this is the main execution method
        # we set things visible, connect a couple of event hooks to methods
        # specifically to handle switching in and our of fullscreen
        self.window.show_all()
        self.window.connect("key-press-event", self.on_key_press)
        self.window.connect("window-state-event", self.on_window_state_change)
        #this starts everything up
        gtk.main() 

    def getTweets(self, *args):
        #Now for the main logic...fetching tweets
        #at the moment I'm just using basic auth.
        #urllib2 provides all the HTTP handling stuff
        auth_handler = urllib2.HTTPBasicAuthHandler()
        #realm here is important. or at least it seemed to be
        #this info is on the login box if you go to the url in a browser
        auth_handler.add_password(realm='Twitter API',
                          uri='http://twitter.com/statuses/friends_timeline.json',
                          user=self.username,
                          passwd=self.password)
        #we create an 'opener' object with our auth_handler
        opener = urllib2.build_opener(auth_handler)
        # ...and install it globally so it can be used with urlopen.
        urllib2.install_opener(opener)
        #switch on whether this is an refresh or a first download
        if self.last_id == None:
            json = urllib2.urlopen('http://twitter.com/statuses/friends_timeline.json')
        else:
            #basically the twitter API will respond with just tweets newer than the ID we send
            json = urllib2.urlopen('http://twitter.com/statuses/friends_timeline.json?since_id='+str(self.last_id)+'L')
        #JSON is awesome stuff. we get given a long string of json encoded information
        #which contains all the tweets, with lots of info, we decode to a json object
        data = simplejson.loads(json.read())
        #then this line does all the hard work. Basicaly for evey top level object in the JSON
        #structure we call out getStatus method with the contents of the USER structure
        #and the values of top level values text/id/created_at
        [self.getStatus(x['user'],x['text'], x['id'], x['created_at']) for x in data]

    def getStatus(self, user,data, id, created_at):
        #at this point user is another JSON structure of lots more values of which we are currently
        #only interested in screen_name
        #append to our list store the values from the JSON data we've been passed for a tweet
        # the funny #NXNXNX type values are colours I chose a slightly blue for the name
        # and black for the tweet. At some point I intend to do some alternating colours for
        # cell backgrounds to make the display clearer
        self.liststore.append([ user['screen_name'],"#2E00B8",data+"\nposted on: "+created_at,"#000000", id])
        #now we process the id, this is so we can do a refresh with just the posts since the latest one we have
        #if we haven't stored the most recent id then store this one
        if self.last_id == None:
            self.last_id=id
        else:
            #if we have an id stored, check if this one is 'newer' if so then store it
            if long(self.last_id) < long(id):
                self.last_id=id

    def newTweet(self, widget, text_widget,*args):
        #The other main need of a twitter client
        #the ability to post an update
        #get the tweet text from the input box
        tweet = text_widget.get_text()
        #see if we have just an empty string (eg eroneous button press)
        if (tweet == ""):
            return

        #we get the text in the input box then we construct the outbound tweet
        #first we need to encode for utf-8
        tweet = unicode(tweet).encode('utf-8')
        #then we need to urlencode so that we can use twitter chars like @ without
        #causing problems
        post = urllib.urlencode({ 'status' : tweet })

        #build the request with the url and our post data
        req = urllib2.Request('http://twitter.com/statuses/update.json', post)
        #setup the auth stuff
        auth_handler = urllib2.HTTPBasicAuthHandler()
        auth_handler.add_password(realm='Twitter API',
                              uri='http://twitter.com/statuses/update.json',
                              user=self.username,
                              passwd=self.password)
        opener = urllib2.build_opener(auth_handler)
        # ...and install it globally so it can be used with urlopen.
        urllib2.install_opener(opener)
        json = urllib2.urlopen(req)
        data = simplejson.loads(json.read())
        #message sent, I'm assuming a failure to send would not continue
        #in this method? so it's safe to remove the tweet line
        # what I don't want is to lose the tweet I typed if we didn't
        # sucessfully send it to twitter. that would be annoying (I'm looking
        # at you Mauku)
        text_widget.set_text("");

    def on_window_state_change(self, widget, event, *args):
        #this just sets a flag to keep track of what state we're in
       if event.new_window_state & gtk.gdk.WINDOW_STATE_FULLSCREEN:
            self.window_in_fullscreen = True
       else:
            self.window_in_fullscreen = False 

    def on_key_press(self, widget, event, *args):
        #this picks up the press of the full screen key and toggles
        #from one mode to the other
       if event.keyval == gtk.keysyms.F6:
             # The "Full screen" hardware key has been pressed
             if self.window_in_fullscreen:
                 self.window.unfullscreen ()
                 #when we toggle off fullscreen set the cell render wrap
                 #to 500
                 self.cell.set_property('wrap-width', 500)
             else:
                self.window.fullscreen ()
                #when we toggle into fullscreen set the cell render wrap
                #wider
                self.cell.set_property('wrap-width', 630)

if __name__ == "__main__":
    #this is just what initialises the app and calls run
    app = Witter()
    app.run()

And that’s it. I used esbox to develop and just had it using SCP/SSH to copy accross to my n810 and execute directly there, which was a pretty easy way to develop.

There are lots of things I will now go on to add to this client. Things like checking for replies/DMs. Being able to make easy reference to an URLs in tweets, and reply to people etc etc. But I wanted to show the code of the bare bones working in case it helped anyone else get started with developing apps for Maemo.

Developing applications for Maemo

When I first got my nokia 770 I intended to develop for it. I had a few ideas of what to write, but discovered apps already existing for just about everything I could think of. I also discovered that it’s not that easy developing for maemo. To write in C you need scratchbox and an emulated environment. It all looked very complicated and I never got into it.
When I got my n810 I figured it would be fun to try and write an app that could render openstreetmap data, as an alternative to maemo mapper which uses pre-rendered tiles. But again I found Navit, already very good.
I thought about getting involved with Navit development. But I have no C experience and the code is somewhat light on comments.

Recently I decided to have a go again. This time I chose to ignore existing apps. The point is to learn, so I have decided to work on my own twitter client. Yes very trendy…there are hundreds around, but only one really for Maemo, which is mauku.
Part of my reason for chosing to write a twitter client is that it’s pretty simple but covers some basics that will be helpful.
Writing a GUI app that displays data and can scroll
calling out to some webservice and processing the response

I figure if I can get those two then I’ll have a good grounding for lots of projects.

The other part of my reason is that mauku has some very annoying ‘features’ which are driving me crazy. Now I could just raise feature requests and bug reports etc. But again, the point is to learn, so I have to start somewhere. At least this way I don’t feel it’s a complete duplicated effort, I will scratch my personal itch and make a twitter client the way I want it.

I started off trying to write a c application. I found esbox which makes the whole developing with scratchbox thing a much nicer prospect than plain text editors. Being an eclipse tool, I feel right at home using it.

I got my C application as far as being able to display a window, with a menu. And in the menu was a fetch tweets option. When selected it would talk to twitter and fetch my friends timeline into memory, then parse each tweet into a simple structure of name:tweet. Which was then inserted into a list in the display. Hurrah.

However C is a horrible language to work in. I just don’t have time to learn memory management etc. I know I should, I know that the potential is to write a much faster application in C. But I got sick of segfaults. I have no idea how to debug C. I am truly a Java boy. Give me stack traces! Trying to debug why my C code would just randomly explode was impossible (for me) It seemed like sometimes something in the reponse from twitter would cause it to segfault. But more likely I was just memory management completely wrong and was just lucky that occassionally it didn’t explode in my face.

So I’ve decided life is too short for C. Instead I’ve moved on to python. I’ve never written any python before, but it at least saves me from managing memory, and it looks like it’s going to be much closer to things I do know, such as perl.

Python can still be developed in ESBOX with pluthon plugin. This basically lets me write an app then use SSH to dispatch it direct to my n810 to run. So I no longer test in a fake environment. It is probably much slower working this way, but the benefit is that I really know that the app runs on the device. Where as I think I’d always have a nagging fear with running in scratchbox that I might hit some difference in how I had it setup to the real device.

So I started my application again. I’ve been able to reuse at least some of what I had learned about GTK development. This is still the area that I know the least about. It took me ages to get to the point I had the ability to display items on the screen, in a TreeView pulling data from a ListStore. Figuring out how to use items that could be added to by triggered methods and generally getting my head around the scope of objects etc.

Having gotten some basic information appearing on the screen, it then took about 30mins to pick uip the liburl2 library and use it to call twitter to get my status feed.

Now I just need to write some code to parse it into some sensible structure and dislpay. I would point out that I have found that someone has already created a twitter API for python which I could just pick up. I’m sure it does way more, way better than my code will. However, I have to remind myself that the point of this exersise is for me to learn, not just for me to glue other peoples code together.
Also I may not bother with a lot of the things I could process from twitter, Instead just writing a fairly minimal application. At least to start with.

First impression of python is that it’s pretty weird the way it uses indentation to imply structure. If you get the indents wrong then the code won’t work. It feels like it was done by someone who got very upset about people having badly formatted code, so decided to invent a language in which the formatting is enforced because that’s the only way the language will work. Not that I have anything against nicely formatted code. But I’d just as soon have an editor do nice formatting for me to suit my preferences.

Once I feel I know what I’m doing a bit more, I shall write up a post with code fragments to explain what I’ve learned. I think if I put up the code I have right now it would just be confusing. I always find that my initial attempts to get something working are very badly commented and horribly structured. At the moment I’m just throwing stuff in based on API doc and examples and I’m not sure I understand the implications of everything yet. Once I understand it, I can comment it and make it presentable.

I’m hoping that given enough time I’ll understand what I’m doing enough that I can start turning out interesting little tools and hacks. And perhaps finally realise my original intention with my internet tablet to do some mobile development.

First steps in a new software project

This week I started a brand new project. This time it’s the start of product development, rather than just an internal project to get something running. And it really is new, so everyone is new and we’re setting up everything from scratch.

So this is my list of things to go from nothing to a fully fledged software engineering project.

Step one, getting the right tools installed for everyone off the bat makes things easier later on. Standardise upfront rather than try to merge half a dozen separate ideas about whats best later.
For this team we will be testing a GUI so Rational Functional Tester (RFT) with Rational Team Concert (RTC) is the basic tool set.

This gives us all the project management tools in RTC, plus source control, and easy collaboration. Then when we get as far as something to test RFT will already be there.

RTC is the best tool I’ve used for starting projects, it lets you start capturing requirements and high level ’stories’ and start hanging individual tasks off of them. Code sharing is very easy, and delivering code changes under tasks makes the project tracking pretty seamless.

Step two, a build server. It’s easy to link into RTC to have a build engine that allows you to schedule builds as frequently as you like. It also allows people to kick off builds of the mainline code plus their ‘local’ changes. So you can build and unit test changes before delivering them to the main codebase. The outline of this can be set up quickly, the detail of full build and kicking off unit tests takes longer but is a top priority.

Step three is a Rational Quality Manager (RQM) server. This can be linked to the RTC server so that it gets notified when builds complete and also allows you to create defects in RTC if tests fail. RTC can then view defects which block test cases. RQM lets you define what environments you will test in, how you will split things into test plans. And makes it easy to start defining test cases and where they will run. Ultimately it can be used to report on test status, with built in dashboards and report generation.

The last step are some test servers which run adapters which connect to the RQM server. For our needs some of these will use a command line adapter, others will use RFT. This allows RQM to kick off tests against the systems and gather the results.

As a set of tools they are not light weight. Requiring at least 4 servers (RTC, RQM, build, test) but then they are a very powerful combination. I don’t expect to have any spreadsheets or hand-crafted chart generation being done by the team. All of the information we require comes straight from the tools, when we report project status we will use the tools not hand-made slides or spreadsheets. I also expect to be able to get tests automated and managed from the start. So no trying to retro-fit automation onto manual tests.

After one week we have RTC setup with the first wave of requirements stories and tasks, and a code base setup with components for our new product. We have an RQM connected and being told about builds. We have the skeleton of a build system. It’s not doing much other than defining the build types we will have, but the framework is in place.

Next week I’ll setup my first test server. Hopefully in a very short space of time we will have everything we need to develop and test a product. Complete with project tracking and reporting, all while most of the team are focussed on what we will produce, not tied up with figuring out infastructure.

Retrospective on an agile project

Yesterday I finished my 5 week project. I still can’t believe it’s been 5 weeks, it’s gone so fast.
I wrote a bit about it a couple of weeks ago. And now that it’s finished I wanted to reflect a little on how it went. This, of course, is part of the agile process. They call it a retrospective, and in theory you do one every iteration to consider what worked, and what didn’t. Since my iterations were just one week long I didn’t write anything down for each one.

One thing I said in my previous post is that I wouldn’t work long hours to achieve results, as it is unfair on future projects to misrepresent what can be done in the time. Almost the day after posting that I found myself breaking that rule.

I worked several long evenings, but I have an excuse-ish. One of our big problems was getting a system set up so that we could start on some of our tasks. But it gave us enormous problems, and I was tired of burning days on what should have been a couple of hours of set up. So I started working into the evening in the hopes of putting us back on track. Regardless it still took almost 2 weeks to get a system we could use. This was incredibly frustrating and I broke my rule because I didn’t consider the task part of what the project had to achieve. It was just a pre-req. But an important reminder that sometimes the things that take the longest are things you don’t even officially plan or size. I should have taken half a day at most, but it burned more like eight and a half.

I also broke my rule for another reason, perhaps one I should have factored in. We often needed to work with a team in the US, specifically with us raising questions or problems, and getting responses. To that end I joined a weekly conference call from 17.30 till 18.30. I also found it was better to check e-mail into the evening, since it was the difference of responding to a question and getting a second response by the next day versus finding the question the next morning. It’s easy to burn through days bouncing question/answer/question back and forth at a rate of one request/response per day. The conference call helped to speed things up. However the reality of relying on a team in another timezone is that you make faster progress if you spend more overlapping work time.

The common theme to these two rule breakers is things that felt unrelated to the *work*. Just stuff we needed to be able to get on with the doing. That said I need to remember in future to factor this stuff into short projects.

In terms of project results, I have mixed feelings. We failed to achieve all I wanted. However, I did achieve the major strategic investment I wanted. I never expected to be ‘done’ in 5 weeks. Just to have kick started it enough that it becomes a long term commitment to the direction. Although my project is over, most of the team will continue. Everyone agreed that we were going in the right direction, and we did enough to convince people that it *can* work.

I’m very happy about achieving the important attitude shift, and establishing the relationships necessary to succeed in the long run. At the same time I’m very frustrated by some of the things we didn’t achieve already which I feel we could have.

For me personally a big part of the project was trying to lead. My natural instinct when people aren’t doing what I’d like, is to wish I had the AUTHORITY. You know? That feeling that it would be easier if I had been bestowed with power such that people must do as I say. Perhaps this is what attracts people to management. To have that explicit power defined. However, I do not want to be a manager. So the trick for me is trying to lead with no explicit authority. And boy can that be frustrating!

I feel I had a particularly challenging crew. On previous projects I’ve worked with keen, enthusiastic people. Willing to listen and discuss. Ultimately accepting the group decisions, and working furiously towards the agreed goals.
This project could not have been more different. One person in particular had very specific views on what they wanted to do, and how. Whilst this fit with what was necessary that was fine, great even as they made great progress. But they would sometimes decide randomly on another priority. They totally didn’t get the idea of agile and committing to deliverables for this iteration. On one particular occasion when I asked on a Monday about getting started on one of our committed items, I was told ‘I’m not going to look at that this week’, other times I asked for where they had gotten to, and what was blocking progress. Only to be deflected with questions about other things. The attitude being that their tasks were none of my business, they didn’t want to explain the problems or the specific progress. They just wanted to be left alone for as long as it took to do it, their way.
On one occasion I got the Instant Message equivalent of being hung up on. Rather than answer my question I was told ‘I need to concentrate’ and they logged off.

It would be easy to say that person was just difficult, and in future I’ll only work with easier people. But of course I realise I have to learn to get the most from people like this. Perhaps I should have been more direct and just asked ‘ why don’t you want to tell me what the problems are?’

In that regard I feel I did badly as a leader. However in other regards I was getting better. One of my problems as a techie is that I come up with the solution I want, and I try to get people to do it my way. For this project I made a conscious effort to define the result I needed, and accept solutions that weren’t ‘my’ way, so long as they would meet the requirements. Sometimes this means watching something take longer than I think it should take. But maybe that’s just because the solution will be better?

So I still have much to improve on. That said, I was explicitly thanked by the manager in charge of the project for my leadership. So it can’t have been all bad

Automation isn’t free, and change isn’t either.

When I started this blog I intended it to be about both woodworking and software engineering. It’s pretty clear that my focus has been woodworking, and more specifically woodturning.
However, that’s not to say I’m not busy thinking about software engineering, I just don’t often think to write about it.

This week I found myself starting another agile project. A little over a year since I wrote about being ‘so agile they’ll need another word‘ and so I thought I’d write up some more thoughts on the subject of software engineering and agile projects.

Last year the project was one I had spent some time building support for, to overhaul our automation infastructure to be more sophisticated. It required a change in the accepted way of delivering results. Which meant convincing project managers that the new information would be *better* than what they were used to. Then convincing management that thoe whole package would be worth the cost of about 4 person months.

I really enjoyed that project, and I learned a lot. Both about how to run an agile project, but also about how to get one off the ground in the first place. I came to the following conclusion

People don’t know what they’re doing.

By which I mean, all of those people who ‘control’, managers, project managers, strategists etc etc. They mostly don’t have grand plans and visions. They are now ochestrating clever and subtle schemes to reach their goals. They basically are just winging it, like the rest of us.

The implications of this realisation are simply that if you have an idea of what you want to change, and can formulate a half decent plan to achieve it. Then there is a good chance that managers and planners will seize upon it with joy. At last! A plan! I don’t mean to undermine what these roles do, they keep projects moving, they understand what is required to deliver a project. They remember what ahs gone before. But the important realisation is that they rarely have time to think about great changes from the ‘norm’ and they will gleefully accept that idea you’ve been toying with, if only you can pitch it in their language.

Last year the project was a great success. And one year on I believe people are still feeling the benefits, looking back over the project it is clear that things went better, ran smoother. The project delivered on it’s promises.

That makes for a good reference when talking again about a project to run. And so it is that I started a few months ago paying attention to general talk around adopting a new product to manage our test tracking and reporting. There was a general feeling that we ought to be looking at it, but by default everyone is already completely commited to plans. And so ‘looking at it’ means, occasionally having a meeting to talk about it, but not actually having any time to *do* anything. It seemed clear that this would be my next target.

Part of the message I try to get accross about why such projects are necessary is : Automation isn’t free. And neither is significant change for the better. In short, you can’t expect to acheive great things for nothing.

For too long auotmating tests was sold as expensive up front, but then you got a lot ‘free’ like platform converage, repeatability etc. I think that this message set the wrong picture. Because many automation infastructures are woefully under funded for on going work. The attitude is very much that ‘we paid the high up front cost, now this bit is free…right?’
Well no, automation isn’t free, it requires people to make sure things keep ticking along. It requires updates to support new things. It requires skill and knowledge in what is often a complex environment. However, it has such a high rate of return, such an amazing return on investment, that it is easy to think of many of the benefits as ‘free’

I struggle to push the message that you get what you pay for, and when you pay into your automation infastructure you get a LOT. But it DOES require on going investment.

In this case I’m not dealing with automation as such, but we are talking about a big change. It has the potential to make many areas of our project ‘governance’ much better. It has fantastic scope to open us up to options that simply can’t happen without the first big step.

And it is this that I told people when looking into adopting this new technology. Anything *less* than a couple of person months dedicated time to get things going is a joke. If you aren’t prepared to stump up the resource for at the minimum that much. Then stop talking about it, stop thinking about it, because you’d just be wasting your time. Sure there are plenty of other things that you need to acheive with that resource, and you need to understand the relative priority of this compared to those things. But don’t for a second kid yourself that the team can effect such large change without some real time to plan, test, try, deploy.

And so here I am, 1 week into a 5 week project, where I have been given time, and a student. To really get to grips with how we will adopt this new technology. My goal is to have it either going into production at the end of the project. Or have a very specific list of blockers, and associated defects and actions to resolve them.

I love working this way. I have a set of stakeholders, people who will be the real customers of the system. and I meet them weekly on Friday’s. In that meeting I show them what has been achieved during the week, and get their approval to mark certain tasks as ‘complete’ Then I work with them to define the tasks of the next week. Sometimes this means making them agree amongst themselves the priority order for new tasks. Then everyone knows where we are, and what we’re doing. Why we are doing this first instead of that. The list of requirements can grow as and when the stakeholders think of them and define what they want. But they also get to help chose which of those on the backlog they *really need* this week. The great advantage of this is that you could keep the project rolling until people run out of *must have* items. You can stop at any time that people feel they’ve had most of the value. In essence the stakeholders are the ones who will ultimately decide some fraction of the requirements they raised, really weren’t that important afterall.

It also means that last weeks work finished on Friday, and next weeks work starts on Monday. That sounds kinda obvious, but pychologically, I am not starting to think about the challenges of next week yet. And those of last week are done, for better or worse I met with our stakeholders and showed them what we had and that was the end of the iteration. Working this way keeps me focussed. Everything boils down to ‘what do I have to show the stakeholders on Friday’. It makes for an intense working week, and sometimes it’s hard to go home at the end of the day and NOT keep working. But I have to remember that if did that it would be a disservice to anyone in the future. It would be dishonest to show a project completing on schedule, if it actually took me a weeks worth of over time to do so.

Week one is always hard in terms of just ramping up, but week two is where things are going to get serious. The level of expecation goes up a few notches, and my little team needs to start delivering real change. I’m looking forward to it, I know that success in this project is more than just delivering what I said I could, it’s showing that this whole way of working delivers consistent results. It means that next time I pitch a project, my job is a little bit easier.

Many projects I’m sure, struggle to justify the cost in terms of the benefits they yeild. I find a different challenge, trying to remind people there is a *cost* and what that is. It easy to see the value, but automation isn’t free, and change isn’t either.