hedgehog lab

Corporate Programming - It's better than a James Bond movie

An excellent talk by Zed Shaw and worth a watch for every budding (or experienced) programmer.

Bug Tracking Best Practices

I recently got asked by a customer about whether we eat our own dog food in using fixx (which we do) and the conversation veered towards what I think are bug tracking best practices.

A lot of the so-called best practice is just common sense (as it always is) but it is surprising how little they are used. Therefore, I thought I would share what we feel are bug tracking best practices from the view-point of 3 different roles.

For end users

When you report a software bug, it is imperative that you detail how the bug was caused in the first place. This will help the developers in finding the root cause and fixing it.

Never provide a solution unless you are a developer in-charge of the functionality. Leave it to the developer to implement the best solution.

For Developers

Be patient! We all know how frustrating bug reports can be. Try to re-assign the issue back to the reporter and ask for clarification rather than dismissing it.

Don't be defensive. I have certainly been guilty of this. No developer likes being told their code is buggy, but try to verify whether something is a bug or not before you dismiss it.

Try to use filtering, reporting and organisation features in the bug tracker to keep your focus on a specific set of issues.

Do not close issues unless you raised them. The only person who should close an issue is the person who raised the issue and can verify if it is resolved or not.

Try to provide estimates that accurately reflect the effort required for a task.

Added by David in comments - If you are working on an open source project, try to provide a suggested solution (in the form of source code patches) along with the bug report.

For Project Managers

Resist the temptation to keep adding new "fields" and meta data. Remember that the focus of the bug tracking system is not to store boat loads of information but to enable you to get things done and fix issues.

Try to keep the workflow simple. The key is to get an issue from open to closed in the fastest time possible. Complex workflows only serve bureaucracy and encourage procrastination in the process.

Trust your developers and testers to make the right decisions. Do not enforce too much control with complicated permissions and umpteen approval/review processes to get a bug resolved.

Try to keep tasks or issues that are small and achievable. "Develop a visitor management system" is not a feature/task, it is a project (as ludicrous as it may seem, this is a real world example).

Don't go overboard with reports. Project Managers love nothing more than constantly generating reports that all say the same thing with a different presentation.

I would love to hear if any of you have any other tips or best practices that you use in your organisation.

Time to make usability a first-class citizen in the software development process.

A year or so ago, I met with a then-potential client, who we pitched our software development process to. I went through our usual routine of explaining the various techniques and methods involved, when the client stopped me on 1 specific slide.

"Requirements workshops? Prototype? User Experience?", he said. "Frankly, I am not sure we can justify the spend on that. We are fairly confident of what we need!" With that, he pushed a well-bound booklet called "System Requirements Specification", which some poor analyst had spent months preparing in an isolated cubicle.

Usability or User Experience (or whatever else you choose to call it), is a fledgling concept (in the larger field of Human Computer Interaction), that is all too often considered a "cost" on software projects. Things are certainly changing, with trend-setters like Google and Apple championing both the practical and aesthetic aspects of usability. However, there is still a distinct problem in how organisations (especially in the software industry) deal with usability.

Organisations that have a high focus on usability separate this function away from the core development team, usually employing people in specialist usability roles or worse, an isolated department. There are, of course, great advantages to having specialists in usability and user experience (as with any other discipline), but the problem lies in most developers taking the "not our problem" stance.

Although advanced usability skills require specialist learning and experience, there is no excuse for developers to not be familiar with basic usability concepts and apply them in their every-day role. The expansive literature that is available on usability makes it a breeze even for a novice to learn the basics.

At hedgehog lab, we expect every developer to be involved in the usability process, both at a design and implementation stage. We have no secret-sauce for training them. Just an open-mind, a constant thirst to learn, and some great usability books like the following,

• Don't make me think
• Elements of User Experience
• About Face
• Designing the obvious
• Information Dashboard Design

It is high-time that software developers listed usability and user experience design as a key-skill in their arsenal. Let's face it; very soon "usable" will cease to be a benefit or a USP and become the norm, and those software developers who do not embrace this concept will be left behind.

[fixx hacks] The Upcoming API

We, here at the lab, appreciate the value of letting the user hack out their own solutions to problems that they may have stumbled upon that don't fit our vision. There's barely a day goes by that you won't hear Mark suggest a rubbish new feature, get instantly shot down, and then insist that he'll write a Greasemonkey script for that functionality anyway.

With this in mind, the fixx team have always maintained that there will be a fixx API, so that developers may adapt fixx to meet their bug tracking needs. While still in relatively early stages, that API is coming together and you can check out the progress in the beta by going to any issues screen and appending .xml or .json to the URI (/issues.json, for example). This was still mostly unusable (unless you want to jump through some pretty hefty hoops), until recently when Sarat committed a change to allow basic HTTP authentication. This change is not available to the masses yet, however, to show off its potential usefulness, I've put together a quick shell one-liner to get the number of issues assigned to me:

curl -u user:password http://SERVER/issues.xml?qAssignedTo=19 | xmlstarlet sel -t -v "count(//issue)"

Which when fed into conky, an ace Linux desktop text rendering app, leaves you with a widget that looks a bit like this:

Useless? yes. Limited? Of course. But if this divvy can use it, you won't have any trouble whatsoever.

User testing on a shoestring budget

If you are a freelancer or small agency designing websites, usability testing the websites you create is a bit of a challenge due to the planning, time and cost involved in setting the tests up. Setting up a good user testing environment and getting your target audience to give you valuable feedback is a tough task and hence it is overlooked.

We had the same problem when we decided to go live with the website. Given that we are a small and agile company, we were not bothered about getting user feedback before we went live as we always envisioned the site to improve incrementally with rapid iterations.

We considered many options for user testing and most seemed far too much effort given our busy period (with the release of fixx coming near) and our limited budget. Therefore, I set out on finding methods that will allow us to get some user feedback without breaking the bank.

Out of all the things we tried (which included pestering our developer friends constantly for feedback), 3 really stood out for us and returned tons of feedback which we are sifting through. I admit that the quality of these tests cannot be on par with setting up a controlled user testing environment and neither am I preaching that these methods are better (no flame wars Information Architects!), but if you have limited resources, reach and budget, I can assure you that these will return better results than by-passing usability testing.

• Usertesting.com

  I found usertesting.com while browsing through some posts at the Joel on Software forums and man are we impressed! For less than $19 a user, usertesting.com allows you to conduct highly targeted tests. At the end of the test, you will have access to a flash recording of the user's session (with voice) and a transcript of their summarised feedback. I won't go into a sales pitch for them, as their home page does a good job of this but I really like the idea of how usertesting.com has leveraged crowd-sourcing to deliver what is an interesting and useful service. Needless to say, we are very happy with the results and quality of feedback. If anyone knows of other services like usertesting.com, please feel to post in the comments.

• LinkedIn

  A curious one but based on 2 past experiences, I have found LinkedIn to be a good source of feedback based on target audience. Their "Answers" feature allows you to select specific categories under which to post a question. Again, the quality of feedback we had from there has been pretty good. Although I guess this doesn't classify into the scientific mould of "testing", a lot of the audience was our target market. The cost of this exercise? Nil! I presume this could be said of any other "Answers" service but I am only going by what our experience is.

• Friends, colleagues and clients

  If you are lucky enough to be in an industry where your target customers are your friends, then getting feedback from them is zero cost. Likewise, colleagues in similar jobs but other companies are very valuable, except if they were your competitors! Finally, the obvious one, if you are lucky enough to have good relationships with you're existing clients, there is no one better to get user feedback from (I know you are going duh! but I have seen many instances where people never talk to their customers other than to bill them or pitch new work!).

Do you have an interesting methodology to conduct user testing that is not obvious and does not cost a lot? We are always looking to improve our processes, so please let us know.

Django: Debug techniques and IPython

Since joining The Lab, I have had the pleasure of working with Django a whole lot more. In this time, I’ve noticed a couple of tricks / niceties that make development run a lot smoother, here are a couple…

Evolution of debugging

Web development is full of little debug tricks, usually involving variations on variable inspection. Django provides lots of such useful methods, and I have found myself progressing through more powerful solutions as I discover them.

1. return HttpResponse({variable to inspect})

   As far as I am aware, this is the most basic form of debugging. Just stick that string at the point in your view where you wish to examine the state and it will short circuit the view code with the variable result. It’s the Django equivalent of alert(). In a lot of development, this will suffice, however this is limited to the scope of view code.

2. print {variable to inspect}

   Usable in any Django Python code running on the Django development server, lob this in and it will publish the result to the console. I usually put some \n’s around it to easier spot it amongst the response codes. For me, the limitation of only being able to use it in the dev server isn’t a big one, because that’s where I do most of my development. The only real issue is when I realise that I want a different variable, change the code and restart the server. Some debug context (solved by the next method) would be nice.

   (#5677 suggests that this is bad form and could cause troubles when using over mod_python, depending on your set-up.)

3. raise Exception({variable to inspect})

   I realised this beaut through the most excellent Django wiki. By raising the exception (with debug = True in your settings.py), you force the lovely Django debug page to display - chocked full with helpful info, tracebacks and context - along with your inspected variable at the top.

4. import pdb; pdb.set_trace()

   This is my personal favourite. It activates the Python Debugger and allows you to step through your code and interactively check each variable.

   Type “help” for a list of commands, “args” prints the arguments sent into the currently running function, “next” executes the next line, p {variable to inspect} prints a variable, pp {variable} pretty-prints a variable and continue tells pdb to run the rest of your code (unless we hit another breakpoint).

My google-fu has recently revealed django-logging, a Django middleware component which provides the ability to spit out variables in your view and based upon the Python Logging module. I am yet to try this, but it looks like another ace string in your Django debugging bow.

In addition to these, there is always the incredibly helpful <% debug %> tag, to spit out your context variables while template coding.

IPython

IPython has spoiled me rotten. It is an enhanced Python REPL environment with a lot of extra bells and whistles. First off, the code completion is fab. django-admin.py recognises that you have IPython installed, so typing “./manage.py shell” allows you to import your models and inspect them with a quick “.<TAB>”.

IPython makes reading code / docstrings a breeze - you don’t know what django.models.cheesypuff is? type “django.models.cheesypuff?” to get information and documentation about the object / method printed to your screen. That doesn’t help? then try “django.models.cheesypuff??” to get documentation AND source code. It’s brilliant.

The best bit about IPython (apart from the above and it also having access to your system shell) is that it is scriptable (in Python, of course). So all sorts of nice little tricks can be used to make life easier, one of my favourites is this little snippet. It tells IPython to automatically import any model associated with your current Django project, I’ve edited mine slightly to print the names as they’re imported so I have a nice index of the models available to me without having to traverse lots of models.py files.

In short, IPython is a fantastic tool - any work I do on querysets or databases I interactively run through in that shell first - it makes it easier to quickly catch gaps in my logic. IPython also stores all of your history (type “%history”), so you can easily copy and paste that into your editor of choice. There’s plenty I haven’t discussed (if you paste in doctests, it strips out the >>>’s allowing you to run them with a quick ctrl+v), but I’ll leave you to have a play.

What is your favourite Django debug technique?

Painless wireframes and agile user-centered design

At hedgehoglab we pride ourselves on the user-centered approach we use in designing our software, with our core product fixx seeing a number of iterations over the past few months, primarily in terms of UI design but more importantly in information architecture surrounding the views and the simplicity of use.

The process that we currently use consists of first deriving use cases of how the user will perform each operation in the system (in the form of user stories), then on to quick mock ups of real application screens in the form of wireframes (using a standard layout and component library) and finally short iterations of focused review by the entire team. We repeat this process until every use case of the system is addressed from a user perspective.

The problem with such agile reviews and lightning changes of the UI and wireframes is that it leaves some poor soul, namely me, to reflect these changes in the wireframes ready for another review. A time consuming job without the right tools believe me!

After hearing many good things from other Information Architects, we originally tried Visio for a while in the hope that this would allow for quicker iterations, but I found Visio lacking in visual appeal and there was always a bottle neck in duplicating effort by translating most Visio wireframes into visual designs after every iteration. In short, Visio sucked at providing quick turn around in terms of having realistic application screens designed.

The second option was opting for mockups in XHTML, thinking we’d save time further down the line when we could just tag some CSS, refine the XHTML and lo and behold, have a completed view. This, however, proved more effort than estimated with the intermediate & final CSS changes and mark-up tweaks ultimately ending in a mash-up of good and bad code, something we ended up throwing away and opting for a re-write. We also found that XHTML wireframes did not allow the flexibility or speed of change we required to make the wireframes as visually complete as possible.

So, what is it that has me ranting about wireframes and turnaround?…

Fireworks, the web developers’ friend!

Although not comparable to the likes of Photoshop and Illustrator (tools my designer chums swear by), Fireworks lends itself well to the needs of most web developers when it comes to slicing and dicing the visuals ready for a quick conversion into finished XHTML and CSS. It also allows us to create wireframes that are visually rich and are as close to the completed product as we would like. This also eliminates the step of producing plain wireframes in Visio first and then designing them out in a graphics package.

Working with Fireworks the past few days has seen about 10 iterations of the fixx UI go from a 5 hour job to no time at all. Each change was painless and immediate (and visually correct for everyone to review). Fireworks does a good job of keeping your pages and layers organised meaning exporting them into a reviewable image format was effortless and because Fireworks is out for both windows and mac (sorry Ubuntu fan boys, no Linux yet), we can all chip in when we need to and work on the UI of the system.

The end result? The fastest and most visually stunning wireframes & mock-ups I’ve produced in a long time with the advantage of ready-made visuals! All they need now are a little bit of eye candy thrown in, in the form of icons and content and we are away!

If you want to see sneak previews of the fixx UI we have put together keep an eye out for my next post where i will be discussing some of the changes we have made to the UI with plenty of screenshot goodness. Oh! And did I forget to mention that fixx will be entering internal beta soon at the lab?

Rails performance tips

We have been recently working on a top-secret Rails app and performance has been a primary consideration. Although I agree that “scaling is a nice problem to have sometime in the future“, it does not mean we throw good software design out of the window. It causes a lot less pain and effort designing something better from the ground-up than re-factoring an entire code-base at a later point.

With this in view, we set about investigating a few pointers to improving Rails performance. A note that all of these are links that have been collected from our days learning Rails from various sources on the web. I just could not let these languish in our internal wiki!

• Optimising ActiveRecord
• Scaling Twitter
• Common performance problems in Rails
• Optimising AJAX in Ruby on Rails
• Optimising Rails resource usage
• Top 10 Ruby on Rails performance tips

Easing MySQL pain when using Locomotive and XAMPP

If you are one of those n00bs that are looking to learn Ruby on Rails on a Mac OS X and happen to use the unfortunate combination of Locomotive and a XAMPP-based MySQL server, you will probably be frustrated with the following error,

           Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)

The secret sauce to solve this problem is to connect to your MySQL using an IP address (to force TCP/IP) rather than the default socket-based connection initiated by using localhost.

Simply change the following line in your Rails config/database.yml from,

  host: localhost

to

  host: 127.0.0.1

iPhone Coding Tips Part 1 : Memory Management

Tracking down the source of bugs while writing code for any type of computer system can sometimes be one of the most interesting and challenging parts of developing software. It's almost like getting a really awesome puzzle book, but rather than having the answers listed somewhere towards the back, their spread across forum posts or SDK documentation, and in some of the more challenging cases just require you to wade off into the unknown, narrowing down the problem until you finally discover the one line that causing the entire thing to come crashing down.

After spending weeks or even months seeing bugs crop up and cause problems that seem to bare no resemblance to one another, you begin so see patterns forming, and rather than just seeing strange console messages and stack traces, you see the same errors cropping up again and again.

Over the next few weeks, I'm going to talk about some of the more frequently occurring bugs that I find in my code, useful tips I've discovered, and hopefully give any developers new to the iPhone scene, a slight head start.

Throughout the time I've been working with the iPhone SDK I've encountered many problems, some of which are one offs that you'd probably only ever need a fixx for once every blue moon, where as others occur more frequently (almost daily). The most major source of problems tends to stem from the way in which objective-c implements memory management. For those of you unfamiliar with how this works here is a quick rundown...

The most basic way to allocate memory for, and release objects is to use the alloc and release methods that will be inherited from the NSObject base class. Basically when we wish to allocate memory for an object we...

//Allocate memory for, and initialize the object
MyClass *aClass = [[MyClass alloc] init];

// When we are finished with the object we release it.
[aClass release];

the release method simply reduces the object reference count by 1. If this reference count reaches 0 then the objects dealloc method will be called, purging the object from memory.

Provided we always have matching pairs of alloc and release no memory should be leaked.

The second way of handling memory is to use Autorelease objects. Autorelease objects and convenience constructors are hugely useful when you only need to use a variable temporarily, or need to return an object from a method and need your code to take responsibility for the memory management. The only problem is there are loads of occasions where the pool memory containing my objects is released, and I find myself getting nil pointer exceptions, where my code is trying to message objects that don't exist any more. My remedy for this is to only use convenience constructors in confined blocks of code, where the object isn't likely to go out of scope and be released.

Now enter threading! If you use autorelease or convenience constructors in methods being run on other threads, make sure you initialise an NSAutoreleasePool at the beginning of your code block and release it at the end of your code block. This is because the default pool runs on the main thread only. If you don't do this you'll end up with memory leaking all over the place! Just keep an eye on the console as autorelease objects that have no pool will log messages warning you...

- (void)randomFunction
{
[self performSelectorInBackground:@selector(someMethod)];
}
- (void)someMethod
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
// ...
// Do some stuff here involving autoreleased objects.
// e.g.
// NSString *aString = [NSString string];
// NSData *someData = [NSData data];
// NSArray *myArray = [[[NSArray alloc] init] autorelease];
// ...
[pool release];
}

But remember that after releasing the pool you can't return any objects/values that were set to autorelease, although in most cases you won't be able to return objects anyway.

The majority of places I use this are in background syncing methods, or other blocks of code where literally tens, hundreds or even thousands of autorelease objects are created, and obviously if not dealt with properly, just float around taking up valuable resources.

Well thanks for reading guys, I hope this micro-rant has been of help. My next post will be all about singleton objects in Objective-c, and the best way to implement these.