If you are a freelancer or small agency designing websites, usability testing the websites you create is a bit of a challenge due to the planning, time and cost involved in setting the tests up. Setting up a good user testing environment and getting your target audience to give you valuable feedback is a tough task and hence it is overlooked.
We had the same problem when we decided to go live with the website. Given that we are a small and agile company, we were not bothered about getting user feedback before we went live as we always envisioned the site to improve incrementally with rapid iterations.
We considered many options for user testing and most seemed far too much effort given our busy period (with the release of fixx coming near) and our limited budget. Therefore, I set out on finding methods that will allow us to get some user feedback without breaking the bank.
Out of all the things we tried (which included pestering our developer friends constantly for feedback), 3 really stood out for us and returned tons of feedback which we are sifting through. I admit that the quality of these tests cannot be on par with setting up a controlled user testing environment and neither am I preaching that these methods are better (no flame wars Information Architects!), but if you have limited resources, reach and budget, I can assure you that these will return better results than by-passing usability testing.
I found usertesting.com while browsing through some posts at the Joel on Software forums and man are we impressed! For less than $19 a user, usertesting.com allows you to conduct highly targeted tests. At the end of the test, you will have access to a flash recording of the user's session (with voice) and a transcript of their summarised feedback. I won't go into a sales pitch for them, as their home page does a good job of this but I really like the idea of how usertesting.com has leveraged crowd-sourcing to deliver what is an interesting and useful service. Needless to say, we are very happy with the results and quality of feedback. If anyone knows of other services like usertesting.com, please feel to post in the comments.
A curious one but based on 2 past experiences, I have found LinkedIn to be a good source of feedback based on target audience. Their "Answers" feature allows you to select specific categories under which to post a question. Again, the quality of feedback we had from there has been pretty good. Although I guess this doesn't classify into the scientific mould of "testing", a lot of the audience was our target market. The cost of this exercise? Nil! I presume this could be said of any other "Answers" service but I am only going by what our experience is.
Friends, colleagues and clients
If you are lucky enough to be in an industry where your target customers are your friends, then getting feedback from them is zero cost. Likewise, colleagues in similar jobs but other companies are very valuable, except if they were your competitors! Finally, the obvious one, if you are lucky enough to have good relationships with you're existing clients, there is no one better to get user feedback from (I know you are going duh! but I have seen many instances where people never talk to their customers other than to bill them or pitch new work!).
Do you have an interesting methodology to conduct user testing that is not obvious and does not cost a lot? We are always looking to improve our processes, so please let us know.
Over the past 6 months, I have interviewed close to 100 plus jobseekers and I have to agree with the core points Seth makes. Most of the resumes I receive are monotonous and you could not tell one apart from another, if it not were for the name at the beginning. Some have cover letters, some do not, but the overriding theme between all of them is common skills, relatively similar experience and a list of hobbies that do not tell me anything about what makes the person exceptional.
There is no surprise then that only 1 of our 3 existing team members actually had to submit a resume to us, which we never read twice. All 3 were hired from our network of influence either because we worked with them previously or because they came highly recommended from people we knew.
As a software company, the best thing you can do is get involved with the community and get to know who the brilliant developers are in your area of expertise. Too often, smaller technology companies get so involved in operations and selling, that they distance themselves from the communities that matter to them. The common excuse revolves around, "We just don't have enough time!" Therefore, recruitment becomes a cumbersome business with the daily drone of skimming through resumes and fighting over fees with your recruitment consultant.
Getting involved with the community means that you get to spot the talented people who have skills that cannot be quantified on paper. I would love to employ a Don Brown or a Gareth Rushgrove, as I am sure many others would. What sets both of them apart is their influence in their respective communities (even ignoring the fact that both of them are really good at what they do) and their involvement with grassroots level open source and standards groups. I doubt either of them will ever need to "apply" for a job.
As a developer, you can boost your chances by getting involved with various open source groups or contributing to the community through conferences, blogs or just plain brilliant code. Sure it will take time and sure it involves a lot of work but at least it will save you the effort of writing one more mundane resume.
Software is overrated! You might think that is a bit rich coming from someone who runs a software company. The problem is that to most people, software is synonymous with the code underlying a system. Sure, writing good code is a tough task but let me assure you that writing code itself isn't all that difficult (software companies create value in more than just code, but that topic is for another day).
Ideas are overrated! I cannot begin to describe the number of times I saw a new product or service (software or otherwise) and thought, "Blimey! They stole my idea."
So if both code and ideas are overrated and hardly ever original (note I say hardly, some ideas and code are original), then why is it that technology companies classify these as "intellectual property"? Why is there so much bickering over who owns what when a clever programmer in his basement can create the next big thing and almost any software system can be cloned without much effort?
When will managers and leaders in technology focused companies realise that their real intellectual property are their people?
“The major problems of our work are not so much technological as sociological in nature.”
Simply put, software companies like to blame technology, law, environment, government and everything else for their problems rather than deal with the real problems of people (employees), customers and management.
What better way to make your failure to run effective projects and deliver on time appear more valid, than blaming "intellectual property" issues and crying foul against competitors?
Here is an idea! Why not invest in a great team, listen to your people, bring in better managers, treat your customers with honesty and transparency, and focus on running effective projects. Then see how much the rest matters!
I was recently trawling through my Google Reader subscriptions and came across a post about workplace experiments, which struck a chord with some of the things we do at hedgehog lab to make it a great place to work for our team.
We don’t call what we do “workplace experiments”, as it is just standard practice for us. I thought it would be good to list out all the unique things that we do to enhance everyone’s jobs here at hedgehog lab. I appreciate (given the heated responses to the 37signals post) that these won’t and can’t apply to everyone, so I am in no way recommending these as best practice (although I have yet to get anything but staunch support from our team on these).
4-day week
One of our founding principles was the 4-day week. With the exception of client demands (as we still do some consulting), we regularly work 4-day weeks. No one is required to come into work on a Friday if they do not want to. The problem we are facing is keeping our team away on a Friday! The reality is, if you create the right environment and give people a goal to work towards to, you will have a tough time keeping them away from work.
Expenses
Although we do not offer everyone a company card, we never say no to anything that someone in the team wants (that is reasonable). Books, software and conferences are paid for by the company as long as we can afford them (see you at @media 2008). This ensures our team gets what they want without spending the time getting them; allowing them to focus on getting on with their jobs.
Working from home & Flexible hours
One of the toughest things you can do as an “employer” is trust your team to get on with their job without constantly looking over their backs. Therefore, most working from home policies for office-based businesses are either half-baked or cosmetic at the best. Not so at hedgehog lab.
Our working from home policy is completely flexible and anyone from our team can work from home at management’s discretion. So whether you are having that brand new laptop delivered home or you need to see a doctor miles away, you can rest assured that it will be stress free.
Similarly, our flexible hours are actually flexible. When most companies use the word “flexible hours” they mean that you can come in an hour later but have to stay back an hour later too! When we say “flexible hours”, we are saying “we appreciate there will be times when you cannot get into work on time or want to leave early and we are fine with that!” There is no need to stay later or come in earlier to “make up the hours”. I have yet to see this being abused at the lab!
Profit sharing
Unfortunately we don’t have a bonus scheme at hedgehog lab as we feel that it depends too much on the whims of the person deciding the bonus and offers no real method of measuring contribution.
At hedgehog lab, we have a flat profit sharing scheme. It works on the basis of setting aside a share of our profits every year and dividing this equally between all team members. No matter what your role, junior or senior, director or developer, you get an equal share of the profit. Of course, there might not be a profit to share but that means no one gets an incentive. We feel this is a great way to reward overall performance of the team and company while ensuring no one feels singled out.
Overtime
Although we frown at overtime for our internal projects, work for clients sometimes dictates that we are forced to ask some of our team to work overtime. We accept this as a by-product of service work but instead of fighting against it, we embrace it by innovating on how people are paid for overtime. Instead of paying overtime rates of 2x or 3x times someone’s hourly rate, we pay them the whole amount that the company makes for that work from the client. You heard that right!
These are just some of things we try and do to make life better for everyone in the team. I would be interested in finding out what works for other people and welcome any debate on our approach.
Intellectual Property or IP, for those who don’t know, is one of those words used by companies that never actually produce anything original but claim domain over everything. Even Wikipedia cannot agree on what it exactly stands for (the article on IP is in dispute). In one of my recent interviews, I (possibly controversially) used the phrase “Vaporware is not Intellectual Property.”
This raised an interesting debate internally about the issue,”At what stage in the production of a software system does it actually become your IP?” I am sure this is a highly debatable issue and I know everyone of us had different ideas of what actually constitutes IP.
One thing we were all in agreement though was that Vaporware or software that is nothing more than an idea, or concept (talked about around a drink at a pub) and a few powerpoint slides is not IP. Please! I have heard too many stories from friends who dreamed about building Facebook in 1995! Tough! You did not and Mark Zuckerberg now has enough money to buy a few houses.
Since joining The Lab, I have had the pleasure of working with Django a whole lot more. In this time, I’ve noticed a couple of tricks / niceties that make development run a lot smoother, here are a couple…
Evolution of debugging
Web development is full of little debug tricks, usually involving variations on variable inspection. Django provides lots of such useful methods, and I have found myself progressing through more powerful solutions as I discover them.
return HttpResponse({variable to inspect})
As far as I am aware, this is the most basic form of debugging. Just stick that string at the point in your view where you wish to examine the state and it will short circuit the view code with the variable result. It’s the Django equivalent of alert(). In a lot of development, this will suffice, however this is limited to the scope of view code.
print {variable to inspect}
Usable in any Django Python code running on the Django development server, lob this in and it will publish the result to the console. I usually put some \n’s around it to easier spot it amongst the response codes. For me, the limitation of only being able to use it in the dev server isn’t a big one, because that’s where I do most of my development. The only real issue is when I realise that I want a different variable, change the code and restart the server. Some debug context (solved by the next method) would be nice.
(#5677 suggests that this is bad form and could cause troubles when using over mod_python, depending on your set-up.)
raise Exception({variable to inspect})
I realised this beaut through the most excellent Django wiki. By raising the exception (with debug = True in your settings.py), you force the lovely Django debug page to display - chocked full with helpful info, tracebacks and context - along with your inspected variable at the top.
import pdb; pdb.set_trace()
This is my personal favourite. It activates the Python Debugger and allows you to step through your code and interactively check each variable.
Type “help” for a list of commands, “args” prints the arguments sent into the currently running function, “next” executes the next line, p {variable to inspect} prints a variable, pp {variable} pretty-prints a variable and continue tells pdb to run the rest of your code (unless we hit another breakpoint).
My google-fu has recently revealed django-logging, a Django middleware component which provides the ability to spit out variables in your view and based upon the Python Logging module. I am yet to try this, but it looks like another ace string in your Django debugging bow.
In addition to these, there is always the incredibly helpful <% debug %> tag, to spit out your context variables while template coding.
IPython
IPython has spoiled me rotten. It is an enhanced Python REPL environment with a lot of extra bells and whistles. First off, the code completion is fab. django-admin.py recognises that you have IPython installed, so typing “./manage.py shell” allows you to import your models and inspect them with a quick “.<TAB>”.
IPython makes reading code / docstrings a breeze - you don’t know what django.models.cheesypuff is? type “django.models.cheesypuff?” to get information and documentation about the object / method printed to your screen. That doesn’t help? then try “django.models.cheesypuff??” to get documentation AND source code. It’s brilliant.
The best bit about IPython (apart from the above and it also having access to your system shell) is that it is scriptable (in Python, of course). So all sorts of nice little tricks can be used to make life easier, one of my favourites is this little snippet. It tells IPython to automatically import any model associated with your current Django project, I’ve edited mine slightly to print the names as they’re imported so I have a nice index of the models available to me without having to traverse lots of models.py files.
In short, IPython is a fantastic tool - any work I do on querysets or databases I interactively run through in that shell first - it makes it easier to quickly catch gaps in my logic. IPython also stores all of your history (type “%history”), so you can easily copy and paste that into your editor of choice. There’s plenty I haven’t discussed (if you paste in doctests, it strips out the >>>’s allowing you to run them with a quick ctrl+v), but I’ll leave you to have a play.
