Whenever Codal puts together proposals for our clients, we typically dedicate a slide or two to potential project risk factors. Collaboratively identified by our designers, engineers, and business analysts, these risks are comprised of any possible roadblocks that could delay the project’s timeline or impact the budget.
Most of the risks depend on the nature of the project, but one that we include every time is the difficulty of recruiting and scheduling subjects for user testing. At Codal, we take user testing seriously—it’s diffused within our UX design process, regularly occurring at nearly every phase.
We recognize it as the only way to truly validate the choices a designer makes, and we recognize it can be a serious headache, especially within the time constraints of some Agile design cycles. Identifying, locating, and convincing a representative sample of potentially several different user bases is no easy task.
But we’ve been doing this for close to a decade at Codal, so we thought we’d offer a few of our own tools, tricks, and strategies for expediting the recruitment process. This isn’t high-level advice; rather, we’re providing a more in-the-weeds perspective from our UX researchers.
HOW MANY DO I NEED TO RECRUIT?
Before we tackle actual recruitment, it’s important to have a solid understanding of how many subjects you’ll need to enlist. At Codal, we typically aim for 5-10 subjects per user type. This usually provides a representative sample of the target demo—enough for the data to reveal definitive conclusions.
Of course, numbers don’t always care about our timelines, and sometimes the results of user testing aren’t conclusive enough. When this happens, Codal will increase the subject pool by 5-10, and repeat testing (pending the client’s approval of further testing of course).
GUERILLA & HALLWAY TESTING
While some applications have highly specialized user types, others are defined with much broader brushstrokes—middle-aged women, maybe, or millennials. For cases such as these, often times UX design agencies will employ guerilla or hallway testing.
Guerilla testing simply consists of approaching potential users, usually in a public setting like a busy street or coffee shop, and conducting the testing on the spot. It’s useful for when your user type is something that can be easily eyeballed, or simple enough that it can be discerned with a brief requisite question e.g. ‘are you bilingual?’.
In a similar vein, hallway testing is conducted on-site, and subjects are recruited simply by asking around the office. Like guerilla testing, it only works if the user type is a broad category—if it’s too specialized, it’s unlikely you’ll get a large enough sample.
Designers utilizing these types of recruitment enjoy the benefit of relatively low costs. All subjects need to be incentivized in some way, and typically those recruited via guerilla testing are satisfied with a $5-$10 gift card.
TASKRABBIT
TaskRabbit is a popular mobile app designed to connect freelance labor with local demand, particularly for everyday tasks such as cleaning, assembling, or moving. Despite it’s errand-centric design, TaskRabbit can also serve as a reliable source for usability test subjects.
Like guerilla testing, it’s best for less specialized user types, but it can often provide a more representative sample than the local Starbucks. You pay for the improved subjects as well: “Taskers” can typically charge $30 at the minimum for their participation in your UX studies.
That works out to about $300 per user type, which could potentially hurt the budget if the platform you’re testing has a particularly diverse user base. Still, it can be a cost-effective option compared to some other user recruitment services.
USERBOB
UserBob is a user testing service that performs just about all of the heavy lifting of usability testing. You simply provide the app, website, or even scenario you’re investigating, and provide a list of tasks you’d like the user to complete.
UserBob does the rest: assembling a representative sample, providing the subjects with access to the platform, tasking them with the prompt, and recording their screen and voice as they talk through their experience. All the UX designer has to do is review the videos.
This is an attractive option for any UXer who dreads the rigamarole of usability testing, but it does have it’s drawbacks. UserBob’s recording equipment doesn’t capture user taps, scrolls, or other gestures when using apps. Codal prefers to use Lookback, a screen capture program with much more functionality.
UserBob charges $2 per user, plus $1 per requested minute of testing. So depending on the test you’ve designed, this could be reasonable pricing or something way out of your budget. On average, Codal finds that our testing sessions typically fall around $100 per user type.
USERLYTICS
Finally, Codal will sometimes turn to Userlytics, the heavy artillery of test subject recruitment. Like UserBob, Userlytics offers testing services, but at a much more comprehensive level.
Leveraging a combination of social media and contextually and behaviorally optimized ad networks, Userlytics can assemble highly representative samples of just about any demographic you can think of. These are high quality samples, and the Userlytics engine can build them in a matter of hours.
It’s a massive time saver, but its efficiency comes with a hefty price tag: the service charges about $90 per user, so you’re getting close to dropping a grand for every user type.
TESTING…TESTING…1, 2, 3
Hopefully a brief look into Codal’s recruitment techniques will help you identify and assemble representative test subjects yourself. While we touched on a smattering of our approaches, the ones detailed here vary for projects of differing scopes and budgets.
Still, the best user testing is the one you don’t have to do at all. If you’re thinking about refreshing your website or app, or want to know what your users really think of it, consider enlisting the services of a UX design agency.
We understand recruitment is difficult, which is why we add it as a risk factor to projects that require usability testing. But we also understand how to mitigate it, how to expedite it, and how to stop it from hindering the flow of a project. That’s what makes Codal one of the top UX design agencies in America.
