User testing in the wild: research at conferences and other events
Sometimes the hardest thing about user testing is simply making time for the logistics. At Twilio, we’ve found that events we’re already going to — like conferences and industry meetups — can be a great way to get easy access to a big pool of potential users. User testing at events will not give you perfect metrics, ideal user profiles, or a deep understanding of interaction details. But it’s a great technique for getting lots of broad feedback to guide detailed research you’ll want to do next.
Here are some tips to help you conduct great interviews in busy and unpredictable event settings.
1. Target and schedule
Begin by defining what you want to learn, your ideal users, and the kinds of metrics you will gather. And remember: Conferences are highly photographed, so don’t test anything that can’t show up on Twitter.
Identify events that attract the kinds of users you want to learn about (your sales and marketing teams can probably help). Schedule 30-minute sessions that align with session times, workshop hours, talks, panel discussions, lunches, and happy hours. Avoid keynotes and product announcement times — people will be too focused on the actual conference to pay attention.
2. Seek permission
Ask the conference organizers for permission to run user testing sessions during their event. Be precise about how, when, and why you want to do it at their conference — even if your own company is putting on the event. Ask where you can set up for your user interviews. If you get pushback, find the root of the hesitation and politely assure the organizers that your user testing sessions will not detract from the conference.
3. Prepare your setup
Here’s a checklist of things you’ll need to prepare in order to conduct an interview at an event:
- Pens and paper for taking notes
- Consent forms (for recorded sessions that will be shared)
- A space where you’ll do interviews. Will it be quiet enough? Will people need to sit down? Will you be interrupted by other people who want to talk to your participants? Make sure you’ve got a plan that covers all of your contingencies.
- A script for the questions you want to ask. Check beforehand to make sure you can easily read it while talking to your tester. Especially when at a busy event and short session, make sure to have your participants think aloud.
- Reliable access to power for your laptop.
- A wireless card for Internet access. If you cannot bring a wireless card, have a local build of whatever you want to test as a backup. At conferences, a good rule of thumb is: The Internet will go down.
4. Recruit
Getting potential testers to commit to an interview before the event can be difficult — they often don’t check the agenda or plan their own schedule until they arrive — so you’ll need to do most of your recruiting on the fly. When recruiting at the event, you’ll need to be a bit of an extrovert and talk to conference attendees throughout the day. Think of these conversations like a pre-interview that will help you figure out if they are the kind of customer you want to interview.
When you find someone to participate in your research, ask them to fill out a small form that asks for:
- Name
- Mobile number
- Email address
- What sessions they do not want to miss
- Screening questions to determine if they’re a good fit for the study
Open a Google form on your phone, fill out as many fields for the participant as possible and then ask them to list which sessions they do not want to miss. At Twiliocon, we recruited users in the morning during breakfast and lunch and ran our research sessions in the afternoon.
If your company is hosting the conference, it gets a lot easier to recruit. You can use marketing channels like email, Twitter, and Facebook to call for interested participants. I link to the same Google form I use at the event in the call-to-action button.
5. Send reminders
Once you’ve recruited users, follow up with an email about what time they’re scheduled and specifically where to meet you (conferences can be hard to navigate). Include your mobile phone number. (I give out a Twilio phone number that forwards to my cell phone to keep my personal number private.) I remind people to call, email, or text me if they cannot make the interview. Finally, include a calendar invitation for their exact session time.
6. Ask for critical feedback
To get critical feedback, we let participants know we’re testing parts of Twilio that people often struggle with or that we have found not to work well. We also emphasize our dedication to improving our products, and tell them the more they find wrong, broken, or confusing, the better.
7. Say thank you
You and your participants will be swimming in new contacts after the event, so follow up with a “thank you” afterward. We send along a gift card as an extra thank you — it’s a memorable way to let users know that their time and feedback were genuinely valuable.
Hopefully these tips will help you take advantage of events (like conferences and meetups) to easily recruit and interview potential users of your product. Have you done user research at events? Please share your experiences in the comments!
