# Software as a service for user testing

**URL:** <https://discourse.opensourcedesign.net/t/software-as-a-service-for-user-testing/199>\
**Category:** Usability reviews and research\
**Created:** [June 4, 2017, 2:58pm UTC](https://discourse.opensourcedesign.net/t/software-as-a-service-for-user-testing/199 "2017-06-04T14:58:40Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![pdurbin](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.opensourcedesign.net/pdurbin/32/3478_2.png) [@pdurbin](https://discourse.opensourcedesign.net/u/pdurbin)\
**Post date:** [June 4, 2017, 2:58pm UTC](https://discourse.opensourcedesign.net/t/software-as-a-service-for-user-testing/199/1 "2017-06-04T14:58:40Z")

</div>

This morning in the #[positivepython](https://typy.io/ca9d7d4ae804c85501c9426222e489a3dca53021dbb1c259fea3acd0ce817383) channel on [freenode](https://webchat.freenode.net/?channels=positivepython) I heard the following desire for infrastructure for doing using testing on open source projects:

> I wish we had a better set of infrastructure for doing f/oss usertesting, seems like most problems in like this (e.g. Debian’s website, Helios, etc), they live in the developer’s blind spot, since they’re not the user and they’re focused on a feature-set that’s interesting to them  
> …  
> like travis, but for user-testing

I just tried a couple searches for “user testing saas” and “user testing software as a service” and didn’t find anything I’d heard of. I’m much more familiar with real world, in-person user testing efforts such as [SpinachCon](https://groups.google.com/d/msg/dataverse-community/G6cyCAv1hbY/iTI7_lCZBwAJ) or the [User Research Center at Harvard Library](https://urc.library.harvard.edu).

**Is there software as a service for user testing that open source projects can use?**

* * *

For more context on the quote above see [BotBot.me + Startup Resources | Startup Resources](https://botbot.me/freenode/positivepython/2017-06-04/) part of which is captured below:

> nedbat: i just did my PSF ballot, and I remain impressed by the over-engineering of the encrypted ballot, and the complete lack of awareness of usability issues.
> 
> sigmavirus24: nedbat: Would you care to email `vote@python.org` with your feedback? Mark and I will work with te Helios team to fix as much as we can
> 
> pdurbin: What are the usability issues?
> 
> sigmavirus24: pdurbin: well, I can think of a few myself
> 
> sigmavirus24: That said, on the whole, I find Helios to be leagues better than eVote
> 
> \*: pdurbin doesn’t believe he’s used either
> 
> nedbat: sigmavirus24: i might send an email, but i feel like i’ve said this in previous years: the election software seems entirely focused on assuring me about the encryption of the results (which I frankly don’t care about), and not concerned much with helping me understand what I am voting for.
> 
> paultag: I wish we had a better set of infrastructure for doing f/oss usertesting, seems like most problems in like this (e.g. Debian’s website, Helios, etc), they live in the developer’s blind spot, since they’re not the user and they’re focused on a feature-set that’s interesting to them
> 
> paultag: something’s perfectly usable if you’re one of the world’s leading experts in that feature-set
> 
> paultag: (I say this, as guilty of the same exact thing)
> 
> pdurbin: paultag: have you seen [https://discourse.opensourcedesign.net](https://discourse.opensourcedesign.net) ? That group talks about user testing and such.
> 
> paultag: I mean, infrastructure that will take a f/oss project and user-test it, or set it up for user-testing (e.g. recorded user sessions) in a lightweight and easy way
> 
> paultag: like travis, but for user-testing
> 
> paultag: I’m sure people talk all day about it
> 
> paultag: but it’s usless unless you put it into practice, and use the issues and feedback created from it to generate usablity issues

---

<div class="post-metadata">

**Author:** ![jdittrich](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.opensourcedesign.net/jdittrich/32/42_2.png) [@jdittrich](https://discourse.opensourcedesign.net/u/jdittrich)\
**Post date:** [June 4, 2017, 5:44pm UTC](https://discourse.opensourcedesign.net/t/software-as-a-service-for-user-testing/199/2 "2017-06-04T17:44:15Z")

</div>

Do you mean something like [usertesting.com](https://www.usertesting.com/), but using an opensource infrastructure?

I assume one reason that you might not have been able to find the SaaS is that it is usually a service. The software is only a part of it and does usually not make much sense itself – the most interesting feature is probably that the services also distribute users with suitable attributes to the tests.

So, such a software would need to have

- Screen recording and audio recording
- Annotations with instructions
- simple feedback form or so

But also:

- A data base of users
- Some mechanism to draw a sample of users with suitable attributes (Age, experience, whatever)
- Connection to the actual testing part above.

---

<div class="post-metadata">

**Author:** ![pdurbin](https://yyz1.discourse-cdn.com/flex029/user_avatar/discourse.opensourcedesign.net/pdurbin/32/3478_2.png) [@pdurbin](https://discourse.opensourcedesign.net/u/pdurbin)\
**Post date:** [June 4, 2017, 11:52pm UTC](https://discourse.opensourcedesign.net/t/software-as-a-service-for-user-testing/199/3 "2017-06-04T23:52:22Z")

</div>

@jdittrich good question. A quick answer:

> pdurbin: paultag: Do you mean something like [usertesting.com](http://usertesting.com), but using an opensource infrastructure?
> 
> paultag: pdurbin: sorta, there are tons of usertesting platforms, but, I was trying to explain something as easy as travis to put into a repo to enable running usertesting, including payment for people’s time and find a way to test new features before they get merged, as well as periodic retests of workflows. So, if you want to look at something like [usertesting.com](http://usertesting.com), something for before and after to help
> 
> paultag: mediate interactions
> 
> paultag: so have people express they’d be willing to test stuff, and a developer (who hopefully knows how to moderate) can jump on, session is recorded, and look across sessions for patterns
> 
> paultag: but like, hosting the test version, etc
> 
> paultag: anyway, whatever, just a thought

@jdittrich please do feel free to hop in #positivepython to chat about this more. I guess I’m just the messenger.