Coming Up with a Product

I was talking to a friend about common frustration in our immediate environments about product development life cycles, and how you communicate to stakeholders what a successful design cycle looks like for a product.

She was toying with this concept of increasing fidelity as you design a product:

You start with Goals, dive further down into Concepts, and then make your way to User Stories.

I think this holds pretty true for standard design R&D, where you’re often working on a specific client’s needs. But what about designing for open source, where a project is often someone’s side project that they started because they thought that they needed it? Or they thought that an existing product wasn’t good enough and they took it upon themselves to fix it or make it better.

Does open source have a method of R&D that’s different from traditional proprietary work, just because of it’s intrinsic nature? Does it make as much sense to step through concepting, goal definition, etc when starting a product? Do you want to encourage that behavior for people starting their own projects? Is that even desirable?

2 Likes

tl;dr yes, I think so.

Re: @jdittrich’s recent topic and diagrams such as the one at https://jdittrich.github.io/userNeedResearchBook/#toc57, any number of things might be done to prepare stakeholders (including yourself) for the inevitability of troughs in emotion.

Some such troughs can result when there’s variance from an agreed timeline. Defocusing from product design …

[spoiler]When managing e.g. mini projects over which I have a reasonable degree of control, I know from experience to not trust my own ‘gut’ estimates about time and resource scheduling. I take my somewhat dyslexic estimate, multiply it by at least four before offering a timescale to anyone who will be involved.

Non-managed projects are a quite different kettle of fish. [/spoiler]

It’s always good to define goals, I should be cautious of predefining too much about the product.


#thinkbubbles computer-free; two mind maps per person, one of which is a third person interpretation of the first person’s spoken mind; rigidity of Gantt; bubbles from the maps on to the lines of the Gantt; multiperson scrunch of the combined bubbles + lines, to tell how it feels; stretch, tear, repair until the remaining scraps become cohesive in a way that’s mutually agreeable.

I would assume that is the standard story to be told afterwards, but I think most dev cycles are different: People will have an idea that they can use X also for Y, or that everybody now needs an Z or… At least in design the process is rather messy and I assume this holds for non design Research and Development (R&D), too (an old, but great overview is “Technology and Change” by D. Schön)

Hypothesis: Design in opensource needs more reflection and finding a common direction.

In the non-open source environment this may be created by two (and more) sources:

  • People from marketing, customer service, user research… exchanging with the people who decide about new products leading ideally to different input and at least to having a common goal and direction (OK, that can also be forced onto people, but I assume this is not great and leading to innovation)
  • The need to succeed on the marked creates incentives to be useful and usable (and also incentives to just give that impression, but thats another story) leading to the question of “are we useful and usable”

scratching-your-own-itch development, in contrast could have these assumed outcomes:

  • No strong incentives to be useful and usable beyond the particular use case of the initiator
  • The possible community around the product will self select and thus be more or less fine with the use of the product.

So maybe there is not a lot of constructive asking of questions, following reflection and deliberately looking beyond a predefined set of parameters.

If that theory makes sense, one could think about possible measures that may help to gain these without loosing the advantages of the opensource model:

  • Frame new ideas in stories, (instead of “let’s have a button there” for x)
  • Bug reports and forum posts can tell a lot about usability and usefulness (instead of being targets for turning down with “you did it wrong”)
  • Strengthen the entanglement of discussions and possible implementations (instead of seeing talking as a distraction from actually building something)
2 Likes

With open source you can put something out there that’s good enough for your own needs and invite others to make their own improvements, building on your work rather than starting from scratch. Some amount of polish is nice I favor getting working code into people’s hands sooner rather than later so that it can grow and evolve. Your goals you can write up in your README after you’re done writing code. I guess I’m talking about fairly small projects. Maybe the amount of up front design that’s necessary depends greatly on how many lines of code we’re talking about rather than if the product is open source or not. For me open source is all about standing on the shoulders of giants and enabling others to take your ideas beyond your imagination.

2 Likes

I find discussions that don’t focus on creation not just a distraction but destructive to the process. When your only incentive for creation is motivation and enthusiasm about it - slapping that enthusiasm down is (and in my experience has been) the death knell of any project.

It tends to, though, create a community of debaters who really enjoy a good debate. And then the project dies or is kept up by either a loyal following trying to follow the directions in the debate and create something, anything, out of it or by people ignoring the debate and just doing something.

It is something that is often forgotten I feel: that the product at the end of the line, for many, is not as relevant as the process getting there and the process dictates the product. If the process for example is defined and pushed forward by “you will all be fired unless this is done” the product will be something that the person who can fire people will enjoy.

2 Likes

To connect it back to the initial question – Do I understand it right that: you say that, yes, it is different (or am I wrong here?)

Does that mean that a discussion on process/modes of collaboration is not useful in your eyes?

No of course it’s useful - but at a certain point. But motivation is the end all, be all of the project if the majority of the work is done by community participants.

Interesting topic :slight_smile:

One of my pet peeves in FOSS is the belief that, because a project started by scratching an itch, the maintainer(s) “always know” which new features make sense. The reality is that, the moment you release your “itch scratching” to the world and user number 1 turns up, you are no longer the user, and you know pretty much nothing :slight_smile: Therefore, there is very much a need for formative design research, as with any commercial product.

This is an easy one: how do you know what “better” is? You need to find out! What’s better for you is probably not “better” for the bulk of the project’s users. Or maybe it is, but you cannot be sure until you validate your ideas.

I think they are more nuances than fundamental differences.

One of them is the fact that part of your user base is bound to also contribute to the product. We must watch for “maintainer bias”, but you cannot disregard the maintainers’ perspective on the basis that they are contributing and that automatically makes them “developers” instead of “users”. Sometimes they are both. I’ve found the way you engage with these users is somehow different to how you engage with users who do not contribute to the project. The usual participatory design and research techniques must be tweaked to accommodate their dual role.

Another one is derived from the high number of communication channels and high volumes of communication in FOSS projects, all of which are completely open and accessible. For FOSS, it is possible to gather tons of insights about users’s goals, needs and context just by combing mailing lists, forums, bug reports and IRC channels. Very often there is no such thing in commercial products.

You also have conferences (you often don’t in commercial settings), where at least part of your user base gathers. You can do a lot of ethnography-like research when attending those, and also carry out short, focused validation studies with users you grab at sessions, booths, or while having beers!

Finally, because of the openness, and because you often have a vocal and technologically inclined user community, the “put something basic out there and check out what happens” becomes a particularly productive approach. You very often don’t have the freedom to do something like that in commercial settings.

It would be really interesting to produce a list of “FOSS particularities” that impact the way you plan and execute formative and summative research + design iterations.

Yes! And most of the time you do somehow, even if it’s not formalised or structured, and you don’t directly involve users in the process.

Yes! And it would be really interesting to come up with a lightweight approach that could be applied to very small, scratch your own itch type of things.

3 Likes

:-)

Some messiness should be welcomed. Kicking the ball around, for want of a better expression … a playful and productive mess.

Whether it will be appropriate to mention anything like a mess, in those communications, will depend on the stakeholders.

[quote=“jdittrich, post:3, topic:151”]…

  • Bug reports and forum posts can tell a lot about usability and usefulness (instead of being targets for turning down with “you did it wrong”)
  • Strengthen the entanglement of discussions and possible implementations (instead of seeing talking as a distraction from actually building something)[/quote]

:thumbsup:

“It doesn’t work”

From last week’s Open Whisper Systems >> Blog >> Reflections: The ecosystem is moving:

… consistency and control right now. They introduced issue templates, but a number of 3rd party GitHub clients don’t support them, so even after creating a thorough issue template for the Signal Android repository, we still get people who post “it doesn’t work please help,” because their client never even showed them the template. That makes me annoyed with GitHub, … potential opportunity for a GitHub competitor that can display issue templates consistently. …

Whilst I empathise with developers’ wishes for consistency in reporting, it is – I believe – equally, if not more, important to allow people to write in their own words. Funneling thoughts into a template can disrupt the train of thought.

Enthusiasm

There’s always a way to tease more out of a person. No need to spend too much time teasing out the information, but sometimes a “doesn’t work” report will become a wonderfully useful report.

Consider:

GitHub aside: any pedantic attitude towards feedback can destroy enthusiasm.

Less often a death knell for a project; more often a quiet killing of personal goodwill.

Design cycle: SysAdm™

SysAdm is probably rolling release, so this may be not a good example, but here goes …

A screenshot of an early version of SysAdm Client:

https://bugs.pcbsd.org/attachments/download/6012/example.png

I foresaw the rotated texts, to the left, becoming a nightmare so I offered early feedback. I can’t find that feedback (it might have been in IRC) but part of it involved a comparison with an early Internet Explorer for Mac, where rotation – in the explorer bar – did work well:

– worked well, because there were only five things in the bar, four of which required almost no computer literacy, each one had a predictable position, and so on.

Would not work well for SysAdm because … I offered a shortlist of reasons. A rational conversation between me and the developer. Some weeks or months later, http://testing.pcbsd.narkive.com/kMwzHiht/pc-bsd-testing-sysadm-texts-rotated-bottom-to-top-instead-of-left-to-right-ltr the second post from me was unusually blunt (with apologies for the bluntness) only because there had been the parallel, deeper conversation through which I was reasonably confident that the bluntness via e-mail to the list would be not too offensive to the developer. Despite that, the overall effect of the thread was probably an emotional trough for the developer, and I felt rotten about triggering that trough (about maybe dampening his enthusiasm), but the exchange of opinions did lead to a better product. Now comparable to what’s pictured at https://sysadm.us … plus e.g. https://github.com/trueos/trueos-core/issues/164 leading to a good overview when the client is opened:


The screenshot of Internet Explorer is from Browsing the Web with Internet Explorer for Mac OS X :: Chapter 13. Surfing the Web :: Part II: Mac OS X: Connecting to the World :: Using MAC OS X v10.2 :: Mac OS :: eTutorials.org

Briefly: I’ve had a pretty negative experience with Signal and how they handle issues.

Yeah this makes sense to me. But then also feels like it equates the design of open source as the design of open source “apps” rather than the design of documentation, libraries, events, communities, and graphics (for logos etc). Open source is a bit of a broader brush than proprietary I think. For a lot of these things, you and your company (because often companies are the producers of open source libraries and frameworks) might still be the primary users. In fact - that’s how a lot of frameworks find support and stay maintained (I’m thinking Ruby on Rails and Django, for example).

You and @jdittrich both mentioned this. Have you seen this done anywhere effectively? I saw someone else mention it on Twitter the other day:

I get this, but as a developer I’ve also often found that people’s own words are most often red herrings.

What do you think this would look like? Is it something we could put in our resources?

1 Like

It was not my intention to imply that, by any means. “You are not the user” applies to every single aspect of the interaction with a technology system (the original meaning of the expression “user experience”). And that includes all the things you mention above.

[quote=“simonv3, post:11, topic:151”]For a lot of these things, you and your company (because often companies are the producers of open source libraries and frameworks) might still be the primary users. In fact - that’s how a lot of frameworks find support and stay maintained (I’m thinking Ruby on Rails and Django, for example).
[/quote]

Probably in some cases, I guess at the very beginning. What I am trying to say is that the time span of the creator being the “primary” user of any mildly successful FOSS project is often much shorter than we think.

By now, neither Basecamp nor Lawrence Journal-World can be considered “primary” users of Ruby on Rails and Django respectively. In my experience, failure to appreciate this fact can stop FOSS projects from embracing user-centred design in a meaningful way.

I think we need to clarify what we mean by “effectively”. In design practice, what you are aiming for when looking at those communications is not academic rigor in their analysis, or proving/disproving a hypothesis, or developing theory. What you are aiming for is gathering accionable insights, nuggets of knowledge that can help you improve a piece of software, or identify new features. For that, rigorous analysis is often not necessary: the biggest problems are most of the time painfully obvious once you start looking. Repetition (they come up over and over again) makes them impossible to ignore.

In this sense, I used insights from communication channels rather effectively in Yocto Project and OpenEmbedded for the 5 years I worked with them. I talk a bit about it at

https://belenbarrospena.github.io/me/inspired_by_ethnography.html

The approach has limitations of course (it ignores users who do not participate in these channels, for example). Like any approach, by itself is not enough, but it can be used in combination with other activities.

Definitely, but whichever shape it takes, it should be a living document that we can edit continuously. In my head, we would start by putting together a list collaboratively. Once we have a few things on the list, we could create a document/page and develop each item a bit further, maybe with a simple structure (list item definition + project examples or something like that), also collaboratively. I suspect that we could learn a lot through the exercise of putting this together :slight_smile:

[quote=“simonv3, post:11, topic:151”]
You and @jdittrich both mentioned this. Have you seen this done anywhere effectively? [/quote]

I use it in my work and hope it to be effectively. But is is done to rarely and it is often not feeling as productive than starting to code right away (at least for me).

Which is why I rarely do surveys. But it is interesting: If I interpret is right, you posted this, because you assume that (user) research means people use surveys. If that is correct – what made you think so?

I’m not sure if I know of a great example of a project combing mailing lists, forums, bug reports and IRC channels for insights but once you’ve gathered those insights, where do you put them? WeWork calls them “nuggets” and puts them in AirTable: https://airtable.com/universe/expShuhNMi0Oc0xpb/polaris-ux-nuggets .

Slightly off topic, but the polaris, the system behind the collection of “Nuggets” was presented as a way of Democratizing UX and I asked Tomer it they consider opening polaris in the comments (Who did not reject the idea, but said that their focus at that time was to stabilize it)

I don’t know, but if anybody has any connection to WeWork it would be great to follow up.