On the impact of using different templates on creating and understanding user stories

TLDR: In this paper, the authors conducted a quasi-experiment to evaluate the impact of different templates in creating user stories and observed statistically significant differences in some of the metrics for accuracy, speed and visual effort.

Abstract: Context: User stories are often used for elicitation and prioritisation of requirements. However, the lack of a widely adopted user story template, covering benefit and the usage (or not) of a persona, can affect user stories’ quality, leading to ambiguity, lack of completeness, or accidental complexity. Objectives: Our goal was to analyse the differences between 4 alternative user story templates when creating and understanding user stories. Methods: We conducted a quasi-experiment. We asked 41 participants to perform creation and understanding tasks with the user story templates. We measured their accuracy, using metrics of task success; their speed, with task duration; visual effort, collected with an eye-tracker; and participants’ perceived effort, evaluated with NASA-TLX. Results: Regarding the impact of the different templates in creating user stories, we observed statistically significant differences in some of the metrics for accuracy, speed and visual effort. For understanding user stories, we observed small differences in terms of visual effort. Conclusions: Although some templates outperformed others in a few metrics, no template obtained the best overall result. As such, we found no compelling evidence that one template is "better" than the others.