Skip to content
1981
Experience Research Part 2
  • ISSN: 2055-2106
  • E-ISSN: 2055-2114

Abstract

Experience goals have been introduced to guide experience-driven design, but there are few tools to test whether those goals are met. This research introduces an evaluation technique that addresses the practical evaluation needs of experience-driven design projects in organizations. The XGoalTest technique was developed through thirteen design students’ experience-driven design course projects in two countries over twelve years. This article reports a literature review and analysis of ten selected course projects. A literature review on UX goal evaluation revealed a research gap in evaluation methods targeted for testing if the design outcomes match the UX goals. The student projects revealed seven important requirements for such an evaluation technique. After two decades of research on how to determine what experience to aim for and to design something that is expected to evoke that experience, this article proposes an instrument to evaluate whether that experience is actually realized. While numerous methods exist for evaluating user experience, the XGoalTest technique offers a novel, structured approach to assess whether design outcomes meet a specific experiential goal.

Funding
This study was supported by the:
  • Business Finland (Award 10586/31/2023)
Loading

Article metrics loading...

/content/journals/10.1386/dbs_00072_1
2026-02-23
2026-04-17

Metrics

Loading full text...

Full text loading...

References

  1. Abdullah, U. N. N., Othman, N., I Romli, F. and Handroos, H. (2018), ‘Investigation on user experience goals for joystick interface design’, Journal of Mechanical Engineering, 15:1, pp. 13342, https://ir.uitm.edu.my/id/eprint/36300. Accessed 7 December 2025.
    [Google Scholar]
  2. Adinda, K. S. and Niwanputri, G. S. (2021), ‘Designing for financial literacy: How adolescent learning through a mobile application’, 2021 8th International Conference on Advanced Informatics: Concepts, Theory and Applications (ICAICTA), Bandung, Indonesia, 29–30 September, IEEE, pp. 16.
    [Google Scholar]
  3. Ahmed, E., Islam, A., Chowdhury, A. I., Rahman, M. M., Chowdhury, S. and Hosen, M. I. (2021), ‘Robo-friend: Can a social robot empathize with your feelings effectively?’, in C. R. Panigrahi, B. Pati, B. K. Pattanayak, S. Amic and K. C. Li (eds), Progress in Advanced Computing and Intelligent Engineering. Advances in Intelligent Systems and Computing, vol. 1299, Singapore: Springer, pp. 77788.
    [Google Scholar]
  4. Aromaa, S., Lappalainen, I., Kaasinen, E. and Öfversten, J. (2022), ‘A case study: Applying a design thinking process and user experience goals in developing solutions for a smart construction site’, in M. M. Soares, E. Rosenzweig and A. Marcus (eds), Design, User Experience, and Usability: Design Thinking and Practice in Contemporary and Emerging Technologies, HCII 2022, Lecture Notes in Computer Science, 16 June, vol. 13323, Cham: Springer, pp. 93105.
    [Google Scholar]
  5. Chowdhury, A., Ahtinen, A. and Kaipainen, K. (2020a), ‘“The superhero of the university” experience-driven design and field study of the university guidance robot’, Proceedings of the 23rd International Conference on Academic Mindtrek, Tampere, Finland, 29–30 January, New York: ACM, pp. 19.
    [Google Scholar]
  6. Chowdhury, A., Ahtinen, A., Pieters, R. and Vaananen, K. (2020b), ‘User experience goals for designing industrial human-cobot collaboration: A case study of Franka Panda robot’, Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society, Tampere, Finland, 29–30 January, New York: ACM, pp. 113.
    [Google Scholar]
  7. Desmet, P. M. (2012), ‘Faces of product pleasure: 25 positive emotions in human-product interactions’, International Journal of Design, 6:2, https://www.ijdesign.org/index.php/IJDesign/article/view/1190/459. Accessed 7 December 2025.
    [Google Scholar]
  8. Desmet, P. M. A. and Schifferstein, H. N. J. (eds), (2011), From Floating Wheelchairs to Mobile Car Parks: A Collection of 35 Experience-Driven Design Projects, Den Haag, NL: Eleven Publishers.
    [Google Scholar]
  9. Desmet, P. M., Hekkert, P. and Jacobs, J. J. (2000), ‘When a car makes you smile: Development and application of an instrument to measure product emotions’, in S. J. Hoch and R. J. Meyer (eds), Proceedings of the Association for Consumer Research annual meeting, Association for Consumer Research, pp. 1117.
    [Google Scholar]
  10. Diefenbach, S., Lenz, E. and Hassenzahl, M. (2013), ‘An interaction vocabulary: Describing the how of interaction’, CHI ‘13 Extended Abstracts on Human Factors in Computing Systems (CHI EA ‘13), New York: Association for Computing Machinery, pp. 60712.
    [Google Scholar]
  11. Fu, J. (2016), ‘Smart TV interface design and evaluation based on goal-oriented’, 2016 Sixth International Conference on Instrumentation & Measurement, Computer, Communication and Control (IMCCC), Harbin: IEEE, pp. 32730.
    [Google Scholar]
  12. Fokkinga, S. F. and Desmet, P. M. A. (2013), ‘Ten ways to design for disgust, sadness, and other enjoyments: A design approach to enrich product experiences with negative emotions’, International Journal of Design, 7:1, pp. 1936.
    [Google Scholar]
  13. Gegner, L. and Runonen, M. (2012), ‘For what it is worth: Anticipated experience evaluation’, Proceedings of the 8th International Conference on Design and Emotions, University of the Arts London, London, UK.
    [Google Scholar]
  14. Hartono, W. S. and Widyantoro, D. H. (2016), ‘Fisheye zoom and semantic zoom on citation network visualization’, 2016 International Conference on Data and Software Engineering (ICoDSE), Denpasar: IEEE, pp. 16.
    [Google Scholar]
  15. Hassenzahl, M. (2003), ‘The thing and I: Understanding the relationship between user and product’, in M. A. Blythe, K. Overbeeke, A. F. Monk and P. C. Wright (eds), Funology: Human-Computer Interaction Series, vol. 3, Dordrecht: Kluwer, pp. 3142.
    [Google Scholar]
  16. Hassenzahl, M. and Monk, A. (2010), ‘The inference of perceived usability from beauty’, Human-Computer Interaction, 25:3, pp. 23560, https://doi.org/10.1080/07370024.2010.500139.
    [Google Scholar]
  17. Hassenzahl, M., Burmester, M. and Koller, F. (2003), ‘AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität’, in G. Szwillus and J. Ziegler (eds), Mensch & Computer 2003: Berichte des German Chapter of the ACM, vol. 57, Wiesbaden: Vieweg+Teubner Verlag, pp.18796.
    [Google Scholar]
  18. Hassenzahl, M., Diefenbach, S. and Göritz, A. (2010), ‘Needs, affect, and interactive products: Facets of user experience’, Interacting with Computers, 22:5, pp. 35362, https://doi.org/10.1016/j.intcom.2010.04.002.
    [Google Scholar]
  19. Hekkert, P., Mostert, M. and Stompff, G. (2003), ‘Dancing with a machine: A case of experience-driven design’, Proceedings of the 2003 International Conference on Designing Pleasurable Products and Interfaces (DPPI ’03), New York: Association for Computing Machinery, pp. 11419.
    [Google Scholar]
  20. Imani, S., Widyani, Y. and Rusmawati, Y. (2021), ‘Designing interaction for zero-waste application using cooperative gamification’, 2021 International Conference on Data and Software Engineering (ICoDSE), Bandung: IEEE, pp. 16.
    [Google Scholar]
  21. Kaasinen, E., Roto, V., Hakulinen, J., Heimonen, T., Jokinen, J. P., Karvonen, H., Keskinen, T., Koskinen, H., Lu, Y., Saariluoma, P., Tokkonen, H. and Turunen, M. (2015), ‘Defining user experience goals to guide the design of industrial systems’, Behaviour & Information Technology, 34:10, pp. 97691, https://doi.org/10.1080/0144929X.2015.1035335.
    [Google Scholar]
  22. Karvonen, H., Koskinen, H., Tokkonen, H. and Hakulinen, J. (2014), ‘Evaluation of user experience goal fulfillment: Case remote operator station’, Virtual, Augmented and Mixed Reality: Applications of Virtual and Augmented Reality, Part II, Cham: Springer, pp. 36677.
    [Google Scholar]
  23. Kymäläinen, T., Perälä, P., Hakulinen, J., Heimonen, T., James, J. and Perä, J. (2015), ‘Evaluating a future remote control environment with an experience-driven science fiction prototype’, Proceedings of the 2015 International Conference on Intelligent Environments (IE ’15), IEEE Computer Society, pp. 8188.
    [Google Scholar]
  24. Kymäläinen, T., Kaasinen, E., Aikala, M., Hakulinen, J., Heimonen, T., Paunonen, H., Ruotsalainen, J., Lehtikunnas, L. and Mannonen, P. (2016), ‘Evaluating future automation work in process plants with an experience-driven science fiction prototype’, Proceedings of the 2016 12th International Conference on Intelligent Environments (IE ‘16), London: IEEE, pp. 5461.
    [Google Scholar]
  25. Laugwitz, B., Held, T. and Schrepp, M. (2008), ‘Construction and evaluation of a user experience questionnaire’, HCI and Usability for Education and Work: 4th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society, USAB 2008, Graz, Austria, 20–21 November, Springer Berlin Heidelberg, pp. 6376.
    [Google Scholar]
  26. Lewrick, M., Link, P. and Leifer, L. (2020), The Design Thinking Toolbox: A Guide to Mastering the Most Popular and Valuable Innovation Methods. Hoboken, NJ: John Wiley & Sons.
    [Google Scholar]
  27. Lu, Y. and Roto, V. (2014), ‘Towards meaning change: Experience goals driving design space expansion’, Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI ‘14), New York: Association for Computing Machinery, pp. 71726.
    [Google Scholar]
  28. Lucero, A. and Arrasvuori, J. (2010), ‘PLEX cards: A source of inspiration when designing for playfulness’, Proceedings of the 3rd International Conference on Fun and Games (Fun and Games ’10), New York: Association for Computing Machinery, pp. 2837.
    [Google Scholar]
  29. Lucero, A., Holopainen, J., Ollila, E., Suomela, R. and Karapanos, E. (2013), ‘The playful experiences (PLEX) framework as a guide for expert evaluation’, Proceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces (DPPI ’13), New York: Association for Computing Machinery, pp. 22130.
    [Google Scholar]
  30. O’Brien, H. L., Cairns, P. and Hall, M. (2018), ‘A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form’, International Journal of Human-Computer Studies, 112, pp. 2839, https://doi.org/10.1016/j.ijhcs.2018.01.004.
    [Google Scholar]
  31. Pine, B. J. and Gilmore, J. H. (1998), Welcome to the Experience Economy, Cambridge, MA: Harvard Business Review Press.
    [Google Scholar]
  32. Preece, J., Rogers, Y. and Sharp, H., (2019), Interaction Design: Beyond Human-Computer Interaction, Indianapolis, IN: Wiley.
    [Google Scholar]
  33. QS (2025), ‘QS world university rankings by subject 2024: Art & design’, https://www.topuniversities.com/university-subject-rankings/art-design. Accessed 10 April 2024.
  34. Rosén, J., Billing, E. and Lindblom, J. (2023), ‘Applying the social robot expectation gap evaluation framework’, in M. Kurosu and A. Hashizume (eds), Human-Computer Interaction, Lecture Notes in Computer Science, vol. 14013, Cham: Springer, pp. 16988.
    [Google Scholar]
  35. Roto, V., Kaasinen, E., Heimonen, T., Karvonen, H., Jokinen, J. P. P., Mannonen, P., Nousu, H., Hakulinen, J., Lu, Y., Saariluoma, P. O., Kymäläinen, T., Keskinen, T., Turunen, M. and Koskinen, H. M. K. (2017), ‘Utilizing experience goals in design of industrial systems’, Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CH ‘17), New York: Association for Computing Machinery, pp. 69937004.
    [Google Scholar]
  36. Roto, V., Bragge, J., Lu, Y. and Pacauskas, D. (2021), ‘Mapping experience research across disciplines: Who, where, when’, Quality and User Experience, 6:1, n.pag., https://doi.org/10.1007/s41233-021-00047-4.
    [Google Scholar]
  37. Ryan, R. M., Mims, V. and Koestner, R. (1983), ‘Relation of reward contingency and interpersonal context to intrinsic motivation: A review and test using cognitive evaluation theory’, Journal of Personality and Social Psychology, 45:4, pp. 73650, https://doi.org/10.1037//0022-3514.45.4.736.
    [Google Scholar]
  38. Sanders, E. B. N. and Dandavate, U. (1999), ‘Design for experiencing: New tools’, in C. J. Overbeeke and P. Hekkert (eds), Proceedings of the First International Conference on Design and Emotion, TU Delft, pp. 8792.
    [Google Scholar]
  39. Sheldon, K. M, Elliot, A. J, Kim, Y. and Kasser, T. (2001), ‘What is satisfying about satisfying events? Testing 10 candidate psychological needs’, Journal of Personality and Social Psychology, 80:2, pp. 32539, https://doi.org/10.1037/0022-3514.80.2.325.
    [Google Scholar]
  40. Sutrisno, A. T. and Yulianti, L. P., Harlili (2021), ‘Designing interaction of food allergy information application using user-centered design approach: Gojek case study’, Journal of Physics: Conference Series, 1803:1, n.pag., https://doi.org/10.1088/1742-6596/1803/1/012028.
    [Google Scholar]
  41. Watson, D., Clark, L. A. and Tellegen, A. (1988), ‘Development and validation of brief measures of positive and negative affect: The PANAS scales’, Journal of Personality and Social Psychology, 54:6, pp. 106370, https://doi.org/10.1037//0022-3514.54.6.1063.
    [Google Scholar]
/content/journals/10.1386/dbs_00072_1
Loading
/content/journals/10.1386/dbs_00072_1
Loading

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was a success
Invalid data
An error occurred
Approval was partially successful, following selected items could not be processed due to error
Please enter a valid_number test