A quantitative experiment setup for testing the effect of desirable difficulties on teaching robotics


  • Achim Gerstenberg Center for Interdisciplinary Research in Space, NTNU Samfunnsforskning, 7491 Trondheim, Norway
  • Martin Steinert Department of Mechanical and Industrial Engineering, NTNU, 7491 Trondheim, Norway




Desirable difficulty; Human subject experiment; Problem solving


Desirable difficulties such as generating one’s own solution instead of replicating a provided solution is associated with improved long-term memory. Disseminating misleading information has shown improved learning in science education over consuming concise and clear learning instructions. We describe an experimental setup aimed at quantifying if a tutorial about programming a mobile autonomous robot that requires having to correct misleading instructions leads to better problem-solving capabilities than providing correct and clear tutorial instructions when asked to solve a complicated open-ended robotics task. The presented experimental setup is aimed towards a controlled comparative human-subject study that compares the effect of desirable difficulties on participant’s performance in solving a complicated open-ended task after completing an introductory tutorial. We explain the experiment timeline, the tasks of the tutorial, as well as the open-ended task including the robot and how this experiment can be executed under very controlled, repeatable and as unbiased as possible conditions. We share and qualitatively discuss some observed problems in this setup from early trials with 8 participants.


Bjork, R.A., 1975, Retrieval as a memory modifier: An interpretation of negative recency and related phenomena. Information processing and cognition, Loyola Symposium, 123 – 144.

Bjork, R.A., 1994, Memory and metamemory considerations in the training of human beings, Metacognition: Knowing about knowing, 185 – 205.

Bjork, R.A., 2011, Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning, Psychology and the real world: Essays illustrating fundamental contributions to society, Worth Publishers, New York, NY, US.

Gerstenberg, A. & Steinert, M., 2018, Open ended problems a robot programming experiment design to compare and test different development and design approaches. Proceedings of NordDesign 2018.

Gerstenberg, A. & Steinert, M., 2019, The Relevance of Testing in Engineering Product Design – Investigations on a Robot Programming Task. Procedia CIRP 84, 586-592.

Muller, D.A., 2007, Saying the wrong thing: improving learning with multimedia by including misconceptions, Journal of Computer Assisted Learning, 24(2), 144 – 155.

Roediger, H.L. & Karpicke, J.D., 2006, Test-enhanced learning: Taking memory tests improves long-term retention, Psychological Science, 17, 249–255.




How to Cite

Gerstenberg, A., & Steinert, M. (2021). A quantitative experiment setup for testing the effect of desirable difficulties on teaching robotics. CERN IdeaSquare Journal of Experimental Innovation, 5(2), 56–60. https://doi.org/10.23726/cij.2021.1077



Original Articles