Skip to content
1981
Volume 11, Issue 2
  • ISSN: 1752-7066
  • E-ISSN: 1752-7074

Abstract

Motion-capturing technology has been traditionally used in the field of sports for the analysis of athletes’ body movements. The application of this multimodal tool to the field of musical pedagogy, however, has yet to be widely explored. Historically, music teachers have been using abstract language such as , for example, to describe melodic phrases. These can function as mental signposts or cues to help a performer navigate through a particular piece especially during performances. Concepts such as – the stretching of time – for instance, are more challenging to describe in terms of concrete lines or shapes, however. Because the push and pull of is so subtle, it can sometimes be challenging to pinpoint and maximize its effectiveness. Thus, in addition to listening to the coach’s verbal explanations, it can be helpful to see the teacher’s gestures displayed simultaneously alongside their students’ gestures during music lessons. The software provides visual feedback in real time and can be played back in slow motion. This device functions much like a mirror, as the performers’ gestures reflect onto the screen in real time. At the same time, their teacher’s gestures can also be juxtaposed onto the screen as a reference. Details of the speed and the precision of the timing can be seen on the screen as well. More importantly, can be a useful source of feedback in the practice room where the teacher is not present; students can record precise gestures during their lessons and revisit what they had learned when they are alone. In this sense, students would not feel lost in the practice room during the week, and they could also hone their music analysis skills through the examination of their body movements. This study aims to catalyse the learning process and to revolutionize the traditional methods of daily music practice.

© 2018 Intellect Ltd
Loading

Article metrics loading...

/content/journals/10.1386/jmte.11.2.149_1
2018-09-01
2026-04-18

Metrics

Loading full text...

Full text loading...

References

  1. Akinaga, S.,, Miura, M.,, Emura, N., and Yanagida, M.. ( 2006;), ‘ Toward realizing automatic evaluation of playing scales on the piano. ’, in ICMPC, Bologna.
     [Google Scholar]
  2. Allsop, L., and Ackland, T.. ( 2010;), ‘ The prevalence of playing-related musculoskeletal disorders in relation to piano players’ playing technique and practising strategies. ’, Music Performance Research, 3:1, pp. 61–78.
     [Google Scholar]
  3. Anderson, L.. ( 1981;), ‘ This sporting life. ’, Tempo (New Series), 3:139, pp. 33–34.
     [Google Scholar]
  4. Arx, V. A. Von. ( 2014), Piano Lessons with Claudio Arrau: A Guide to His Philosophy and Technique, New York, NY:: Oxford University Press;.
     [Google Scholar]
  5. Barry, N. H.. ( 2007;), ‘ A qualitative study of applied music lessons and subsequent student practice sessions. ’, Contributions to Music Education, 34, pp. 51–65.
     [Google Scholar]
  6. Bott, J. N.,, Crowley, J. G., and LaViola Jr, J. J.. ( 2009;), ‘ Exploring 3D gestural interfaces for music creation in video games. ’, in Proceedings of the 4th International Conference on Foundations of Digital Games, April, Orlando, FL:: ACM;, pp. 18–25.
     [Google Scholar]
  7. Brown, W.. ( 2008), Menahem Pressler: Artistry in Piano Teaching, Bloomington, IN:: Indiana University Press;.
     [Google Scholar]
  8. Chaffin, R.,, Imreh, G., and Crawford, M.. ( 2005), Practicing Perfection: Memory and Piano Performance, London:: Psychology Press;.
     [Google Scholar]
  9. Clarke, E. F.. ( 1993;), ‘ Imitating and evaluating real and transformed musical performances. ’, Music Perception: An Interdisciplinary Journal, 10:3, pp. 317–41.
     [Google Scholar]
  10. Dammers, R. J.. ( 2009;), ‘ Utilizing internet-based video-conferencing for instrumental music lessons. ’, Update: Applications of Research in Music Education, 28:1.
     [Google Scholar]
  11. Davis, N.,, Popova, Y.,, Sysoev, I.,, Hsiao, C. P.,, Zhang, D., and Magerko, B.. ( 2014;), ‘ Building artistic computer colleagues with an enactive model of creativity. ’, in Proceedings of the Fifth International Conference on Computational Creativity: The International Association for Computational Creativity, pp. 38–45.
     [Google Scholar]
  12. Desain, P., and Honing, H.. ( 1992;), ‘ Time functions function best as functions of multiple times. ’, Computer Music Journal, 16:2, pp. 17–34.
     [Google Scholar]
  13. Duke, R. A., and Simmons, A. L.. ( 2006;), ‘ The nature of expertise: Narrative descriptions of 19 common elements observed in the lessons of three renowned artist-teachers. ’, Bulletin of the Council for Research in Music Education, 170, pp. 7–19.
     [Google Scholar]
  14. Ekin, A.,, Tekalp, A. M., and Mehrotra, R.. ( 2003;), ‘ Automatic soccer video analysis and summarization. ’, IEEE Transactions on Image Processing, 12:7, pp. 796–807.
     [Google Scholar]
  15. Ferrario, V. F.,, Turci, M.,, Lovecchio, N.,, Shirai, Y. F., and Sforza, C.. ( 2007;), ‘ Asymmetry of the active nonweightbearing foot and ankle range of motion for dorsiflexion-plantar flexion and its coupled movements in adults. ’, Clinical Anatomy, 20:7, pp. 834–42.
     [Google Scholar]
  16. Fortney, P. M.. ( 1992;), ‘ The effect of modeling and silent analysis on the performance effectiveness of advanced elementar, instrumentalists. ’, Research Perspectives in Music Education, 3:1, pp. 18–21.
     [Google Scholar]
  17. Furuya, S., and Kinoshita, H.. ( 2008;), ‘ Expertise-dependent modulation of muscular and non-muscular torques in multi-joint arm movements during piano keystroke. ’, Neuroscience, 156:2, pp. 390–402.
     [Google Scholar]
  18. Gabrielsson, A., and Bradbury, R.. ( 2011), Strong Experiences with Music: Music Is Much More Than Just Music, Oxford:: Oxford University Press;.
     [Google Scholar]
  19. GECO MusicTech, ‘ GECO documentation. ’, http://uwyn.com/geco/docs/. Accessed 7 November 2018 .
  20. Geringer, J. M., and Kostka, M. J.. ( 1984;), ‘ An analysis of practice room behavior of college music students. ’, Contributions to Music Education, 11, pp. 24–27.
     [Google Scholar]
  21. Gil-Gómez, J. A.,, Lloréns, R.,, Alcañiz, M., and Colomer, C.. ( 2011;), ‘ Effectiveness of a Wii balance board-based system (eBaViR) for balance rehabilitation: A pilot randomized clinical trial in patients with acquired brain injury. ’, Journal of Neuroengineering and Rehabilitation, 8:1, p. 30.
     [Google Scholar]
  22. Goebl, W., and Palmer, C.. ( 2008;), ‘ Tactile feedback and timing accuracy in piano performance. ’, Experimental Brain Research, 186:3, pp. 471–79.
     [Google Scholar]
  23. Gorodnichy, D. O., and Yogeswaran, A.. ( 2006;), ‘ Detection and tracking of pianist hands and fingers. ’, in Computer and Robot Vision. The 3rd Canadian Conference, IEEE, Quebec, Canada, 19 June, p. 63.
     [Google Scholar]
  24. Grosshauser, T., and Tröster, G.. ( 2013;), ‘ Finger position and pressure sensing technique for string and keyboard instruments. ’, in NIME 13, Daejeon, Korea:: KAIST;, pp. 27–30.
     [Google Scholar]
  25. Hadjakos, A.,, Aitenbichler, E., and Mühlhäuser, M.. ( 2008;), ‘ Potential use of inertial measurement sensors for piano teaching systems: Motion analysis of piano playing patterns. ’, in Proceedings of the 4th i-Maestro Workshop on Technology-Enhanced Music Education, Darmstadt, Germany, pp. 61–68.
     [Google Scholar]
  26. Hamann, D. L.,, Lucas, K. V.,, McAllister, P., and Teachout, D.. ( 1998;), ‘ An investigation into the factors contributing to individual practice. ’, Journal of Band Research, 34:1, p. 59.
     [Google Scholar]
  27. Hewitt, M. P.. ( 2001;), ‘ The effects of modeling, self-evaluation, and self-listening on junior high instrumentalists’ music performance and practice attitude. ’, Journal of Research in Music Education, 49:4, pp. 307–22.
     [Google Scholar]
  28. Ho, M. R.,, Smyth, T. N.,, Kam, M., and Dearden, A.. ( 2009;), ‘ Human-computer interaction for development: The past, present, and future. ’, Information Technologies & International Development, 5:4, p. 1.
     [Google Scholar]
  29. Hutchinson, L.. (n.d), ‘ Leap motion controller. ’, https://arstechnica.com/gadgets/2013/07/hands-on-with-the-leap-motion-controller-cool-but-frustrating-as-hell. Accessed 7 November 2018 .
  30. Huron, D. B.. ( 2006;), Sweet Anticipation: Music and the Psychology of Expectation. , Cambridge, MA:, Empirical Musicology Review, 2:2.
     [Google Scholar]
  31. Jackson, R.. ( 2013), Performance Practice: A Dictionary-Guide for Musicians, New York:: Routledge;.
     [Google Scholar]
  32. Kenny, D.. ( 2011), The Psychology of Music Performance Anxiety, Oxford, UK:: Oxford University Press;.
     [Google Scholar]
  33. Kessler, G. D.,, Hodges, L. F., and Walker, N.. ( 1995;), ‘ Evaluation of the CyberGlove as a whole-hand input device. ’, ACM Transactions on Computer-Human Interaction (TOCHI), 2:4, pp. 263–83.
     [Google Scholar]
  34. Kostka, M. J.. ( 2002;), ‘ Practice expectations and attitudes: A survey of college-level music teachers and students. ’, Journal of Research in Music Education, 50:2, pp. 145–54.
     [Google Scholar]
  35. Kruse, N. B.,, Harlos, S. C.,, Callahan, R. M., and Herring, M. L.. ( 2013;), ‘ Skype music lessons in the academy: Intersections of music education, applied music and technology. ’, Journal of Music, Technology & Education, 6:1, pp. 43–60.
     [Google Scholar]
  36. Leapmotion Developer, ‘ Leap visualizer. ’, http://users.itk.ppke.hu/~horan/LeapImageJ/LeapSDK/docs/csharp/supplements/Leap_Visualizer.html. Accessed 7 November 2018 .
  37. Lee, J. C.. ( 2008;), ‘ Hacking the nintendo wii remote. ’, IEEE Pervasive Computing, 7:3, pp. 39–45.
     [Google Scholar]
  38. Lehmann, A. C.. ( 1997;), ‘ The acquisition of expertise in music: Efficiency of deliberate practice as a moderating variable in accounting for sub-expert performance. ’, Perception and Cognition of Music, pp. 161–87.
     [Google Scholar]
  39. Lehmann, A. C., and Ericsson, K. A.. ( 1997;), ‘ Research on expert performance and deliberate practice: Implications for the education of amateur musicians and music students. ’, Psychomusicology: A Journal of Research in Music Cognition, 16:1&2, p. 40.
     [Google Scholar]
  40. Leimer, K., and Gieseking, W.. ( 1972), Piano Technique Consisting of the Two Complete Books the Shortest Way to Pianistic Perfection and Rhythmics, Dynamics, Pedal and Other Problems of Piano Playing, New York:: Courier Corporation, Dover Publications;.
     [Google Scholar]
  41. Madsen, C. K., and Geringer, J. M.. ( 1981;), ‘ The effect of a distraction index on improving practice attentiveness and musical performance. ’, Bulletin of the Council for Research in Music Education, 66/67, pp. 46–52.
     [Google Scholar]
  42. Maris, B. E.. ( 2000), Making Music at the Piano: Learning Strategies for Adult Students, New York, NY:: Oxford University Press on Demand;.
     [Google Scholar]
  43. Mark, T. C.,, Gary, R., and Miles, T.. ( 2003), What Every Pianist Needs to Know about the Body: A Manual for Players of Keyboard Instruments: Piano, Organ, Digital Keyboard, Harpsichord, Clavichord, Chicago, IL:: GIA Publications;.
     [Google Scholar]
  44. Maynard, L. M.. ( 2006;), ‘ The role of repetition in the practice sessions of artist teachers and their students. ’, Bulletin of the Council for Research in Music Education, pp. 61–72.
     [Google Scholar]
  45. Mazzola, G., and Zahorka, O.. ( 1994a;), ‘ The RUBATO performance workstation on NEXTSTEP. ’, in ICMC, Aarhus.
     [Google Scholar]
  46. Mazzola, G., and Zahorka, O.. ( 1994b;), ‘ Tempo curves revisited: Hierarchies of performance fields. ’, Computer Music Journal, 18:1, pp. 40–52.
     [Google Scholar]
  47. Menache, A.. ( 2000), Understanding Motion Capture for Computer Animation and Video Games, San Diego, CA:: Morgan Kaufmann;.
     [Google Scholar]
  48. Mikilabo (n.d), ‘ Controlling Arduino car via Leap Motion. ’, http://www.instructables.com/id/Controlling-Arduino-car-via-Leap-Motion. Accessed 7 November 2018.
  49. Miksza, P.. ( 2007;), ‘ Effective practice an investigation of observed practice behaviors, self-reported practice habits, and the performance achievement of high school wind players. ’, Journal of Research in Music Education, 55:4, pp. 359–75.
     [Google Scholar]
  50. Motion Controller, ‘ Motion controller. ’, https://www.pubnub.com/blog/2015-08-19-motion-controlled-servos-with-leap-motion-raspberry-pi. Accessed 7 November 2018 .
  51. Ng, K. C.,, Weyde, T.,, Larkin, O.,, Neubarth, K.,, Koerselman, T., and Ong, B.. ( 2007;), ‘ 3D augmented mirror: A multimodal interface for string instrument learning and teaching with gesture support. ’, in Proceedings of the 9th International Conference on Multimodal Interfaces, 12–15 November, Nagoya, Aichi, Japan:: ACM;, pp. 339–45.
     [Google Scholar]
  52. Nielsen, S.. ( 2001;), ‘ Self-regulating learning strategies in instrumental music practice. ’, Music Education Research, 3:2, pp. 155–67.
     [Google Scholar]
  53. Piek, J. P.,, Dawson, L.,, Smith, L. M., and Gasson, N.. ( 2008;), ‘ The role of early fine and gross motor development on later motor and cognitive ability. ’, Human Movement Science, 27:5, pp. 668–81.
     [Google Scholar]
  54. Pingali, G.,, Opalach, A., and Jean, Y.. ( 2000;), ‘ Ball tracking and virtual replays for innovative tennis broadcasts. ’, Proceedings of ICPR’00, Barcelona;, pp. 4152–46.
     [Google Scholar]
  55. PSD graphics ( 2011;), ‘ Blank projection screen. ’, http://www.psdgraphics.com/psd/blank-projection-screen-psd. Accessed 7 November 2018.
  56. Rink, J.. (ed.) ( 2005), The Practice of Performance: Studies in Musical Interpretation, Cambridge:: Cambridge University Press;.
     [Google Scholar]
  57. Rosenblum, S. P.. ( 1988), Performance Practices in Classic Piano Music: Their Principles and Applications, vol. 680, Bloomington, IN:: Indiana University Press;.
     [Google Scholar]
  58. Santiago, P. F.. ( 2004;), ‘ An exploration of the potential contributions of the Alexander technique to piano pedagogy. ’, doctoral dissertation, London:: Institute of Education, University of London;.
     [Google Scholar]
  59. Schlömer, T.,, Poppinga, B.,, Henze, N., and Boll, S.. ( 2008;), ‘ Gesture recognition with a Wii controller. ’, in Proceedings of the 2nd International Conference on Tangible and Embedded Interaction, 18–20 February, Bonn, Germany:: ACM;, pp. 11–14.
     [Google Scholar]
  60. Schön, D. A.. ( 1987), Educating the Reflective Practitioner: Toward a New Design for Teaching and Learning in the Professions, San Francisco, CA:: Jossey-Bass Publishers;.
     [Google Scholar]
  61. Shaffer, H.. ( 1984;), ‘ Timing in musical performance. ’, Annals of the New York Academy of Sciences, 423:1, pp. 420–28.
     [Google Scholar]
  62. Su, Y. H.. ( 2010;), ‘ Effects of using relaxation breathing training to reduce music performance anxiety in 3rd to 6th graders. ’, Medical Problems of Performing Artists, 25:2, pp. 82–86.
     [Google Scholar]
  63. Taubman, D.. ( 1984;), ‘ A teacher’s perspective on musicians’ injuries. ’, in The Biology of Music Making: Proceedings of the 1984 Denver Conference. St. Louis, MMB Music, Missouri:, p. 144.
     [Google Scholar]
  64. Upright Piano, ‘ Upright Piano. ’, https://www.123rf.com/clipart-vector/upright_piano.html. Accessed 7 November 2018 .
  65. Visi, F.,, Schramm, R., and Miranda, E.. ( 2014;), ‘ Gesture in performance with traditional musical instruments and electronics: Use of embodied music cognition and multimodal motion capture to design gestural mapping strategies. ’, in Proceedings of the 2014 International Workshop on Movement and Computing, 16–17 June, Paris, France:: ACM;, p. 100.
     [Google Scholar]
  66. Wang, J. R., and Parameswaran, N.. ( 2004;), ‘ Survey of sports video analysis: Research issues and applications. ’, in Proceedings of the Pan-Sydney Area Workshop on Visual Information Processing, June, Sydney, Australia:, Australian Computer Society, Inc.;, pp. 87–90.
     [Google Scholar]
  67. Wong, E. L.,, Yuen, W. Y., and Choy, C. S.. ( 2008;), ‘ Designing wii controller: A powerful musical instrument in an interactive music performance system. ’, in Proceedings of the 6th International Conference on Advances in Mobile Computing and Multimedia, 24–26 November, Linz, Austria:: ACM;, pp. 82–87.
     [Google Scholar]
  68. Xiao, X.,, Aguilera, P.,, Williams, J., and Ishii, H.. ( 2013;), ‘ MirrorFugue iii: Conjuring the recorded pianist. ’, in CHI Extended Abstracts, April, Cambridge, MA:, pp. 2891–92.
     [Google Scholar]
  69. Xiao, X., and Ishii, H.. ( 2011;), ‘ MirrorFugue: Communicating hand gesture in remote piano collaboration. ’, in Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction, 22–26 January, Funchal, Madeira, Portugal:: ACM;, pp. 13–20.
     [Google Scholar]
  70. Xiao, X.,, Tome, B., and Ishii, H.. ( 2014;), ‘ Andante: Walking figures on the piano keyboard to visualize musical motion. ’, in NIME, Cambridge, MA:, pp. 629–32.
     [Google Scholar]
  71. Yin, J.,, Wang, Y., and Hsu, D.. ( 2005;), ‘ Digital violin tutor: An integrated system for beginning violin learners. ’, in Proceedings of the 13th annual ACM International Conference on Multimedia, November, Singapore, Singapore:: ACM;, pp. 976–85.
     [Google Scholar]
  72. Yin, R. K.. ( 2003), Case Study Research: Design and Methods, vol. 5, Thousand Oaks, CA:: Sage Publications;.
     [Google Scholar]
  73. Zakaria, J. B.,, Musib, H. B., and Shariff, S. M.. ( 2013;), ‘ Overcoming performance anxiety among music undergraduates. ’, Procedia-Social and Behavioral Sciences, 90, pp. 226–34.
     [Google Scholar]
  74. Cheng, M.. ( 2018;), ‘ Introducing motion-capturing technology into the music practice room as a feedback tool for working towards the precision of rubato. ’, Journal of Music, Technology & Education, 11:2, pp. 149–170, doi: 10.1386/jmte.11.2.149_1
     [Google Scholar]
/content/journals/10.1386/jmte.11.2.149_1
Loading
Introducing motion-capturing technology into the music practice room as a feedback tool for working towards the precision of rubato
Journal of Music, Technology & Education 11, 149 (2018); https://doi.org/10.1386/jmte.11.2.149_1
/content/journals/10.1386/jmte.11.2.149_1
/content/journals/10.1386/jmte.11.2.149_1
Loading

Data & Media loading...

  • Article Type: Article
Keyword(s): gesture; Leapmotion; motion-capture; music; performance; technology

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a success
Invalid data
An error occurred
Approval was partially successful, following selected items could not be processed due to error
Please enter a valid_number test