Can I read Virtual Piano using Computer Vision on EtoBox?

Virtual Piano using Computer Vision by Kang, Seongjae; Kim, Jaeyoon; Yoon, Sung-eui is a scholarly article available to read on EtoBox.

What is Virtual Piano using Computer Vision about?

In this research, Piano performances have been analyzed only based on visual information. Computer vision algorithms, e.g., Hough transform and binary thresholding, have been applied to find where the keyboard and specific keys are located. At the same time, Convolutional Neural Networks(CNNs) has been also utilized to find whether specific keys are pressed or not, and how much intensity the keys are pressed only based on visual information. Especially for detecting intensity, a new method of utilizing spatial, temporal CNNs model is devised. Early fusion technique is especially applied in temporal CNNs architecture to analyze hand movement. We also make a new dataset for training each model. Especially when finding an intensity of a pressed key, both of video frames and their optical flow images are used to train models to find effectiveness.

Author

Kang, Seongjae; Kim, Jaeyoon; Yoon, Sung-eui

Published

2019

Language

EN