Talking with Hands and Feet

A digital investigation of the coordination between Kathak dancer and Tabla drummer.

Topic

Kathak, a classical northern Indian dance form, is danced to the rhythmical beats of the tabla drums. A deep communication between dancer and musician is obvious in the beautiful, improvised synchronization of dance and music. However, the way in which this communication is achieved is not fully understood. Together, the institute for ethnomusicology of the university of music and performing arts Graz and the IT:U will conduct an experiment to investigate this phenomenon with digital and ethnomusicological methods.

Invitation

We invite a public audience to see a performance by Kathak dancer Sumedha Bhattacharyya and tabla player Haider Khan on Monday, June 24th, 2024, at 17:00. We will record all motions of the artists with a motion capture system synchronized to the audio and dance moves. Analysis of this data promises to answer questions about the communication between artists. Our methodology will serve as a groundbreaking example for further research into digital humanities.

Details

Ethnomusicological perspective: We hypothesize that Kathak dancer and Tabla player conduct a rich dialog through body and eye movements, as well as through musical and choreographic interpretations, to synchronize and improvise the art they collectively create. While participatory observation and interviews provide insights into this process, an additional external, data driven perspective will likely serve to validate the hypothesis. Finding time shifted correlations between patterns in music and dance, but also between patterns in movement of body and eye and music and dance, would provide focal points from which the question of causality could be investigated in dialogue with the artists. A successful digital research methodology would also benefit further investigation of countless other art forms with similar structure.

Digital perspective: Motion capture translates the movement of objects in space, and in particular of kinetic models of the human body, into digital trajectories which can be used in applications ranging from entertainment to medicine to design and digital humanities research. Traditional marker-based motion capture methods have struggled with complex setups like detailed finger joints, and the need to place markers always puts subjects in an uncommon situation. Markerless motion capture methods rely on camera data from RGB cameras and sophisticated machine learning models to capture biomechanic models directly from images. Audio recordings may be captured and frame synchronized to exactly match the recorded motion. Eye movements may be captured by post-processing video data or from dedicated eye trackers, which are no more uncomfortable or artificial than wearing glasses. Post-capture data analysis can reveal time shifted correlations between patterns in the various channels, including audio, trajectories and gaze.

Experiment: The planned experiment will see a Tabla and Kathak performance take place in a traditional setting, artists standing and sitting on the floor, in front of an audience also seated on cushions on a floor. Artist Haider Khan will play Tabla and dancer Sumedha Bhattacharyya will dance Kathak. The performance will take place on Monday, June 24th, 2024, at 17:00, in the motion capture lab of IT:U, Altenberger Straße 66c, SCP4, OG1. A public audience will attend. The whole performance will be recorded in audio and video and motion captured. The resulting data set will be shared between KUG and IT:U and analyzed in a collaborative research effort.

Research: Researchers from both IT:U and KUG will jointly analyze the data set and look for the correlations described above. Each side is responsible for its own effort. The cleaned and formatted data set will, together with a research paper about it, be published and provided as open source. Student projects of both KUG and IT:U will be based on the data and develop new insights.

Artists

Sumedha Bhattacharyya, Kathak. Haider Khan, Tabla.

A cooperation between University of music and performing arts Graz, institute for ethnomusicology, and IT:U.