SYNAESTHESIA
IASAS Moscow Synaesthesia Meeting 2019
Synaesthesia, Creativity & Consciousness: Music and Arts
Abstract - Synaesthesia is a relatively rare condition characterized by cross-communication between the sense modalities wherein a sensory experience in one modality (e.g. an auditory stimulus) automatically elicits a concurrent perceptual experience in another (e.g. a visual percept). Some synaesthetes utilize these cross-modal pairings for creative purposes, as in the case of Vincent van Gogh, Wassily Kandinsky, and Olivier Messiaen, all of whom appear to exhibit chromesthesia, where sounds are converted to colors. Here we report the contents of interviews with three synaesthete artists who all experience auditory stimulation in the form of visual experiences. Each of the three describes the nature of their synaesthetic experience and the process by which they produce their artistic output in response to specific musical compositions. We highlight some examples of sensory transposition by artists, including visualization of musical texture as “landscape”, a 4D digital model of musical-space synaesthesia, the type of synaesthesia symbolist composer Alexander Scriabin possibly had. We discuss what these cases reveal about the nature of synaesthesia, as well as how visual digital applications may benefit from considering aspects of multi-sensory design based on synaesthetic sensory pairings.
Public Outreach Concerts: Dr. Svetlana Rudenko is a researcher and concert pianist who performs in concert with accompanying visuals, demonstrating how sound and music can be perceived with other senses, including visual, tactile, potentially aromas, and even taste. Awareness of sensory experiences elicited by music broadens our understanding of the processes of imagery, mind, and consciousness. Some of the performances include a Synaesthesia Concert in Maynooth University, 2020; a Music & Consciousness event in University College Dublin, 2020 (both postponed due to Covid-19), Cognitive Musicology via Synaesthesia and Cross-Modal Associations at the BNA Festival of Neuroscience in Dublin, April 2019, and at the IASAS Synaesthesia conference Moscow, October 2019. Collaborations with visual artists Maura McDonnell, Timothy Layden, Ninghui Xiong, Sofia Areal, Geri Hahn, Ann le Pore, and others, have pushed the boundaries of how audio-visual art could be assistive to classical music analysis. Some performances were Svetlana’s improvisations on Art, translating visual experiences into the musical genre. These projects have the potential of developing into AR digital environments for therapeutic purposes, and potentially the prevention or reduction of dementia via the stimulation of brain plasticity mechanisms.
Review - IASAS Synaesthesia conference Moscow, October 2019 | www.theiasas.org
American Synesthesia Association - 12th ASA Conference, October, 2017
Harvard University, Cambridge, Massachusetts
The American Synesthesia Association, Inc., a not-for-profit organization, was created in 1995 by Carol Steen and Patricia Lynne Duffy to provide information to synesthetes and to further research into the area of synesthesia. The mission of the ASA is to foster and promote the education and the advancement of knowledge of synesthesia and to promote and provide a means for the people who experience and/or study synesthesia to be in contact. As part of its educational mission, the ASA will provide information to scientists and other researchers, health professionals, academicians, artists, writers, musicians, lay persons, and people who experience synesthesia.
Relatively little was known in 1999 about this ability called Synesthesia, but research over the past 18 years has shown that synesthesia is not nearly so rare as once believed. Although its existence has been recognized for several centuries, it is only in recent decades that synesthesia has received serious scientific attention. Two developments have contributed importantly to this greater awareness of and attention to synesthesia: the development and use of methods of neuroimaging, especially fMRI, and the spread of the Internet. The ability to monitor ongoing human brain activity has launched numerous studies of synesthesia in research laboratories around the world and the widespread and easy access to the Intern.
Τhe Official page of the American Synaesthesia Association (ASA)
Ninghui Xiong on my performance of Chopin Ballade N1
Neuroscientists are searching for the answer to brain plasticity
Professor Takao K. Hensch, Molecular and Cellular Biology, Harvard University. Artist Carol Steen, Touro College, New York City
Special thanks to:
Greta Berman, Julliard School New York City; Daphne Maurer, McMaster University, Hamilton, Ontario, Canada; Radhika Gosavi, Edward M. Hubbard, Department of Educational Psychology, University of Wisconsin-Madison; Ninghui Xiong, China Synesthesia Alliance; Carrie C. Firman, Edgewood College, Wisconsin
Artists and Musicians embrace creativity through Synesthesia
Abstract - Cross–Sensory Visualization of Musical Texture
Music is initially associated with our sense of hearing and we know that it has a strong power to stimulate our brain in terms of feelings, memories, or even to make our body move rhythmically. But some people can experience music in additional ways- not just heard but seen, felt, smelt. This cross-modal perception, or synesthesia, allows them to perceive a particular stimulus with multiple responses, such as the shape or touch of sound, the aroma of timbre, etc.
Artists use multi-sensory experiences from synaesthesia to capture sound impressions in artistic forms. The musical texture of some composers, such as Scriabin, Messiaen, Liszt, Rachmaninoff (synesthetes /pseudo synesthetes), are organized in such a way that makes an image/archetype both visually and audibly recognizable. Music analysis could be the source for developing a methodology of symbolic artistic visualizations. Scriabin as a performing pianist stretched the piano to new limits of tone and touch. His performing tempo fluctuates continuously but the average tempo coincides with metronome indication: ‘he kept, side by side, two timelines’ (Leikin, 2011).
I argue that visualization of musical texture, (Interviews, Rudenko, 2018) has a great influence on sound perception: timing of the musical phrases, dynamics of rhythm, harmony ‘tastes’, and the whole interpretation of the composition. My digital project, 4D visualization of musical texture, demonstrates volumetric properties of sound as a terrain ’‘landscape’ in musical-space synesthesia perception.
4D Digital Visualization of Musical Textures, Dublin, May, 2017
4D Visualization of Musical Texture
A computational model of Musical-Space Synaesthesia perception, developed in Processing, live code reacting on three layers of musical texture, demonstrating perception of sound as a "landscape" and dynamic shapes of the melody. According to recent research (Akiva-Kabiri et al, 2014): “In musical-space synaesthesia, musical pitches are perceived as having a spatially defined array...unlike the vertical and horizontal representation of musical pitch tones in the general population, synaesthetes describe a linear diagonal organization of pitch tones." My hypothesis is that mental visualization of melodic/harmonic shapes influences time-perception and can save hours of monotonous repetitive practice to overcome technical virtuoso difficulties in music repertoire.
First Synesthesia Exhibition in China, October, 2016
Synaesthesia, Cross Sensory Practice in Art, China, October, 2016
1st Synaesthesia Art Exhibition and Forum in China “Cross-Regional Synaesthesia Survey”
Svetlana Rudenko participated in the project events in Beijing, Xu Zhou, and Hang Zhou in 2016.
Please, refer to the Artecitta Website (http://www.artecitta.es) with relevant photos, videos, and media reports.
1st Synaesthesia Art Exhibition and Forum in China
Cross-regional Synaesthesia Survey Beijing, XU Zhou, and Hang Zhou
Synaesthesia and Cross Sensory Practice in Art
Project Team Report: (Ninghui Xiong Media report)
Additional video: Mobile phone application
My Interview: RTE Radio 1 at 33:50 Inside Culture#7, May 2016
“Inside Culture #7“ @ 33:50 | (Minister Humphreys, Roger Casement, Music Perception and Cloudlands)
The broadcast was produced on May 23rd, 2016.
Link to Interview and transcript: https://soundcloud.com/insideculture/inside-culture-7
International Conference at Trinity College Dublin in Conjunction with UK Synaesthesia Association, April 2016
Paper presentation, Musical-Space Synesthesia: Image Processing and Space/Time Organization of Musical Texture
Trinity College Dublin hosted a conference on Synaesthesia and Cross-modal Perception, organized by Kevin Mitchel and Fiona Newell in conjunction with the UK Synaesthesia Association: ’The aim of the meeting is to provide a forum to present and discuss the latest findings on cross-modal perception and synesthesia. We aim to include scientific input from many different disciplines, with a variety of perspectives from molecule to mind, relating to cross-modal perception.’
Kevin Mitchell: ‘For millennia, philosophers have mused over the nature of perception, how closely it mirrors ”reality” and whether different people might, quite without knowing it, subjectively perceive the world in very different ways.’
The conference gathered amazing people from different professions and backgrounds from all around the world who are passionate to SEE and HEAR the world with other senses and from different angles. Thanks to scientists who supported ‘irrational’ experiences with feedback on a brain wiring to give confidence to artists, musicians, educators, designers to carry on their gift to people, and particularly children. Recent research proved that the brain of a newborn is cross-modal and it is an obligation to adults to nurture young age creativity and give the best understanding and new learning methods for the next generation. Synaesthesia Artforum Blog - [View Page]
V International Congress Synesthesia, Science & Art, May 16th-19th, 2015
The musical texture of synesthete, pseudo-synesthete composers such as Scriabin, Gubaidulina, Messiaen, is organized in a certain way, making an Image/Archetype visually and audibly recognizable. According to recent research (Linkovski, Akiva-Kabiri, Gertner and Henik, 2012):’ In musical-space synesthesia, musical pitches are perceived as having a spatially defined array…unlike the vertical and horizontal representation of musical pitch tones in the general population, synesthetes describe a linear diagonal organization of pitch tones’.
This paper will explore the connection between three dimensional hearing/vision of an Image in musical-space synesthesia perception and its realization in compositional writing, or interpreting, elements of musical texture. It is hypothesized that most professional musicians develop this sense through training. Scriabin’s performing tempo fluctuates continuously but the average tempo coincides with metronome indication:’he kept, side by side, two timelines'(Anatole Leikin). Synaesthetic hearing and brain wiring led him to insert a color organ into the score of Prometheus illustrating slow and fast timelines by colors. Messiaen’s ‘independence of rhythm and pitch, duration, intensity and attack'(Robert Sherlaw Johnston) indicate that Image is a primary source organization of a musical texture. The synesthetes are considered to be more creative in Arts and Music. Understanding their musical perception, Image processing and space/time organization of musical texture could be the key to creative interpretation and memorable performance.
Paper details below:
Musical-Space Synaesthesia: Visualisation of Musical Texture Svetlana Rudenko 1,2,∗ and Maria José de Córdoba Serrano 1 1 Faculty of Fine Arts, University of Granada, Spain 2 School of Computer Science and Statistics, Trinity College Dublin, College Green, Dublin, Ireland (Online Publication) | (Book)