Multisensory Creation: Can Visual Experience Be Expressed Musically?

“Does visual art simply convert into music, or does it emerge into a new structure through music?”

The combination of sight and sound in art has long been discussed under the umbrella of 'synesthesia,' but approaching it from the perspective of the physical structure's transfer and reconstitution, going beyond simple subjective impression, can be a different matter. For me, this contemplation has recently been distilled into the keywords: symmetry, non-linearity of mapping, and entropy.

• Transformation inevitably entails information loss.

• Loss diffuses like heat.

• The remaining structure condenses into motion.

• The result is emergence, not translation.

If symmetry has been primarily treated as the balance of planar forms or a static correspondence, in a contemporary view, symmetry is redefined as 'invariance'—the property of a system that remains unchanged even during the process of transformation. Mapping presupposes symmetry, and symmetry is multi-dimensional.

"Mapping presupposes symmetry, and symmetry is multi-dimensional."

Due to the inherent dimensional constraints and topological differences of the senses, this mapping process can never form a linear one-to-one correspondence. The visual cognitive system simultaneously captures the 2D or 3D composition of space, whereas the auditory cognitive system is inevitably accompanied by time, an irreversible and linear single axis.

The musicalization of visual art necessarily involves the process of projection and re-encoding. Projection functions as a creative filtration process that extracts the core topological information of the object and assigns a new context. The key point is that the overlap and omission of information that occur during the projection process are not technical limitations, but a decisive mechanism where the artist's interpretation intervenes to generate new aesthetic value. In this process, microscopic distinctions are averaged out and transition to a broader statistical state.

The phenomenon where visual details merge into a musical atmosphere or texture is an irreversible, necessary condition for creation where individual pieces of information transition into a macroscopic emotional current. Therefore, 'loss' during the transformation process is not evidence of failed translation, but the process of endowing the work with physical vitality and temporal irreversibility. It is not disorder, but the process where microscopic distinctions are averaged out into a macroscopic state. In (my own) technical terms, it is called "lumping" (퉁친다). When lumped in this way, the phenomenon of visual details merging into a musical texture can be viewed not as a mere deficiency, but as a statistical reorganization.

Mapping, which presupposes a multi-dimensional symmetric structure, is a process of holistic reorganization. Through this, the artistic transformation shifts its center of gravity from 'representation' to 'emergence.' The resulting product becomes a structural event that transcends mere sensory translation. This process is the foundation for many artistic expressions, not just the musicalization of visual experience.

Visual art is not reduced to music, but rather, through the medium of auditory form, it acquires a new mode of existence that was impossible in its previous dimension. At this point of emergence, the artist transcends being a mere designer to become a mediator who orchestrates the flow, encountering a new dimension of artistic reality born at the boundary's collapse.

https://www.youtube.com/watch?v=hcLB5V0UciQ

There was a day when I felt that improvisational performance notes could be modeled as "vectors," and that direction and movement could be modeled through the concept of "minimum path selection." Although it’s called the minimum path, it is not about moving side-by-side in a cramped manner within a three-dimensional space. I wondered if our spatial perception is perhaps too narrow, which is why we cannot understand cases where space is folded, or where the law of conservation is maintained, disregarding spacetime like in quantum entanglement. What if space were far more multi-layered? If we hypothesize various non-linear possibilities, we might be able to break away from a kind of teleological worldview. Could we perhaps simulate a breakthrough of biological limits by escaping the spatiotemporal center and a global temporal cross-section, rather than being confined to forms like Kiseungjeongyeol (起承轉結, Introduction-Development-Turn-Conclusion) or Gikyeonggyeolhae (起景結解, Start-View-End-Resolve)?

Although it was canceled due to low popularity, I really like the 'Stargate Universe' series. A spaceship launched a million years ago by the Ancients (Lantian) navigates through deep space based on subtle signals that possess the initial structural patterns of the universe. Along the way, it drops 'Stargates' to secure relay points. I liked this image of a journey.

I don't have the resources to launch a spaceship into deep space, but by expressing the imagination of a multi-dimensional space into an irreversible form of art, I indulge in my own version of journeying toward the deep cosmos.