Visual Interpretation for "Six Seasons" | Art & Science
The Chukchi Sea, north of Alaska, is one of the most inaccessible places to humans on earth. Six seasons in the Arctic, according to the Inuit, are not demarcated by a fixed calendar, but by what we hear in the changing environment. Hydrophones were placed about 300 meters below the sea surface at a seafloor recording location 160 km north of Utqiagvik, Alaska. They capture the sound of sea ice, marine mammals, and the underwater environment throughout an entire year. Our journey begins on October 29, 2015, just three days after new ice had started to form – the birth of ice. ---- By Lei Liang and Joshua Jones
"Six Seasons" is a collaborative research project directed by Lei Liang and Joshua Jones, where I (Mingyong Cheng) create a visual interpretation using generative AI and visual computing for an experimental and interdisciplinary music series. This work guides listeners through the six seasons of the Chukchi Sea, north of Alaska, where Liang and Jones's team integrate real-world recordings of sea ice, marine mammals, and underwater environments with speculative sounds crafted by performers.
My visual response consists of audio-reactive video artworks that respond to the music's nuances. The visual exploration begins with transcendent terrains reflecting satellite observations of the Arctic, where each season is realized through distinct strata: a living scenic landscape shifts imperceptibly while foreground elements move in liquid formations guided by acoustic elements. Throughout the seasons, traditional elements of Shan Shui (Chinese landscape painting aesthetic) gradually emerge, culminating in full expression during the Coda, where ice becomes liquid ink and oceanic motions transform into elegant lines.
Drawing inspiration from Liang and Jones's musical methodology, I construct speculative landscapes by combining real NASA satellite imagery with AI-generated synthetic imagery. The background layers are generated using a Lora fine-tuned Stable Diffusion model, trained on satellite imagery and referenced against traditional Chinese landscape paintings through ControlNet. For the coda, which is fully in the Shan Shui style, I process the audio signal into dynamic noise patterns that serve as input for our fine-tuned model in Stream Diffusion. This creates real-time generated Arctic landscapes that respond to and evolve with the musical elements. The generated landscapes are then composited and transformed through visual computing techniques to achieve the final Shan Shui aesthetic. Additional elements, including satellite imagery data and open-source footage, are processed in TouchDesigner to react to the soundscape.
This approach is deeply influenced by Shan Shui aesthetics, which transforms observed landscapes into emotional and spiritual realms through memory and imagination. In Chinese art, this technique represents a harmonious relationship between humanity and nature—a concept I extend to encompass all life forms shaping our ecology. This theme culminates in the coda, where a whale's journey of separation and reunion becomes a metaphor for our collective existence.
See the full collection below:
If the embedded showcase could not display normally on your device, please visit the showcase via: https://vimeo.com/showcase/11366903
Credit:
This project is led by Composer and Artistic Director Lei Liang, with advice from Oceanographer and Principal Scientific Advisor Joshua Jones. The violin and viola d’amore are performed by Marco Fusi, while the audio systems are designed and software developed by Charles Deluga. Audio software development is handled by Zachary Seldess. New media artist Mingyong Cheng responds to "Six Seasons" soundscape by reimagining the Arctic through creative computing and generative AI, merging the serene motifs of traditional Chinese paintings with Arctic landscapes.