Metasyn is an interface that allows visitors to explore the collection of contemporary art in Roskiilde. The visualization includes an interactive 3D browser that is among the best I’ve seen. Items are organized in the space as follows:
The objects are lined up vertically by year showing the distribution of objects over time. For a given object, its vertical order is a product of the ‘grade of dominance’ that the related artist has. The objects that are made by artists whose objects are commonly accruing in the collection are placed closer to the ground plane. This results in an organisation where the most dominant artists are represented close to ‘the core’ of the structure, while the less known artists ends up in the periphery. This decision was made to support the impression of exploring the unknown in the outer areas of the collection, and to increase chances additionally that the museum’s choice of popular artists are promoted.
Music on the Spiral Array . Real-Time (MuSA.RT) explores the use of Chew’s Spiral Array model in real-time analysis and scientific visualization of tonal structures in music.
Tone-based music consists of sequential arrangements of notes that generate pitch structures over time. An expert listener is able to ascertain these structures over time. MuSA.RT allows listeners to see tonal structures as they hear them. Real-time tracking of tonal patterns in music also has widespread applications in music analysis, information retrieval, performance analysis, and expression synthesis.
MuSA.RT shows the names of the pitches played, the triads, and the keys, as the music unfolds in a performance. The structures are visualized and computed using the three-dimensional Spiral Array model. Two trackers, called Centers of Effect (CEs), one for longterm and one for shortterm information, show the history of the tonal trajectories.
The three-dimensional model dances to the rhythm of the music, spinning smoothly so that the current triad forms the background for the CE trails. The real-time MIDI (Musical Instrument Digital Interface) input can be captured from an acoustic piano through a Moog piano bar.
MuSA.RT was designed using François’ Software Architecture for Immersipresence, a general formalism for the design, analysis and implementation of complex and interactive software systems.
The graph above plots tempi in the first movement, in terms of average beats per minute; the fastest is Hermann Scherchen, at 174.58, and the slowest is Otto Klemperer, in 1970, at 110.74.
Hot of the presses, here are the sides for the tutorial that Justin and Paul are presenting at ISMIR 2009 on October 26.
Note that the live presentation will include many demonstrations and videos of visualizations that just are not practical to include in a PDF. If you have the chance, be sure to check out the tutorial at ISMIR in Kobe on the 26th.
This project explores several different ways of visualizing sets of extracted audio features in real-time. These visualizations are realized in a toolkit for the Max/MSP/Jitter programming environment. The primary purpose is to visualize timbral changes in the sense of exploratory data analysis. The program has four main parts: feature extraction, visualization, similarity, and audio control. Features are calculated by using a combination of pre-existing libraries, as e.g. the zsa.descriptors and the CNMAT analyser object. Additionally, we introduce a simple notion of timbral distance, which can be used in real-time performance situations, and present its performance for a set of different textures. The visualizations are further used to inform the control of audio effects by feature trajectories.
Joris Klerkx has built a visualizer of the emotions in lyrics. Joris has integrated a karaoke player and Synesketch, a framework for visualizing 6 basic emotions, defined by Ekman (happiness, anger, fear, surprise, sadness, disgust). The player takes a song, plays it, and with each line of text that plays in the lyrics, the strongest emotion of that line is visualized. In the image above, on the left hand side, you’ll see the 6 emotions and their visualization. On the right hand side, 2 screenshots of demo’s of the prototype.
Some video of the player in action:
Thriller by Michael Jackson: emotions fear, angry, sad & disgust are well visible in the end.
Jorik points out that it can be interesting to see how the visualizations contrast with how the song sounds since offten times the emotion and mood of the lyrics of a song contrast with how the song sounds
MarGrid is a visualization that uses Self-Organizing Maps to organize music collections into a two-dimensional grid based on music similarity. On the MarGrid website you can use find a flash-based interface that will let you explore a 1,000 song music collection.
The MarGrid interface is incorporated into AudioScapes, a framework for prototyping and exploring how touch-based and gestural controllers can be used with state-of-the-art content and context-aware visualizations. AudioScapes provides well-defined interfaces and conventions a variety of different audio collections, controllers and visualization methods so they can be easily combined to create innovative ways of interacting with large audio collections.
Here’s an AudioScape video that shows the MarGrid in an iPhone app that is designed to to help people with disabilities navigate through their personal collections. There are more videos worth watching on the AudioScapes site.
Creator:
MarGrid and AudioScapes is a project being built by researcher Steven Ness and George Tzanetakis at the University of Victoria It is built using the venerable Marsyas audio framework
Fast Visual Music Discovery Via Locality-Sensitive Hashing
mHashup is a novel visual interface to large music collections, such as today’s million-song download services, for discovering musical relationships among tracks. Users engage in direct on-screen query and retrieval of music fragments in an instantaneous feedback flow performed by a locality sensitive hash table in secondary storage.
mHashup facilitates both professional music searches (such as musicologists and copyright lawyers seeking the origins of sampled music with location markers precisely given for each returned track) and end-user music applications (such as discovery of “dark media” by its relationship to known “hot” items). The visual/auditory display of results incorporates summaries of retrieved tracks and facilitates a user-interaction feedback cycle for refining and expanding music discovery processes. mHashup’s visual interface uses the core functionality of a content-based search engine as a visual grammar to be explored by direct manipulation.
The FM4 Soundpark is a web platform run by the Austrian public radio station FM4, that visualizes an audio similarity music space. Soundpark incorporates purely content-based rcommendations based upon a seed track and provides a 2D visualization based on audio similarity as well as an interactive 3D visualization based upon a combination of audio and metadata features. Features of Soundpark:
Music Recommendation: The core of all applications is a content based music similarity function. The similarity is automatically computed and based on models of the songs’ audio content. Musical instruments and voices exhibit specific frequency patterns in the audio signal. These patterns are estimated with statistical models and used to compute the audio similarity.
Soundpark Player: Whenever a visitor of the Soundpark listens to a song, a recommendation of five or more similar songs is provided. These recommendations are visualized in a graph-based representation. Users can interactively explore the similarity space by clicking on songs in the recommendation graph.
Soundpark 3D: The entire database of songs in the Soundpark is visualized as an audio landscape of sea and islands. Songs from the same genre inhabit the same islands, within islands songs are grouped according to their audio similarity. Users can travel through the landscape and explore their own audio path through the data base.
Automatic generation of playlists: Visitors of the Soundpark can choose a start and an end song from the data base and a playlist of eight more songs “in-between” is automatically computed. The playlist is a smooth transition from the start to the end song. This functionality is not online any more.
