Metasyn is an interface that allows visitors to explore the collection of contemporary art in Roskiilde. The visualization includes an interactive 3D browser that is among the best I’ve seen. Items are organized in the space as follows:
The objects are lined up vertically by year showing the distribution of objects over time. For a given object, its vertical order is a product of the ‘grade of dominance’ that the related artist has. The objects that are made by artists whose objects are commonly accruing in the collection are placed closer to the ground plane. This results in an organisation where the most dominant artists are represented close to ‘the core’ of the structure, while the less known artists ends up in the periphery. This decision was made to support the impression of exploring the unknown in the outer areas of the collection, and to increase chances additionally that the museum’s choice of popular artists are promoted.
Music on the Spiral Array . Real-Time (MuSA.RT) explores the use of Chew’s Spiral Array model in real-time analysis and scientific visualization of tonal structures in music.
Tone-based music consists of sequential arrangements of notes that generate pitch structures over time. An expert listener is able to ascertain these structures over time. MuSA.RT allows listeners to see tonal structures as they hear them. Real-time tracking of tonal patterns in music also has widespread applications in music analysis, information retrieval, performance analysis, and expression synthesis.
MuSA.RT shows the names of the pitches played, the triads, and the keys, as the music unfolds in a performance. The structures are visualized and computed using the three-dimensional Spiral Array model. Two trackers, called Centers of Effect (CEs), one for longterm and one for shortterm information, show the history of the tonal trajectories.
The three-dimensional model dances to the rhythm of the music, spinning smoothly so that the current triad forms the background for the CE trails. The real-time MIDI (Musical Instrument Digital Interface) input can be captured from an acoustic piano through a Moog piano bar.
MuSA.RT was designed using François’ Software Architecture for Immersipresence, a general formalism for the design, analysis and implementation of complex and interactive software systems.
Hot of the presses, here are the sides for the tutorial that Justin and Paul are presenting at ISMIR 2009 on October 26.
Note that the live presentation will include many demonstrations and videos of visualizations that just are not practical to include in a PDF. If you have the chance, be sure to check out the tutorial at ISMIR in Kobe on the 26th.
This project explores several different ways of visualizing sets of extracted audio features in real-time. These visualizations are realized in a toolkit for the Max/MSP/Jitter programming environment. The primary purpose is to visualize timbral changes in the sense of exploratory data analysis. The program has four main parts: feature extraction, visualization, similarity, and audio control. Features are calculated by using a combination of pre-existing libraries, as e.g. the zsa.descriptors and the CNMAT analyser object. Additionally, we introduce a simple notion of timbral distance, which can be used in real-time performance situations, and present its performance for a set of different textures. The visualizations are further used to inform the control of audio effects by feature trajectories.
Joris Klerkx has built a visualizer of the emotions in lyrics. Joris has integrated a karaoke player and Synesketch, a framework for visualizing 6 basic emotions, defined by Ekman (happiness, anger, fear, surprise, sadness, disgust). The player takes a song, plays it, and with each line of text that plays in the lyrics, the strongest emotion of that line is visualized. In the image above, on the left hand side, you’ll see the 6 emotions and their visualization. On the right hand side, 2 screenshots of demo’s of the prototype.
Some video of the player in action:
Thriller by Michael Jackson: emotions fear, angry, sad & disgust are well visible in the end.
Jorik points out that it can be interesting to see how the visualizations contrast with how the song sounds since offten times the emotion and mood of the lyrics of a song contrast with how the song sounds
Zune offers a rather rich music browsing experience on the web showing all sorts of artist info including songs, videos, bios, news, reviews and artist popularity data. One rather nifty tool is their MixView. When browsing an artist you can click on MixView to display a variety of information related to the seed artist in various-sized boxes. Each box is clickable, which brings focus of the related item into view, and in turn, a new set of related boxes appear. Additionally, each box has other actions such as “play” and “learn more” depending on the view that allows the user to jump to different places in the Zune Marketplace. I like how MixView combines different types of information in one view. In one view they show related artist, artist influences, artist albums, related albums and so on. It is a well done browser – and one of the first that I’ve seen implemented in Silverlight.
MarGrid is a visualization that uses Self-Organizing Maps to organize music collections into a two-dimensional grid based on music similarity. On the MarGrid website you can use find a flash-based interface that will let you explore a 1,000 song music collection.
The MarGrid interface is incorporated into AudioScapes, a framework for prototyping and exploring how touch-based and gestural controllers can be used with state-of-the-art content and context-aware visualizations. AudioScapes provides well-defined interfaces and conventions a variety of different audio collections, controllers and visualization methods so they can be easily combined to create innovative ways of interacting with large audio collections.
Here’s an AudioScape video that shows the MarGrid in an iPhone app that is designed to to help people with disabilities navigate through their personal collections. There are more videos worth watching on the AudioScapes site.
Creator:
MarGrid and AudioScapes is a project being built by researcher Steven Ness and George Tzanetakis at the University of Victoria It is built using the venerable Marsyas audio framework
Fast Visual Music Discovery Via Locality-Sensitive Hashing
mHashup is a novel visual interface to large music collections, such as today’s million-song download services, for discovering musical relationships among tracks. Users engage in direct on-screen query and retrieval of music fragments in an instantaneous feedback flow performed by a locality sensitive hash table in secondary storage.
mHashup facilitates both professional music searches (such as musicologists and copyright lawyers seeking the origins of sampled music with location markers precisely given for each returned track) and end-user music applications (such as discovery of “dark media” by its relationship to known “hot” items). The visual/auditory display of results incorporates summaries of retrieved tracks and facilitates a user-interaction feedback cycle for refining and expanding music discovery processes. mHashup’s visual interface uses the core functionality of a content-based search engine as a visual grammar to be explored by direct manipulation.
The Fidg’t Visualizer allows you to play around with your network. You interface with the Visualizer through Flickr and LastFM tags, using any tag to create a Magnet. Once a Tag Magnet is created, members of the network will gravitate towards it if they have photos or music with that same Tag.
This simple mechanic lets you visualize your Network in a unique way, demonstrating its Predisposition towards certain things. What is more popular amongst people in your Network – rock or electronic music? Are photos of buildings more popular than photos of sunsets? Based on how your network reacts to those Tags, you might get an answer. The Visualizer also shows how your Network compares to a random sampling of the networks of other Fidg’t users, letting you see how your network stacks up to others?
For good measure, you can also search through the network for certain users, and check out their recent photos and music. This visualizer is just one example of some of the cool Address Book applications you could build on top of our web services.
RAMA is a prototype web-based application for visualizing and interacting with networks of music artists. It uses data of roughly 200000 artists and 3 million tags, collected from Last.fm’s API. Data includes artists similarities, associated tags and popularity.
RAMA provides two simultaneous layers of information:
a graph built from artist similarity data, modeled as a physical system representing nodes as negatively charged particles and edges as springs;
overlaid labels containing user-defined tags.
A number of different features aim at providing enhanced browsing experiences to users: RAMA emphasizes commonalities as well as main differences between artists, users can interact with the graph in different ways (changing the graph’s initial ramification R, the depth D and how the ramification decays with depth, the population factor P), etc. Optionally, users can edit graphs manually, removing some artists and expanding artist’s neighbors.
