Everything that surrounds us is nothing but a collection of atoms, particles and electromagnetic fields, vibrating without any apparent meaning. When these impulses are interpreted by our mind, they become colours, tastes, music, memories and emotions: the foundations of what each one of us perceives as reality.
Over the course of our lives, our perception of reality evolves; our understanding of space and time changes, and along with it, the way in which we perceive the surrounding universe: a vast flow of events and dynamic currents, interconnected and constantly interacting with one another. Everything that surrounds us becomes a unique landscape built by our minds, a universe that self-generates in our unconscious, in which we dance with all the marvel of a child. We fight against the time of adults, and we grasp the peace of a sunset. In this continuous movement, everything changes along with us, from the tiniest atom to the great galaxies, in an unstoppable flow of events and currents that unites humanity in a cycle of eternal renewal.
Dökk is a journey through the subconscious mind, where reality is represented by worlds and universes that take form and dissolve within the mind, as it constantly seeks a balance between light and darkness.
Dökk is an Icelandic word that means darkness. In some cultures, the absence of light may be interpreted as a metaphor for life on Earth and of the perception of reality represented as a shadow cast by a light that cannot be seen yet the existence of which may only be sensed. Starting out from this concept, the narrative was developed through the creation of ten rooms that make up a circular path in which the end coincides with a new beginning.
The Italian term stanza (‘room’) may reflect a variety of meanings: it could be a well-defined physical space that isolates us from the outside world, but it could also be imagined as a mental space inside which to construct our own vision of reality, or in the literary field, it refers to a verse of a longer poem.
The rooms of Dökk are the result of the coming together of these meanings in a single form of symbolism that accompanies the whole narrative. Inside these places of the mind, universes are constructed that evoke the various stages of life.
The journey starts by recounting the sensation of wonder and total interconnection with the world, typical of childhood. As we grow, all this is lost, and reality becomes something to fight against: a suffocating and oppressive force that leaves a sensation of emptiness, disorientation and bewilderment. The path comes full circle with the rediscovery of that connection with the universe that one has as a child, yet with the awareness of those who have known and crossed through the darkness, having found harmony in opposing forces. At that point, everything may end, only to restart in another life, another journey through the shadow of light, into the darkness: Dökk. The stage becomes a space in which it is possible to lose one’s perception of space and time: a place of the mind in which reality is reconstructed as the result of one’s own actions, and in which every gesture has its own particular consequence. Thus our awareness grows of the power of our own gesture in the ‘creation’ of the world around us.
MAKING OF - REAL TIME
Dökk is the upshot of three years of work aimed at the creation of a work capable of stimulating a sense of profound empathy, transmitting two key concepts: the synchronicity and the unforeseeableness of human existence.
In order to obtain this result, a system has been developed capable of elaborating the result of close interaction between various data generated in real time on the stage: the analysis of sound, the movement of the performer, his/her heartbeat and the sentimental analysis of contents shared on social networks. The combination of these data thus ensures that every performance takes on ever different and unique connotations as a result of the random, unforeseeable nature of the information analysed.
Every time the show starts, a stream of tweets filtered through the trending topics of that precise moment is analysed. For every tweet, a sentimental composition is extracted from an algorithm based on the open-source library developed by U. Krcadinac. And so it happens that while Dökk is performed, if a particularly relevant event in the world is going on, the visual and audio landscapes will change, taking on different connotations. In this way, the audience is stimulated to an involvement which goes beyond the physical space where the work is unfolding thanks to thousands of invisible connections with the rest of the world. The element that indicates this kind of variation in the various settings is the colour red, which in turn represents the intensity of human warmth. As shown by the study Bodily maps of emotions, various emotional states correspond to specific thermal spectrums of various parts of the body.
The data extracted from the sentimental analysis of the tweets act on the ‘warmth’ of the scenes, modulating the shade of red in every single moment, as well as altering other graphic details that modify the atmosphere of the single rooms.
Also, the soundscape is influenced by these data: there are six ‘ghost’ tracks, one for each base emotion, which are mixed with the main soundtrack. The mixing of these tracks changes on the basis of the percentages of the emotion distribution analysed, that the spread of single sounds in the 4.2 surround system that envelops the audience also depends on.
MOTION TRACKING AND HEARTBEAT DETECTION
The interaction between choreography and the scene elements represents one of the focal points of the project. The goal set right from the start was to communicate a sense of union and synchronicity between the digital and the bodily components.
In order to do this, we made use of a Perception Neuron: a motion capture system characterised by 18 accelerometers positioned directly on the body of the performer, allowing for the real-time tracking of every single movement. These data are then cross-checked with the two Kinect units placed on the stage in order to monitor her positioning on the scene and provide and even more profound level of interaction.
Another fundamental element that characterises this experience from the beginning to the end is the heartbeat. The heart is the first organ to be formed when a new life is created, and it is the last to stop when it ends. Likewise, the start and the end of Dökk are marked out by the real heartbeat of the performer, acquired through a BLE heart rate sensor, which transmits information via Bluetooth to a special software that in turn elaborates and sends the necessary data via network messaging.
The digital landscapes through which the narrative unfolds are visual metaphors designed to try and portray the perception of reality in the subconscious mind. This representation takes place on the stage, a physical space defined by precise confines which, during the development phase, we felt the need to overcome in order to express the vastness of thought. From here, the need arises to display the universe itself as a symbol of the human spirit.
In Dökk, visualisations alternate of what our universe is, based on a variety of data sources: in some cases the upshot of real observations of space, in others obtained by simulations. Every room is characterised by peculiar behaviours defined by specific physical laws that determine their interaction with the various data analysed in real time.
In the first part of the journey, the visualisation of the universe is put together with data from studies (links below) using observation of the redshift through the analysis of the spectrum of light emitted by celestial bodies. On the basis of the composition of the celestial body, we can find the difference between light measured and that predicted, going on to assess the shift in the frequency of certain components through the Doppler effect. ln the case of redshift, the value found indicates that the celestial body is moving away from us; in the case of blueshift on the other hand, it is drawing closer. From the value of the redshift it is possible to measure the speed with which the body is moving compared to our position, and whether or not it is nearing the Earth. The depiction of the universe that opens the show was obtained by placing particles in the real points of ascension and declination of the galaxies present in the databases, exploiting the redshift parameter to attribute a distance to the particles from the centre (i.e. the Earth – us) on the basis of the Hubble constant.
This very characteristic form, made up of two almost symmetrical sectors, is actually the result of a problem of observation of space because of which man can only observe beyond the distribution of the stars of our Milky Way, which in fact creates two large blind spots where no particle has been placed (or the very few that are visible).
THE HEARTBEAT AND TWINKLING STAR
From the start of the journey up to the final room, the real-time analysis of the heart rate is connected to the twinkling of one of the stars that make up the universe visualised in the scene. The pulsation of this star remains connected to that of the heart from the start to the end of the show. This idea was suggested by the discovery of the existence of a group of binary stars of which the luminosity is actually described by a graph very much like an electrocardiogram. The positioning of this star in the representation of the universe in Dökk respects the real coordinates of this binary system.
THE GREAT ATTRACTOR
In the third room, the universe starts to deform into a spiral that only on first sight seems to be centred on the position of the Earth and of the protagonist (at the centre of the two sectors); in actual fact, the distortion takes place around galaxy 16154609-6055071 (ESO_137-_G_008) which is to be found roughly in correspondence with what is defined as the Great Attractor: the point where a gravitational anomaly has been identified, powerful enough to deviate all the galaxies within Laniakea, the supercluster of which the Earth is also part (square red dot in the following texts).
Because of the expansion of the universe in accordance with the Hubble Flow, the redshifts of the galaxies (as observed in various studies) find a correspondence, but they also have variations that demonstrate this anomaly of the Great Attractor: the cause of a peculiar velocity.
In room six, the scale changes, and the filaments that initially described a universe are transformed into axons and dendrites through a construction of a stigmergic nature, in which the particles go on to make up tracks that depend on those paths carried out by other particles. The result is a network structure of which the parameters are modified by gestures, which in turn become stigmergic paths within this network.
In room seven, the networks moves in around a sphere that envelops the performer. The rotation of all that which surrounds her follows her movements, which in turn simultaneously control and modify the music.
EVOLUTION AND ASSEMBLY OF GALAXIES AND THEIR ENVIRONMENTS
From room eight, the landscape gradually returns to be viewed as a universe, yet one which is no longer the result of correspondence with the observations of redshift, but rather becomes an accurate representation of the universe that surrounds us.
The simulation used as a starting point for the positioning of the particles is that known as ‘EAGLE’ (Evolution and Assembly of GaLaxies and their Environments) created by the Virgo Consortium. The data represent various states in the evolution of the universe, starting from a ‘time zero’ when the universe was completely uniform, right up to the current age of the universe in which filaments, voids and galaxies can be seen and identified, just like those we may observe ourselves. The incredible correspondence between what we may observe and reality is the confirmation of the exactness of the latest scientific discoveries.
In room ten, which brings the show to an end, the same EAGLE simulation engine has been exploited, but applied here to an invisible system of particles that go on to influence a gravitational field.
The simulation that we put together through the calculus software GADGET-2 ha operated on our servers for almost a week, generating 500 ‘snapshots’ of the state of matter in a uniform universe made up of 276,498 particles. These elements represent the contribution to the formation of a vector field of velocity, the final result of our study, represented by a cube of which each side is 100 million light years in length. By exploiting this field, we managed to move the particles used previously to generate a vortex around the protagonist, as if to try and trace the evolution of our own Laniakea.
The music of Dökk is designed to provide the narrative voice that accompanies the entire show, making it possible through sound to recount a whole series of emotions and states of mind that it would be hard to describe in words.
According to the theory of synchronicity, chance – meant as a sequence of non-linear events – does not exist, but that on the contrary it may give a sense that helps to define a direction of events. Also the method used for the production of Dökk, and in particular of the soundtrack, was influenced by a non-linear approach, recording and editing generative musical sequences within compositions capable of creating unique audio events and random successions of notes and sounds.
A major contribution to the creative direction of the sound and structure of a number of pieces was the chance to take part and record during a music therapy session, along with children and kids affected by autism. Using these recordings as a starting point to create the sonorities of the first rooms, they represent childhood and the joy of the discovery.
During the production and mixing of the sounds, various audio synthesis tools and processes were adopted, from the use of semi-modular synths, to sound-design e field-recording techniques, right up to audio programming with the use of Max MSP (a graphic development environment), creating chains of effects for the elaboration of sound and patches of audio synthesis in real time for the interactive parts of the performance. One of these patches uses a system that includes four granular synthesis modules, in which each parameter is controlled by the position in space of the Neuron perception sensors, located on the performer’s hands and feet. Every movement makes it possible to explore and manipulate various audio samples taken from ambient recordings, triggering little sound fragments defined as ‘grains’ which have their own specific volumes, dimensions and frequencies.
The music for Dökk was designed, composed and mixed to be played through a 4.2 surround system so as to envelop the audience and create a multisensory experience which is as immersive as possible.
Direction and Executive Production: Mattia Carretti, Luca Camellini
Concept: Mattia Carretti
Performer, Choreographer: Elena Annovi
Software Supervision: Luca Camellini
Software: Paolo Bonacini, Matteo Mestucci, Samuel Pietri
Sound Design: Riccardo Bazzoni
Hardware Engineering: Matteo Mestucci
Production Management: Filippo Aldovini
Support for Concept Development: Giulia Caselli
Scientific Consultant: Margherita Carretti
Collaborators: Mark van de Korput, Clizia Welker, Enrico Viola
Light Engineering: Marcello Marchi
Video Report: Matteo Torsani
Photo Report: Enrico Maria Bertani, Emmanuele Coltellacci
Dökk is developed using openFrameworks and powered by NOITOM’s MOCAP Perception Neuron.
We acknowledge the help of the Virgo Consortium for making the EAGLE simulation data available. Thanks to CAT Centro Armonico Terapeutico for allowing us to record a session of musical expression with the guys from the Spazio Me group.
In collaboration with La Corte Ospitale – Progetto residenziale 2016
Special thanks to friends, family and the whole FUSE*FACTORY crew for their support.
 Krcadinac, Uros, et al. “Synesketch: An open source library for sentence-based emotion recognition.” IEEE Transactions on Affective Computing 4.3 (2013): 312-325.
 Nummenmaa, Lauri, et al. “Bodily maps of emotions.” Proceedings of the National Academy of Sciences 111.2 (2014): 646-651.
 Huchra, John P., et al. “The 2MASS Redshift Survey—description and data release.” The Astrophysical Journal Supplement Series 199.2 (2012): 26.
 Colless, Matthew. “The 2df galaxy redshift survey.” Looking Deep in the Southern Sky. Springer, Berlin, Heidelberg, 1999. 9-15.
 Crain, Robert A., et al. “The EAGLE simulations of galaxy formation: calibration of subgrid physics and model variations.” Monthly Notices of the Royal Astronomical Society 450.2 (2015): 1937-1961.
 Furlong, M., et al. “Evolution of galaxy stellar masses and star formation rates in the EAGLE simulations.” Monthly Notices of the Royal Astronomical Society 450.4 (2015): 4486-4504.
 McAlpine, Stuart, et al. “The EAGLE simulations of galaxy formation: public release of halo and galaxy catalogues.” Astronomy and Computing 15 (2016): 72-89.