The EyesWeb open platform was originally conceived for supporting research on multi-modal expressive interfaces and multimedia interactive systems. EyesWeb has also been widely employed for designing and developing real-time dance, music, and multimedia applications.
It supports the user in experimenting computational models of non-verbal expressive communication and in mapping gestures from different modalities (e.g., human full-body movement, music) onto multimedia output (e.g., sound, music, visual media). It allows fast development and experiment cycles of interactive performance set-ups by including a visual programming language allowing mapping, at different levels, of movement and audio into integrated music, visual, and mobile scenery.
EyesWeb has been designed with a special focus on the analysis and processing of expressive gesture in movement, MIDI, audio, and music signals.
EyesWeb consists of a number of integrated hardware and software modules which can be easily interconnected and extended. The EyesWeb software includes a development environment and a set of libraries of reusable software components (blocks) which can be assembled by the user in a visual language to build patches as in common computer music languages inspired to analog synthesizers. A patch can be used as a module in a higher-level patch.
EyesWeb includes a software Wizard enabling users to extend the system with new modules, data types, and libraries.
The software runs on Windows and is based on the Microsoft COM/DCOM standard. It supports Steinberg VST and ASIO, OSC (Open SoundControl), and FreeFrame plugins.
Automation support is provided. This let developers invoke EyesWeb from other languages, such as Microsoft Visual Basic. The execution of a patch can be controlled from external applications. Any scripting language that supports automation can be used for this purpose (e.g., Python, VBScript, JavaScript, etc.).
Modules include passive modules (e.g., filters) and active modules, e.g., sources and modules with an internal dynamics (i.e., modules which receive inputs as any other module but may send outputs asynchronously with respect to their inputs).
EyesWeb libraries include:
- Input: support for frame grabbers (from webcams to professional videocameras), wireless on-body sensors (e.g., accelerometers), audio and MIDI input, serial, network, keyboard, mouse.
- Math and filters (e.g., operations with scalars and matrices).
- Imaging (processing and conversions of images).
- Sound and MIDI Libraries.
- Communication (e.g., MIDI, OSC, TCP/IP, serial, DCOM).
- Motion Analysis: motion trackers (e.g., feature tracking, tracking of multiple colored blobs), modules for extraction of global features from movement (e.g. amount of detected motion), analysis of the use of the space.
- Mapping of extracted features in real-time generation of audio and visual outputs.
- Output: visual, audio, MIDI, serial, network.
http://www.infomus.dist.unige.it/
