The most important thing about MAPI ...

is to turn on rendering by pressing the green circle where it says gemwin (0).

The following is a (rather crude) documentation of MAPI, I hope it is useful. Please check out the MAPI wiki as well ...

What is it all about?

I'm trying to use PD / GEM to do some VJing. The main reason behind this is that I was somewhat swamped by the human/computer interfaces of existing VJ tools. The result is MAPI (download), here's a screenshot:

Installation

Grab the ZIP. Put all abstractions in your abstrations-folder, for cleaning up, just delete mapi*.pd. Open the left part of the MAPI window (where it says pd routing), scroll to pd initstuff and edit the settings in there to match your setup.
MAPI depends on a lot of externals: I'm recalling to have used cyclone, maxlib and the iemlibs at least. OSC is being used for the messaging. GEM is also a must :-)

How does it work?

MAPI is built around GEMs - and, for that matter, OpenGLs - rendering paradigm. Each of the little boxes (1) in the center of your monitor represent a render chain.

All render chains consist of:

• one input (2). At its current state, MAPI provides the following image sources: 2 live inputs (cam0/cam1, I'm usually running it with a DV camera and an USB web cam), 2 movie clips (mov0/mov1 with 3 different loop modi: no repeat, repeat and zigzag, the loop points can be set freely; the clips will advance either through scratching, automatically at a variable frame rate or triggered by external events), 2 static pictures (pic0/pic1), a collection of 7 pictures (7pics0, only one of these pictures will displayed at the same time) and another collection of 7 pictures (7bpics0, any combination of these pictures will be displayed using one out of 5 blending modes).
• effects (3). The following are available: rtx (GEM), ndance (inspired by EyesWeb, delays RGB components), backlight (GEM), lumaoffset (GEM), refraction (GEM), convolution (general convolution, can be used to do sharpening, softening, edge detection and the like), motionblur, halftone (GEM), colour manipulations (thresholds for RGB, adjusting RGB and alpha values, brightness, contrast). After any combination (to be precise: any combination that doesn't take up 100% of your CPU time) of these effects has been applied to the input, it will be rendered on one or more geos.
• output (4). Here you decide on which geo you want your texture to be rendered. Available are rectangles, cubes, spheres, texts, 3d-models (obj) and waves. You can also use some special effects, which will render the texture on more than one geo: there is a matrix with a customisable number of elements in the x and y direction, an oscilloscope, and a spectrum analyser. You can also change all rotation and translation parameters of the geo(s) here.

How do I control this mess?

MAPI can be controlled via mouse and keyboard or with a midi controller (highly recommended). On the left-hand side of the screen is a bunch of bangs, they can be clicked with the mouse. If you use a midi device, they will light up if MAPI receives a corresponding signal. There is a top-down-pattern: say you want to change the amount of red of the video clip that is rendered by mov0 on a cube. You click on "fx", then "mov0", then "col-adj" and then "R". Now, nothing happens. Two things to note: don't forget to turn on the colour-adjustment filter for the mov0 chain (above the "R"), and moreover, you need to assign a controller to control this effect. With this task, you'll probably spend most of your time with MAPI, and yeah - it is a bit clumsy. However: turn on controller learn mode (5), hit "R" again and move one of the sliders on top of the window to its zero position (left) and them more to the right. See the "/fx/mov0/values/col-adj/R" appear? Move the slider and watch your output. Got it? Controller assignments are valid as long as you don't assign the same controller to another effect (by moving it to zero and then a bit to the right). The sliders are mapped to MIDI control change messages on channel 16, this can be changed in the preferences.

What about the other stuff?

Lights (6) should be fairly self-explaining, although I must admit that I didn't use them very much. Metros (7) are - well, metros. They bang every so-and-so milliseconds and produce an envelope limited by floor and ceiling values. Envelopes are either a cheap attack-delay-release (times are fractions of the metro interval, note that they do not range from 0 to 100 per cent, but from 0 to 127 per cent), a sine wave or a weird sine-cosine-mixture (2*sin(x)-cos(2*x), IIRC). The values produced by the metros can - and should - be assigned to effects via the toggles on top of the MAPI window (8). Trigger (9) are pretty much the same and are assigned to effects by the other row of toggles (10). They produce an envelope and are triggered by a beat detection mechanism (bass - using Millers bonk) or by MIDI input (midi0/midi1, useful if you're performing live with some expert knob twiddler). amplitude just outputs the amplitude (no envelope), adsr0 and adsr1 are adsr-anvelope generators triggered by pressing a key on the midi controller.

okay ...

That's pretty much all about MAPI. It has some quirks: the worst thing is that it eats up a lot of CPU doing nothing - I suspect a tcl-/tk-issue. In my case, MAPI burns about 25 % CPU cycles when sitting idle (Pentium IV, 2.6 GHz). By using pd-carmen, I was able to bring this number down to 15 per cent.

things to make and do:

• improve usability ... the next big thing on my to-do-list is state saving for render chains (will probably use rradical and go to the preset-section). I'm also thinking about splitting MAPI in two versions: one would be a "mess around with me" version, whih would have a gui quite similar to the current one. The other one would be aimed at performing live, its gui should both be more intuitive and less cpu consuming (see above) than the current one. Since all messaging within MAPI is done with OSC, Flash could be used for that ... pd's built-in gui elements are definitely not up to this task.
• employ more fx ... appearantly, FreeFrame-plugins can be used with GEM now. This is great, I'll incorporate them as soon as someone comes up with a new compiled version of GEM for windows
• employ more sfx ... cyrilles l-systems are looking pretty good, something similar to rashid karim's digipop-stuff might be worth a try as well
• employ more inputs ... a snapshot of the GEM window might be a good idea, gotta check the speed of this object

Feedback ...

is very much welcome, drop me a mail: thoralf_schulze@yahoo.de.