"Talk Show Host" [ft. Jibaro’s Sara Silkin ] – Is this the future of motion capture? + Breakdown

choreography and performance by: Sara Silkin
vfx: myself

In collaboration with Sara, I transformed an iPhone recording of this beautiful performance, into this multi-angle audiovisual piece.

I managed to do in using no ultra-expensive equipment, nor full-production budget. All in a single platform + editing software. [A few years ago, this would have costed several thousand bucks.]

Breakdown:

I started from the original dance/performance video and split it into 3-10s clips if I wanted to use the camera angle present in reference image, or up until 30s if I wanted to preserve original camera angle from video source.

Then I used Uisato Studio’s Kling Motion Control mode for generating the interventions.

Inputs were:

1. the original performance video as the reference video
2. a target image with the robot / bio-tech aesthetic as the reference image for each section. You can use the "capture frame" function to intervene one of input video's frames using Gemini, or you can bring your own intervened [reference] images. As I said before, here's the place in which you can introduce different point-of-view for the interevened scene.
3. a brief [balanced] prompt describing what I wanted beyond the motion transfer; "an avant-garde humanoid android performer dancing (...)" / "you might introduce subtle robotic precision while still following the original dance (...)"
4. while standard the "std" kling-3 model performs really well, I went with "pro" for that tiny, but noticeable overall improvement

In all sections I added some [10] overlapping frames at the start and the end between each, just in case I wanted to have some room for later transitioning between section on editing.

For some particular parts of the piece, I created duplicated sections for having variations of a single shot.

Once everything has been set I generated the clips in a single go, and then assembled the final piece in editing.

Voilá, single-character motion capture on-a-budget²