The Chromakey feature in the Slingstudio has got me thinking about how one might enhance a livestream with visuals behind the band during a livestream.
Here's what we are going for. We have traditional lighting set up in the space to add some mood to the set and make it look more like a stage, but our backdrop is an ugly, beige orange-peel wall. We could just get some fancy heavy curtains (red or purple velvet-like) and hang them and we'd have more of a stage feel already. But what would be really cool is if we had graphics behind the band like you see in the big shows, so how do we get that, especially without a big-band budget?
Today, we're going to cover 4 different methods/designs for background replacement, and discuss the pros and cons for a band in a small room.
What makes this particular use case challenging is the band setting. If we were recording just one or two people for a newscast or a youtube video, we would be using all white lights. In a band setting, that would be really boring...we want lighting of different colors to create mood and for that mood to be complemented by the background video. Additionally, having a small space to do it in adds further challenges, which we'll discuss through this post.
Classic Chromakey Methodology (green screen)
Green screen's were invented back in the 1930's and are still in heavy use today in both broadcast TV and movie production sets. The definition of chromakey is pretty simple: chroma - meaning color, and key. The software needed to do implement a chromakey is pretty commonly available. For example, Adobe Premiere Pro, Adobe After Effects and Davinci Resolve all have built-in chromakey software which is fairly decent for many applications. Broadcast/livestream software also has bulit-in functions from companies like SlingStudio and Vimeo Livestream Studio. And then there are 3rd party applications, like Primatte Keyer from Red Giant, which I own. There are also hardware solutions out there, like those from Black Magic, but those are well out of my price range.
The way it works is that you use a pointer to click on your background (green, blue, whatever) to select the color (chroma) of your key, and then there are various ways to manipulate the specific color range of the key (chroma values). The software then removes that portion of the image, making it transparent. That way when you layer that video on top of another video, whatever the video is below it will show through the transparent parts of the video on top.
How to make it work right
A classic Chromakey green screen approach involves a green or blue screen behind the subject. It needs to be both well lit, as well as evenly lit, in order for your software to easily identify the color and mask it out. Ideally it is also far enough behind your subject(s) to where the light reflected off the green or blue material does not hit your subject, or you get what's called "spill." There are both wardrobe and lighting limitations with this method. Your subjects cannot wear the same color as the screen that you'll apply the Chromakey to, and you cannot light your subjects with the same color lighting as the background screen or part of your subjects will disappear and be replaced with your background video. This can create kind of a cool effect if done intentionally, but usually when this happens it looks like garbage.
Another problem with this setup is the spill from the studio lights. In a classic green screen "talking head" shot, you light the subject with white light, and if any of that light happens to spill into the green screen, it's no big deal, it's just extra light. With colored mood lighting, if any of that light spills onto the screen, or is refracted from particles in the air (think haze machine) you get the opposite kind of spill problem...now part of your background replacement disappears.
I am using my basement as my space for live-streaming, and there are some serious challenges to overcome. Even though spill is not a major issue, the white lights that I use (5 of them) add ambient white light to the room because they are sitting right behind the drummer, and that lessens the impact of the other lighting I am using considerably. The lighting is still a bit uneven on the green screen, so I also have to increase the tolerance setting in my livestream software to compensate for that, which, if done too much, creates other undesirable artifacts and generally degrades the quality of the foreground images, particularly edge detail.
I had mixed results. In the first example, you can see what worked well. This shot is before background replacement.
And this is after background replacement:
And here's a scene that didn't go as well. As you can see from the picture below, the light is not particularly even, which makes for a pretty bad key.
Pre-Chromakey replacement:
Another method for lighting is to use a projector. A projector is an interesting light because it's designed to evenly spread light across the entire defined area. If you can get it positioned so that it doesn't hit any of the subjects, you should be able to project a nice even while light across your area for about $250. I've not tried this personally, but it's an interesting option and probably ends up being significantly cheaper than purchasing LED lights.
Yet another interesting method for lighting is to build your own lights using 9 foot florescent tubes and placing them into a commercial ballast. There are lots of videos on Youtube about how to do this, just search on "build your own green screen lighting on the cheap" and you'll find dozens of them.
Reflective Green Screen Approach
There is an alternative approach to doing a green screen, which I guess would be called a reflective green screen. The key enabling technology is a special reflective material from companies like what you would use if you were making night-jogging clothing. Instead of using green cloth as your backdrop, you put this material up and it reflects 90-95% of the light sent at it back in the same directly. If you pair this with a green light that is very close to your lens, the camera will see the screen as green, but no one else in the room will unless they start standing right next to the light. It will just look dark grey from every other angle in the room. If one of your other lights in the studio hits it, say a red light, the camera won't see it.
I found a turn-key solution in this space called ReflecMedia, and they are selling kits starting at around $2,000. For a space my size, I am probably looking at more like $20,000 for a solution. While I am willing to spend a little money to test something out, that's way beyond my available test budget, so instead I plan to borrow a pair of ADJ Mega Par Profile Plus lights from my rig along with two yards of 53" wide reflective fabric I was able to pick up on amazon for $50. These guys at ReflecMedia charge $1,800 just for a green LED ring-light to attach to your lens, which makes for a very green reflection without shadows, but I prefer the idea of using a DMX-controlled light sos that my DMX controller system has full control over the intensity, and whether it is even on or not, and as an added bonus, they are less than 1/10th the cost of one of their green ring lights.
Here are the results of my initial test.
Screen as seen with my phone camera placed right next to the light, wit the intensity of the light at approximately 30% and no other light's on (except that errant one in the upper right).
Moving to a different spot in the room and taking another shot. Yes, the green light is still on, but it's completely invisible because the green light is a good 30 degrees axis from the position this was taken in.
And finally, a screen shot from the Android app Green Screener (also available for apple).
You can see that even with the tolerance set to medium, the quality of the key is really good. Far better than I ever got with the green fabric lit with as many lights as I could place. There are a couple of reflections off the kit that are showing up in the key as well, but I don't think those are going to cause any significant issues.
And finally, here's a test of that green screen I ran in Vimeo Livestream Studio.
As my initial tests with this fabric are very promising, I ordered another 20 yards of the non-stretch variety. If you go directly to the fabricempire.com web site, you can actually find it a little bit cheaper than amazon. I also ordered some new lights that have a 90 degree angle, so I should be able to cast a nice green light across my entire backdrop with just a single light. The items I have coming in are the American DJ Dotz Par, a basic 36W RGB 3-in-1 LED, which should do the job nicely.
Forward Image Projection Methodology
This is another approach that has been around a really long time, and unlike chromakeying, this approach requires no special software (except with very advanced setups). Once you have it set up, any ordinary camera will pick it up and carry it to your feed.
If I had a space large enough, I would do this all day long. A somewhat recent example of where this was used is in the making of the SciFi movie Oblivion. In this behind the scenes look at the making of the movie, huge projectors, like the kinds used in iMax theater's, are projected against a screen. One of the cool things about this approach is that the ambient light created by this approach light your subjects, creating a realistic and immersive set.
Even if we don't have the space, or the budget for those big projectors (thousands each), we can still think about how we might use a projector to do background replacement. The idea is pretty straightforward. Take your video and pump it into the projector against your white background. And done! Add a little haze in the studio, and you get a cool effect as the images shine though and reflect off of the particulate matter in the air.
You can even cast the image over the top of your subjects, making for a cool effect. For some examples, check out this Youtube tutorial on using projector's to create cool effects.
The downside of this approach is that the lighting in our studio will shine on the white background and wash it out with the color, so we have to be extra careful with where we aim the lights on our lighting rig to avoid this. Truthfully, it may be impossible to achieve in our small space, so I will not be testing this.
Rear Projection Methodology
If we had the space in our studio to place a projector well behind our subjects and shine the light onto a transparent screen, this would be a pretty great solution. The chief advantage to this way of doing a background replacement over the forward projection approach is that our lighting for the background will be separated from the studio lighting and spill should not be an issue.
Unfortunately, I do not have the space for this, but current technology presents us with another choice; LED Panels.
The stuff they use in the big band live shows and EDM stages are quite expensive. This 6 panel LED system (which isn't even great resolution, but is bright enough for an arena) goes for 9 grand!
We don't need that kind of brightness in our small studio, however, this presents another design choice. We could go with consumer televisions! With one of these HDMI devices, we could take a single HDMI signal and split it into 4, or even 9 different screens. For our screens, we could get 9 50" TV's for around $250 ea., or 4 60" TV's for $500 ea., effectively a 155" screen and 121" screen respectively.
As pricey as this is for a small band budget, it's affordable as a last ditch option if all else fails. Let's see how the reflective fabric option looks once we get it all set up and I'll report back here.