Panasonic Auto Tracking enters virtual production space
By Adam Noyes
AV Magazine recently published a great article going over Panasonic’s Auto Tracking System and how it now goes into the virtual production space.
A leading event and content marketing agency in Norwood, Massachussetts, Cramer is well known for creating impactful virtual experiences for its prominent clients. To improve the workflow and reduce the production time of its virtual productions, the agency has implemented Panasonic Auto Tracking Software, working with the company’s flagship AW-UE150 4K 60p PTZ camera, in an innovative, time-saving way.
When shooting some scenes using a manual jib with the AW-UE150, Cramer was finding it next to impossible to track the subject in traditional ways.
“After some rehearsal, I could use the mouse to stay with the subject but when the talent would move, it was very hard to stay with the movement,” said Brad Richelson, vice president of engineering at Cramer. “Much time was dedicated to rehearsing the shot when tracking in manual. This added significantly to the time that’s been budgeted and scheduled time for the shoot. And when you have many people standing around waiting during the rehearsals, it adds up quickly.”
Richelson explained that he had used an earlier version of Panasonic’s Auto Tracking software and while it was a significant improvement over using a joystick or mouse, it was clear that an increase in precision was needed. “The improvements in the software are working so well that only in special cases do I go back to manually tracking of the subject. Overwhelmingly, most of the time the director will ask me before we start shooting the next scene or shot to keep Auto Tracking on.’”
Another key benefit of the Auto Tracking software, said Richelson, is that it doesn’t get tired. “Auto Tracking doesn’t care that the shoot has already gone on for 10 hours or more where human effort starts to plummet,” he said.
Available for single-camera operation (AW-SF100) or multi-camera operation (AW-SF200), the main use for Panasonic’s Auto Tracking Software has been to support automated lecture and presentation capture with high-precision tracking using motion detection, facial recognition and new body detection powered by Deep Learning Algorithms. The AW-UE150 has been the industry’s choice for a 4K/HD PTZ camera, offering high-end production features including 4K/UHD 60p capture, FreeD protocol for camera tracking in augmented/virtual (AR/VR) reality productions, a V-LOG video output, a wide 75.1-degree viewing angle, SRT protocol support for 4K/60p video streaming, and RTMP/RTMPS support for direct video uploads to live-streaming services such as YouTube and Facebook Live.
In describing how Cramer used the Auto Tracking on a recent production, Richelson said, “We had one subject who at specific points starting dancing. If we did multiple takes, his dancing would change each time – it was not choreographed. Auto Tracking helped significantly with this situation since there’s no way to rehearse this if I wanted to track the subject in manually.”
For its virtual event production, Cramer utilises Pixotope software (for creating virtual studios and AR), and the Unreal Engine (to create virtual sets that can track with a camera’s motion around actors and objects and be rendered in real time). The UE150 is mounted atop a Mo-Sys Startracker base. The UE150, Startracker base, two PC laptops and Pixotope workstation computer are all networked together.
Richelson explained that the agency takes the UE150’s 12G SDI output into an AJA Kona 5 card in the Pixotope computer. They use a high-end, GPU-equipped PC laptop for the Auto Tracking software and another PC laptop using Chrome to change any UE150 settings. For Cramer’s virtual shoots, the UE150 records in 1080/29.97p or 2160/29.97p and content is almost always shot in V-Log.
Cramer is building a mobile AR/VR rack, including the UE150, for virtual shoots
The agency, Richelson added, is building out a mobile AR/VR rack that will contain almost everything, including the UE150, they’ll need to setup virtual shoots, and allow them to quickly set up between their three different studios.