Going Virtual: Beyond REMI Sports Production
By Adam Noyes
LOS ANGELES–About a year ago I wrote about REMI (REMote Integration) sports production (also known as “At Home”), which is a trend for covering live sports that requires less equipment and personnel on-site and shifts much of the live production activities to the broadcaster’s central production center/network center.
Enabled by robust and affordable telecommunications capabilities from the stadium/arena, the camera (and microphone) signals are sent back to the central production center, integrated with graphics, replays, effects and b-roll materials. Usually a small utility truck is sent to the remote site, with cameras and microphones as well as video encoders.
Local camera operators and utility crew roll out the cameras and microphones while an A2 is usually on-site (and may have driven the truck) to set up coms and in some cases provide a sub-mix of the natural sound (crowd). Examples of this type of production are used extensively in college sports (except football and men’s basketball) as well as some professional sports. In smaller productions (1-5 cameras) the truck is replaced by a portable field pack.
BEYOND THE CENTRAL PRODUCTION CENTER
However in many cases REMI still uses first-class professional cameras and production gear. These require operators who are well trained in the use of this equipment. For some sports, this is still a high bar in terms of cost, especially the huge capital cost to build such facilities.
In response to REMI, a new type of live sports production has evolved, one that does not need a central production facility and therefore can be produced anywhere there is a solid internet (IP) connection. Further, instead of hard-wired cameras and microphones, this new production called “Virtual Production” by Sony uses wireless connectivity from camera (and microphone) to a cloud-based production service. No heavy production equipment, no wires!
The sports event can be produced via an app on a tablet, laptop PC or even a smartphone. The “control point” can be augmented with multiple video monitors and audio speakers/headsets. The solution even includes return channels for communications (IFB) and video returns. Further, there can be multiple productions simultaneously using the same camera and mic feeds but outputting specialized line cuts. Finally, this may not be limited to just a professional producer, the same cameras and audio can be available to fans to generate a personalized feed that can be shared via social media.
Taking advantage of this freedom of wires and professional gear, even professional productions can cover parts of an event that are near impossible to cover using conventional TV coverage techniques. For instance, at the Red Bull Alpenbrevet alpine motorcycle race where there isn’t space or room for a production vehicle, a roving camera operator–connected via 4G wireless modem–can cover any part of the event that is accessible.
REQUIREMENTS
Is this Virtual Production (VP) the future of live sports coverage? What are the challenges and downsides of going virtual in the cloud? Some necessary technologies that enable Virtual Production are large wireless bandwidth on site with low latency, connectivity to the internet that is reliable and high bandwidth, and a VP service available and affordable in a data center relatively close to the event (to minimize delays). Finally, what is also needed is a production client app that can enable all the features needed by a production, with low latency of both signals and commands, that can run on typical laptops/tablets/smartphones with online support during the production in case of problems as well as help in setting up the VP service.
For now the offering from Sony is somewhat limited to switching up to six camera feeds, graphics insertion, limited effects (picture in picture, transitions) and audio mixing. Camera synchronization–and to some extent error recovery–requires around 2 seconds of delay, which is deemed to be acceptable for some productions. The other unique advantages that VP provides is a “pay as you go” approach to cost, thanks to implementing VP on the Amazon AWS cloud. For those interested in delivering via the internet using social media live features, this approach has a clear advantage of already being in the cloud. In addition, new features and functions can be rolled out without bringing the truck back to base and wait for the retrofit. Once upgraded on the hosted servers, the new features are immediately available.
[Read: HPA 2016: Remote Live Production]
The costs associated with VP are based on a monthly subscription for a certain number of hours of production. Included are up to six camera feeds, two playout servers, two logo keyers as well as the VPS production user interface. Additional costs may occur due to the wireless camera feeds that use wireless carrier 3G/4G (and in the near future 5G) data services. Finally, there is the cost to distribute the live feed over the internet, which could be free for some social media services (i.e. YouTube Live) or if using a commercial grade CDN such as Akamai, the cost is typically based on the number of concurrent streams. In addition, if the show is archived there will be costs to store the content and cost to retrieve it later.
OTHER VARIATIONS ON THE VP MEME
While the Sony VPS is a full IP/cloud-based approach using wireless cameras, there are other applications that take advantage of the new video technologies. NEP in their facility outside Amsterdam, offers a production facility for REMI production using their own data center, connected via fiber to major stadiums and arenas. The production facilities are rented (much like a remote truck) but only charged for the time used by the production. NEP has also used wireless IP cameras using point-to-point links (Ubiquiti Wireless) at the Coachella music festival to enhance coverage of the event. The use of IP connectivity allowed the production to remote control the cameras (Panasonic PTZ type). This obviated the need to run fiber optic cables to each camera in order to cover the sprawling event.
Teradek has a live production solution called Live: Air Action. Using their bonded wireless encoder/modems(VidiU or CUBE), connecting cameras (including iPhones/Android Smartphones) via a cloud server (Teradek CORE Integration) the Live:Air Action application lets a producer switch cameras, with limited effects, add graphics, record and playback, and has an audio mixer. Stream live over social media sites or via CORE.
Globo TV, the biggest TV network in Brazil, wanted to cover The World Surf League 2017 finals from Honolulu. Using three Sony FS5 cameras with VidiU Pro encoders with 4G LTE modems, Globo integrated with Live: Air Action to switch the cameras and stream using a Bond 4G/LTE modem to connect to Facebook live. Live:Air Action enabled the producers to add graphics including stats as well as integrate commentary and interviews with the third camera feed. Using the iPad app for Live:Air Action meant the producers could be close to the beach and the live event.
Newtek Tricaster has been used for years to provide alternative coverage, usually for social media or OTT streaming. They have developed an IP video networking environment, NDI, which allows for the use of IP cameras as well as interconnecting third-party equipment.
Recently Thaler Media used Tricaster and NDI along with LiveU wireless camera systems to cover the 2018 Western Amateur Championship in Northridge Illinois (near Chicago) while producing the live feeds at their facilities located in North Palm Beach Fla. The streams were sent to the Golf Channel’s digital platform.
On site at the Sunset Ridge Country Club were a producer, editor, four camera operators and sound mixer. Three of the cameras were mobile and used LiveU’s LU600 HEVC and LU500 IP-based transmission system to send the camera signals back to a LU2000 server located at the master control facility in Palm Beach.
The camera of choice was a Sony F55, which provided first class imagery. They set up a talk show set in the Sunset Ridge clubhouse and had the program’s co-anchors provided commentary and analysis, while a virtual set (greenscreen) provided the set background. An additional third anchor position was set up at the master control room in Florida, where the local program anchors could throw to the anchor at the main control studio. Tying it all together was a NewTek TriCaster TC1 IP production switcher with 16 inputs and 4 M/E decks, chromakey, graphics/titles, transitions, video server and virtual set support. NewTek 3Play replay systems provided SLOMO replays as well as aerial footage captured by camera flying on a drone over the course. Four Skype channels were used to provide return feeds to the Chicago on-air announcers including mix-minus and IFB.
While not up to the level of a major PGA event broadcast coverage, for the first time the Western Amateur Championship was covered live in HD, an event that in the past drew future golf greats including Tiger Woods, Jack Nicklaus, Phil Mickelson and Ben Crenshaw.
HOW WILL SPORTS BROADCAST PRODUCTION EVOLVE?
Clearly today there are many choices to capture live events in video, from big broadcast TV trucks to REMI to REMI in the cloud and finally, smartphone cameras and live video apps. For large events, with commensurate rights fees and audience sizes, professional broadcast OB vans, connected via satellite and/or fiber-optic telecom, will be the choice of live production for the foreseeable future.
On the other end, with all the available tech available in our hands (smartphones) coupled with social media sites providing distribution for live video, your child’s AYSO soccer match can be covered with multiple cameras, graphics and replays (and sound) to be seen by your friends and family on YouTube or any of the other social websites. In between big time pro sports productions events and Little League games, productions are using lower-cost solutions that use mobile wireless networks or WiFi networks, along with cloud-based apps to produce a video program that has many of the aspects of the big pro productions as well as providing professional level tools such as graphics and stats, IFB, and replays. Ingenious production teams, motivated to cover events that in the past were not economically viable for live coverage, are leveraging these new tools and providing to modest sized audience, content that in the past would have been just highlight clips in the evenings sportscast.
The keys to success is finding the right balance between cost and coverage, having robust wireless connectivity, reliable and deterministic Internet bandwidth, and a team able to adapt and learn this new way of producing live content.