Game changed - Remote Production
- By Adrian Pennington
Remote production is now a permanent fixture for live events with new technologies coming on stream fast to improve management of synchronisation, latency & bandwidth… By Adrian Pennington
Remote production has been a game-changer for live events but it’s actually a subset of the larger trend toward distributed production. Finding the right distributed production model is a challenge. It differs from on company to another, from one job to another, necessitating different cameras, different networks and different workflows. However, two critical issues will always need addressing: the demand for high quality images and the demand for low latency networks.
“These will only increase as the industry seeks to differentiate itself,” says Norbert Paquet, Head of Live Production, Sony Professional Europe. “Technically speaking, the latency challenge comes with different constraints which technology can resolve either by better quality links or use of low latency compression.”
This is where Sony and Nevion joined forces to look at network environments, such as LAN, WAN, 5G and cloud uplinks/downlinks, “so that we can propose the most suitable solution delivering reliable, secured transport and processing.”
Sony points to “decades” of building solutions for remote and distributed production. Its Media Backbone Hive news production system can be deployed on prem, in cloud or as a hybrid architecture and was recently chosen by Reuters for its international operations.
“Our latest Media Solutions Toolkit is built around a modular approach which allows businesses to choose and combine a number of cloud-native, specialised services, each addressing a particular function, like import or workflow management,” Paquet says.
In Live Production, Sony’s offers a cloud native solution enabling a production to bring any content from anywhere in the cloud, produce live and distribute to multiple web platforms or CDN.
This was adopted by Viacom CBS for the production of some events during the pandemic.
Grass Valley elaborates on this philosophy arguing that the most important aspects of the tech stack are handling timing, latency, and bandwidth. Each of these can be very different depending on the topology of remote production being used.
“On one end of the spectrum, you may have all the production equipment at the venue/event while your operations team control the equipment remotely,” explains Marco Lopez, General Manager, Live Production. “In this case, latency is of critical importance because the team need to make real-time production decisions thousands of miles away from the equipment. Timing is no different than when a truck rolls to the event. Bandwidth is manageable as control and monitoring signals typically use very low bandwidth.
“The other end of the spectrum is where both equipment and operators are centralized in a remote centre. Here, timing is of greater concern because of the sheer number of signals that must be hauled back to the production centre. Depending on the distance travelled, it could take more than 24 hours to get the timing right on international feeds along with the telco service provider.”
Moreover, high-quality, low latency compression like JPEG-XS is needed to cut down on data bandwidth costs, especially when working with 1080p or 4K-UHD formats.
Grass Valley’s answer is to use cloud-based technology, such as GV AMPP. This is built on a series of microservices, with “innate low-latency and intelligent timing management capabilities.”
Lopez says, “AMPP marries broadcast-quality production tools with the agility and elasticity of deploying cloud compute, memory, and storage. It is a collection of many production applications that can be used in a wide variety of different workflows and spun up and down as required in a matter of seconds – ideal for the expanding demands of live. Moreover, GV AMPP supports a wide range of compression technologies, starting from proxy quality for monitoring all the way to contribution and distribution, enabling the creation and delivery of must-watch content.”
Since remote production environments depend on maintaining the timing relationship of various feeds (via IP multiplexing) LTN is developing a codec-agnostic network. This enables customers to migrate over LTN’s IP network and achieve reliable low-latency transport, multi-point delivery, unique monitoring tools, and IP visibility, according to Mike Burk, General Manager, LTN Create.
“One option provides a managed interface to the LTN Network, allowing broadcasters to use their existing hardware investments in encoders, decoders, and multiplexers, and we can add dedicated, reasonably priced hardware if they need it,” he says. “We also offer a turnkey solution that includes all hardware.”
Net Insight views time synchronisation as critical when deploying remote workflows. Video, audio, and graphics from several different remote sources need to be synchronised, says CTO Per Lindgren. Packet loss, latency, and jitter can destroy the content and the ability to remotely manage and control the production. Synchronization techniques can be used to smooth out jitter and wander across IP networks and ensure frame alignment in multi-camera productions.
“Remote production workflows need to set and manage time-stamps accurately and require an accurate time source at both ends of the remote production,” says Lindgren. “For ST2110 workflows then PTP (IEEE1588v2) is used to provide this timing.
“For broadcast-quality production, many use GPS receivers but preferably the network is capable to distribute high-precision time (PTP) to the end-points. When moving to new IP workflows, security aspects are becoming key, especially when running over public telco networks.”
Net Insight introduced the Media Pro App at the beginning of this year to enhance the security and scalability of remote IP workflows. Lindgren says, “With its ultra-low latency 100 Gbps IP Trust Boundary, it addresses the security challenges while ensuring the quality and performance required. The Nimbra products also includes support to distribute PTP transparently over the IP network, without requiring full timing support in every hop.”
In addition, Net Insight has introduced its cloud-based media delivery platform for contribution and primary distribution using internet and cloud, natively supporting ARQ protocols (RIST, SRT, Zixi). Nimbra Edge includes support for stream alignment and improved security and is now also integrated into cloud production platforms such as Grass Valley AMPP and Simplylive.
Software capabilities for remote production
Over the last year broadcasters have sent as few people as possible to venues. This extends to camera operators located at the studio rather than the venue running pan/tilt capabilities. For local area video, Media Links is also seeing switch capabilities managed remotely.
“This can literally be done from someone’s home with a secure connection into the Network Management System (NMS) to then operate specific links, observe their provisioned links or trouble-shoot,” says John Dale, Director/CMO.
This transition to a software and controllable ecosystem is likely to be as important in future as customers want to continue having fewer people in the field supporting an increasing number of sporting events.
“Managing all of the different signal types and locations can be a daunting task, which is where a robust and interoperable NMS comes into play,” says Dale. “Network operators and broadcasters can quickly bring new services online for studio, remote and field locations, all while monitoring and maintaining the reliability of active circuits, trunks, and network devices
“Media Links’ ProMD EMS system offers broadcasters a comprehensive tool for media network management including service scheduling, operational monitoring, troubleshooting and so much more. This ensures network providers and broadcasters can maintain and operate their mission critical, contribution quality media over IP networks. Multi-level topology views and cross-linked device, card, circuit, and log screens make service provisioning and proactive troubleshooting over the network fast and accurate.”
As remote production workflows become standardised, Riedel says better tools are needed to ensure that non-technical, creative people can do their job effectively.
“We need streamlined comms with intuitive UIs and straightforward signalling,” says Carsten Voßkühler, Senior Project Manager. “Improved automated signal processing and remotely controllable solutions will also be crucial. Another great way to assist creative folks whenever they feel overwhelmed by the technical complexities of remote productions is remote support, as provided by our Remote Operations Center (ROC).”
The ROC is a described as a one-of-a-kind communications and signal transmission hub that enables remote monitoring of multiple channels of video, audio, telemetry and other data, and even remote control of systems and networks.
“Staffed 24/7 by a knowledgeable team, and equipped with 12 workspaces boasting the latest Riedel gear, the ROC gives broadcasters flexibility to address multiple applications and to adapt instantly to the changing needs,” says Voßkühler.
Riedel’s ROC has supported countless live sports productions this past year including the “world’s first fully distributed live event production” - ‘Around the World in 80 ms’. For this virtual live event, the ROC enabled all audio, camera and lighting operators to work remotely and in realtime across three continents.
Faced with constantly changing production demands, broadcasters and service providers need equipment that can adapt, no matter if the underlying infrastructure is on-premise or off-premise, local or remote, or even cloud. Lawo believes this requires efficient and secure management of IP broadcast infrastructures with features including automated discovery and registration of devices, connection management, flow control, software and firmware management, scalability and security.
HOME, Lawo’s management platform for IP based media, addressed this. It is natively built on a “cloud-ready microservices architecture, enabling users to connect, manage and secure networked production setups from the ground up.” It provides centralised access and control for all Lawo gear within a setup.
“HOME helps broadcast professionals address some of the most demanding requirements of modern IP infrastructures, including automated discovery and registration of devices, connection management, flow control, software and firmware management, scalability and security,” says Wolfgang Huber. “HOME significantly increases efficiency in setting up IP system installs and provides the architecture for our customers to scale with their agile business requirements.
The platform is based on open standards such as ST2110, NMOS, IEEE802.1x and RADIUS and follows LUX, Lawo’s unified experience design principles, which provide a consistent workflow across all Lawo IP products.
LiveU’s VP Marketing, Ronen Artman, says, “We don’t see any major issues that need addressing at this juncture because most have already been resolved. Pre-pandemic, we had already seen significant growth in the use of LiveU technology for remote productions, given the fundamental suitability of our solutions and IP-based service ecosystem.”
He says, “It’s more a case at the moment of understanding exactly what type and scale of remote production a customer wants to achieve and then working closely with them at all stages to do precisely that. Then there’s the promise that 5G, in conjunction with remote production, offers. We strongly believe that we are still only at the tip of the iceberg.”
LiveU’s LU800 is an all-in-one production-level field unit designed for live news and dynamic sports coverage. “Multi-camera production and superior video and audio capabilities are combined with mission-critical transmission in a native 5G unit,” Artman says.
It supports up to four fully frame-synced feeds in high resolution from a single unit, using powerful IP bonding of up to 14 connections. The unit also offers up to 4Kp60 10-bit HDR transmission, as well as up to 16 audio channels for high-end productions.
LiveU have a range of cloud partnerships, with pre-integrated solutions, “to provide dynamic and flexible live workflows” including with Avid, Grass Valley, Blackbird, Haivision, Grabyo, and Vizrt.
