VTV Project: An Integrated Digital Archive System

VTV REQUIREMENTS
VTV’s primary need consist in digitizing its large tape library to subsequently store them in a digital archive system able to initially hold 15,000 hours of media, this system will must provide future retrieval either just for viewing, for repeat transmission, or editing. VTV’s technology department has summarized the characteristics that the solution must include in the following key points: • Capability for a large number of users (operators, producers, etc) to access archived high resolution content including their low resolution copies • Capability of restoring/enhancing poor-quality media before storing it in the archive • Combination of high storage capacity with video formats versatility to deliver content to the various VCTV playout channels • Ability to communicate with Avid NLE systems for allowing producers to retrieve media for editing without having to restore to video tape first • Support for archiving media coming from satellite feeds or live sources • Possibility of archiving media at different resolutions to improve their re-transmission and editing • A workflow that permits operators to decide which media will be archived and which not, as well as monitoring workflow operations in real-time • Added assets value due to the generation of frame-accurate metadata • Complete integration with all media equipment for ingest, preview, storage and playout, and with other internal and external systems under a digital environment.

OVERALL PROPOSED SOLUTION
Etere will provide VTV with an “Integrated Digital Archive System” to store and transparently manage its archive material over the long term in a tape robotic system. Etere will also allow VTV’s editors to access the digital archive directly and efficiently within their digital production environment over the intranet. Etere aims to be the core of VTV’s facility as a total system integrator that ensures the reliability of the global project. An Etere-based digital archive system is able to interface existing sub-systems and encompass future expansions while maintaining the consistent set of characteristics that makes of it the right solution for an enterprise management of digital content under a file-based workflow environment, VTV will be mainly provided with the following key features: • A patented distributed architecture to avoid any single point of failure • Reliable control at high-performances of the equipment integrated within the global system • Enterprise management and transport of media between the storage system and the production environments • Best flexibility on digitizing media content from multiple sources • Robust media management including high resolution files, browse copies and metadata • Seamless integration with existing and co-existing systems • Quality management to ensure the safety of archived assets over short and long terms • Intelligent and customized workflow management • Safe and fully-tracked access to the content archive • Transparent media transferring as a part segment or as a whole, in the correct format into the intended target system • Capability of NLE stations for uploading/downloading contents • Fast and efficient transferring connection between internal –and external- systems. Furthermore, the implementation of an Etere system will provide external agencies and internet editors to access valuable material with the highest standards of security. An important aspect of the selected Etere solution is functionalities which support the preservation and high-availability of media content archived and catalogued by the Etere system. By implementing an Etere solution, VTV will acquire a total integrator able to interface and make work together under a seamless distributed framework the following systems: • 2 ingest servers • 5 HSM managers interfacing 1 tape library • 2 low-res encoding stations • 2 hi-res playout transcode stations • 25 browsing stations • 1 replicated content storage system • 25 web browse stations for NLE systems • 1 playout system.

ETERE ARCHITECTURE
Etere is based on a distributed architecture which allows different modules to run on different workstations interconnected via a local area network. All system configuration parameters, security roles, user data, and pre-defined rules are stored in a reliable SQL database supporting backup and redundancy operations. Etere’s distributed architecture allows achieving a top-level availability of resources and reliability of operations across the entire broadcast workflow thanks to its redundant capabilities to improve the fault resilience on any hardware or software failure. A Distributed System ETERE is a distributed, modular and fully integrated broadcasting system composed by a set of applications specifically oriented to efficiently perform each complex phase of the broadcasting chain synchronously within the same database environment, being all managed by suitable user- defined workflows that ensure an efficient overall system controlling. Multi-level File Access Hierarchy Etere offers a hierarchical storage management by organizing in user-specific access levels an unlimited number of content versions offering ‘instant access’ (from video servers), 5 minutes access (from NLE) and 15 min access (from archives), being all these levels available to the operator under a simple and user-friendly interface. Tape Library Management Etere allows VTV to carry out the management of tapes (i.e.: video tapes, data tapes, discs, etc) by providing them with a set of modules specifically designed to improve the most important tasks involved in the logical management of tapes. Etere provides full support for managing tape libraries within the system, being possible to monitor its status and view specific contents in real-time.

MEDIA MANAGEMENT VIA WORKFLOW
Etere's solution features an integrated and professional workflow management to optimize VTV's entire broadcasting system, reduce operating costs and facilitate overall process control. Etere Workflow permits modules to for example, seek confirmation for sensitive process, follow specific rules, enhance the efficiency and reliability of process, and manage multiple workflows to perform different tasks simultaneously and independently. Multiple Storage Management Etere reduce the complexity of managing storage devices by arranging physical storage devices present across the system into metadevices (logical devices), the use of metadevices improve the overall media management by offering the following features:     ■ Automated management via workflow of logical devices including arching, restoring, transcoding, etc     ■ Monitored storage space owing to the set of restrictions     ■ Increased storage and better performance since metadevices acts as a virtual device representing several logical disks or disk systems.     ■ Distributed storage according to specific requirements without the need of creating partitions, just associate individual disk volumes to different: Custom Design Workflow All workflows can be customized to fit the real needs of the station and thus give complete control over the overall system management which offers:     ■ Clear definition of each complex step of the broadcasting process     ■ Visual representation of each step mapped out on a PC not in a paper document     ■ Set of instructions and authorizations that must be followed in order to move forward     ■ Complete log of all steps carried out, operations denied etc. A comprehensive and user-friendly workspace allows creating suitable workflows based on custom actions just by dragging and dropping the necessary elements into it: Video Files Quality check Workflow An Etere quality check workflow is able to automatically ask operators to assign a quality value to a certain asset(s) after browsing its video content: Video Files Content Check via Workflow Etere counts with a workflow action called content check, that once inserted into a workflow and attached to an asset, searches on its related proxy video file for defective video issues to subsequently mark (into their EDL list) all encountered defective segments including black scenes, scene changes and freeze video. Video Files Archiving Workflow Etere allows you to design a workflow for archiving your assets on LTO tapes, including a quality and content check, a proxy copy generation and a final email indicating the result of the process Video Files Restore Workflow Create a workflow to automatically restore any scheduled asset for its playout by searching for them amongst a group of devices arranged on basis of their priority Video Files Transcoding and Uploading Workflow In case you need to make your assets available for a web-server, just create a workflow that will automatically retrieve them from the archive, transcoding them automatically to a more suitable codec (e.g. low-res mpeg1, wm9, QuickTime, mpeg4, etc) and upload it to a web server Video Server Space Check Workflow A simple workflow will keep the video server always clean of "not scheduled" assets (e.g. on next 24 hours), allowing in addition to except all those which are commercials Video Files Checksum MD5 Verification Etere offers an enterprise control of video files integrity; it keeps a log of the hash md5 of video files such in a way that it is possible to verify at any time if they have been modified after their approval. All video files registered on the Etere’s database can be verified through an md5 checksum, this control is performed via workflow, each time that a video file is moved from one device to another, its initial hash md5 is calculated to allow a future checking. The workflow editor allows creating custom Checksum workflows to either generate or check the MD5 hash of a video file. Video Files Conversion Etere HiRes Transcoder handles a wide array of critical operations including format conversions, workflow operations that can be launched for example, immediately after a content capture.

DETAILED FUNCTIONALITIES
ETERE is an integrated broadcasting solution that implements a modular system formed by a set of modules specifically oriented to cover each complex phase of a broadcasting system, focusing to efficiently carry out specific operations such as media ingest, archiving, transferring, browsing, etc. All these operations are synchronously performed within the same database environment and managed by suitable user-defined workflows that ensure an efficient overall system control; these are some of the main features that make of ETERE a solution that can easily fit VTV’s media management workflow. All modules that make part of the Etere’s proposed solution will be treated throughout this chapter, explaining how its distributed architecture and integrated complementation are key parts of the success of the global system where a top-level performance and reliability is reached. ETERE INGEST: An Enterprise Capturing System Etere Ingest is a versatile set of modular applications that significantly improves the digitization process inside a broadcasting system, this software covers any particular requirement of the entire process such as automatic and scheduled ingest: Etere Ingest supports multiple parallel ingest streams, managed automatically either on a single workstation or across various workstations, allowing also to schedule the video files to be ingested. Etere ingest allows recording satellite signals to any storage target including video tapes, video servers, etc, just by configuring GPI devices to trigger the ingest process. Once ingested, video files are transcoded into the specific format of the destination device on which they will be stored, in the same way; video files are transcoded each time they are moved from one device to another. The image below illustrates how Etere allows setting the workflow to be launched at the start and end of an ingest process to for example, create a low resolution version or normalize the audio of captured contents: ETERE HSM: Integrated Long Term Archiving Etere HSM is the cost-effective solution to radically streamline the management of expensive tape libraries; allowing stations to optimize the migration of contents including high and low versions as well as associated metadata. Etere HSM improves the management of libraries by controlling their mechanical movements through the HSM Robotics Control and HSM Data Pump applications, which are able to run several data pumps on different machines to boost their throughput, while offering access to real-time logs, reports and statistics. Etere HSM distinguish four different archiving levels into a broadcasting workflow, these levels required distinct access times which vary from 0 minutes (video server) to 15 minutes (standard video tapes). All these levels are managed “virtually”, that is, you can use logical devices (metadevices) based on physical devices to free design your storage layout, enriching in this way the entire system with the benefits derived from the use of metadevices: • Carry out loan-balanced movements on an intelligent multi-volume scenario • Extend your storage space by joining physical devices into one metadevice, without altering the archiving workflow • Categorize your storage devices by dividing them into metadevices with no partitioning required • Space limits and storage distribution are defined by the user and not by devices itself • Classify metadevices in media pools in order to automate their management • Background defragmentation and online/offline tape management • Scheduled archiving of devices, media contents and entire databases. Etere HSM forms a tandem with Etere Data Mover to be the only solution in the market with an embedded multi-level and multi-rule cache that offers an intelligent management which ensures the best performances with low investments. Owing to Etere’s comprehensive character, these applications are perfectly integrated with other modules (e.g.: Ingest, EtereWeb, etc) to allow all these modules to use shared resources and have unlimited communication. ETERE MEDIA MANAGER: File-based Data Transfer The Media Management solution proposed to encompass VTV’s content transfer and archiving goes beyond of a simple copy concept by moving video files based on custom policies, transcoding video files when required and offering a full track of all operations. VTV’s video contents will be transferred between the various department (e.g.: near-line storage, archive, post-production, playout, browsing, etc) by Etere Media Manager; this migration process also includes rewrapping and transcoding capabilities. Etere’s approach is oriented to “virtualize” the entire media management process, improving it with flexibility, customization and most important cost-effectiveness. Etere manages (logical) metadevices instead of (physical) devices, this approach results in a wide range of possibilities for the media management, for example, it is possible to control with one click the available space of all metadevices. Etere Data Mover is the application used to perform the physical storage and retrieval of video files, a typical Data Mover operation would be to move a video clip from a video server to an archive based on custom actions which are defined and executed via workflow. Additionally, the crucial logging function is available for all Etere applications, log files are written by the software each time it performs a task so it will be possible to trace their execution status, interaction level, and final result. Log files can be very helpful to understand and solve software and devices problems. Etere MAM: Proxy Preview Etere enables comprehensive search and browse and cataloguing of rich media, its very intuitive interface establishes a bridge between the ingest department and the production department, allowing contents to be browsed simultaneously from various workstations thus enabling low-res proxy browsing over the network. Etere also allows restoring only a part of video files by creating either high or low resolution video files including specific segments described on the source video EDL, it is important to note that as usual on Etere’s operations, the conforming of video files is fully performed via workflow.Conforming can also be performed from different sources, to create a single final video file containing all scenes selected by the operator, use this function to for example perform the dubbing of video contents. The image above illustrates the user-friendly interface on which operators creates new video sequences of MAM captions which includes all relevant metadata associated to the inserted scenes. ETEREWEB: Post-Production Integration Etere Web is the web service seamlessly integrated with the playout and media management system to permit arriving contents to be managed digitally, resulting into a faster and more efficient delivery process which also includes digital signing features for any delivered content. EtereWeb integrates the latest streaming technologies for video distribution and a comprehensive rights management system that gives to authorized users the possibility to access via web to a user-friendly interface. Etere Web works perfectly behind a DMZ router so remote access and ftp transfers are drastically improved. NLE systems can deliver contents via Etere Web as a digital equivalent of physical reception, where selected people can deliver video and metadata to the station, but owing to its digital nature, operations are perfectly organized, performed and logged, avoiding loss of any content information.

CONCLUSIONS
This paper has described how the development and deployment of a comprehensive Etere-based “Integrated Digital Archive System” is able to provide VTV with a large number of operational benefits and advantages derived from the correct use of ultimate media management technology; Etere will entirely support VTV on its migration to a digital environment by providing them with the following key features: • Efficiency, reduced need for repetitive manual operations, allowing to define them in advance and then include them in the ingest workflow, thus increasing productivity • Flexibility, on meeting all requirements by proving a versatile media management system tightly integrated with all capturing and storage devices present on the station • Scalability, for increasing the number of capturing channels and devices without altering the system workflow complexity, thus minimizing operational overheads and reducing overall costs • Workflow Reliability, all operations automatically generates fully customizable logs to track both the overall and individual functioning of the entire system • Accuracy, from the media management to the final delivery, reducing the risk of mistakes when retrieving data since all archived contents are continuously checked.