A Central Archive Management System

For the full report on a central archive management system, please refer to the attachment.

Introduction
Etere provides broadcasters, media companies, content providers and house productions with an integrated archive and distribution system which combines powerful servers with an easy to use user interface. Etere is able to closely interact with other systems present within the global solution; it allows searching, browsing and delivering media files stored across the various subsystems present in the global solution.

This paper describes Etere as a total system integrator, a solution that ensures the reliability of the global project, implementing a robust media archive system able to provide instant access to any file, at anytime, from any server and most importantly, with the maximum of speed, characteristics that will ensure the improvement of each single broadcasting area with a wide set of cutting edge modules that goes from content acquisition to content delivery.

2. ETERE SOLUTION: OVERVIEW
Etere’s solution consist of a system intended to be the core of the central archive, providing media archive management and delivery services to the global system including instant access and delivery of media files; all this media management is cemented on a file-based workflow framework featuring a wide range of function-specific workflows for ingest, production and playout with additional integration capabilities. The modules that Etere will implement across the system are briefly described below:
■ Etere MAM, a rock-solid central archive completely based on an Etere system that will be the core of the media management system, able to handle a commercial archive of about 100'000 ads,
■ Etere Ingest, Implementation of two content capturing channels with support for future scalability,
■ Etere Media Manager, Guaranteed timing and effectiveness on media transfers between devices,
■ Etere Workflow, automatic workflow-based processes for a centralized and monitored execution, including federated management of assets from their search and browsing to their delivery,
■ Etere HSM, An automatic tape-based storage system for long-term archiving,
■ Etere Automation, a fault tolerant playout automation system to send on-air all daily schedules made up of commercial and non-commercial programs,
■ Etere Asset Management, an ensured tight integration with the global system including existing systems such as the archive server (Dayang NLE), the news server (Final Cut Pro), the production server (Final Cut Pro) and the satellite TV 8 CH server (Final Cut Pro).

An Etere-based central archive system is able to interface existing systems and encompass future expansions while maintaining the consistent set of characteristics that makes of it the right solution for an enterprise management of digital content under a file-based workflow environment; the station will be mainly provided with the following key features:
■ A patented distributed architecture to avoid any single point of failure,
■ Reliable control at high-performances of the equipment integrated within the global system,
■ Enterprise management and transport of media between storage devices,
■ Best flexibility on digitizing media content from multiple sources,
■ Robust media management including high resolution files, browse copies and metadata,
■ Seamless integration with existing and co-existing systems,
■ Quality management to ensure the safety of archived assets over short and long terms,
■ Intelligent and customized workflow management,
■ Safe and fully-tracked access to the content archive,
■ Transparent media transferring, the correct media format will be always delivered,
■ Capability of NLE stations for uploading/downloading contents,
■ Fast and efficient transferring connection between internal –and external- systems.
An important aspect of the selected Etere solution is the wide range of functionalities which support the preservation and high-availability of media content archived and catalogued by the Etere system.

Furthermore, the implementation of an Etere system will integrate the tape robotic systems present in the station to store and transparently manage the long-term media archiving; NLE systems will be able to access the digital archive directly and efficiently through a proper production environment, making use of the highest security standards.

3. ETERE SOLUTION: ARCHITECTURE
Etere is based on a distributed architecture which allows different modules to run on different workstations interconnected via a local area network. All system configuration parameters, security roles, user data, and pre-defined rules are stored in a reliable SQL database supporting backup and redundancy operations.

3.1 A Distributed System
ETERE is a distributed, modular and fully integrated broadcasting system composed by a set of applications specifically oriented to efficiently perform each complex phase of the broadcasting chain synchronously within the same database environment, being all managed by suitable user-defined workflows that ensure an efficient overall system controlling.

Etere’s distributed architecture allows achieving a top-level availability of resources and reliability of operations across the entire broadcast workflow thanks to its redundant capabilities to improve the fault resilience on any hardware or software failure.

3.2 A Federated Management
Etere’s federated capabilities will permit the station to take full advantage of federated technologies, allowing them to import/export assets between systems such in a way that it will be possible to manage external assets as local ones. The diagram below clearly illustrates how Etere allows to handle external assets and also to orchestrate external workflows:

Etere's enhanced federated capabilities allow integrating different systems and provide the global system with the following features:

■ An extensible access and extended catalog with optimized retrieval performances,
■ Search, open and manage assets’ data and content stored across diverse sources,
■ Centralized processing of media data with further replication over different systems,

The local Etere system connect itself with federated systems making best use of currently available proven technologies such as SQL (federated search), XML (federated import/export) and web services (federated workflows).

3.2.1 Federated Search
Etere implements a 'federated search' system based on Microsoft SQL technology, it permits to search across assets belonging to different systems. During the search it is possible to highlight assets according to their source installation and once retrieved, they can be managed as local assets for almost all intents and purposes (i.e.: consultation, workflows, etc).

3.2.2 Federated Workflows
Etere's advanced workflow management allows all data and digital content of all external installations (repositories) to be able to participate in the federated workflow processes, that is, it will be possible to execute workflows for external assets locally.

The federated workflow management is based on Etere Web Services; a web service that permits to remotely execute workflows, this key feature will take the station to a top media management level in which media files can be transferred remotely.

3.2.3 Federated Import/Export
The federated management is based on the import of external assets into the local system; import/export can be automatically and periodically performed according to the needs of the station. Asset's data included during this process comprehend local ID, external ID, code, description, type, metadata, etc. Once defined (i.e.: imported), federated installations can be customized by assigning them a descriptive code, icon and color for a differentiated visualization and easier recognition.

3.3 Multi-level File Access Hierarchy
Etere offers a hierarchical storage management by organizing in user-specific access levels an unlimited number of content versions offering ‘instant access’ (from video servers), 5 minutes access (from NLE) and 15 min access (from archives), being all these levels available to the operator under a simple and user-friendly interface:

3.4 Tape Library Management
Etere allows stations to carry out the management of tapes (i.e.: video tapes, data tapes, discs, etc) by providing them with a set of modules specifically designed to improve the most important tasks involved in the logical management of tapes:

Etere provides full support for managing tape libraries within the system, being possible to monitor its status and view specific contents in real-time:

4. ETERE SOLUTION: MEDIA FUNCTIONALITIES
Etere’s solution features an integrated and professional approach based on a workflow management to optimize the station’s entire broadcasting system, reduce operating costs and facilitate overall process control. Etere Workflow permits modules to for example, seek confirmation for sensitive process, follow specific rules, enhance the efficiency and reliability of process, and manage multiple workflows to perform different tasks simultaneously and independently.

4.1 Multiple Storage Management
Etere reduce the complexity of managing storage devices by arranging physical storage devices present across the system into metadevices (logical devices), the use of metadevices improve the overall media management by offering the following features:
■ Automated management via workflow of logical devices including arching, restoring, transcoding, etc,
■ Monitored storage space owing to the set of restrictions,
■ Increased storage and better performance since metadevices acts as a virtual device representing several logical disks or disk systems
■ Distributed storage according to specific requirements without the need of creating partitions, just associate individual disk volumes to different

4.2 Custom Design Workflow
All workflows can be customized to fit the real needs of the station and thus give complete control over the overall system management which offers:
■ Clear definition of each complex step of the broadcasting process,
■ Visual representation of each step mapped out on a PC not in a paper document,
■ Set of instructions and authorizations that must be followed in order to move forward,
■ Complete log of all steps carried out, operations denied etc.
A comprehensive and user-friendly workspace allows creating suitable workflows based on custom actions just by dragging and dropping the necessary elements into it:

4.3 Integration between workflows
Etere allows calling workflows from others just by inserting an action that can perform a specific task (i.e.: attach, attach and start, start, abort, reset, restart, detach, etc) on a certain workflow:

4.4 Remote launching of external workflows
Etere counts with predefined templates to allow an easy creation of workflows able to launch operations on other Etere installations, Etere will connect with remote installations using Etere Web Services, a web service that makes the most important Etere functions (e.g. data consultation, workflow execution, etc) available to other systems through the use of standardized XML messages. Defining and launching remote workflows is an easy task with Etere; it needs following four steps only: Once launched, the remote workflow will be started remotely.

4.5 Video Files Quality check Workflow
An Etere quality check workflow is able to automatically ask operators to assign a quality value to a certain asset(s) after browsing its video content:

4.6 Video Files Content Check via Workflow
Etere counts with a workflow action called content check, that once inserted into a workflow and attached to an asset, searches on its related proxy video file for defective video issues to subsequently mark (into their EDL list) all encountered defective segments including black scenes, scene changes and freeze video:

4.7 Video Files Archiving Workflow
Etere allows you to design a workflow for archiving your assets on LTO tapes, including a quality and content check, a proxy copy generation and a final email indicating the result of the process:

4.8 Video Files Restore Workflow
Create a workflow to automatically restore any scheduled asset for its playout by searching for them amongst a group of devices arranged on basis of their priority:

4.9 Video Files Transcoding and Uploading Workflow
In case you need to make your assets available for a web-server, just create a workflow that will automatically retrieve them from the archive, transcoding them automatically to a more suitable codec (e.g.: low-res mpeg1, wm9, QuickTime, mpeg4, etc) and upload it to a web server:

4.10 Video Files Checksum MD5 Verification
Etere offers an enterprise control of video files integrity; it keeps a log of the hash md5 of video files such in a way that it is possible to verify at any time if they have been modified after their approval. All video files registered on the Etere’s database can be verified through an md5 checksum, this control is performed via workflow, each time that a video file is moved from one device to another, its initial hash md5 is calculated to allow a future checking:
The workflow editor allows creating custom Checksum workflows to either generate or check the MD5 hash of a video file.

5. ETERE SOLUTION: COMPONENTS
ETERE is an integrated broadcasting solution that implements a modular system formed by a set of modules specifically oriented to cover each complex phase of a broadcasting system, focusing to efficiently carry out specific operations such as media ingest, archiving, transferring, browsing, etc. All these operations are synchronously performed within the same database environment and managed by suitable user-defined workflows that ensure an efficient overall system control; these are some of the main features that make of ETERE a solution that can easily fit any media management workflow.

All modules that make part of the Etere’s proposed solution will be treated throughout this chapter, explaining how its distributed architecture and integrated complementation are key parts of the success of the global system where a top-level performance and reliability is reached.

5.1 ETERE INGEST: An Enterprise Capturing System
Two ingest channels will be implemented in the Central station Archive, they will be managed by Etere Ingest, a versatile set of modular applications that significantly improves the digitization process inside a broadcasting system, this software covers any particular requirement of the entire process such as automatic and scheduled ingest.

Etere Ingest supports multiple parallel ingest streams, managed automatically either on a single workstation or across various workstations, allowing also to schedule the video files to be ingested.

Etere ingest allows recording satellite signals to any storage target including video tapes, video servers, etc, just by configuring GPI devices to trigger the ingest process. Etere Ingest permits to capture contents from a wide range of video storage devices including the broadcast industry standard Sony XDCAM.

Etere counts with a XDCAM-dedicated module to acquire contents from professional XDCAM discs either manually or automatically, via SDI or FTP:

Moreover, Panasonic’s DVCPRO P2 series products are fully supported by Etere, that access either the PC or the NLE system where the Panasonic P2 cards are placed, permitting operators to manage them as simple metadevices with immediate availability (e.g.: via LAN, FTP, etc).

Once ingested, video files are transcoded into the specific format of the destination device on which they will be stored, in the same way; video files are transcoded each time they are moved from one device to another.

The image below illustrates how Etere allows setting the workflow to be launched at the start and end of an ingest process to for example, create a low resolution version or normalize the audio of captured contents:

5.2 ETERE MEDIA MANAGER: A Digital Archiving and Delivery
The Media Management solution proposed to encompass station’s content transfer and archiving goes beyond of a simple copy concept by moving video files based on custom policies, transcoding video files when required and offering a full track of all operations.

Video contents will be transferred between the various departments (e.g.: near-line storage, archive, post-production, playout, browsing, and even non-Etere systems) by Etere Media Manager; this migration process also includes rewrapping and transcoding capabilities. Etere’s approach is oriented to “virtualize” the entire media management process, improving it with flexibility, customization and most important cost-effectiveness.

Etere manages (logical) metadevices instead of (physical) devices, this approach results in a wide range of possibilities for the media management, for example, it is possible to control with one click the available space of all metadevices:

Etere Data Mover is the application used to perform the physical storage and retrieval of video files, a typical Data Mover operation would be to move a video clip from a video server to an archive based on custom actions which are defined and executed via workflow.

Additionally, the crucial logging function is available for all Etere applications, log files are written by the software each time it performs a task so it will be possible to trace their execution status, interaction level, and final result.

5.3 ETERE HSM: A Tape Based Archiving
The Quantum tape libraries present in the station will be managed by Etere HSM, the cost-effective solution to radically streamline the management of expensive tape libraries; allowing stations to optimize the migration of contents including high and low versions as well as associated metadata.

Etere HSM improves the management of libraries by controlling their mechanical movements through the HSM Robotics Control and HSM Data Pump applications, which are able to run several data pumps on different machines to boost their throughput, while offering access to real-time logs, reports and statistics.

Etere manages (logical) metadevices instead of (physical) devices, this approach results in a wide range of possibilities for the media management, for example, it is possible to control with one click the available space of all metadevices:

Etere Data Mover is the application used to perform the physical storage and retrieval of video files, a typical Data Mover operation would be to move a video clip from a video server to an archive based on custom actions which are defined and executed via workflow.

Additionally, the crucial logging function is available for all Etere applications, log files are written by the software each time it performs a task so it will be possible to trace their execution status, interaction level, and final result.

5.3 ETERE HSM: A Tape Based Archiving
The Quantum tape libraries present in the station will be managed by Etere HSM, the cost-effective solution to radically streamline the management of expensive tape libraries; allowing stations to optimize the migration of contents including high and low versions as well as associated metadata.

Etere HSM improves the management of libraries by controlling their mechanical movements through the HSM Robotics Control and HSM Data Pump applications, which are able to run several data pumps on different machines to boost their throughput, while offering access to real-time logs, reports and statistics. ) to allow all these modules to use shared resources and have unlimited communication.

5.4 ETERE WEB SERVICES: Integration with another systems
Etere Web Services is a web application programming interface provided by Etere to render some of the most Etere's important functions (e.g. assets management, workflow control, etc) available to mainly Non-Etere programs through the use of standardized messages, thus, Etere allows to the Stations to communicate its own software with Etere.

The station will be provided with a reliable and easy-to-use application based on cutting-edge technology for web services, improving the overall Etere’s distributed architecture for bringing you a quality service, this service is accurately illustrated in the image above.

In essence, through Etere Web Services, a client invokes the web service by sending an XML message, then waits for a corresponding XML response which can be a data retrieval (e.g. a list of certain assets) or a function execution (e.g. an asset deletion).

5.5 ETERE AUTOMATION: Playout System
Etere Automation is the powerful, reliable and modular playout system able to enhance TPI’s potential in terms of functions and workflow design, it is based on a unique approach which combines in a single product real-time device control and media asset management, offering a powerful mix of solutions and capabilities under a graphical user-friendly interface displaying for each event its source, type, description, properties, live status, secondary events, time code, GPI status, scheduled and real times, etc:

5.5.1 Secondary Events Management
Etere Automation manages all the secondary events intended to be transmitted by dedicated devices (e.g.: Logo Generators, Crawl Generators, Subtitlers, etc) with a simple graphical tool, allowing previewing secondary events in low res before their playout through a browsing application:

5.5.2 Live Events Management
Etere Automation offers complete support for live events present on the daily schedule, being possible to manage various different live inputs that can be switched at any time, few minutes before the event broadcasting or even during its transmission:

Additionally, Etere allows managing one video router per automation, being possible to create links between routers so when a channel is switched in the Main Router; the equivalent channel is also switched in the Backup Router:

5.5.3 As-Run Logging
Moreover, the ability to export As-Run logs containing the schedule “actually” transmitted allows an easy reconciliation between planned and real playout, being possible to send to multiple (UNC and FTP) destinations a frame-accurate log in any of the available formats.

6. Conclusions
This paper has described how the development and deployment of a comprehensive Etere-based “Central Archive Management System” is able to provide the station with a large number of operational benefits and advantages derived from the efficient use of ultimate media management technology; Etere will manage the digital contents of the station, from acquisition to delivery, while tightly integrating other non-Etere systems thanks to the use of federated capabilities that will allow the integration of web services. The many key benefits that Etere will provide to the station have been condensed in the following points:

■ Reliability, workflow-based operations from ingest to playout that permits to monitor the individual system functioning while increasing productivity,

■ Integration, a global distributed system with the power of a single system that combines Etere’s core functions with the capabilities of other existing systems,

■ Federation, a federated data server that permits to query distributed data as if it were in the local system,

■ Scalability, for increasing the number of capturing channels and devices without altering the system workflow complexity, thus minimizing operational overheads and reducing overall costs,

■ Efficiency, during the whole media management process, reducing the risk of mistakes when retrieving data since all archived contents are continuously and accurately checked.

To download the complete case study, please refer to the attached PDF.