BackGCI: Digital Archive Solution
10 October 2008GCI is looking to move towards a tapeless broadcasting environment by carrying out the implementation of an effective Digital Archive Solution.
INTRODUCTION
GCI is looking to move towards a tapeless broadcasting environment by carrying out the implementation of an effective Digital Archive Solution. Such a solution will streamline the ingest, storage and retrieval of digital data as well as reducing the time and cost associated with content production resulting in an optimization of media asset management. The digital archive needs to be accessible (both remotely and centrally) to three main customers:
1. Universal
2. Digicast
3. Italian Tennis Channel
The current size of the archive is approximately 20,000 tapes predominantly made up of digital betacam, however it also includes analog beta and sx. This proposal looks at an end-to-end digital archive solution from Ingest through to Archive and Media Asset Management. The key benefits of the proposed system are as follows:
- Reduced operational costs
- Increased operational efficiency
- Reliable and accessible archive system
- Increased return on investment of media content
GCI_Digital_Archive_Solution.pdf
The principal objectives that must be met for GCI’s Digital Archive Solution are as follows:
- Speed in the ingest and digitization of video content
- Integrated and highly effective quality control check
- Automatic generation of multiformat, multipurpose proxy files
- Seamless integration with the existing Automation system
- Effective long-term archive of digitized content with the possibility of both nearline and deep archive
- Frame by frame content indexing with full SMPTE dictionary support
- Effective integration with post production
- Complete archive content visibility (browsing) and control via intranet and internet portals for multi-user access.
One of the key issues that GCI’s customers face is the dispersed nature of their users. It is imperative that users, wherever they are, are able to access the media archive remotely in order to make amends or take decisions based on real time information. The following document outlines how the above objectives can be achieved through Etere software and details the key system requirements that will be met with the proposed Digital Archive system.
- Speed in the ingest and digitization of video content
- Integrated and highly effective quality control check
- Automatic generation of multiformat, multipurpose proxy files
- Seamless integration with the existing Automation system
- Effective long-term archive of digitized content with the possibility of both nearline and deep archive
- Frame by frame content indexing with full SMPTE dictionary support
- Effective integration with post production
- Complete archive content visibility (browsing) and control via intranet and internet portals for multi-user access.
One of the key issues that GCI’s customers face is the dispersed nature of their users. It is imperative that users, wherever they are, are able to access the media archive remotely in order to make amends or take decisions based on real time information. The following document outlines how the above objectives can be achieved through Etere software and details the key system requirements that will be met with the proposed Digital Archive system.
Flexibility
The system must be as open, scalable and flexible as possible in order to encompass future growth and changes.
The solution will provide a consistent and reliable system that supports basic functions and both manual and automatic operation modes.
Ingest and Digitization
Current tape content (approximately 30,000 hours) must be ingested and ready for playout within a strict timeframe (approximately 3 months).
Given these current constraints, speed of ingest and digitization is key. In the proposed system the automatic ingest of existing tape content will be carried out using multiple flexicarts. The proposed figure is for 4 flexicarts to be used (one flexicart has the capacity to ingest 80 hrs/day and therefore 2400 hrs/month of video content). Multiple flexicarts can be managed and controlled by Etere using its intuitive VCC Controller and Etere Cart application. The latter ensures complete visibility over the status of content ingest and allows for efficient automated digitization. It must be noted that the Etere is also able to manage the flexicart should in the future the decision be taken to use it as a VTR for direct playout. The flexible nature of Etere Resource Editor allows complete user-defined configuration of the Flexicart.
Workflow
During the ingest process and before any content is archived it is both quality checked and transcoded. The below shows the proposed workflow relevant to the digitization process of the media content
Quality Control
Media content quality is fundamental, however quality control checks can be time consuming and often dependant on the professionalism of the operator in question. In order to produce flawless media content, there are 2 principle levels of quality control that the media content will go through during and after the ingest and digitization process.
Tape Error Control
The VTR error system integrated in the flexicart logs any tape errors by inserting cue points. This automated error system greatly increases operator efficiency as only the cue points need to be checked in order to approve the ‘error-free’ files.
Content Quality Control
Following ingest the unique, highly effective and fully integrated Etere Content Management System (CMS) automatically controls the quality of the ingested media. It can run without an operator being present and can run 8x faster that real time on a P4 1G CPU. The quality check process is fully configurable and uses standard MS Windows XP OS and multiple videos can be run on the same facility simultaneously. Etere CMS checks the media content in order to identify:
- Black video
- Freezed video
- No audio
- Scene changes entry points
Only once both the tape error control and the content quality control have been passed and the latter one approved by the operator, will the media content be accepted for archive ensuring quality standards are met every time.
Transcoding
Optimizing the ingest process is not just about transferring data from one storage point to another but it is also about maximizing the potential of your media content and making it available to multiple users via multiple channels. The proposed system is able to generate multiformat, multipurpose proxy files using the Etere Transcoding Engine. These frame accurate proxy files are not only important for the repurposing of said media in order to be able to serve multiple channels (intranet, internet, mobile phones etc) but also from an integral part of content media management. K2 provides the transcoding needed to do with Ampstream software. The following gives an example of the most basic formats that can be transcoded using the proposed system:
- AVC intra
- MPEG2 422 and 420
- DV
- DVCPRO wrapped in MXF, AVI or quicktime
Media Content Management
Effective media content management should both reduce the operational cost of content production and facilitate the overall handling of a company’s media assets. With a key focus on increasing the return on investment of said assets, the Etere MAM application integrates perfectly into this proposal. A fundamental benefit of an effective digital library is the ability and speed at which media content can be retrieved and made available for re-use.
The optimization of a digital workflow requires a comprehensive understanding of the importance of the following content management tools:
1. Indexing
2. Search & Retrieval
3. Browsing
4. Asset Management
5. User Interface
Indexing
Accurate media content inventory is dependent on the logging capabilities of the respective process via which the media content is ingested. The proposed system offers advanced indexing tools with, amongst others, the following capabilities:
- Basic existing metadata is inserted automatically during the ingest process
- Manual insertion of an unlimited amount of metadata
- Configurable metadata classifications and sub-classifications
- Fully editable metadata fields post insertion allowing for fully flexible manipulation of data
- Integrated indexing ensures metadata is stored inside content file
- Key frame insertion either manually or automatically
Etere MAM facilitates this indexing process ensuring optimized data logging, crucial for the efficient search and retrieval of said data.
Search & Retrieval
Exceptional search and retrieval tools facilitate and speed up media content management. The system allows multiple search criteria including:
- Asset id and or description
- Free search
- Date search
- Expired media
- Synonyms
- Artist/Genre/Media Type
- Full-text
For increased operational efficiency, it is also possible to predefine, save and sort searches
using Query Manager.
Browsing
The ability to preview media content greatly facilitates content management. Etere Browsing supports the streaming and downloading of various media files from VC1 codecs through to MPEG1/2&3 files. It allows for frame accurate clip management supporting the incredibly useful jog&shuffle and audio scrubbing process. It also permits the creation and automatic playback of frame accurate EDLs created through a simple drag and drop function and the use of storyboards. Multiple users have access to the same clip at any given time using Etere Browsing and the integrated nature of the software results in media content being fully accessible at any time, for example during the scheduling process, during asset creation and management, during the assessment of the as run logs, during the recording of the respective media file etc. Digital Watermarking is also possible using Etere Browsing helping to ensure that digital content is secure and protected from non-licensed reproduction. Etere browsing also allows you to specify bit rates, setting different bit rates for different media
categories and support parallel encoding (allowing your PC to encode both hires and lowres media at the same time).
Asset Management
Effective asset management doesn’t stop at indexing. For complete asset control and in order to fully optimize the complete media library, a comprehensive asset management application is required. Etere Asset Management allows you to fully coordinate, amend and view all data associated to
an asset: associated media (both high and low res data, storage location, EDLs etc), metadata, properties (event type, associated secondary events, logos etc) quality control details, technical data, operations associated to said asset etc. The Etere Asset Form consolidates all data relevant to a particular asset in one location, greatly facilitating asset management. Proxy browsing is also possible from within an asset data form making sure at all times during the asset management process that complete knowledge and visibility of media content is available.
User Interface
A user-friendly, common user interface is important for typical user actions. The Etere interface
also supports a level of configurability.
Remote Web Content Management
Archived media content can also be managed remotely through the Etere Web application which allows you to search, view and edit video assets through a user-friendly web interface. Etere Web permits users to access a whole array of data remotely greatly extending the accessibility of the entire system. Key benefits of Etere Web include the following:
- Full integration with Etere HSM and Media Manager for real time information on the archive status of media content
- Video streaming through integrated browsing
- Real time schedule access and manipulation
- Full integration with Etere Memory for access to post broadcast as run logs and aired content
- Comprehensive rights management system for configurable user access levels
- Integral security system to ensure secure data transfer using firewall protection
- Supports integrated editing (Avid via Transfer Manager, Adobe Premier, Apple FCP etc) Physical distance from the digital media archive is not a problem as all previewing, editing and general media content access is supported using the Etere Web interface.
Content Archive
A comprehensive archive facility made up of 2 storage hierarchies will hold the digitized media content:
1. Deep archive containing 400 slots and 2 LTO4 drives (LTO4 cassettes have a capacity of 800MB)
2. Additional nearline storage x3 including one nearline proxy storage for browsing to facilitate content visibility and management Media content stored in the archive (both nearline and deep) will be managed by Etere HSM. This intelligent hierarchical storage management system works as a background process accessed via a powerful and easy to use interface. This intuitive media management system ensures fast movement between encoders, nearline storage, the tape library and the playout system managing both high and low resolution media files. Fully integrated with the existing Etere Automation system, HSM responds to commands and moves media as and when required through both a manual and automated process. The quantity of video that can be moved is:
- 200 hours/day@IMX30
- 6 TB/day
Etere HSM controls cache management and cache cleanup and supports the following:
- Multiple caches
- Multiple tape libraries
- Modern tape drive technologies (e.g. LTO4, AIT3 etc)
- Configurable caching/archiving policies
- Incremental tape writing
- Tape de-fragmentation
- Multiple content copies
- Offline tape management
- Library partitioning
- Mixed mode configurations (multiple libraries/tapes/drives/caches)
- Library-to-library or media-to-media migration policies
- Priorities control (including manual change)
- Queues control (including manual actions e.g. remove, postpone etc)
The comprehensive nature of the system ensures that at any point you are able to clearly identify the current status of your archive with details available on what files are on what tape, how much unused space is available on each tape, status of each tape etc.
Cluster Database Server
The implementation of a cluster database server allows multiple servers to work together increasing the availability of applications and centralizing operational control. This will ensure improved performance and automated failover.
Schematic Diagram
The below shows a schematic diagram of the proposed system from ingest through to archive
System Administration
The system allows the administrator to configure the majority of the media asset management settings and also to specify at least two levels of administration:
1. Full
2. Partial
Integration and Scalability
The proposed system is able to interface with the existing system infrastructure. It is also fully
scalable for future requirements and expandable to meet future operational and businessneeds.
Monitoring and Surveillance
The proposed system allows a comprehensive monitoring of the infrastructure (for example,reports on file usage, database status, file server, encoders, and media servers etc) including SNMP traps generation. It also includes advanced logging and auditing capabilities. Users’ and Administrators’ actions are authored and all system actions fully logged. Log files are kept according to configurable log keeping rules.
Configuration and Administration
The system will have the following GUIs incorporated:
- Configuration: control and logging of queues and running tasks
- Backup/restore of configuration
- Start-up and shutdown of management system
After power up, the management system regains the last active configuration.
The system must be as open, scalable and flexible as possible in order to encompass future growth and changes.
The solution will provide a consistent and reliable system that supports basic functions and both manual and automatic operation modes.
Ingest and Digitization
Current tape content (approximately 30,000 hours) must be ingested and ready for playout within a strict timeframe (approximately 3 months).
Given these current constraints, speed of ingest and digitization is key. In the proposed system the automatic ingest of existing tape content will be carried out using multiple flexicarts. The proposed figure is for 4 flexicarts to be used (one flexicart has the capacity to ingest 80 hrs/day and therefore 2400 hrs/month of video content). Multiple flexicarts can be managed and controlled by Etere using its intuitive VCC Controller and Etere Cart application. The latter ensures complete visibility over the status of content ingest and allows for efficient automated digitization. It must be noted that the Etere is also able to manage the flexicart should in the future the decision be taken to use it as a VTR for direct playout. The flexible nature of Etere Resource Editor allows complete user-defined configuration of the Flexicart.
Workflow
During the ingest process and before any content is archived it is both quality checked and transcoded. The below shows the proposed workflow relevant to the digitization process of the media content
Quality Control
Media content quality is fundamental, however quality control checks can be time consuming and often dependant on the professionalism of the operator in question. In order to produce flawless media content, there are 2 principle levels of quality control that the media content will go through during and after the ingest and digitization process.
Tape Error Control
The VTR error system integrated in the flexicart logs any tape errors by inserting cue points. This automated error system greatly increases operator efficiency as only the cue points need to be checked in order to approve the ‘error-free’ files.
Content Quality Control
Following ingest the unique, highly effective and fully integrated Etere Content Management System (CMS) automatically controls the quality of the ingested media. It can run without an operator being present and can run 8x faster that real time on a P4 1G CPU. The quality check process is fully configurable and uses standard MS Windows XP OS and multiple videos can be run on the same facility simultaneously. Etere CMS checks the media content in order to identify:
- Black video
- Freezed video
- No audio
- Scene changes entry points
Only once both the tape error control and the content quality control have been passed and the latter one approved by the operator, will the media content be accepted for archive ensuring quality standards are met every time.
Transcoding
Optimizing the ingest process is not just about transferring data from one storage point to another but it is also about maximizing the potential of your media content and making it available to multiple users via multiple channels. The proposed system is able to generate multiformat, multipurpose proxy files using the Etere Transcoding Engine. These frame accurate proxy files are not only important for the repurposing of said media in order to be able to serve multiple channels (intranet, internet, mobile phones etc) but also from an integral part of content media management. K2 provides the transcoding needed to do with Ampstream software. The following gives an example of the most basic formats that can be transcoded using the proposed system:
- AVC intra
- MPEG2 422 and 420
- DV
- DVCPRO wrapped in MXF, AVI or quicktime
Media Content Management
Effective media content management should both reduce the operational cost of content production and facilitate the overall handling of a company’s media assets. With a key focus on increasing the return on investment of said assets, the Etere MAM application integrates perfectly into this proposal. A fundamental benefit of an effective digital library is the ability and speed at which media content can be retrieved and made available for re-use.
The optimization of a digital workflow requires a comprehensive understanding of the importance of the following content management tools:
1. Indexing
2. Search & Retrieval
3. Browsing
4. Asset Management
5. User Interface
Indexing
Accurate media content inventory is dependent on the logging capabilities of the respective process via which the media content is ingested. The proposed system offers advanced indexing tools with, amongst others, the following capabilities:
- Basic existing metadata is inserted automatically during the ingest process
- Manual insertion of an unlimited amount of metadata
- Configurable metadata classifications and sub-classifications
- Fully editable metadata fields post insertion allowing for fully flexible manipulation of data
- Integrated indexing ensures metadata is stored inside content file
- Key frame insertion either manually or automatically
Etere MAM facilitates this indexing process ensuring optimized data logging, crucial for the efficient search and retrieval of said data.
Search & Retrieval
Exceptional search and retrieval tools facilitate and speed up media content management. The system allows multiple search criteria including:
- Asset id and or description
- Free search
- Date search
- Expired media
- Synonyms
- Artist/Genre/Media Type
- Full-text
For increased operational efficiency, it is also possible to predefine, save and sort searches
using Query Manager.
Browsing
The ability to preview media content greatly facilitates content management. Etere Browsing supports the streaming and downloading of various media files from VC1 codecs through to MPEG1/2&3 files. It allows for frame accurate clip management supporting the incredibly useful jog&shuffle and audio scrubbing process. It also permits the creation and automatic playback of frame accurate EDLs created through a simple drag and drop function and the use of storyboards. Multiple users have access to the same clip at any given time using Etere Browsing and the integrated nature of the software results in media content being fully accessible at any time, for example during the scheduling process, during asset creation and management, during the assessment of the as run logs, during the recording of the respective media file etc. Digital Watermarking is also possible using Etere Browsing helping to ensure that digital content is secure and protected from non-licensed reproduction. Etere browsing also allows you to specify bit rates, setting different bit rates for different media
categories and support parallel encoding (allowing your PC to encode both hires and lowres media at the same time).
Asset Management
Effective asset management doesn’t stop at indexing. For complete asset control and in order to fully optimize the complete media library, a comprehensive asset management application is required. Etere Asset Management allows you to fully coordinate, amend and view all data associated to
an asset: associated media (both high and low res data, storage location, EDLs etc), metadata, properties (event type, associated secondary events, logos etc) quality control details, technical data, operations associated to said asset etc. The Etere Asset Form consolidates all data relevant to a particular asset in one location, greatly facilitating asset management. Proxy browsing is also possible from within an asset data form making sure at all times during the asset management process that complete knowledge and visibility of media content is available.
User Interface
A user-friendly, common user interface is important for typical user actions. The Etere interface
also supports a level of configurability.
Remote Web Content Management
Archived media content can also be managed remotely through the Etere Web application which allows you to search, view and edit video assets through a user-friendly web interface. Etere Web permits users to access a whole array of data remotely greatly extending the accessibility of the entire system. Key benefits of Etere Web include the following:
- Full integration with Etere HSM and Media Manager for real time information on the archive status of media content
- Video streaming through integrated browsing
- Real time schedule access and manipulation
- Full integration with Etere Memory for access to post broadcast as run logs and aired content
- Comprehensive rights management system for configurable user access levels
- Integral security system to ensure secure data transfer using firewall protection
- Supports integrated editing (Avid via Transfer Manager, Adobe Premier, Apple FCP etc) Physical distance from the digital media archive is not a problem as all previewing, editing and general media content access is supported using the Etere Web interface.
Content Archive
A comprehensive archive facility made up of 2 storage hierarchies will hold the digitized media content:
1. Deep archive containing 400 slots and 2 LTO4 drives (LTO4 cassettes have a capacity of 800MB)
2. Additional nearline storage x3 including one nearline proxy storage for browsing to facilitate content visibility and management Media content stored in the archive (both nearline and deep) will be managed by Etere HSM. This intelligent hierarchical storage management system works as a background process accessed via a powerful and easy to use interface. This intuitive media management system ensures fast movement between encoders, nearline storage, the tape library and the playout system managing both high and low resolution media files. Fully integrated with the existing Etere Automation system, HSM responds to commands and moves media as and when required through both a manual and automated process. The quantity of video that can be moved is:
- 200 hours/day@IMX30
- 6 TB/day
Etere HSM controls cache management and cache cleanup and supports the following:
- Multiple caches
- Multiple tape libraries
- Modern tape drive technologies (e.g. LTO4, AIT3 etc)
- Configurable caching/archiving policies
- Incremental tape writing
- Tape de-fragmentation
- Multiple content copies
- Offline tape management
- Library partitioning
- Mixed mode configurations (multiple libraries/tapes/drives/caches)
- Library-to-library or media-to-media migration policies
- Priorities control (including manual change)
- Queues control (including manual actions e.g. remove, postpone etc)
The comprehensive nature of the system ensures that at any point you are able to clearly identify the current status of your archive with details available on what files are on what tape, how much unused space is available on each tape, status of each tape etc.
Cluster Database Server
The implementation of a cluster database server allows multiple servers to work together increasing the availability of applications and centralizing operational control. This will ensure improved performance and automated failover.
Schematic Diagram
The below shows a schematic diagram of the proposed system from ingest through to archive
System Administration
The system allows the administrator to configure the majority of the media asset management settings and also to specify at least two levels of administration:
1. Full
2. Partial
Integration and Scalability
The proposed system is able to interface with the existing system infrastructure. It is also fully
scalable for future requirements and expandable to meet future operational and businessneeds.
Monitoring and Surveillance
The proposed system allows a comprehensive monitoring of the infrastructure (for example,reports on file usage, database status, file server, encoders, and media servers etc) including SNMP traps generation. It also includes advanced logging and auditing capabilities. Users’ and Administrators’ actions are authored and all system actions fully logged. Log files are kept according to configurable log keeping rules.
Configuration and Administration
The system will have the following GUIs incorporated:
- Configuration: control and logging of queues and running tasks
- Backup/restore of configuration
- Start-up and shutdown of management system
After power up, the management system regains the last active configuration.
OVERVIEW OF ‘KEY OBJECTIVES’
The principal objectives that must be met for GCI’s Digital Archive Solution are as follows: - Speed in the ingest and digitization of video content - Integrated and highly effective quality control check - Automatic generation of multiformat, multipurpose proxy files - Seamless integration with the existing Automation system - Effective long-term archive of digitized content with the possibility of both nearline and deep archive - Frame by frame content indexing with full SMPTE dictionary support - Effective integration with post production - Complete archive content visibility (browsing) and control via intranet and internet portals for multi-user access. One of the key issues that GCI’s customers face is the dispersed nature of their users. It is imperative that users, wherever they are, are able to access the media archive remotely in order to make amends or take decisions based on real time information. The following document outlines how the above objectives can be achieved through Etere software and details the key system requirements that will be met with the proposed Digital Archive system.
The principal objectives that must be met for GCI’s Digital Archive Solution are as follows: - Speed in the ingest and digitization of video content - Integrated and highly effective quality control check - Automatic generation of multiformat, multipurpose proxy files - Seamless integration with the existing Automation system - Effective long-term archive of digitized content with the possibility of both nearline and deep archive - Frame by frame content indexing with full SMPTE dictionary support - Effective integration with post production - Complete archive content visibility (browsing) and control via intranet and internet portals for multi-user access. One of the key issues that GCI’s customers face is the dispersed nature of their users. It is imperative that users, wherever they are, are able to access the media archive remotely in order to make amends or take decisions based on real time information. The following document outlines how the above objectives can be achieved through Etere software and details the key system requirements that will be met with the proposed Digital Archive system.
SYSTEM FUNCTIONALITY
Flexibility The system must be as open, scalable and flexible as possible in order to encompass future growth and changes. The solution will provide a consistent and reliable system that supports basic functions and both manual and automatic operation modes. Ingest and Digitization Current tape content (approximately 30,000 hours) must be ingested and ready for playout within a strict timeframe (approximately 3 months). Given these current constraints, speed of ingest and digitization is key. In the proposed system the automatic ingest of existing tape content will be carried out using multiple flexicarts. The proposed figure is for 4 flexicarts to be used (one flexicart has the capacity to ingest 80 hrs/day and therefore 2400 hrs/month of video content). Multiple flexicarts can be managed and controlled by Etere using its intuitive VCC Controller and Etere Cart application. The latter ensures complete visibility over the status of content ingest and allows for efficient automated digitization. It must be noted that the Etere is also able to manage the flexicart should in the future the decision be taken to use it as a VTR for direct playout. The flexible nature of Etere Resource Editor allows complete user-defined configuration of the Flexicart. Workflow During the ingest process and before any content is archived it is both quality checked and transcoded. The below shows the proposed workflow relevant to the digitization process of the media content Quality Control Media content quality is fundamental, however quality control checks can be time consuming and often dependant on the professionalism of the operator in question. In order to produce flawless media content, there are 2 principle levels of quality control that the media content will go through during and after the ingest and digitization process. Tape Error Control The VTR error system integrated in the flexicart logs any tape errors by inserting cue points. This automated error system greatly increases operator efficiency as only the cue points need to be checked in order to approve the ‘error-free’ files. Content Quality Control Following ingest the unique, highly effective and fully integrated Etere Content Management System (CMS) automatically controls the quality of the ingested media. It can run without an operator being present and can run 8x faster that real time on a P4 1G CPU. The quality check process is fully configurable and uses standard MS Windows XP OS and multiple videos can be run on the same facility simultaneously. Etere CMS checks the media content in order to identify: - Black video - Freezed video - No audio - Scene changes entry points Only once both the tape error control and the content quality control have been passed and the latter one approved by the operator, will the media content be accepted for archive ensuring quality standards are met every time. Transcoding Optimizing the ingest process is not just about transferring data from one storage point to another but it is also about maximizing the potential of your media content and making it available to multiple users via multiple channels. The proposed system is able to generate multiformat, multipurpose proxy files using the Etere Transcoding Engine. These frame accurate proxy files are not only important for the repurposing of said media in order to be able to serve multiple channels (intranet, internet, mobile phones etc) but also from an integral part of content media management. K2 provides the transcoding needed to do with Ampstream software. The following gives an example of the most basic formats that can be transcoded using the proposed system: - AVC intra - MPEG2 422 and 420 - DV - DVCPRO wrapped in MXF, AVI or quicktime Media Content Management Effective media content management should both reduce the operational cost of content production and facilitate the overall handling of a company’s media assets. With a key focus on increasing the return on investment of said assets, the Etere MAM application integrates perfectly into this proposal. A fundamental benefit of an effective digital library is the ability and speed at which media content can be retrieved and made available for re-use. The optimization of a digital workflow requires a comprehensive understanding of the importance of the following content management tools: 1. Indexing 2. Search & Retrieval 3. Browsing 4. Asset Management 5. User Interface Indexing Accurate media content inventory is dependent on the logging capabilities of the respective process via which the media content is ingested. The proposed system offers advanced indexing tools with, amongst others, the following capabilities: - Basic existing metadata is inserted automatically during the ingest process - Manual insertion of an unlimited amount of metadata - Configurable metadata classifications and sub-classifications - Fully editable metadata fields post insertion allowing for fully flexible manipulation of data - Integrated indexing ensures metadata is stored inside content file - Key frame insertion either manually or automatically Etere MAM facilitates this indexing process ensuring optimized data logging, crucial for the efficient search and retrieval of said data. Search & Retrieval Exceptional search and retrieval tools facilitate and speed up media content management. The system allows multiple search criteria including: - Asset id and or description - Free search - Date search - Expired media - Synonyms - Artist/Genre/Media Type - Full-text For increased operational efficiency, it is also possible to predefine, save and sort searches using Query Manager. Browsing The ability to preview media content greatly facilitates content management. Etere Browsing supports the streaming and downloading of various media files from VC1 codecs through to MPEG1/2&3 files. It allows for frame accurate clip management supporting the incredibly useful jog&shuffle and audio scrubbing process. It also permits the creation and automatic playback of frame accurate EDLs created through a simple drag and drop function and the use of storyboards. Multiple users have access to the same clip at any given time using Etere Browsing and the integrated nature of the software results in media content being fully accessible at any time, for example during the scheduling process, during asset creation and management, during the assessment of the as run logs, during the recording of the respective media file etc. Digital Watermarking is also possible using Etere Browsing helping to ensure that digital content is secure and protected from non-licensed reproduction. Etere browsing also allows you to specify bit rates, setting different bit rates for different media categories and support parallel encoding (allowing your PC to encode both hires and lowres media at the same time). Asset Management Effective asset management doesn’t stop at indexing. For complete asset control and in order to fully optimize the complete media library, a comprehensive asset management application is required. Etere Asset Management allows you to fully coordinate, amend and view all data associated to an asset: associated media (both high and low res data, storage location, EDLs etc), metadata, properties (event type, associated secondary events, logos etc) quality control details, technical data, operations associated to said asset etc. The Etere Asset Form consolidates all data relevant to a particular asset in one location, greatly facilitating asset management. Proxy browsing is also possible from within an asset data form making sure at all times during the asset management process that complete knowledge and visibility of media content is available. User Interface A user-friendly, common user interface is important for typical user actions. The Etere interface also supports a level of configurability. Remote Web Content Management Archived media content can also be managed remotely through the Etere Web application which allows you to search, view and edit video assets through a user-friendly web interface. Etere Web permits users to access a whole array of data remotely greatly extending the accessibility of the entire system. Key benefits of Etere Web include the following: - Full integration with Etere HSM and Media Manager for real time information on the archive status of media content - Video streaming through integrated browsing - Real time schedule access and manipulation - Full integration with Etere Memory for access to post broadcast as run logs and aired content - Comprehensive rights management system for configurable user access levels - Integral security system to ensure secure data transfer using firewall protection - Supports integrated editing (Avid via Transfer Manager, Adobe Premier, Apple FCP etc) Physical distance from the digital media archive is not a problem as all previewing, editing and general media content access is supported using the Etere Web interface. Content Archive A comprehensive archive facility made up of 2 storage hierarchies will hold the digitized media content: 1. Deep archive containing 400 slots and 2 LTO4 drives (LTO4 cassettes have a capacity of 800MB) 2. Additional nearline storage x3 including one nearline proxy storage for browsing to facilitate content visibility and management Media content stored in the archive (both nearline and deep) will be managed by Etere HSM. This intelligent hierarchical storage management system works as a background process accessed via a powerful and easy to use interface. This intuitive media management system ensures fast movement between encoders, nearline storage, the tape library and the playout system managing both high and low resolution media files. Fully integrated with the existing Etere Automation system, HSM responds to commands and moves media as and when required through both a manual and automated process. The quantity of video that can be moved is: - 200 hours/day@IMX30 - 6 TB/day Etere HSM controls cache management and cache cleanup and supports the following: - Multiple caches - Multiple tape libraries - Modern tape drive technologies (e.g. LTO4, AIT3 etc) - Configurable caching/archiving policies - Incremental tape writing - Tape de-fragmentation - Multiple content copies - Offline tape management - Library partitioning - Mixed mode configurations (multiple libraries/tapes/drives/caches) - Library-to-library or media-to-media migration policies - Priorities control (including manual change) - Queues control (including manual actions e.g. remove, postpone etc) The comprehensive nature of the system ensures that at any point you are able to clearly identify the current status of your archive with details available on what files are on what tape, how much unused space is available on each tape, status of each tape etc. Cluster Database Server The implementation of a cluster database server allows multiple servers to work together increasing the availability of applications and centralizing operational control. This will ensure improved performance and automated failover. Schematic Diagram The below shows a schematic diagram of the proposed system from ingest through to archive System Administration The system allows the administrator to configure the majority of the media asset management settings and also to specify at least two levels of administration: 1. Full 2. Partial Integration and Scalability The proposed system is able to interface with the existing system infrastructure. It is also fully scalable for future requirements and expandable to meet future operational and businessneeds. Monitoring and Surveillance The proposed system allows a comprehensive monitoring of the infrastructure (for example,reports on file usage, database status, file server, encoders, and media servers etc) including SNMP traps generation. It also includes advanced logging and auditing capabilities. Users’ and Administrators’ actions are authored and all system actions fully logged. Log files are kept according to configurable log keeping rules. Configuration and Administration The system will have the following GUIs incorporated: - Configuration: control and logging of queues and running tasks - Backup/restore of configuration - Start-up and shutdown of management system After power up, the management system regains the last active configuration.
Flexibility The system must be as open, scalable and flexible as possible in order to encompass future growth and changes. The solution will provide a consistent and reliable system that supports basic functions and both manual and automatic operation modes. Ingest and Digitization Current tape content (approximately 30,000 hours) must be ingested and ready for playout within a strict timeframe (approximately 3 months). Given these current constraints, speed of ingest and digitization is key. In the proposed system the automatic ingest of existing tape content will be carried out using multiple flexicarts. The proposed figure is for 4 flexicarts to be used (one flexicart has the capacity to ingest 80 hrs/day and therefore 2400 hrs/month of video content). Multiple flexicarts can be managed and controlled by Etere using its intuitive VCC Controller and Etere Cart application. The latter ensures complete visibility over the status of content ingest and allows for efficient automated digitization. It must be noted that the Etere is also able to manage the flexicart should in the future the decision be taken to use it as a VTR for direct playout. The flexible nature of Etere Resource Editor allows complete user-defined configuration of the Flexicart. Workflow During the ingest process and before any content is archived it is both quality checked and transcoded. The below shows the proposed workflow relevant to the digitization process of the media content Quality Control Media content quality is fundamental, however quality control checks can be time consuming and often dependant on the professionalism of the operator in question. In order to produce flawless media content, there are 2 principle levels of quality control that the media content will go through during and after the ingest and digitization process. Tape Error Control The VTR error system integrated in the flexicart logs any tape errors by inserting cue points. This automated error system greatly increases operator efficiency as only the cue points need to be checked in order to approve the ‘error-free’ files. Content Quality Control Following ingest the unique, highly effective and fully integrated Etere Content Management System (CMS) automatically controls the quality of the ingested media. It can run without an operator being present and can run 8x faster that real time on a P4 1G CPU. The quality check process is fully configurable and uses standard MS Windows XP OS and multiple videos can be run on the same facility simultaneously. Etere CMS checks the media content in order to identify: - Black video - Freezed video - No audio - Scene changes entry points Only once both the tape error control and the content quality control have been passed and the latter one approved by the operator, will the media content be accepted for archive ensuring quality standards are met every time. Transcoding Optimizing the ingest process is not just about transferring data from one storage point to another but it is also about maximizing the potential of your media content and making it available to multiple users via multiple channels. The proposed system is able to generate multiformat, multipurpose proxy files using the Etere Transcoding Engine. These frame accurate proxy files are not only important for the repurposing of said media in order to be able to serve multiple channels (intranet, internet, mobile phones etc) but also from an integral part of content media management. K2 provides the transcoding needed to do with Ampstream software. The following gives an example of the most basic formats that can be transcoded using the proposed system: - AVC intra - MPEG2 422 and 420 - DV - DVCPRO wrapped in MXF, AVI or quicktime Media Content Management Effective media content management should both reduce the operational cost of content production and facilitate the overall handling of a company’s media assets. With a key focus on increasing the return on investment of said assets, the Etere MAM application integrates perfectly into this proposal. A fundamental benefit of an effective digital library is the ability and speed at which media content can be retrieved and made available for re-use. The optimization of a digital workflow requires a comprehensive understanding of the importance of the following content management tools: 1. Indexing 2. Search & Retrieval 3. Browsing 4. Asset Management 5. User Interface Indexing Accurate media content inventory is dependent on the logging capabilities of the respective process via which the media content is ingested. The proposed system offers advanced indexing tools with, amongst others, the following capabilities: - Basic existing metadata is inserted automatically during the ingest process - Manual insertion of an unlimited amount of metadata - Configurable metadata classifications and sub-classifications - Fully editable metadata fields post insertion allowing for fully flexible manipulation of data - Integrated indexing ensures metadata is stored inside content file - Key frame insertion either manually or automatically Etere MAM facilitates this indexing process ensuring optimized data logging, crucial for the efficient search and retrieval of said data. Search & Retrieval Exceptional search and retrieval tools facilitate and speed up media content management. The system allows multiple search criteria including: - Asset id and or description - Free search - Date search - Expired media - Synonyms - Artist/Genre/Media Type - Full-text For increased operational efficiency, it is also possible to predefine, save and sort searches using Query Manager. Browsing The ability to preview media content greatly facilitates content management. Etere Browsing supports the streaming and downloading of various media files from VC1 codecs through to MPEG1/2&3 files. It allows for frame accurate clip management supporting the incredibly useful jog&shuffle and audio scrubbing process. It also permits the creation and automatic playback of frame accurate EDLs created through a simple drag and drop function and the use of storyboards. Multiple users have access to the same clip at any given time using Etere Browsing and the integrated nature of the software results in media content being fully accessible at any time, for example during the scheduling process, during asset creation and management, during the assessment of the as run logs, during the recording of the respective media file etc. Digital Watermarking is also possible using Etere Browsing helping to ensure that digital content is secure and protected from non-licensed reproduction. Etere browsing also allows you to specify bit rates, setting different bit rates for different media categories and support parallel encoding (allowing your PC to encode both hires and lowres media at the same time). Asset Management Effective asset management doesn’t stop at indexing. For complete asset control and in order to fully optimize the complete media library, a comprehensive asset management application is required. Etere Asset Management allows you to fully coordinate, amend and view all data associated to an asset: associated media (both high and low res data, storage location, EDLs etc), metadata, properties (event type, associated secondary events, logos etc) quality control details, technical data, operations associated to said asset etc. The Etere Asset Form consolidates all data relevant to a particular asset in one location, greatly facilitating asset management. Proxy browsing is also possible from within an asset data form making sure at all times during the asset management process that complete knowledge and visibility of media content is available. User Interface A user-friendly, common user interface is important for typical user actions. The Etere interface also supports a level of configurability. Remote Web Content Management Archived media content can also be managed remotely through the Etere Web application which allows you to search, view and edit video assets through a user-friendly web interface. Etere Web permits users to access a whole array of data remotely greatly extending the accessibility of the entire system. Key benefits of Etere Web include the following: - Full integration with Etere HSM and Media Manager for real time information on the archive status of media content - Video streaming through integrated browsing - Real time schedule access and manipulation - Full integration with Etere Memory for access to post broadcast as run logs and aired content - Comprehensive rights management system for configurable user access levels - Integral security system to ensure secure data transfer using firewall protection - Supports integrated editing (Avid via Transfer Manager, Adobe Premier, Apple FCP etc) Physical distance from the digital media archive is not a problem as all previewing, editing and general media content access is supported using the Etere Web interface. Content Archive A comprehensive archive facility made up of 2 storage hierarchies will hold the digitized media content: 1. Deep archive containing 400 slots and 2 LTO4 drives (LTO4 cassettes have a capacity of 800MB) 2. Additional nearline storage x3 including one nearline proxy storage for browsing to facilitate content visibility and management Media content stored in the archive (both nearline and deep) will be managed by Etere HSM. This intelligent hierarchical storage management system works as a background process accessed via a powerful and easy to use interface. This intuitive media management system ensures fast movement between encoders, nearline storage, the tape library and the playout system managing both high and low resolution media files. Fully integrated with the existing Etere Automation system, HSM responds to commands and moves media as and when required through both a manual and automated process. The quantity of video that can be moved is: - 200 hours/day@IMX30 - 6 TB/day Etere HSM controls cache management and cache cleanup and supports the following: - Multiple caches - Multiple tape libraries - Modern tape drive technologies (e.g. LTO4, AIT3 etc) - Configurable caching/archiving policies - Incremental tape writing - Tape de-fragmentation - Multiple content copies - Offline tape management - Library partitioning - Mixed mode configurations (multiple libraries/tapes/drives/caches) - Library-to-library or media-to-media migration policies - Priorities control (including manual change) - Queues control (including manual actions e.g. remove, postpone etc) The comprehensive nature of the system ensures that at any point you are able to clearly identify the current status of your archive with details available on what files are on what tape, how much unused space is available on each tape, status of each tape etc. Cluster Database Server The implementation of a cluster database server allows multiple servers to work together increasing the availability of applications and centralizing operational control. This will ensure improved performance and automated failover. Schematic Diagram The below shows a schematic diagram of the proposed system from ingest through to archive System Administration The system allows the administrator to configure the majority of the media asset management settings and also to specify at least two levels of administration: 1. Full 2. Partial Integration and Scalability The proposed system is able to interface with the existing system infrastructure. It is also fully scalable for future requirements and expandable to meet future operational and businessneeds. Monitoring and Surveillance The proposed system allows a comprehensive monitoring of the infrastructure (for example,reports on file usage, database status, file server, encoders, and media servers etc) including SNMP traps generation. It also includes advanced logging and auditing capabilities. Users’ and Administrators’ actions are authored and all system actions fully logged. Log files are kept according to configurable log keeping rules. Configuration and Administration The system will have the following GUIs incorporated: - Configuration: control and logging of queues and running tasks - Backup/restore of configuration - Start-up and shutdown of management system After power up, the management system regains the last active configuration.