VIMN: Bringing Traditional Vietnamese Music to the World

This case study will describe the Etere’s solutions with which VIM has been provided in order to implement an “Enterprise Musical Archive Management System” able to cover all aspects of ingest, catalogue, storage (online, nearline and offline), browsing, tape management and delivery of the Vietnamese Musical Archive; making emphasis on the process of digitizing content stored on tapes into hires and lowres files ready for being distributed over the world’s largest digital distribution mean, internet.

For more information, please refer to the attachement.

Introduction
In addition, a supported barcode generation and recognition capabilities will permit VIM to easily and quickly identify offline tapes before either loading them into the system or simply consulting their main information and contained assets. The Vietnamese Institute for Musicology (VIM), formerly the Musicology Division of the Culture and Arts Bureau, was established in 1950. VIM at present is an institution belonging to Vietnam National Academy of Music under the Ministry of Culture and Information. VIM is mainly responsible for collection, preservation, research and dissemination of Vietnamese traditional music within the nation and on worldwide basis. VIM headquarters are officially located in Hanoi, in a modern building large enough to house all VIM divisions including Studio, filming division of performance, the Showroom of Vietnamese Traditional Musical Instruments, the division of Datfabank on Folk Music and Traditional Performing Arts of Vietnam, etc. The primary challenge of the project was to manage the big and precious musical heritage present in the VIM archive which is composed from about 10’000 hours of video (225TB), 15’000 hours of audio (14TB) and documents and pictures (10TB), however, the first need consisted in digitizing the VIM’s entire video/audio/document library from old analog tapes/disks into a digital archive system able to provide powerful retrieval capabilities for viewing, repeating transmissions or editing, all these, through a simple and easy-to-use interface in Vietnamese language.

The provided solution will improve also the VIM’s audio ingest process, enabling operators to easily load existing digital audio files into the system and even record them directly. Once ingested and before being archived, acquired content will be enriched in just one click using preset metadata profiles or using a streamlined speech-to-text module for inserting metadata at specific time points without typing, thus saving time and money to the organization. This case study will describe the Etere’s solutions with which VIM has been provided in order to implement an “Enterprise Musical Archive Management System” able to cover all aspects of ingest, catalogue, storage (online, nearline and offline), browsing, tape management and delivery of the Vietnamese Musical Archive; making emphasis on the process of digitizing content stored on tapes into hires and lowres files ready for being distributed over the world’s largest digital distribution mean, internet. In addition, a supported barcode generation and recognition capabilities will permit VIM to easily and quickly identify offline tapes before either loading them into the system or simply consulting their main information and contained assets.

Overall Solution
Based on the increasing quality, prestige and recognition that Etere solutions receives in the Vietnamese market, VIM has relied upon Etere for the deployment of a musical archiving system empowered with a high-speed transmission line to bring its whole media content to internet to disseminate Vietnamese music to the world more effectively, thus making of VIM not only a reliable address for the musical circle but also a destination of those who interested in Vietnamese music, especially, Vietnamese traditional music.

Due to the complexity and importance of the project, VIM has been provided with an all-in-one Etere Ecosystem solution, a system able to manage all the activities -from content ingest and transcoding to worksheets organization and content delivery- under a centralized and integrated framework capable of managing production costs as well as distributing media content to millions of users with the maximum of reliability thanks to a well-structured permissions system.

A centralized and highly effective acquisition solution for ingesting digital audio content has been tailored developed to streamline not only the ingest process but also the indexing, storage and retrieval of archived content. The ingest system will allow VIM to digitize video, audio and document content present in its data bank from analog tapes to a digital format, the approximate amount of data to be digitized by Etere has been illustrated in the following table.

Once digitized, content is catalogued and enriched with either manual metadata (typed) or automatic metadata (selected). The media content catalogue will enable VIM to provide an accurate search engine for consulting contents through a desktop and web browsing interface. Etere will manage digitized content using the best-of-breed in transcoding technology, thus allowing the transfer of content between departments with the maximum of quality preservation and media content availability. A lowres version will be always available during cataloguing and browsing in order to improve the search capabilities of the system. In addition to all the aforementioned points, it’s worthy mentioning that Etere’s consolidated diffusion in the Asian market has permitted VIM to get a solution empowered with a Vietnamese graphical interface for allowing even inexpert users to easily search and consult the big digital archive made available by the central system. Etere has fully supported all the hardware present in VIM facility taking the most of each device and integrating it with the cutting-edge technology of Etere modules, thus allowing the overall system to achieve a full performance; among the equipment managed by Etere along the VIM installation it can mentioned:

■ A PetaSite tape library (500 LTO5 slots for 400 TB)
■ 15 AVID/FCP non-linear editing systems
■ 2 MTX ports (full resolution video ingest),
■ 4 MTX ports (audio ingest)
■ 1 non-linear storage device (80 TB for high and proxy resolution),
■ 2 video tape recorders
■ 30 workstations
■ Web/FTP servers, Network switchers, book scanners, barcode readers, barcode printers, audio cassette players, etc

Technical Information
The VIM system has been implemented with the primary aim of creating a tape archive for storing all the video and audio material managed by the Vietnamese Institute for Musicology. The digital archive has been classified and detailed thanks using SMPTE metadata, the system has been rendered available through 4 touchscreen computers (NLE6,7,8,9) enabled with Etere Web, thus allowing researches to consult the archive and download media files (with a proper authorization web request).

3.1. ASSETS AND METADATA
■ Enrich musical content with custom SMPTE information through a group of metadata profiles which allow the immediate association of metadata
■ The “kind of music” of content is managed through the use of fully customizable “genres"
■ Further music information is defined under the asset properties, more specifically, as the asset’s music data, thus allowing a specialized management of musical content
■ The “content of song” field is managed under a specific field which permits to enter an extended description of the related musical content
v Vocal music has been improved with the inclusion of song “lyrics” under a dedicated field
■ The “Artist Manager” allows creating artist profiles and associates them with specific roles (e.g. as “performers”, “producers”, “craftsmen”, etc.) to musical content
■ A robust search engine that permits operators retrieving content based on filters and full-text keywords as well as sorting and grouping results

3.2. INGEST
Within the VIM institute, the implemented system allows to ingest content as following detailed:
■ Video ingest from Etere Ingest on MTX 1, 2 workstations in analogic 4:3 SD format. A workflow (i.e. archiving and transcode) has been set to be automatically launched once ingest is completed
■ Files upload from NLE (AVID) and MAM workstations in XDCAM MXF, QuickTime reference MOV and MPG format
■ Audio ingest from Etere Media Library on Audio 1, 2 workstations. The function used is an Audio Recorder which is only available for material with “audio” property enabled
■ Files import from Etere Media Library using video file import and multi-import functions

3.3. ARCHIVING
Once ingested, audio and video files are archived, transcoded and deleted according to the following criteria:
■ IMX30 wrapped files (either ingested from MTX or upload and transcoded from
EtereWeb) are archived in: \\Nearline\Archive\
■ Wave audio files are archived in: \\Nearline\Audio\
■ AIF files are archived in: \\Nearline\AIF

A workflow is automatically launched after each ingest/upload operation, transcoding ingested files in WMV format and IMX30 format respectively. Once transcoded, the same workflow archives all files (including IMX30, WMV, WAV and AIF) in the LTO tapes of the asset-related media pools
■ Hires files are deleted from the Nearline about one month after their archiving; instead, lowres files are kept since they are used for preview via web
■ WAV audio files ingested using the “Audio Recorder” are archived through a manual workflow whereas files imported using the “Videofile Import” and “Videofile Multi-Import” functions are archived by an automatic workflow (Etere Configuration > Configuration Editor > Workflow designer > Default workflows > After video file import) WAV and AIF audio files are archived on LTO tapes, in the meanwhile, a WMA version is created for each archived file for preview purposes
■ WAV and AIF audio files will be automatically deleted from the nearline storage after “x” days of their archiving

3.4. WEB UPLOAD
The deployed system allows VIM receiving digital files via web, currently managed formats include AVID MXF (XDCAM MXF), AVID MPG (MPG) and AVID MOV (QuickTime Reference). The number of managed formats will be increased with time.

A java-based upload form will permit to select the AVID NLE codec with which files will be
uploaded:
■ AVID MXF files will be uploaded in the \\Nearline\Webupload\ metadevice
■ AVID MOV files will be stored in the \\Nearline\Mov\ metadevice

A post-upload workflow (specific for each format) will transcode all uploaded files into an IMX30 MXF Wrapped format to be stored in the \\Nearline\Archive\ metadevice, creating also, for all files, a WMV copy in the \\Nearline\Proxy\ metadevice. Files uploaded via web are processed by the “Operations by workflow” module installed in the Transcoding workstation.

Originally uploaded files stored in the \\Nearline\Webupload\ metadevice are automaticallydeleted via workflow through a “deletion workflow” automatically launched by the “Operations by Workflow” module installed in the Transcoding1 workstation.

3.5. TRANSCODING
AVID NLE files incoming via web are, as explained before, transcoded into an IMX30 format and stored in the \\Nearline\Archive\ metadevice and then archived on LTO tapes according to their codec.

In order to be able to transcode AVID MOV files, it has been required to install an AVID client on each transcoding workstation (i.e. transcoding 1 and 2) to allow the transcoder to find all the needed files stored in the AVID share subfolders.

For all IMX30 files, the transcoder creates a lowres version in the following metadevices:
■ \\Nearline\Proxy\: The proxy files (WMV). Currently, these files are not deleted since they are used for preview from Etere Web and Media Library
■ \\Nearline\WMA : WMA files (transcoded from WAV and AIF formats) used for previewing assets via web

Deletions and File length checks operations are respectively performed by the Transcoding1 and Transcoding2 workstations. It’s worth mentioning that audio files imported from the Media Library don’t require the file length action since the import itself automatically updates the media duration.

3.6. NLE
The first 5 NLE workstations are used for editing and web uploading of files generated by AVID. The NLE workstations (AVID editors) generate files in MOV (QuickTime Reference) and MXF (XDCAM) format which are subsequently uploaded into \\Nearline\Webupload\ (MXF) or the \\Nearline\Mov\ (MOV) and then archived into LTO tapes according their codec.

When uploading a file, users must select the “AVID NLE” transcoding codec and the “FTP Space” storage metadevice (\\Nearline\Webupload\ or \\Nearline\Mov\). Once the file is uploaded, the user selects a post-upload workflow according to the codec depending on the real uploaded codec (MXF, MOV or MPG).

3.7. MAM
MAM users are able to open, create and modify asset forms to prepare their EDL, this, through the following functions provided by Etere MAM:

■ Create visible segments on EDL,
■ Retrieve “visible segments” through a full-text search and insert SMPTE metadata,
■ Partially restore retrieved media segments,
■ Conforming of assets and segments (licensed but not yet used)
■ Dictate metadata using a speech-to-text module (pending configuration and testing)

Detailed Solution
The solution provided by Etere has permitted VIM to effectively manage its enterprise musical archive based on the more comprehensive and flexible solution available on the market, Etere MERP, a distributed and integrated system able to tightly combine powerful servers with an easy to use graphical user interface. The installation of Etere has been smooth although it represented a big jump for VIM; the content migration to a digital environment has been fully supported with the most of accuracy and reliability derived from the intelligent use of workflowbased technology.

The provided Etere MERP solution now takes care of the Vietnamese musical history archived in the VIM facility, all this, making use of a wide series of modules, specifically designed and tightly integrated, to cover each single operation required by the customer, all Etere modules used across the VIM installation will be listed and explained in detailed through the following chapters, illustrating not only their single efficiency but specially their stunning integrated overall functioning.

4.1. CONTENT CREATION, DESCRIPTION AND RETRIEVAL
Etere’s Media Library module has been able to manage all VIM’s required musical metadata, providing both, an Etere desktop and an Etere Web interface. This chapter details the requirements concerning the metadata management of music asset forms on the VIM Project.

a. Content creation
The Media Library allows creating a placeholder (i.e. asset form) for managed content even before their acquisition, thus making it possible to insert their metadata and media. The flexibility of Etere has permitted to introduce into the Media Library all the metadata required to enrich musical content with custom SMPTE information. A group of metadata profiles has been defined to allow their immediate association with just one click.
The “kind of music” of content is managed through the use of fully customizable “genres”, just as illustrated in the picture below:

Further music information is defined under the asset properties, more specifically, as the asset’s music data, thus allowing a specialized management of musical content:

In addition to the previous metadata, the “content of song” field is managed under a specific field which permits to enter an extended description of the related musical content:

Vocal music has been improved with the inclusion of song “lyrics” under a dedicated field.
The “Artist Manager” allows creating artist profiles and associates them with specific roles (e.g. as “performers”, “producers”, “craftsmen”, etc.) to musical content.

d. Content retrieval
The Media Library counts with a robust search engine that permits operators retrieving content based on filters and full-text keywords as well as sorting and grouping results:

4.2. MEDIA AND METADATA MANAGEMENT
Etere MAM is the module that has permitted VIM to index and enrich media content through a comprehensive, fast and robust interface, allowing to search, browse and catalogue media with metadata profiles (either user-defined or SMPTE):

Etere MAM, thanks to a very intuitive interface establishes a bridge between the ingest department and the production department, allowing contents to be browsed simultaneously from various workstations thus enabling low-res proxy browsing over the network (with further import/export functions from and to other systems respectively). Furthermore, Etere has provided VIM with a streamlined speech-to-text module for enabling operators to enrich contents without typing not even a word; the speech-to-text module allows dictating metadata and inserting it at specific time points while previewing contents, thus ensuring the best accuracy on content detailing as well as on future content retrievals:

4.2. INGEST AND STORAGE
The chosen solution for ingesting and previewing audio and video content has relied on Etere MTX, the cost-effective software solution that combines Matrox HD/SD digital technology with Etere distributed architecture. Etere MTX allows ingesting audio and video content from VTR devices in analog format (documents are acquired using specialized scanners) including:
■ Audio format: Linear 16 bits 48khz
■ Video format: Mpg2 long GOP 24Mb/sec MXF (or IMX50 or Uncompressed)

Etere MTX is the ingest server used by VIM for capturing content, the acquisition of content has been integrated in the Etere’s Media Library for a fastest capture (and workflow-based transfer to the proper media pool). Among the functions implemented by Etere for quickly ingesting content (e.g. stored in P2 cards) directly from the Media Library the following can be mentioned:

a. Video file import: Either, standard (browse for the video file to be imported into the MTX server and automatically associated to the asset form, afterwards, a workflow will be launched for processing the imported asset) or multi-file (browse for the video file to be imported into the MTX server and automatically associated to the asset form, one asset form will be created for each file, afterwards, a workflow will be launched for processing each imported asset):

b. Audio file multi-import: Browse for the audio file to be imported into the MTX server or record a new audio file to be stored into the MTX server

4.3. DATA TRANSFERS
Etere Media Manager allows moving media files between different devices within the VIM archive, thus guaranteeing through a workflow-based media management a superb timing and effectiveness on media transfers between devices. Etere Media Manager goes beyond the simple copy concept since it not only moves content based on custom policies but also transcodes content when required due to its tight integration with the Etere HiRes Transcoder module:

VIM has expressed its satisfaction about the efficiency of data transfers, it’s all owned to the Etere’s distributed processing capability for using an independent agent per data transfer and parallel processing for simultaneously performing different instances of one data transfer; these features have permitted VIM to exploit high computing resources to use a single workstation to perform multiple transfers.

4.4. ARCHIVING REDUNDANCY
All multimedia contents are archived using Etere HSM, the best solution for improving the redundancy of valuable content by storing it on long-term tapes. It is worthy mentioning that, for each ingested content, there are produced three different copies, two of them are kept online in the tape library whereas the third one is archived on another facility for a reliable redundancy.

The VIM institute has guaranteed the archiving of media content for an unlimited time due to the use of a PetaSite tape library provided with 500 LTO5 slots (about 400 TB) smartly piloted and managed by Etere HSM, which will automatically archive and restore content via workflow every time it is required by the main system.

4.5. TRANSCODING CAPABILITIES
Etere Hi-Res Transcoder is the provided solution that allows VIM to perform a completely file-based media transcoding processor supporting any-to-any format conversions, video standard conversions and audio leveling.

The implemented transcoding system has been tightly integrated with the workflow-based Etere Media Manager, thus allowing VIM to produce a browsing copy of all content video (WMV), audio (WMA) and picture (JPG) content in order to deliver archived content in the right and best format to all users.

4.6. QUALITY CONTROL
The file-based quality check of archived content is performed automatically via workflow by Etere CMS, the module that ensures detecting and logging audio/video issues (e.g. freeze frames, black frames, scene changes, audio loss) according to their quality.

Etere CMS has streamlined the operational efficiency of file-based quality controls across the entire content life cycle, either prior (e.g. playout, repurpose, archiving) or after (e.g. encoding, ingest, editing, transcoding, archiving) a media operation.

4.7. WEB CONNECTIVITY
Etere Web is the solution implemented by Etere to make the VIM musical archive accessible via internet to the entire world. Etere Web is a module empowered with a robust search engine as well as several other Media Library features that will allow researches all around the globe to not only retrieve and preview content data everywhere using any web browser (e.g. Internet Explorer, Mozilla Firefox and Apple Safari) but also to manage content as in the main facility:

Etere Web allows authorized users previewing content in compressed format (WMV) through a module based on the latest MS Silverlight technology. Moreover, the upload and download of content is also possible, being required for web users to apply a request to the system administrator in order to download an online media file. Additionally, Etere Web has permitted integrating NLE systems (e.g. FCP and Avid) with the VIMs workflow, providing non-linear editors with an authentication web portal for managing media contents through a user-friendly interface.

4.8. WORKFLOW-BASED DELIVERY
Etere Workflow is an integrated and professional approach for optimizing the workflow of any media company, it facilitates the overall process control while reducing operational costs while improving media management processes such as the archive and transcoding of media files:

The implemented Etere Workflow solution has permitted VIM to implement a delivery system for receiving assets and metadata from anywhere in any format (thus allowing NLE stations to deliver their produced content in any “mov”, “mxf” or “mpg” format) as well as a download system which permits web user to seek confirmation for sensitive download operations, set specific delivery rules or launch automatic post-upload operations. All workflows have been implemented according to VIM to fit the real requirements and
giving complete control over the overall system management which offers: Clear definition of each complex step of the broadcasting process, Visual representation of each step mapped out on a PC not in a paper document, Set of instructions and authorizations that must be followed in order to move forward and a complete log of all steps carried out, operations denied etc.

4.9. WORK ORDERS MANAGEMENT
The organization tasks and costs derived from the VIM digitizing process is completely managed by Etere’s Work Order Management, the flexible solution that permits VIM to manage all operations involved in media management (e.g. tape copying, media editing, etc.). As shown in the pictures above, the Etere’s Work Orders Management has also allowed VIM to administrate the requests arriving from web-users for downloading specific content, thus permitting VIM to have complete control over the content distribution through a time-fixed permissions system.

Work orders are based on a custom structure and are fully integrated into the workflow management, being possible to create user-defined work orders and include them in automated workflows which current status can be consulted at any time in real-time from either a desktop or web interface.

4.10. SOA SUPPORT
An implemented Etere SOA service-oriented interface has allowed VIM to connect its main system with a custom application for distributing its enterprise content archive across Vietnam. More specifically, the Etere Web Services product has been selected as the web-based solution for interfacing the VIM’s archive with other external SOA-compliant systems, thus allowing VIM to turn its media archive into a web-resource.

4.11. TAPE MANAGEMENT
Etere Tape Management has allowed VIM to manage its current tape archive (video tapes,
data tapes, discs) through a set of modules specifically designed to perform the VIM’s most important tape-related operations; these modules are:
■ Tape Archive: Insert and define tapes into the tape inventory and search them through a robust search engine that permits to locate tapes based on specific properties
■ Tape Library: Easy upload/download of tapes from racks; view the main information of a tape and manage its content directly under a rack-view
■ Tape Loan: Perform loans and returns of tapes and consult all the information related to these operations (e.g. loaner, borrower, responsible, dates, etc.)
■ Tape Measurements: Assign new/existing tapes to assets and detail their tape recording information required for their subsequent ingestion (e.g. SOM, EOM, EDL)
■ Racks: Locate tapes within specific racks and libraries and consult some of their more important storage properties (e.g. free space, used space, etc.)

Etere has take note of the importance of the management of tapes within VIM workflow, that’s why VIM has been provided with a wide range of features for facilitating and improving the daily management of tapes, most important tape-related functions includes:
■ Barcode generation and recognition
■ Highly detailed tapes information
■ Standard and quick loan procedures
■ Automatic notification of requested tapes
■ Required permissions for retrieval operations
■ Simple location of tapes within physical racks
■ Easy dubbing for tapes copying
■ Periodical quality control of stored contents,
■ SNMP monitoring of all system operations

4.12. REAL-TIME MONITORING
VIM has been provided with two Etere SNMP Consoles for allowing supervisors to monitor all Etere modules and processes based on custom notification policies (content, validity, filters). All the system information (errors, warnings, etc.) is collected and stored through locally installed agents which send messages containing this information to a remote Etere SNMP Console. Moreover, the system has been improved with the inclusion of a specific database maintenance plan and the installation of a powerful and versatile reporting service.

4.13. SYNCHRONIZED TIMES
An Etere Time Flooder solution has been installed across VIM to keep all network computers synchronized to a reference time, thus ensuring a frame-accurate environment which is crucial for any enterprise media workflow. A Time Flooder module has been implemented based on a timecode board to provide a time-reference to all Time Receivers connected to the network.

5. PRESENT AND FUTURE BENEFITS
This paper has described how the development and deployment of a comprehensive Eterebased “Enterprise Musical Archive Management System” system has been able to provide VIM with a large number of operational benefits and advantages derived from the correct use of ultimate media management technology; VIM’s digital contents is now managed by Etere from acquisition to delivery thus providing VIM with the following present and future benefits;
■ Usability, a simple and sophisticated interface guarantees a smooth use whereas a robust metadata definition and a Vietnamese GUI ensure an easy and intuitive management
■ Reliability, derived from the use of streamlined workflows provided with detailed logs for tracking both the overall and individual functioning of the entire system
■ Flexibility, on meeting all requirements by proving a versatile media management system tightly integrated with all the metadata and equipment managed by the customer
■ Scalability, for increasing the number of capturing channels and devices without altering the workflow complexity, thus minimizing operational overheads and reducing overall costs
■ Efficiency, achieved by reducing repetitive manual operations, allowing to define them in advance and then include them in automatic preset workflows, thus increasing productivity
■ Accuracy, during the whole media management process, reducing the risk of mistakes when archiving data since all archived contents are continuously checked and monitored
■ Integration, Non-linear editing systems will be able to access the digital archive directly and efficiently through a web interface empowered with the highest security standards
■ Security, all operations are based on specific user permissions based on either the station’s Active Directory domain or custom groups defined for accessing and using media
■ Suitability, for meeting all the customer requirements, providing solutions for easily ingesting audio, quickly associating metadata profiles or dictating speech-to-text metadata

About Etere
Etere is an international leader in the media market. Etere develops and distributes a wide range of high technology software for broadcasting and media businesses. With more than 25 years of experience, Etere provides powerful, flexible, cost-effective, high-performance, end-to-end media solutions. Etere is the only company worldwide that can offer you a solution to all your media needs in one single package.

Etere is the only solution 100% workflow based for all broadcast and media environments. It’s a common framework where there is real-time sharing of all the data among several applications to manage all media business requirements. The workflow approach allows a fully customized design with edge performances. Etere guarantees the best after-sales support service on the market with engineers ready to give professional assistance 24 hours a day, 7 days a week. The service includes voice, email, VPN and VoIP with unlimited calls and connection time, and a pro-active system to help diagnose problems before they appear.