BMWK-funded research project focuses on intelligent methods for media analysis to support employees of media companies in their decision-making process
Artificial intelligence (AI) methods offer forward-looking applications and an increasingly important financial basis for media companies with a rapidly growing amount of digital content. AI can help reveal information in data and link it in a sensible way to support actions and decisions, for example by recommending suitable content.
The aim of the research project AI4MediaData, funded by the Federal Ministry for Economic Affairs and Climate Action of Germany (BMWK), was to analyze media data using AI methods and use the information obtained for data-based decision-making support. ZDF Digital, the Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS, DDG AG, and Mainz University of Applied Sciences worked together on the project, which ran from mid-2019 to the end of 2022. With the presentation of the final report on June 30, 2023, the project is now concluded.
Together with two media partners, two AI-based prototype applications were developed to support employees of media companies:
Asset Match enriches existing metadata with additional scene- and image-based metadata and is able to offer its users content-based recommendations. The application can, for example, support the sales team of media companies in the search for suitable content from their own archives and help them to reuse existing material more effectively.
The Highlight Search prototype offers an approach to identifying emotional highlights in reality TV programs. Based on audiovisual features and usage data, an algorithm was developed that identifies particularly emotional scenes in entire episodes of a TV program and outputs them for use by editors in the form of crop marks in customizable lengths to simplify the creation of short highlight clips. These clips can then be marketed after the initial broadcast across other (social media) platforms in generated formats of various lengths.
Volker Denkel, as the representative of consortium leader ZDF Digital, is delighted with the results: “Our research results unlock promising opportunities for the media industry. With AI4MediaData, we have created a solid basis for optimizing the handling of media content to ultimately better meet the needs of viewers.”
The AI4MediaData project has shown that artificial intelligence and advanced data analytics can have truly transformative power for the media landscape. The applications developed present new opportunities to discover, understand, and use media content to better target viewers and improve interaction. According to Prof. Dr. Sven Pagel, head of the WIMM research group at Mainz University of Applied Sciences, “the good collaboration between the four consortium partners and with the two media partners, which offers ideal opportunities for a target-oriented transfer of scientific findings to application-oriented (media) companies,” helped achieve this.
To make the results available to stakeholders in the media industry and to publish them in an industry-oriented manner, a series of publications on the overall project AI4MediaData (issue 10/2022) and the two sub-projects Asset Match (issue 11/2022) and Highlight Search (issue 12/2022) was published in the FKT - Fachzeitschrift für Fernsehen, Film und elektronische Medien.
You can find more information about AI4MediaData and the results here: ai4mediadata.hs-mainz.de.
ZDF Digital
ZDF Digital Medienproduktion GmbH, as consortium leader of AI4MediaData, has many years of experience in the development and marketing of media products. With around 350 employees, ZDF Digital is a medium-sized company that has already achieved a leading position in the field of media technologies. Due to its close affiliation with the ZDF broadcasting family, ZDF Digital is in partnership with and acts as a service provider for other media companies. In addition to leading the consortium, ZDF Digital was responsible for the development of Asset Match (together with Fraunhofer IAIS and Mainz University of Applied Sciences). ZDF Digital developed the matching algorithm for the semantic analysis and, in addition to the architecture, also designed and implemented the front end for the prototype of a mining platform based on user requirements.
DDG AG
DDG AG (formerly DDG-Digital Devotion Group GmbH) is a private company in the field of venture building. It transforms the challenges of SMEs into new digital business models. The solutions invariably arise from real-world challenges and are therefore as close to the market as possible. Within its unique co-innovation model, DDG AG combines the knowledge of SMEs with its technological and commercial expertise and pursues collaborative partnerships. This enables the transformation of traditional business models together with SME partners. DDG assumed responsibility for the development of Highlight Search together with Mainz University of Applied Sciences.
Fraunhofer IAIS
As part of the largest organization for application-oriented research in Europe, the Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS is one of the leading scientific institutes in the fields of artificial intelligence (AI), machine learning, and big data in Germany and Europe. The NetMedia department at Fraunhofer IAIS contributed its many years of experience in the development of AI-based technologies for media data analysis and indexing to the project. The experts integrated their established audio mining technology into the Asset Match application. As part of AI4MediaData, Fraunhofer IAIS also developed a number of other video mining services (such as facial recognition, object recognition, and concept recognition), which were also successively integrated into the Asset Match prototype.
WIMM research group at Mainz University of Applied Sciences
Mainz University of Applied Sciences, with its three schools of Design, Engineering and Business, was involved in the research project with its WIMM research group headed by Prof. Dr. Sven Pagel (Professor of Information Systems and Media Management, wimm.hs-mainz.de). The group’s main fields of research are moving image communication, user research, and digital transformation. The WIMM research group was primarily involved in UX and requirements analysis to ensure the user-friendliness of the developing solution. WIMM has already demonstrated this expertise in other research projects (EU EFRE, BMBF) and applied it to the funded project through the use of data thinking workshops and user tests.