Audio-Novel

Analysis of using GPT-3 as a co-narrator in live improvisational theatre. Covers methodology, audience/performer feedback, and implications for human-AI creative collaboration.

Comprehensive technical documentation and resources about audio-novel technology and applications.

Research on retrieving audio using free-form natural language queries, introducing new benchmarks and baselines for cross-modal audio retrieval.

Analysis of AudioBoost, a system using LLMs to generate synthetic queries from audiobook metadata to improve retrieval and query suggestions in Spotify's cold-start scenario.

Analysis of Audiobook-CC, a novel speech synthesis framework for generating coherent, emotionally expressive multicast audiobooks with fine-grained control and long-context modeling.

Analysis of Audiobook-CC, a novel TTS framework for generating coherent, emotionally expressive, and contextually consistent multicast audiobooks with fine-grained control.

Spotify's innovative 2T-HGNN system using Graph Neural Networks for audiobook recommendations, achieving +46% start rate and +23% streaming rate improvements.

A novel approach for weakly-supervised temporal grounding of procedural steps in instructional videos using multi-modal alignment of frames, narrations, and step descriptions from wikiHow.

A novel meta-learning approach (MAMLCon) that mitigates catastrophic forgetting in continual few-shot learning for spoken word classification, outperforming existing methods like OML.

Analysis of integrating mobile audiobooks to develop listening comprehension skills in EFL college students, covering advantages, selection criteria, teaching phases, and assessment.

Analysis of Movie101v2, a large-scale bilingual dataset for movie narration, including its three-stage task roadmap, baseline evaluations, and future research directions.

An analysis of research investigating how background music with and without lyrics impacts spoken-word recognition, with implications for social settings and future work.

An interactive authoring tool that simplifies the creation of animated data videos by linking narration to chart animations and providing structure-aware animation effects.

This paper introduces the novel task of narration generation for videos, presents a dataset from Peppa Pig, and proposes models for timing and content generation.

Analysis of a computational model exploring phonetic perception's role in non-native word processing, challenging traditional phonological explanations.

A two-stage framework for embedding spoken words with both phonetic and semantic information, enabling advanced spoken document retrieval beyond simple term matching.

Research on predicting prosody attributes (pitch, volume, rate) from narrative text using NLP and language models, improving TTS for audiobook generation.

Research on computational identification of unreliable narrators using LLMs, featuring the TUN A dataset and classification of intra-narrational, inter-narrational, and inter-textual unreliability.

A research paper exploring how to use noisy audio narration as weak supervision to train video action detection models, reducing annotation costs while leveraging multimodal features.

A zero-shot approach for generating expressive audiobooks using multimodal speaker personas and LLM-based script instructions, eliminating costly training and manual annotation.

A technical analysis of MultiActor-Audiobook, a novel zero-shot system for generating expressive audiobooks using multimodal speaker personas and LLM-based script instructions.