Skip to yearly menu bar Skip to main content


A Statistical Investigation of Long Memory in Language and Music

Alexander Greaves-Tunnell · Zaid Harchaoui

Pacific Ballroom #253

Keywords: [ Time Series and Sequence Models ] [ Natural Language Processing ] [ Deep Sequence Models ] [ Deep Learning Theory ]


Representation and learning of long-range dependencies is a central challenge confronted in modern applications of machine learning to sequence data. Yet despite the prominence of this issue, the basic problem of measuring long-range dependence, either in a given data source or as represented in a trained deep model, remains largely limited to heuristic tools. We contribute a statistical framework for investigating long-range dependence in current applications of deep sequence modeling, drawing on the well-developed theory of long memory stochastic processes. This framework yields testable implications concerning the relationship between long memory in real-world data and its learned representation in a deep learning architecture, which are explored through a semiparametric framework adapted to the high-dimensional setting.

Live content is unavailable. Log in and register to view live content