Submit to Entropy Review for Entropy Propose a Special Issue

Journal Browser

Information-Theoretic Approaches to Explaining Linguistic Structure

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: closed (15 December 2021) | Viewed by 16908

Share This Special Issue

Special Issue Editors

Dr. Kathleen Currie Hall

E-Mail Website
Guest Editor

Department of Linguistics, University of British Columbia, Vancouver, BC V6T 1Z4, Canada
Interests: laboratory phonology; information theory; corpus phonology; signed languages

Dr. Uriel Cohen Priva

E-Mail Website
Guest Editor

Department of Cognitive, Linguistic & Psychological Sciences, Brown University, Providence, RI 02912, USA
Interests: phonology; functional pressures on language; information theory and language

Dr. Richard Futrell

E-Mail Website
Guest Editor

Department of Language Science, University of California Irvine, Irvine, CA 92697, USA
Interests: language processing; typology; syntax

Special Issue Information

Dear Colleagues,

Information theory is a highly generic and powerful mathematical framework for analyzing communication systems. In recent years, there has been renewed interest in using this framework to understand linguistic structures. This is in part because language is a communication system that enables effective communication, subject to cognitive, physical, and social constraints, on how we encode, transmit, receive, decode, and store linguistic content. Information theory provides ways not only to formalize these constraints, but also ways to study how they affect the structure of the resulting communication systems. Information theory thus provides a bridge between linguistic function and linguistic form.

In this Special Issue, we invite contributions applying information theory to explain why and how particular linguistic phenomena arise at all levels of linguistic analysis, such as phonetics, phonology, morphology, syntax, semantics, and pragmatics, as well as cross-cutting areas such as sociolinguistics, historical linguistics, acquisition, and language processing.

We hope this collection of papers will serve to create a theoretical common ground both within linguistics, and between linguistics and other fields that use information-theoretic principles as explanatory tools.

Dr. Kathleen Currie Hall
Dr. Uriel Cohen Priva
Dr. Richard Futrell
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

information theory
linguistic structure
linguistic explanation
phonetics
phonology
morphology
syntax
semantics
pragmatics
historical linguistics
sociolinguistics
language processing
language acquisition

Published Papers (6 papers)

Download All Papers

Research

15 pages, 7156 KiB

Open AccessFeature PaperArticle

Grammatical Gender Disambiguates Syntactically Similar Nouns

by Phillip G. Rogers and Stefan Th. Gries

Entropy 2022, 24(4), 520; https://0-doi-org.brum.beds.ac.uk/10.3390/e24040520 - 07 Apr 2022

Cited by 1 | Viewed by 2427

Abstract

Recent research into grammatical gender from the perspective of information theory has shown how seemingly arbitrary gender systems can ease processing demands by guiding lexical prediction. When the gender of a noun is revealed in a preceding element, the list of possible candidates is reduced to the nouns assigned to that gender. This strategy can be particularly effective if it eliminates words that are likely to compete for activation against the intended word. We propose syntax as the crucial context within which words must be disambiguated, hypothesizing that syntactically similar words should be less likely to share a gender cross-linguistically. We draw on recent work on syntactic information in the lexicon to define the syntactic distribution of a word as a probability vector of its participation in various dependency relations, and we extract such relations for 32 languages from the Universal Dependencies Treebanks. Correlational and mixed-effects regression analyses reveal that syntactically similar nouns are less likely to share a gender, the opposite pattern that is found for semantically and orthographically similar words. We interpret this finding as a design feature of language, and this study adds to a growing body of research attesting to the ways in which functional pressures on learning, memory, production, and perception shape the lexicon in different ways. Full article

(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)

► Show Figures

Figure 1

17 pages, 343 KiB

Open AccessArticle

Evolution and Trade-Off Dynamics of Functional Load

by Erich Round, Rikker Dockum and Robin J. Ryder

Entropy 2022, 24(4), 507; https://0-doi-org.brum.beds.ac.uk/10.3390/e24040507 - 05 Apr 2022

Cited by 1 | Viewed by 1888

Abstract

Functional load (FL) quantifies the contributions by phonological contrasts to distinctions made across the lexicon. Previous research has linked particularly low values of FL to sound change. Here, we broaden the scope of enquiry into FL to its evolution at higher values also. We apply phylogenetic methods to examine the diachronic evolution of FL across 90 languages of the Pama–Nyungan (PN) family of Australia. We find a high degree of phylogenetic signal in FL, indicating that FL values covary closely with genealogical structure across the family. Though phylogenetic signals have been reported for phonological structures, such as phonotactics, their detection in measures of phonological function is novel. We also find a significant, negative correlation between the FL of vowel length and of the following consonant—that is, a time-depth historical trade-off dynamic, which we relate to known allophony in modern PN languages and compensatory sound changes in their past. The findings reveal a historical dynamic, similar to transphonologization, which we characterize as a flow of contrastiveness between subsystems of the phonology. Recurring across a language family that spans a whole continent and many millennia of time depth, our findings provide one of the most compelling examples yet of Sapir’s ‘drift’ hypothesis of non-accidental parallel development in historically related languages. Full article

(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)

► Show Figures

Figure 1

16 pages, 1094 KiB

Open AccessArticle

Frequency, Informativity and Word Length: Insights from Typologically Diverse Corpora

by Natalia Levshina

Entropy 2022, 24(2), 280; https://0-doi-org.brum.beds.ac.uk/10.3390/e24020280 - 16 Feb 2022

Cited by 5 | Viewed by 3313

Abstract

Zipf’s law of abbreviation, which posits a negative correlation between word frequency and length, is one of the most famous and robust cross-linguistic generalizations. At the same time, it has been shown that contextual informativity (average surprisal given previous context) is more strongly correlated with word length, although this tendency is not observed consistently, depending on several methodological choices. The present study examines a more diverse sample of languages than the previous studies (Arabic, Finnish, Hungarian, Indonesian, Russian, Spanish and Turkish). I use large web-based corpora from the Leipzig Corpora Collection to estimate word lengths in UTF-8 characters and in phonemes (for some of the languages), as well as word frequency, informativity given previous word and informativity given next word, applying different methods of bigrams processing. The results show different correlations between word length and the corpus-based measure for different languages. I argue that these differences can be explained by the properties of noun phrases in a language, most importantly, by the order of heads and modifiers and their relative morphological complexity, as well as by orthographic conventions. Full article

(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)

► Show Figures

Figure 1

21 pages, 1894 KiB

Open AccessFeature PaperArticle

Quantifiers in Natural Language: Efficient Communication and Degrees of Semantic Universals

by Shane Steinert-Threlkeld

Entropy 2021, 23(10), 1335; https://0-doi-org.brum.beds.ac.uk/10.3390/e23101335 - 14 Oct 2021

Cited by 12 | Viewed by 2800

Abstract

While the languages of the world vary greatly, they exhibit systematic patterns, as well. Semantic universals are restrictions on the variation in meaning exhibit cross-linguistically (e.g., that, in all languages, expressions of a certain type can only denote meanings with a certain special property). This paper pursues an efficient communication analysis to explain the presence of semantic universals in a domain of function words: quantifiers. Two experiments measure how well languages do in optimally trading off between competing pressures of simplicity and informativeness. First, we show that artificial languages which more closely resemble natural languages are more optimal. Then, we introduce information-theoretic measures of degrees of semantic universals and show that these are not correlated with optimality in a random sample of artificial languages. These results suggest both that efficient communication shapes semantic typology in both content and function word domains, as well as that semantic universals may not stand in need of independent explanation. Full article

(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)

► Show Figures

Figure 1

14 pages, 340 KiB

Open AccessArticle

Characterizing the Typical Information Curves of Diverse Languages

by Josef Klafka and Daniel Yurovsky

Entropy 2021, 23(10), 1300; https://0-doi-org.brum.beds.ac.uk/10.3390/e23101300 - 02 Oct 2021

Cited by 1 | Viewed by 1885

Abstract

Optimal coding theories of language predict that speakers will keep the amount of information in their utterances relatively uniform under the constraints imposed by their language, but how much do these constraints influence information structure, and how does this influence vary across languages? We present a novel method for characterizing the information structure of sentences across a diverse set of languages. While the structure of English is broadly consistent with the shape predicted by optimal coding, many languages are not consistent with this prediction. We proceed to show that the characteristic information curves of languages are partly related to a variety of typological features from phonology to word order. These results present an important step in the direction of exploring upper bounds for the extent to which linguistic codes can be optimal for communication. Full article

(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)

► Show Figures

Figure 1

35 pages, 507 KiB

Open AccessArticle

A Refutation of Finite-State Language Models through Zipf’s Law for Factual Knowledge

by Łukasz Dębowski

Entropy 2021, 23(9), 1148; https://0-doi-org.brum.beds.ac.uk/10.3390/e23091148 - 01 Sep 2021

Cited by 3 | Viewed by 2689

Abstract

We present a hypothetical argument against finite-state processes in statistical language modeling that is based on semantics rather than syntax. In this theoretical model, we suppose that the semantic properties of texts in a natural language could be approximately captured by a recently introduced concept of a perigraphic process. Perigraphic processes are a class of stochastic processes that satisfy a Zipf-law accumulation of a subset of factual knowledge, which is time-independent, compressed, and effectively inferrable from the process. We show that the classes of finite-state processes and of perigraphic processes are disjoint, and we present a new simple example of perigraphic processes over a finite alphabet called Oracle processes. The disjointness result makes use of the Hilberg condition, i.e., the almost sure power-law growth of algorithmic mutual information. Using a strongly consistent estimator of the number of hidden states, we show that finite-state processes do not satisfy the Hilberg condition whereas Oracle processes satisfy the Hilberg condition via the data-processing inequality. We discuss the relevance of these mathematical results for theoretical and computational linguistics. Full article

(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)

Journal Menu

Journal Browser

Information-Theoretic Approaches to Explaining Linguistic Structure

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (6 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI