Special Issue "Information-Theoretic Approaches to Explaining Linguistic Structure"

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: closed (15 December 2021).

Special Issue Editors

Dr. Kathleen Currie Hall
E-Mail Website
Guest Editor
Department of Linguistics, University of British Columbia, Vancouver, BC V6T 1Z4, Canada
Interests: laboratory phonology; information theory; corpus phonology; signed languages
Dr. Uriel Cohen Priva
E-Mail Website
Guest Editor
Department of Cognitive, Linguistic & Psychological Sciences, Brown University, Providence, RI 02912, USA
Interests: phonology; functional pressures on language; information theory and language
Dr. Richard Futrell
E-Mail Website
Guest Editor
Department of Language Science, University of California Irvine, Irvine, CA 92697, USA
Interests: language processing; typology; syntax

Special Issue Information

Dear Colleagues,

Information theory is a highly generic and powerful mathematical framework for analyzing communication systems. In recent years, there has been renewed interest in using this framework to understand linguistic structures. This is in part because language is a communication system that enables effective communication, subject to cognitive, physical, and social constraints, on how we encode, transmit, receive, decode, and store linguistic content. Information theory provides ways not only to formalize these constraints, but also ways to study how they affect the structure of the resulting communication systems. Information theory thus provides a bridge between linguistic function and linguistic form.

In this Special Issue, we invite contributions applying information theory to explain why and how particular linguistic phenomena arise at all levels of linguistic analysis, such as phonetics, phonology, morphology, syntax, semantics, and pragmatics, as well as cross-cutting areas such as sociolinguistics, historical linguistics, acquisition, and language processing.

We hope this collection of papers will serve to create a theoretical common ground both within linguistics, and between linguistics and other fields that use information-theoretic principles as explanatory tools.

 

Dr. Kathleen Currie Hall
Dr. Uriel Cohen Priva
Dr. Richard Futrell
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All papers will be peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • information theory
  • linguistic structure
  • linguistic explanation
  • phonetics
  • phonology
  • morphology
  • syntax
  • semantics
  • pragmatics
  • historical linguistics
  • sociolinguistics
  • language processing
  • language acquisition

Published Papers (3 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

Article
Quantifiers in Natural Language: Efficient Communication and Degrees of Semantic Universals
Entropy 2021, 23(10), 1335; https://0-doi-org.brum.beds.ac.uk/10.3390/e23101335 - 14 Oct 2021
Viewed by 708
Abstract
While the languages of the world vary greatly, they exhibit systematic patterns, as well. Semantic universals are restrictions on the variation in meaning exhibit cross-linguistically (e.g., that, in all languages, expressions of a certain type can only denote meanings with a certain special [...] Read more.
While the languages of the world vary greatly, they exhibit systematic patterns, as well. Semantic universals are restrictions on the variation in meaning exhibit cross-linguistically (e.g., that, in all languages, expressions of a certain type can only denote meanings with a certain special property). This paper pursues an efficient communication analysis to explain the presence of semantic universals in a domain of function words: quantifiers. Two experiments measure how well languages do in optimally trading off between competing pressures of simplicity and informativeness. First, we show that artificial languages which more closely resemble natural languages are more optimal. Then, we introduce information-theoretic measures of degrees of semantic universals and show that these are not correlated with optimality in a random sample of artificial languages. These results suggest both that efficient communication shapes semantic typology in both content and function word domains, as well as that semantic universals may not stand in need of independent explanation. Full article
(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)
Show Figures

Figure 1

Article
Characterizing the Typical Information Curves of Diverse Languages
Entropy 2021, 23(10), 1300; https://0-doi-org.brum.beds.ac.uk/10.3390/e23101300 - 02 Oct 2021
Viewed by 586
Abstract
Optimal coding theories of language predict that speakers will keep the amount of information in their utterances relatively uniform under the constraints imposed by their language, but how much do these constraints influence information structure, and how does this influence vary across languages? [...] Read more.
Optimal coding theories of language predict that speakers will keep the amount of information in their utterances relatively uniform under the constraints imposed by their language, but how much do these constraints influence information structure, and how does this influence vary across languages? We present a novel method for characterizing the information structure of sentences across a diverse set of languages. While the structure of English is broadly consistent with the shape predicted by optimal coding, many languages are not consistent with this prediction. We proceed to show that the characteristic information curves of languages are partly related to a variety of typological features from phonology to word order. These results present an important step in the direction of exploring upper bounds for the extent to which linguistic codes can be optimal for communication. Full article
(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)
Show Figures

Figure 1

Article
A Refutation of Finite-State Language Models through Zipf’s Law for Factual Knowledge
Entropy 2021, 23(9), 1148; https://0-doi-org.brum.beds.ac.uk/10.3390/e23091148 - 01 Sep 2021
Viewed by 915
Abstract
We present a hypothetical argument against finite-state processes in statistical language modeling that is based on semantics rather than syntax. In this theoretical model, we suppose that the semantic properties of texts in a natural language could be approximately captured by a recently [...] Read more.
We present a hypothetical argument against finite-state processes in statistical language modeling that is based on semantics rather than syntax. In this theoretical model, we suppose that the semantic properties of texts in a natural language could be approximately captured by a recently introduced concept of a perigraphic process. Perigraphic processes are a class of stochastic processes that satisfy a Zipf-law accumulation of a subset of factual knowledge, which is time-independent, compressed, and effectively inferrable from the process. We show that the classes of finite-state processes and of perigraphic processes are disjoint, and we present a new simple example of perigraphic processes over a finite alphabet called Oracle processes. The disjointness result makes use of the Hilberg condition, i.e., the almost sure power-law growth of algorithmic mutual information. Using a strongly consistent estimator of the number of hidden states, we show that finite-state processes do not satisfy the Hilberg condition whereas Oracle processes satisfy the Hilberg condition via the data-processing inequality. We discuss the relevance of these mathematical results for theoretical and computational linguistics. Full article
(This article belongs to the Special Issue Information-Theoretic Approaches to Explaining Linguistic Structure)
Back to TopTop