The Dichotomy of Neural Networks and Cryptography: War and Peace

Abstract

In recent years, neural networks and cryptographic schemes have come together in war and peace; a cross-impact that forms a dichotomy deserving a comprehensive review study. Neural networks can be used against cryptosystems; they can play roles in cryptanalysis and attacks against encryption algorithms and encrypted data. This side of the dichotomy can be interpreted as a war declared by neural networks. On the other hand, neural networks and cryptographic algorithms can mutually support each other. Neural networks can help improve the performance and the security of cryptosystems, and encryption techniques can support the confidentiality of neural networks. The latter side of the dichotomy can be referred to as the peace. There are, to the best of our knowledge, no current surveys that take a comprehensive look at the many ways neural networks are currently interacting with cryptography. This survey aims to fill that niche by providing an overview on the state of the cross-impact between neural networks and cryptography systems. To this end, this paper will highlight the current areas where progress is being made as well as the aspects where there is room for future research to be conducted.

1. Introduction

In recent years, artificial neural networks (ANNs), simply referred to as neural networks (NNs) have been of great interest to the research community. They consist of layered networks of nodes meant to mimic a human brain, where the nodes represent biological neurons and the connections between them represent the synapses. Neural computing, as technology and a field of research has a wide ecosystem. It is in close interaction with many scientific and technological fields. NNs support a range of technological fields including medical technology [1] as well as image processing [2], cloud computing [3], aerospace technology [4], meteorology [5], and especially in security-related technologies [6,7]. Moreover, several technologies and sciences such as chaos theory [8], frequency-domain transforms [9], genetic algorithms [10] and Digital Signal Processing (DSP) [1] are supporting neural networks as enablers.

In this paper, we focus on the intersection of neural computing with cryptography. Cryptography is the study of mathematical techniques as a means of securing data communication and storage. Like the case of neural computing, cryptography has a broad ecosystem consisting of different scientific and technological fields. It supports a variety of technologies including Internet of Things (IoT) [11], cloud computing [12], Fog computing [13], etc. [14]. It also serves to security-related scenarios such as information hiding [15], authentication [16,17], privacy [18], etc. Furthermore, cryptography is supported by a variety of enabling technologies and sciences including radix $2^n$ [19] and modular arithmetics [20], quantum computing [21,22], coding and information theory [23,24], Very Large Scale Integration [25], chaos theory [26], and error management techniques [27] are used to support cryptography.

Neural computing and cryptography frequently appear in the ecosystems of each other. They come together, in war and peace, in many ways; NNs can play adversarial roles against cryptosystems while they can support and be supported by cryptography at the same time. This tight interaction forms a dichotomy. Figure 1 illustrates the two sides of this war-and-peace dichotomy.

In Figure 1, the two bears facing each other represent the war or the adversarial role of NNs in breaking cryptographic systems. The teddy bear represents the peaceful give and take between the two technologies including the use of cryptography to secure NNs or the use of NNs in order to improve cryptosystems.

A comprehensive survey on the war-and-peace dichotomy shown in Figure 1 can pave the way for future research. In this paper, we first summarize some existing relevant surveys. We highlight the shortcomings of these surveys to motivate this research. We will then discuss the current state of the war; the ways in which NNs are being used to break cryptogrphic systems. We will then study the state-of-the-art in the peace side; how the two technologies work in concert with each other, cryptography providing confidentiality and privacy guarantees for NNs, and NNs adding to the functionality and the security of cryptographic systems. Lastly, we will discuss how quantum computing might fit into the future of this dichotomy. We briefly discuss what the future may hold for the war-and-peace dichotomy under the impact of quantum computing.

1.1. Contributions

This survey aims to highlight the current state of the dichotomy described above, with the goal of highlighting areas for future research where the two technologies intersect. This research is unique in the broadness of it’s scope. Our goal is to cover as many of the interactions between cryptography and NNs as possible. There are some surveys that cover limited aspects of this dichotomy, some covering out of date research, and others failing to look to the future of the field. We aim to address these niches via providing a broad study on different aspects of the of NN-crypto dichotomy, and using that for a look into the future.

The contributions of this paper can be listed as follows.

1.

Our paper is the first of its kind to analyze the cross-impact between neural networks and cryptography and decompose it into two opposite sides.

• War: In this side, we study the aggressive activities assisted by neural networks against cryptography. We investigate the role of NNs in cryptanalysis and attacks against cryptographic systems.
• Peace: This side consists of the ways cryptography and NNs mutually support each other. We study this mutual support in the following lines.

  -

  Cryptographic techniques, mechanisms and devices can be used to provide confidentiality for NNs and their processed data.

  -

  Neural networks can be applied in the design of cryptosystems aiming at improved security and efficiency.

2.

In addition to shedding light on the current state in the dichotomy of NNs and cryptography, we establish a future roadmap for further research in this area. This roadmap is developed in consideration of future computing paradigms and expected advancements.

1.2. Organization

The rest of this paper is organized as follows. Section 2 studies existing surveys. Section 3 reviews research works focusing on the use of NNs against cryptography. Section 4 shows how neural computing and cryptography can coexist, cooperate and support each other. Section 5 provides directions for future research on the interaction between neural computing and cryptography. Lastly, Section 6 concludes the paper.

2. Existing Surveys and Motivations

Although there might be some related surveys, their shortcomings motivate the work of this paper. These shortcomings can be itemized as follows.

• There are a few reviews on the role of artificial intelligence (AI), and especially NNs in cryptography. However, to the best of our knowledge, there is no comprehensive survey on the roles of cryptography in secure neural networks. Moreover, there is no survey on the role of neural networks in aggressive activities against cryptography.
• Some existing surveys are too outdated for such a dynamic research area [28,29,30].
• Most of them fail to develop directions for future research in this area [31,32].

The above shortcomings highlight the importance of our work in this paper with the contributions mentioned in Section 1.

In the following, some relevant surveys are briefly discussed ordered by their publication year.

There have been several past surveys studying the relationship of AI and cryptography. Figure 1 shows the general relationship between the two technologies, in places they are adversaries and others they combine to create a benevolent system. Some older surveys study technology which is no longer relevant to the current discussion. Many of the newer surveys take too narrow field to garner insight into the future of the field.

Vandewalle et al. looked at the current state of cryptography by defining the goals of security systems and discussing possible solutions [28]. Concerning NNs this study mentions the application of cellular NNs (CeNNs) too implement Boolean mappings and the application of cellular automata (CeA) to encrypt data.

Ten years later Schmidt et al. summarized uses of NNs in private key systems, an image encryption scheme, pseudo random number generation, the analysis and generation of digital watermarks [29]. That same year Hu and Wang proposed using a specific NN model for 802.16 to share private keys over a public channel through training synchronization [30].

Hadke and Kale provided another broad review of NN’s ability to improve on current cryptographic techniques [31]. This paper fails to mention any techniques that the the 2008 review did not mention, but it gives an overview of some attacks on NN-improved cryptography systems.

Sharma and Sharma compared key exchange schemes using NNs and quantum computers as means to improve on current key management systems in 2016 [32]. This paper concluded that NN Key exchange is more practical but has yet to be practically implemented and the theory needs hardening against unconventional attacks.

ÖzÇakmak et al. published a paper in 2019 referencing the work of Sharma and Sharma also comparing NNs and quantum computers ability to improve key exchange systems [33]. This paper also come to the conclusion that current quantum solutions cannot be broadly implemented and NNs could be used but it does not bring up any specific implementations or steps needed to further research.

Su et al. studied current image encryption techniques using NN to propose a possible architecture for encrypting medical images [34].

Meraouche et al. [35] published a comprehensive survey documenting various NN cryptography techniques along with verified attacks and some published solutions for the attacks. Through this survey Meraouche et al. found that cryptographic NN systems are trending in two directions, providing a low computation cost encryption solution and being a viable post-quantum encryption primitive.

Table 1. Summary of existing surveys.

Survey	Year	NN-Peace-Crypto	Crypto-Pace-NN	NN-War-Crypto	Future Roadmap
[28]	1998	yes	no	no	no
[29]	2008	yes	no	no	no
[30]	2008	yes	no	no	no
[31]	2016	yes	no	no	no
[32]	2016	yes	no	no	yes
[33]	2019	yes	no	no	no
[34]	2020	yes	no	no	yes
[35]	2021	yes	no	no	yes

provides a summary of existing surveys in order to make it easy to see their shortcomings and compare them with our survey.

In Table 1, each entry in the first column contains one of the surveys studied in this section. The second column indicates the publication year of the related survey. Outdated surveys can be identified using this column. The next three columns indicate the aspects of the NN-crypto dichotomy (partially) covered by each survey. The third column indicates whether or not the survey discusses the roles of NN in improvement of cryptosystems. The fourth column contains “yes” if the survey studies the roles of cryptography in protection of NNs. It contains “no” otherwise. Surveys focusing on the adversarial role of NN against cryptography are designated by a “yes” in the fifth column. Lastly, the sixth column demonstrates whether or not the related survey builds a future roadmap.

3. War: Neural Computing against Cryptography

There is no research work focusing on the activity of cryptography against NNs. However, NNs are used against cryptography in different ways.

Cryptography schemes are designed so that it is hard to garner information from their output. If implemented correctly many cryptography schemes should be secure against analysis. However there are several proposed methods to use neural computing to break or otherwise perform cryptanalysis on different encryption schemes.

3.1. Detecting Malicious Encryption

Detecting and locating malicious encrypted data can help detecting ransomware and other kinds of malware. Some researchers have focused on the applications of NNs in this area [36]. Some research works focusing on malicious encryption detection in software code and network traffic are briefly reviewed in the following.

Cryptography can be used to hide malicious data until it is needed. When encrypted data is detected and it cannot be decrypted, there is normally no way to tell if it is malicious or not. It has been proposed to use machine learning models to identify encrypted files and network traffic as malicious.

3.1.1. Software Code

Cryptography is a tool that can protect the confidentiality of data, while often used to the benefit of users it can be used by attackers to hide malware on a system until it is needed or obfuscate network traffic. It can be difficult to identify malware that is able to encrypt data as it might require reverse engineering the suspicious program and applying a thorough understanding of many encryption methods [37]. It has been found that NNs could simplify the process of detecting encryption in obfuscated programs. This section will discuss some researchers applying NNs to detecting encrypted malicious code.

Wright and Manic [38] built a NN system trained with error back propagation that analyzes the ratio of certain opcodes that are commonly used by cryptographic algorithms. Trained on functions in OpenBSD it was found this system could identify most typical encryption functions regardless of compiler optimization or specific implementation, but failed to detect methods that operate sufficiently different from typical encryption methods such as elliptical curve cryptography and public key encryption.

Jia et al. [37] propose solving this issue with a NN model called K-max-CNN-Attention that looks for common instruction patterns rather than relative instruction density. The improvements this model brings is in the move to a convolutional NN (CNN), which interpret blocks of data maintaining the original structure and a better preprocessing scheme which simplifies the input enough to be meaningful to the NN but leaving more information to be interpreted, while these changes improve on performance and accuracy of existing techniques Jia et al. [37] speculate that better accuracy could be achieved by changing the preprocessing and classification models to consider non-sequential execution of code.

3.1.2. Network Traffic

Due to the increase in VPN usage by average users, companies have begun looking for solutions and security models in order to distinguish between a legitimate connection and a malicious connection [39]. One group has used flow-based statistics to classify TCP layer traffic as VPN or Non VPN connections. These flow statistics are TCP flows taken as time-based statistics and other key features found using Pearson’s correlation coefficient algorithm [39]. Using a multilayered perceptron NN trained from the features selected, the model was successfully able to distinguish 92% of the data given during the validation phase making it feasible to tell whether a connection is using a VPN or not [39]. What this model of VPN identifying does not consider is the identification of non VPN traffic that may be malicious and encrypted. However, other models that use a combination of encryption and abnormality detection can solve this problem [40]. In order to provide high performance and data integrity, this model will use clustering and feature vector expansion to improve the quality of their data [40], while this model does seem provide better performance and classification rates that traditional methods, it is subject to the data imbalance problem due to the diversity of encrypted malicious attacks.

3.2. Cryptanalysis

Cryptanalysis is the processes of finding vulnerabilities in ciphers by studying their operation. This is usually completed with knowledge of some combination of the cipher text, plain text, and key. Though it is often associated with attacks on cryptogrphic systems, cryptanalysis can be used to audit current systems to improve them. In some cases, cryptanalysis can be considered as an aggressive activity against cryptosystems, because it can be conducted in an initial stage of an attack. There have been some attempts at creating cryptanalysis attacks to directly decrypting cipher text using NNs. Apolinhrio Jr. et al. found an early cryptanalysis application for NNs reordering blocks of audio data scrambled with time segment permutation scrambler [41]. Their Hopfield network, trained with simulated annealing with a “genetic algorithm” approach on a small set of words, was able to meet the performance of exhaustively searching for a solution [41].

Ruzhentsev et al. attempted to apply neural nets to decrypt 8 bit cipher texts from a substitution permutation cipher using the same key for the training set and the test set [42], while they did not find complete success decrypting cipher texts this way, they found it possible to decrypt 232 out of the 256 possible cipher texts with the average number of wrong bits in each erroneous decryption being 1.3.

Other papers focused on applying NNs to determine some amounts of the key used to encrypt a cipher text, possibly reducing the amount of time needed to guess the correct key. Abassal and Wahdan propose an attack specific to Feistel block ciphers using NNs to determine the key but was limited to only the cipher the neural net is designed for [43].

Danziger and Henriques studied how effective NNs could be at cryptanalyzing S-DES ciphers [44]. In particular they attempted to determine key bits based on a given plain text and it’s corresponding cipher text. Their test found that certain bits of the key were more easily determined due to problems with the substitution box being used by the cipher.

Xiao et al., utilized NNs to quantify the strength of ciphers by predicting cipher text based on the plain text [45]. Using this approach only ciphers as strong as DES-3 were successfully mimicked, though the researchers suggested that using a neural net architecture that more closely reflected that of the cipher being analyzed would be more capable.

NNs have also been used to analyze cipher operation in order to encrypt data. Khan et al. tested the application of neural nets to directly replicate the functionality of a cipher by training some models on cipher, plain text pairs with and without knowledge of the key [46]. Hsiao et al. proposed a different system, training a NN to analyze the output of multiple time-delay chaotic system in order to approximate its output and synchronize a second multiple time-delay chaotic system as a means for establishing an encrypted communication over a public channel [47].

3.3. Vulnerability Analysis

Vulnerability analysis is the act of evaluating threats to security or cryptographic systems and networks. This process will identify and assess these systems and networks for flaws that could lead to exploitation by malicious actors.

Like the case of cryptanalysis, vulnerability analysis can help an adversary design the attack scenario against a cryptosystem. There are a few research works where NNs have been used to analyze the vulnerabilities of cryptosystems. The use of feed-forward NNs FNNs has been applied to the vulnerability analysis of Physical Unclonable Functions (PUFs) [48]. More specifically FNNs are being used to model out attack scenarios against the Challenge Response Pairs (CRPs) of the PUFs [48]. It was found that given very few CRPs as a baseline, an FNN using the Dragonfly Algorithm(DA) was able to accurately predict more challenge–response pairs with a 85.2% accuracy attacking the Configurable Ring Oscillator PUF and 71.3% against the XOR-inverter Ring Oscillator PUF [48]. DA works by moving neurons as dragonflies, moving them closer to the goal (food sources) and away from bad predictions (enemies) by using the neurons as dragonflies on a dimensional grid with speed and velocity [48].

3.4. Attack

Some researchers have been able to develop attacks on cryptogrphic systems leveraging the features of NNs. In some research works, NNs have been directly used to attack cryptosystems. It has been shown that symmetric ciphers can be broken by using Real-time Recurrent NNs (RRNN) with Chosen Plaintext Attack (CPA) [49].

As another example, UFnet, a NN using ReLU activation functions and Xavier initialization techniques, can predict the responses of double-arbiter physically unclonable functions (PUFs) [50].

4. Peace: Coexistence and Alliance

NNs have also been integrated with cryptography systems. Section 4.1 will discuss NNs trained on encrypted data and Section 4.2 will discuss encryption schemes that make use of NNs on some level.

4.1. Coexistence

It has been proposed to use NNs trained on encrypted data. Since there is a noticeable performance drop when using encrypt data, either in processing time accuracy or both. The methods to address this are twofold, developing infrastructure that can better support using encrypted data or tailoring the encryption scheme and the Neural net to improve accuracy and reduce training and processing time.

4.1.1. NNs Adapted to Encrypted Data

Researchers are working on the design of NNs capable of being applied on encrypted data. To this end, NNs need to be able to be trained over encrypted datasets, and process encrypted input data. Training NNs on normal data can be computationally expensive. However training on encrypted data can be even more expensive. To reduce the extra cost of using encrypted data researchers have proposed several efficiency increasing methods.

NNs Trained over Encrypted Datasets

To train NNs on encrypted data, very large and diverse datasets are needed. Instead of coming up with individual training datasets for each new model it is common to create a database of standard training data.

Xu et al. proposed a framework to securely share encrypted data sets from multiple sources, comparing the model training time and accuracy to that of MINST data sets [51]. Xu et al. proposed another different framework which applies functional encryption scheme to cloud AI service architectures where user supplied data is processed by the service provider [52].

Another consideration is the extra complexity of creating a model to understand information that is not meant to be readable as would be the case creating a model which can process encrypted data. In order to train a NN on data that has to be permuted to maintain privacy, Molek and Hurtik proposed using a fully connected auto encoder as a preprocessor for a convolutional NN to make the encoded data more readable [53]. Similarly Nandakumar et al. developed a method of training a NN on fully homomorphicaly encrypted data [54]. By making some optimizations in training a small drop in accuracy is traded to reduce the time needed to train on encrypted data from 6.5 h to 40 min.

NNs Capable of Processing Encrypted Input Data

This section will list many of the ways that NNs are being implemented to use encrypted data. In the following, we study some types of neural computing processes possible to applied on encrypted data.

1.

Classification: Different kinds of input data can be classified in their encrypted form by special types of NNs. Some of these types are discussed below. Similar to identifying encrypted malicious data, NNs have been trained to classify other kinds of encrypted data to protect the confidentiality of the contents while still providing useful classification.

• Encrypted Network Traffic: This type of input data can be classified by NNs for anomaly detection [55] or application identification [56,57] purposes. Some papers focused on improving the classification of TLS/SSL traffic since it is commonly used to protect web traffic. Zhang et al. designed a NN combining, stereo transform, and convolutional NNs to classify TLS/SSL traffic with up to 95% accuracy [58]. Yang et al. proposed using a Bayesian NN system that observes non-encrypted handshake packets, the specific cipher being used and the compression method to classify TLS connections [59]. Zhou and Cui improved upon the Alexnet deep NN by developing multi-scale convolution, a deconvolution operation, and batch standardization in order to reduce training time [60]. Other papers focused their efforts further on classifying encrypted VPN traffic. Song et al. applied a text convolutional NN system to classifying VPN traffic [61]. In order to avoid imbalances in class identification caused during training a loss function and class weighing method were implemented. Zou et al. proposed a novel deep NN that takes series of three packets as input so that features of packets that are continuous between packets can be considered by the system [62]. He and Li method interprets encrypted packets as greyscale images [63]. The images are classifies the by a convolutional NN. In implementation a convolutional NN was trained on VPN traffic and was able to classify similar traffic with 97.3% accuracy. Wang et al. [64] on the other hand evaluated their novel NN’s ability to classify mobile data sourced from 80 different mobile applications. The novel design combines long short-term memory recurrent networks with convolutional NNs, for pattern and signature recognition, respectively. Some researchers designed systems focusing on factors external to the specifics of one encryption method. Wang and Zhu made an early attempt in 2017 at end-to-end encrypted traffic classification using a 1D convolution NN [65]. While many approaches rely on deep learning which require a lot of training Cheng et al. propose using lighter weight system utilizing multi-headed attention and 1D convolutional NN which has to consider far less parameters and halves the training time of comparable systems [66]. In order to perform classification using information provided by observing feature attributes Cui et al. propose an improvement to CapsNet which weighs effective traffic more and now considers the spacing of features [67]. Yang et al. developed a classification method using an auto encoder and convolutional NNs taking packet length and arrival time into account [68]. More recently Wang et al. proposed a similar method using the combination of stacked autoencoder and convolutional NN but raising the number of parameters to 26, including both statistical data and information from the raw packets, to supply high level classification features [69].
• Encrypted Image: In order to efficient perform ITS image evaluation, images need to be secured. This group of researchers proposes using a convolutional NN to classify encrypted images based on partially decrypting them to reveal only nonsensitive information [70].
• Encrypted Speech: In order to retrieve encrypted speech, a deep NN using deep hashing is proposed with two different models: both convolutions and convolutional recurrent. With their high quality deep hashing capabilities, a two stage retrieval strategy is proposed [71].
• Encrypted Application: Uses an end-to-end encryption application for encrypting network traffic based on a 1 dimensional convolutional NN using spatial and temporal features [72].

2.

Other Processes: In addition to classification, researchers have proposed NNs for applying other processes such as compression [73] or visual quality assessment [74] on encrypted input data.

4.1.2. Cryptographic Technology Adapted to Neural Computing

Cryptosystems are being adapted to neural computing. They are trying to encrypt data in a way that the encrypted data can be efficiently processed by NNs. Homomorphic encryption is the most common attempt made towards achieving this goal. While classification can be performed on normal encryption data, researchers have proposed some systems which use specially designed encryption schemes which obfuscate the data in a way that better allows processing. Homomorphic encryption is a form of encryption that allows a user to perform valid computations with the ciphertext. Upon decryption of the ciphertext the plaintext will have the same computations applied.

In order to maintain privacy in the current world filled with powerful data mining techniques and massive networks of data gathering sensor it is becoming increasingly obvious that Fully Homomorphic Encryption (FHE) will be needed for end-to-end encryption [75].

As this field has been growing rapidly we have seen many changes, as recently as 2018 it was shown that current homomorphic systems were too slow for large amounts of data and NNs were not yet secure enough to work with encrypted data [76]. The same year it was thought a possible solution would be to perform the homomorphic encryptions as a part of a cloud environment using multiple parties [60].

However, this too showed to be too slow, but brought to light possibilities of using GPU’s to increase efficiency [60]. The authors of [77] have taken into consideration the space-efficiency problems with FHE functions and have begun work on a method named DOReN. Their method is used to instantly evaluate multiple quantized ReLU-activated neurons in the NN which are processing encrypted data. This considerably cuts down on the space needed to perform these neuron activations.

Other research groups have tackled FHE’s time efficiency problem by transforming all the operations into bit-wise functions and transforming the input encrypted data into binary format; this technique roughly equates to a 6.3 times increase in the speed of CNN’s [78].

More recent work has shown that the security of NNs lead to Fully Homomorphic Encryption over Torus (TFHE), which is a scheme that uses NN’s to effectively evaluate encrypted input data [75]. TFHE along with the protection against backdoor attacks into NNs has provided NNs with security against some forms of malicious attacks [79].

While optimizations of how we use FHE are still in production, some groups have already begun applying these systems to facial recognition and English to Arabic translation [79,80].

Although the translation time for even small words still has a massive run-time to provide adequate accuracy, facial recognition on ciphertext using CNN’s provides real-time usage for high accuracy verification [79,80].

4.2. Alliance

Aside from training NNs on encrypted data, NNs have also shown to be useful to improve the functionality of cryptographic systems and vice versa.

4.2.1. The Role of NNs in Cryptography

NNs have been applied to improving several aspects of cryptographic systems from encrypting several data types to key management and generally securing different aspects of cryptographic systems.

Neural Cryptography:

Neural cryptography refers to the application of mutual learning, self learning, and stochastic behavior of neural networks as well as similar algorithms in the design, implementation, or evaluation of any cryptographic algorithm, device, system, or scenario. Particularly, neural networks have been used as enablers in the design of several cryptographic mechanisms, which are implemented as individual modules in cryptosystems. This should be considered an important aspect of neural cryptography.

• Key Management: Neural Cryptography has been applied to key management in many different ways, some have researched its use in concealing keys in Deep NNs [81], while others have researched the use of NN’s using tree parity machines as a way to distribute keys of a symmetric encryption system [82]. A more novel approach uses Artificial Spiking NNs (ASNNs) to create keys for a symmetric block cipher algorithm with the ability to use any block size [83]. This method provides no need for key exchange [83]. The environment systems itself uses a semblance of public key cryptography where the public key is the seed used to generate the private key on both sides [83]. Additional approaches to neural cryptography symmetric key exchanges involve using a 3D cube algorithm in order to induce secrets on the receiver side or search guided gravitational neural keys [84,85].
• Random Number Generation: Neural cryptography has guided the verification of Pseudo Random Number Generators (PRNGs) by picking up on statistical biases unknown to humans [86]. This is achieved using neural cryptography to detect the difference between actual output and desired ideal random numbers [86].

The use of neural cryptography for encryption [87] and decryption [88] has received a focus from the research community in recent decades [89,90,91]. Different security models have been proposed based on neural cryptography [92] and different kinds of NNs [93] have been used for design and implementation of cryptosystems [94]. This effort has led to the development of different types of neural cryptosystems [94,95]. In the following, we use the research literature to establish an ecosystem for neural cryptography. This ecosystem consists of applications, enablers, and challenges.

• Applications: The applications of neural cryptography can be studied in the following lines.

  -

  Encrypting Different Content Types: Neural cryptography has been successfully tested on different content types, among which one may refer to the following.

  *

  Image: Regular scrambling-diffusion image encryption suffers from many vulnerabilities [96]. Particularly both the scrambling and diffusion are performed independently meaning an attacker can attack each separately [96]. With neural cryptography this vulnerability can be resolved. More specifically using an algorithm that performs the initial scrambling and diffusion in parallel then a second diffusion from a Hopfield chaotic NN trained [96]. This allows not only for the protection from the aforementioned independent cracking of the scrambling and diffusion steps, but also resists chosen plaintext attacks [96]. Other groups have also implemented parallelization in their neural cryptography encryption algorithms, electing instead to perform these operations using cellular NNs and block encryption to create an algorithm based on the feistel framework [97]. Cellular NNs are being used in all kinds of Image Encryption software, including an encryption scheme that uses the hyper chaotic system sequences of a cellular NN to shuffle around the bit of an image before performing a bit-wise XOR [98]. It is important to note that this method uses asymmetric RSA for key exchanges [98]. This can pose an issue since the security of the model relies then on the RSA key and not the Neural Cryptography system [99]. To resolve this issue a NN at the receiver end and a stochastic encryption method at the senders end can be used to eliminate the need for key exchanges all together [99]. Finally, Wavelet Chaotic NNs (WCNN) and chaotic NNs have also been used for secure encryption and decryption of images [100]. However, research has shown that WCNN provides stronger ciphertext [100]. Furthermore, during transmission the only data that would need to be sent is approximation coefficients, reducing the size of the ciphertext drastically [100].

  *

  Video: For video encryption of the MPEG-2 video code, research has shown that using Chaotic NNs to encrypt the bitstream results in high entropy and high key sensitivity, both desirable results for security [101]. This model transmits data via the Orthogonal Frequency Division Multiplexing (OFDM) modulation technique and controls the bit rate and quality of the decrypted video [101]. A hybrid chaos and NN cipher encryption algorithm for compressed video signal transmission over wireless channel.

  *

  Text: While image and video encryption introduces new types of encryption, some approaches taken for text encryption have been to improve upon existing systems to provide new cryptographic schemes. Some groups have used Neural cryptography to encrypt plaintext, generating both a secret key and a hash using the Auto Encoder NNs (AENN) [102]. AENN is a NN meant to provide the least possible distortion to the resulting ciphertext, this allows ciphertext normalization to still appear as ascii [102]. Another improvement of schemes is the use of secret dimensions of a NN model as key instead of relying on asymmetric keys and trapdoor functions [103]. The application of delayed Chaotic NNs to generate binary sequences has also been researched in text encryption [104]. The binary sequence is used to create the key for the first level of encryption [104]. Then used in conjunction with DNA cryptography to create a secure ciphertext [104].

  -

  Applications in Security-Related Scenarios: There are some security-related scenarios, which depend on cryptography. Neural networks have been used by researchers in many of these scenarios. To mention a few, we may refer to the following.

  *

  Privacy: The security of ubiquitous computing has seen great improvement due to Neural cryptography. The idea of neural synchronization to generate shared keys is currently one that provides real-time security for systems already in place [105].

  *

  Authentication: While issues with WiMAX have been thoroughly documented [106]. Neural Cryptography proposes solutions to authentication and authorization by creating neural synchronized key pairs [106]. To achieve this neural synchronization two NNs are created with the same weight changing algorithm and passed the same input [107]. To achieve neural synchronization boundary conditions are set, whenever both weights shift to the same direction and one of the networks touches the boundary, the boundaries close tighter eventually leading to neural synchronization [107]. RFID has seen many problems due to having no international standards and posses security risk, one proposed solution [108]. Involves using a tree parity NN in order to perform key generation [108]. Biometric recognition for authentication has also seen support from deep recurrent NNs in order to increase accuracy and performance of models [109].

  *

  Steganography: Stenography is the study of hiding messages within something that is not a message, in some cases an image. One way to achieve this using neural cryptography is to first perform Discrete Cosine similarity Transform and Elliptic Curve Cryptography to first encrypt the image you would like to hide [110]. Then using a Deep NN this message is embedded into a host image [110]. Other groups have achieved similar results of image using Self-Organizing Map (SMO) NNs with 26 clusters for every letter of the alphabet [111]. Research has also been conducted to hide messages within sound [112]. To accomplish sound stenography, SMO’s are used again with 27 clusters, 1 for every letter in the alphabet and then a cluster for the space between words [112].

  *

  Visual Cryptography: One drawback of visual cryptography is its lack of evaluation criteria [113]. One group proposed a method of evaluating the desirable results of visual cryptography would be encryption-inconsistency and decryption-consistency [113]. Visual cryptography via NNs can be achieved by passing a Q’tron NN a set of greyscale images and the output be a set of binary images [114,115]. Other types of NNs used for visual cryptography includes Pi-Sigma NNs, which is a double layered feed forward network [116]. Allowing for fewer communications between sender and receiver with higher security [116].

  -

  Technological Applications: The recent literature comes with several successful applications of neural cryptography in the technology. Some of these applications are studied below.

  *

  Applications in Industry: A large contribution to NN in technology comes from its applications for secure wireless communication [108,117,118]. After proof of its security was published [119]. Particularly, NN have been used with Fast Handover Protocol in place of MIPv6 to replace its short comings, allow the encryption of large scale satellite images for secure transmission and decryption efficiently and lightweight implementation for key systems in an IoT environment [117,120,121]. Other applications of Neural Cryptography has allowed for homomorphic encrpytion to be applied to cloud services for secure communication and noise compression, as well as intelligent transportation systems to allow confidentiality of personal information [122,123]. Finally, chaotic NNs has seem many applications as well [124,125,126]. To note, hyper chaotic systems and chaotic Feistel transform and time synchronization with multiple dimensions have allowed the resistance of plain text attacks and brute force attacks within the physical layer [125,126].

  *

  Applications in Medical Technologies: Applications of neural cryptography in the field of medicine have come from the requirement of keeping patient images confidential [127]. One approach uses a Hermite Chaotic NN in two rounds, first a chaotic sequence is generated from a logical mapping and used to train the NN, then the image is passed into the network to generate a key for encryption [128]. Other methods of security proposed involve using the Region of Non-Interest in the image in order to watermark the image [127].

• Challenges:

  -

  NN Type Selection: A look at the literature shows that different kinds of NNs are useful for different applications. NN type selection is critical to the ability of neural cryptography to be successful, one group has even used NNs to effectively create new cryptography based off NN training [129]. Investigated the use of Complex-valued tree parity machines in order to perform key synchronizations and how CVTPM’s can be seen as more secured to create key synchronizations than using simple tree parity machines [130]. Achieved postquantum key exchange protocols by using NNs in order to augment Diffie–Hellman key exchange protocols by using multivariate cryptosystems [131]. Explored the relationship between cryptographic functions and the learning abilities of RNN [132,133]. Used Principle Component Analysis NNs to generate random numbers for a chaos encryption system [134]. Other groups have experimented with cellular NNs with iterative interchangeability to produce encryption that allows flat historgrams for randomness and bias [135]. Back-propogating NNs in order to provide strong image compression-encryption using a fractional-order hyperchaotic system [136]. unbounded inertia NNs with input saturation in order to obtain good cryptographic properties [137]. Memristive bidirectional associative memory NNs for colored image encryption [138]. Uses recurrent NNs parallel processing speed to increase the performance of encryption, also proposes a symmetric encryption scheme allowing for variable message and block sizes for data integrity and data encryption [139,140]. A look at the literature shows that different kinds of NNs are useful for different applications.

  -

  Hardware Implementation: A successful implementation of Izhikevich’s neural model has been created using SIMECK block cryptography to allow the spiking NN to perform authentication [141].

  -

  Neural Physically Unclonable Function (PUF): A Physically Unclonable Function is a physical device that when provided with challenges provides a response that acts as a digital finger print. The uniqueness of these fingerprints relies on the physical variations created during manufacturing of the device. Neural PUF are PUFs with NNs embedded into the hardware in an attempt to make them resistant against attacks from NNs learning the outcome of challenge response pairs [142]. It is well known that Strong PUF’s can have their pattern recognized by NNs, thus it is suggested to used a WiSARD NN in order to add machine learning resistance to strong PUF’S [142]. Further ways to disallow NNs to learn from challenge responses of PUF’s is to use analog NNs [143]. Moving on to hardware purposes, researches have created a 1-bit PUF with a 2 neuron CNN with good metrics for robustness [144]. Other uses for NNs in the space of PUF’s involve Error Coding Correction for keys which provides more efficient corrections than standard models [145]. Finally, Tests have been conducted to show there is feasibility in using NN based PUF’s for authentication purposes [146].

  -

  Security Evaluation: Here we discuss attacks on previously mentioned cryptographic systems [147,148]. First we note a majority attack on neural synchronization via NN to provide secret keys, this attack is possible due to many cooperating attackers [147]. Then we view the lack of side-channel resistance in tree parity machine NNs and how you can obtain the secret weight vector via this side channel attack [148]. Finally, we look at a power analysis attack on NNs in order to discover their secret information and then propose resistances against these types of side channel attacks [149]. Although NNs are susceptible to the aforementioned attacks, it also provides resistances to the more commonly known vectors of classical cryptography [150].

  -

  Synchronization: Synchronization of NNs is when a client and network exchange output of NN’s with the goal of having identical weights for synapses. Following with the derivation of a shared key using these keys. Researchers use Period Self-Triggered Impulses to attempt synchronization of NNs and then applied the NN to encrypted images [151]. There has also been study into the generalization of synchronization by using Discrete Time-Array Equations [152]. Other papers investigate the use of lag within the neuron activation functions of a network of NNs in order to provide secure synchronization [153]. Papers have also tested different reaction-diffusion technique of Lyapunov time-dependent impulses within NNs to see its applicability to encrypting images [154]. Synchronization for arrays in a network system can also be achieved by using master-slave synchronization of a delayed NN [155]. The use of memristor-based models and its chaotic properties have also been studied in regards to its image encryption capabilities [156]. Other memristive models using lyapunov functions have also been used for image encryption [157].

  -

  Asynchronous Neural Cryptography: Asynchronours Neural cryptography is neural cryptography where synchronization of the sender and receiver model need not be conducted, in fact they can calculate their weights separately based on information passed to each other via encryption schemes such as one time pad [158]. Produce a chaotic time series using a chaotic NN and use that to encrypt plaintext [158], while the method does have its errors the proposed encryption scheme to use in conjunction, a one-time pad, does alleviate those problems [158].

• Enablers: By “Enablers”, we mean technologies sciences and techniques used to support the design of neural cryptography systems. Some of these enablers are discussed below.

  -

  Chaos Theory: Chaos theory is a branch of mathematics that aims to understand and accurately describe systems which are highly sensitive to their initial conditions [159]. In image encryption, three separate chaotic functions are used for each rgb color in order to allow image encryption via Hopfield NNs [160]. Other uses of Hopfield NNs include using creating asymmetric cryptography by using the semblances of the NN with the human body to do synchronization [161]. Other uses for Hopfield NNs and its human-like similarities is using it in conjunction with DNA cryptography [104]. Signal encryption using the chaotic nature of some NNs have been explored to create digital envelopes [162]. Signal encryption has also been achieved using VLSI architecture for chaotic NNs [163]. Further broad-brand signal encryption utilize Chain Chaotic NNs [164]. Other uses for Chaos NNs is pseudo random number generations using a peice wise linear chaotic map [165,166]. Besides using chaotic maps for pseudo random generation, it is also being used in research in conjunction with S-boxes in order to improve upon public key cryptography [167,168], while encryption via NNs may be possible, a comparison with AES shows it provides better performance at the cost of security for larger files [169].

  -

  Genetic Algorithms: Genetic algorithms are heuristics based of the theory of evolution where the best performing individuals will be used to create the next generation of individuals for further optimization in the hopes to converge to an optimization. A version of genetic synchronization has been used to create a key where the hidden weights of both NNs acts as a key between parties where weights are taken as the distance between chromozones of the NN [170]. Other symmetric key applications use Genetic algorithms with error back propogating NNs to instead create the key to be used for other encryption schemes [171]. Some encryption schemes have also seen improvement, by using AES with a GA algorithm in a NN, the SP-Box portion of AES can be greatly improved [172]. Public Key cryptography enhancements via genetic algorithms have also been of interest to some researchers, in fact it has been used for key generation here as well [173].

  -

  Error Management Codes: Error management codes such as Cyclic Redundancy Check (CRC) have been used by some researchers in the design of neural cryptosystems. CRC is a type of checksum which is primarily used to verify there are no errors in a message. It accomplishes this by executing some polynomial operations on the body of a message, the result of the operations can then be used to verify the messages integrity [174]. One way researchers have augmented the security of neural cryptography is using the DTMP algorithm to produce erroneous bits into the bits transferred between NNs during the synchronization phase [175]. However it was found this two allows potential attacks [176]. The researchers continued by finding three algorithms to build upon the original idea of DTMP [176].

  -

  Frequency-Domain Transforms: Frequency-Domain transforms methematical algorithms which can be used to obtain a description of a function in the frequency domain as opposed to the time domain [177]. Researchers created a data transmission method using classification of speech patterns via NNs to leave no speech signals available on the phone line [178].

  -

  Blockchain: A blockchain is collection of data organized into blocks which are connected to each other by a chain of cryptographic hash digests, commonly implemented in a way such that modifications to the blockchain need to be made through a peer to peer network [179]. To aid in the authentication of users performing key synchronization, researchers have proposed using a second secret value for implicit identity authentication based on block chain technologies [180].

  -

  Combinatorics: Combinatorics is the study of arranging discrete structures, and can expand into other fields including enumeration, graph theory and algorithms [181]. With the power of cellular NNs and its chaotic sequences along with 5 distinct latin squares are used for greyscale image encryption [182].

  -

  Existing Cryptographic Algorithms: In existing cryptography some researchers propose using the synaptic connections of a NN and an input image in order to generate the secret key for an AES encryption [183]. Other researchers propose the use of a variety of AES encryptions for files using the same NN key structure [184].

According to the above discussions, the ecosystem of neural cryptography is illustrated by Figure 2.

NN-Improved Cryptography:

Researchers have shown that NNs can be used in conjunction with colored image encryption schemes to reduce the noise leftover after decryption [185].

Cryptography Using Neural-like Methods:

Neural-like methods are the application of neural functions and methodology without the use of a NN. For example, the neural-like method proposed by the authors of [186] is capable of using a hardware neural-like data decryption/encryption system based on a geometric transformations model.

4.2.2. The Role of Cryptography in Secure Neural Computing

Cryptography has also been used to ensure the confidentiality of NN data. The following subsection will discuss different approaches to encrypting the data that NNs use.

Encrypted NNs

Architectures for encrypted NNs have been proposed in order to preserve the privacy of data being processed through integrating some kind of cipher, usually homomorphic, into the normal operation of the system. This section will discuss some proposed methods to work on data sets that have been processed to protect the privacy of it’s subjects.

Li and Han propose a framework for a federated learning scheme which protects the data sets of clients contributing to a central server [187]. This is performed by encoding the calculated gradient before it is sent by the client. The gradient update is decoded while being combined with the gradients from other clients, effectively preventing interception as well breaches from server.

Other approaches focused more heavily on working on encrypted data.

Choi et al. designed, implemented and tested a convolutional NN ASIC system with integrated AES functionality [188].

Emmanuel et al. propose a system similar to Li and Han’s [187] scheme, which only sends data in the form of encoded gradients. This system however never decodes the data as it is encrypted homomorphicaly [189].

Mefta et al. also designed a hardware solution, but this system differs as it is based on a deep NN that is working on homomorphicaly encrypted data [77].

Crypto-Enabled NNs

Another vector for compromising the confidentiality of NNs is through the data stored in memory [190]. A solution could be encrypting any data needed for operating the NN until it is used. This would protect the data but would introduce a significant loss in performance. There are however a few specialized cryptographic techniques the enable NNs to maintain the confidentiality of their data without much loss of performance [190].

Enabling In-Memory NNs Using Cryptography

Cai et al. proposed an encryption scheme that only encrypts a small number of weights in memory called sparse fast gradient encryption [190]. Encrypting only a few of the weights has far less overhead compared to encrypting all of the weights, and in tests it was still enough to protect the model. The same group of researchers also extended sparse fast gradient encryption to the storage level when non-volatile memory is used [191].

Data Encryption for Securing NNs

Considering the large overhead that comes with using encrypted data, Hu et al. proposed a deep NN system processing homomorphicaly encrypted data with the goal of achieving real-time [192]. The proposed changes are to use the NN blocks whichcan be hardware accelerated.

Cantoro et al. take a different approach, they proposed using encrypted weights not only as a method for protecting the training data set and operation of the NN for highly sensitive systems but also as a fault detection mechanism [193].

Image Encryption for Privacy-Preserving NNs

Dealing with text data that is encrypted come with performance challenges as the patterns in the data are harder to recognize. encrypted image data compounds the issue, as images can be far larger in size.

Sirichotedumrong et al. present pixel-based image encryption, an image encryption scheme that obfuscates image data while maintaining a deep NNs ability to analyze it after a little preprocessing [194]. Sirichotedumrong et al. then propose a cipher-text only attack on images that have been encrypted with pixel-based image encryption when a victim uses the same key [195].

Sirichotedumrong et al. propose encryption scheme that is resistant to the known cipher-text only attack involving using another novel image encryption algorithm and training NN models on encrypted images that use unique keys [196]. Sirichotedumrong et al. published another paper building on the cipher-text only attack resistant scheme. They trained models on encrypted images as well as non encrypted images extending their models ability to classify encrypted images, to also classifying plain images as well [197].

5. Future Roadmap: The Promise of Secure AI

As quantum computing becomes a more mature technology, viable applications will become more clear. A possible field quantum computing could move into is cryptography and neural computing. Thus, we anticipate that the war and peace will be between quantum NN and quantum-inspired NN-assisted cryptography in the future. Our reason for such an anticipation is the existence of the trends discussed in Section 5.1–Section 5.3.

5.1. Quantum NNs in Cryptography

Some researchers are looking at the application of fractional-order quantum cellular NNs (QCNN) in an attempt to solve the nonlinearities that exist in image cryptography [198]. On the other hand, researchers have instead proposed multilayer quantum NNs trained with synchronization to create a new cryptosystem [199]. Continuing the trend of developing quantum cryptosystems with NN, one group proposes a multivariate cryptosystem for a post cryptography world [200].

5.2. NNs in Quantum Cryptography

In Quantum Cryptography, the use of artificial NNs provide good security and efficiency for error correction provided the model meets necessary requirements [201]. Other groups have introduced the idea of noise diffusion using Chaotic Recurrent NNs by using chaotic keys from NNs with quantum noise [202].

5.3. Quantum-Inspired AI

Quantum computing is finding its applications in a variety of technological areas ranging from navigation [203] to channel coding [204] and IoT [205]. Quantum-inspired AI is a recent research trend dealing with the applications of quantum computing in artificial intelligence. In the following, we review some research works focusing on quantum-inspired AI.

Some researchers have proposed a quantum inspired reinforcement learning (QiRL) solution in the application of trajectory planning for unmanned aerial vessels(uavs) in order to create a greedy AI [206]. QiRL has also been studied in the used of Markov chains in order to utilize collapse phenomenon and amplitude amplification in Robotic navigation [207]. In contrast, some groups have developed AI frameworks based on ’re-experiencing’ based events (inputs) using quantum technology in order to further develop sophisticated models [208]. Quantum inspired multi directional associative memory has also been proposed using a model suited for self-convergence and iterative learning [209]. Finally, a model for Fuzzy based NNs in two-class classification has been proposed using using iterative learning by integrating fuzzy clustering techniques with quantum computing [210].

Figure 3 shows a possible future for the neural network–cryptography dichotomy.

6. Concluding Remarks

As the relationship between NNs and cryptography grows more matured, the cross-impact between them is growing more and more complex. Spanning over a decade makes it pertinent to study this dichotomy. This paper has been a look at the many ways NN and cryptographic technologies have come together as well as the future of both given recent computing paradigms and related advancements. This paper can be concluded in the following lines.

6.1. State-of-the-Art: War and Peace

The recent literature suggests the following aspects for the war-and-peace dichotomy of neural networks and cryptography.

• The application of neural networks in the cryptanalysis of cryptographic schemes and attacks against them (War);
• The application of neural networks towards improving the security as well as the efficiency of cryptosystems (Peace);
• The application of cryptography towards the confidentiality of neural networks (Peace).

6.2. At the Horizon: Quantum Advancements

Quantum computing stands in a position to create a similar dichotomy. Once quantum computing finds its way into cryptographic and neural computing schemes, it will inherit the problem of balancing data confidentiality with computational efficiency. It will remain a key concern protecting legitimate confidential data from analysis techniques, while still being able to defend against malicious data being hidden through similar means. Thus, quantum computing will need to bring cryptography and neural computing together in ware and peace again. In the era of quantum computing, the war-and-peace dichotomy of neural networks and cryptography will lead to another dichotomy; quantum neural networks and quantum cryptography.

6.3. Contributions

This paper aimed to highlight areas where current research is going on as well as related topics in need for future research. The authors hope that this survey will provide some insight for those looking to do further work in the field.

6.4. Further Research

In this paper, we have not covered the adversarial role of cryptography against neural networks. The reason is the lack of adequate research in this area. This gap is shown by the red box in Figure 4. To address this gap researchers can continue the work of this paper, when the literature comes with adequate relevant works, via studying the ways cryptanalysis and cryptographic attacks can be conducted against encrypted neural networks. Moreover, we would recommend future research investigating defensive provisions for cryptography schemes against neural attacks.