New Autonomous Intelligent Sensor Design Approach for Multiple Parameter Inference ^†

Abstract

The determination of multiple parameters via luminescence sensing is of great interest for many applications in different fields, like biosensing and biological imaging, medicine, and diagnostics. The typical approach consists in measuring multiple quantities and in applying complex and frequently just approximated mathematical models to characterize the sensor response. The use of machine learning to extract information from measurements in sensors have been tried in several forms before. But one of the problems with the approaches so far, is the difficulty in getting a training dataset that is representative of the measurements done by the sensor. Additionally, extracting multiple parameters from a single measurement has been so far an impossible problem to solve efficiently in luminescence. In this work a new approach is described for building an autonomous intelligent sensor, which is able to produce the training dataset self-sufficiently, use it for training a neural network, and then use the trained model to do inference on measurements done on the same hardware. For the first time the use of machine learning additionally allows to extract two parameters from one single measurement using multitask learning neural network architectures. This is demonstrated here by a dual oxygen concentration and temperature sensor.

1. Introduction

In its most general form, a sensor is some kind of device (analog or digital) that responds to some input from the physical environment. Examples are light [1,2], motion [3,4], pressure [5,6], moisture [7,8] sensors just to cite a few. The output is typically some human-readable information that can be visualised directly on the sensor or can be transmitted over a network for further analysis or processing. To decode the input from the environment in human-readable format, a mathematical model of some form is required. For example, in a mercury-based glass thermometer the height of the mercury column can be translated to temperature thanks to the thermal expansion properties of mercury. The scale printed on the thermometer itself is obtained by using a mathematical formula that link the volume of the mercury with its temperature, often obtained by calibration (meaning by measuring the temperature with a reference sensor, and comparing it with the mercury column height). Today, sensors are of course more complex, and are often at the fore front of technology. Examples incude motion sensors, lidar, radar, chemical sensors and so on. Regardless of their complexity, all of them still rely on some form of mathematical model that needs to be known and tuned during construction and calibration. Often those mathematical models contains artificially introduced parameters that help to take into account variations due to tolerances in the construction, cross-interferences or limitations of the sensor itself. Those parameters are then determined in a calibration phase, but may be (and often are) different for each sensor produced. Look-up tables with values of those parameters in different input ranges are sometimes used to increase the range of applicability of sensors. All the aspects described above make innovation in the sensor field difficult and expensive. In fact, improving the performance of specific devices, often involves materials or construction processes that are more expensive and more difficult to use. As a consequence, new and better sensors tends to be expensive and therefore less accessible by a larger public, including, for example, third-world countries.

With the advancement of artificial intelligence, in the last years, a new research field has appeared that shows the biggest potential for innovation and disruption: smart sensor technology. New sensors that uses machine learning have been discussed in many fields, from agriculture to smartphones [9,10,11]. For example, technological advancements in this field offer impressive innovation in medical devices [12]. These sensors can provide unprecedented medical analysis and diagnosis methods in rural and underdeveloped areas. Demand for devices in health care by governments of several countries is one of the main factor that will increase the demand in the sensor market for smart devices in the next five years [13].

This increased demand in smart sensor technologies poses the question of how the new generations of smart sensors must look like, and what are the main characteristics that they must have to differentiate themselves from classical sensors. It is fundamental to move to a completely new generation of sensing technologies that will be easier, less expensive, more portable and less fragile than what is available today.

Democratising sensors, or in other words make them available to a much wider population, will mainly drive the innovation in smart sensor technology in the next five to ten years, due to an exceptional demand [13]. This can only be achieved by eliminating the need of expensive materials, complicated calibration procedures and large mathematical models with many parameters that needs to be tuned.

Examples of applying artificial intelligence to sensors have appeared in growing numbers in the past few years, but no one, to the best of the author’s knowledge, so far have yet described a new type of sensor that does not rely on a mathematical model, can be built with cheap materials, is portable and can reach the performance of existing commercial sensors.

2. Autonomous Intelligent Sensor

The authors were able for the first time, in the field of oxygen concentration measurement, to build a first prototype of this new generation of sensors [14], that have been called Autonomous Intelligent Sensors (AISs). The built prototype demonstrates how a paradigm shift from the classical description of the response of a sensor through an approximate model, to the use of a machine learning approach, is not only possible but a necessity for future generations of sensors.

A schematic representation of an AIS, as described by the authors, can be seen in Figure 1.

An AIS can learn the complex inter-parameter dependencies and sensor-specific response characteristics from a large amount of data automatically collected. This new method will enable to build sensors even if the response of the system to the physical quantities is too complex to be described by a mathematical model. An AIS will train a machine learning algorithm with a training dataset collected autonomously, and then use the same algorithm for inference on input data measured by the same hardware that generated the training dataset. The mathematical model will be, if the reader accept an intuitive explanation, learned directly from the sensor itself. No need of knowing an a-priori mathematical model will be necessary anymore. Such approach will also allow the use of less expensive construction processes or materials, as the machine learning algorithms will learn to compensate for defects or less precise construction processes.

2.1. AIS Prototype for Oxygen Sensing

The protype built by the authors in [14] is a luminescence based sensor, that was used to measure at the same time oxygen concentration and temperature. The simultaneous determination of multiple physical quantities (multiple sensing) is known to be a hard problem, and is normally solved by complex measuring processes or by the combination of different sensors in one single device [1,15,16,17,18,19,20,21,22,23,24,25,26].

The measuring principle is dynamical phosphorescence quenching. When colliding with molecular oxygen, the energy of the excited luminophore is reduced due to radiationless deactivation [27]. The AIS built by the authors enables accurate dual-sensing by measuring one single quantity using Multi Task Learning (MTL) neural networks [28] that can predict both oxygen concentration and temperature [11,29]. MTL architectures have been used because they can learn correlated tasks [30,31,32,33,34,35]. The authors previously showed with a purely theoretical study that used only synthetic data, that MTL architectures can be flexible enough to address multi-dimensional regressions problems [11] as required by an AIS.

The work to build the first prototype of an AIS, as reported in the two main author’s papers [11,29], demonstrate for the first time that a new generation of sensors (AISs) are possible and can satisfy most of the needs of a future generation of sensors.

3. Future of Smart Sensors

In this section the authors would like to summarize how a new generation of smart sensors should look like, and indicate which research topics are the most important to develop the necessary new technologies for future generations of sensors. The list is not ordered by importance, and future research in this field may well bring up new topics or make some of those listed obsolete. The list is also not intended to be exhaustive, but has the goal of raise awareness in the research community to the most important aspects of this discussion. This new generation of sensors will be indicated here with the abbreviation of AIS.

• An AIS should not need a mathematical model to decode the physical or chemical response into something human-readable. Machine learning algorithms can be used to let the sensor find the necessary mathematical formulas autonomously. Neural networks, as the authors proved in one particular case [11,29], are an extremely efficient class of algorithms in this regard.
• An AIS should not be expensive to build, and should rely on materials and electronics that are commercially easy to find.
• An AIS should not need a complicated calibration processes. The training of the machine learning algorithm must be possible on the field by everyone with minimal technical training. This will make possible the deployment of such technologies in rural areas and underdeveloped countries.
• An AIS should not rely on, unless needed by specific requirements, an active internet connection. Updates to software and models should not be a necessity, and should be avoided if possible.
• When developing an AIS, focus should be given to the software, and in particular to the machine learning algorithms used and not to material or complicated construction. The goal is to build sensors that can be used in the most different conditions and should not require a laboratory environment to be used in.

The authors hope that this list will foster research in these areas for future generations of sensors.

4. Conclusions

In this paper the authors have described a new generation of sensors that does not need any a-priori mathematical model. In [11,29] they also describe an AIS prototype that can extract multiple physical quantities (oxygen concentration and temperatures) at the same time from one single measurement. This prototype can extract two physical quantities without knowing in advance how they are related to each other. This capacity has profound implications for developing new generations of sensors in this particular field, as they will become easier and cheaper to build since no separate temperature measurement or hardware will be necessary. In this paper the reasons why a new generation of sensors is needed is discussed addressing aspects as innovation potential, easy of build and deployment challenges in underdeveloped and rural areas. The authors have also given a list of the most important characteristics that a new generation of sensors should have to enable a democratisation of sensors.