Study on the Implementation of Multimodal Continuous Authentication in Smartphones: A Systematic Review

—Profound societal shifts result from the inception of the 4.0 age of the Industrial Revolution and rapid technological advancements. The widespread adoption of e-services has resulted in substantial reliance on smartphones to access diverse offerings. Even so, account breaches and data leaks are risks that users take when they rely so heavily on their smartphones. Authentication is an essential method of safeguarding personal information. The purpose of this study is to undertake a thorough review of the literature on the deployment and trends of multimodal biometric authentication on smartphones. The studies will look at several biometric modalities, such as behavioral and physiological characteristics, and the algorithms for pattern recognition used in continuous authentication systems. The results show various biometric authenticators and emphasize the importance of behavioral features in smartphone authentication. In addition, the research underlines the significance of machine learning algorithms in pattern identification for rapid and accurate analysis. This study helps to understand the present authentication technique landscape and gives ideas for future advances in safe and user-friendly smartphone authentication systems.


INTRODUCTION
The Fourth Industrial Revolution gave birth to significant social disruption, characterized by rapid and highly advanced technological advancements with a particular emphasis on artificial intelligence, big data, and integration systems [1].This trend encourages every element of society to use electronic service systems in every activity, ranging from the world of business, banking [2], transportation [3], and social organization [4].
In addition, many people use smartphones to access various electronic services via the internet, with 6.4 billion users or 79 percent of the world's total users.The utilization rate of intelligent mobile devices is significantly higher than other devices, such as portable computers and tablets [5].
Internet connection on intelligent mobile devices carries the risk of security vulnerabilities such as account theft and data leakage of its users [6].Authentication is a meaningful way to keep personal information, such as personal data and more, from falling into the wrong hands [7].Android devices have used authentication schemes, including pin codes or passwords, patterns, fingerprints, and biometrics [8], where patterns, pins, and alphanumeric passwords are still the preferred way to log into Android devices [9], computers, and web applications such as email, cloud storage, and online shopping services [10], [11].
A knowledge-based authentication, physiological biometrics authentication, behavioral biometrics authentication, and multifactor authentication are the essential components that comprise the taxonomy of user authentication systems on mobile devices [12].Regarding security, knowledge-based verification uses information that only people and systems know.The secret can be text, like PINs, codes of letters and numbers, or a picture, like a pattern [12].
Traditional or knowledge-based single-factor authentication has become a significant concern for security practitioners and researchers.While still a viable option due to its simplicity, using PINs, passwords, and patterns as authentication inputs comes with several vulnerabilities, such as surfing and smudge attacks and susceptibility to intercept [13].Also, password vulnerability causes most users to use passwords that are easy to guess and do not change regularly [14].The Cybersecurity and Infrastructure Security Agency (CISA) has also added singlefactor authentication, such as password matching, to gain access to a system as a bad practice [15].
Other approaches initially anticipated that multi-factor authentication would enhance security [16] and ease ongoing protection for computing devices [17] and other critical services [18] from unauthorized access by using more than two types of credentials [19], such as biometrics and secret knowledge [12].However, the enhanced security force is still limited.This limitation is supported by other studies that have understood the failure of multi-factor authentication on mobile devices [20].One flaw in this scheme is that an attacker attempts to intervene in a communication between two interacting parties and modify the message or information transmitted so that the attacker can gain access to confidential data or perform unlawful acts.Furthermore, synchronization issues, hardware alterations, or faults in the implementation of authentication protocols might be used by attackers to gain sensitive information or carry out illicit acts.
Continuous user authentication approaches on smartphones through sensors, multimodal behavioral biometrics, and machine learning models have been introduced in recent research [21], [22], [23] to resolve the previously mentioned issue, improve accuracy, and reduce interference in authentication mechanisms.This method gives a higher level of www.ijacsa.thesai.orgprotection while accessing electronic services via mobile devices, such as smartphones.Previous research in [24] has also conducted systematic reviews of continuous multimodal biometrics but has yet to focus on smartphone implementation.Therefore, this study investigates the possibility of implementing continuous multimodal authentication in mobile devices such as smartphones.
The structure of this paper is as follows.Section II presents the related works.Section III clarifies the concept of multimodal biometrics.Section IV describes the methods used in the study.Section V provide the study's findings and discuss its implications.In Section VI, we present the conclusion and outline future work.

II. RELATED WORK
We discovered limited papers focusing on multimodal continuous authentication in smartphones.Researchers have examined authentication in various ways.For instance, [12] conducted surveys of existing authentication methods on mobile devices, while [13] suggested a behavioral biometric authentication scheme as secure and convenient.Moreover, [17] concluded that biometric authentication alone was insufficient and proposed multi-factor authentication mechanisms for more robust security.The study in [20] explored security vulnerabilities in multi-factor authentication schemes on mobile devices, while [21] identified continuous authentication with behavioral biometrics in smartphones as insightful and challenging for adoption.Furthermore, [23] found that continuous multimodal biometric authentication offers high accuracy and improved security, and [24] suggested implementing and evaluating such systems to demonstrate their feasibility.Based on these studies, we aim to investigate the implementation of multimodal continuous authentication in mobile devices, such as smartphones.

III. MULTIMODAL BIOMETRIC
Biometric refers to recognizing patterns that establish a person's identity by comparing biological or behavioral features of biometric attributes.Biometric traits are a highly convenient means of verifying an individual's identity, as they offer high security (difficult to replicate) and cannot be stolen, forgotten, or misplaced [25].
Fingerprints, palm prints, hand geometry, faces, eyes, ears, electrocardiograms, and electroencephalograms are all physiological biometrics used in modern smartphones.Tapping behavior, hand motions, noises, gait, and daily activities are all examples of biometric behavior [12].
A biometric system is, in essence, a pattern recognition system that collects biometric data from a person, extracts a set of features from that data, and then compares the extracted features to a background of templates saved in a database.This process is known as "biometric matching."To put it another way, a pattern recognition system is what a biometric system is.Its performance is determined by the context in which it is used; for example, depending on the context, it may function in either a verification or identification mode [25].
Unimodal and multimodal represent two distinct categories within biometric systems, with their primary distinction lying in the number of modalities employed for authentication purposes.Unimodal biometric systems, exemplified by fingerprint and facial recognition technologies, are easier to create since they only require one identity.However, they are vulnerable to problems like spoofing and poor identification.On the other hand, multimodal systems, such as those combining facial and voice recognition or iris and fingerprint recognition, provide enhanced protection, durability, and adaptability to external influences by employing multiple characteristics.Their intricacy, however, resides in figuring out what, when, and how to combine data for authentication across several modalities [24].

IV. METHODOLOGY
The research strategy used for this project was called a Systematic Literature Review (SLR).The SLR technique is a tried-and-true research approach when gathering and analyzing data on a specific issue.We have used the PRISMA guidelines provided by Matthew J. [26].This SLR consists of four main steps: primary study planning and search, study collection, data extraction, and data synthesis.Section IV(A) identifies research objectives and questions as the first step.In Section IV(B) and IV(C), search strategy steps involve study selection criteria, study selection procedures, keyword formulation for research, and search queries.In Section IV(D), the final step requires quality assessment.

A. Research Question and Objectives
The primary purpose of this SLR is to explore the implementation of continuous multimodal authentication in Smartphones.We create research questions to focus on the objectives of this research.

RQ1: What biometrics is used for authentication on smartphones?
RQ2: What technique is used for pattern recognition in continuous authentication on smartphones?
Based on this research question, the focus of this research objective is to review trends that have occurred in recent years, particularly the use of continuous multimodal authentication on smartphones, and explore biometric combinations used for authentication on smartphones.

B. Search Strategy
In this study, we searched using the electronic databases IEEE Xplore and Scopus.We prepared several lists of keywords to search for relevant literature on multimodal biometric authentication in smartphones from selected electronic databases.The search query utilized was: "continuous" AND (*biometric* OR *multi*) AND ("authentication" OR "verification" OR "validation") AND "smartphone" OR "mobile phone." Queries applied to article titles, abstracts, and keywords to get relevant articles from electronic databases.

C. Selection Criteria
We analyze the query results that have been obtained by removing duplicate articles.Filtering is also done based on the article's title, abstract, and keywords.In addition, we also use www.ijacsa.thesai.orginclusion and exclusion criteria.Fig. 1 shows the PRISMA diagram for this meta-analysis.We use the following inclusion criteria.The paper or article should talk about authentication in smartphones.In addition, papers or articles must be in English and published from 2018 -2022 in journals or conference proceedings.
We used several article selection exclusion criteria in this study.Papers or articles discuss the continuous multimodal biometric authentication but not in smartphones.In addition, papers or articles in the title, abstract, or keyword section do not mention authentication; articles with survey methods or systematic reviews are not attached to the results of this study.

D. Quality Assessment
Quality assessment is used to assess the quality of the selected article.The quality assessment also evaluates whether the selected article is fully accessible and can answer our review.To determine consistency, we formulated some quality assessment questions.QA1: Does the article mention the use of continuous multimodal biometric authentication, and is it clearly stated?QA2: Does the article provide an answer to the formulated RQ? QA3: Are the aims of the research clearly stated without ambiguity in the paper?Yes or no can answer each question with weights of 1 and 0, respectively.The results are evaluated after an assessment of the quality of the entire article has been carried out.The quality assessment process is intended for all research articles according to quality assessment questions.Therefore, this review includes all 31 selected articles.

V. RESULT AND ANALYSIS
The study's conclusion will involve presenting research papers that investigate continuous authentication using multimodal methods.Additionally, the second step consists of conducting a mapping exercise to explore the application of biometrics based on specific biometric properties.

A. Significant Journal
In this literature review, 17 journal articles and 14 conference proceedings discuss continuous multimodal authentication.Here is a brief overview of the distribution of publications journals over the past five years.The result is shown in Table I.Intriguingly, the scholarly landscape unveils the distinguished International Journal of Advanced Computer Science and Applications as the sole journal within the www.ijacsa.thesai.orgintellectually captivating Q3 quartile, signifying its profound impact with a singular publication.

B. Biometric Authenticators
A biometric is a pattern recognition system that establishes a person's identification by comparing biological or behavioral traits of biometric characteristics [26].
Table II presents a captivating exploration of the landscape surrounding biometric authentication within continuous multimodal biometric authentication systems on smartphones, offering a comprehensive overview of the diverse range of biometric authentication methods utilized in intelligent mobile authentication systems that seamlessly integrate multiple biometric modalities sustainably.From this table, we can identify and analyze various biometric authenticators employed in these systems, unveiling a rich tapestry of authentication techniques.
In addition, Fig. 2 illustrates the prevalence of different biometric modalities in authentication systems, revealing that 84% of behavioral characteristics are widely used for smartphone authentication, 13% choose to use physiological factors, and 3% combine behavioral and physiological characteristics.Notably, human gait, routine activities, and touch/swipe functions are emerging as highly preferred options for enhancing the security of continuous authentication processes.An innovative smartphone unlock scheme to improve user authentication through swipe behavior becomes a unique approach where the user selects a background image and performs a swipe action at a specified location on the smartphone screen.This combination ensures secure and reliable authentication, providing an additional layer of protection for smartphone users [27].Other combinations, such as swipe and tap, result in increased security through continuous user authentication by paying attention to factors such as vibrations from walking, the effects of different positions, and trembling hands in cold temperatures [31] and the number of tap gestures implemented without any combination [49], [50], [57].
The continuous implementation of authentication and identification security is also demonstrated using behavioral characteristics in everyday life.Activities of daily living, such as walking, typing, and clapping, can be used for authentication and biometric identification.This demonstrates the feasibility of using natural activities for continuous biometrics with the help of smartphone motion sensors and inertial measurement datasets [53], [55].Another approach through an innovative scheme also offers a dynamic and personalized user validation process by analyzing six everyday activities: walking, running, standing, sitting, walking up, and walking down.The variety of positions for smartphone placement on the user's body affects the user's recognition of each specific activity.This can optimize sensor placement and improve the overall performance of recognition systems [40].
The development of human gait recognition for smartphone access shows the advantages of biometric authentication methods to enhance security and prevent illegal user access [28].The system's hidden nature, which does not require user interaction, further confirms its advantages as a convenient and user-friendly layer of security [30].By utilizing smartphones' built-in inertial sensors, data collection can be done smoothly without burdening the user, making it a highly efficient, scalable, and robust modality for smartphone user authentication [33], [52].
Another approach in the realm of physiological characteristics, using the front-facing camera on devices, enables capturing the user's facial features and facial attributes (e.g., eyes) [39], [41].Current face authentication approaches train the system using facial data from a single context or several contexts with no separation.However, camera exposure recognition is essential to improve facial recognition performance in different circumstances.The contingency for elevated accuracy thresholds arises when illumination conditions play a pivotal role in facial image recognition, owing to their substantial impact [39].
MetaEar is a cutting-edge method of modeling and authenticating Ear-Related Transfer Function (ERTF) biometrics from the human ear using Frequency-Modulated Continuous Wave (FMCW) ultrasonics.The system uses FMCW ultrasonic waves and twin microphones to record and analyze the feedback sound wave for extracting ERTF characteristics [48].

C. Pattern Recognition Techniques
Pattern recognition techniques play an essential role in smartphone continuous biometric authentication systems, enabling the automatic identification and classification of patterns in data for efficient and accurate analysis.Recent advances in machine learning algorithms have revolutionized pattern recognition, offering powerful tools for extracting meaningful information from complex data sets.Fig. 3 depicts utilization patterns of classification techniques in continuous multimodal biometric authentication systems based on an SLR.The study conducted by Benegui used four different Convolutional Neural Network (CNN) architectures with varying depths as embedding extractors [30].These networks have a similar structure, using SoftMax activation at the classification layer and Rectified Linear Units (ReLU) at 84% 13% 3% The distribution of various biometric modalities within the authentication system.In addition, CNN shows superior performance compared to traditional machine learning classifiers such as Support Vector Machine (SVM), k-nearest Neighbors, Random Forest, and Linear Discriminant Analysis (LDA) when applied to behavioral characteristics based on routine activities [33].The inherent time-invariant nature of CNN makes it particularly suitable for processing time series data in behavioral biometrics, such as the analysis of hand movements [42].

Behavioral
The Long Short-Term Memory (LSTM) model undergoes an optimization process to determine the optimal set of parameters for a particular task.Compared to CNN and ConvLSTM architectures, six-layer CNN outperforms ConvLSTM in terms of accuracy and generalization [30].An LSTM-based architecture is used in the authentication model to capture user behavior patterns while holding their smartphone, regardless of activity.As shown through keystroke dynamics analysis, LSTM classifiers show promising potential in predicting user behavior, even with limited data availability [54].
Unsupervised user verification demonstrates remarkable performance with minimal training time.The authentication system effectively handles various activities and routines using a dedicated single-class SVM model, resulting in exceptional accuracy [37].Moreover, SVM classifiers are trained on facial characteristics using advanced methods like face warping and textual feature extraction, ensuring precise face identification [41].The authentication process further incorporates the analysis of touchscreen interactions and subtle micro-gestures, enhancing the overall accuracy and reliability of the system [31].
In other techniques, Random Forest (RF) is often used for prediction and classification because the computational complexity is relatively low, and the training is faster [50].RF technique is also utilized to identify motion status, touch gesture characteristics, keyboard patterns, and short-term activities.[29], [45], [50].The pattern recognition field also extensively uses another statistical method called logistic regression.On the other hand, the author favors this option due to its simplicity and dependability and the fact that it is included in the Weka library [29], [39].

D. Discussion
We discovered 13 biometric authenticators that use behavioral and physiological aspects for authentication.Behavioral factors are used most frequently for smartphone authentication, followed by physiological and mixed techniques.In addition, we identified 16 pattern recognition approaches critical for constructing a continuous multimodal biometric authentication system.Fig. 3 depicts the ranked pattern recognition techniques that were used.
Based on these findings, it can be concluded that behavioral traits have been widely implemented in smartphone authentication.The combination of biometric authentication ensures secure and reliable authentication, providing an additional layer of protection for smartphone users.Furthermore, selecting pattern recognition techniques is crucial in continuous biometric authentication systems on smartphones.This ensures the automatic identification and classification of data patterns for efficient and precise analysis.

A. Conclusion
The SLR method was employed in this study to investigate the implementation of continuous multimodal authentication in Smartphones.The study's findings revealed a diverse collection of journals across different quartiles, with notable publications discussing continuous authentication in Q1 and Q2 journals.The examination of biometric authenticators revealed a diverse set of methodologies, with behavioral traits being the most often utilized for smartphone authentication.Behavioral variables, such as human movement and customary activities, are used for smartphone authentication, followed by physiological characteristics and a mix of both.
Additionally, the research emphasized the popularity of pattern recognition algorithms, including neural networks based on convolution and long-term and short-term memory models.These showed more extraordinary performance when evaluating behavioral biometrics.Support Vector Machine, Random Forest, and Logistic Regression were also used for classification and prediction.Overall, the study provided insights into the landscape of biometric authentication and pattern recognition techniques in continuous multimodal biometric authentication systems on smartphones.This research contributes to understanding the implementation of continuous multimodal authentication on smartphones.The findings highlight the importance of behavioral characteristics and the effectiveness of pattern recognition techniques in enhancing security and user authentication.The diverse range of journals and the prevalence of advanced machine-learning algorithms in the field demonstrate significant advancements in this area of research.Future studies can further explore the performance and usability of these authentication methods and investigate new approaches to enhance continuous authentication on smartphones.

B. Limitations and Future Works
The research had certain limitations.Initially, it concentrated primarily on multimodal continuous authentication in smartphones.Second, it only looked at multimodal biometrics, not unimodal biometrics.Future studies could investigate a broader range of devices that require authentication techniques.Furthermore, additional research might focus on unimodal biometrics or compare unimodal and multimodal biometrics across different devices.
-depth analysis of the provided data reveals a diverse collection of journals across different quartiles.Within the esteemed Q1 quartile, we find a constellation of scholarly publications, including Computers & Security, Journal of Network and Computer Applications, IEEE Transactions on Industrial Informatics, IEEE Access, IEEE Internet of Things Journal, Human-centric Computing and Information Sciences, and IEEE Signal Processing Letters.Remarkably, these journals exhibit varying publication frequencies, ranging from 1 to 2, which discuss continuous authentication.Transitioning to the intellectually stimulating Q2 quartile, we encounter an array of influential journals that contribute significantly to their respective fields.Noteworthy publications such as Electronics Microprocessors and Microsystems, IEEE Transactions on Information Forensics and Security, International Journal of Distributed Sensor Networks, and Sensors grace this quartile, each showcasing their research prowess with a single publication.

Fig. 2 .
Fig. 2. The proportion of diverse biometric modalities utilized in the authentication system.

Fig. 3 .
Fig. 3.The utilization patterns of classification techniques in continuous multimodal biometric authentication systems.

TABLE I .
DISTRIBUTION OF PUBLICATIONS

TABLE II .
A COMPILATION OF RESEARCH AND BIOMETRIC AUTHENTICATORS USED FOR CONTINUOUS MULTIMODAL AUTHENTICATION