Sentiment-Driven Forecasting LSTM Neural Networks for Stock Prediction-Case of China Bank Sector

.


I. INTRODUCTION
The digital epoch ushered in a plethora of tools and platforms, with search engines emerging as crucial conduits for gauging public sentiment.
Shiller [1] accentuates the importance of grounding research in actual human behavior to comprehend how individuals genuinely think and act.This sentiment concurs with the dynamics of the digital realm, where search query fluctuations concerning financial entities can unveil early signs of shifting public interest or potential concerns.These subtle shifts, characteristic of the digital age, harbor the potential to impact stock market trends [2].
This study endeavors to tap into public sentiment within China's financial market, probing how sentiments harvested from digital platforms can act as a predictive lens for scrutinizing and foreseeing market dynamics.Central to this exploration is the Baidu index, a tool often drawn parallel to China's version of Google, emblematic of the digital transition.Baidu, surpassing its initial function as a search engine, has evolved into a dynamic digital nexus, reflecting the collective sentiment of its extensive user base.This discernment elevates tools like the Baidu index to a significant pedestal in contemporary financial prediction frameworks, particularly against the backdrop of digital transformation sweeping through banks and financial institutions [3].
Banks are pivotal players in China's economic narrative, entrusted with capital provisioning, trade facilitation, and financial stability maintenance [4].The banking sphere, headlined by colossal entities like the Industrial and Commercial Bank of China (ICBC) and the Agricultural Bank of China (ABC), commands substantial influence over China's economic fabric.In the digital era, their roles and impact have evolved, with stock performance mirroring not only the health of individual institutions but also the broader economic vitality [5].Observing the stock trends of these institutions can significantly inform both domestic economic strategies and global financial outlooks.Traditional stock prediction methodologies, shackled by inherent constraints, often grapple with accuracy challenges, paving the path for Neural Networks-specifically, the Long Short-Term Memory (LSTM) networks-renowned for their prowess in time series predictions owing to their capability to process long-term data dependencies.
Nonetheless, the nuanced realm of stock market predictions often finds the exclusive reliance on raw financial metrics inadequate.The digital epoch grants stakeholders a platform to voice their opinions, assumptions, and concerns.Tapping into this vast sentiment reservoir, particularly reflected in search patterns on platforms like Baidu, can unveil precious insights.The fusion of sentiment analysis with LSTM models ushers in a novel frontier for stock prediction, especially within crucial sectors like China's banking domain.
This inquiry embarks on an innovative journey, integrating sentiment indices from the Baidu platform with LSTM neural networks to refine the prediction framework for China's banking stocks.Our scrutiny especially shines a light on stalwarts like Industrial and Commercial Bank of China (ICBC), Agricultural Bank of China (ABC), Bank of Communications (BoCom), CCB (China Construction Bank), www.ijacsa.thesai.orgChina Merchants Bank (CMB), Shanghai Pudong Development Bank (SPDB), and Industrial Bank (IB).Overlaying sentiment metrics with historic stock data, we endeavor to ascertain the predictive potency of sentiments, gauged via the Baidu index, in molding future stock valuations within the banking sector.This endeavor distinctively enriches the literature by elucidating the intertwined roles of banks in financial stability, regulatory landscapes, investor interests, and the digital transformation of the banking sector.Accurate forecasting of bank stock prices enhances our comprehension of impending financial scenarios and vulnerabilities.By leveraging sentiment analytics, we aim to foresee shifts in the continually evolving regulatory labyrinth banks traverse, equipping stakeholders with insights into potential policy alterations.Recognizing the magnetic appeal the banking sector exudes for investors, our model strives to equip them with precise forecasting tools.Additionally, as the banking sector navigates through transformative digital innovations, our sentiment-driven approach seeks to decipher public perceptions, offering insights into how fintech advancements might reshape bank stock trajectories.
Our exposition is organized as follows: Section II delves into pertinent literature, mapping the journey of sentimentimbued stock predictions.Section III unravels our methodology, elucidating the integration of Baidu's sentiment indicators with LSTM networks.In Section IV, we disseminate our empirical findings, accentuating the merits and limitations of our model.Finally, Section V encapsulates a reflective summary and contemplates potential directions for future exploration in this domain.

II. RELATED WORK
Research posits that investor sentiment plays a predictive role in cross-sectional stock returns, suggesting that the collective mood of investors can drive stock price fluctuations and impact their anticipated returns [6,7,8].Moreover, an increasing body of literature is exploring the influence of Internet search activities on financial markets, delving into how online information-seeking behaviors may reflect or shape investor sentiment and, in turn, market dynamics.
Li et al. [9] marked one of the initial strides into understanding how financial news articles sway stock prices.Their contention was that while many had mapped the influence of news in a bag-of-words paradigm, the more profound underlying sentiment-bridging the chasm between lexical patterns and stock fluctuations-remained an undercharted territory.
By implementing sentiment dictionaries, their work enriched the sentiment space, and the results affirmed that sentiment-infused models yield superior accuracy over the conventional bag-of-words counterparts.
Pivoting to the realm of digital social interactions, Guo et al. [10] embarked on an exploration capitalizing on user comments from Xueqiu, a focal social networking site in the Chinese stock market landscape.Their research sought to dynamically unravel the web of interactions between investor sentiment and the stock market.One salient revelation was that sentiment data's efficacy as a predictive tool is tethered to the degree of public attention the stock garners.This insight underscores the dynamism and contingent nature of sentiment's predictive power.
Smales in [11] delved deeper into the mechanics of how investor sentiment could cause asset prices to deviate from their fundamental equilibrium.He asserted that understanding these deviations is paramount, given their potential ramifications on capital allocation and its costs.Through a comprehensive analysis spanning a quarter-century, he demonstrated that sentiment indicators, exemplified by the Volatility Index (VIX), forge a robust bond with stock returns.Further granularity revealed that some stocks, especially those in tech or smaller capitalization brackets, display heightened susceptibility to sentiment fluctuations.Intriguingly, recessional periods amplify this sentiment-driven volatility.
Bringing sophisticated machine learning into the fold, Xing et al. [12] championed the sentiment-aware volatility forecasting (SAVING) model.Their venture sought to elucidate the bidirectional dance between asset prices and market sentiment.This synthesis of deep learning and sentiment analysis manifested in results that eclipsed traditional forecasting tools and rival neural network architectures in terms of precision.
Lastly, the ubiquity of social media platforms in shaping stock predictions has witnessed academic spotlight.Swathi et al. [13] echoed the notion that platforms like Twitter, given their massive user engagement, have metamorphosed into sentiment barometers.By fusing the Teaching and Learning Based Optimization (TLBO) technique with Long Short-Term Memory (LSTM) frameworks, they demonstrated an unprecedented accuracy in forecasting stock price movements based on social media sentiment.

III. METHODOLOGY
This section elucidates the systematic approach employed to forecast the stock prices of significant Chinese banks, leveraging sentiment analysis and LSTM Neural Networks.We initiate with an introduction to LSTM and its aptness for time series forecasting and subsequently incorporate sentiment data derived from the Baidu Index.Acknowledging the latent impact of sentiment on stock market behavior, the Baidu Index is introduced as a lagged variable.Employing grid search, the model's hyperparameters are optimized.Evaluation metrics such as R 2 , RMSE, MAE, and MAPE are then utilized to determine the model's predictive accuracy.

A. Long Short-Term Memory (LSTM) Networks
LSTM (Long Short-Term Memory) networks, proposed by Hochreiter and Schmidhuber [14], are a specialized subset of Recurrent Neural Networks (RNN), meticulously crafted to grasp long-term dependencies within sequential datasets.Their intrinsic capability to retain patterns across extended sequences renders them particularly proficient for tasks such as stock price prediction.Mathematically, the operations of an LSTM unit are encapsulated as:

Refer to
where:


˜ are the forget gate, input gate, cell candidate, cell state, output gate, and hidden state at time t respectively.


and are the weights and biases for the respective gates.


is the sigmoid function.

B. Sentiment Index
In this paper, the Baidu index, a popular sentiment index in China, gauges the public sentiment by analyzing search query frequencies and user behaviors on the Baidu search engine.To harmonize the scale between stock prices and the Baidu Index during pre-processing, Min-Max scaling is applied.Recognizing the potential latency in sentiment influence on stock prices, a time-lagged version of the Baidu Index is integrated.This latency captures the inherent delay in investor reactions, attributable to the time taken to interpret and act upon sentiment-driven information, thereby encapsulating market inertia.
For model input augmentation, at each time step t, our LSTM model receives a concatenated input vector, represented as: Where:  is the stock price at time .
 represents the Baidu Sentiment Index at time .
The LSTM, characterized by its inherent memory, updates its cell state c, factoring in both the stock prices and sentiment index, thereby accounting for sentiment influence across sequences.During the LSTM's forward propagation, this merged input not only impacts immediate outputs but also modifies the hidden states.Consequently, the sentiment information guides predictions and tweaks the LSTM's internal memory.The chosen loss function, Mean Squared Error (MSE), benchmarks model forecasts against real stock prices.During backpropagation, the gradients, inherently influenced by the Baidu Index, fine-tune the model's weightings throughout its training phase.

C. Grid Search Optimization
Grid search is an exhaustive search approach employed for hyperparameter tuning, ensuring optimal model performance.By systematically working through multiple combinations of hyperparameter sets, it evaluates which combination gives the best performance based on a scoring technique [15,16].
In the optimization process of the LSTM model, several pivotal hyperparameters are meticulously evaluated.The batch size, dictating the number of samples processed before a model update, emerges as a key factor under consideration.The efficacy of various optimizers, notably RMSprop and Adam, is also examined [17].These algorithms adjust the model's weights in alignment with the data and its corresponding loss function.To fortify the model against overfitting, dropout layers are integrated within the LSTM's architecture [18].The dropout rate specifies the percentage of neurons randomly nullified during each training phase, enhancing model generalizability.A detailed visualization of the grid search, illustrating the synergy of these hyperparameters and their bearing on model performance, is provided in Fig. 2.

D. Forecasting Evaluation Criterias
To gauge the accuracy and reliability of the model's forecasts, followed by Paudel et al, [19], several statistical metrics are employed: (Coefficient of Determination): RMSE (Root Mean Squared Error): MAE (Mean Absolute Error): where is the actual value at time ˆ is the predicted value at time , and ‾ represents the mean of the actual values.

E. Summary
The proposed technique for predicting stock prices by incorporating the Sentiment Index employs a systematic process, seamlessly integrating the power of Long Short-Term Memory (LSTM) networks and the insights from the Sentiment Index.
Initially, input data comprising of stock prices and a oneday lagged Sentiment Index is collected.This data undergoes normalization to ensure that the range of values is consistent, thereby facilitating effective training.A time-series dataset is then constructed to ensure sequences of data are fed to the LSTM network, aiding in the capture of temporal dependencies.
The dataset gets bifurcated into training and test sets.The LSTM model, designed with two LSTM layers and a dense output layer, is structured to efficiently incorporate and process this information.The first LSTM layer, activated by the tanh function and equipped with dropout, returns sequences which are fed into the second LSTM layer.This layer not only processes the sequences but also integrates the Sentiment Index data.The dense layer at the end aids in producing the final prediction.
Upon constructing this model, it undergoes compilation wherein various aspects like optimizer, loss function, and evaluation metrics are defined.To determine the most optimal parameters for training, a grid search technique is employed.This process considers parameters such as optimizer choice, dropout rate, and batch size, ensuring the model's robustness and accuracy.The model, trained with the best parameters identified by the grid search, is then used to make predictions on the test set.
Once predictions are made, they are de-normalized to revert them to their original scale.Finally, these predictions are evaluated using several metrics to determine the model's efficacy.The metrics include RMSE, MAE, MAPE, and R 2 .
The entire technique, encapsulated in the flowchart (see Fig. 3), presents a comprehensive approach to effectively predict stock prices by harnessing the dual power of LSTM networks and sentiment indices.

A. Data Description
The study employs daily stock prices from August 1, 2022, to August 1, 2023, of seven eminent banks listed on the Shanghai Stock Exchange, including ICBC, ABC, BoCom, CCB, CMB, SPDB, and IB (see Fig. 4).This data, sourced from Yahoo Finance, assures precision.Considering Baidu's significant presence in China, its index is a robust reflection of the public's sentiment towards these banks, sentiment indicators is derived from the Baidu index specific to each bank depicted in Fig. 5.For modeling purposes, the data is split: 80% for training and 20% for testing.Preprocessing steps involve data normalization to ensure consistency.www.ijacsa.thesai.org

B. Hyperparameter Determination
Using grid search, various combinations of batch size, dropout rate, and optimizer type are examined.Each bank's data undergoes this analysis individually, catering to the distinct behavior of each bank's stock prices.The optimal hyperparameters obtained are delineated below in Table I.

C. Forecasting Performance
Table II showcases the forecasting performance of both the LSTM and Sentiment-integrated LSTM (SB-LSTM) models for each bank.For ICBC, the SB-LSTM model exhibits a remarkable increase in R 2 by 6.67% compared to the standard LSTM, and the RMSE also drastically reduces from 0.0510 to 0.0107.In the case of ABC, the R^2 in the SB-LSTM model increases by 2.01% and sees a significant reduction in RMSE from 0.0411 to 0.0185.BoCom's results indicate that the SB-LSTM has an RMSE of 0.0338, an improvement from 0.0553 as observed in the standard LSTM.
Interestingly, for CCB, the standard LSTM outperforms the SB-LSTM in terms of R 2 and RMSE metrics.This suggests that the integration of sentiment does not consistently lead to improved forecasts for every stock.For CMB, the SB-LSTM model dramatically reduces the RMSE from 0.7648 to 0.2316.Similarly, IB's forecasting with the SB-LSTM exhibits a substantial enhancement in R 2 , marking an increase of 7.29% compared to the LSTM.In essence, while the SB-LSTM model generally showcases superior accuracy and precision in forecasting for most stocks, there are notable exceptions, such as CCB.This underscores the idea that sentiment integration typically enhances prediction capabilities, but the individual characteristics of stocks must be taken into account.In summary, the SB-LSTM models consistently demonstrated higher classification accuracy across all stocks when juxtaposed with the LSTM models, emphasizing their superior capability in predicting stock price movements.

V. CONCLUSION
This study ventured into the realm of sentiment-driven stock prediction, specifically within China's banking sector, utilizing a Sentiment-integrated LSTM (SB-LSTM) model.Our findings predominantly showcased the SB-LSTM model's superior accuracy over traditional LSTM in forecasting stock prices, echoing the potential of sentiment analysis in financial forecasting as suggested by previous studies [20,21,22].Particularly, the use of the Baidu Index highlighted how sentiment data can enhance predictive accuracy.The consistent enhancement in classification accuracy across most stocks by the SB-LSTM models further corroborates the promise of integrating sentiment analysis with LSTM networks for more precise stock price forecasting.While the use of the Baidu Index effectively demonstrated how sentiment data can enhance predictive accuracy, a key limitation lies in the reliance on a single sentiment data source, which may not capture the full spectrum of market sentiments.Additionally, the model's efficacy outside the Chinese banking sector remains untested, suggesting the need for further research to evaluate its applicability across different sectors and geographical regions.This endeavor not only enriches the literature but also paves the way for future explorations in leveraging unstructured sentiment data for financial forecasting amidst the digital transformation sweeping across the financial sector.This transformation is increasingly recognized as crucial in the evolving landscape of financial analytics.
Fig. 1 for a visualization of the LSTM model, structured as follows:  1 Input Layer: Accepts the sequence of stock prices and sentiment indices.www.ijacsa.thesai.org 2 Hidden Layers: Composed of LSTM units that can remember patterns in data over long periods. 1 Output Layer: Produces the predicted stock price.

Fig. 6
delineates the confusion matrices for the LSTM and SB-LSTM models across six prominent banks: ICBC, BoCom, ABC, CCB, CMB, and IB.For the ICBC stock, the LSTM model yielded 138 true negatives and 84 true positives but was marred by 131 false positives and 132 false negatives.www.ijacsa.thesai.orgConversely, the SB-LSTM showed a marked improvement, boasting 247 true negatives, 212 true positives, and significantly fewer false positives and negatives.A similar trend was observed for BoCom, where the SB-LSTM displayed enhanced accuracy with 232 true negatives and 227 true positives, overshadowing the LSTM's 134 true negatives and 102 true positives.ABC's LSTM model presented 166 true negatives and 58 true positives, but the SB-LSTM again outperformed by producing 242 true negatives and 189 true positives.This pattern of SB-LSTM superiority persisted with CCB, where the LSTM's 144 true negatives and 86 true positives were eclipsed by the SB-LSTM's 251 true negatives and 210 true positives.For CMB, while the LSTM model recorded 144 true negatives and 111 true positives, the SB-LSTM surged ahead with 257 true negatives and 220 true positives.Lastly, IB's results from the LSTM, comprising 132 true negatives and 96 true positives, were also surpassed by the SB-LSTM's impressive 257 true negatives and 225 true positives.

TABLE I .
OPTIMAL HYPERPARAMETERS FOR EACH BANK