A Multi-dimensional Credibility Assessment for Arabic News Sources

Due to the advances in social media, it has become the most popular means of the propagation of news. Many news items are published on social media like Facebook, Twitter, Instagram, etc. Facebook is a huge source for spreading and consuming daily news, but it is an unstructured way of producing news about domains (Art, Health, Education, Sport, Politics, etc.). Thus, this paper will present a model to assess the credibility of news sources over the social context in a particular domain through a particular period of time from a multidimensional perspective. Based on these dimensions of credibility, this model will be designed, evaluated, and implemented by using machine learning algorithms and Arabic NLP approaches to assess the credibility score for Arabic news sources on Facebook. In addition, the study will visualize their scores at different data analysis levels to make the assessment more precise and trustworthy. The proposed model has been implemented and tested over some real Arabic news sources for specific domains and over a period of time to produce a credibility score for each one, whereas the user can display these scores and choose the most credible news sources. The credibility assessment model will be more specific and accurate for a specific domain and time with an accuracy of 98%. Keywords—Information credibility; social media; machine learning; Arabic Natural Language Processing (ANLP)


I. INTRODUCTION
Recently, online social media has become a great way to connect people with others. Users of social media share news, communicate with other people, and create more posts and tweets related to the news than they consume. Consequently, a huge amount of incredible news is created and propagated through social media, which has a serious impact on society and individuals. Though this impact can make political gains, increase advertising revenue, and attract attention, it can harm the reputation of businesses. The news spreads rapidly on social networks and may lead to many problems. Various domains on social media such as politics, education, finance, art, sports, and health are not categorized for the news. The objective of the research community in social media now is to reach out the authenticity of the information from the trusted sources to build trusted news displayed on the internet in different domains. In this regard, information credibility has become an important indicator for citing particular news sources over others in different domains at a particular time.
Here, it is worth mentioning that there are many problems on setting and defining credibility. Firstly, the lack of assessing credibility in multiple domains at the same time, such as arts, sports, and education, etc. Secondly, there are a few studies work on the Arabic language for the purpose of assessing credibility because its structural essentials are complex. Finally, there is a strong need to assess the credibility of the news sources automatically on Facebook social media which has become the most popular mean to spread the news rabidly.
Information credibility is a topic that has become very important for assessing credibility of information to avoid the conflict from huge number of information due to the growth of technology and internet. In this section will present the main types of information credibility, Level of assessment and approaches used in the assessment process.
There are two main types of information credibility, offline and online. Offline information credibility the news spreads through means of 'traditional media that do not require the use of digital technologies like magazines, journals, etc., while online information credibility concerns the news will spread through digital technologies such as social media or the web. Online information credibility has two types of information credibility are content or social information credibility. The content-based information credibility involves the news collected from various websites, web blogs through a search engine, or fact-checking websites while the socialbased information credibility the news published over social media like Facebook, Twitter, Instagram, etc. [1].
Moreover, there are several levels of credibility that can be used to assess the new source these levels are, Post-Level which the assessment of credibility occurs over one post or tweet, and the assessment includes accurate information about a certain topic. It's semantic and characteristic of the message either image, videos, or audio may be considered and sentiment features will be considered also, while topic -Level which the assessment of the credibility can be calculated for a certain topic hashtag or trending topics that attract any users and the users start to comment or retweet about this topic, and member-level applies the assessment on each user account over the social media and measures the reliability of a user by the following features: age group, sex, school degrees, the number of followers, friends, tweets, and retweets, finally hybrid-level (post -topic -member) the credibility assessment contains (topic, post, and user) assessment [2].
There are approaches for assessing the information credibility, automated-based approaches, humanbased approaches, or hybrid approaches. The automated approaches consist of several algorithms that can be followed, these algorithms are weighted and information retrieval algorithms (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 12, No. 9, 2021 317 | P a g e www.ijacsa.thesai.org which the algorithms for measuring credibility for a certain claim collected from different credible independent sources and provides evidence for the similarity about the claim from different sources or knowledge graph algorithms: which used to build an information graph of facts. The graph has made up of relations to all derived information and their origin, and finally, machine learning algorithms which can be supervised techniques, unsupervised techniques, semi-supervised techniques, or reinforcement techniques while Human-Based approaches can be cognitive and perception or crowdsourcing: or voting approaches, and hybrid approaches can be automated or human using experts who are specialized in a specific domain [2]. Motivated by the above, there are many contributions of the research are:  Build a model based on social-context credibility assessment using a hybrid level (content and member) assessment executing over three dimensions: calculating a credibility score of the news source in a specific domain at a specific time to enhance the credibility assessment and give more precise and accurate results.
 Visualize the results over different data analysis.
This paper is organized as follows. Section 2 introduces the related work for the existing studies accessing credibility. Section 3 presents the text categorization methodology. Section 4. The details of suggested model, Section 5 displays the experiment results, and finally, Section 6 concludes this works.

II. RELATED WORK
Some studies have developed approaches attempt to assess the credibility in Social Media. Popat, Kashyap, et al. [3], proposed approaches to leverage the stance, reliability and trend of sources of evidence and counter evidence for credibility assessment of textual claims. And provide explanations for the credibility verdict in the form of informative snippets from articles published by reliable sources that can be easily interpreted by the users. Another one, Ahmad, Faraz, and Syed Afzal Murtaza Rizvi [4], presented a survey approaches for detection of rumors on social networks. They present the data collection methods, extract the features which responsible for finding and estimating credibility using different machine learning techniques. And, Pasi, Gabriella, and Marco Viviani [5], Present Three important issues in the assessment information credibility: in the review sites, it detects the opinion spam, detect the fake news in microblogging, and the credibility assessment of online health-related information. They present a concise survey of the approaches and methodologies.
There are some frameworks assess the credibility, Jaho, Eva, et al. [6], proposed an Alethiometer framework to improve trustworthiness and the validity of contents in the midst of overloads of information. This paradigm was utilized for evaluating the veracity of news consumed on social media. It measures the content by deriving a single metric that takes into account the quality of the contributor and the content, in a unified manner, providing a few preliminary statistical data from the examination of 10 million Twitter users, which offers useful insights into social media data characteristics. Maps of features value on a qualitative scale manner.
Moturu, Sai T., and Huan Liu [7], proposed also, a twostep unsupervised, feature-driven framework for health content, and proposed a various unsupervised scoring models for the user content based.
Mahmood, Saba, et al. [8] present a framework which predict the credibility of web information for a typical user in order to improve decision-making. This framework decides the content credible if based on expert" reviewer. It uses the past behavior of the entities and opinion of others as reputation for expert ranking. Reputation based credibility assessment computing the credibility of the content by evaluating the reputation of the expert" reviewer.
Some researches presented algorithms which assess the credibility, Gupta, M., et al. [9], enhanced the Basic Credibility Analyzer (BasicCA) for Twitter algorithm, which automatically assesses the credibility of Twitter events by performing event graph optimization (EventOptCA). The EventOptCA better than the Basic CA because it depends on stronger event associations inferred from shared unigrams (event similarity) which gives more accurate performance for credibility scores. Abbasi, Mohammad-Ali, and Huan Liu [10], propose a method to measure user credibility in social media. They assess the credibility of the content and user (source of the information) based on the user"s profile by using the CredRank algorithm, this algorithm measures the user credibility in social media and analyzes social media users" online behavior to measure their credibility. Gupta, Aditi, et al. [11], present a semi-supervised ranking model for scoring tweets according to their credibility" TweetCred" it is available as a browser plug-in and score credibility on tweets user"s timeline. Mitra, Tanushree, and Eric Gilbert [12], present CREDBANK algorithm combining machine and human computation on tweets, topics, events and associated human credibility judgements.
There are tools assessing the credibility, Saikaew, Kanda Runapongsa, and Chaluemwut Noyunsan [13], developed a tool which is a chrome extension of fb credibility for Facebook users to evaluate the credibility of by manual human"s labelling by using Support Vector Machine (SVM). Horne, Benjamin D. et al. [14], introduced a tool called News Landscape (NELA)Toolkit is an open source toolkit for the systematic exploration of the news landscape. NELA allows users to assess the credibility of news articles using the content based features, filter and sort article predictions based on the user needs. The NELA allows users to visualize the media landscape at different time slices using a variety of features computed at the source level. Also, Saez-Trumper and Diego [15], presented a web application called "the Fake Tweet Buster (FTB)", which identifies fake tweets" images and users that are consistently uploading and/or promoting fake information on Twitter.by reverse image searching, user analysis and a crowd sourcing approach to detect that kind of malicious users on Twitter. Finally, Lorek, Krzysztof, et al. [16], presented a tool called "TwitterBOT" which able to www.ijacsa.thesai.org score submitted tweets by using an automated credibility assessment on Twitter. Using random forest algorithm as an automatic classifier.
Today, several studies have developed models to assess the credibility in Social Media. Podobnik, Vedran, et al. [17], proposed a model on a Facebook application named "Closest Friends", which calculates the user"s closest friends on the Facebook social network to calculate who-trusts-whom and implement that knowledge in the social recommender, it record the detail of all possible relations between users in the social network and evaluate those relations properly.
Bauskar, Shubham, et al. [18], present a novel machine learning model based on Natural Language Processing (NLP) techniques for the detection of "fake News" by using both content-based features and social features of news.
Some researches assess credibility for Arabic news it is difficult and have a few papers talk about it, Jardaneh, Ghaith, et al. [19], utilized machine learning algorithms to identify fake news Arabic tweets based by content and user features supervised classification model.by extract important features for classification purposes and employ sentiment analysis to generate new features for the detection of fake Arabic news. Sentiment analysis led to improving the accuracy of the prediction process.
Chadi Helwe et al. [20], developed methods for assessing the authenticity of Arabic blogs using deep co-learning, a semi-supervised end-to-end approach, and deep learning. This deep co-learning technique is based on co-training, which employs several classifiers that learn from one another using distinct data characteristics. The classifiers are trained using a small train set in a totally supervised way initially. The small, fully-labeled dataset trains two deep learning models for assessing the credibility of Arabic blog posts. The two classifiers are based on a convolutional neural network (CNN) architecture. The first model uses continuous bag of words (CBOW) word embeddings as features, while the second uses character level embeddings.
Al-Eidan, Rasha M. BinSultan et al. [21], proposed a system to measure information credibility of Arabic web content automatically focused on weblogs and need to enhance the selected credibility features by using SVM machine learning technique to be suitable for the Arabic language.
Floos, Ahmad Yahya M. [22], the author illustrates the difficulty of Arabic rumor identification in twitter social platform by studying the impact based on Arabic tweet content. And explains how these content features are too influential in measuring the credibility of those Arabic tweets.
Mouty, Rabeaa, and Achraf Gazdar [23], focus on discover the credibility of Twitter publishing in Arabic by integrating the social media mining techniques with the Natural Language Processing (NLP) by using the random forest classifier.
El Ballouli, R., et al. [24], presented a Credibility Analysis of Arabic Content on Twitter (CAT) model which uses a binary classifier that classifies a given tweet as either credible or not and uses both content based and user-based features.
The authors [25] enhanced in "TweetCred" algorithm, but there is difference between "TweetCred" algorithm and CAT algorithm which the first one relies in its classification on realtime features only, such as, count of re-tweets, and count of friends. However, CAT utilizes the tweeter"s history for any clues that might be helpful in deciding on the credibility of the tweet.
Al Zaatari, A., et al. [24], creating two corpora for credibility analysis and validate their usefulness for Arabic blogs, the two corpora manually labeled as credible, fairly credible or non-credible by a number of human judges.
Finally, Alrubaian, Majed, et al. [2], introduced a comprehensive study about the information credibility assessment at different levels of features. It addressed a new taxonomy of credibility analysis and assessment techniques. And a cross-referencing of literature review and suggested a new topic for future studies of credibility assessment in social media context. This research suggested the credibility researchers commonly use text analysis tasks. However, analysis of multimedia (images, audio, and video) must be explored further, Text analysis has been employed effectively; nonetheless, semantic analysis of text content has not been explored, the feature levels of credibility assessments require further investigation, especially in terms of relative importance. In some cases, certain features are more important than others, leading to misjudgment of the trustworthiness of the content and source, expand the hybrid models to formulate automation relevant to social media content credibility judgment.
Different from the above work, this paper proposes an efficient assessment model to enhance the accuracy of assessing credibility for news source in online social network specially over the Facebook social media. Online users can check the veracity of this news from credible sources specialized in field or domain at a certain time. Although all the above work assesses the credibility of users or post but didn't assess the credibility of a news source in a specific domain and a time to trust all news from the most credible sources to preserve the authenticity of the news to spread the news through the network correct. Therefore, this research paper will be introduced a Multi-Dimensional credibility assessment to assess the credibility of the news source in a specific domain and time.

III. MULTI-DIMENSIONAL CREDIBILITY ASSESSMENT MODEL
The proposed model assesses the credibility score for different Arabic news sources from multi-dimensional manner. The dimensions are: The credibility Score C i , for a specific domain (Art, Health, Sport, Education and Politics) D and at a specific epoch of Time T using a hybridlevel (Topic-Post-Member) at a social-based information credibility type. The proposed model will pass on three main phases shown in Fig. 1. The first is the preparation of the dataset, the second is feature extraction phase, and the last one is measuring the credibility of the news source phase. These phases will be explained in details in the next sections. www.ijacsa.thesai.org

A. Preparing the Dataset Phase
In this phase, the proposed model will use the "SMAD" dataset which is a new Arabic social media corpus collected and labeled for credibility from Facebook social media for the credibility assessment purpose. it consists of 15,240 textual topic of news collected from several websites (El-youm7, BBC Arabic, El-watan, etc.) scrapped for five domains Sports, Art, Education, Health and Political with size 2MB.
It is trained on 4200 textual topics in different domain in the training phase and about approximately 120 textual topics news item extracted from Facebook social media for testing phase. It preprocessed using Arabic Natural Language Processing (ANLP) and evaluated and gives accuracy of 98% using the TF-IDF classifier algorithm.

B. Features Extraction Phase
There are four types of Social-Based Information Credibility assessment (Post -Topic -Member -Hybrid levels) which can assess the credibility of them. This paper will use the hybrid level to give robust and more accurate information. The following tables, Table I and Table II show the features extracted at each level to be used in the calculation of the credibility scores.

1) Post features:
The credibility of the content is the level of the believability of the content and the source which produces the news. The degree assigned to the post is an indication of the post believability. Table I illustrates four features of Facebook posts used for computing credibility. The reason for choosing these features is that they are the indicator for the user interaction of that post.
2) Member features: Table II illustrates four features of Facebook's user account used for computing credibility at the member level. The values of these features can be accessed via the puppeteer library.

C. Credibility Measurement Phase
Unfortunately, social network users are unable to directly observe how well someone is trusted in a particular domain as a source of credibility. The credibility will be assessed based on content trust and source trust in a particular domain of interest to maximize relevance, credibility, and the quality of the information received to consume the news.
Our model will use a uniform distribution to derive the significance of the news source page on Facebook. The calculating credibility score is based on the Simple Aggregated Score presented in [6] [7] [16] [19], where each score for each feature in a specific domain at the period of time for each news source will be added to provide an aggregated score. The credibility score will be calculated based on features and is rated on a discrete 3-point scale. The rating of the credibility of the news source is based on threshold values a0, a1, a2 with the following mapping: [0, a0) → Highly Credible (HC), [a0, a1) → Neutral (N), [a1, a2) → Highly Non-Credible (HNC).
Each news source NS has several features F. The significance will be calculated for each feature on Facebook posts in a specific domain D in an epoch of time T. Then commutate them to calculate the total significance. The features are like (e.g. number of likes, number of comments, number of shares, if the news source is verified or not, and the number of followers).
To calculate the significance of each feature S there are some procedures that must be applied. The first is rescaling the feature values using a min-max scaling algorithm is the process of transforming the extracted values to be in the range of In the same manner, the calculation will be done in all features to get the significance of news sources. The values are S (F L ), S(F C ), S (F S ). The second is calculating the total significance value as well as calculate the weighted significance of each feature.
To calculate the weight, first, calculate the dispersion of each feature value around its mean (standardization). Standardization is the transformation of each feature and it has a mean of 0 and a standard deviation of 1. The idea is that the closest a value is to the mean; the more reliable it is, whereas the farthest values are outliers. Standardization is applied by using scikitlearn"s and StandardScaler packages.  The weight of (#of Likes) W feature F L will be calculated by Eq. (3).
while that the weight of (#of Comments) W feature F C will be will be calculated by Eq. (4).
W(F C ) = (4) and the weight of (#of Shares) W feature F S will be calculated by Eq. (5).
Finally, the total significance of the news source i (S i ) will be calculated by Eq. (6).
The following algorithm illustrates how to calculate the credibility score for each news source NS in a specific domain D in time T. new source, we scrap using pupputeer library about 2200 post in all domains to calculate its credibility. In "Arabic BBC " news source , we scrap about 1200 post. In (El-youm7) ‫اليوم‬ ‫السابع‬ new source, we scrap about 2250 post. In ‫(الوطن‬El-watan) news source, we scrap 2510. In ‫نيوز‬ ‫سكاى‬ (Sky news) news source, we scrap about 2400 post. For testing the result for classification posts into domains using TF-IDF and KNN algorithms, the experiment results show the algorithm accuracy is 0.98%. The following figures will visualize the credibility scores over different data analysis for different levels and investigate the credibility score at different domains and visualize these scores into graphs.

Algorithm 1: CREDIBILITY SCORE Algorithm
In the first level of data analysis, the credibility score will be calculated for a specific news source in all different domains at any period of time. Fig. 2 depicts the credibility score for the "Arabic BBC" page on Facebook social media through the four weeks of June, in all domains (Education, Art, Health, Sport, and Political).
Also, Fig. 3 shows the credibilty score at a point of time applied on ‫السابع‬ ‫اليوم‬ (El-youm7) page.
The second level of data analysis, the credibility scores Cxi for all news sources NSi in all different domains Di at a point of time T. Fig. 4 depicts the credibility scores for the five news sources " (El-youm7) ‫السابع‬ ‫,اليوم‬ (Arabic BBC) BBC ‫,عربي‬ ‫(الوطن‬El-watan), (Akhbar El-youm) ‫اليوم‬ ‫اخبار‬ and ‫سكاى‬ ‫نيوز‬ (Sky news) "as a sample of news sources in five domains (Education, Art, Health, Sport, and Political) to show the highest and lowest news source in each domain.    The third level of data analysis, the credibility scores for all different news sources at a certain period of time in specific domain. The following figures depict the variation of credibility scores over time for five news sources such as ‫اليوم‬ ‫السابع‬ (El-Youm7), (Arabic BBC) BBC ‫,عربي‬ (El-Watan) ‫,الوطن‬ ‫اليوم‬ ‫اخبار‬ (Akhbar El-youm) and (Sky News) ‫سكا‬ ‫ى‬ ‫نيوز‬ in same domain. Fig. 5 shows the investigation for all news sources in June in the Art domain. The news source (El-Watan) ‫"الوطن"‬ has the highest credibility score in the first week. While the news source (El-Youm7) ‫السابع"‬ ‫"اليوم‬ has the highest credibility score in the second week. In the third week, the news source ‫السابع"‬ ‫"اليوم‬ (El-Youm7) is the highest. Finally, in the last week of the month, the highest one is ‫السابع"‬ ‫"اليوم‬ (El-Youm7). Thus, the overall investigation of data analysis of Fig. 5 the most credible source in the Art domain over the last four weeks is the ‫السابع"‬ ‫"اليوم‬ (El-Youm7) news source. The least credible source in the Art domain over the last four weeks is the (Akhbar El-youm) ‫اليوم"‬ ‫"أخبار‬ news source. Fig. 6 shows the credibility scores of all news sources over a period of time in the Education domain. The highest credible score is the (El-Youm7) " ‫الس‬ ‫اليوم‬ ‫ابع‬ " news source. Therefore, it is the most credible one during this period. The lowest credible source is the ‫اليوم"‬ ‫"أخبار‬ (Akhbar El-youm) news source.
Therefore, it is the least credible Fig. 7 shows the credibility scores of all news sources over a period of time in the political domain, while Fig. 8 shows the credibility score in the health domain.    Finally, Fig. 9 despites the credibility score of the Sports domain for all the news sources over one month.

V. DISCUSSION
Prior studies  have presented the importance of assessing the credibility of the news source and posts. Most of the mentioned work, the most popular social media platform used in the information credibility assessment is Twitter, due to its ease of use and because it has a 140-280 characters" limit in tweets. The second one is the web content or web sources' information credibility assessment, the third is microblogging and weblogs, and the last one is Facebook. While Facebook is the most popular social network worldwide, where there are 2.89 billion monthly active users, However, there is one tool that works on Facebook to assess the credibility assessment [13]. It assesses the credibility at the post-level and uses crowdsourcing to justify its results. Different from the above, our model assesses the credibility of Arabic news sources using hybrid levels (Post-Topic-Member) and works especially on the Arabic language due to the complexity of the essential structure of the Arabic language used as a challenge. Our model assesses credibility along three dimensions. The first dimension involves assessing the credibility of Arabic news sources automatically. The second dimension, the credibility of the news sources will be calculated in a specific domain (Art-Health-Education-Politics-Sports) to be more accurate and help the audience or consumers of the news to support these sources or not and to be guidance and trustable to them to rely on. The final dimension that should be taken into consideration is that news sources can vary their credibility scores within a period of time. Thus, our model assesses the credibility of news sources in a specific domain at an epoch of time with an accuracy of 98%. Finally, our model also, created a new dataset called "SMAD" that contains 5000 Arabic news items classified from Facebook social media. The SMAD corpus gives accuracy of about 98% in five domains. Each domain has been estimated and compared with other different datasets according to quality measurement metrics (precision, recall, Fmeasure and accuracy). In the sports domain, its precision is 0.95, recall is 1 and the F-measure is 0.98. In the education domain, its precision is 0.98, recall is 1 and the F-measure is 0.99. In the arts domain, its precision is 1, recall is 0.96 and Fmeasure is 0.98. In the health domain, its precision is 0.99, recall is 0.97 and F-measure is 0.98. Finally, the political domain precision is 0.98, recall is 0.97, and F-measure is 0.98.

VI. CONCLUSION AND FUTURE WORK
To sum up, the present study constructs a new model assessing the credibility of Arabic news sources on Facebook social media in a multidimensional manner for news sources, in different domains, and at different epochs of time. This model will be visualizing this data in different ways from the existing benchmark dataset for the news to calculate the credibility scores quickly related to topics from social media posts. This model visualizes the results at different data analysis levels: visualizes the credibility assessment scores of El-youm7 at a specific epoch of time in different domains (Art, health, Education, Political and sports), visualizes the five Arabic news sources in all domains at a specific epoch of time and finally visualizes each domain with respect to the scores of five Arabic news sources at a specific epoch of time.
There are number of open issues can be considered in future: First one, construct an Arabic Fact-checking website to visualize the credibility scores to be readable for online users and help them to retrieve the most credible news sources to rely on them. Second, use Semantic similarity analysis methods to detect near-duplicated and similar news retrieved from different sources about a certain claim in a specific domain and a specific period of time should be handled using Natural Language Processing (NLP) methods. Third, sentiment analysis can be taken into consideration to analyze positive and negative comments to get the credibility scores more accurate and precise. Finally, use the Blockchain technology to assess the credibility of each news source in its domain at a time and retrieve all the true news from the most credible sources, which will join into the creditable blockchain network to preserve the authenticity of the news and spread it through the network correctly.