Question Answering Systems : A Review on Present Developments , Challenges and Trends

Question Answering Systems (QAS) are becoming a model for the future of web search. In this paper we present a study of the latest research in this area. We collected publications from top conferences and journals on information retrieval, knowledge management, artificial intelligence, web intelligence, natural language processing and the semantic web. We identified and classified the topics of Question Answering (QA) being researched on and the solutions that are being proposed. In this study we also identified the issues being most researched on, the most popular solutions being proposed and the newest trends to help researchers gain an insight on the latest developments and trends of the research being done in the area of question answering. Keywords—Question answering systems; community question answering systems


INTRODUCTION
In this paper we present a study of the latest research being done on question answering systems.We attempt to give an answer to questions like: Are researchers gaining or losing interest in QAS?What are the characteristics of QAS being given most attention to?What are the topics of the research being given most attention to?What are the challenges faced by researchers in this area?What kinds of solutions are being proposed?What are the newest features being applied?What are possible trends of the research in this area?We collected publications from top conferences and journals on information retrieval, knowledge management, artificial intelligence, web intelligence, natural language processing and the semantic web in the last three years and made a quantitative and topicbased analysis of these publications.Our work can be used to help researchers gain an insight on the present state and latest trends of the research being done in the area of question answering systems.
Unlike related work [1], [2] that classify and report the state of the art of question answering systems, our study makes a quantitative analysis on the amount of research being done in the area of question answering as well as topic-based classification and research trend identification.To the best of our knowledge this is the first review of QAS from this perspective.
The rest of this paper is organized as follows: In Section 2 we describe the methodology used in our study and define objectives and research questions.Section 3 makes a quantitative and topic-based analysis of the collected research.
Section 4 discusses the results and conclusions derived from our study.Finally, we list the selected papers in Appendix A.

A. Research Questions
As a primary step in the investigation, retrieval and selection of the most accurate publications for our review we have defined the following research questions:

B. Search Keywords and Source Selection
In order to extract the most relevant information for our review we used the following keywords and their combination and synonyms.The search string below was used as a query to search for publications in different online digital libraries: (-Question answering‖ OR -question answer‖ OR -question answering system‖ OR -question answering systems‖).The search for these keywords was done on the title of the publication, as well as the abstract.www.ijacsa.thesai.org

C. Inclusion Criteria
Table 2 lists the inclusion and exclusion criteria that we used to collect papers.

Inclusion criteria Exclusion criteria
Relevant to the topic of our review Review papers Papers that have been published in the last three years (2014 -2016) Reports Published in top conferences and journals on information retrieval, web intelligence, artificial intelligence, natural language processing and the semantic web We did not collect review papers and reports because our aim is to analyze the existing implementations and developments of QAS.

III. QUANTITATIVE AND TOPIC-BASED ANALYSIS
We first make a quantitative analysis of the collected research.We had divided the papers into two categories: new systems and existing system improvement.Table 3 shows the total number of publications for each year, as well as the number of publications for each category.These data suggest that QAS are gaining popularity and interest from the research community.We also notice that the majority of contributions are new QAS.This suggests that QAS is a rapidly growing and evolving field of research where new ideas are being implemented continuously with success.This also justifies the fact that a considerate amount of research is being done on improving and implementing new ideas to existing state of the art QAS and incremental results are being achieved.As regards RQ1 we can say that there is a growing trend in publications indicating an increased interest in this area from the research community.
For the topic-based analysis we make a classification of the systems described in the collected papers.We identify the amount of research being done according to this classification and try to answer the research questions posed in Section 2.
We studied the systems from three different points of view: 1) system characteristics; 2) research topic; 3) solution approaches.

A. System Characteristics
We identified five main characteristics of QAS: 1) System domain: open domain vs closed domain; 2) System type: Community Question Answering System (CQAS) vs noncommunity QAS; 3) Question type: factoid vs non-factoid questions; 4) Information source: documents vs structured Knowledge Base (KB); 5) Information source type: single vs multiple.

1) System domain
This characteristic describes the domain of the questions that a QAS can accept.Closed domain QAS accept questions only from a specific domain while open domain QAS do not have this limitation.The greatest part of the systems we studied is open domain with a ratio of 117 open domain to 12 closed domain QAS, translating to a percentage of 90.6% open domain to 9.4% closed domain.

2) System type
This characteristic describes the type of the system from a community perspective.Original QA systems were closed encyclopedic-like systems with the system relying on its own knowledge for answering questions.Some of the modern QA systems like Quora1 or Yahoo!Answers2 are communitybased where the users rely on expertise from the community to get an answer for their question.The majority of the systems we studied were non-CQA with a ratio of 73 non-CQA to 56 CQA, translating to a percentage of 56.6% non-CQA to 43.4% CQA.This indicates that CQAs have gained an important part in QA research.

3) Question type
This characteristic describes the type of the questions the system can accept.A factoid QAS is a system that provides concise facts like -What is the population on Earth?‖In contrast in a non-factoid QAS the system can be asked to provide an answer to a math question, how to change the oil of the car or even more complicated answers like those on Quiz www.ijacsa.thesai.orgBowl3 .The majority of the systems we studied were factoid QAS with a ratio of 111 factoid to18 non-factoid QAS, translating to a percentage of 86% factoid to 14% non-factoid.We consider worth mentioning the fact that there is an increase in publications regarding non-factoid QAS throughout the years with 2 publications in 2014, 6 publications in 2015 and 10 publications in 2016.

4) Information source
This characteristic describes the source of information the QAS uses to generate the answer.We identified two types of information source: documents and structured KB.For the first type, the QAS information is organized as a set of documents from which it tries to make a match between question and answer.For the second type, the QAS information is organized in a form of structured KB where the data are linked by semantics.The majority of the systems we studied use documents as information source with a ratio of 71 KB-centric to 56 document-based QAS, translating to a percentage of 56.6% KB-centric to 43.4% document-based.
We consider worth mentioning the fact that for the year 2016 we identified five systems described in (P34), (P68), (P84), (P87) and (P90) that deal with image-based information source.The system described in (P68) has both text and image-based information source, while the others are entirely image-based.This kind of systems is not present during 2014 and 2015.

5) Information source type
This characteristic describes the types of information source the system uses.We identified two types of information source: single and multiple.Single information source systems use only internal information to generate an answer.This information may be organized either in a structured KB or as separate documents.A multiple information source type system uses external data like documents, web search, query logs, or even entire KBs besides its own internal information to generate an answer.The majority of the systems we studied are single information source systems with a ratio of 120 single information to 9 multiple information source systems, translating to a percentage of 93% single information to 7% multiple information source systems.We consider worth mentioning the fact that 75% of the contributions on multiple information source QAS were made in 2016.

B. Research Topics
We identified three research topics from the papers we collected: 1) question processing; 2) information source and organization; 3) answer processing.

1) Question processing
We identified 80 publications dealing with this topic.This number comprises 62 % of the total number of publications.We divided this topic into three subtopics: 1) question analysis and generation; 2) question routing; 3) question-answer matching.
The question analysis and generation subtopic deals with user query analysis, identification of query intent, generating of possible candidate questions from user query and selection of the most relevant question.
The question routing subtopic deals with finding possible answerers to a question posed by a user.This is relevant in CQAS routing a question to the right users improves overall system accuracy.
The question-answer matching deals with finding possible matches between user question and document text or KB entries in the information source.Some of the publications we studied dealt with mixed subtopics totaling an amount of 14, translating to a percentage of 17.5% out of 80 publications.
Table 4 illustrates the number of publications for each subtopic.

2) Information source and organization
This topic deals with the way the information is organized in the QAS and its sources.We identified 13 publications dealing with this topic, comprising 16.25% of the total publications.We divided this topic into three subtopics: 1) knowledge base creation; 2) knowledge acquisition; 3) knowledge base linking.
The knowledge base creation subtopic deals with the way information is organized semantically in the QAS.
The knowledge acquisition subtopic deals with getting information from multiple sources in order to gain knowledge about the question topic to be able to find an answer.
The knowledge base linking subtopic deals with finding the right sources that can answer the user query and integrating partial answers from different sources.

3) Answer processing
We identified 65 publications dealing with this topic, making up for 50.3% of the total number of publications.We divided this topic into three subtopics: 1) answer detection and ranking; 2) answer summarizing and generation; 3) answer validation and selection.
Answer detection and ranking deals with detecting possible answers for a user question and ranking them according to question relevance.
Answer summarizing and generation deals with aggregating answers from possible different sources as well as summarizing and generating the final answer.
Answer validation and selection deals with validating possible candidate answers and selecting the most relevant one.
Some of the publications we studied deal with multiple subtopics from the same topic, such as answer detection and ranking, as well as answer validation and selection.There is also a considerable amount of publications dealing with both the question processing topic and answering process topic.This phenomenon occurs also for some publications dealing with question processing, information source and answer processing.The topic overlapping occurs for 26 distinct publications, translating to a percentage of 20% of the total number of publications.
Table 6 illustrates the number of publications for each subtopic.

C. Research Challenges
In order to address RQ4 we identified the main research challenges involved in the selected publications.We divided them into two categories according to system characteristics: 1) KBQA; and 2) CQA.

1) Research challenges in Knowledge Base Question Answering Systems
We identified the following challenges for KBQA systems:  Lexical gap between natural language and structured semantics of the knowledge base: We identified it as the most frequent problem.It concerns differences in sentence representations between the unstructured natural language question and the structured knowledge base.It also concerns the many ways of expressing knowledge in a knowledge base.
 Entity identification and linking: This was another prominent challenge.The challenge of entity identification and linking concerns the ability of the system to correctly identify the subject entity in question and link it to a triple in the knowledge base.
 Questions involving multiple entities: It concerns the ability of the system to identify and reason over multiple subject entities in question and link it to the relevant triple in the knowledge base.
 Passage question answering: This is a challenge on non-factoid question answering where the answer is in the form of a paragraph.Question-answer matching is a challenging task as it requires effective representations that capture the complex semantic relations between questions and answers.

2) Research challenges in Community Question Answering Systems
We identified the following challenges for CQA systems:  Lexical gap between questions: It was one of the most frequent problems in the selected publications.It concerns differences in natural language formulation of questions.Different users ask for the same information but they formulate the question in different ways.This results in many questions that are semantically equivalent but differ lexically.
 Lexical gap between questions and answers: This was another frequent problem.Similar to the lexical gap between questions, sometimes question and answers can be highly asymmetric in the information they contain.There is also a technical terminology gap between questions and answers.Questions are posed by novices or non-experts who use less technical terminology while experts who answer questions use the correct terms.
 Deviation from question: It concerns the phenomenon of answer thread becoming irrelevant to the question.Answers are given in the form of comments but sometimes users engage in discussion and deviate from the original question.

D. Solution Approaches
The systems described in the papers we collected use techniques of Natural Language Processing (NLP) and machine learning to complete their tasks.We identified three approaches: 1) neural networks; 2) probabilistic model; 3) algebraic model.
For the first approach, the neural networks are used as reasoning agents that select candidate answers and determine their relevance to the given question.For this approach we identified 36 publications translating to a percentage of 27.9% out of 129 publications.
In QAS that use probabilistic model, similarities are computed as probabilities that an answer is relevant to a given question.The answers are ranked based on their probability of relevance to the question.The process of answer selection is treated as a probabilistic inference.For the probabilistic models approach we identified 57 publications, translating to a percentage of 44.1% out of 129 publications.www.ijacsa.thesai.orgIn QAS that use the algebraic model, the question and candidate answers are represented as vectors in a multidimensional space.The system computes the similarity between these vectors as a scalar value.The more similar an answer vector is to a question vector, the more likely it is that the answer is relevant to the question.For this approach we identified 36 publications, translating to a percentage of 27.9% out of 129 publications.
Table 7 summarizes these results.

IV. DISCUSSIONS AND CONCLUSIONS
In this paper we presented a study on the current state of research on question answering systems.We can answer RQ2 from three different points of view: domain type, question type, system type.From the domain type point of view, the QAS that are most popular and are being given more attention to are open domain QAS.This is justified by the need of modern systems to be extensive and inclusive of all areas of information and knowledge.
From the question type point of view, the QAS that are most popular and are being given more attention to are factoid question answering.However, we noticed a growing number of contributions, especially in 2016, on non-factoid QAS.This fact suggests a growing interest in the research community for this kind of QAS and a possible trend towards systems that are more intelligent and closer to humans.
From the system type point of view, the QAS that are most popular and are being given more attention to, are non-Community QAS with the most number of contributions.However we noticed that a great amount of research is being done on CQAS and the difference in publications for the two systems is not very big.This reflects the increasing role that social networking and online communities have in the acquisition of knowledge.
To answer RQ3 we identified the topics of research with more contributions.We can say that most of the research is being done on issues regarding question processing.This is justified by the need to understand user questions better in order to provide a more accurate answer.We also find worth mentioning that a considerate amount of research is being done on issues involving all the answering process from information source organization to question analysis and answer generation.
As regards RQ4, the most prominent challenge is the lexical gap.It is evident in the difference between questions expressed in natural language and the semantically structured information of the KB.The lexical gap is also present in CQAS as the difference between user questions asking for the same thing using different words, as well as between answer and question which can, sometimes, differ considerably from a lexical point of view.Another prominent challenge for KBQA systems was the question entity identification, especially in questions involving multiple entities.The lexical gap can have a negative effect on this problem and increase the difficulty of entity identification.
As regards RQ5 we can say that the solutions being applied to solve various issues of the answering process are natural language processing and machine learning methods implemented with neural networks, algebraic and probabilistic models with the latter having the most number of contributions.
To answer RQ6 we identified some new characteristics that are recently being integrated into QAS and tried to identify possible research trends.We noticed a growing number of contributions on multiple knowledge base QAS, with 75% of them during the year 2016.This is indicative of increased research interest in this type of systems and a future research trend justified by the need to create more flexible systems that obtain and validate answers from multiple and possibly external sources in cases when a single KB is not enough to answer the question.We also noticed constant increase in the amount of contributions on non-factoid QAS.We can identify this as an increased research interest and future research trend towards systems that are more intelligent and closer to humans.
We consider worth mentioning a new type of information source for QAS that is being researched on during the last year.This is image-based information retrieval where the information source for finding the answer is either entirely composed of an image database or is a text and image hybrid.We can identify this as a research trend motivated by the need to create QAS that go beyond the traditional boundaries of text based systems towards a more complete artificial intelligence.
As a last point of discussion we find worth mentioning an overlapping of some of the QAS that we studied with other areas such as user behavior (P59) and decision support systems (P9), (P111).However, there is a limited number of research contributions and we cannot identify possible trends.www.ijacsa.thesai.org RQ1: Are researchers gaining or losing interest in QAS? RQ2: What are the characteristics of QAS being given most attention to?RQ3: What are the topics of the research being given most attention to?RQ4: What are the challenges faced by researchers in this area?RQ5: What kinds of solutions are being proposed?RQ6: What are the trends of research in this area?

TABLE III .
PUBLICATIONS THROUGHOUT THE YEARS

Table 5
illustrates the number of publications for each subtopic.

TABLE IV .
NUMBER OF PUBLICATIONS FOR QUESTION PROCESSING SUBTOPICS 7 www.ijacsa.thesai.org

TABLE VI .
NUMBER OF PUBLICATIONS FOR QUESTION PROCESSING SUBTOPICS