Service Design for Developing Multimodal Human-Computer Interaction for Smart Tvs

A Smart TV integrates Internet and Web features into a TV, as well convergence between computer and TV and can utilize as a computer. Smart TV devices facilitate the curation of content by combining Internet-based information with content from TV providers. Many techniques, such as those that focus on speech, gestures, and eye movement, have been used to develop various human computer interfaces for Smart TVs. However, as suggested by several researchers, user scenarios and user experiences should be incorporated with development techniques to meet user demands on Smart TVs. Thus, this study applies the service design approach for scenario planning and user experience analysis of multimodal interaction development for Smart TVs. This research begins with the service design process and derives the Quality Function Deployment matrix (QFD Matrix) for initial decision-making. Analytical Hierarchy Process (AHP) is then applied to evaluate the priority and relevance of features proposed in the QFD Matrix. Research results show the service design approach is an efficient way for an interdisciplinary team to communicate. The proposed two-stage decision-making processes provide qualitatively analyze and quantitatively measure the priority and relevance of features derived from the service design process. The technique team can then develop prototypes that facilitate multimodal humancomputer interaction on Smart TVs. Keywords—Smart TV; Service Design; Human-Computer Interaction; Quality Function Deployment; Analytical Hierarchy Process


INTRODUCTION
A smart TV is either a TV with integrated Internet capabilities or a television with a set-top box that offers advanced computing abilities and connectivity than a typical TV.Smart TVs may be considered a TV that also hascomputer operating system that often allows users to install and run advanced applications on a specific platform.
A Smart TVcan broadcast broadband web content [1].It has the potential to seamlessly integrate the strengths of TV broadcasting and broadband network services [2].Smart TVs currently provide access to the Internet and legacy web services, and specify which content services are immediately coupled to broadcast content that is rendered on the terminal device [3].
Although a Smart TV attempts to serve audiences through its innovative services, a number of questions remain about the mechanism delivering services to different users via the same platform.Additionally, Smart TVs require innovative human-computer interactions to provide enhanced services and fulfill user requirements [4].Differing from a conventional TV with a remote control, new Smart TV features, such as web search, social networking, multi-user, personalized services and applications development, require innovative -natural‖ human-computer interactions.Using keystrokes on remote controls, touching the TV screen, or using the touch panel on smart handheld devices are inconvenient and impose limitations on users.Some user groups, such as the disabled or elderly in particular face problems when using these services [5].The availability of accessible user interfaces that can adapt to the specific needs of users with impairments is very limited.Notably, no method automatically adapts to multimodal interactions, such that they cannot automatically fit the requirements of users with different impairments [5].
To improve the multimodal human-computer interaction of Smart TVs, one must bring together technicians and designers inter-disciplinary integration to generate a comprehensive roadmap for development and identify the future requirements for Smart TVs [4,5].This research uses the service design approach to organize a cross-discipline professional team, including of computer science and interaction design professionals, to evaluate the features of human-computer interaction mechanisms of Smart TVs.The principles of service design were implemented in scenario planning [6].The quality function deployment (QFD) matrix, a qualitative approach, systematically assesses the correlation between user requirements and technical features [7].Finally, the analytical hierarchy process (AHP) synthesizes the features in the QFD matrix and ranks alternatives.Therefore, both qualitative and quantitative criteria can be weighted and prioritized using informed judgments [8,9].The remainder of this paper is organized as follows.Related work is analyzed in Section II.Section III discusses the design mechanism and proposed methodology.Implementation results of QFD and AHP are then discussed in Section IV. Conclusions and future works are presented in Section V.

II. RELATED WORK
According to [10,11], -service interfaces are designed for intangible products that are, from the customer's point of view, useful, profitable and desirable, while they are effective, efficient and different for the provider.‖The method for making this process integral and holistic is to incorporate the particular visions of all stakeholders, including users, www.ijacsa.thesai.orgdesigners, investors, researchers, technicians, policy makers, consultants and competitors [12].Moggridge asserted that -service design is the design of intangible experiences that reach people through many different touch-points‖ [11].That is, service design is a process of continual updates based on the responses of users who are observed and monitored.
Any application of service design to the multimodal interaction development of Smart TV must consider aspects of both product design and interface design.As pointed out by Obrenovic and Starcevic [13], multimodal interfaces move the balance of interactions closer to the human and offer expressive, transparent, efficient, and robust human-computer interactions.In human-computer interactions, the term modality typically refers to the five human senses-sight, hearing, touch, smell, and taste.Oviatt [14] offered a more practical definition, stating that multimodal systems coordinate the processing of combined natural input modalities, such as speech, touch, hand gestures, vision, head and body movements, with multimedia system output.Thus, by applying a service design approach to the multimodal interaction of Smart TVs, this work follows Oviatt's definition [14] and focuses on applications of speech, touch, hand gestures and visualizing.Moreover, this work follows some features and characteristics of service design that were summarized by Blomkvist and Holmlid [15,16], including the following.
1) Assessing services from a holistic and detailed point of view.
2) Considering both artifacts and experiences.
3) Making services tangible and visible via visualizations.
In addition to the service design approach, the QFD matrix and AHP are also utilized simultaneously to systematically identify the criteria derived from service design scenario planning, and to weight and prioritize criteria.
As proposed by many researchers, the QFD matrix transforms customer requirements (CRs) into technical requirements.Originally a quality improvement tool, the QFD matrix has been widely used to develop new products [7,17,18].For effective product design, a design team must be cognizant of what they are designing and what users will expect.The QFD matrix is a systematic design approach based on an in-depth awareness of customer desires, coupled with integrated corporate functional groups.The QFD matrix translates customer desires into design characteristics for each stage of product development.The ultimate goal of is to translate often subjective criteria into objective criteria that can be quantified and measured and which can then be used to design and manufacture the product.According to Akao [17], weighting customer requirements is a critical step in building a QFD matrix.The simples tmethod is to ask an expert panel to apply a point scale, such as 1-5 or 1-9 and score each CR.However, this simple system has two weaknesses: it does not prioritize customer requirements; and weights are subjective and depend on panel consensus.
To overcome these weaknesses, several researchers and practitioners have advocated using the AHP to weight CRs.The AHP is a structured technique for dealing with complex decisions [19][20][21].Conditions of uncertainty arise from subjective information (presented as quantitative and qualitative values) used in decision-making processes.This uncertainty is based on incomplete decision knowledge about the properties of an object, insufficient confidence in the accuracy of expert judgments, knowledge inconsistencies, and information fuzziness [22].Therefore, implementation of a decision-making problem under uncertainty requires a comparison of factors lacking quantitative characteristics or a simultaneous comparison of quantitative and qualitative characteristics.In the capacity of tool for such problems decision heuristic methods based on expert judgments may be used in addition to the AHP [9].
The AHP enables groups of people to interact and focus on a certain problem, modify their judgments and, as a result, combine group judgments in accordance with the main criteria [23].Applying the AHP to weight CRs in a QFD matrix provides a rational framework for structuring a decision problem.The combined AHP-QFD approach can quantify CRs and elements, relate those elements to overall CR goals and evaluate alternative solutions.The combined AHP-QFD approach has been used successfully to assess customer needs based on a multiple-choice decision analysis [24].Gupta et al. [25] reviewed uses of the QFD-AHP to evaluate and select methodology for an innovative product design concept.The methodology combining QFD-AHP was mainly used as a multi-criteria decision method for evaluating user requirements.By considering the multimodal interaction requirements of Smart TVs and characteristics of service design, this work uses this methodology to evaluate the multimodal HCI design and development of Smart TVs.

III. METHODOLOGY
This work integrates design thinking with technology development process for developing a multimodal Human Computer Interface (HCI) for Smart TVs. Figure 1 shows the comprehensive structure of the integration of design thinking concept with technology development an inter-disciplinary approach.www.ijacsa.thesai.orgThis structure has 5 major phases (Fig. 1).The first phase outlines the vision for developing the Smart TV multimodal HCI design and is based on a review of development trends and visions for Smart TVs.The second phase proposes a plan for inter-disciplinary integration of domain experts in technology and interaction design by holding a service design workshop for brainstorming.The third phase, which is the section for integration, defines the user scenario and technology benchmark with user-centered design insights.The fourth phase develops the applications and multimodal interaction prototype by integrating technologies into the user interface.The final phase evaluates user experiences with the prototype on a system usability scale that can collect scientific data (e.g., eye-tracking system) for objective analysis.Evaluation results are then feed back to the inter-disciplinary team to modify and adjust the prototype.This paper will present the results and evaluation of the first three phases.
For practical implementation, this work follows the implementation process (Fig. 2), the details of which are as follows.
Figure 3 shows inter-disciplinary team discussion and character map of service design workshop.This workshop helped the team gain a clear understanding the features of multimodal HCI design.The many ideas generated were then narrowed down from global thoughts into specific and applicable features that meet user requirements and are applicable to technical features development.

A. Quality Function Development (QFD) Matrix
All features derived from the workshop are listed in a QFD matrix.Figure 4 presents the conceptual diagram of the QFD.The QFD matrix shows the importance of each feature via correlation analysis of user requirements and features of technical requirements.It also shows user recognition by describing their experiences to competitors by giving a value to their importance.The importance range is 1-5 and their thinking is limited to strong, moderate, or poor.This method tells us how strongly the features (product characteristics) are related to user requirements and reflects the strengths of existing products.This work uses the QFD matrix to systematically list the features of the multimodal HCI design.

B. Analytic Hierarchy Process
The three basic AHP steps in thisresearch are as follows.
1) Describe a complex decision-making problem as a hierarchy.
2) Use pairwise comparison techniques to estimate the relative priority of various elements on each level of the hierarchy.
3) Integrate these priorities to develop an overall evaluation of decision alternatives.
The AHP calculation template provided by Goepel [27] is used for primitive analysis of analytical results.The workbook consists of 20 input worksheets for pairwise comparisons, a sheet for consolidating all assessments, a summary sheet for systematic results, a sheet with reference tables (random index, limits forthe geometric consistency index (GCI), and judgment scales) and a sheet for solving the eigenvalue problem when using the eigenvector method (EVM).
The algorithm and formula used to weight and for pairwise comparisons are as follows.

a) Multi-criteria decision
In terms of Multi-Criteria Decisions, the AHP uses a threelevel hierarchical decision system: the first level considers a decision goal G; on the second level, it has n independent evaluation criteria-C 1 , C 2 ,...,C n , such that , where w(C i ) > 0, i= 1,2,...,n, w(C i ) is a positive real numberweight, or, relative importance of criterion C i subject to goal G; on the third level m variants (alternatives) of decision outcomes V 1 , V 2 ,...,V m are considered, such that again , where w(V r , C i ) is a non-negative real number-an evaluation (weight) of V r subject to the criterion C i , i= 1,2,...,n.This system is characterized by the super matrix W, where whereW 21 is the n1 matrix (weighting vector of the criteria), i.e., , andW 32 is the mn matrix: The variants can be ordered according to these priorities.Z = W 32 W 21 (4) In real decision systems with three levels, typical interdependences exist among individual elements of the decision hierarchy (e.g., criteria or variants).Consider now the dependences among the criteria.This system is then given by the super-matrix W: In general, matrix ( 5) is not column-stochastic; hence, the limiting matrix does not exist.The Stochasticity of this matrix can be retained by additional normalization.A limiting matrix W  then exists and the vector of weights Z can be calculated by formula (6).

(6)
As matrix W 22 is resembles the zero matrix, and the dependences among criteria are generally weak, this result can be replaced by the first four terms of Taylor's expansion: b) Priority Calculation Priorities p i in each input sheet are calculated using the row geometric mean method (RGMM).With the pairwise N x N comparison matrix A = a ij Thus,

c) Inconsistencies
To find the most inconsistent comparison, this work looks for the pairi,j with ) max( Consistency ratios (CRs) are calculated in all input sheets and in the summary sheet.With  max, the calculated principal eigenvalue-either based on the priority eigenvector derived by the RGMM in the input sheet or derived by the EVM in the summary sheet-the consistency index (CI) is given as The Alonson/ Lamata linear fit is applied, yielding CR:

d) Aggregation of individual judgments (Consolidation of participants)
The consolidated decision matrix C (selected participant -0‖) combines all k participants' inputs to obtain the aggregated group result.The weighted geometric mean of the decision matrices elements a ij(k) using the individual decision maker's weight w k , as given in the input sheets, is used:

ln exp e) AHP consensus indicator
The AHP consensus is calculated in the summary sheet based on the RGMM results of all inputs using Shannon alpha and beta entropy.The consensus indicator ranges from 0% (no consensus) to 100% (consensus).

AHP consensus indicator S
where?H , ,  is the α, β, γ Shannon entropy for the priorities of all K decision makers/participants.

Shannon alpha entropy
One must adjust for the maximum score c max of the AHP scale and where N is number of criteria, and K is the number of decision-makers/participants.

A. Quality Function Deployment Matrix Results
Figure 5 shows the QFD matrix results.Based on QFD matrix analysis.the smart interactive user interface and privacy settings are two of the most important features of Smart TVs, followed by gesture and voice control, customization of personal settings, and layout adaptation.These visualized results show that the multimodal interaction design is very important to Smart TVs.

B. Analytical Hierarchy Process Results
Features in the QFD matrix are further processed as criteria in a questionnaire.To collect pairwise comparison results, 30 questionnaires were dispatched to inter-disciplinary experts, including faculty, researchers, and professionals in the fields of computer science, electronic engineering, and interaction design.The quantitative results are then applied in the AHP template to weight and prioritize each feature.Figure 6 shows the table of analytical results for the user interface class in the category of customer demands.According to the ranking, the smart interactive user interface is followed by intuitive operation, and gesture and voice control.
Next, one must turn this matrix into a ranking of criteria (Fig. 7).According to Saaty [21], the eigenvector solution was the best approach.The computed eigenvector gives the relative ranking of criteria.The most important criterion is smart interactive user interface(38.8.%), followed by intuitive operation (35.4%), and gesture and voice control(25.9%)(Fig. 4).Also, the CR is <1.5%, meaning the ranking is credible.The weight scale from the QFD matrix and weights and ranking by the AHP method are correlated (Table 1).
As shown in Table 1, there is a correlation between the QFD Matrix's weight scale and the weights and ranking from AHP method.The most demanded feature of Smart TV HCI is the user interface (31%) then the visual design.(27%).There are not much difference between the personal setting (22%) and layout development (20%).However, the top 5 design priority for Smart TV HCI design feature are: layout adaptation (46%), smart interactive user interface (39%), personal customization setting (39%), nature design in visual design (36%), user interface intuitive operation (35%).The top 3 feature from the AHP are similar to the QFD weight.However, the fourth and fifth design feature priority are very different from OFD results.The gesture and voice control feature, has a high priority in the QFD matrix, differing markedly from its low weight by the AHP method.The likely reason is that this customer requirement differs from the technical perspective.Additional efforts are needed in gesture and voice control when designing multimodal interaction for Smart TVs.The results show some guideline for industry to the development of Smart TV HCI design.

V. CONCLUSIONS AND FUTURE WORK
The AHP, based on the hierarchy principle, assumes consecutive decomposition of multiple aims with degree increasing toward lower levels.Hierarchy development conforms with the principles of system approaches toward task analysis and can facilitate the process of creation and formalization of Participatory Technology Development (PTD) priorities.One main advantage of the AHP is the determination of subjective criteria and scores based on pairwise comparisons.Another advantage involves the structural organization of problem components.The AHP provides consistent assessment tools, analyzes alternative sensitivities, uses relatively simple mathematic equations, and allows participation of different specialists or groups.
A strong point of the AHP is the independence of its application from the activity sphere.The AHP results show the service design approach is an efficient way for communication among interdisciplinary team members.The proposed twostage decision-making processes qualitatively analyze and quantitatively assesses the priority and relevance of features derived from service design process.The technique team can then develop a prototype that demonstrates multimodal interaction with confidence, thereby fulfilling user demands.
Three possibilities directions exist for future study.
1) Include raw prices (retail prices) in the QFD matrix method and AHP.This would be comparative, as this criterion may affect user demands.For example, if the Kinect Sensor price is excessive, and users think it is not as effective as, say, the motion leap sensor embedded in a remote control could be considered as a criterion to be evaluated.
2) In-depth understandings of current and existing demands are essential.Failure probability still exists as the AHP does not work well when evaluating quantitative values; it is much better at creating qualitative values.
3) Implementing these methods is acceptable.For future work could identify Smart TV features.If field report results could be evaluates and joined with questionnaire results, this project would generate relevant and effective content.

Fig. 1 .
Fig. 1. the Comprehensive Structure of Inter-disciplinary Integration of Design Thinking with Technology Development for Smart TVs

Fig. 3 .
Fig. 3. Inter-disciplinary Team Discussion and Character Map of the Service Design Workshop

Fig. 4 .
Fig. 4. Conceptual Diagram of Quality Function Deployment ijacsa.thesai.organd facial recognition are highly prized by respondents.Privacy via encryption and decryption, and traditional/single sign-in on account management are also required by customers.Respondents agreed that Apple TVs and Smart TVs have user-friendly interfaces.The privacy feature has already been developed by Apple TV, general Smart TV, and Google TV.The QFD matrix results comprehensivelyshow a significant role to help the development.These results are also evaluated and calculated via the AHP.Each criterion is compared to another, such that the importance weight is derived.

Fig. 6 .
Fig. 6.Summary of the User Interface Class in the User Requirement Category-AHP Results

TABLE I .
COMPARISON OF QFD AND AHP RESULTS