Menu Positioning on Web Pages. Does It Matter?

—This paper concerns an investigation by the authors into the efficiency and user opinions of menu positioning in web pages. While the idea and use of menus on web pages are not new, the authors feel there is not enough empirical evidence to help designers choose an appropriate menu position. We therefore present the design and results of an empirical experiment, investigating the usability of menu positioning on web pages. A four condition experiment was conducted by the authors. Each condition tested a different menu position. The menu positions tested were left vertical, right vertical, top horizontal and bottom horizontal. The context was a fictitious online store. The results, based on statistical analysis and statistically significant findings, suggest that the top horizontal and left vertical positioned menus incurred fewer errors and fewer mouse clicks. Furthermore, the user satisfaction ratings were in line with the efficiency aspects observed.


INTRODUCTION
As the internet has been around for some years now, many users around the world have developed an adaptability when using web sites.This is because web sites do not always follow a predefined structure for their content or a predefined navigation structure.This in turn has forced users to become adaptable to the web sites they visit.This in some cases can lead to more errors or other potential problems when navigating around a web site.
There are potentially many different designs one can use to help users achieve their navigation aims whilst using a web site.Some examples include, having menu options placed on a web page on the left vertically, right vertically, top horizontally and bottom horizontally.Some web designers have even used other options on live web sites, e.g.having a menu in the middle of a web page [10].Furthermore, there are respected works published which aim to help with menu design, such as the work of Dix, Finlay, Abowd and Beale [5], Benyon [3] and Rogers, Sharp, and Preece [14].
However, the authors of this paper feel that not enough is known about which menu design or positioning on a web page is better in terms of efficiency and user preference.Although there has been previous work done in this area, the authors feel that overall there is not clear enough evidence to show which menu positioning may be overall better.This is because in several studies carried out over the years, the results presented are not categorically in one single direction.This lack of clear knowledge can then in turn negatively affect the aim of achieving fully universally designed web pages, e.g.menus placed at non-optimal positions on a web page could affect the universal design principles of 'Simple and Intuitive Use' and 'Tolerance for Error' [15].
Therefore in this paper some of the main works of others conducted with similar themes will be discussed.Then this paper is linked to some earlier work done by the authors and then will follow the description of an empirical experiment conducted to try and gain a better understanding of menu positioning on web pages.Lastly the main results from the data collected will be presented along with some conclusions based on our findings.

II. PREVIOUS WORK
Web navigation and investigating better ways to achieve it has been looked into by other researchers, e.g.Gwizdka and Spence [8] looked at ways of trying to predict 'lostness' and possible task success whilst navigating the web.Their reasoning suggested that such predictions could help in designing better web sites.Some researchers have also looked at the aspect of satisfaction whilst navigating and using web sites.De Wulf, Schillewaert, Muylle and Rangarajan [4] adopted a mostly empirical approach to reach a conclusion that essentially suggests that if a web site is designed to give the users some pleasure the web site will tend to be more successful than if it does not give any pleasure.The authors suggested in their paper that features of a web site, such as the 'content, organisation and technology' can have an impact on the users' pleasure.They also found that pleasure or lack of pleasure can affect 'success variables'.By the authors' own discussion, they do suggest that their study has some limitations in their methodological approach to carrying out the study/data collection, e.g.participants supplied their own data etc.Also Benbunan-Fich [2] conducted an evaluation of a commercial web site.A form of think aloud protocol was used and various usability problems were found, including problems with the navigation.She also found problems with the web site's content and interactivity.Some of the issues found with the navigation, involved users not being able to find certain categories of item, not being able to see controls for moving to another stage of the interaction and unclear links and/or buttons.One of the aspects that could have been stronger in this study would have been to use a larger sample of participants.However the work shows how potentially mainstream web sites can fail on simple usability issues.www.ijacsa.thesai.org In another study by McCarthy, Sasse and Riegelsberger [13] menu positioning was investigated in the context of a commercial web site which was manipulated to be either simple or complex.They tested three different menu positions (left, top and right of the page).Further, their study used eye tracking to examine where users were looking at certain points in the interaction.As one would expect, task time was significantly longer with the complex site.The different menu positions did not incur any statistical significance when averaged across the tasks done and the simple and complex sites.However there was better performance with the left positioned menu in relation to the use of the first page of the web site.When users then moved to a second page there were no differences for performance.The eye tracking data showed that the 'glances' made by the users had the heaviest distribution in the middle area of the page.
Overall the authors concluded that humans adapt quickly to a 'different' layout on a web page and that then the performance is not negatively affected.The only difference is with the use of the first page as mentioned above.They suggest that if there is some evidence to indicate that some other layout is good, then designers should not feel they cannot use it due to existing conventions.
One of the issues with this aspect, is that the reduction in performance with the first page a user visits is still not a desirable outcome.It is not clear from this study if such an approach would have a commercial impact, e.g.fewer visits and then possibly less profit.Furthermore in the study discussed above by De Wulf et al [4] some emphasis was placed on the pleasure factor of a web site.It is unclear if the approach suggested by McCarthy et al [13] would lead to less pleasure on the part of the users and then potentially less visits etc.
Furthermore, another interesting study by Fang and Holsapple [6] conducted an investigation into navigation structures for web sites.They tested three types of hierarchy ('subject-oriented, usage-oriented' and a combination of the first two) in relation to a series of simple and complex tasks.The authors found that the 'usage-oriented' and the combined form provide better usability than the 'subject-oriented' type.
However one aspect worth noting about the study is that the participants used in the study all had a basic knowledge of production and operations management (POM) via a course they had all taken.This was very good for ensuring that probably most participants had approximately equivalent knowledge about the topic.This is relevant as the web site prototype was also in the context of POM.Another aspect worth mentioning is that based on the screenshots included by the authors, the actual appearance of the web site prototype could have been more realistic looking.We feel that this could have had unknown effects on the results as web sites in 2007 were usually much more 'advanced' looking when compared with the prototype.Also in a study by Yu and Roh [16], they investigated three different menu types.These were 'a simple selection menu, a global and local navigation menu and a pull-down menu'.In order to test these menus, they developed a prototype to look like a shopping web site.A group of participants were then recruited to take part in a series of tasks involving the finding of some information in relation to certain products.The main results showed that the pull-down menu incurred better searching performance in terms of speed and 'browsing task performance' was better with the global and local navigation menu.This was in terms of speed.The subjective feelings of the participants did not differ significantly across the three experimental conditions.One aspect worth mentioning concerns the structure of the prototype used in the study.This had 'four pages at each of the four levels' and could have had some unexpected effect, e.g.most real web sites are not structured so precisely.
Lastly, one of the authors of this paper has been involved with research into menu design for some time and in [12] the results of an empirical experiment were presented.In this study, a left vertical menu and a fisheye menu placed horizontally at the top of a page in a prototype supermarket web site were tested against a real supermarket web site using a horizontal menu placed at the top of a page.Overall no real differences in the comparison made were found.However on closer examination of the work, it was felt that perhaps the tasks that were designed could have been more challenging.If all the tasks are easy, then it could be difficult to observe differences in different kinds of menus.The authors of this paper have therefore tried to address this issue in the experiment described in this paper, by having some tasks that are slightly more difficult compared to our previous study [12].Further, we have also tried to test more menu positions (four in total) than in the first experiment with the aim of being even more thorough.
Having discussed some of the previous relevant work done by others, the rest of this paper will now describe the details of our experiment, including the results and then some overall conclusions will be presented.

A. Hypotheses
Four hypotheses (null and positive counterparts) were devised for this work and the main themes concerned task times, errors, mouse clicks and user satisfaction: 1) H 0 : There will be no difference between the left, top, right and bottom positioned navigation bars for task time.
H 1 : The navigation bar on the left hand side of the web site will incur shorter task times than the top, right and bottom positioned navigation bars.

2) H 0 : There will be no difference between the left, top, right and bottom positioned navigation bars for errors.
H 1 : The navigation bar on the left hand side of the web site will incur fewer errors than the top, right and bottom positioned navigation bars.
3) H 0 : There will be no difference between the left, top, right and bottom positioned navigation bars in terms of the number of mouse clicks used to complete the tasks.
H 1 : The navigation bar on the left hand side of the web site will incur fewer mouse clicks than the right, top or bottom navigation bars to complete the tasks.www.ijacsa.thesai.org4) H 0 : There will be no difference between the left, top, right and bottom positioned navigation bars in terms of user satisfaction whilst completing the tasks.
H 1 : The navigation bar on the left hand side of the web site will incur more user satisfaction than the top, right and bottom positioned navigation bars whilst completing the tasks.

B. Users
56 participants were recruited.The participants chosen for this experiment were recruited by means of a carefully designed recruitment questionnaire.The authors decided to select users with certain characteristics.The main characteristics were high computer usage experience, high confidence in using computers and experience of using the internet.Linked to these characteristics, the authors specifically decided to exclude anyone with no internet experience or anyone under 18 years of age.These choices were made because the authors wanted the data collected from participants to not be affected in any way with bias in relation to someone not having adequate IT skills.Also the focus of the study did not include minors.Furthermore the sample recruited had a mixture of male and female participants across the 18-59 age range.

C. Experimental Design
A between users design was used for this experiment, in which the 56 participants were randomly allocated to one of four conditions, the conditions were: Apart from the navigation being in a different position on the web site all of the web site versions had the same content and appearance on them and all the instructions given to the participants were identical irrespective of the condition under consideration.
The between users design was chosen so that possible learning effects would be removed as the tasks used were identical across all four conditions.Linked to this, strong attempts were made to recruit individuals with similar backgrounds in terms of experience and confidence (see previous section for details).The similarity on backgrounds helped to offset potential participant effects in the use of a between users design.

D. Variables
The independent variables were the four navigational web site menus positioned at the four different locations on the web site and the type of tasks which involved finding items/information on the prototype web site.
The dependent variables were the performance and the participants' subjective opinions.
The dependent measures were the task time, the number of errors (i.e.wrong click(s) on a navigation link and/or the wrong item(s) selected), the number of mouse clicks and participants' subjective opinions regarding various features of the web site and the navigation used.The perceptive opinions were elicited by means of a post-experiment questionnaire which covered detailed aspects of the design of the basic web site, the navigation bar used and aspects of the participants' feelings during the experiment.A Likert [9] type scale ranging from 1 to 7 was used for all the questions, where for all questions a 7 score was the highest possible positive score that could be allocated.

E. Apparatus and Materials
The following materials were used in the experiment:  A desktop PC running Windows 7, 2.00GB RAM and a 22" monitor,  The four prototype web sites which were identical in content and style with the exception of the actual aspect being investigated, i.e. the navigation bar positioning (see Experimental Design section above for the actual positions used in the experiment).
 A stopwatch to record the time taken  BB FlashBack [1] screen reader, used for recording the path the participant took in order to navigate to the items during the task, counting errors and mouse clicks.
 A recruitment questionnaire including a consent statement completed before the experiment.
 A post-experiment questionnaire eliciting perceptive opinions about the user interface and navigation.
 An observation protocol used in conjunction with the screen recording software.

 An instruction sheet providing the participants with information about what the experiment involved.
There were six tasks for the experiment.Each of these was designed to simulate a typical shopping type activity where a user may be looking for a few items with an intent to buy.The tasks were further designed to ensure that the participants had to use the navigation bar on the web site.

F. Procedure
In order to conduct the experiment the procedure described below was followed.www.ijacsa.thesai.orgParticipants were recruited by means of the recruitment questionnaire and only individuals meeting the criteria identified in the Users section above were asked to take part.
Once the 56 participants were recruited and had given their consent, they were then randomly allocated to one of the four experimental conditions.This clearly resulted in each condition having 14 participants.The conditions were as follows: The experiment was conducted with each participant individually.Each participant was treated with a welcoming manner.Before beginning the tasks, the participants were asked to read an information sheet.This contained information about confidentiality and use of the data collected.It also informed the participants about the use of a screen recorder and what it was being used for.Finally, participants were asked to click the 'purchase item' link when they had found the required item pertaining to a task.At this point, each participant was issued the first task and then on completion of the first task the next task was issued to the participants, until all six had been completed.Each task had the starting point at the home page which was set by experimenter.The decision of issuing one task at a time was taken so that participants would focus only on one task at a time and also not potentially be tempted to take a mental note of the other task items and look out for these as they navigated the web site.The web site was designed so as to have only one item that was the correct item for a task, e.g.there was only 1 e-book that was priced at £7.00 (Task 4), etc.
During each task, the screen reader recorded the activity on the screen, where the main aspects of interest were the errors made and the number of mouse clicks.The observation protocol was used by the experimenter to record the time taken from the stopwatch.The data (errors made and the number of mouse clicks) recorded by the screen reader were decoded after the participant had left the experiment location.
Once all the tasks were completed, each participant was asked to complete a post-experiment questionnaire (see Variables section above for a summary of the areas covered by the post-experiment questionnaire).
Finally, when the questionnaire was completed the participants were thanked for their time and participation.

G. Results
The data was initially explored by looking at the distributions and overall pattern (not included in this paper for brevity).The data was judged to be too non-normal for use with a parametric test.Therefore a Kruskal-Wallis [11] test was applied to all the performance data (task times, errors and mouse clicks) and perceptive opinions data.Where significant differences were identified by the test, these were followed up by Mann-Whitney U [11] tests for isolating the differences.
For task times, no significant differences were indicated by the Kruskal-Wallis test and will therefore not be discussed further in this section.
However, the Kruskal-Wallis test (please see Appendix A -Fig. 1 for the Mean Ranks for the errors discussed here) indicated that there was a significant difference in the number of errors made during Tasks 4 and 5 and this in turn significantly affected the total number of errors recorded: Task 4 Errors: H (3) = 11.34,p = 0.01, Task 5 Errors: H (3) = 10.64,p = 0.01, Total Errors: H (3) = 15.02,p = 0.002.The Mann-Whitney U tests suggest that across all the tasks the top navigation bar incurred overall fewer errors than the bottom navigation bar U = 27.50, n1 = 14, n2 = 14, p = 0.001, r = 0.61.Also the Mann-Whitney U tests suggest that the left navigation bar incurred significantly fewer errors for Tasks 4 and 5 compared to the bottom navigation bar: Task 4: U = 35.00,n1 = 14, n2 = 14, p = 0.002, r = 0.58.Task 5: U = 40.50,n1 = 14, n2 = 14, p = 0.004, r = 0.55.The differences in errors for the top and right navigation bars were not significant.
Regarding the number of mouse clicks recorded for each task under the four types of navigation bar, the Kruskal-Wallis test (please see Appendix A -Fig. 2 for the Mean Ranks for the mouse clicks discussed here) indicated that there was a significant difference in number of mouse clicks for Tasks 3, 4 and 5 and this in turn significantly affected the total number of mouse clicks recorded: Task 3 Mouse Clicks: H (3) = 9.28, p = 0.03, Task 4 Mouse Clicks: H (3) = 7.96, p = 0.05, Task 5 Mouse Clicks: H (3) = 10.87,p = 0.01.Total Mouse Clicks: H (3) = 11.79,p = 0.01.The Mann-Whitney U tests suggest that across all the tasks the top navigation bar incurred overall significantly fewer mouse clicks than the bottom navigation bar U = 37.00, n1 = 14, n2 = 14, p = 0.01, r = 0.53.Also the Mann-Whitney U tests suggest that the left navigation bar incurred significantly fewer mouse clicks for Task 5 and for the overall total mouse clicks, compared to the bottom navigation bar: Task 5: U = 41.50, n1 = 14, n2 = 14, p = 0.003, r = 0.56, Total Mouse Clicks: U = 39.50, n1 = 14, n2 = 14, p = 0.01, r = 0.51.
The data concerning perceptive opinions about the various navigation bar positions were also part of the Kruskal-Wallis test (please see Appendix A -Fig. 3 for the Mean Ranks for the perceptive opinions discussed here).The results of the test suggested significant differences concerning opinions about the ease of navigation, the ease of locating correct web pages and whether the current position of the navigation bar was well liked: Ease of Navigation: H (3) = 8.20, p = 0.04, Ease of Locating Correct Web Pages: H (3) = 7.84, p = 0.05, Current Position of Navigation Bar Well Liked: H (3) = 32.35,p < 0.001.
Lastly the participants' feelings during the experiment were also analysed as part of the Kruskal-Wallis test and no significant differences were indicated and will therefore not be discussed further in this section.

IV. DISCUSSION AND CONCLUSIONS
The research carried out and described in this experiment, has helped to gain more understanding about which menu position may be better.As discussed above, in this experiment attempts were made to make some of the tasks more difficult than those in an earlier study done by one of the authors of this paper [12] and the experiment aimed to examine more menu positions.
Regarding the hypotheses devised, the positive hypothesis for hypotheses 1 is rejected, as there was no significant statistical difference in terms of times for the tasks and menu positions.However, for hypotheses 2, 3 and 4, the positive hypotheses are partially accepted.These suggested that the left vertical menu would incur fewer errors, fewer mouse clicks and more user satisfaction.However, while this is shown in the data analysed, the data also shows that the top horizontal menu did not perform any worse than the left vertical menu for errors, mouse clicks and user satisfaction.
Therefore overall, the results of this experiment suggest that menus placed at the top of a page horizontally or vertically at the left of a page seem to elicit better performance in users.This is in terms of errors and mouse clicks.No statistically significant differences in terms of task times were found.Furthermore, for the subjective opinions elicited from the participants, the statistically significant indicators were in line with the performance values.
The worst performing menu was the one placed at the bottom of the page horizontally.This should alert web designers that potentially the trend of recent years to place many items on a 'menu-type' configuration at the bottom of a web page is perhaps not the best place to position these options.
The authors would suggest that using menus placed at the top of a page horizontally or vertically at the left of a page would also go some way to helping in having web pages that are universally designed.Concerning the universal design principles of 'Simple and Intuitive Use' and 'Tolerance for Error' [15], the data suggests that using one of these two menu options would be better.The sample of participants had never seen our test web site and therefore it is suggested that the menus placed at the top of a page horizontally or vertically at the left of a page are simpler and more intuitive to use.The data collected for errors, mouse clicks and user satisfaction suggests this.Further this is linked with the aspect of being error tolerant.The significant differences in errors between the different menu positions suggest that menus placed at the top of a page horizontally or vertically at the left of a page would incur fewer errors.
Although it is felt that this study helps to increase our knowledge regarding menu and navigation design, more work could still be done to make things more clear.Some examples include investigating further other menu designs and other menu configurations.The authors hope to be able to engage in further studies around these contexts.www.ijacsa.thesai.org The navigation bar on the left hand side of the web site  The navigation bar on the right side of the web site  The navigation bar at the top of the web site  The navigation bar at the bottom of the web site

The tasks were as follows:  Task 1 : Task 2 : 3 , Task 3 : Task 4 : Task 5 : Task 6 :
Find the child's book Giant Land, Find the head phones accessories for PlayStation Find the Music DVD Lady Gaga, Find an E-Book that costs 7.00 GBP, Find the film that does not have an image, Find the item that has the wrong title.
Left hand navigation bar,  Right hand navigation bar,  Top navigation bar,  Bottom navigation bar.