Definition of Tactile Interactions for a Multi-criteria Selection in a Virtual World

—Tablets, smartphones are becoming increasingly common and interfaces are predominantly tactile and often multi-touch. More and more schools are testing them with their pupils in the hope of bringing pedagogic benefits. With this new type of devices, new interactions become possible. A lot of studies have been done on the manipulation of 3D objects with 2D input devices but we are just at the beginning of studies that made a link between needs of pedagogy and possibilities of these new types of interactions. FINGERS© is an application for learning spatial geometry. It's written for pupils from 9 to 12 years old. Interactions have been designed with teachers. Some interactions are specifics for 3D geometry (3 DOF translations, rotations, nets, combinations of cubes, etc) and someone are general like designation or multi-selection. A lot of grammars of gesture propose a set of interactions to select an object or a group of objects. Multi-taps or lasso around an area are commonly adopted interactions. Performing geometry exercises needs imaging another interactions. For example how selecting all Cubes, how selecting all green objects. The real question is how introduce a parameter in selection. After presenting the limits of current solutions, this communication presents the solutions developed in FINGERS©. We explain how they allow a " parameterized " selection.


INTRODUCTION
The commercial success of tablets requires researchers in human-computer interaction to imagine new ways to interact with these devices.Today, someone use it to individual way [19], in groups [12], as part of multi-display environments [8], and for fun and entertainment [20].These devices provide interaction techniques that are often intuitive and easy to use in 2D.However, manipulation of objects in 3D is still a challenge.Manipulations in 2D have been defined very simply (selection, moving, designation).Things become more complex when it's necessary to perform manipulations in space.In 3D environment, widgets [13] have been largely used to make 3D manipulation easier.Moscovich [14] had shown how to design touchscreen widgets that respond to a finger's contact area.Schmidt et al. [17] have presented an interface for 3D object manipulation in which standard transformation tools are replaced with transient 3D widgets invoked by sketching context-dependent strokes.Designate an object in 2D or 3D is relatively simple.A user would naturally choose a direct touch on an object.Select a group of objects can be done either by increasing touches or by drawing an area around a group of objects (the lasso technic or selection box).Both approaches seem to meet all the needs and few works propose solutions for more complex designations.For example, what grammar of gestures to select all objects with the same geometric shape?What grammar of gestures to select objects with the same color?This paper proposes to explore innovative and intuitive solutions to provide simple solutions to a problem that can be complex.
The paper proposes to use the characteristics of objects to answer these questions.An object has different parameters as a geometric shape, color and others.By identifying designated objects, it's possible to identify common features and deduct User's intention.

II. RELATED WORKS
Multi-touch surface computing provides a set for interactions that are closer analogues to physical interactions of windowed interfaces.Building natural and intuitive gestures is sometime a difficult problem.Sometimes the gesture is not natural.How to define a gesture to move an object in virtual world along 3 directions with only one hand?[2] or twohanded [12].When you have only one hand to point, to move and to turn an object, your possibilities to interact with is entity are poor.In 2009 [14] Moscovich shows how to design touchscreen widgets that respond to a finger's contact area and gives limitations on the design of interactions based on sliding Widgets.Recreating new interactions (grammar of gestures) became a necessity.In 2008 [17] Schmidt and Al.present an interface for 3D object manipulation in which standard transformation tools are replaced with transient 3D widgets invoked by sketching context dependent strokes.The majority of works try to define a quantitative and qualitative surface gesture used by users.Understanding users' mental model will improve a better knowledge of relationships between technology and users.In 2002 Poggi [16] build a four dimensions topology, where gestures differ.These four categories are: mapping of meaning, semantic content, spontaneity and relation to speech.From an analyse of people collaborating around a drawing table, Tang [18] observe that gestures appear as an element of simulation for operations, referring to an area of interest in connection with users.Some studies focus more specifically for handling objects in a workspace in 3D.Gestures are more complex and less intuitive.It is common to incorporate the users to define the input systems and mainly grammars gestures.Cohé and Hachet www.ijacsa.thesai.org[6] conducted a user study to better understand how nontechnical users interact with a 3D object from touch-screen inputs.The experiment has been conducted while users manipulated a 3D cube with three points of view for rotations, scaling and translations (RST).Their study shows a wide disparity for gestures suggested by users.Figure 1 illustrates this disparity for rotation.The most suggested gesture by users embodies only 17.9% of proposals.The sixth gesture gets even 8.6% of suggestions.In 1986, Bier [4] introduces two classes (two widgets): "anchors" and "end condition" to precise placement of shapes relative to each other.Since this first definition and tactile interface building, widgets used in 3D manipulation are in rapid succession.When you use your finger to point, to move and to turn, your possibilities to interact with an entity are reduced.Some studies explore multi-touch controls to manipulate several degrees-of-freedom at the same time.Hanckock and al. [9][10] proposed to use from one to three fingers to handle objects in shallow depth.Martinet, Casiez and Grisoni [12] explored the design of free 3D positioning techniques for multi-touch displays to exploit the additional degrees of freedom provided by multi-touch technology.Theirs contributions are twofold: in a first time an interaction technique to extend the standard four viewports technique found in commercial CAD applications by adding a teleportation system, and in a second they introduce a technique designed to allow free 3D positioning with a single view of the scene: The Z-technique (Fig 2).

Fig. 2. Z-technique and multi-touch viewport technique
From a short preliminary study, Cohé and al. [5], show that selection of the DOF controls is difficult as soon as the graphical elements project close to each other on the screen.They note that it is difficult to control all the DOF when they are displayed at the same time.They propose an alternative approach and built a tBox controlled with a finger (Figure 3).User-centered design is a way of designing humancomputer interfaces.But you have a gap between users and designers.Users behavior are often complex to develop and often inefficient for design.Foley et al. [7] observe that a usercomputer dialogue is at the beginning of all languages of inputs and outputs.As in speech recognition, a feedback is inevitable to developing an exchange between two entities (humans or not).
In its work on user-defined gesture, Wobbrock [21] trying to control this feedback to prevent revision by user of his of mental model.They perform gestures on tactile table (Microsoft table).Participants performing a gesture to pan a field of objects after a learning animation.The initial hypothesis is that any action or command cannot be performed by a gesture."So what is the right number of gestures to employ?"He developed au field experience with 20 participants.They presented them, like Cohé and Hachet [6], a set of 27 commands and they asked then to imagine corresponding gesture.From this analysis we extracted two specific items related to the selection of objects (Line 3 and 12).The 27 participants consider designating an object is a simple action.Make a multiple selection is already considered more complex.In addition, the multiple-selection was basic.Participants were just designate several patterns; selection was not constrained by specific characteristics such as colors or shapes.

III. CRITIQUE OF ACTUAL SOLUTIONS
With a small quantity of objects or in specific situations the problem of multi-selections is always simple.Gesture created to solve this problem are easy to understand, simple to realize and very efficient.The two main options using a lasso or a designation by multi-touch (Figure 5).These gestures are often presented as efficient and affordant solutions but they are not fitted when the complexity increases.
They does not answer to the following questions:  How selecting all the squares / cubes / circle? How selecting all red objects? How selecting all wired objects?More generaly, how selecting objects by providing parameters like form, color or representation.According to teachers of primary school, FINGERS© propose different categories of selection.After presenting FINGERS© and its functionalities we describe the solutions adopted for the selections of objects.
IV. FINGERS APPLICATION FINGERS© (Find INteractions for Geometry learNERS) is an application on tactile tablets that help young students to learn geometry in 3D space [1][2] [3].Study is restricted to mobile devices like iPad (This tablet is present in a large number of schools in France).The main goal is to manipulate a solid accurately even if one.Moreover the scene can contained several mathematical objects like Cube, Sphere, Pyramid and parallelogram.To permit the largest possibilities of manipulation, each solid had to be independently manipulated as well as the entire scene.To test potential pedagogic benefits of our set of interactions, a prototype is implemented with different functionalities:

A. Creation and suppression of solids
Creating or deleting a solid, FINGERS use tangible solids or eraser (Fig 6).Putting a solid on the screen incorporate a virtual solid under the tangible object.Putting a tangible eraser, like a rubber, on a virtual solid delete it.You can also delete an object by sliding to the edge of the tablet.Fig. 6.A tangible cube used to create a virtual cube in the prototype.

B. Selection
A long press on a solid makes its reference system appear (Fig. 8a).One more time and selectable vertices appear (Fig. 9).Another long touch and you draw edges (Fig 8b).Another long touch and wire mode appears.The selection system is cyclic (Fig 8c).

C. Translation
FINGERS© permits to translate an objet along screen plan in indirect mode (a touch out of object) if the solid is selected, or direct mode (a touch on objet and move along screen).If you add a second touch (1 touch + 1 touch different from 2 synchronized contacts) you have possibility to control object along z-axis (the gesture is similar to zoom).
The initial distance between the two fingers is recorded as reference distance.Depth translation is performed by the variation of the distance between the two contacts.If the two contacts move with a constant distance the solid moves along the screen plane.Moving the two contacts with a variation of the distance between them in the same time permits to manage 3DOF.www.ijacsa.thesai.org

D. Duplication
Duplication is managed by a three-way interaction contacts.You just put one direct contact to designate the solid and two indirect contacts to indicate the position where the duplicated solid appears.If the two indirect contacts begin closed to the solid, the duplication is restricted along the -x or x-axes, the -y or y-axes.You have possibility using one touch on object and two contacts anywhere on screen plan for positioning a free duplication.

E. Manipulation of a net of polyhedron
To generate and manipulate a net of polyhedron the solid need to be selected.Two hands are using for interaction with four indirect contacts.By moving two fingers by hand in an opposite direction (Fig. 7) we fold/unfold a net of the polyhedron.When a net is open, a double tap places the net on the screen plane to be modified.A direct contact on a face permits to move it.Pupils can check the validity of the new resulting net by folding it.Also they can experiment different possibilities and visually check them.A net of polyhedron can be manipulated in the same way than a solid.

F. Rotation
All rotation interactions from our set are indirect.Rotations are mapped with two fingers interactions.A solid must be selected to be rotated.Rotations-interactions are categorized in three classifications depending on the reference system.The three reference systems employed are the screen frame, the object frame centered on the object and the scene frame (Fig. Rotations in the reference system of the screen use two fingers to rotate the object around the axes.According to Nacenta [15] a magnitude filtering technique is used to minimize non-wanted rotations.Rotations in the reference system of the object are constrained by a defined axis.When a solid is selected its reference system appears (Fig. 8a).A touch on the sphere or the cone of each axis selects it.The solid can only rotate around the selected axis.Rotations in the reference system of the scene are all the others rotations where axis is defined by two vertices of the solid.The main problem is to define the axis of rotation.To solve the problem two states of selection are introduced.Our selection system is cyclic.A onesecond long press on a solid makes its reference system appear.One more second makes selectable vertices appear.A rotation axis is defined by selecting two vertices (Fig. 9).A two fingers slide rotates the solid.

G. Changing position of observer
We went in schools to observe 3D geometry lessons.Pupils turned around a real model of the exercise to verify their results.This interaction was so intuitive that this functionality is our solution to turn around the scene.www.ijacsa.thesai.orgA one-second long press with one finger on each side of the tablet begins or stops moving the observer.A new background color gives a visual feedback.The gyroscopic sensor is used to modify observer's position around the scene.It's the video camera metaphor.The tablet acts as a window onto the scene and moving the tablet in space changes the viewpoint into the scene (Fig. 10).In addition to conventional interactions, some functions of selections are incorporate into FINGERS© for a single object or group of objects.The problem was to build a simple grammar of gesture to select objects with different criteria.

V. PROPOSITIONS FOR MULTI-SELECTION
Figure 11 shows a simple example of problematic.Imagine that user wish selecting all green cubes or all parallelograms of different colors (green, red, yellow and blue).Using Lasso technic or selecting one by one objects is difficult indeed impossible.It's possible also argue by subtraction.How to select all objects except the pyramids or all green cubes except the one in bottom left corner?In FINGERS© application, a long touch is used to select an object.Simple tap is not used like a principal action on an object.The easiest way was using these interactions to build an action of multiple selections.The consistency is maintained with the other actions, the interaction is easy to remember and very efficient.According to Kammer [11] we define the syntax of our grammar gesture that is an extended Backus Naur Form.A small proportion of this grammar used for selection is:

A. Selecting all same objects
For example, to select all blue cubes you perform a long touch on initial object and a tap on destination object.Initial and final objects have same form, same color, same representation (plain or wire) and different position in space.FINGERS© understands that selection must be realize for all green cubes and select them with only one gesture (Fig 13).

B. Selecting all same forms
To select objects with the same shape (not necessarily the same color) you must apply the same interaction of selection on the two objects (Long touch on one and a tap on other one).Figure 14 shows for example how to select all colored cubes.

C. Selecting all same colors
Similarly it's possible selecting all objects with the same color (whatever their form or modeling).Figure 15 shows the selection of red objects.

D. Selecting all objects
There is still a function that is the selection of all objects.When the action involves two unrelated objects, interaction must be applied for all objects of 3D space.A long touch on object and tap on another one (having no common features) will select all elements (Figure 16).From a selection mode, an object has two states.In one hand a state "selected" in other hand an "unselected" state (it's a binary state).
(blue cube) +  (blue cube): It's interaction to do to select all objects.This final state is represented figure 17 by a symbolic red cross on all green cubes.A long touch on empty space unselecting all objects (Figure 19).With the development of tactile devices, software designer has imagined interactions to perform complex tasks.Sometimes, actions deemed simple have not been a particular attention "cuius rei demonstrationem mirabilem sane detexi.Hanc Marginis exiguitas not caperet" (Fermat's last theorem).The problem of selecting multiple items is a good example.This paper shows that the problem may be more complex and the solutions developed are insufficient.Our proposition is a simple and effective approach based on two common gestures a long touch and a tap.The proposed actions are intuitive.A user can select groups of objects by varying different criteria.Our selection mode allows designation with three different criteria (shape, representation, color).Without common criteria, our gesture will be interpreted as a selection of all the elements.It's possible to imagine other criteria such as position, orientation or size by changing only a small part of interaction.This solution is integrated soon in our FINGERS© and tested in actual software.

Fig. 1 .
Fig. 1.Categorization of gesture for rotation around X axis

Fig. 9 .
Fig. 9. Axis of rotation defined by two vertices

Fig. 10 .
Fig. 10.Gyroscopic sensor and video camera metaphor to change the position of observer

Fig. 11 .
Fig. 11.Example of situation containing multiple objects in FINGERS© The generalization of problem is how to introduce a parameter like number, color, form, representation indeed position in a tactile selection query.


 for a long touch   for a tap  + Two gestures performed in a asynchronous manner  * Two gestures performed in a synchronous manner An action of multiple selections can be written: Multi::=(Initial object) +  (Destination object).

Fig. 15 .
Fig. 15.Gesture to select all red objects

Fig. 16 .
Fig. 16.Gesture to select all objects E. Unselect one object from selection Sometime it's necessary to realize an incomplete selection.For example, if user need to delete all green cubes except the cube localize at the left-down corner.Intuitive step is:  Selection all objects  Unselect one cube (left-down corner)  Delete all selected cubes.