Computer Science HCI and UI – Assignment

CHAPTER

•· The wheel is an extension of the foot, the book is an extension

of the eye, clothing, an ex tension of the skin, electric circuitry '' an extens ion of the centra l nervous system.

CHAPTER OUTLINE 10.1 Introduction

10.2 Keyboards and Keypads

10.3 Pointing Devices

10.4 Displays

Marshall McLuhan and Quentin Fiore

The Medium Is the Message, 1967

337

338 Chapter 10 Devices

10.1 Introduction

Input and output devices represent the physical medium through which users oper­ ate computers. Along with improvements in computer processor speeds and storage capabilities in the past 50 years, their physical form factor and basic functionality have also changed dramatically. Only two decades ago, the standard computer plat­ form was the desktop or laptop personal computer equipped with a screen, a mouse, and a keyboard, but mobile devices have revolutionized the face of computing to the point that many people do not realize that their ever-present smartphones, tablets, or portable MP3 players are, indeed, powerful computers. Computing has reached a point where it is deeply woven into the very fabric of our everyday existence (Dourish and Bell, 2011). With easily more than 5 billion mobile devices in existence, more than 25% of them "smart" and able to access the internet, compared to some 800 million personal computers, it is clear that mobile computing is the universal computing platform for the world (Baudisch and Holz, 2010). What's more, unlike the previous generation of personal computers, this new pervasiveness of mobile computing is no longer restricted to industrialized parts of the world but is quickly becoming an integral arld iI1tegrated aspec t of life even in rural, poor, and underde­ ve loped regions (Pew Research Center, 2014). In fact, many regional efforts, such as Data Wind's $35 Aakash tablet designed for the Indian market (Fig. 10.1) as well as the Uhuru Tab by Rig Communications Limited in Ghana, are making sigruficant inroads toward making advanced computing available to everyone.

The veritable explosion of new and exciting computing technology has increased the importa nce of interaction design so as to accommodate such a wide diversity of input and output modalities. To keep up with this rapid pace of change, successful designers are increasingly employiI1g micro-HCI and macro­ HCI theories to transcend the specific capabilities and characteristics of individual devices. Some of these theories involve micro -scale ideas on consistency, respon ­ siveness, discoverability, hierarchies, mformation archi tecture, and feedback as well as macro-scale ones on contex t, social setting s, emo tions, learning, and per­ sonalization. Refer to Chapter 3 for a discussion on micro-HC I and macro-HCI.

Despite increased complexity, devices also represent the part of computer tech­ nology that perhaps has the largest capacity for game-changmg irmovation for user interfaces. Indeed, the hype of such innovations often concerns the user-interface aspects of a new device. The Apple iPhone and iPad changed smartphones and tab­ let computing overnight when they were introduced, mainly on account of their

See also:

Chapter 7, Direct M anipulat ion and lmm ersive Environments

Chapter 8, Fluid Navigation

~ourc 1enl

FIGURE 10.1 Indian IT minister Kapil Sibal announcing the Aakash, a $35 tablet for the Indian market, in 2011.

10.1 Introduction 339

smooth and natural user experi­ ence. The Nintendo Wiimote and the Xbox Kinect introduced ges­ tural and full -body interaction, respecti vely, to the living rooms of millions of people around the world. Finally, the Oculus Rift and the Microsoft HoloLens are bring­ ing vir tual and augmented reality to life for everyone (see Chapter 7).

Given such diversity and scope, this chap ter gives a brief introduc­ tion to the most important fami­ lies of input and output devices. The chapter first reviews text

en try (Section 10.2), including keyboards and keypads as well as their layout, physical design, and accessibility adaptations. It also discusses text entry tech­ niques for mobile de vices . Pointing (Section 10.3) is another common interac ­ tion for interfaces, and the section below reviews ideas for ho,-v to make pointing mor e efficien t, more accurate, and more accessible. Section 10.4 presents both traditi onal as well as novel display technologies, focusing on particularities of large and small disp lays as well as wearable computing

FIGURE 10.2 Many baby monitors use video cameras to provide a real -time feed of the baby's activities on a remote display device. There are also some wearable baby monitors that incorporate advanced features such as a baby's heart rate, respiration patterns, and skin temperature, which parents can track using their smartphone.

340 Chapter 10 Devices

and shape -chan ging disp lays. Examples of possible solutions for users with disabilities are distributed throughout the chapter.

1 0.2 Keyboards and Keypads

Text entry is one of the most common input tasks, and the primary mode of text entry is still the keyboard (Fig. 10.3). Despite having received much criticism over the years, the keyboard is very successful and still represents the most effi­ cient text-entry mechanism. Billions of people use keyboards; although the rate for beginners is genera lly less than one keystroke per second and the rate for average office workers is five keystrokes per second (approximately 50 words per minute), some users achieve speeds of up to 15 keystrokes per second (approximately 150 words per minute). Contemporary keyboards generally per­ mit only one keypress at a time, although dual keypresses are used for capita ls (Shift plus a letter) and special functions (Ctrl or Alt plus a letter).

More rapid data ent ry can be accomplished if several keys can be pressed simul taneously (i.e., chording). An inspiration might be the piano keyboard, an impressi ve data-entry device that permit s severa l finger presses at once and is responsive to different pressures and durations. Similarly, chord keyboards use multiple keypresses that represent several characters or entire words. While this requires trainiI1g ai1d continued use, such chorded keyboards can allow for text entry outside the sta11dard office environment, such as one-han ded or eyes-free typing on small mobile devices or for wearable computing. In the courtroom,

•

-·

I

FIGURE 10.3 An Apple MacBook Air laptop with a QWERTY keyboard (left) showing the inverted T movement keys at the bottom right and function keys across the top. A multi-touch trackpad supports pointing. On the right, a detail photograph of a Lenovo laptop keyboard shows a pointing stick (also called a trackpoint) mounted between the G and H keys on the keyboard.

10.2 Keyboards and Keypads 341

such devices are called stenotypes and allow court reporters to rapidly enter the full text of spoken arguments at rates of up to 300 words per minute. Hoovever, this feat requires months of training and frequent use to retain the complex pat­ terns of chord presses.

10.2.1 Keyboard layouts The Smithsonian Institution's National Museum of American History in Washington , DC, has a remarkab le exhibit on the development of the type­ writ er. During the middle of the nineteenth century, hundreds of attemp ts were made to build typewriters, with a stunning ,,ariety of positions for the paper, mechanisms for producing characters, and layouts for the keys. By the 1870s, Christopher Latham Sholes's design was becoming dominant – it had a good mechanical design and a clever placement of the letters that slowed down the users enough that key jamming was infrequent. This so-called QWERTY layout puts frequently used letter pairs far apart, thereby increasing finger travel distances.

Sholes's success led to such widespread standardization that, more than a century later, almost all keyboards use the QWERTY layout or one of its varia­ tions deve loped for other languages. The development of electronic keyboards eliminated the mechanica l prob lems of typewriters and led many twentieth­ century inventors to propose alternative layouts to reduce finger travel dis­ tances. The Dvorak layout could increase the typing rate of expert typists from about 150 words per minute to more than 200 words per minute and even reduce errors. Its failure to gain acceptance is an interesting examp le of how even docu ­ mented improvements can be impossible to disseminate because the perceived benefit of change does not outwe igh the effort required to learn a new, nonstan­ dard interface.

A third keyboard layout of some interest is tl1e ABCDE style, which has the 26 letters of the English alphabet laid out in alphabetical order. The rationale here is that non-typists will find it easier to locate the keys. A few data-entry terminals for numeric and alphabetic codes still use this style, though studies have shown no advan tage for the ABCDE style; users with little QWERTY expe­ rience are eager to acquire this expertise and often resent having to use the ABCDE layout.

Number pads are a further sour ce of controversy. Telephones have the 1- 2- 3 keys on the top row, but calcula tors place the 7-8- 9 keys on the top row. Studies have shown a slight advantage for the telephone layout, but most computer keyboards use the calculator layout.

Some researchers have recognized tha t the wrist and hand placement required for standard keyboards is awkward and have proposed more ergo­ nomic keyboards. Various geometries have been tried with split and tilted keyboards, but empirical verification of benefits in typing speed, accuracy, or reduced repetitive stra in injury is elusive.

342 Chapter 10 Devices

10.2.2 Accessible text entry While people with motor impairments often can still use regular keyboards, albeit very slowly, several approaches to aid such users exist. Early solu tions were based on large menus of fixed choices, but methods currently used in practice include adaptive keyboards, where keys are lowered instead of raised to aid acquisition, as well as on-screen keyboards accessed using alternative input devices like head pointers or oversized trackbal ls. All such text-entry methods can be improved sig­ nificantly by incorporating dictionary-based auto-completion as well as automatic error correction (Kane et al., 2008). In contrast, visua lly impaired users represent a particular challenge for text entry. Perklnput (Azenkot et al., 2012) and BrailleTouch (Southern et al., 2012) both provide nonvisual input methods for one-handed or two-handed Braille typing on multi-touch smartphone displays.

Some technique s go beyond the traditional keyboard. Dasher predicts proba­ ble characters and words as users make their selections in a continuous 2-D stream of choices and has been adapted to brain-computer interfaces (BCI), where people use their brain alone to input text (Wills and MacKay, 2006). Also, orbiTouch's Keyless Keyboard replaces the keys with two inver ted bowls, on top of which the user's hands rest comfortably (Fig. 10.4). A combination of small hand movements and small finger presses on the two bowls selects letters or con­ trols the cursor. No finger or wrist movement is needed, which might be I-1elpful to users with carpal tunnel syndrome or arthritis.

Finally, yet another approach reconsiders the use of a keyboard entirely. One idea is to rely on pointing devices such as mice, touchpads, or eye-trackers for data entry. Another builds on wearab le devices, such as a wris tband or ring form factor, to e11ter text (Ye et al., 2014). Common among many accessible text­ entry methods, particular ly for mobile settings, is the increasing use of speech input for this purpose (Chapter 9).

FIGURE 10.4

orbiTouch Keyless Keyboard with integrated mouse functionality (http://orbi tou ch .org /). The orbiTouch requires sma ll finger presses and no actual hand motion to operate yet supports high-performance typing and point ing.

10.2 Keyboards and Keypads 343

10.2.3 Keys Keyboards keys have been refined carefully and tested thorot1ghJy in research lab­ oratories and the marketplace. The keys tend to have slightly concave surfaces for good contact with fingertips and a matte finish to reduce both reflective glare and the chance of finger slips. Keypresses require a 40- to 125-gram force and a dis­ placement of 1 to 4 millimeters, which enables rapid typing with low error rates while providing sujtable feedback to users. An important element in key design is the profile of force displacement. When the key has been depressed far enough to send a signal, the key gives way and emits a very light click. llis tactile and audi­ ble feedback is extremely important in touch typing; hence, membrane keyboards that use a nonmoving surface are difficult to use for extensive touch typing. How­ ever, such keyboards are durable and therefore acceptable for challenging environ­ ments such as fast-food restaurants, factory floors, or amusement parks.

Certain keys, such as the space bar, Enter key, Shift key, or Ctrl key, should be larger than others to allow easy, reliable access. Other keys, such as Caps Lock and Num Lock, should have a clear indication of their state, such as by physical locking in a lowered position or by an embedded light. Large-print keyboards are available for vision-impaired users. The placement of the cursor-movement keys (up, down, left, and right) is important in facilitating rapid and error-free use. The popular and compact inverted-T arrangement of arrow keys (Fig. 10.3) allows users to place their middle three fingers in a way that reduces hand and finger movement. The cross arrangement is a good choice for novice users. Some large keyboards reuse the peripheral eight keys on the numerical keypad (all keys except the central 5 key) to simplify diagonal movements . For such key­ boards, the Num Lock key is used to toggle between keypad and arrow mode. In some applications, such as games, where users spend hours using the move­ ment keys, designers reassign letter keys as cursor-movement keys to minimize finger motion between the movement keys and other action keys. The WASD keys are often used for this pt1rpose . Finally, the au to-repeat feature, where rep­ etition occurs automatically with continued depression, may improve perfor­ mance, but control of the repetition rate must be provided to accommodate user preferences (thi s is particularly impo rtan t for very young users, older adult users, and users with motor impairm ents).

10.2.4 Mobile text entry As computers morph into new form factors – such as tables , tablet s, and phones-as well as become universally usable for a broader population of users from different backgrounds, nationalities, and capabilities, text entry is also changing beyond the traditional keyboard. Most older or low-cost mobile devices provide only a numeric keypad. Entering text using keypads requires multiple taps, where users hit a number key multiple times to cycle through

344 Chapter 10 Devices

several letters assigned to that key. Using the same key for consecutive letters requires the user to pause between letters. Predictive techniques, such as T9® by Tegic Communications, use dictionary-based disambiguation to speed up text entry and are often preferred for writing longer texts. Similarly, LetterWise uses the probabilities of prefixes and facilitate s the entry of non-dictionary words, such as a proper nouns, abbreviations, or slang. After training, users were able to type 20 words per minute with LetterWise compared with 15 words per min­ ute wi th multi-tap (MacKenzie et al., 2001).

While the current generation of sma rtphones, heralded by the initial release of the Apple iPhone (Fig. 10.6) in 2007, tends to eschew physical keyboards in favor of soft keyboards, some still use a traditional QWERTY keyboard. Fig. 10.5 shows two such mobile devices. Physical keyboards are still preferred by many mobile users who need to enter large amoU11ts of text usiI1g tl1eir phones, such as to manage e-mail on the go . With practice, users can reach speeds of 60 words per minute when using both thumbs with those mechanical keyboards or more when the device auto-corrects "off-by-one" erro rs where the user accidentally presses a key adjacent to the one intended (Clawson et al., 2008).

Nevertheless, the proliferation of touchscreen technology means that physi ­ cal mobile keyboards are increasingly being replaced by virtual, or so-called "sof t," keyboards, where the keyboard is merely a visual representation on the tou chscree n (Dunlop and Masters, 2008). Projection keyboards, where the physical world is appropriated (Harrison, 2010) to disp lay an image of the key­ board, are based on the same principle. The benefit of soft keyboards is that they can be dynamically relabeled, such as for a new charac ter set or layout, as well as rescaled and rotat ed to fit the physical display and device orientation

~ ··~~– – – · . • ! ! • . I ' ~ 'm ID m I

#

.f " 8 -.. • • l tj • 12) fJ lfl ID 0

, ' ~

+I I •• * 5 " ~ st "

, ..• , 2 3 4 5 • 7 ' a · 9 0 a., 0 ' IV . E • T y " u 0 p • ., A . 5 D F c; H J K " L , … • ~ .. • – z •. X" C · V , -~· 8 – N • .. ~ •

FIGURE 10.5 A Blackberry 010 (http://www.b lackberry.com) shown here on the left with a small physical QWERTY keyboard; users typically type with one finger or with both thumbs. On the right, a larger keyboard uses the longer dimension of a LG Cosmos 2 device and can be sl id back into the device when not needed ( http://www. LG .com/).

10.2 Keyboards and Keypads 345

oogsitter'?

– tter? P.9.g§t …… .

Hey!

Are you available

t

q w e r

F GURE 10.6 Soft keyboard on an Apple iPhone with the Shift technique (Voge l and Baudisch, 2007). Shift displaces a pressed or hovered key to a position above the user's finger to reduce t he so-called "fat finger" prob lem, whe re the finger occludes the touch target. The iPhone keyboard also increases precision by allowing repositioning and then activates on lift-off .

(Fig. 10.6). However, because soft keyboards lack the tangible and tactile feedback of a physical keyboard, they are difficult to use for eyes-free operation and typi­ cally yield only modest performance, around 20 to 30 words per minute. Still, one study demonstrated that providing tactile feedback using the phone's vibration motor could improve typing speed (Hoggan et aL, 2008), and another study found that expert typists can reach an average of 59 wpm and 90% key accuracy withou t even seeing a visual representation of the keyboard (Findlater et al., 2011).

Several methods exist to improve text entry on touchscreens. Just like keypad ­ based text-entry methods can make use of dictionary-based or predictive text-entry algorithms, current touchscreen text-entry methods commonly sugg est possible word completions given an input string. More advanced techniques use language models to predict the word the user is trying to write based on the current sentence; such language models are incorporated in new Apple iOS and Android mobile operating systems. Similar ly, Swype and ShapeWriter (Zhai and Kristensson, 2003)

346 Chapter 10 Devices

enable typing by tracing letters using a single touch gesture v.rithout the need to lift the user's finger, resolving conflicts using a language model. Finally, while origi­ nally designed for pen-based interfaces, the Shift technique has been adopted to mitigate the "fat finger" problem in text entry, for example, by the Apple iPhone, where the user's own finger occludes the key being pressed (Fig. 10.6).

Another text entry method is simply to write by hand on a touch-sensitive surface, typically with a stylus, but character recognition remains error-prone. Contextual clues and stroke speed plus direction can enhance recognition rates, but successful gestural data-entry methods are based on simplified and more eas­ ily recognizable character sets, such as the unistrokes used by Graffiti® for Palm OS devices. Another promising method is to allow shorthand gesturing on a key­ board instead of tapping on a touchscreen keyboard, using shapes that match the tapping patterns. Lor1g-term studies confirm that it is possible to achieve good text-entry performance with this technique (Kristensson and Denby, 2009).

For some languages, such as Japanese or Chinese, handwriting recognition has the potential to dramatically increase the number of potential users. On the other hand, users with disabilities, older adults, and young children may not have the necessary fine -motor control to use such interfaces on tiny touch -sensi ­ tive surfaces. For them, innovations such as EdgeWrite (Wobbrock et al., 2003) might be helpful. EdgeWrite relies on the use of a physical border to frame the drawing area and uses a modified character set that can be recognized by iden­ tifying the series of corners being hit instead of the pattern of the pen stroke, resulting in higher accuracy for all users compared with Graffiti. The Edge Write character set has also been used successfully with trackballs or eye-trackers to address the needs of users with disabilities (Wobbrock et al., 2008).

Finally, the pro liferation of so-called smartwatches (Fig. 10.23) has led to new research efforts on making text entry practica l even on such devices. Of course, the challenge with smartwatches is that their effective display and input area is approximately on the order of 1 inch (about 2 to 3 on). Zoom Board (Oney et al., 2013) is one approach that uses iterative zooming to make impossibly tiny keys usable on a small display; other methods exist or are likely forthcoming.

10.3 Pointing Devices

The new generation of touch displays invites users to tap, drag, and pinch the images on the screen directly. Furthermore, with complex information displays such as those found in computer -assisted design tools, drawing tools, or air­ traffic-contro l systems, it is often convenient for the user to point at and select items. This direct-manipulation approach (see Chapter 7) is attractive because the users can avoid having to learn commands, reduce the chance of typographic errors on a keyboard, and keep their attention on the display. The resu lts are

10.3 Pointing Dev ices 347

often faster performance, fewer errors, easier learning, and higher satisfaction. Pointing devices are also important for small devices and large wall displays that make keyboard interaction less practical.

The diversity of tasks, the variety of devices, and the strategies for using them create a rich design space (Hinckley and Wigdor, 2011). There are many ways to categorize pointing devices, such as physical device attributes (rotation or linear movement), number degrees of freedom (horizontal, vertical, yaw, pitch, etc.), and positioning (relative or absolute). The description below focuses on tasks and degree of directness as organizing dimensions.

10.3. 1 Pointing tasks and control modes Pointing devices are useful for seven types of interaction tasks:

l. Select. Choosing from a set of items. This technique is used for traditional menu selection, tl1e identification of objects of interest, or marking an object in a slide deck.

2. Position. Choosing a point in a one-, two-, three -, or h ighe r-dimensiona l space. Positioning may be used to place shapes in a drawing, to place a new window, or to relocate a block of text in a figure.

3. Orient. Choose a direction in a two-, three-, or higher-dimensional space. The direction may rotate a symbol on the screen, indica te a direction of motion, or control the operation of a device, such as a robot arm.

4. Path. Define a series of positioning and orientation operations. The path may be realized as a curving line in a drawing program, a character to be recognized, or the instructions for a cloili-cutting or other type of machine.

5. Quantify. Specify a numeric value. The quantify task is usually a one­ dimensional selection of integer or real values to set parameters, such as the page number in a docwnent, the velocity of a vehicle, or ilie music playback volume.

6. Gesture. Perform an action by executing a pre-defined motion. Examples of gestures include dwel ling on an object to bring up a context menu, swiping to the left (or right) to turn a page forward (or backward), and pinching (or separating) your fingers to zoom out (or in).

7. Text. Enter, move, and edit text iI12-D space. The pointing device indicates ilie location of an insertion, deletion, or change; see Section 10.2 for details on text­ entry devices. More elaborate text tasks include centering, setting margins and font sizes, highlighting (boldface or underscore), and page layout.

vlhile all of iliese tasks can be performed using a keyboard by specifying directions, coordinates, and distances using a command language, this is an indirect and inefficient method that requires training. In ilie past, the keyboard '"'as used for au of these purposes, but now most users employ pointing devices to perform the tasks more rapidly and w iili fewer errors; expert users can

348 Chapter 10 Devi ces

furthe r impr ove performance by using keyboard short cuts for tasks that are invoked frequently (e.g., Ctrl-C followed by Ctrl-V to copy and pas te).

Pointing devices can be grouped into those that offer direct control on the screen surface, such as the touchscreen or sty lus, and those that offer indirect control away from the screen surf ace, such as the mou se, trackball, joys tick, graphics tablet, or touchpad. Within each category are many variations, and nove l designs emerge frequently (Box 10.1).

DOX 10.1 Pointing devices.

Direct control devices (easy to learn and use, but hand may obscure display)

• Touchscreen (sing le- and mul t i-touch)

• Stylus (passive and acti ve )

Indirect control devices (take time to learn)

• Mouse

• Trackball

• Joystick

• Point ing stick (trackpoint )

• Touchpad

• Graphics tablet

Novel devices and strategies (for spec ial purposes )

• Bimanual input

• Eye-trackers

• Sensors (accelerometer , gyroscopes, depth cameras )

• 3-D trackers

• Data g loves

• Haptic feedback

• Foot contro ls

• Tangible user interfaces

• Digital paper

Criteria for success

• Speed and accuracy

• Efficacy for task

• Learning time

• Cost and reliabi lity

• Size and weight

10.3 Pointing Dev ices 349

Another way to think about pointing de,,ices is whether they use absolute or relative input. Touchsc reens, graphics tablets, and eye-trackers also use an input model where the input (motor) space is directly mapped to the output (visual) space. This is called absolute input, since one point in motor space corresponds to one point in visua l space . Relative input, on the other hand, deals with transla­ tions (and rotations) from a current position and includes devices such as the mouse, joystick, and trackball. While this distinction is not used in the below discussio11, the reader may want to think about each device in absolute versus relative terms as well.

10.3.2 Direct-control pointing devices Touchscreens are the canonical direct control pointing devices and allow users to interact directly with the visual content of the screen by touching it with their fin­ gers. Because of their natural affordance, i.e. their form inviting appropriate action, touch-enabled screens are often integrated into applications directed at novice users in which the keyboard can be eliminated and touch is the main interface mechanism.

Early touchscreen imp lementations had problems wi th imprecise pointing, as the software accepted the touch immediately (the land-on strategy), denying users the opportunity to verify the correctness of the selected spot. These early designs were based on physical pressure, impact, or interruption of a grid of infrared beams. High-precision designs dramatically improved touchscreens. The resistive, capacitive, or surface-acoustic-wave hardware often provides up to 1600 x 1600 pixel resolution, and the so-called lift-off strategy enabled users to point at a single pixel. This lift-off strategy has three steps: Users touch the surface and then see a cursor that they can drag to adjust its position; when they are satisfied, they lift their finger off the display to activate.

High-preci sion touchscreens have transformed mobile devices (Section 10.3.6), such as tablets and phones, to the point that it has become natural for users to be able to directly point to objects on the mobile display. Combined with device miniaturization, where users are perpetually asking for mobile devices to become smaller, lighter, and more powerful, touch computing has led to current mobile devices consisting almost entirely of a touchscreen. In fact, researchers are inves­ tigating ways to increase the input and output surface of mobile devices, either by using the back of the device (Baudisch ai1d Chu, 2009) or by appropriating ilie surrounding physical world using projectors and input sensors (Harrison, 2010). However, pointing using the user's own fingers is prone to the aforementioned "fat finger" problem (Section 10.2.4), where the user's hand and fingers occlude on-screen content. New techniques such as Shift (Vogel and Baudisch, 2007) and occlusion -aware interfaces (Vogel and Balakrishnan, 2010) try to remedy this by displacing the screen content based on the user's touch interaction.

Another way to avoid the fat finger prob lem is to use a stylus, which has a familiar and comfo rtabl e feel for most users while sim ultan eously minimizing

350 Chapter 10 Devices

hand-screen occlusion. These advantages, however, must be balanced against the need to pick up and put down the stylus. Most stylus interfaces (also called "pen-based interfaces") are based on touchscreen technology; users can write with a stylus for more natural handwriting and increased motion control but can also use a finger for quick selection (Vogel and Baudisch, 2007). In fact, common capacitive touchscreens, which form the majority of today's tablets and smartphones, can be interacted with using a low-cost blunt-tipped stylt1s wi th a capaci tive tip. Results show that ever, such inexpensive stop -gap measures improve accuracy and performance for drawing and sketching tasks on standard touchscreens (Badam et al., 2014). However, using a stylus on a standard touch display may result in uninten­ tion al touch es if users rest their hands 011 the display; such a situa tion calls for palm rejection techniques tha t discard interaction resulting from the hand based on shape or on the timing of finger and stylus input. There is also risk of losing the stylus.

Beyond mobile devices, the availability of high-precision tou chscreens has opened the door to many professio11al applications in banking, medical, or military systems. Furthermore, because touchscreens can be made to be very robust, they are particularly appropriate for public-access kiosks and mobile appl icatio ns. Designers of public-access systems va lue touchscreens because there are no moving parts and durability in high-use environmen ts is good (the touchscreen is the only input device that has survi, ,ed at Walt Disney World ® theme parks). Strategies have been described to provide access to touchscreen systems, such as for information kiosks or voting sys­ tems for people who are visio n-impaired or blind, are hard of hearing or deaf, ha ve trouble reading or are unable to read at all, or ha ve physica l dis­ abilities (Vanderheiden et al., 2004). For kiosk designs, arm fatigue can be a problem, which can be addressed by tilting the screen and providing a sur­ face on which to rest the arm. On the other hand, kiosks are generally not used for extensive interactive sessions. In general, arm fatigue for mid-air or unsupported interaction can be measured using the Consumed Endu rance metric, which is based on a biomechanical model of the arm (Hincapie­ Ramo s et al., 2014).

10.3.3 Indirect-control pointing devices Indirect pointing devices separa te the input (motor) space from the output (dis­ play) space, thus minimizing hand fatigue, by providing a surface for the hand to rest as well as eliminating hand-screen occlusion, by keeping the spaces apart. However, they require the hand to locate the device and also demand more cog­ nitive processing and hand / eye coordination to bring the on-screen cursor to the desired target.

10.3 Pointing Dev ices 351

The 1nouse is the most common indirect pointing device and is appealing because of its low cost and wide availability. While using a mouse, the hand rests in a comfortable position, buttons on the mouse are easy to press, long motio11s can be done rapidly by moving the forearm, ai1d positioning can be done precisely with small finger movements. However, users must grab the mouse to begin work, desk space is consumed to operate it, and users must sep­ arate their attention between the motor and display space. Other problems are that pick-up and replace (also called clutching) actions are necessary for long motions and some practice is required to develop skills (usually from 5 to 50 minutes, but sometimes much more for older adults or users with disabilities). The variety in terms of mouse technologies (physical, optical, or acoustic), num­ ber of buttons, placement of the sensor, weight, and size indicates that designers and users have yet to set tle on one preferred design. Personal preferences and the variety of tasks to be done leave room for lively competition. The mouse may be simple or may incorporate a wheel and additional buttons to facilitate scrolling, web browsing, or spec ific applications (Fig. 10.7). Such addi tional mouse features can sometimes be programmed to perform common tasks of special-purpose applications, such as adjusting the focus of a microscope and switching its magnification level.

The trackball is controlled by spinning a ball along two axes and has sometimes been described as an up side-down mechanical mouse. It is usually implemented

——-~

FIGURE 10.7 The Apple Magic Mouse 2 wireless mouse on the left has on ly one button activated by pressing down on the whole mouse (http://www.apple.com). The Razer Ouroboros ® gaming mouse on the right has two standard buttons, a center mouse wheel that can also be clicked, and an additional set of nine buttons that can be programmed for specific gaming settings (http://www.razerzone.com/gaming-mice/ razer-ouroboros).

352 Chapter 10 Devices

FIGURE 10.8 The Logitech Trackman Wheel Optical is a popular trackball device (http://www .logitech.com/).

as a rotating ball, 1 to 15 centimeters in diameter, that moves a cursor on the screen as it is moved (Fig. 10.8). The trackball is wear-resistant and can be firmly mounted in a desk to allow users to hit the ball vigorously and to make it spin. Trackballs ha, 1e also been embedded in control panels for air-traffic-control or museum information sys tems, and they are commonly used iI1 video game controllers.

The joystick, whose long history began in aircraft-control devices and early computer games, has doze11s of versions with varying stick lengths and thick­ nesses, displacement forces and distances, anchoring stra tegies for bases, and placement relative to the keyboard and screen. Joysticks are appealing for track ­ ing purposes (to follow or guide an object on a screen), partly because of the relatively small displacements needed to move a cursor, the ease of direction changes, and the oppor tunit y to combine the joystick with additional buttons, wheels, and triggers (Fig. 10.9).

The directional pad (or 0 -pad) originated in game consoles and consists of four directional arrows arranged in a cross with a trigger button in the center. An example is the Wii remote contro l (left part of Fig. 10.10). This system is also used in mobile devices for navigation in menus. Beyond the Wii, curren t "eighth­ generation" video game consoles also use similar controllers that integrate both joysticks and 0 -pads; Fig. 10.10 (right) shows the Sony PlayStation 4 DualShock controller.

The pointing stick (or trackpoint) is a smalJ isometric joystick embedded in keyboa rds between the letters G, B, and H (Fig. 10.3). It is sensitive to pressure and does not move. It has a rubber tip to facilitate finger contact, and with modest practice, users can quickly and accurately use it to control the cursor while keeping their fingers on the keyboard home position. The pointing stick

10.3 Pointing Devices 353

FIGURE 10.9 A flight simulator control, combining a joystick (right) and a throttle (left) for two-handed operation.

l

FIGURE 10.10

… … … …

Wii

The left image shows a Nintendo Wii remote controller (Wiimote, right hand) with attached Nunchuck (left hand). The Wiimote includes a three-axis acce lerometer that detects movement in three dimensions . The right image shows a Sony Dua1Shock4 wireless controller for the PlayStation 4 video game console .

354 Chapter 10 Devices

is particularly effective for applications such as word processors that require constant switches between the keyboard and the pointing device. Because of their smal l size, pointing sticks can easily be combined with other devices such as keyboards or even mice to facilitate 2-D scrolling.

Touchpads offer the convenience and precision of a touchscreen while keeping the user's hand off the display surface. Users can make quick movements for lo11g-distance traversals and can gently rock their fingers for precise positioning before lifting off. Often embedded below tl1e keyboard, the touchpad can be used with the thumb s while keeping tl,e l,ands in typing position. Their lack of moving parts and thin profile make touchpads appealing for laptops . Further­ more, current touchpads often have multi -touch capability, allowing for up to five simultaneous touch es; windowing systems such as Apple OS X and Microsoft Windows use this capability for gestures, for example, for scrolling, panning, and zooming a document or graphical view.

The graphics tablet is a touch-sensitive surface separate from the screen, usu­ ally laid flat on the desk/tabl e or in the user 's lap. This separation again allows for comfortable hand positionit,g and keeps the users' hand s off the screen . The

FIGURE l 0. 11 A digital art ist using a Wacom ® 13HD Touch graphical tablet with a wireless stylus (http ://www.wacom.com /). The Wacom pressure -sensitive sty lus and graphics tablet allow t he precise pointing and accurate control that artists need .

10.3 Pointing Dev ices 355

graphics tablet is appea ling when users' hands can remain with the device for long periods without switching to a keyboard. For this reason, graphics tablets are often popular with digital artists who engage in drawing and sketching operations. Furthermore, the graphics tablet permits adding application options, such as pa lettes, tools, and brushes, beyond the screen itself to its surface, thereby preserving valua ble screen space and providing both guidai1ce to nov­ ice users as well as easy access to experts. Graphics tablets are typically operated using a finger, pencil, puck, or stylus through acoustic, electronic, or con tact position sensing. Artists tend to prefer wireless pens for high precision and free ­ dom (Fig. 10.11).

Among the above indirect pointing devices, the mouse has been the greatest success story. Given its rapid, high-precision pointing abilities and comfortable hand position, the modest training period is only a small impediment to its use. Most desktop computer systems offer a mouse, but the battle for the laptop continues, with many vendors offering multiple pointing devices on a single machine.

10.3.4 Comparison of pointing devices Early studies found that direct pointing devices such as a touchscreen were often the fastest but the least accurate devices. Decades of studies have consis­ tently shown the merits of the mouse over alternative devices for speed and accuracy. The pointing stick has been found to be slowe r than the mouse due to tremors during fine finger movements (Mithal and Douglas, 1996). Trackballs and touchpads fall somewhere in between. Users' tasks matter when compar­ ing devices. For example, when browsing the web, users are constantly involved in both scrolling and pointing. One study showed that a mouse with a finger wheel did not improve users' performance over a standard mouse. Future research might provide a better understanding of the benefits and limi­ tation s of each device.

Common wisdom states that pointing devices are faster than cursor­ movement keys for selecting objects, but this assertion depends on the task. When a few (2 to 10) targets are on the screen and the cursor can be made to jump from one target to the next, the cur sor keys can be faster than using point­ ing devices . For short distances and for tasks that mix typing and pointing, cursor keys have also been shown to be faster than and preferred to the mouse. However, many users never learn keyboard shortcuts (e.g., Ctrl-Z to undo), despi te the fact that menu selec tions can be performed much faster using those shortcuts than by using a pointing device (Grossman et al., 2007).

Users with motor disabilities often prefer joysticks and trackba lls over mice, as the location of such devices remains fixed, they have a small footprint (allowing them to be mounted on wheelchairs), and they can be operated by small residual movements. On the other hand, touch-sensi tive devices are

356 Chapter 10 Devices

useful when applying force is difficult-for instance, for users with motor disabilities-but designers should attempt to detect inadvertent or uncontrolled movements and smooth out trajectories. Using active target areas that are larger than the butto11 or icon to be selected is effective to sl1orte11 selection time and reduce frustration for every user and, in some cases, might be all that is needed to render an application usable by a much wider audience.

Pointing devices are extremely challenging for users who have vision impair­ ments. Well-designed cursors of adjustable size and shape may help users witl1 limited vision impairments, but indirect-control devices sucl1 as the mouse are simply not practical for users with severe vision impairments who have to rely on the keyboard. Alternative keyboard or keypad navigation options should be provided whe11ever possible. Toucl1screen interfaces can more easily be explored and memorized when speech syn thesis or sonification is available to describe the display, read menu options, and confirm selections. For example, in a touchscreen voting kiosk, users can use arrow keys to navigate through lists of candida tes whose names are read aloud via headphones (Fig. 10.12). Successful examples demonstrate that it is possible to design powerful systems that are truly accessible to the general public, including users with a wide ran ge of dis­ abilities (Vanderheiden et al., 2004). Finally, tactile graphics can be produced by using thermal paper expansio n machines and placed on top of touchscreens for use by blind users.

In summary, individua l differences and the user tasks are critica l when select­ ing a pointing device. The touchscreen and trackball are durable in public ­ access, shop-floor, and laboratory applications. The mouse, trackball, trackpoint, graphics tabl et, and touchpad are effective for pixel-level poin ting. Pens and styli are appreciated for drawing and handwriting, and simple gesh1res can be used to specify actions and quantify their parameters. Cursor jump keys remain attractive when there are a small number of targets. Joysticks are appealing for games or specialjzed navigation applications.

10.3.5 Fitts's Law One of the few scientific models of human-computer interaction is Paul Fitts's (1954) law of human hand movement. Often referred to simply as Fitts' s Law ( or even Fitts' Law), this micro-scale HCI theory allows designers to decide on the optimal locations and sizes of buttons and other elements when laying out screens as well as indicates wluch pointing devices are best suited to performing common tasks. Fitts noticed that the time required to complete hand movements was dependent on the distance users had to move, D, and the target size, W. Doubling the distance (say, from 10 cm to 20 cm) resulted in longer completion time s, but not twice as long. Increasing the target's size (say, from 1 cm2 to 2 cm2)

enabled users to point at it more rapidly.

FIGURE 10.12

PRESIDE NT ANO VlCE PRESIDENT OF THE UNITED STATES

Vote fOf OM

Al Goro, O Joo Utt»etrn,n

0.-,>I

Hatph Nad~r I Q WlftOn.a UOu"ko

Grf'f'" PJ,ty

G~rgo W Oi.lth I O o,c• c11<noy

R~tit~n

0 WUL~tn Blank

10.3 Pointing Dev ices 357

Users of this touchscreen voting tablet need only touch any text on the screen to have it read aloud, with the sound communicated to them via headphones. Touching the check box marks the vote, with verbal confirmation if headphones are used. Users who are completely blind or have severe physica l disabilities that prevent them from using the touchscreen (even with voice) can use a detachable keypad – with or without voice. The keypad also allows con nection of custom switches voters bring with t hem (http://www.trace.wisc.edu/).

Since the time to start and stop moving is constant, an effective equation for the movement time (MD for a givei1 device, such as a mouse, turns out to be

MT= a+ b l og2 (DIW + 1)

where a approximates the start/ stop time in seconds for a given device and b measures the inherent speed of the device. Both a and b need to be determined experimentally for each device. For example, if a were 300 milliseconds, b were

358 Chapter 10 Devices

…… 0

FIGURE 10. 13 A blind student uses a Touch Grap hics tactile map mounted on a touchscree n. On t he right, a dif ferent embossed overlay is used to learn the location of states and capitals, with a pen providing aud io desc rip t ions (htt p://www .touchgrap hi cs.com/).

200 msec/bit, D were 14 cm, and W were 2 cm, then the movement time MT would be 300 + 200 log2(14/2 + 1), which equals 900 milliseconds.

Several versions of Fitts's Law are used, but this equation has been demon­ strated to provide accurate predictio11s in a wide rai1ge of situations. Tl1e varia­ tions are due to differences such as the direction of motion (horizontal or vertical), device weight (heavier devices are harder to move), device grasp, shape of targe ts, and arm position (on a table or in the air). MacKenzie (2013) lucidly describes what Fitts's Law is, how it has been applied, and refinements for cases such as 2-D pointing. Studies of high -precision touchscreens have shown that in addition to the gross arm movement predicted by Fitts, there was also a fine-tuning mo tion of the fingers to move in on small targets such as sin­ gle pixels. A three-componen t equation was thus more suited for the precision­ pointing movement time (PPMT):

PPMT =a+ b 1092 (0/W + 1) + c 1092(0/W) .

The third term, time for fine tuning, increases as the targe t wid th, W, decreases. This extens ion to Fitts's Law is qui te understandable; it suggests tha t the precision-pointing movement time consists of the start/ stop time (a), a time for gross movement, and a time for fine adjustment. Other studies deal with a greater range of arm motion with pointing in 3-D space or witl1 two- thumb text entry. Alternative extensions (e.g., Chapuis and Dragicevic, 2011) focus on other aspects of the model.

Fitts's Law is well-established for adult users, but it may need refinements for special popula tions such as young childre11 or older adults. In one study, 13 four­ year-olds, 13 five-year-olds, and 13 young adults performed point-and-click selection tasks (Hourcade et al., 2004). As expected, age had a significant effect

10.3 Pointing Dev ices 359

.

Young adults

-·~ – · 5 year olds

.,

t'

4 year olds

,. ·- ' '•

FIGURE 10.14 Tracing the trajectory of the pointer during a repeated target -selection task il lustrates the dramatic difference between children's and young adults' use of the mouse (Hourcade et al., 2004).

on speed and accuracy (and of course trajectories, as shown in Fig. 10.14). A detailed analysis showed that Fitts's Law models children well for the first time they enter the target, but not for the time of final selection.

The open problem remains: how to design devices that produce smaller con­ stants for tlte predictive equation, in effect "beating" Fitts's Law (Balakrishnan, 2004). One study has shown that multi -scale pointing with zooming works best with two-handed input and a constant zoom speed (Guiard et al., 2001). Another stud y looked at crossing-based interfaces in which targ ets are merely crossed

360 Chapter 10 Devices

(e.g., as in how a finish line is crossed) instead of pointed at. The target-crossing completion time was found to be shorter than or equal to pointing performance under the same index of difficulty and depended on the type of task performed (Accot and Zhai, 2002). The quest for faster selection times continues.

10.3.6 Novel pointing devices The popularity of pointing devices and the quest for new ways to engage diverse users for diverse tasks have led to provocative innovations. Improving the match between the task and th e device and refining the input plus feedback strategies are common themes (Kortum, 2008).

Birnanual input – input using two hands simultaneously – can faci litate multi ­ tasking or compound tasks. In such settings, the nondominant hand sets a frame of reference in which the dominant hand operates in a more precise fashion. A natural application of bimanual operation for desktop applications is that the nondominant hand selects actions (for example, the Fill command of a paint program) while the dominant hand precisely selects the objects of the operation (see Chapter 8 for more on supporting navigation).

Since users' hands might be busy on the keyboard, designers have explored alternative methods for selection and pointing. Foot controls are popular with rock-music performers, organists, dentists, and car drivers, so maybe computer users could benefit from them as well. A foot mouse was tested and was found to take about twice as much time to use as a hand-operated mouse, but benefits in special applications may exist-for examp le, switches and pedals activated by foot migl1t be effective to specify modes.

Eye-trackers are gaze-detecting controllers that use video-camera image recog­ nition of the pupil position to give one- or two-degree accuracy (Figs. 5.3 and 5.4). Fixations of 200 to 600 milliseconds are used to make selections. Unfortunately, eye-tracking easily results in the "Midas touch" problem, since every gaze has the potential to activate an unintended command. Combining eye-tracking with mai1ual input is one way to address this problem (Stellmach and Dachselt, 2013), but for now, eye-tracking remains mostly a research and evaluation tool (Sec­ tion 5.3.1) and a possible aid for users with motor disabilities (Wobbrock et al., 2008).

Multipl e-degree-of-freedo111 devices can sense multiple dimensions of spatial position and orien tation. Control over 3-D objects seems a natural application, but comparisons with oth er strat egies reveal low precision and slow responses. Support for virtua l reality (Chapter 7) is one motivation, but many design, medical, and other tasks may require 3-D input or even six degrees of free­ dom to indicat e a position and an orientation. Commercial tracking devices include the Log itech company's 3Dconnexion ®, Ascension ®, Int ersense ®, and Polhemus ™.

Ubiquitous computing and tangible user interfaces (Dourish and Bell, 2011) depend on embedding sens ing technologies into the en vironment. For example,

10.3 Pointing Dev ices 361

active badges with Radio Frequency Identification (RFID) tags can trigger the preloading of personal files into a room's computer when users enter a room. The positioning of physical objects can specify modes or trigger actions. Ambient light, sourld, or airflow can be modified to present a small amou11t of informa­ tion to users. Entertainment and artistic applications use video cameras or body sensors to track body positions and create enticing user experiences. Early explorations by performance artist Vincent John Vincent led to 3-D environ­ ments for theatrical exploration such as Mandala, in which performers or amateur users touch images of harps, bells, drums, or cymbals, and the instruments respond. Myron Krueger's artificial realities contain friendly video­ projected cartoon-like creatures that playfully crawl on your arm or approach your outstretched hand. Such environments invite participation, and the serious research aspects fade as joyful exploratio11 takes over and you step inside the computer's world (Section 7.6). StoryRoom is another such application that enables children to actively construct their own interactive environments, using props and magic wands to crea te stori es that other children are invited to experience (Montemayor et al., 2004).

Even paper can be used as an input device. Early applications demonstrated the benefits of capturing annotations on large documents such as sketchbooks, blueprints, or Jab notebooks. Pens such as the LivescribeTM 3 sty lus (Fig. 10.15) with Anoto® functionality facilitate interaction, particularly in mobile situati ons . The pen has a small camera in its tip, and it records pen strokes drawn on a spe­ cial paper printed with a unique pattern that identifies the location of each stroke. The handwriting can then be transferred to a computer or a mobile phone. The

FIGURE 10.15

~ . . . . ••• .,., -if'!!!·· ~ ……… ,: …..

~ : ·-··•~•· .. "' . .. <-'""–. . • • . .

,.-. ….. –

The Livescribe 3 smartpen with Anoto techno logy records the strokes of ink written on augmented paper (paper with a unique dot pattern), and the data are transferred wirelessly to the tablet. A microphone allows users to record audio at the time an annotat ion is being made to be replayed later via the embedded speaker by tapping on the annotation (http://www .livescribe.com/) .

362 Chapter 10 Devices

ease of learning might help novice users: Paper Augmented Digital Documents (PADDs) can be edited both in digital and paper form (Liao et al., 2008), or trans­ lation s can be requested by writing words on paper (Fig. 10.15).

Mobile devices can also be used as input devices. For examp le, Carnegie Mellon University's Pebbles project (Myers, 2005) and University of Maryland's PolyChrome project (Badam and Elmqvist, 2014) explored how mobile devices can be used to communicate with personal computers, other mobile devices, large displays, home appliances, automobiles, or factory equipment. Mobile devices can act as intelligent universal remote controls, potentially empowering all users by reading aloud product information or menu options, translating instructions written in a foreign language, or offering speech recognition when needed. Finally, the images captured by cameras of mobile phones can be used as input to augmented-reality applications (Rohs and Oulasvirta, 2008) or everl to control general apps (Hansen et al., 2006).

Sensors mounted in the environment or added to handheld devices can enrich the interaction with the devices themselves. For example, accelerometers allow the Apple iPhone to detect changes in the device's orientation, causing the display to switch dynamical ly between portrait and landscape orienta ­ tions. Similar functiona lity exists in most digital cameras, allowing them to automatically determine whether a picture should be shown in landscape or portrait orientation . As users become familiar with gesture interaction, design­ ers may be able to find additional natural uses of movement information. For example, users may be able to zoom and pan on a map by adjusting the prox ­ imity or lateral position of mobile devices in fro nt of them. Tilting the device could scroll through a list of names, and bringing the device near the ear could answer an incoming call. The remote control of the Nintendo Wii video game console inc ludes a three-axis accelerometer that can detect movement in three dimensions and respond to gestures. For example, to hit a tennis ball, users swing the controller like a racket with a realistic arm motion. The Wii has inspired many app lications that require users to be more active and has successfu lly attracted more female and older adult users to video games (Fig. 10.10). Fina lly, commodity depth ca1neras, such as the Xbox Kinect, allow for cap turin g the motion of the user's entir e body using a combination of stan­ dard and infrared cameras. Recent work has explored how to use this capabil­ ity to reconstruct the user's hand pose for more refined touch computing (Murugappan et al., 2012).

Speciali zed hand sensors may be able to determine the exact posture and posi­ tion of each finger joint in a user's hand. The Leap Motion controller uses infra­ red cameras to track a user's hand at 200 frames per second in a volume above the sensor without the need for specia lized markers or even touching the sensor (Fig. 10.16). This enables detecting the hand position and posture at high accuracy. Similarly, the VPL DataGlove appeared in 1987 and attracted researcl1ers, game developers, cyberspace adventurers, and virtual reality devotees. Descendants

10.3 Pointing Dev ices 363

FIGURE 10.16 Th e Leap Motion contro ller tracks the user's hand in three dimensions using infrared cameras, precise ly reconstructing the posture of each f inger (http://www.leapmo t ion .com/).

of the original data glove have attached fiber-optic sensors to measure angles of finger joints (Fig. 7.13). The displayed feedback can show the relative placement of each finger; thus, commands such as a closed fist, open hand, index-finger pointing, and thumbs-up gesture can be recognized. Combined with a hand­ tracker, complete 3-D placement and orientation can be recorded. Devotees claim that the naturalness of gestures will enable use by many keyboard-averse or mouse-phobic users. But while the simple gestures of the Nintendo Wii games are easily learned because their number is small and they mimic the actions being simulated (e.g., the swing of a golf club or the rotation of a wheel), data glove and Leap Motion users require substantial training to master more than l1alf a doze11 gestures, especially if they do not map to 11aturally occurring gestures. Still, gestural input with the glove can make special applicatio11s possible, such as the recognition of American Sign Language or virtual musical performances.

Pointing devices with ha.ptic feedback are an intriguing research direction (Kortum, 2008). Several technologies have been employed to allow users to push a mouse or other device and to feel resistance (for example, as they cross a window boundary) or a hard wall (e.g., when navigating a maze). 3-D versions, such as SensAble Technology's PHANTOM, are still more intriguing, but commercial applications are slow to emerge . Because sound and vibratio11s are

364 Chapter 10 Devices

often a good subs titute for haptic feedback, the use of advanced haptic devices remains limited to special-purpose applications (such as training surgeons for heart surgery), while devices using simple vibrations have become mainstream in game controllers and even the common mouse.

Finally, a recent development in pointing has been the introduction of hybrid pointing techniques that combine several of the above techniques. This development goes hand in hand with the increasing prevalence of large as well as small displays, since pointing on such devices breaks from the stan­ dard; it is simply not practical to use a mouse or even your own finger touches on a large display, but this may require distant pointing. At the same time, such distant pointing using laser pointers or even the user's own gaze or finger is error-prone . New work that successfully integrates botl1 sma ll and large displays achieves this by combining direct and indirect pointing (Forlines et al., 2006; McCallum and Irani, 2009). Further advanced pointing techniques that use novel sensors will likely appear in the future, such as head or sho uld er movements, a light blow of air in a tube, the blink of an eye, and even faint myoelectric currents generated when muscles are being tensed. Such reality -based (Jacob et al., 2008) and "natural" (Wigdor and Wixon, 2011) interaction mechanisms are indicative of a broader trend of going beyond th e traditional mouse and keyboard for the new generation of comp uting devices.

10.4 Displays

The display is the primary source of visual feedback to users from the compu ter . It has many important characteristics, such as:

• Physical dimensions (usually the diagonal dimension and depth)

• Resolution (the number of pixels available)

• Number of available colors and color correctness

• Luminance, contrast, and glare

• Power consumption

• Refresh rates (sufficient to allow animation and video)

• Cost

• Reliability

Usage cha ra cteri s tics also distinguish display devices. Portability, pri­ vacy, saliency (need to attract attentio11), ubiquity (likelihood of being able to locate and use the display), and simultaneity (number of simultaneous

10.4 Disp lays 365

users) can be used to describe displays (Raghunath et a l., 2003). For example, mobile phones are perceived as personal devices and provide portable and private displays, whereas large displays allow social interactions between, for example, multiple users controlling characters in video games. Similarly, salient information displays found in malls or museums might offer store location information to a single user or an emotional theatrical experience to dozens of impressed visitors. Whiteboard displays allow collaborators to share information, brai11storm, and make decisions. Finally, immersive dis­ plays can transport a user into an imaginary world for recreation or to learn a new skill.

10.4. 1 Display technology The classic raster-scan cathode-ray tubes (CRTs) have now most ly vanished, replaced by liquid-cn;stal displays (LCDs) with their thin form, light weight, and low electricity consLLmption. Like LCDs, plasma displays have a flat profi le, but they consume more electricity. They are very bright and readable even from the side, making them valuable for mounted wall displays in control rooms, public displays, or conference rooms. Light-en1.itting diodes (LEDs) are now avai lable in many colors and are being used in large public displays. Matrices of miniature LEDs are also used in some head-mounted displays. Manufacturers are actively developing new displays using organic light­ emitting diodes (OLED). These durable organic displays are energy-efficient and can be laid on flexible plastic or metallic foil, leading to new opportunities for wearable or rollable displays.

New products attain paper-like resolution using electronic ink techno logy. They contain tiny capsules in which negatively charged black particles ai,d pos­ itively charged white particles can be selectively made visible. Because elec­ tronic ink displays use power only when the display content changes, they have an extended battery life over other types of displays and are well-suited for ebooks (e.g., Amazon's Kindle™, shown in Fig. 10.17, the Barnes & Noble Nook, or Bookee11's Cybook). Slow display rates allow some animatio11 but no video displays.

Tiny projectors, so-called pico projectors (Dachselt et al., 2012), are able to project color images on the wall from mobile devices and make collaboration using those devices more practical. Braille displays for blind users provide up to 80 cells, each displaying a character. A coup le of cells can be mounted on a mouse, and small displays can fit above the keyboard. Prototypes of refreshable graphic displays with up to severa l thousand pins are being developed. Manu­ facturers and government agencies are addressing health concerns relating to the different types of visual displays, such as visual fatigue, stress, and radiation exposure. Adverse effects seem for the most part attributab le to the overall work environment more than the visual display uruts themselves.

366 Chapter 10 Devices

FIGURE 10. 17

'Wy du, a.ct.• acl Mt "'""""· ……. e-• f • "11 t,,bo –~ ..,,._ ,ue.llo .. ia thl , … ar &""'l 1ot. : .-: afo.-.r ,:,,•..-"•' cl In -i.m.t>:, flr.!.Otl ,,_ 11>! ~ ..,.,. !ht ~ I, bl,,.r," '" 11,t-..

'1 dltu•ylll". bit II>•- ,la M,ho lholillllo)ol-fl,,d,..v o;,ut,b1b,o11: – tla-c,l,, l.eld,.,.. 1,·•-!J~•.J.W, .,_ Ko l,J ……. """' th- u•doci,-1,.t….,.. 11p d<k<,m ,.i.:, u:tftVII IMN b,tb- ,,!'$..,.., ,.,.,,..rcl:lt• pobt • ..,, nt-.:1 r4 r,.,.,._.., !lit 111,.r Y.al. ~ CUii<. •di • pod n&l>IN'd

-!>On ., – .. •-h "'c~I ,m "' -O'nblt O,ai; I atfl hff " """' -•·1-~ .,.11,1,.., ,,__ ….aid ;,o-hly ~­ C11V-4-~·~1,tp-,,~ .

·1- • titf~ (b.Wn .. ,…i ….. ,,,.. …. "' ~y::lll.'(ttOAC~ .. ,111,

An Amazon Kindle (http://www.amazon.com/) book reader being used to browse Bleak House by Charles Dickens. The Kindle uses E-lnk® technology (http://www.eink .com/), providing a bright disp lay that uses power only when the display changes, and can be read in direct sun light and at varying angles, which can improve reading comfort (see Section 14.4 fo r a discussion of reading on paper versus on a display).

10.4.2 Large wall displays The ubiquity of computer displays, from desktops to mobi le devices, projectors, and large televisions, lets us envision how integrating all those displays could provid e mor e producti ve work and play environm ents (Ardito et al., 2015). The differentiation might fade in the futur e, but there are currently thre e types of large wall displays. Informational wall displays provide shared views to users standing far away from the disp lay, while interactive wall displays allow users to walk up to the display and interl eave interaction and discussion among partici ­ pants. Finally, users sitting at their desks can connect multiple-desktop displays to their computers to have a larger number of windows and documents visible at the same time and within reach of the mouse. Of course, hybrid combinations are possible (see, for example, Badam and Elmqvist, 2014).

Large informational wall display s are effective in control rooms to provid e overviews of the system being monitored (Fig. 10.18); details can be retrieved on individual conso les. Military command and control operations, uti lity man­ agement, and emergency response are common applications, as large displays

10.4 Disp lays 367

F GURE 10.18 Multip le high-reso lution disp lays tiled together to present weather, traffic, message sign status, and road-condition informat ion to the operato rs in the State of Maryland's Highway Admin istration cont rol room (http ://www .cha rt.sta te. md.us/) .

help to build situation awareness through a co1nmon understanding of the informa tion presented and to facilitate coordination. Wall displays also allow teams of collaborating scientists or decision makers to look at applications that may be running on different computers, locally or remotely, but are presented on a single display.

Or iginally built w ith ma trices of CRTs and made popular in commercial or entertainment settings, wall displays now often use rear-projection tech­ niques or tiled LCD displays. Rear projection has become popular because improved calibration and alignment techniques allow for seamless displays. When seen from a distance, informational wall displays require bright projec­ tors, but the resolution does not need to be very high – 35 dots per inch is suf­ ficient. However, when users want to vievv or interact with the display at close range, the low reso lution of current projectors is often insufficient. Fur ­ thermore, rear projection requires a large amount of space for the throw dis­ tance of each projector. In such situations, tiled displays consisting of a grid of LCD monitors may be the most cost -effective solution (extreme ly large LCD displays are still prohibitively expensive to build) even if the display bezels yield visib le seams between each tile. The Stony Brook University Reality Deck

368 Chapter 10 Devices

FIGURE 10. 19

Users discussing and pointing at details on the Stony Brook University Reality Deck (Papadopoulos et al., 2015). an immersive gigapixel display consisting of 416 thin­ bezel LCD displays and powered by 18 graphics workstations connected using a high-speed network (https://la bs .cs.su nysb. edu/la bs/vis lab/rea I ity-deck-home/).

(Fig 10.19) is an example of an extreme-scale tiled LCD displ ay consisting of more than 400 individual high-resolution screens (Papadopoulos et al., 2015). Regardless of the technical solution, these multi-display environmen ts typi­ cally need specialized software frameworks to distribute the renderir1g and merge input across multiple computers; PolyChrome is an example of such a framework (Badam and Elmqvist, 2014).

For interact ive wall displays (Figs. 10.18, 10.19, and 10.20), the traditi onal desktop interaction techniques, st1ch as indirect-control pointing devices and pull-down menus, become impractical. New techniques are being devised to maintain fluid interaction with freehand sketching or novel menu techniques (Chapter 8). Even on large interactive group displays, space is limited and designers are exploring new ways to dynamically scale, summarize, and man­ age the information presented or generated by users on the display.

Digital whiteboard systems, such as the SMART Board® from SMART® Tech­ nologies, Inc., provide a large touch-sensitive screen on which a computer image is projected. Their functionality is identicaJ to that of the desktop machine, using

10.4 Disp lays 369

FIGURE 10.20 Tw o users collaboratively control a lens on a gigapixe l image of Paris, France, using a tab let touchscreen as well as an interactive cursor (Chapuis et al., 2014). Photo © lnria – H. Raguet, with permissio n.

users' fingers as pointing devices. Colored pens and a digital eraser simula te a traditional whiteboard, augmented with annotation recording and a software keyboard.

Facilitating collaboration between local or remote users (Chapter 11), manag­ ing the recording and reuse of brainstorming information, providing new cre­ ative tools for artists and performers, and designing new interaction methods using mobile devices are examples of challenges and opportunities created by interactive wall displays.

Multiple-desktop displays usuall y employ traditi onal flat panels, which intro­ duce discontinuities in the overall display surface (Fig. 10.21). Those displays can also be of different size or resolution, adding the possibility of misalignments. On the other hand, users can continue interacting with applications in the familiar way, eliminating training as users simply spread windows across displays. Another concern is that multiple-desktop displays may require users to stand, or at least rotate their heads or bodies , to attend to all displays, and even attentive users might not notice warnings or alarms that are far from their foci of attention. Organized users might assign displays to particular functions (for examp le, the left-hand display may always show e-mail and calendar applications and the

370 Chapter 10 Devices

FIGURE l 0.21 A nalyst interacting wit h cap ital markets data on a six-monito r Bloomberg Terminal. One of approx imately 320,000 units, these terminals are leased to clients on a subscription basis at approximate ly $20,000 per user (http://www.b loomberg.com/ professional/products-solutions/). Note the non-over lapping user inter face and the specia lized keyboard.

front display always a word processor), but this strategy can be detrimental when the current task woul d benefit from using the entire display space.

Multi ple -desktop disp lays are particularly useful for persona l creative appli­ cations. For example, creating an interactive web application in JavaScript might require a timeline, a stage, graphic-component editors, a scripting-language editor, a directory browser, and a preview window, all open at the same time. Multiple-desktop displays might also facilitate side-b y-side comparisons of documents, software debugging, or reasoning based on a large numb er of infor­ mation sources . They are usually great ly appreciated by users, and empirical evidence of their benefits is emerging (Andre ws et al., 2011).

Of course, there is a danger that cluttered displays will become more cluttered as their physical size grows. Also, direct manipulation on large displays can become a challenge because of the distance between objects. On the other hand, results show that having access to large displays even for individual use makes reasonh1g easier because the user can intelligently organize information in space (Andrews et al., 2011). Innovations can mitigate many of these challenges. Refinements should be made so that the 1nouse cursor can easily be found and tracked across displays. Rapid focus swi tching between windows might be facilitated by clicking on small overviews placed at stra tegic locations on the display. Furtl1ermore, strategies for automatic window layout and coordination among windows wi ll become critical

10.4 Disp lays 371

(refer to Section 12.3). Neverthe less, as the cost of displays continues to drop, it seems clear that multiple-desktop displays or simply larger single displays will become prevalent.

10.4.3 Tabletop (horizontal) displays While ,,vall displays promote coordination and consensus, horizontal surfaces have been shown to invite collaboration and discussion (Rogers and Lind ley, 2004). For this reason, tabletop displays have become an in teresting platform for deeply collaborative settings, such as for creative design, prob lem solving, or real -time resource management and planning. Such digital tabletops are gener ­ ally equipped with 1nulti-touch touchscreens, which allow a single user to use both hands or multiple fingers at once or allow multiple users to work together on a shared surface. The Microsoft Surface, Surface 2, Perceptive Pixel, and Sur­ faceHub are examp les of such devices. Circle Twelve's DiamondTouch TM dis ­ play allows the application to tell which user touched the screen, allowing better identification of personal versus collaborative interactions. With hori zontal dis­ plays, users can be positioned anywhere around the tab le, so applications that can be used in any orientation are desirab le (Fig. 10.22). Physical objects might also be used to mark positions and facilitate design sessions. Using stereoscopic displays, volumetric displays, or head-mounted displays, it may become possi­ b le to design effective 3-D tabletop interactions (Grossman and Wigdor, 2007). Furthermore, combining shared tabletops with user's personal mobile devices is also particularly powerful (McGrath et al., 2012).

FIGURE 10.22

• • • . "'. • • —-

• • … -• ·­•

Two people collabora t ing on a real estate tas k using a tabletop display and mobile tablet (McGrath et al., 2012). The tabletop serves as a shared and public disp lay where changes affect all col laborators, whereas the tablet is perceived as a private d isplay that allows users to work independently.

372 Chapter 10 Devices

Multi-touch displays-both in terms of tabletops as wel l as mobile devices­ have ushered in a vocabulary of simple gestures for complex direct manipula­ tion. Such gestures include pinching, grasping or spreading apart two fingers (e.g., to zoom out or in 011 an image), and dwelling, holding a finger touch for an extended amount of time (e.g., to bring up a context menu on the selected object).

10.4.4 Heads-up and head-mounted displays Personal-displa y technology involves smal l portab le monitors, often made with LCDs in monochrome or color. A heads-up display projects information on the partially silvered windscreen of an airplane or car, for example, so that the pilots or drivers can keep their attention focused on the surroundings wh ile receiving computer-generated information.

An alternative, the head-n1ounted display (HMO) used in virtual reality or aug ­ mented reality applications (Section 7.6), lets users see information even while turning their heads. In fact, tracking the user's head allows for dynamically changing the information that is seen as a function of the direction in which the user is looking. Different models provide varying levels of visual -field obstruc ­ tion, audio capabilities, and resolution. Google's Glass project was an attempt at blending HMDs with mobile devices, yielding a wearable computer with a dis­ play in tl1e comer of the user's vision. The Oculus Rift and Microsoft HoloLens devices (Chapter 7) represent reimaginings of the HMO concept, again mostly targeted at recreational and non-professional users.

10.4.5 Mobile device displays The use of mobile devices is becoming widespread in personal and business applications and has the potential to improve medical care, facilitate learning in schoo ls, and contr ibut e to more fulfilling sightsee ing experiences. Medical monitors can alert doctors when a patient's life signs reach a criti cal level, schoo lchildren may gather data or solve problems collaboratively using ltandheld devices, and emergency rescue personnel ca11 evaluate their situa ­ tion in dangerous environments by using small devices fixed on their suits. Small disp lays are also finding ways int o our homes, with reprogrammable picture frames and other devices, and even onto our bodies, with ever more powerfu l wr istwatches with customization features to fit the needs of the moment. The new generation of so-called "smartwatches," such as the Apple Watch and Fitbit Surge (Fig. 10.23), integrate step counters, heartbeat, and GPS as well as additional advanced functionality such as text, e-mail, calendar, voice recognition, and even electronic payment options into the wristwatch form factor.

10.4 Disp lays 373

FIGURE 10.23 The Apple Watch on the left supports both fitness as well as personal information management app lications, such as e-mail, calendar, and electronic paymen t. The Fitbit Surge smartwatch on the right is designed mainly for personal fitness applications and contains a step counter, heart rate monitor, and GPS.

Guidelines are emerging from experience with mobile devices (Ballard, 2007). Industry has been leading th e way by providing useful design case studi es and detailed guidelines such as those developed for Android devices or the iPhone. Such guidelines can be seen as micro -HC I theories.

Ballard (2007) sees mobile devices being developed in four classes, depending on their intended usage: (1) general-purpose work (similar to the Blackberry or Pocket PC), (2) general-purpose entertainment (which focus on multimedia features like the App le iPod), (3) general-pu rpose communication and control (extensions of today's phones), and (4) targeted devices that do only a few tasks (e.g., the United Parcel Service drivers' DIAD IV). Mobile devices are often used for brief but routine tasks. Therefore, it is critical to optimize the designs for those repetitive tasks while hiding or eliminating less important functions. Whenever possib le, data entry should be reduced and complex tasks offloaded to the desktop.

While researchers and deve lopers are steadily in creasing the scope of appli­ cations for mobile devices, a framework for thinking about the range of actions may be he lpful. Whether the application is financial -, medical-, or travel-related, the following five pairs of actions shou ld be considered: (1) n1onitor dynamic information sources and alert when app ropriate , (2) gather information from many sources and spread out information to many destinations, (3) participate in

374 Chapter 10 Devices

a group and relate to individuals, (4) locate services or items that are not vis ible (for example, the nearest gas station) and identifiJ objects that are seen (for exam­ ple, the name of a person or flower), and (5) capture information from local resources arld shn.re your information with future users.

Poor readability will be an issue in low light or for users with poor eyesight, and users will appreciate tl1e ability to adjust the font size. Reading on small screens might also be improved with rapid serial visual presentation (RSVP), which presents text dynamically at a constant speed or at a speed adapted to the con tent. Using RSVP, although no differences were found for long texts, a 33% improvement was measured in speed of reading for short texts (Oquist and Goldstein, 2003).

Successful interaction design for 1nobile devices requires adapting content to the display, regardless of its size and capabilities (color depth, resolution, update frequency). This is particularly importa11t for web designers, as people are increasingly using their mobile devices to access the web . Responsive iveb design (RWD) is an approach to creating webpages and web applications so that the result is optimized for wha tever device is being used to view a web­ site . Responsive web design can be summarized in a small se t of basic princi­ p les: (1) mobile-first design, rather than desktop first, so that the constraints of limited resources become more apparent to the designer; (2) unobtrusive dynaniic behavior, so that a website is not wholly dependent on JavaScript; and (3) progressive enhance1nent, where websites are layered with increasingly advanced functiona lity so that backward compatibility is retained for basic browsers while advanced browsers can take full advantage. Fig. 10.24 shows responsive web design in practice on three different display sizes.

Mobile users often have only one hand available and rely on their thumbs to interact with the de vices . Guidelines for mobile interfaces that support one ­ handed interaction include placing targets close to one another to minimize grip adjustment, allowing users to configure tasks for either left- or right-handed oper­ ation, and placing targets toward the cen ter of the device (Karlson et al., 2008).

LCD Monitor Tablet Phone

□ DD

FIGURE l 0.24 Examples of responsive web design. The monitor layout on the left is automat ically adapted to the smaller d isplay space of a tablet (m idd le) and a smartphone (right) . Also see Chapter 8.

10.4 Displays 375

Another challenge facing designers of mobile-device applications is the growing diversity of devices, which may require finding interaction styles that adapt to multiple screen sizes and can be activated by multiple input mechanisms (QWERTY keyboards as well as touchscreens, keypads, or directional pads).

As mobile devices become information appliances, they may contribute to the goal of bridging the digital divide. These low-cost devices, which are easier to master than desktop computers, may enable a wider range of people to ben­ efit from information and commmucation technologies. Developing countries are seeing a rapid spread of mobile technology, which requires less local infra­ structure than providing stable electricity. For users with disabilities, mobile telecommunication devices offer a unique opportunity to design n1odalihJ trans­ lation services, as described by a project of the Trace Center at the University of Wisconsin. Remote services can provide instant translation from one presenta­ tion mode to another, anywhere and at any time, via mobile devices. This per­ mits text-to-speech, sign language, international language, and language-level translation as well as print recognition and image/video description services. Modality translation could benefit people with disabilities and people who have no disabilities but experience functional limitations when driving their cars, vis ­ iting a foreign country, or touring a museum without reading glasses.

l 0.4.6 Deformable and shape-changing displays Current displays are flat due to reasons of both manufacturing and tradition, but this will likely not remain the case in the near future. In terms of manufacturing, new technological advances in shape displays, digital fabrication, and programmable matter will soon allow hardware companies to build displays that are of virtually any shape. In terms of tradition, while computer interfaces have long been displayed on planar surfaces, human history, culture, and technology are full of examples of non-planar manifestations of data and content, such as sculptures, statues, tokens, souvenirs, paintings, medals, and mementos. In other words, future displays will not only go beyond the flat plane, but they will also be shape-changing in that they bend, move, and respond to not only virtual but also physical interactions.

Some examples of future non-flat and shape-changing displays are given in Fig. 10.25. Physical visualizations are 3-D embodiments of data graphlcs ren­ dered on the screen Gansen and Dragicevic, 2013). They represent examples of static physicalizations(http://dataphys.org/) of data and can often be fabricated using a 3-D printer (which also can be seen as a form of display or output device). Tilt displays challenge the flat LCD surface by mounting multiple small displays on actuators {Alexander et al., 2012). The displays move in response to the data and can also be manipulated themselves, making them also input devices. The Paper Phone is an exercise in flexible displays and is a prototype of a smartphone based on bendable paper (Lahey et al., 2011). Just like the tilt display, the Paper­ Phone is also an input device with several gestures defined for how to make and

376 Chapter 10 Devices

FIGURE 10.25 The left image shows a physical bar chart visualization displaying comp lex data (Jansen and Dragicevic, 2013). The midd le shows the tilt disp lay that consists of multiple small displays mounted on actuators (Alexande r et al., 2012). On the right is the PaperPhone, a flexible smartphone prototype that supports bending interaction (Lahey et al., 2011 ).

take calls, navigate in the address book, and so on. Finally, Fig. 10.26 shows the inFORM dynamic shape display that serves as both output as well as input dis­ play for data physicalization (Follmer et al., 2013).

FIGURE 10.26

A telepresence applicat ion being used on the inFORM dynam ic shape display, an actua ted and touch-respons ive shape-c hanging display (Follmer et al., 2013).

Researcher's Agenda 377

Practitioner's Summary

Choosing hardware always involves 1naking a compromise between the ideal and the practical. The designer's vision of what an input or output device should be must be tempered by the realities of what is commerc ially available within the project budget. Devices should be tested in the application domain to verify the manufacturer's claims, and testimonials or suggestions from users should be obtained.

New devices and refinements to old devices appear regularly; device-indepen­ dent architecture and software permit easy integration of novel devices. Avoid be­ ing locked into one device, as the hardware is often the softest part of the system. Instead, aim to transcend device hardware by focusing on the task rather than the mechanics of performing it. Also, remember that a successful software idea can be­ come even more successful if reimplementation on other devices is easy to achieve and if cross-modality permits users with disabilities to access the system. Remember Fitts' s Law to optimize speed of performance, and consider two-handed operations.

Keyboard entry is here to stay, but consid er other forms of input when text en­ try is limited. Selecting rather than typing has many benefits for both novice and frequent users. Direct-pointing devices are faster and more convenient for novices than are indirect-pointing devices, and accurate pointing is possible, but remember that users on the go are likely to use the devices with a single hand. Simp l.e gestures can trigger a few actions and are show ing promising applications. Mobile devices ha ve become the universal computing platform of the world, and many people nowadays exclusively use their smartphone for both work and play. For this rea­ son, designers may want to design mobile-first interfaces that are responsive to varying devices.

Display technology is moving rapidly, and user expectations are increasing. Multi-touch displays are now expected by most users, and these come with a standardized vocabulary of touch gestures. Beyond mobile devices, large high­ resolution displays are becoming prevalent. Tl1e current next big thing in dis­ plays are those that go beyond the plane and become volumetric, deformable, and shape-changing, in effect blurring the border between input and output.

Researcher's Agenda

Novel text-entry keyboards to speed input and to reduce error rates ,vill have to provide significan t benefits to displace the well-entrenched QWERTY design. For the numerous applications that do require extensive text entry or run on mobile device applications, many opportunities remain to create special-purpose devices

378 Chapter 10 Devices

or to redesign the tasks to permit direct-manipulation selection instead of key­ boarding. Increasingly, input can be accomplished via conversion or extraction of data from online sources. Another input source is optical character recognition of printed text or bar codes (for example, in magazines or bank statements) or RFID tags attached to objects, clothing, or even personalized dolls. Finally, while it is not possible to truly "beat" Fitts's Law for pointing situations, there exist many ways to assist with or minimize tasks that require pointing.

The range of display sizes available has widened enormously, and users need applications that can operate on mobile devices, desktops, and large wall or table­ top displays. Researchers need to understand how to design plastic or multi-modal interfaces that allow users to adapt their interfaces depending on the environ­ ment, their preferences, and their abilities. What are the strategies for increasing productivity with multiple screens? Sensors embedded in the e11viro1unent and in many mobile devices can provide information about users' locations or activi­ ties to enable development of context-aware applications. The benefits may be large, but inconsistent behavior and pri vacy concerns will have to be addressed before adoption become s widespread. Finally, the new breed of sl1ape-changing disp lays requires considerab le further study in order to realize its full potentia l.

WORLD WIDE WEB RESOURCES

www. pearsonglobaleditions . com/shneiderman

Input and output computing hardware is a major industry, and each manufac­ t urer have its ow n websites and resources for more information . The follow ing are the most dominant manufacturers for keyboards and pointing devices .

• Logitech: http://www.logitech .com/

• Microsoft: http://www.microsoft.com • Dell: http://www.dell.com

Touchscreens are now commonplace and have no centralized resource, but

Wacom has long been the leader in graphical t ablets.

• Wacon: http://www.wacom.com

Common brands for large -scale to uchscreens, such as tabletops and wall ­ moun ted screens, inc lude:

• SMART Technologies: http://www .smarttech.com

• Microsoft: http://www .microsoft.com

Fina lly, the Trace Center has exce llent resources for accessible and inclusive

input and output devices:

• Trace Center:

http://trace.wisc .edu/

References 379

Discussion Questions

1. A company is designing a kiosk that can display weather information in pub­ lic locations. The kiosk wi ll feature a touch screen so users can select a city by pointing on a map. Give three reasons why a touch screen is an effective device for this application.

2. Explain the difference between direct-control and indirect-control pointing devices. Name a task when the one type is a more appropria te device than the other.

3. Alhat are the different interaction tasks for which pointing devices are useful? How can the chal lenges faced by visually impaired people while using point­ ing devices be addressed?

4. Define responsive design. What characteristics of a display would make an individual state that the design they are viewing seems responsive?

5. What are the advantages of large wall displays? What are the limitations?

6. Give a definition of context-aware computing. Provide an example of one appli­ cation of context-aware computing that would meet the user needs of a tourist.

References

Accot, Johnny, and Zhai, Shumin, More than dotting the i's: Foundations for crossing­ based interfaces, Proceedings of the ACM Conference on Hun1an Factors in Con1puting Systems, ACM Press, New York (2002), 73-8 0.

Alexander, Jason, Lucero, Andres, and Subramanian, Sriram, Tilt displays: Designing display surfaces with multi-axis tilting and actuation, Proceedings of the ACM Conference on Human Con1puter lnteraction with Mobile Devices and Services, ACM Press, New York (2012), 161-170.

Andrews, Christopher, Endert, Alex, Yost, Beth, and North, Chris, Information visu­ alization on large, high-resolution displays: Issues, challenges, and oppor tunitie s, Info1·n1ation Visualization 10, 4 (2011), 341- 355.

Ardito, Car1nelo, Buono, Pao lo, Costabile, Maria Francesca, and Desolda, Giuseppe, Interaction with Large Displays: A Survey, ACM Con1puting Surveys 47, 3 (2015), 46.

Azenkot, Shiri, Wobbrock, Jacob 0., Prasain, Sanjana, and Ladner, Richard E., Input finger detection for nonvisual touch screen text entry in Perklnput, Proceedings of Graphics lnterface, Canadian Information Processing Society, Toronto, ON, Canada (2012), 121-129.

Badam, Sriram Karthik, and Elmq vist, Niklas, PolyChrome: A cross-device framework for collaborative web visua lization, Proceedings of the ACM Conference on interactive Tabletops and Surfaces, ACM Press, New York (2014), 109-1 18.

380 Chapter 10 Devices

Badam, Sriram Karthik, Chandrasegaran, Senthil, Elmqvist, Niklas, and Ramani, Karthik, Tracing and sketching p erformanc e using blunt -tipped styli on direct-touch tablets, Proceedings of the ACM Conference on Advanced Visual Interfaces, ACM Press, New York (2014), 193-200.

Balakrishnan, Ravin, "Beating" Fitts' law: Virtual enhancements for pointing facilita­ tion, International Journal of Human-Coniputer Studies 61, 6 (2004), 857-874.

Ballard, Barbara, Designing the Mobile User Experience, John Wiley & Sons, New York (2007).

Baudis ch, Patrick, and Chu, Gerry, Back-of-devi ce interaction allows creating very small tou ch devi ces, Proceedings of the ACM Conference on Hurnan Factors in Cornputing Systerns, ACM Press, New York (2009), 1923-1932 .

Baudisch, Patrick, and Holz, Christian, My new PC is a mobile phone, XRDS: Cross­ roads, The ACM Magazine for Students 16, 4 (2010), 36-41.

Chapui s, Olivier, and Dragicevic, Pierre, Effects of 1notor scale, visual scale and quan­ tization on s1nall target acquisition difficulty, ACM Transactions on Cornputer-Hurnan Interaction 18, 3 (2011).

Chapuis, Olivier, Bezerianos, Anastasia, and Frantzeskakis, Stelios, Smarties: An input sys tem for wall displa y development, Proceedings of the ACM Conference on Human Factors in Co,nputing Systems, ACM Press, New York (2014), 2763-2772.

Clawson, Jam es, Lyons, Kent, Rudnicky, Alex, Iannucci , Jr. , Rob ert A., and Starn er, Thad, Automatic whiteout++: Correcting mini-QWERTY typing errors using keypre ss timing, Proceedings of the ACM Conference on Human Factors in Co1nputing Syste,ns, ACM Press, New York (2008), 573- 582.

Dachselt, Raimund, Hakkila, Jonna, Jones, Matt, Lochtefeld, Markus, Rohs, Michael , and Rukzio, Enrico, Pico projectors: Firefly or bright future? ACM Interactions 19, 2 (2012), 24-29.

Dourish, Paul, and Bell, Gene vieve, Divining a Digital Future: Mess and Mythologi; in Ubiquitous Con1puting, MIT Press (2011).

Dunlop, Mark, and Masters, Michell e, Pickup usability dominates: A brief history of mobile text entry research and adoption, International Journal of Mobile Human Co111- puter Interaction 1, 1 (2008), 42- 59.

Findla ter, Leah, Wobbrock, Jacob 0., and Wigdor, Daniel, Typing on flat glass: Examin ­ ing ten-finger expert typing patterns on touch surfaces, Proceedings of the ACM Confer­ ence on Hurnan Factors in Cornputing Systenzs, ACM Press, New York (2011), 2453-2462.

Fitts, P. M ., The information capacity of the human motor syste m in controlling ampli­ tude of movement, Journal of Experin1ental Psychology 47 (1954), 381-391.

Folbner, Sean, Leithinger, Daniel, Olwal, Alex, Hogge, Akitmitsu, and Ishii, Hiroshi, inFORM: Dynamic physical affordances and constraints through shape and object actuation, Proceedings of the ACM Symposium on User Inteiface Softivare & Technology, ACM Press, New York (2013), 417-426.

Forlines, Clifton, Vogel, Daniel, and Balakrislman, Ravin, HybridPointing: Fluid sw itching between absolute and relative pointing with a direct input device, Proceedings of the ACM Symposiuni on User Inteiface Sofhvare & Technology, ACM Press, New York (2006), 211-220.

Grossman, Tovi, and Wigdor, Daniel, Going deeper: A taxonomy of 3D on the tabletop, Proceedings of the IEEE International Workshop on Horizontal Interactive Humnn-Cornputer Syste,ns, IEEE Press, Los Alamitos, CA (2007), 137- 144.

References 381

Grossman, Tovi, Dragicevic, Pierre, and Balakrishnan, Ravin, Strategies for accelerating on-line learning of hotkeys, Proceedings of the ACM Conference on Hun1an Factors in Computing Systen1s, ACM Press New York (2007), 1591- 1600.

Guiard, Yves, Bourgeois, Frederic, Mottet, Deni s, and Beaudouin -Lafon, Michel, Beyond the 10-bit barrier: Fitts' law in muJti- scale elech·onic worlds, in People and Comput- ers XV-Inte raction Without Frontiers (Joint Proceedings of HCI 2001 and IHM 2001), Springer -Verlag, London, UK (2001), 573-587.

Hansen, Thomas Riisgaard, Eriksson, Eva and Lykke-Olesen, Andreas, Use yotu head: Exploring face tr acking for mobile interaction, Extended Abstracts of the ACM Confer­ ence on Hun1an Factors in Con1puting Systems, ACM Pre ss, New York (2006), 845-850.

Harrison, Chris, Appropriated interaction surfaces, IEEE Con1puter 43, 6 (2010), 86- 89.

Hin capie-Ramos, Juan David, Guo, Xiang, Moghada sian, Paymahn, and Irani, Pourang, Consumed end ur ance: A metric to quantify arm fatigu e of mid -air intera ctions, Pro­ ceedings of the ACM Conference on Hun1an Factors in Computing Systems, ACM Pre ss, New York (2014), 1063- 1072.

Hinckley, Ken, and ,Vigdor, Daniel, Input technologies and techniqu es, in Jack o, Julie, and Sears, Andrews (Editors), The Hun1an-Co1nputer Interaction Handbook, Laurence Erlbaum Associates, Mahwah, NJ (2011), 95-1 32 .

Hoggan, Eve, Brewst er , Stephen A., and Johnston, Jody, Investigating the effectiveness of tactile feedback for 1nobile touchscreens, Proceedings of the ACM Conference on Hurnan Factors in Conzputing Systenzs, ACM Press, New York (2008), 1573-1582.

Hou rcade, Juan-Pablo, Bederson, Benjamin 8 ., Druin, Allison, and Guirnbretiere, Fran co is, Difference s in poin ting ta sk performance between preschool children and adu lts u sing mice, ACM Transactions on Con1puter-Hun1an Interaction 11, 4 (December 2004), 357-386.

Jacob , Robert J. K., Girouard, Au dr ey, Hirshfield, Leanne M., Horn, Michael S., Sha er, Orit, Solovey, Erin Treacey, and Zigelbaum, Jamie, Reality-based intera ction: A framework for post-WIMP interfaces, Proceedings of the ACM Conference on Hun1an Factors in Cornputing Systerns, ACM Pres s, New York (2008), 201- 210.

Jansen , Yvonne, and Dragic ev ic, Pierre, and Fekete, Jean-Daniel, Evaluating the efficiency of physical visualizations, Proceedings of the ACl',,f Conference on Human Factors in Con1puting Systems, ACM Press, New York (2013), 2593-2602.

Kane, Shaun, Wobbrock, Jacob 0., Hami ss, Mark, and John son, Kurt L., TrueKeys: Identifying and correcting typing errors for peop le w ith motor impairments, Proceedings of the ACM Conference on Intelligent User Interfaces, ACM Press, New York (2008), New York, 349-352.

Karlson, Amy, Bederson, Benjamin B., and Contreras-Vidal, Jose L., Understanding one handed use of mobile dev ices, in Lumsden, Johanna (Editor), Handbook of Research on User Interface Design and Evaluation for Mobile Tech.n.ologi;, lrtformation Science Refer ­ ence/ IGI Globa l, Hershey, PA (2008), 86- 101.

Kortum , Philip (Editor), HCI Beyond the GUI: Design for Haptic, Speech, Olfactory and Other Nontraditional Interfaces, Elsevier/Morgan Kaufmann, Amsterdam, Nether land s (2008).

Kristensson, Per Ola, and Denby, Leif C., Text en try pe rformance of state of the art un constrai ned handwriting recogn ition: A longitudin al use r stu dy, Proceedings of the ACM Conference on. Hurnan Factors in. Con1puting Systerns, ACM Pres s, New York (2009), 567- 570.

382 Chapter 10 Devices

Lahey, Byron, Girouard, Audrey, Burleson, Winslow, and Vertegaal, Roel, PaperPhone: Understanding the use of bend gestures in mobile devices with flexible electronic paper displays, Proceedings of the ACM Conference on Hu111an Factors in Co·mputing Systen'IS, ACM Press, New York (2011), 1303-1312.

Liao, Chunyuan, Guimbretiere, Fran<;ois, Hinckley, Ken, and Hollan, Jim, Papier­ Craft: A gesture-based command syste1n for interactive paper, ACM Transactions on Co1nputer-Hun1an Interaction 14, 4 (2008), 14.

MacKenzie, I. Scott, Kober, Hedy, Smith, Derek, Jones, Terry, and Skepner, Eugene, Letter Wise: Prefix -based disambiguation for mobile text input, Proceedings of the ACM Sy111posiun1 on User Interface Software and Technology, ACM Press, New York (2001), 111-120.

MacKenzie, Ian Scott, Human-Computer Interaction: An E,npirical Research Perspective, Morgan Kaufmann, San Francisco, CA (2013).

McCallum, David, and Irani, Pourang, ARC -Pad: Absolute+ relative cursor positioning for large d isplays with a mobile touchscreen, Proceedings of the ACM Sy1nposiu111 on User Interface Software & Technology, ACM Press, New York (2009), 153-156.

McGrath, Will, Bowman, Brian, McCallum, David, Hincapie-Ramos, Juan David, Elmqvist, Niklas, and Irani, Pourang, Branch-Explore-Merge: Facilitating rea l-time revision control in collaborative visual exp loration, Proceedings of the ACM Conference on Interactive Tabletops and Surfaces, ACM Press, New York (2012), 235- 244.

Mithal, Anant Kartik and Douglas, Sarah A., Differences in movement microstructure of the mouse and the finger-controlled isometric joystick, Proceedings of the ACM Conference on Human Factors in Computing Systenis, ACM Press, New York (1996), 300-307.

Mon temayor, Jaime, Druin, Allison, Chipman, Gene, Farber, Allison, and Guha, Mona Leigh, Tools for ch_ildren to create physical interactive storyrooms, ACM Con1puters in Entertainrnent 1, 2 (2004), 12.

Murugappan, Sundar, Vinayak, Elmgvist, Niklas, and Ramani, Karthik, Extended mul­ titouch: Recovering touch posture and differentiating users using a depth camera, Proceedings of the ACM Sy111posiurn on User Interface Software and Technology, ACM Press, New York (2012), 487-496.

Myers, Brad, Using handhelds for wireless remote control of PCs and appliances, Inter­ acting ivith Con1puters 17, 3 (2005), 251- 264.

Oquist, Gustav, and Golds tein, Mikael, Towards an improved readability on mobile devices: Evaluating adaptive rap id serial visual presen tation, Interacting with Co,nputers 15, 4 (2003), 539-558.

Oney, Stephen, Harrison, Chris, Ogan, Amy, and Wiese, Jason, ZoomBoard: A diminu­ tive QWERTY soft keyboard using iterative zooming for ultra-sma ll devices, Proceed­ ings of the ACM Conference on Hun·,an Factors in Camputing Systems, ACM Press, New York (2013), 2799- 2802.

Papadopoulos, Chariloas, Petkov, Kaloian, Kaufman, Arie E., and Mueller, Klaus, The Reality Deck – An immersive gigapixel display, IEEE Con1puters & Graphics 35, 1 (2015), 33-45.

Pew Research Center, En1erging Nations Embrace Internet, Mobile Technology, February 2014.

Refe rences 383

Raghunath, Mandayam, Narayanaswami, Chandra, and Pinhanez, Claudio, Fostering a symbiotic handheld environment, IEEE Con1puter 36, 9 (2003), 57-65.

Rogers, Yvonne, and Lind ley, Sian E., Collabora ting around vert ical and horizontal large interactive displays: Which way is best? Interacting with Computers 16, 6 (2004), 1133- 1152.

Rohs, Michael, and Oulasvirta, Antti, Target acquisition with camera phones when used as magic lenses, Proceedings of the ACM Conference on Hun1an Factors in Computing Systems, ACM Press, New York (2008), 1409- 1410.

Sou thern, Ca leb, Clawson, James, Frey, Brian, Abowd, Gregory 0., and Romero, Mario, An evaluation of Braille Touch: Mobile touchscreen text entry for the visually impaired, Proceedings of the ACM Conference on Hurnan-Cornputer Interaction with Mobile Devices and Services, ACM Press, New York (2012), 317- 326.

Stellmach, Sophie, and Dachselt, Raimund, Still looking: Investigating seamless gaze ­ supported selection, positio n ing, and manipulation of distan t targets, Proceedings of the ACM Conference on Hu,nan Factors in Con1puting Systen1s, ACM Press, New York (2013), 285-294.

Vande rheiden, G., Kelso, D., and Kr ueger, M., Extended usability versus accessibility in voting systems, Proceedings of the 27 RESNA International Annual Conference, RESNA Press, Arlington, VA (2004).

Vogel, Daniel, and Balakrishnan, Ravin, Occlusion-aware interfaces, Proceedings of the Conference on Hu,nan Factors in Cornputing Syste,ns, ACM Press, New York (2010), 263-272.

Vogel, Danie l, and Baud isch, Pa trick, Shift: A technique for ope rating pen -based inter ­ faces using touch, Proceedings of the ACM Conference on Human Factors in Co,nputing Systen1s, ACM Press, New York (2007), 657-666.

Wills , Sebastian A., and MacKay, David J.C., Dasher – An efficient writing system for brain-computer inte rfaces? IEEE Transactions on Neural Systen1s and Rehabilitation Engineering 14, 2 (2006), 244-246 .

Wigdor, Daniel, and Wixon, Dennis, Brave NUI World: Designing Natural User Interfaces for Touch and Gesture, Morgan Kaufmann (2011).

Wobbrock, J., Myers, B., and Kembe l, J., EdgeVVrite: A styl us-based text entry method designed for high accuracy and stab ility of motion, Proceedings of the ACM Syrnposiurn on User Interface Soft1vare and Technologi;, ACM Press, New York (2003), 61- 70.

Wobbrock, J. 0., Rubinstein, J., Sawyer, M. W., and Duchowski, A. T., Longitudinal evalu­ ation of d iscrete consecutive gaze gestures for text entry, Proceedings of the ACM Sy1npo­ siun1 on Eye Tracking Research and Applications, ACM Press, New York (2008), 11-18.

Ye, Han lu, Malu, Meethu, Oh, Uran, and Findlater, Leah, Current and future mob ile and wearab le device use by peop le with visua l impa irments, Proceedings of the ACM Conference on Hunzan Factors in Cornputing Systerns, ACM Press, New York (2014), 3123- 3132.

Zhai, Shumin, and Kris tensson, Per Ola, Sh orth and writing on stylus keyboa rd, Proceed­ ings of the ACM Conference on Hu1nan Factors in Computing Systems, ACM Press, New York (2003), 97- 104 .

CHAPTER

Co r.m r.m u@ ica t • Callao.a • •

written in co llaboration with Nicho las Diakopoulos

•• Coming together is a beginning, staying toge ther is progress, and working together is success . ''

Henry Ford

•• Drop the 'The .' Just 'Facebook.' It's cleaner. ''

CHAPTER OUTLINE 11. 1 Introduction

11.2 Models of Collaboration

11.3 Specific Goals and Contexts

11.4 Design Considerations

Justin Timb erlake as Sean Parker The Social Network, 2010

385

386 Chapter 11 Communication and Collaboration

11. 1 Introduction

Constant and immediate communication and interaction with family, friends, collaborators, colleagues, coworkers, and even pets are now commonplace it, the mcreasingly networked world. Communication and collaboration systems are re-making entire swaths of how lives are lived, including how people do work, find romance, exchange and deliberate policy, engage in civic participa­ tion, produce software and other creative wares, play and entertam themselves, shop for and select products and services, fmd support when they are m ill health, gain an education, and receive and produce information (Rainie and Wellman, 2012). The intrmsically motivating role of mterpersonal connected­ ness (Deci and Flaste, 1995) dri, res human bemgs to want to communicate and interact with others across tl,e full range of experiences.

Origmally launched as a microblogging platform in 2006 for people to post short, 140-character text messages about what they are doing, Twitter allows users to subscribe to each other's feeds, forming a "publish and subscribe" social network. In the 10 years since its launch, the platform has become an important way for people to share information, find experts, surface breaking news photos and videos, coordinate efforts during natura l disaster responses, and maintain an awareness of the pulse and reactions of those on the network (Vieweg et al., 2010; Diakopoulos et al., 2012). It's reshaping how news and information are dissemmated, moving away from a traditional gatekeeping model where professional editors had ultimate control over what was pub ­ lished or promoted to a system in which social activity, personal connections, and algorithmic ranking are just as important. Facebook is another major socia l network site (boyd and Ellison, 2007) and in 2015 had close to 1.5 bil lion global users who log on at least once a month. Facebook is similarly changing the information production and consumption landscape and has also been shown to have substantial benefits for individuals' relationships. People often join Facebook in order to keep up with friends or to solidify relationships with acqua intances like dormmates, classmates, and work colleagues. Early research has shown that Facebook helps people build social capital-the resources available to an individual as a result of having a durable network of relationships (Ellison et al., 2007). While Facebook and Twitter are dominant in the Uni ted States, other social network sites and chat platforms are flour­ ishmg elsewhere around the world, such as Weibo and Renren m China, VKontakte in Russia, and Kakao Talk in South Korea. Such platforms allow users to easily and cheaply maintain connections and crystallize relation­ ships. The positive outcomes from such platforms are wide-ranging, from job seekers hoping for support or a lead on a new job (Burke and Kraut, 2013) to citizens seeking to organize a political pro test to fight oppression. Social

11.1 Introduction 387

network sites can even enable positive public health outcomes such as contributing to smoking cessation programs by providing social support (Phua, 2013).

But it's also important for designers to co11sider and account for the down­ sides and negative exigencies of such systems. Criminals, terrorist recruiters, and oppressive political leaders can use social network sites for negative pur­ poses. A miscreant can bully or deceive others online, and hate groups can spew their propaganda. Some may become addicted to communication tools or waste time that could be more productive, while otl1ers will inadvertently share pho­ tos or other information that damages their reputation and is difficult to remove from their online profile. Social media can make behaviors such as stalking and public shaming easier (Ronson, 2015). Trusting or nai've children and adoles­ ce11ts may become targets of predatory adults seeking underage sexual relation­ ships. Ideas can become polarized when liberal and conservative thinkers cluster together based on homophily and pay less attention across the philo­ sophical divide. Sherry Turl<le has been critical of the role that mobile communi­ cation techi1ologies play in distracting people from fully participating in real-life conversation (Turkle, 2015). Important questions for society to consider are how new forms of communication change the way people think, build relationships and communi ty, and practice political organization. Designers must be aware of such behaviors and possible outcomes and consider design op tions that circum­ vent or mitigate the worst possibilities. Good design, effective community lead­ ership, and thoughtful governa nce policies and strategies can lead to more positive social outcomes.

Despite their hug e populari ty, social network sites are just one particular form of online communication. A Pew survey of Americans in 2014 found that four of the five top uses of smartphones were communication applications with modalities including text messaging, voice or video calls, e-mail, and socia l net­ working (Smith and Page, 2015). Different communicat ion channels and tools are more or less suited for different tasks and human needs, whether they be chatting with a friend or coworker, writing a collaborative document with someone, posting to a discussion forum or QI A site, participating in group project mana gement, coordinatin g a real-wo rld community gat herin g like a meetup, sharing files, or teleconferencing, among others. The academic field that emerged in the 1980s to study technology used by two or more peop le is called computer-supported cooperative work (CSCW), though 30 years later the field has also adopted social computing as an umbrella term that implies less of a strict devotion to collaboration and work per se and includes cooperation, collaboration, and competition as wel l as non-work activities like gaming and romance.

The communication and collaboration tools that designers create shape the ability to work and accomplisl1 shared goals with one another. The degree of interactivity, the social cues present in the interface, and tl1e mobility of

388 Chapter 11 Communication and Collaboration

communication technologies are but a few of the design dimensions that affect use in different contexts (Baym, 2015). Design is just a starting point for behav­ ior, though, and often people will quickly repurpose communication technolo­ gies to accommodate their specific needs. On Twitter, the need to re-share information while attributing it to the original source (a social norm) led to lin­ guistic innovations like via, retiveeting, and R/f before ultimately catching on as RT a compressed version of retweet that took up minimal space within the 140 characters allowed in a message (Kooti et al., 2012). Years later, Twitter formal­ ized this convention by building the abili ty to retweet a message directly into the platform. The social shaping of technology perspective reflects the idea that the tools that designers craft do not precisely determine how people will use them but rather interact with human goals as the technology co-evolves: "The consequences of techno logies arise from a mix of 'affordances' – the social capa­ bilities technological qualities enable-and the unexpected and emergent ways that people make use of those affordances" (Baym, 2015). Interface and experi­ ence design mus t be an itera tive and cons tantl y evolving endeavor as people and techno logy co-evolve.

Research in collaboration and communication interfaces is often more com ­ plex than in single-user interfaces. The multiplicity of users makes it difficult to condu ct experiments that control for group var iabi lity . Differences in phys­ ical distribution of participants can make the application of some research methods considerably more arduous. Studies of small-group psychology, industrial and organizational beh avior, sociology, and anthropology can pro ­ vide useful research paradigms (Lofland and Lofland, 2005). Content analysis methods can be used to ana lyze and create typologies of the types of messages that individuals post, leading to insights not only about content but also about the relationships between individua ls (Riffe et al., 2013). Also, as questions of macro-HCI are considered (see Chapter 3), studying communication platforms at scale requjres that methods in data science be adapted . Commun ication texts, including chat logs, tweets, Facebook posts, and on line comments can be ana lyzed using natural language processing (NLP) algorithms. These methods are useful for identifying, counting, or scoring texts (Diakopoulos, 2015), such as according to positive or negative sentim ent s expressed . Text analysis can also be combined with struc tural understanding using methods from social network analysis (Hansen et al., 2011; Leetaru, 2011). For examp le, topical network maps have elucidated structures among online Twitter groups like polarized crowds, tight crowds, brand clusters, community clusters, and broadcast or support networks (see Fig. 11.1). Such methods allow better understanding of how individual s organize and communicate online, elucidating structures and strategic locations or roles within the network (Smith et al., 2014).

Questio11s of ethics become paramount when stud ying open communicatio11 networks as people may share sensitive personal information without realizil1g

G 1: my2k tcc:0 2 obamn tlot women college cool lnyhbt gop

• ~

•­·–·-

–

FIGURE 11.1

• ··–·

I …:….

11.1 Introduction 389

G2: my2k ows p2 fb teat p2p con~e!eft gopmedicorekillers elec p2b -i.J1 . – . – —

– –

A network map made with NodeX L software shows the polarized nature of the conversation on Twitter around the #My2k hashtag that emerged over U.S. budget strugg les in 2012.

that a researcher might scoop it up and analy ze it for other purposes. Research­ ers must carefully consider whether they should anonymize names and remove other identifying information from subsequent published analyses (Bruckman et al., 2015) as well as more generally consider tl1e risks and benefits of such research. A Facebook study indicating that users' emotional state could be modtllated based on the positive or negative sentiments they were exposed to in their newsfeed (Kramer et al., 2014) resulted in widespread questioning of the ethics of such experimental manipulations. A survey of researchers who use online data found consensus around several ethical guidelines, including notifying participants about why data is being collected, sharing researc h results with participants, removing individuals from datasets upon their request, and being cautious when sharing results with identifiable outliers (Vitak et al., 2016). Studying others' platforms leads to additional challenges like a lack of control of the interface and an inability to know how the platform may be shifting or dynamic due to A/B tests (see Chapter 5.3.4). One way to overcome some of these issues is to build proprietary social software and

390 Chapter 11 Communication and Collaboration

develop a user base that is genuinely motivated to inhabit that online space. This approach is difficult but has been successful for projects like Group Lens at the University of Minnesota, which has been able to run large-scale studies of recommender systems and other online communities in tl1is fashion.

The following section (Section 11.2) presents a model to help orient readers within the design space. In Section 11.3, different collaboration arld communi­ cation goals and contexts are presented to illustrate how design can adapt to support different user needs . And finally, in Section 11.4, several design con­ siderations and challenges related to communication and collaboration technology are articulated.

11.2 Models of Collaboration

Consider a typical day for a digital native: Wake up and check socia l network­ ing accounts to get the latest news, go to work and collaboratively edit a report, chat with an office colleag11e about the new intern, post a question to a Q/ A site about a statistical test needed to complete the report, then on the way out of the office text message a significant other to coordinate dinner plans, and after dinner receive a crowd-based recommendation for a movie to watch. Each of these activities hinges on communication – a process in which infor­ mation is exchanged between individuals-if not active collaboration involv­ ing achieving or doing something with those individuals. Yet the wide variety of types of communication and collaboration raises the question of how to make sense of the design space. Which of these daily activities is more similar and more different from a design point of view? A descriptive model or frame­ work for design can help start putting this into perspective and provide the ability to recognize, compare, and discuss the features and demands of vari­ ous design contexts.

The traditional way to decompose collaborative interfaces is by using the time/ space matrix, which has four q11adrants: same time, same place (e.g., shared table display, wall display); same time, different place (e.g., teleconferencing); different time, same place ( e.g., public display); and different time, different place (e.g., e-mail, discussion forums, version control). The terms synchronous (same time), asynchronous (different time), co-located (same place), and rem.ate (different place) are often used. Certainly time and space are both important dimensions to consider when designing such collaboration tools, but the binary nature of the matrix is somewhat of an oversimplification. In terms of time, for instance, modern communication tools like Slack or Facebook blur the line bet,<l"een asyn­ chronous messaging and synchronous chat and are not distinctly asynchronous or synchronous.

11.2 Models of Col laboration 391

A more contemporary framework that operates at the mezzo and macro level is the Model of Coordinated Action (MoCA), which incorporates the tra­ ditional model but expands it into a set of seven dimensions and shifts toward a deeper understanding of "coordinated action" in order to encompass goal­ directed activities that are not traditionally considered to be work (Lee and Paine, 2015). Lee and Paine define coordinated action as the "interdependence of two or more actors who, in their individual activities, are working toiuards a particular goal through one or 1nore overlapping fields of action." This definition accounts for situat ions where participants may be collaborating in a diffuse or even indirect way, such as crowdsourcing or collaborative recommendation engines. A "field of action" need not be the same for all collaborators as they tak e on different tasks in order to accomplish some greater goal. The seven dimensions, described in detail next and shown in Fig . 11.2 are syrlchror licity, ph ysical distribution, scale, number of communities of practice, nascence, planned permanence, and turnover. In some cases, these dimensions will reflect on the design of the comm uni cation tool or platform, but just as often, the nature, qualities, and irltents of the people involved in the coordinated actions play just as large a role in achieving a successful outcome. Universal designs for collaboration and communication systems fluidly accommodate users across the spectr um of these dimensions.

FIGURE 11.2

Synchronicity

Physical Distribution

Scale (Number of Participants)

Number of Communities

of Practice

Nascence

Planned Permanence

Turnover

asynchronous

same location

2

0

routine

short-term

.. low

synchronous

different locations

N

N

developing

long-term

.. high

The seven dimensions of the MoCA model (Lee and Paine, 2015).

392 Chapter 11 Communication and Collaboration

11.2.1 Synchronicity Coordinated actions can take place along a spectrum of synchromcity, ranging from actions that are entirely asynchronous to those that are entirely synchro­ nous. Importantly, this dimension allows for actions to be a mixture of synchronous and asynchronous rather than enforcing a distinct boundary between extremes. In ongoing work processes, a larger context of asynchronous interaction often embeds episodic synchronous activity (Olson and Olson, 2000). Examples of more synchronous communication channels include voice or video conferencing (e.g., Skype), whereas more asynchronous channels include things like messaging systems or QI A forums (e.g., iMessage and Stack Overflow, respectively). The degree of synchronicity of a channel (i.e., the delay between turns) can also be a function of its context of use or social expectations. Imagine users in a chat session on their phones, happily exchanging messages in a near­ synchronous fashion when one chat partner introduces a large delay into the next response. Maybe the partner got distracted, or his or her attention otherwise shifted and was reprioritized, but the result is that the chat we11t from being more synchronous to more asynchronous.

Newer project 1nanagement tools reflect the idea that collaborations often require a mixture of asynchronous and synchronous communication and allow users options within that range. Google Docs is a writing program that allows collaborators to co-write texts. It allows different users to write asynchronously across different time zones or work shifts as well as allowing for real-time edit­ ing by multiple users. Moreover, for situations where there is a need to work concurrently and resolve questio11s, there is chat fu11ctionality built in so that collaborators can exchange messages and discuss any edits they may need to make (see Fig. 11.3).

11.2.2 Physical distribution Teams working together can exist along a continuum ranging from being at the same shared desk to the same room, bui lding, campus, city, country, continent, or planet. A collaboration can thus be more or less physically distributed. Despite all of the internet commumcation channels available to users, the actual physical distribution of collaborators still matters (Olson and Olson, 2000). Physical presence can afford unplanned interactions and rapport building that are unavailable through other channels-sometimes the trust built over such "watercooler" talk is essential to project success or resilience.

Pl1ysical location can be a proxy for cultural differences that might include expectations for things like pauses in conversation or who has permission to speak when and to whom (Olson and Olson, 2013). Different countries may have differ­ ent holiday schedules or work hours. For instance, in parts of the Middle East, Sunday is a work day, whereas in the Umted States it is 11ot. Time zones can make

11.2 Models of Col laboration 393

DTU16 Rolling agenda II f'tle Edit Vie-N ltlsert F«mat TOOis Tab'le Add-ons … S..n ewehengN

e, ,r. ,… r' '°"" • M.-dl119t • t~ … • 1, · U I il A · oo O W ■ =-i ■ tll · Mot-. •

FIGURE 11.3

– Agenda.:

• calling lnstructlons see below • chapter ,1.aws

o 2 sets of chapters due Oct 16 • d°"""),

o naming oon~tions o folders with figures o u,l)CJate t&ble of contents o u-pdale spreadsheet o note 10113.XSne When reedy

• re!erences dois??, sample roadmap???, quotes {no change) • MW(M'ku.odalt • flyer• improved, sta at bttp;hbcil2.g .OO)d,edu/OJUl§I • figures • pd)lishe, Into • Topica we dont et:Ner?77 robots, afftdive UI

MeetlnJ: Present Steve, Maxine, Catherine, Ben, Niltlas, Nick

• Renumbering and delivery for Oc:10ber 16 o Chapter 1 – ready 10 go (Ben) o Chapter 2 • ready 10 oo (Ben) o Chapter 3 – ready 10 go {Ben) o Chapter4 • ready 10 oo (&moll!) (N3<1as)

• Textbooks inste ad of research artlc:!es • New image • convergeneeldivetgenoe, double diamond

o Cha~er 5 – filclng and de-anlng up figures (Maxine) ■ Portabte usability setup in high resokllion • Catherine • try to help

o Chapter6 • almos.tready to go (Steve) • Need new apple IITlage (ourrent 100 low res)

o Chap<er 8 • almos.t ready to go (C&therine) • Perfeq;: . H#J !1we!ldone • WiUl>ei

(4) Celtlenne Pltlisant. Maxine _

(I me 144PM It k>oks lite we're converging en ow Oct 160, clelldllne ., cloet #'l'fOl"I• nMd tiny final~ Ofl rciutet?

The Google Docs interface showing how mu ltiple users can simultaneously edit a document. Note that co lored flags for different users offer concrete feedback that signals who is editing a particular part of the document. In the lower right corner, a chat box invites users to synchronously converse w ith each other around the document.

it difficult to sched ule synchro11ous communications that are convenient to all parties: a S p.m. call in Norway is an 8 a.m. call in California, at the extremes of the work day and probably not terribly convenient for either party. Who on the team deserves to be more inconvenienced in case a common time cannot be found?

11.2.3 Scale The number of participants involved, or the scale of the collaboration, is an important dimension that affects the nature and type of interactions that emerge. The difference is substantial between co-writing a paper with one other person and contributing to a Wikipedia article with 10 or 100 others. In smaller collabo­ rations, each person might know everyone else by name, whereas in much larger-scale collective actions, there may in fact be little direct contact between individuals. Many users may remain in lightweight contact as occasional lurk­ ers or provide fine-grained contributions like voting or tagging. Wikipedia man­ ages what is often a large scale of contributions from different users by allowing for a range of granularities of contributions, anywhere from fixing spelling or

394 Chapter 11 Communication and Co llaboration

✓ Yourd was~ .

Article Talk Flead Edit V',ew lllSIOfy * More • search

Cont8"1 mvfew and curat/on am lundamoota/ to Wiklm8dla pro/6cls. Do )'OU have Ideas to ® ~ improve how existing content is assessed and organized in your projects? The Inspire ~ campaign is offering funding for new projects in March. Help review and propose your ideas

today/

University of Maryland Human- Computer Interaction Lab

The Human-Computer lnteracUon Lab (HCtl) at the Unl'erslty of Marytand,

College Pa.r1t is one of the oldest and longest running HCI labs In the world. Founded in 1983 by Ben Shneiderman, HCIL members de6-gn, irrc>lement,

and evaluale new intet1ace lechnOlogies that are uni'o'ersally usable. usefUI.

efficient and appe~ 10 a titoad cross-section of l)eOl)lt, To this end, the HCIL deYelops advanced user Interfaces and de99'1 methods. Primary

ectM'tles Include oonabofative research, publication and the sponsorship of

open house&. woc1<shops and symposia.

The HCIL iS an inl«disciphnary lab YMl'I laculfy and stuoeints lrom lntOffl'latiOn

Studies, Computer Science, Education, English, Business, and Psychology. Current watt includes nt1'N approaches to inklrma1ion visualization, interfaces

for digital lbtaries, muhimecia resources tor learning oomrTU'lities., and

The HCL logo In cake torm

zooming u&er ~e,1aoe {ZUls). HCIL pioneorod technology design methods Wlh and for children, mobile and pon•bosod

con-.:,uting, and Instruments 1or evaluating use, 1n1ertace led'lnol0Qie$. tts many prl)leets are ShOWCa.sod at the atW'IUal

Human-Co ter Interaction Lab S sium.111

FIGURE 11 4 A Wikipedia page after editing. In the UI tabs at the top of the page, users can quickly access the "Edit" tab where they can direct ly ed it and then save a new version of the page . For more substantial editing decisions, t hey might visit the "Talk" tab to discuss with other editors first.

adding a missing comma to conceptually restructuring the article (see Fig. 11.4). In traditional organizations, the typical way to deal with larger-scale tasks is to introduce a hierarcl1y that decomposes tasks and clarifies authority and accoU11t­ ability. Hierarchical task decomposition, integration of work, and quality over ­ sight have been explored in large-scale collaborations like crowdwork (Kittur et al., 2013). As efforts scale, the role of leadership and exper tise becomes appar ent in e11su ring a successful outcome (Luther et al., 2010).

11.2.4 Number of communities of practice A community of practice refers to the idea that over time individuals form a group as they teach and learn from one another and develop coherent va lue s, norms, and practices (Wenger, 1998). A group or team may reflect many differ­ ent communities of practice that must come together to coordinate and work toward a common goal. The notion of interdisciplinarity is key here: When sci­ entists want to start a company around their brilliant new discovery, they need disciplines like engineering, law, accounting, communications, and business to develop that idea into a viable product for the market. Yet the ways of thinking and the vocabulary in these different communities of practice may differ

11.2 Models of Col laboration 395

substantially-to an engineer "code" is something that a computer interprets, whereas for a legal professional it may correspond to a statute that needs to be adhered to. Different communities may have different goals or notions of impact, differe11t standards, or differe11t toolsets that reflect disciplinary educa­ tional patterns. At 011e extreme of the spectrum, a group may consist of a single community of practice, whereas at the otl1er end, there is a huge diversity of people in the world that may come together from various disciplines, ways of working, language, and culture to collaborate. Working in diverse teams means tha t different communities of practice may need to be bridged.

11.2.5 Nascence

New groups of peop le are constantly popping into existence to coordinate their actions, while others have been around for a 1011g time. Some of these groups only last for a short while, such as for coordinating a response to a local natura l disaster, whereas other groups may be around for many years or even genera­ tions. Nascence refers to the degree to which coordinated actions are already established and routine or if they are un-established, new, and developing. For instance, early on, different (and potentially diverse) team members may need to aligi1 goals and develop common ground. The organization of collaborative work is often in flux on the birthing of a new group, team, or community, but that is not to say that communities that have been around mucl1 longer can't still continue to develop. Periods of routine work may need to adapt all of a sudden to new demands, contexts, or norms. Research has show11 that the char­ acteristics and behaviors of founders early in a group's lifespan predict how long it will survive (Kraut and Fiore, 2014). Actions like visiting the group fre­ quently, having multiple group administrators, and articulating a group description and logo during the group's most nascent stage (i.e., the first week) predicted group survival.

11.2.6 Planned permanence

Some coordinated actions are shorter-term, whereas others are longer -term. For instance, responding to a crisis event may take place over the course of hours, days, weeks, months, or even years, and it may be apparent at the outset based on the magnitude of the response needed that the timescale would fit into any of those buckets. Regardless of whether collaborations are temporary or permanent, the participants will need to develop shared vocabulary and coordinate work practices and output. When participants know that a collaboration should and -vill endure for a longer time frame, they may also begin to develop their own standards that coalesce ideas from different communities of practice. Plai1ning for longer-term collaboration can oftentimes entail a higher overl1ead in terms developing and agreeing on administrative and work frameworks or policies.

396 Chapter 11 Communication and Co llaboration

FIGURE 11.5 Th ree examp le badges used in the DUST alternate reality game (http://fal li ngdust.com) to signal different achievemen ts during co llaborative play.

11.2.7 Turnover Turnover refers to the stability of the people invo lved in a collaboration in terms of how frequent ly new participants enter and leave the group. On one end of the spectrum are coordinated actions that may have a very slow churn, such as an e-mail list of school administrators, whereas on the other end are collaborations where new people are constantly coming and going, such as online discussion boards that don't require registration. For instance, an analysis of the online commenting activity on The Econon1ist's Graphic Detai l blog showed that over the course of eight months, about 79% of users who commented did so only once, indicating a smaller group (21 % of users) who repeatedly left comments (Hullman et al., 2015). Such a high turnover and steady inflow of new contribu­ tors can pose difficulties to deve loping policies and behavioral expectations and norms for the group. One design approach toward this issue is to give users badges that indicate their tenure within the community or that otherwise mark them as "verified" or "trusted" (see Fig. 11.5). Arlother approach is to welcome newcomers to a community by creating positive onboarding experiences and interactions with established community members (Morgan et al., 2013).

11.3 Specific Goals and Contexts

People collaborate because doing so is satisfying or productive. Collaboration allows individuals to reap the emotional rewards of socializing and iI1teractiI1g with others, to accomplish greater goals than they could alone, or to meet and transact with people who they otherwise couldn't. In this section, a macro-HCI perspective is taken by exploring the dimensions of the MoCA model in connec­ tion with diverse contexts in which collaborations emerge. Contexts vary not only in terms of the goals and tasks that primarily concern users but also accord­ ing to the social and physical context (e.g., mobile, in a car, in a classroom, in a public space) as well as along the dimensions of MoCA.

11.3 Specific Goals and Contexts 397

11.3. 1 Communication and conversation

One of the essentia l coordinated actions that most people participate in on a daily basis is exchanging ideas, information, and knowledge with other people via conversation. Users can do so via their voice, by writing down those ideas, or by using their faces to emote and their bodies to gesture. Different conversation tools make possible the use of one modality or another, but people will make use of whatever channels they have available . For instance, on the telephone, users lose the ability to scowl at their interlocutor as a sign of disagreement, whereas on a Skype video conference, users do ha ve the ability to use nonverbal visua l cues like facial expressions, hand gestures, body posture, or direction of gaze in order to help express an idea. At the same time, while unhappy telephone users can't convey a scowl, they can still modulate their voice to make a disagreement known. Telepresence (see Chapter 7) goes even further by providing a panoramic multi-video view and a more imrnersive (e.g., 3-D virtual reality) experience or by physically extending one participant into the space of another using robotics. Design dimensions such as the physical e11vironment, mobility of participants, and visual feedback, among others, are factors in such systems (Rae et al., 2015).

Conversation systems often vary in the degree of synchronicity they suppor t. Voice or video conferencing systems tend to be highly synclronous, whereas chat systems can support synchronous or asynchronous modes, and discussion boards or e-mail listservs tend to be less synchronous. Some chat systems have explored the implications for how interac­ tions change when comm uni cations are ephemeral by design. For instance, Snap­ Chat (shown in Fig. 11.6) is a popular app that allows users to share silly photos, v ideos, and doodles with their friends that can only be viewed for up to 10 sec­ onds before they disappear. This raises an interesting design question about how limiting the planned per1nanence through a design cons traint can lead to interesting new genres of communication.

Conversation systems often also vary along the dimension of scale: chatting with a best friend over iMessage is a very differ­ ent kind of experience from participating

FIGURE 11.6 SnapChat is an app that allows for th e composit ion of photos, dood les, and emojis that when sent to friends can only be viewed for up to 10 seconds as shown by the sma ll clock with the "10" in the left corner of the composition UI.

398 Chapter 11 Communication and Collaboration

in an e-mail group listserv, which is different still from wri ting a public comment on a New York Tinies article. In large-scale discussions, the pace may be slower and users may have little or no knowledge of the identity, reputation, back­ ground, or physical location of other participants. In order to coalesce cortversa­ tions that are dispersed in a social network site like Twitter, the use of hashtags has emerged so that disconnected users can find each other by referencing and searching for the hashtag. Anonymity or even just the lack of persistent identity can result in flaming behavior, including swearing, name calling, or other ad hominem attacks (Diakopoulos and Naaman, 2011). Missing identity informa­ tion can mask large disparities in the number of co1nmunities of practice and the turnover of the participants making it difficult to build common ground and shared vocabulary.

11.3.2 Online markets

Buying, selling, and trading-they've been driving commerce for a few thou­ sand years. But the past 20 years have seen the nature of these activities change dramatical ly as people adopt the internet for shopping, trading, buying, sell­ ing, and de livery of goods. Online marketplaces like Amazon, Airbnb, eBay, and Etsy present a multitude of options, allowing buyers to connect and do business not only with traditional corporations but also with individual collec­ tors, artisans, or serv ice providers. This allows for hither to unseen scale and physical distribution of participants in a transaction. Etsy users can buy a hand­ crafted steampunk costume item from the other side of the world as easily as the other side of the city (though maybe with different shipping costs). Markets often tend to be low er on the synchronicity scale in order to accommodate con­ venience for buyers and sellers shopping and fulfilling orders on their own clock as needed. Engendering trust in participants who have never met a1td may not even have a lot of background on each other is a key challenge for designing effective online marketplaces.

To cope with the scale of people, items, and content that is present in online markets, collaborative filtering algorithms have been developed (Linden et al., 2003). One approach to collaborative filtering works by representing the pur­ chase or preference data of each individual in the market and then identifying other people who have a sim ilar profile. Items from other users that are similar to a given user are then ranked and presented to that user as "related." The think ­ ing is that if two users are similar in their preferences and purchases, then they might have good product recommendations for each other. A/hat's interesting about collaborati ve filtering is that it is an implicit form of collaboration: Two individuals may have never interacted directly or even know about each other ­ in fact, they may be completely anonymous to one another. Other forms of feed­ back have also emerged in onlin e markets. eBay, for instance, has a feedback score that is simply calcula ted as +1 poin t for a positive rating and -1 point for a

11.3 Specific Goals and Contexts 399

Star Colo r Numbe r of rati ngs

* Yellow 10 lo 49

* Blue 50 to 99

* Turquoise 100 to 499

Purple 500 lo 999

* Red 1,000 lo 4,999

* Green 5,000 to 9,999

Jr Yellow shooting star 10,000 lo 24,999

~ Turquoise shooting star 25,000 lo 49,999

~ Purple shooting star 50,000 to 99,999

; Red shooting slar 100,000 lo 499,999

ji Green shooting star 500,000 lo 999,999

• S~ver shooting star 1,000,000 or more (Wowl) J;i

F GURE 11.7 The eBay rating scale uses icons that correspond to different levels of positive feedback.

negative rating. Higher scores are signaled via icons of stars of variou s colors and styles being shown next to a user's account, allowing users to quickly assess whether they want to do business with each other (see Fig. 11.7).

11.3.3 Meeting coordination

Sometimes users mere ly need the aid of communication tools in order to coor­ dinate a real -world meeting time and location. Such tools can allow user s to create group s and communiti es that can come together as needed to coordinate IRL (in real life). A powerful example of this is the Meetup platform, which according to its website in 2015 claims to facilitate more than 9,000 local group meetings every day by helping those groups self-organize. The platform inte ­ grate s capabilities to schedule and locate meeting event s, to send e-mails out to group members, to record RSVPs and manage attendance, to upload and share media like photos and videos after the event, and to comment on events and share information or opinions that are persistent for others. Meetup is squarely targeted at supporting communities that are in a relatively nearby physical

400 Chapter 11 Communication and Collaboration

distribution, such as the same city. Its websi te's features are also mostly oriented toward the asynchronous end of the S1JnchronicihJ spectrum (i.e ., asynchronous planning for a synchronous event), making it ideal for casual users to come in and ou t of the event planning on tl1eir own time. The platform is agnostic to dimensions sucl1 as number of communities, turn.over, and planned permanence, allowing for a great deal of flexibility for group leaders to define tl1e scope of their community as needed.

Another very popular context for meeting coordination platforms is online dating sites. The needs of such tool s are very different to a platform like Meetup, though; the scale of a meeting, for instance, is (most often) fixed at two. Potential romantic partners need ways to break the ice in what can sometimes be an awkward situa tion, they need ways to chat some times synchrono u sly and sometimes asynchronously, and tl1ey need ways to arrai1ge for dates where they feel comfortable and safe. The ways in which users are able to portray their identity and personality are important. In late 2015, eHarmony boasted an average of 438 marriages a day as a result of partners meeting on the site, and the OKCupid site reported more than 7 million messages exchanged per day among hundreds of thousands of users. Yet the demands of the romantic meeting context are quite different than those of a more general -purpose social network site. Turnover of users is quite high as old users are matched and drop off the platform. The frequent desire to meet romantic partners face to face means that strong filtering for the physical distribution of matches is a key fea­ ture. The types of communications afforded need to support an entire range of planned pernianence from a sing le message exchange to a single IRL meeting to a mor e involved relationship that results in a long -term relationship. The com­ munication on such platforms can adapt as the p lanned permanence of the interactions shifts.

11.3.4 Creative production Whether it's developing new software, writing an online encyclopedia, remix­ ing or animating a movie, or conducting an international science experiment, big creative projects demand that users work together. Work needs to be broken down into pieces and re-assembled, contingenc ies and interdependencies require planning, quality must be ensured, different roles and skills must be brought to bear, and supporting administrative duties underlie it all. Because creative productions often involve original and innovative output, there is some times no obvious or clear path forward, making group leadership espe­ cially important. Creative collaborations touch on and encompass some of the previous contexts mentioned above, namely communication, conversation, and meeting coordination. In addition, there is often an informational substrate, suc h as data or media, that needs to be manag ed in the course of the creative work . For instance, Bootlegger is a mobile app (see Fig. 11.8) tha t facilitates

11.3 Specific Goals and Contexts 401

•• J ~· :: J:

FIGURE 11.8 The Bootlegger app al lows users to coordinate the creative production of videos around live events.

collabora tive work around the substrate of video, coordina ting its collec tion and editing around live even ts. In other applica tions, features like access control can become important so that some participants are only allowed to contribute at particular stages of the process or to specific tasks (Olson and Olson, 2013). Various platforms and tools exist to help facilitate these activities, from socia l coding platforms like GitHub to file-sharing tools like Dropbox and project management tools like Slack or Basecamp.

Because of the breadth of possibilities, collaborative creative endeavors can more or less exist almost anywhere within the design dimensions of the MoCA framework. For instance, the scale could be a coupl e software developers work­ ing on an open source widget, or it could be hundreds contributing to the Mozilla Firefox codebase. The physical distribution of work is often wide so that creative projects can tap talent wherever it may reside. Oftentimes the nu1nber of conununities of practice will be greater than one in order to marshal different peop le with different talents, such as an animator, sound designer, and pro­ grammer working on an interactive game together. As a result, leadership and experience are often needed to coordinate different ways of working and think­ ing about the project (Luther et al., 2010).

11.3.5 Crowdsourcing and crowdwork

A growing number of online services exist to help people find paid work as con­ tract ors online. While there are a range of crowdwork platforms available, such as Fiverr, CrowdFlower, or TaskRabbit, a representative example of a paid crowdwork system is the Amazon Mechanical Turk platform, where "request ­ ers" can specify a task (called a HIT which stands for Human Intelli gence Task), an amo unt to pay, and a time frame for the task, and workers (or "Turkers" as they're often called) can browse for available tasks and sign up to comp lete

402 Chapter 11 Communication and Co llaboration

those that they find interesting or rewarding. This allows requesters to tap labor across a wide physical distribution, and it allows workers flexibility to step in and out of working without having a formal employment relationship. Crowd work is in slight contrast to other forms of crowdsourcing, which includes otl1er activ­ ities like serious games or citizen science that motivate users to participate for reasons besides money. A popular crowdsourcing project was the New York Public Library's effort to digitize historical menus from the library's archives. It was able to transcr ibe 8,700 menu s in just four month s by placit1g the digitized images online and allowing visitors to click a menu item, type in what it was, and submit (see Fig. 11.9).

A model of crowdwork recently published by Kittur and colleagues (2013) articulates a number of research areas, including workflows that support task decompo sition, dependence, and synthesis; task assignment to match worker abilities and skills to tasks; hierarchy to create leadership structures; latency to support real-time tasks; synchronous collaboration; quality control; job and task design to maintain interes t; reputation and credentials; and moti va tion and rewards. The power imbalance between requesters and workers on man y crowdwork platforms also suggests the need to consider ethical issues and

MOSO A V , DHCKM BER 81, 1906 .SOUPS

~ uree or grren pt'M with rice …… . . .. ….. :M ~ onsornmc Pleury ………………. ………. … 25 (!jo1uato iJl X croutons …………………….. 25 ~ ock Turt.lc …………… ………………… 2~

~ p. fl ot or Cold Conso1n111e …………. . ……… . ……………………… . 16 Iii.SH

~& iled fresh mnckc rcl and stc"·ed pot3 lC)Cj in cream … …….. ….. ………….. . ……. ,t,'j ~ ed. smelts. sauce taru..rc ___ •····• .. , .. _, __ ,,, …………….. .. . …. , ..••……… GO

rie<l scal lops "'•ith bncon ………••…•……….•.•……..•••••.•••• ••.. . .•.•••••..•. 50 ~ ariuirtc herring au<l boiled polatoes ……… . …………….. . ………… 40

ENTRE.E.S ~~ •'ala Modo with potato dumpllng . …………………………………………………….. $0

amb etow with ve,l.etablea ……………………………………………………………….. $0 ~ ot ro.ut beef l!ollnd,iC:h ………………….. . …. … . … . .. . .. . . .. … …………………. ~

(';, oiled bone sirlo in steak a la Nh-ernaisc …………… … ………….. __ t 06 ~ mall temlerJoin steak, smnthucd onions and potal0t$ en casst ro1e70

~~ 1ldcr of 'Cal with salad ………. …. ………………… _.J,.. ……… ftO oiled plO'er on toast with green 1>eas …… . ……………… 80

~ utton clioJ)6 sautc, grten }X"a, 011d mn.-.hed potatocs ………………. . ……………………. 50 ~~ tbr~d patties a le Pi1111uderc …………………… __ ………… … .. …………… 50

.5.w~_l ':_t"~I ~ i~lncy will! P':!l~t~ pancake …………… ………………………… ….. . 60

FIGURE 11.9

0.25

/ 0,…….,Flewy 0.25

/ Ton:woSOUpAUX a.o.-

0.25

/ Mcdt1"nl,Soup …. / Ctlp, Hot CoMCJrlUrlt , …. / O,p, Cold°""""""' 0.15

/ lln>lo!Fmh>laclo…i And Sccwed Potatoes la

0-45

c…m / F>lolsmeo.,saua, .,..,..,. 0-5

/ """ Scollop, \'11!1 ….. o..s

/ ….,.izteH . AM Bolled .,._::c …

/ led"ALaJ.SodeWrth ……….. Imp

… / Lamb $:,e.w With

Vca,:tabla 0.5

/ Hoc Rout """5aNMch 6'15

/ Btolled Bone Slrloln Sink A La Nh-emailt

1.0

/ Smll!Tmdnto&ISl,u. Smolhcnd Onions Alld

.. , P1IQtoa F.DCUlttOle

/ Sboulder Of Veal V.1tll

"""' 0-5

A histor ical 1906 menu from Fleischmann's Bakery digitized by the New York Public Library in co llaboration with thousands of crowdsource volunteers who helped type in individua l menu items. Note the UI list at righ t reflects and allows for navigation of items to be digitized.

11.3 Specific Goals and Contexts 403

worker rights outside of traditional employment arrangements. Crowdwork may be characterized by high turnover and nascence since workers may come and go from participating in a given task of their own accord. Such work arrangements tend toward the short-term end of planned permanence, as a given requester may need a specific task accomplished after which time the labor force can be disassembled easily. Some of the research challerlges posited by Kittur et al. also touch on dime11sions of the MoCA framework, such as how to support crowdwork that is high in synchronicihJ and how to assign tasks to a diverse labor force that may draw on a higher number of co1nmunities of practice.

11.3.6 Entertainment and gaming It's human nature to seek amusement and mirth in the company of others. Many online services and communities exist to help meet the need to meet and "play" with one anothe r, including, for instance, massive mu ltiplaye r online role-playing games (MMORPGs) with titles like Ultima Online, World of Warcraft, and Star Wars: The Old Republic. Whether fantasy, sci-fi, or otherwise themed, people enjoy playing such immersive games as they allow not only for the tradi­ tional fun of games, with goals, scoring and advancement, competition, and reward mechanisms, but also for social interaction and teamwork in the pursuit of goals. Each player controls a character in a virtual world and develops an identity and an avatar with a unique constellation of skills and attributes. In World of Warcraft (WoW), for instance, p layers form gui lds that collaborate to engage in various quests, raids, and role playing (Rheingold, 2014). The guilds coordinate different members' skills in order to succeed and give guild mem­ bers ai1 opportunity to build team collaboration skills in a safe, playful environ­ ment. Early research on WoW studied the ways in which people engaged in guilds within the game and found that players used in-game relationships to meet new people as well as extend real-life relationships (Williams et al., 2006). The popular online comedy series The Guild lampoorls these permeable bound­ aries between in-game and real-life relationships.

Interesting ly, factors like the scale of the guild and the turnover- or, as Williams calls it the "guild churn"-are key in defining the nature of the interac­ tions that occur. For instance, larger-scale guilds tend to be more goal-oriented toward game goals, whereas smaller guilds tend to be more focused on social bonds (Williams et al., 2006). The nascence of a guild that was developing is sometimes found to create conflicts between players with different expectations for friendliness, sharing, and leadership.

Online role-playing and guild behavior are of course just one type of, albeit very popular, online play. Other online social games include those that are integrated into social network sites like Facebook or simply provide a portal through which players can find and compete against others in classic games like poker.

404 Chapter 11 Communication and Collaboration

11.3.7 Education Recent years have witnessed an explosion of interest in interactive course materials availab le online from platforms like Coursera, Udacity, and EdX. These courses have come to be known as n1assive open online courses (MOOCs) because they often offer open enrollment and attract anywhe re from 100 to 10,000 students. Online communication and collaboration systems have become common for distance educa tion courses both as supp lements to face-to-face classes as well as stand -alone offerings student s can engage with to suit their ongo ing learnin g needs. Such systems not only offer new ways for students to receive information like lectures but also enable possibilities to engage and learn from other students from across the globe, take interactive qui zzes and exams, and develop collabora tive class projects. One study found that a tool for arrang­ ing and gu idin g synchronous video discussions among culturally and geo ­ graphically diverse students in a MOOC led to better learning outcomes including higher performance on qui zzes and exams (Kulkarni et al., 2015). Even for campus-based courses, communication techno logy now provides a means for a rich, collaborative learning environment that exceeds the traditional classroom in its ability to connect students and make course materials available on an around-the-clock basis.

Online collaborative education in MOOCs defines a unique coordinate within tl1e MoCA model. The scale is poten tially immense, and the physical distribution can likewise be very broad. Because such courses attract an interna ­ tional audience of users who could not otherwise access educational opportu­ nities and because there may not be a good way to enforce prerequisites, the set of interacting students on these platforms may also come from very diverse con1munities of practice. An educational course is something with a defined start and end date, which defines a distinct planned per1nanence of a few weeks or months and which in turn means that the educational community around a certain course or topic is refreshed, or turned over, periodically. MOOCs are a11 active area of research that demand more study to assess and address how edu ­ cation can scale effectively.

11 .4 Design Considerations

The re is a catalogue of features that designers might design into communication and collaboration systems. Why are some features important, and how can they suppor t cer tain types of tasks or in terac tions? An excellent reference is Kraut and Resnick's book Building Successful Online Communities (2012), wlucl1 lays out a series of evidence-based design claims that connect cer tain observ ­ able conditions of a community to certain expected outcomes. An example of

11.4 Design Considerations 405

such a design claim from that reference is: "Pub licly displaying many examples of inappropriate behavior on the site leads members to believe such behavior is common and expected." The claim makes a specific connection between a pos­ sible design feature (i.e., displaying inappropriate behavior) and expected reception of that information by community members. However useful, such design claims can suffer the drawback of not being context-specific, and it is crucial to understand the context in which the designer is designing: not only the tasks but the diversity of participants along all of the dimensions discussed in Chapter 2, such as personality, cultural and international differences, older versus younger users, and cognitive or physical disabilities.

The remainder of this chapter examines design considerations rather than specific claims. Instead of making declarative statements about expec ted out­ comes from a particular feature, the goal is to help with understanding why each design dimension ought to be considered and to see the connection between a feature and the tasks that might need to be accomplished across the range of contexts as articulated in the last section of this chapter. Design considerations are organized according to their impetus: cognitive factors, individual factors, and collective factors.

11.4.1 Cognitive factors

Common ground Establishing common ground – the knowledge that com­ municators have in common – as well as jointly understood references during comm uni cation can be essen tial for effective collaboratio n. What do users mean when they say "this button" or "that menu"? If a user is standing next to some­ one and points with a finger at one of the buttons in an elevator, the user can be pretty sure that's the "this" being referred to. Pointing to and referencing objects is called deictic reference, or deixis. Other forms of reference include general (e.g., "upper left"), definite (e.g., using named entities like organization or place names), or detai led (e.g., described by distinctive attributes like "the red button") (Heer and Agrawal, 2008). When users engage in mediated communi­ cation, the channel may support referencing to a greater or lesser extent and thus require different levels of effort for people to achieve common ground (Olson and Olson, 2000). For instance, in video meetings, screens are often shared so that there is a common visual reference for discussion. But it can still be easy to get lost or not understand what users are talking about if they say "this button" because there may not be feedback on the sha red screen for pointing to the button that the spoken "this" refers to. In full video meetings, users will often gesture with their hands as they talk, which can provide deictic informa­ tion that makes what they're saying more easily and precisely understood. On social media, referencing is supported in several standard ways, including the @usemame syntax, which indicates a person with the given usemame is being referenced, and the #hashtag syntax, which indicates a topic is being referenced,

406 Chapter 11 Communication and Co llaboration

Nick Diakopoulos ndiakopoulos • Aug 6

Very interest ing comp utational journalism too l from @FILWD's lab

Char les Ornste in @char1esomstein

To analyze Yelp's 1.3 million health reviews, we used an amazing tool built for us by @nyupoly. Details: enrico.bertini.io/ revex

RElWEETS FAVORITES

4 7

9:56 AM – 6 Aug 2015 · Details

* ,II •••

FIGURE 11.10 References can be embedded in a tweet on Twitter in several ways, including referencing another person's account (in this example, @FILWD) as we ll as referencing and quoting another person's entire tweet, providing vita l context and citation for the information.

as well as embedding and directly referencing other's posts (see Fig. 11.10). In­ terfaces often include explicit referencing via threading of comments so that it's clear who is responding to whom in a larger conversation space. In designing communication systems, it's worthwhile to consider the nature of the tasks that need to be accomplished and how different forms of referencing may need to be supported in order to make those tasks efficient-or indeed possible at all.

Social cues Beyond reference, there are a variety of other nonverbal cues that can also enhance communication, including facial expressions, gaze direction, posture, proximity, and bodily orientation (Baym, 2015). For instance, giving two thumbs-up on a video conference sends the message to collaborators that they are in full agreement with a sugges tion. A furrowed brow or a smile and nod can convey a wealth of important feedback to a conversational partner who is explaining a difficult concept and wants to gauge understanding. Again, some media provide more possibilities for expressing these other channels of commurtication than others, yet even in "less ricl1" channels, like text, users adapt and develop mechanisms to convey social cues and emotions. The most popular of these are the emoticons that have become so prevalent in chat-based systems. Chat users routinely combine regular textual characters on the key­ board to indicate ai1d convey vario us emotions. Research on Twitter has show11 that emoticon use varies across culture, with Asian cultures favoring vertical emoticons using eye shape, (e.g.,"""" is a positive emoticorl meaning "happy")

11.4 Design Considerations 407

0.5%

0.4%

0.3%

0.2%

0.1"

0.0'6

F GURE 11. 11

DIVIDED BY A COMMON LANGUAGE (UK vs US vs Canada vs Australia English)

Emoji use varies across cultures.

• ~

$

poop

eun

cheers

money

and Western cultures favoring horizontal emoticons using mouth shape (e.g., ":p" is positive but with a tongue sticking out) (Park et al., 2013). Emojis are a cousin to emoticons but use actual iconographic representations of items (not just faces) to convey additiona l meaning and emotion in chat or social media. In an effort to respond to the desire for users to express emotional cues to one an­ other, Facebook began experimenting in late 2015 with emoji reactions to posts that allowed for affective responses to indicate "love," "haha," "yay," "wow," "sad," and "angry." Emojis have proliferated with their integration into popular mobile phone software that allows users to type using the icons. Again, not all emoji use is the same around the world: A 2014 study from Swiftkey found that even within English, there is substantial variation in use of icons across US, UK, Canadian, and Australian users as shown in Fig. 11.11.

Activity awareness The notion of social tran slucence argues that making socia l behavior vis ible facilitates awareness and ultimately accountability for one's actions (Gilbert, 2012). This might include making vis ible information such as "who sees vvhat," "who's done what," and "who knows that I know." For instance, collaborators often need to maintain an understanding of what others have accomp lished in a joint work activity around shared artifacts (Olson and Olson, 2013). Alerts and other interface signals are used to indicate

408 Chapter 11 Communication and Co llaboration

Contributors Traffic Commits Code frequency Punch card Network Members

Nov 16, 2014 – Sep 26, 2015 Contrit:uions to master, exclKling merge commits

..

.. 10

·– 2015 ·- – …. – £- NickDiakopoulos #1 simransachar W 66 commits 83,2'34 ..+ 158 •· Z3 commits 27,3.50 ++ 24,23,S •·

– – FIGURE 11.12

– • — –

ContnbutiOns: Commits •

– 12

– A Github dashboard charting project activity over t ime and ind icating two users w ho are active in the project including their overa ll coding activity as we ll as volume over time . More details on activity are available by drilling into individua l users.

when something has changed and by whom (e.g., "document updated on Monda y at 4:23 p.m. by Andy D."). Not only does this allow collaborators to track how the work is evolving , but it also provides some level of accountability. So, for instance, if the last ed itor mad e a chang e that needs to be reve rted , then the team knows whom to talk to about it. Awareness information has been stud ­ ied extensively in collaborative software production, and research shows that cues relating to recency and volume of activity, sequence of actions over time, attention to artifacts or peop le, and detailed information about an action can help support social inferences like interest and level of commitment, intentions be­ hind actions, importance to community, and personal relevance (Dabbish et al., 2012). Transparency of actions can also support learning by others who can then readily observe how a process works. On Github, a "contributors" tab on each project page offers an overview of activity and opportunities to drill into speci fic details of hov.r individuals have contributed to the project (see Fig. 11.12).

Interruptions Interruptions iI1 or by communication channels can affect their usability. For instance, during a synchronous phone call or video call, a chopp y connection due to poor internet service can cause unnatural break s in the con­ versational flow. This can be frustrating to recover from since pausing and turn taking are normal components to conversation, and technic al glitches cai1 make it unclear whether a pause is intended (e.g., while someone is thinking of what to say next) or imposed (e.g., by tl1e technology). For communication

11.4 Design Considerations 409

technologies that tend more toward the asynchronous end of the synchron icity dimension, interruptions pose a different problem: When a new message, text, or e-mail arrives, a new alert is usually generated to notify the user. In an office environment, it's not uncommon to have the phone blinking at the same time as the phone or tablet beep and an icon starts bounci11g at the bottom of the laptop screen. Research has examined the design space of interruptions and articulated various dimensions such as the symmetry of interruptions, the obtrusiveness (e.g., focal or peripl1eral), and the temporal gradient (e.g., historical, current, or predicted availabili ty) among otl1ers (Hincapie-Ramos et al., 2011). Where a communication technology fits within this design space will affect how users integrate it into their workflow.

11.4.2 Individual factors

Privacy One of the issues that arises in conjunction with greater activity aware­ ness in a system is the concomitant loss of pri, racy. If activity is collected implic­ itly based on actions within the system-rather than explicitly recorded or set by the user – this could affect the use or adoption of the system, as users may not want others to see every little action they take. In many cases, it's important to know who contributed what in a collaborative work project so that issues, corrections, or applause can be directed and so that the provenance of the result can be better understood. In more open forums like social network sites, there may also be a need for privacy in cases where users wan t to communicate more sensitive information only to certain connections. The idea of context collapse re­ flects the possibility that communications meant for a limited audience might in fact be visible beyond that audience. For instance, a user might not want her mother or boss to see the photo she posted at 4 a.m. from a club 011 Mallorca, but she could be entirely okay with her close friends seemg it. On Facebook, users can tweak their privacy settmgs for a number of things such as who can see a post, what people can see on their profile, whether to hide a given post on their timeline, and whether the sys tem suggests tags for photos based on facial recog­ nition teclmology . In some cases, user s' privacy may be vio lated because of al­ gorithmic inferences. For instance, by analyzing what users "like" on Facebook, algorithms can be used to predict a range of sensitive personal information like sexual orientation, personality trait s, ethni city, and mental heal th (Lee, 2014). When designing communication systems, it's important to consider situations or contexts in which users may want different amounts of privacy and to offer some degree of control, adaptability, or facility to opt out.

Identity Onlme communities open up a raft of questions regarding how peo­ ple represent and portray themselves when people's physical bodies are not shown and text or ava tar s become the primary medium of communi cation. In an online game like World of Warcraft, an older man could play the charac ter

410 Chapter 11 Communication and Collaboration

of a young woman, or a teenager could role-play as a sage and aged magician. Less-media-rich channels provide flexibility in how people choose to express their identity or identities. One of the most crucial elements to identity is the name chosen (Baym, 2015). In some cases, such as in financial transactions, real names are necessary, whereas sometimes pseudonyms (i.e., unique monikers not tied to real names) or even full anonymity is more appropriate. Some social network sites, like Facebook, have a real name policy, but oftentimes forums al­ low people to use pseudonyms, which allow people to have one or more identi­ ties that they can use to interact in the same community but in different ways. For instance, a well-to-do attorney in town occasionally comments on the lo­ cal paper's business articles, but sometimes also really wants to trash talk the local sports team without that being tied back to his lawyer identity. Having a differer1t pseudonym for each type of comment supports the user's needs it1 this case. Research has shown that certain topics on the anonymous messaging app Whisper, including NSFW ("Not Safe For Work"), LGBTQ ("Lesbian Gay Bisexual Transgender and Queer"), "Drugs and Alcohol," and "Black Markets," reflect considerably more desire for anonymity, with older users being more sensitive to the need for anonymity in these categories of content than younger users (Correa et al., 2015). The disinhibition afforded by anonymity, while it can lead to crude and anti-social behaviors and cue de-individuation and mob be­ havior, may also contribute to experimentation and creativity (Bernstein et al., 2011). Moreover, anonymity in online communication can reflect legitimate hu ­ man needs, such as a desire to make a confession of guilt or shame, share sensi ­ tive personal health information, test an idea without fear of reputational harm, or supply information for which users may be punished or fear other retribution if their true identity would become public (Diakopoulos and Naaman, 2011).

Trust and reputation Related to identity is the notion of reputation and the ability to develop a sense of trust around that reputation. Trust can be defined as a reliance on a piece of information (or a person) and is particularly important in marketplace contexts where goods or services may be sought or exchanged. For instance, Yelp is an online listing service that helps consumers find and evaluate services or businesses that they may be interested in patronizing. It allows prior patrons to write reviews and to leave a 1-to-5-star rating. These ratings and re­ views are then aggregated and presented back to others who are searching for that type of service. If users see a restaurant with 742 reviews and an average 4.5 stars, it's a pretty strong signal that they can trust that they will have a good meal there. Moreover, the interface allows the user to drill into the reviews and see individual write-ups, ratings, social activity, and feedback about other users, which can be helpful for evaluating their credibility (see Fig. 11.13). If someone who left a lousy review for a restaurant has previously written 20 other 4- and 5-star reviews, this is an indicator that the perso11 l1as a reputation for writit1g very positive reviews and that the 1-star restaurant review must have been a spectacularly bad experience. When users engage in large-scale marketplaces

011vid '• Ptofil •

a … – GI Au•—

m …..

FIGURE 11.13

David "Ayman" S. San f ranceco, CA

.. . ~ Q o,.$~ mm~

Reviews

so,u,,-o…

-eu,

–tt . o.. • 1ia, ~ ., __ C..,..C 111itt … .._

D DD D r., ,,.,,.,,

1"-'°doll. l lw.-elo•tlwlf'~ . INMllONM!b~Mld ,.._.~ttotlQWlhf'l'l'Jlllhftl~ Wfl{IS-..)'N._,,_'19 iv- h-'/1-._USA. t……,_..srd'IW-"~'"Y•pir.uoo-• t.'11119 tw _tu ~ta"'•"'° ll)'tlg l "°- '-IClllln:noltieollll Wltellog1-!l,mt•…ia-tpo,,wr.~—-·ff'

n….,_…-1.or,g,._ U• ..,_:.. CodMI..,_ ,,…,. …….. ,. Sln~ .Q. .. 19

a aa ac ,,.,,,,, ., .. ….. (),.10fo il if . ….. ~ (ol!M ~ · ,.ttt, -'-"O'/ ""'°" of,lf$ ,-,grv oertn0 NW rGll'IO'~,er;ers, l'Mltl.,_. __ 1Nn

– -• bOCll'I 'MIi' • -.tow o1 11• mr Dul 10li0y •• a 1111 or 'Mlill'I •~-'Ohtrt. TIMlltnttolor:Jltlla lltlta-""'<r.-.di'I__,._ boe:IWMt'ey- e …. ubtlood lln!lcltk~ …………. ,

11.4 Design Considerations 411

u.. MltMenil

0 c. ,, rn

■ -­ -♦ ,…..Oi,wld ..

"',m-..,~

-,~ow~• A 8oo'""81'1tt1:t t'lfwnll6liCW!IOCl 2

O r.i.i,oa • ~ o ljjl un 10

tt'.14 C. ntA; m "-

..

D II ~ Ill m lill 1122 1 11 • •

CI D • »

A user page on Yelp showing a variety of social activity context including vo lume of activity like reviews and photos, an "elite" badge, a graph of ratings previous ly made, feedback on the user's reviews (including if they were useful, funny, or cool), and other comp liments. This rich information can help others understand the reliability of this user.

like this, reputation systems that track and reflect the ratings and previou s behavior s of other actors become important components to communicating trustworthiness of those actors.

Motivation As in any interactive system, it is crucial to understand why people engage in collaboration and communication. There is a strong intrinsic motivation for interpersonal connectedness (Deci and Flaste, 1995), but there are a wealth of other reasons people also partake in and sustain interest in col­ laboration and communication, such as altruism, reciprocity , reputation or sta­ tus, and habit (Preece and Shneiderman, 2009). One way to understand this is using the Uses and Gratifications framework (Ruggiero, 2000), which describes how and why active media consumers engage media in order to satisfy spe­ cific needs. The framework offers a typology of gratifications that people typi­ cally seek from media, including exposure to information, an opportunity to test their personal identity and see where it fits, a chance to interact socially, and pure amusement and entertainment. In a study of an online news com­ menting community , all of these gratifications were observed when participants

412 Chapter 11 Communication and Collaboration

were asked about what motivated them to either write comments or read them (Diakopoulos and Naaman, 2011). Other studies of online communities have also applied the Uses and Gratifications framework and found that motivations can shift: Gratifications sought can differ from the gratifications that are ulti­ mately obtained (Lampe et al., 2010). Motivations can also change over time, behooving designers to understand how this affects turnover in a community. For instance, in the context of citizen science, research has shown that initial par­ ticip ation is motivated by things like personal interest, self-promotion, and so­ cial responsibility, whereas sustained longer-term participation relies on users receiving acknowledgment for their efforts and mentorship and being reminded of common goals (Rotman et al., 2014). In still other contexts of collaboration, like crowdwork, monetary rewards also come i11to play (Kittur et al., 2013). The diversity of motivations in online communities underscores tl1e mandate for designers to understand that diversity, to conduct surveys or focus groups to better understand the value that users are receiving, and to consider U11iversal usabil ity in designing u ser experiences that accommodate different motivations.

Leadership Leadership constitutes an ability to guide and direct a group's activities. It' s complicated in online scenarios because it can be harder to main ­ tain awareness of others' activities and to develop and maintain rapport and tru st (Olson and Olson, 2013). Leadership is particularly important when there is a high degree of task interdependence such that different team members are relying on each other for intermediate work products. Leaders are often respon­ sible for developing and managing a work plan, mediating disputes or other problems as they arise, clarifying roles and objectives, making sure the righ t informati on gets to the right team members, monitoring progress and qual­ ity, and enforcing policies. Leaders are often also the members of a group who synthesize and articulate higher-level group ideas or goals and who tend to take responsibility when problems emerge (Preece and Shneiderman, 2009). Research in collabora tive creativity tasks suggests that although formal leaders are present and will initiate projects, tools can support alternative leaderships styles where, for instance, leadership responsibilities can also be delegated and redistributed across the group (Luther et al., 2013). For designers of systems that enable creative production or crowd work, careful consideration should be given to how leaders can be empowered to initiate and lead groups, accomplish the other demands of managing group work, and maintain their motivation to continue in their role.

11.4.3 Collective factors

Deviance A social norm can be defined as "a stable, shared conception of the behavior appropriate or inappropriate to a given social context, that dictates expectancies of others' behavior, and provides 'rules' for one's own behavior"

11.4 Design Considerations 413

(McKirnan, 1980). Different societies, cultures, and sub-cultures may have their own social norms for what constitutes acceptable behavior within that group, but when a member of a group violates a social norm it is considered a socially deviant action. People who are known to intentionally violate group norms are often termed "trolls" and their activities are referred to as ''flaming". Trolls will post inflammatory comments that poke, prod, and antagonize other commu­ nity members for their own amusement (Lee and Kirn, 2015). Another form of deviant behavior is based around selfish manipulation – for instance, in online marketplaces a manipulator might create shill accounts and leave fake reviews in order to falsely hurt or help the reputation of another (Kiesler et al., 2012). In crowdwork platforms, deviant behavior might involve signing up for work and then doing just enough to make it appear as though the shoddy or rushed work is acceptable. People aren't always perfect, and in some cases they may not be aware of the specific norms of the community they're participating in. For these reasons, designers must be keen to consider various ways in which deviant behavior can be regulated or to make social norms more apparent and salien t so that non-normative bel,aviors are reduced and their impacts on the community lessened.

Moderation Given that deviant behavior is to be expected in some measure within online communities, one of the approaches to cope with the issue is to have moderators evaluate contributions and take various actions on the postings. For instance, a moderator could delete a post that harasses another user, or the moderator could demote the post and make it less visible . Moderators can be professionals, as is the case for the commenting system at the New York Tin1es, or they could be members of the community itself, such as on the Slashdot site. In some cases, automated text analysis algorithms are used to assess posts and determine if they use language in an un savory and potentially inappropriate way. On the Yelp platform, algorithms are used to automatically identify reviews that may be fake in an effort to minimize their impact; the fake reviews are de-emphasized in the interface but not deleted en tirely. Oftentimes community members are able to flag certain content that they believe is in violation of the community norms. These flags are then reviewed by professionals in order to make a final determination of whether the content should remain published (Diakopoulos and Naaman, 2011); however, such approaches strugg le to scale for very large communities. Of course, no moderation system is perfect, and people who have their postings removed will likely want to know why. Transparency in the moderation criteria can lend legitimacy to the process so that users understand how the decision ./as made (Kiesler et al., 2012). Another technique that can be used to moderate a community conversation" is to gag or ban users eitl-,er temp orarily or perma­ nently. Sometimes a cool-down period can be an effective way for signaling to users that they need to reform their behavior.

414 Chapter 11 Communication and Co llaboration

Policies and norms Policies, rules, and norms can be important signals to users in online communities so that they know what constitutes acceptable versus unacceptable behavior and so that protocols for adjudication of modera­ tion or other decisions are apparent. Knowing the etiquette for a given channel or community may not always be immediately apparent. Thus, policy docu­ ments are often posted in places where users can easily find them. For instance, on Reddit, a social commenting site, the Reddiquette for the site lists a variety of guidelines for behavior, including "Use proper grammar and spelling," "Look for the original source of content," and "Search for duplicates before posting." These rules of good behavior are useful for newcomers as well as existing users. Another way for users to learn about accepted norms is to observe and under­ stand others' behavior, including which behaviors are sanctioned and which are praised. System designers can make such behaviors more or less salient to ease the learnability of the system. For instance, on the New York Tin1es site, norms about the acceptability of comments are communicated by labeling ex­ ceptional comments as "Times Picks," which offers a valuable feedback signal to both the commenter as well as to the rest of the community (see Fig. 11.14). Policies and norms can be enforced either through technical regulation (i.e., the system makes it difficult to violate rules) or through social processes of regu ­ lation (i.e., rules may be broken but later sanctioned through social processes such as the moderation discussed above). The method of application of policies and rules is an important component to consider in a broader sociotechnical design of a communication technology. The way that users ,-vill behave using a tool is not just a matter of the tool's features but also of how other actors, like administrators, moderators, and other users, are perceived and act, including the ways in which policies are enforced and norms are made salient.

All 4447 Readers' Picks 1518 NYT Picks 85

Texas September 11, 201.l::

Say what you will about the Russians and Mr. Putin in particular .

… in peace with each other if only we would really try.

~ 2196 Recommend

FIGURE 11.14 In the New York Times commenting system, moderators mark some comments as "Times Picks" with a bright yellow badge, indicating they are exceptiona l comments and signaling norms about what cons t itutes an interes t ing and valuable con t ribution to the comment thread.

Researcher's Agenda 415

Practitioner's Summary

Communication and collaboration tools are continually evolving to support human interactions across the full range of human experience. While there are many positive outcomes derived from using such tools, designers mu st also be aware that negative behaviors are possible and should be prepared to consider mitigating design alternatives. It is essential to understand the myriad contexts in which users may employ communication and collaboration systems from conversations to markets, meetings and creative work, entertainment, crowd­ sourcing, and education. Models such as MoCA can help in thinking through these various contexts during tl1e design process to understand what may be similar or different about the particular design context being addressed. Other design considerations that can affect the user experience and usability of com­ munication and collaboration systems include common ground, social cues, activ ity awareness, interruption, privacy, identity, reputation, motivation, lead­ ership, deviance, moderation, and norms. Interface and experience design of communication tools must be an iterative endeavor as people and technology adapt and co-evolve.

Researcher's Agenda

There remaiI1s a rich variety of open questions that relate to the desigi1 and understanding of communication and collaboration tools. Perhaps most impor­ tantly, predictive theories that connect design decisions to specific outcomes still need to be developed. Designers will benefit from improved theories that can guide their work in making decisions in the contexts described in tlus chap­ ter. In line with this are questions that approach the larger macro questions of organizational and societal impacts of communication and collaboration sys­ tems: How will home and work life be changed? Can such technologies restore communi ty social capital, or will time onlme only iI1crease distance from neigh­ bors and colleagues? Will patients, consumers, and students become more or less informed and trusting? How wi ll important social issues relating to public health, democracy, international relations, and humanitarian crises be affected? Answering such questions w ill require taking a long view and examming behavior at the macro level. Some of the attraction for researchers stems from this vast uncharted territory: Theories are still needed, controlled studies are difficult to arrange, and analysis of big data has its own challenges. In short, there is a grand opportunity for researchers to influ ence a sti ll-emerging field and st udy some of the biggest questions of our time.

416 Chapter 11 Communication and Collaboration

WORLD WIDE WEB RESOURCES

www.pearsonglobaleditions.com/shneiderman

Using communication and collaboration tools is an effective way to deve lop

intuition for their design. Many sites and apps exist to experiment with, including:

• Facebook: http://www.facebook.com

• Twitter: http://www. twitter.com • Reddit: http://www.reddit.com

• Slack: http://www .slack .com • eBay: http://www.ebay.com

To get started with social network ana lysis, NodeXL is a powe rful too l that

fac ilitates bot h gathering and visualizing data:

• NodeXL: http://www.smrfoundation.org/tools/

An evolving list of data collection and analysis tools for social media is curated by:

• Deen Freelon:

http://dfreelon .org/2015/01 /22/soci a 1-media-col lecti on-tools-a -cu rated -list/

Discussion Questions

1. Take a position on whether you. feel user interfaces for work will remain iso­ lated or if they will become mor e collaborative. Presen t ev idence to support your argument.

2. How doe s collaborative filtering contribute to online marketit1g?

3. Differentiate the roles of face-to-face encounters and collaborative interfaces. Explain the limitations and benefits of each type of communication.

References 417

4. Below are the seven dimensions of the MoCA model (Lee an d Paine, 2015). Cite examples of each and how you feel it might influence a successful collaboration.

Synchronicity .. .. asynchro nous synchronous

Physical Distribution

diffe rent same locat ion locations

Scale (Number of Participants)

2 N

Number of Communi ties

of Practice 0 N

Nascence routine developing

Planned Permanence

short-term long-term

Turnover low high

5. Explain how collaborative interfaces can improve or harm teamwork.

6. Explain how an interface designer can protect us ers of a collaborative inter ­ face from hostiJe or malicious behav ior .

References

Baym, Nancy, Personal Connections in the Digital Age. Polity, 2nd Edition (2015).

Bernstein, M., Monroy-Hernandez, A., Harry, D., Andre , P., Panovich K., Vargas, G. 4chan and /b /: An analysis of anonymity and ephemerality in a large online community . Proceedings of the international Conference on Web and Social Media (ICWSM). (2011).

boyd, danah 1n., and Ellison, Nicole B., Social network sites: Definition, history, and scholarship, Journal of Cornputer-Mediated Cornrnunication 13, 1 (2007).

Bruckman, A., Lu ther, K., and Fies ler, C., When sho uld we use rea l names in published accounts of Internet research?, in Hargitta i, E., and Sandvig, C. (Editors), Digital Research Confidential: The Secrets of Studying Behavior Online, MIT Press (2015).

Burke, M., and Kraut, R., Using Facebook after losing a job: Differential benefits of strong and weak ties, Proceedings of the Computer Supported Cooperative Work and Social Co·mputing (CSCW) (2013).

418 Chapter 11 Communication and Collaboration

Correa , Denzil, Araujo Silva, L., Monda!, M., Benevenuto, F., and Gummadi, K. P., The many shades of anonymity: Charact erizing anonymous social media content , Proceedings of the International Conference on Web and Social Media (ICWSM) (2015).

Deci, Edward, and Flaste, Richard, Why We Do What We Do: Understanding Self-Motivation, Penguin Books (1995).

Dabbish, L, Stuart, C., Tsay, J., and Herbsleb, J., Social coding in GitHub: Transparency and collaboration in an open software repository, Proceedings of the Con1puter Supported Cooperative Work (CSCW) (2012).

Diakopoulos, N., De Choudhu ry, M., and Naama n, M., Finding and assessing social media infor1nation sources in the context of journalism, Proceedings of the ACM Conference on Hurnan Factors in Cornputing Systen1s, ACM Press, New York (2012).

Diakopoulos, N., and Naaman, M., Towards quality discourse in onl ine news comments, Proceedings of the Conference on Cornputer Supported Cooperative Work (CSCW) (2011).

Diakopoulos, N., Picking the NYT picks: Editorial criteria and automation in the cura­ tion of online news comments, #ISO/ Journal (2015).

Ellison, N ., Steinfie ld, C., and Lampe, C., The benefits of Facebook "friends:" Social capi tal and college students' use of on line social network sites, Journal of Computer­ Mediated Com,nunication (JCMC) 12 (2007).

Gilbert, E., Designing social translucence over social networks, Proceedings of the ACM Conference on Hun1an Factors in Computing Systems, ACM Press, New York (2012).

Hansen, D., Shneiderman, B., and Smith, M., Analyzing Social Media Networks with NodeXL: lnsights from a Connected World, Morgan Kaufmann (2011).

Heer, J., and Agrawal, M., Design considerations for collaborative visual analytics, lnforn1ation Visualization 7 (2008), 49- 62.

Hincapie -Ramos, J. D., Voida, S., and Mark, G., A design space analysis of ava ilability­ sharing systems, Proceedings of the Sy1nposiu1n on User Interface Software and TechnologtJ (UIST) (2011).

Hullman, J., Diakopoulos, N., Momeni, E., and Adar, E., Content, context, and critique: Commenting on a data visua lization blog, Proceedings of the Con·1puter Supported Cooperative Work and Social Cornputing (CSCW) (2015).

Kiesler, S., Kraut, R., Resnick, P ., and Kittur, A., Regulating behavior in online commu ­ nities, in Kraut, R., and Resnick, P. (Editors) , Building Successful Online Com1nunities: Evidence-Based Social Design, MIT Press (2012).

Kittur, A., Nickerson, J., Bernsetin, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M., and Horton, J., The future of crowd work, Proceedings of the Co,nputer Supported Cooperative Work and Social Computing (CSCW) (2013).

Kooti, Farshad, Yang, Haeryun, Cha, Meeyoung, Gummadi, Krishna P., and Mason, Winter A., The emergence of conventions in online social networks, Proceedings of the International Conference on Web and Social Media (ICWSM) (2012).

Kra1ner, A., Guillory, J., and Hancock, J., Experimental evidence of massive-scale emotional contagion through social networks, Proceedings of the National Acaden,ies of Science (PNAS) 111, 24 (2014).

Kraut, R., and Fiore, A., The role of founders in building online groups, Proceedings of the Co,nputer Supported Cooperative Work and Social Co1nputing (CSCW) (2014).

References 419

Kraut, R., and Resnick, P., Building Successful Online Co1nn1unities: Evidence-Based Social Design, MIT Press (2012)

Kuklarni, C., Cambre, J., Kotturi, Y., Bernstein, M., and Klemmer, S., Talkabout: Making di stance matter with sma ll gro up s in ma ssive classes, Proceedings of the Cornputer Supported Cooperative Work and Social Co1nputing (CSCW) (2015).

Lampe, C., Wash, R., Velasquez, A., and Ozkaya, E., Motivations to participate in online communities, Proceedings of the ACM Conference on Hun1an Factors in Con1puting Systems, ACM Press, New York (2010).

Lee, Charlotte, and Paine, Drew, From the matrix to a mode l of coordinated action (MoCA): A conceptual framework of and for CSCW, Proc. Cornputer Supported Cooperative Work and Social Co,nputing (CSCW) (2015).

Lee, Newton, Facebook Nation: Total inforn·1ation Awareness, 2nd Edition, Springer (2014).

Lee, So-Hyun, and Kim, Hee-Woong, Why people post benevolen t and malicious comments online, Com,nunications of the ACM 58, 11 (November 2015).

Leetaru, K., Data Mining Methods for the Content Analyst: An introduction to the Co-mputa­ tional Analysis of Content, Routledge Communica tion Series (2011).

Linden, G., Smith, B., and York, J., A1nazon.com recommendation s: Item-to-item collaborative filter ing, IEEE Internet Co,nputing 7, l (2003).

Lofland, L., and Lofland, L., Analyzing Social Settings: A Guide to Qualitative Observation and Analysis, 4th Edition, Wadsworth Publishing (2005).

Luther, K., Caine, K., Ziegler, K., and Bruckman, A., Why it works (when it works): Success factors in online creative collaboration , Proceedings of the ACM Conference on Supporting Group Work (GROUP) (2010).

Luther, K., Fiesler, C., and Bruckman, A., Redi stributing leadership in online crea tive collaboration, Proceedings of the Computer Supported Cooperative Work and Social Coniputing (CSCW) (2013).

McKirnan, D., The identification of dev iance: A conceptualization and initial test of a model of social norms, European Journal of Social Psychology 10, l (1980), 75-93.

Morgan, J., Bouterse, S., Walls, H., and Stierch, S., Tea and sympathy: Crafting po sitive new user experie nce s on Wikipedia, Proceedings of the Co111puter Supported Cooperative Work (CSCW) (2013).

Olson, Gary, M., and Olson, Judith S., Distance matters, Hu-m.an-Con1puter interaction 15, 2 (2000), 139- 178.

Olson, Judith S., and Olson, Gary M., Working Together Apart: Collaboration over the Internet, Synthesis Lectures on Human -Cent ered Informatics (Ed. John M. Carroll) (2013).

Park, J., Barash, V., Fink, C., Cha, M. Emoticon Sty le: Interpreting differen ces in emoticons across cul tures. Proceedings of the International Conference on Web and Social Media (lCWSM). (2013).

Phua, J., Participating in health issue-specific social networking sites to quit smoking: How does online social interconnectedness influence smoking cessation self-efficacy? Journal of Comn·1unication 63 (2013), 933-952.

Preece, J., and Shneiderman, B., The Read er-to-Leader Framework: Motivating technology-mediated social participation, AIS Transactions on Hurnan-Cornputer Interaction 1, 1 (2009).

420 Chapter 11 Communication and Collaboration

Rae, I., Venolia, G., Tang, J., and Molnar, D., A framework for understanding and designing telepresence, Proceedings of the Co1nputer Supported Cooperative Work and Social Computing (CSCW) (2015).

Rainie, Lee, and Wellman, Barry, Networked: The Nezu Social Operating Systen1, MIT Press (2012).

Rheingold, Howard, Net Srnart: How to Thrive Online, The MIT Press (2014).

Riffe, D., Lacy, S., and Fico, F., Analyzing Media Messages: Using Quantitative Content Analysis in Research, 3rd Edition, Rout ledge Communication (2013).

Ronson, J., So You've Been Publicly Shamed, Riverhead Books (2015).

Rotman, D., Hammock, J., Preece, J., Hansen, D., Boston, C., Bowser, A ., and He, Y., Motivations affecting initial and long-term participation in citizen science projec ts in thr ee countries, Proc. iConference (2014).

Ruggiero, T., Uses and Grats theory in the 21st century, Mass Communication & Society 3, 1 (2000).

Smith, M. A., Rainie, L., Himelboim, I., and Shneiderman, B., Mapping Twitter topic networks: From polarized crowds to commun ity clusters, Pew Research Center Report (2014).

Smith, A., and Page, D., U.S. smartp hone use in 2015, Pezv Research Center Report (2015).

Turkle, S., Reclai,ning Conversation: The Power of Talk in a Digital Age, Penguin Press (2015).

Vieweg, S., Hughes, A. L., Starbird, K., and Palen, L., Microblogging during two natural hazards events: What Twitter may contribute to situational awareness, Proceedings of the Conference on. Hun1an Factors in Con1puting Systerns (CHI) (2010).

Vitak, J., Shilton, K., and Ashktorab, Z., Beyond the Belmont Principles: Ethical challenges, practices, and beliefs in th e online data research conununity, Proceedings of the Cornputer Supported Cooperative Work and Social Co,nputing (CSCW) {2016).

Wenger, Etienne, Communities of Practice: Learning, Meaning, an.d Identity, Cambridge University Press (1998).

Williams, D., Ducheneaut, N., Xiong, L., Zhang, Y., Yee, N., and Nickell, E., From tree house to barracks: The social life of guilds in World of Warcraft, Gaines and Culture 14 (2006).