Usability-raw
From Grokdoc
Contents |
Usability-raw
This page is for raw reports of usability study sessions. Describe the background of the participant, the computer hardware and software, and what is observed during the session.
Never used a computer
NU1
Windows user, never used Linux
WU1
Transcript
A neighbor came to use my Los Alamos Computers laptop with 2GHz CPU, 1GB RAM, 60GB hard drive, RH9.
I spent a little over two hours this morning with a neighbor who had used Windows for several years, and could do email, web browsing, printing, and burning CDS, but not much else in Windows, and had no Linux experience at all.
I set up a user-id for him on my laptop RH9 machine, with default configuration, and he started by successfully logging off my user-id. Here are my notes from the session.
9:18 log out. OK. Typed user-id with a capital letter. I had warned him, but he forgot, and I had to remind him. Second try worked.
I told him to bring up the internet. He didn't know that he needed a web browser, and started Kmail. I told him that was not what was wanted, but it took a while for him to find Mozilla on the main menu (he did not notice the Mozilla icon next to the Redhat main menu icon).
Mozilla started with the Redhat Welcome as home page and DSL configured. I asked him to display another web page, and suggested Google. He tried typing in www.Google.com and clicking Search. This did not work, and he could not figure out what to do next. I told him to click Help. The Help page displayed, but it did not show the same welcome page that actually appeared on starting the browser.
9:31. He started navigating Help, but it took several tries to find "Move to another page", and he did not know the meaning of "location bar". After reading Help, he exited it before he had memorized what he read. (It reminded me of the first thing the 911 operator says: "Don't hang up until I tell you to.") After restarting Help and looking again, he left Help running, went back to the browser, entered the URL, and typed the ENTER key, which is all that he had been missing.
Google did not start because I had accidently left the network disabled. Since I felt that this was beyond the scope of the usability test, I started the network for him, and he was able to go to www.google.com.
9:45. He tried to enter a search without putting in a keyword, but realized his mistake, and his second try worked to get a list of links. He linked to pages, used the Back button. I asked him to make Google his home page, and, when he mentioned Favorites, I switched to making a bookmark for Google.
He tried to do this using Save As, then corrected himself, and after some looking around found a way to do it using one of the top menus. He then went via a link to another page, and used the bookmark to go back to Google.
9:55. I asked again for him to make Google his home page. He went to Help, and the machine hung while he was trying to select a command from a context menu. Neither of us had any good ideas until I suggested the ESC key. After two tries, the machine came back.
He wanted to know what a side bar was, since descriptions of side bars appeared when "Customizing Mozilla" was selected. He tried "Managing Profiles". No information on setting the home page. He needed to look at the browser and Help pages at the same time, and spent a while figuring out how to maximze, restore, and size pages. He was slowed down because the Mozilla default is to use maximum size for the restored page.
The directions for setting the home page were finally found by going to the Viewing your Home Page section, and linking to specifying how Mozilla starts up.
10:11 The Help directions were not quite enough. He misread the bullets as being sequential operations rather than alternatives. After getting past this problem, he could not figure out how to complete the operation, and I finally had to tell him to click the OK button. I then explained that Mozilla is a web browser like Netscape.
10:27. I asked him to write a letter. He found and executed Open Office. OO asked for address data source. He correctly decided to put the address in manually, and exited the box. He accidently went to Mozilla, but when I told him he was in the wrong place, he went back to OO. He wrote the letter, and saved it. He performed the right action to print the letter, if the printer had been connected. He exited OO.
I asked him to pretend that he had forgotten where the file was, and to find it again. He executed OO, and opened the file from the recent files list. I asked him to pretend that the file was so old that it was not on the list. He executed the Open function in OO, and found it in his home directory.
I asked him to pretend that he didn't even know what directory it was in, and to find it without opening OO. He used the main menu, tried Window preferences, and then did a search for test2, which was not found. Surprised, I suggested that he try to find it via the Browse operation. After a couple of tries, he found that the file name was test2.sxw, so he tried the search again, and it worked.
10:40. I was thinking of a file manager all along, so I asked him if he could find the file via a file manager. He could find a file manage, but I told him to keep looking. Finally, he found Home Folder in the main menu, and brought up Nautilus.
10:52. I asked him to do email. He found and executed Evolution. He started to fill in the setup information, but did not know what POP meant. Since we were running out of time, I told him to select POP. The next box was Receiving Email, and it took me a few minutes to discover that he did not have enough information to fill in the Host field, and neither did I. F1 did not help. He used Google to search for information on the net, but, of course, the general information there did not help. I bypassed the problem by using my email information for email address, host name, and SMTP address.
At this point, he received some email that had been sent to me, composed a reply, and sent it.
It took over two hours to get about the same amount done that I normally do in the first two or three minutes after I start using the computer.
I plan to respond to this post with an analysis of the usability implications. What usability implication can you find?
Comments
First, a defense against some criticisms. I agree with PJ that, at first, we do not want a formal usability test: we want to get the creativity that goes with chaos.
I had intended to have email already set up, but when I failed, I added it to the test, and my neighbor and I both worked on it.
My experience with Linux is that it is very easy to use for tasks that you already know how to do, but, whenvever you need to do something new, you run into a tree of earlier things that you need to learn and do first. I wanted to verify that my neighbor, with no prior experience in Linux would see the same phenomenon. I didn't want him to try the supposedly more complex tasks of installation and configuration for this first test, so I gave him a system that had started pre-configured, but had been further configured by an amateur (me).
The questions I am trying to investigate are: what problems does a Linux user encounter when trying to do something new; what usability features of Linux alleviate, or would alleviate such problems?
I am not equipped to answer such questions fully, but I can try things out, and see what happens. I can encourage others to ask similar questions. I am confident that if thousands of us do that, something useful will emerge, and that may include a formal comparison of the features of Windows, Linux, and MacIntosh that contribute most to usability .
Several of my conclusions might seem like the classic proof that all Indians walk single file: the one I saw did. However, all investigations have to start with guesses at the general principals governing random experiences, and this one is no exception. This test does not prove anything. It is a beginning of a process of collection of data that will be needed to formulate hypotheses to be tested. Formal tests will have to wait for refinements of the data collection process and formulation of the hypotheses to be tested.
What did I conclude from the test?
My neighbor started without enough knowlege to see what was on the display screen and read the words that described what computer facilities were available. I tried to state the tasks that I wanted him to accomplish in the simplest, most general terms, and to avoid giving him hints about what tools he needed to accomplish the tasks.
He ran into problems trying to do the simplest of tasks, but once started, he quickly learned to do more complex tasks, and ended able to access the web, use online help and do Google searches to get information to solve problems, set Mozilla preferences, write and save a letter in Open Office, find files using several techniques, send and receive email.
This is a large percentage of what I have to do daily on my computer. I could have done the same work in 2 or 3 minutes, but now he can too. On the other hand, I am sure that he would experience the same kind of difficulties getting started with anything new on the computer, and would not usually have a mentor sitting with him to stop him when he goes off in the wrong direction.
Usability problems encountered.
1. Logon should have made it clear to him that the user name and password were case sensitive 2. A Welcome dialog should have made sure that he knew the basic terminology, how to find help, what Linux applications on the system corresponded to Windows applications for performing basic tasks, and how to get started on each of the basic tasks. 3. Online help should be user task oriented, explain terminology, and allow him to find information in an organized way, starting with just a vague idea of the information needed. Attention to detail is needed: e.g., failure to specify that OK should be clicked was part of the reason he missed the entire point of the explanation of how to set the home page. 4. Many of the stumbling blocks he encountered would have been bypassed more easily if clear error messages had occurred.
WU2
I'm keeping a running log on My page of my first exposure to distro selection, downloading, installing ... and, if I get it running! ... using...
Linux with Senior Citizens
SC1
Transcript
I am a senior citizen, well, the local senior center declares me a senior; I'm 59. I retired in 2001 after 30+ years in the software development industry. I do various volunteer work for the center.
In early 2003, the center was given several computers, surplused from local government offices. These machines were complete systems, all fully functional, capable, but older machines (Pentium III, 300MHz). The disks of these machines had been securely wiped before they were given to the center.
I volunteered to set up a computer lab. The center was unwilling to spring for the bux to buy Win98SE for the machines. The local ILEC donated bandwidth and installed the inside wiring. I donated a router/firewall. I installed RedHat 9.0 using NIS for login authentication. I prepared a brief user's guide, showing what I believed to be the "installation specific" thingies that any user unfamiliar with a new environment would need to know. I provided paper forms for registering to get an account in the lab. I gave personal hands on tutorials to new account holders. I prepared and delivered a 4 session, 2 hours per session, class for those completely unfamiliar with computers.
After one year of experience with the lab, the center decided to close it down because there were few who had signed up for accounts, few of those actually used the lab. In early June 2004, the lab was disassembled and the computers were scrapped.
Why was this computer lab unsuccesful?
I believe that there are a number of factors that made this project unsuccessful. No single cause did it in, but a combination of factors.
The community is a small town in a rural lumber industry county. The community is below national median in education, economic achievement. The community, like others in rural, agrarian areas has some significant demographic "problems": it is weighted heavily to older people; it is heavily blue collar; it has a powerful "no-growth" county and city government. Think right-wing eco-terrorists.
The center has a unique clientele. The average age of the members at the senior center is 76 (min 50, max 93). With a membership of over 600, only about 50 regularly come in for any of the activities; it is believed that the others contribute because it is a "good thing to do". Those regularly attending are older, more infirm, more socially isolated in the community. Very few were from office centered backgrounds where they would have used computers at work.
My personal observations suggests that those who came to the center regularly were not interested in the internet. For example, when I asked one person who had gotten an account on the lab why she had done it, she responded that when she complained to her children for not writing, they told her to get email. She was resistant to training, and did only the minimum to send and receive email. She was easily confused when anything out of the ordinary happened, such as when her kids would send pictures of the grand kids; she forgot one session to the next how to view the attachments.
Others observed that many were blatently hostile to computers in the center. One fellow was constantly recommending that the lab be dismantled and the floor space reallocated to some other purpose. He never suggested an alternative purpose, just that the lab be dismantled.
When I taught one class, a hands on lecture-lab, one gentleman was openly hostile. He complained about "having to be there" (I believe his wife made hime go to the class). In his frustration, he beat on the keyboard, threw the mouse and swore profusively. He was excused from the remainder of the class.
There was good news. A few, with some prior computer experience, picked up Linux in a flash. The brief user guide was enough to get them started with a few minutes hand holding. All transferred their Windows experience directly to using Mozilla for web browsing, email and usenet. One gentleman, over 85 years old, was a regular, 3 times per week. He did email with his children and grandchildren; he surfed the web.
In conclusion, It comes down to no failure of Linux itself. Where individuals wanted to learn it and were physically able (no dementia, no severe physical limitations) they got it quickly and used it effectively. The lab failed because the candidate clientele was just not interested. You just cannot teach a pig to sing. It wastes your time and it bothers the pig.
As a footnote, one of my volunteer activities was helping seniors at home with computer problems there. The common thead was that all, without exception, who had computers at home who had asked for help, had been given computers by their children for the purpose of communicating via email. Without exception, these computers had Windows 9x. Without exception, all of these computers had one or more spyware and or worms or viruses. The users were absolutely clueless and resistant to learning more than was the absolute minimum to send and receive email and to surf the web. As a result of this experience, I have become reluctant to help with "tech support" tasks for the center's members.
