Examining Student Perception of an Open Statistics Book

The rise of Open Educational Resources (OER) research provides data that Open Textbooks and other forms of OER may be one cost saving approach for college and university students. Yet little research has been conducted around the attitudes and perceptions of the students using these Open Textbooks. This paper examines the perceptions that students have of the different versions of an open statistics textbook used over several years in one community college. Survey results show that students generally had at least as good of an experience using the open textbook compared to traditional textbooks. Suggestions for further research are discussed.


Introduction
For community college students, textbook expenses and course supplies can be more expensive than tuition and fees.For example, two twelve-credit semesters of tuition at California community colleges cost $1104.If students take a total of ten classes, and those textbooks cost an average of ninety dollars each (Hilton, Robinson, Wiley & Ackerman, 2014), their textbook costs will approach that of tuition.Reducing costs of education can aid students in lessening their debt and potentially help them complete their education faster.Shrinking college budgets may make decreasing tuition costs difficult; however, there are promising possibilities in terms of lowering the costs of textbooks.Open Educational Resources (OER) are one possible answer to lowering the cost of student educational expenses.OER are defined as "teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use and re-purposing by others" (Hewlett, 2013).
One of the best-known and earliest examples of OER is the Massachusetts Institute of Technology (MIT) OpenCourseWare program.This program increased access to MIT's teaching materials and has expanded to a variety of websites where OER are available for teachers to use as substitutes for traditional textbooks.In the past decade, many OER have been made available, and today, many websites provide college instructors with the ability to locate OER that could be substituted for traditional learning materials.Because textbooks are traditionally used as a staple in college Open Praxis, vol. 8 issue 3, July-September 2016, pp. 265-276 courses, open textbooks, a subset of OER, are among the most useful OER to potentially use in the college classroom.The Minnesota Open Textbook Library (http://open.umn.edu/opentextbooks/) and California's Cool4Ed project (http://cool4ed.org/)are examples of searchable directories of open textbooks that include faculty reviews of these textbooks.Yet little research has been conducted to understand the student perceptions and attitudes of those actually using the OER textbooks.
The purpose of the present study is to examine student perceptions of different versions of an open statistics textbook over several years at a community college in California.The overall research questions for this study are: 1) What are student's perceptions of overall quality of an open textbook compared to traditional textbooks?2) What are student's perceptions of cost of an open textbook compared to traditional textbooks?
We begin with a review of literature of open textbooks broadly and then provide specific background on the textbook that is the focus of the present study.

Open Educational Resources
In addition to the above mentioned definition of OER, the phrase "Open Educational Resources" has also been characterized as, "The open provision of educational resources, enabled by information and communication technologies, for consultation, use and adaptation by a community of users for non-commercial purposes" (UNESCO, 2002, p. 24).Wiley, Bliss and McEwen (2014) provide an overview regarding the history of OER, including some challenges to OER such as business models of sustainability and discoverability.However, one clear and immediate advantage of open textbooks is cost-savings to students, who spend approximately $90.00 per class on textbooks (Hilton et al., 2014).Yet, for faculty members who make decisions about which textbooks to adopt, more is required than simple cost savings; for them, other factors such as student performance may be the most vital issue.Thus, Allen and Seaman (2014) found that college professors rate "proven efficacy" and "trusted quality" as the two most important criteria for selecting teaching resources.
Several studies indicate that OER have similar efficacy to traditional textbooks.For example, Allen, Guzman-Alvarez, Molinaro and Larsen (2015) examined an OER that was substituted for a chemistry textbook.An experimental class of 478 students used the OER, while a control class of 448 used a traditional textbook.In order to minimize confounding variables, these two classes were taught by the same faculty member and teaching assistants.Both sets of classes used identical midterm and final exams and were taught at back-to-back hours.The researchers also used pretests at the beginning of the semester to determine that there were no significant differences between the groups.Although there was a large difference in the cost of the resources, the researchers found no significant differences in the tests scores between the two groups.Hilton (2016) reviewed eight additional efficacy studies focusing on OER and determined that, in general, students perform as well or better when OER are implemented.
Hilton (2016) also reviewed nine studies where teachers and/or students share their perceptions about the quality of OER.A general finding from these studies is that approximately half of teachers and students find OER to be of equal quality with traditional textbooks, with a larger number of students and faculty reporting that OER had higher quality than traditional textbooks than those who stated that traditional learning materials had higher quality.Examining Student Perception of an Open Statistics Book Open Praxis, vol. 8 issue 3, July-September 2016, pp. 265-276 Many of the studies regarding student and faculty perceptions of OER ask for the opinions of students regarding several different textbooks, or in some instances, a small sample discussing a single text (e.g., Petrides, Jimes, Middleton-Detzner, Walling, & Weiss, 2011;Lindshield & Adhikari, 2013).This present study was designed to examine student perceptions of quality of a single OER textbook, originally known as Collaborative Statistics, across several years and versions.We next describe the history and background of Collaborative Statistics, later renamed Introductory Statistics, the textbook that is the subject of the present study.

History and background of Collaborative Statistics
Elementary Statistics at De Anza College is generally considered to be a service course.Students planning on transferring to a California public university are required to complete a college level mathematics course before transfer.Many majors, excluding engineering and mathematics, require this particular algebra-based statistics course.In addition, as the course can be used to satisfy most other majors' mathematics graduation course requirements, Elementary Statistics has become the default college level mathematics course for non-STEM (science, technology, engineering and mathematics) students.
Barbara Illowsky and Susan Dean (now retired), mathematics faculty at De Anza College in Cupertino, CA, have been teaching introductory statistics courses for over 25 years to hundreds of students each year.In the mid-1990s, they wrote Collaborative Statistics, a textbook that incorporated technology, multiculturalism, collaborative learning, writing answers, data driven problems, and was at a reading level suitable for English Second Language students.
In 2006, FHDA and Connexions (a division of Rice University) received a grant from The William and Flora Hewlett Foundation to develop a prototype community college OER textbook, as well as train educators in the adoption and use of open textbooks.Dean and Illowsky worked with the Connexions team to produce the OER textbook in an attempt to produce a high quality open text that could be a positive disruptor in the market.Students and faculty were able to access the book freely from the Connexions site, print their own text, modify the text to suit the needs of individual faculty members, or purchase a hard copy version (Illowsky & Dean, 2008).
The pace of innovation accelerated as an ecosystem around OER evolved.The level of innovation around the original work has greatly exceeded the textbook authors' expectations.One such way was the multiple industry innovations derived from the book, such as the iBook textbook version, Kno version, WebAssign's learning system integration, and a new edition of the textbook called Introductory Statistics that was published by OpenStax College in 2014.Collaborative Statistics/ Introductory Statistics became the prototype for OpenStax College's open textbook model.Dr.Richard Baraniuk (2016), OpenStax College at Rice University said, "This is the project that really started it for us.I doubt that we would be reaching nearly 1,000 adoptions, hundreds of thousands of students, and millions of web learners if it weren't for Collaborative Statistics."

Differences between Collaborative Statistics and Introductory Statistics
Introductory Statistics is an updated version of the original Collaborative Statistics.The original text was in black and white only.The printed version appeared as black and white, camera-ready text.The online version was ADA compliant, but still without color.OpenStax College added color and formatting to give the appearance, both online and when printed, of a more traditional textbook.OpenStax College upgraded the graphs and images.It also provided more examples, narrative, and editing.In addition, faculty using the text from a variety of colleges around the United States  (OpenStax, 2013).
To date, one study has been performed on this textbook.Petrides et al. (2011), surveyed instructors and students who used Collaborative Statistics.In total, 31 instructors and 45 students shared their perspective on Collaborative Statistics."Cost reduction for students was the most significant factor influencing faculty adoption of open textbooks" (Petrides et al., 2011, p. 43), partly because it increased student access.Researchers also found that 65% of students surveyed reported a preference for using open textbooks in the future because they stated that they are generally easier to use.Our purpose in this study is to further examine the perceptions that students have of the different versions of this textbook over several years.

Methods
The research in this present study was conducted between 2013-2015 on student use and perceptions of Collaborative Statistics and Introductory Statistics at De Anza College.De Anza College is a large suburban community college in California, located in the region known as Silicon Valley.It operates on the quarter system.The college serves approximately 23,000 students (over 40% full-time).Locally, it has the highest graduation rate (associate degree) with over 60% of its full-time students earning the degree within three years.Asians (38%) and Latinos (24%) are the two largest ethnic groups at the college.To evaluate student perceptions, a questionnaire was developed to identify the student assessment and perceptions of the open textbook.The initial surveys were on the book when it was known as Collaborative Statistics, with the final survey being done on the later version of the book, known as Introductory Statistics produced by OpenStax College.

Collaborative Statistics Version of the Textbook
The initial questionnaire was sent to all of the students who used this text in the spring and fall 2013 quarters at De Anza College to participate by filling out a survey questionnaire.Requests were made via an email, which contained a link to the questionnaire.The requests were sent to statistics instructors towards the end of each respective term, after the last date for students to withdraw from the course, with an invitation for them to pass the questionnaire on to their students.Seventeen out of a possible 28 different instructors had students from their classes that submitted responses.The questionnaire included multiple-choice, multiple-response, and text entry questions.This survey was based in part on a survey utilized by Bliss, Hilton, Wiley & Thanos (2013).

Student Questionnaire
While taking the questionnaire, students would see between sixteen and nineteen questions depending on the answers they chose.No responses were mandatory, so a student could skip any of the questions if they chose to do so.The first few questions were basic demographic questions (e.g.instructor, gender).The next questions asked about students' financial situations, such as if they had received loans, Pell Grants, or fee waivers to fund their education.Other questions asked about student textbook usage, both in general and specifically in their statistics class.The last group of questions inquired about students' usage and perceptions of the open statistics textbook.Two questions inquired how often they used the textbook and what version they used most.Another examined how they would rate the quality of the text as compared to other textbooks and then asked them to comment briefly on their response.The questionnaire did not explicitly define the Examining Student Perception of an Open Statistics Book Open Praxis, vol.8 issue 3, July-September 2016, pp.265-276 term "quality," which was left open to student interpretation in order to accurately capture their perceptions.Other items asked students to comment on what they liked best about the text, as well as their biggest complaints.The final questions probed students to explain their overall opinion of the text, as well as how likely they were to consider taking another course that used this kind of text.We used descriptive statistics to analyze the quantitative data and emergent coding to identify themes from the qualitative questions.

Demographics
Of the 231 students who completed the survey in 2013, 126 were female (54%) and 106 were male (46%).Twenty students (9%) had received loans to fund their education, and 85 (37%) had received Pell Grants or fee waivers.

Student Perception of Cost
As mentioned previously, the text for this course was offered free of charge online; however, there was also a hard copy available for purchase for under $30.Forty-eight percent of students reported purchasing a text for this class.The rest of the students either used the online version or printed off a PDF copy of the text.Of the 56 students who reported purchasing texts, all but eleven reported spending $100 or less, and 57% reported spending under $40.Considering the cost of the hard copy, the "$100 or less" could refer to one instructor whose students purchased the optional hardcopy note pack from the campus bookstore instead of downloading and/or printing them.
Many students (61%) reported printing materials for the course.Of those who did, 82% reported spending $30 or less, with 46% spending less than $10.Many students reported that they did not purchase any texts for the course.When we asked them why not, the vast majority of them (84%) answered "the text was available free of charge online."In an answer to our second research question of student perceptions about cost of OER, our findings indicate that whether students purchased a hard copy of the text or printed off pages, it appears that most students experienced significant savings relative to the average costs of college textbooks.

Student Perception of Quality
In order to answer the first research question of overall student perceptions of the OER textbook being used, a variety of questions were asked and analyzed.One indicator of text quality may be the frequency of student use.We asked students how often they normally use books in all their classes.More than half of students (66%) reported that they used their textbooks at least twice a week.When asked how often they used the textbook for this course, 65% reported using it twice a week or more.Thus, these students used this text about as much as they would any other textbook.
When specifically asked how they would rate the quality of this text as compared to other textbooks they have used, 143 (62%) said that it was the same as books in their other courses, 57 (25%) rated it as better than other texts and 31 (13%) rated it as worse than other texts they have used.This finding is similar to other research on the perception of open textbooks (Bliss et al., 2013).Figure 1 illustrates percent responses from students as to the quality of this text compared to a traditional text.
Open Praxis, vol.8 issue 3, July-September 2016, pp.265-276 When students were asked to imagine a future course in which there were two sections, one offering traditional printed texts and the other offering texts such as the one they used in this course, 50% of students said they would choose the class with texts like those offered in this course.Only 19% said they would enroll in the course with the traditional printed text, and the remaining 32% said they would have no preference.Similarly, when asked how likely they were to register for future courses using books like this one, 73% of students said that they were either "somewhat likely" or "very likely" to do so.Thus, in each case, it appears that students generally had a favorable view of the textbook.
In order to gauge the reasons behind the answers the students gave, we asked them to provide answers to free-response questions designed to help us understand what aspect of the textbook were appealing (and not appealing) to students.Students who rated the textbook quality as lower than average believed that the textbook lacked clarity and organization.For example, one student said, "[The textbook] was very confusing to understand exactly where everything was, and I had a hard time finding something when I needed it." In contrast, students who rated the textbook quality higher than average offered different opinions when they were asked why they rated the textbook highly.When asked what made it better, their comments fell into two main categories: clarity and examples.Examples of positive student comments about the text are as follows: • "The examples and summary pages were the most helpful portions and worth the 26 dollars to actually purchase the text."• "The book is written simply and clearly.This made it easy to understand and less 'taxing' to read.The collaborative aspect of the course built in the text encourages group learning which I have found to be beneficial to my learning."In addition to the foregoing questions, students were also asked a free-response question regarding what they thought of the text overall.We divided their responses into positive, negative, and neutral comments.The negative comments fell into three main categories: clarity, relevance, and examples.For purposes of this analysis, we have defined relevance as how well the text fit in with students' classes.Some examples of negative relevance comments follow: • "It needs improvement as our instructor substituted material for chapters that were not very comprehensive."• "It was a good text but not good enough to rely on for the sole purpose of learning, I prefer teacher notes since it simplifies everything for me." • Some examples of negative student comments about the text's clarity follow: • "Overall, this text is ok.It was simple to understand in the beginning, but then got too complicated for an Intro to Statistics book.It needs to be simplified a bit more."• "I find reading the textbook necessary but very dull and not very engaging so it makes it hard to focus on the information being provided." Examples of negative comments about the examples used in the text are as follows: • "The answers to the homework needed to be a bit more robust; I noticed that the questions that were qualitative did not have sentence answers, which made it difficult for me to gauge whether I was understanding the material (vs.following the formulas provided)."• "Too many examples, not enough explanation of the problems." The positive comments to the overall question mainly fell into three categories (other than general comments such as "Good," "Great," etc.).These categories were Examples, Clarity, and Organization.Sample positive comments regarding "examples" are as follows: • "I think it was a good choice for this course.It had a good variety of homework problems and labs that appeal to most of the students."

• "Many examples are given to illustrate the topics discussed." • "I liked all the examples given because they were very alike with the homework problems." Examining Student Perception of an Open Statistics Book
Open Praxis, vol.8 issue 3, July-September 2016, pp.265-276 • "Free. ..I have attended other institutions and this course is the first one that offered a free textbook.I am very appreciative both of the access to knowledge in a form that is digital, online and hard copy.Also, cost is prohibitive for textbooks sometimes, and not having to pay for textbooks is a direction I'd like to see education move towards." • "It has an online component to it, so I don't have to carry it around all the time." • "I liked the fact that it was online the most because I didn't have to go out and buy/rent a heavy textbook to uselessly keep or return once I finished the class."• "The convenience, I liked that it was available online." • Representative examples of comments regarding price include the following: • "It is free and it is just like regular textbooks." • "I would recommend the text to future students.I loved how students had a choice between downloading the free text and/or owning the hard copy.The course material felt really accessible for low-income students, like myself, who often can't afford expensive texts, and have to wait several weeks into the course to buy one." Figure 4 summarizes the answers between the first two free-response questions: "What made it better?"and "What made it worse?"

Introductory Statistics Version of the Textbook
In the spring of 2015, seventeen teachers at De Anza College used a revised OpenStax version of the textbook now called, Introductory Statistics.We invited those teachers to survey their students regarding their perceptions of the textbook.In total, students from nine of those teachers completed the survey; and 94 students in total completed the survey.A strong majority of the respondents (68%) all came from one class.Demographics were similar to the previous results.54% of respondents were male, and 46% were female.13% of students had received loans to fund their education and 41% had received Open Praxis, vol. 8 issue 3, July-September 2016, pp. 265-276 Pell grants or fee waivers to fund their education.Seventy percent of students stated that they did not purchase any textbooks for this class; their reason for not purchasing the textbook was primarily because the textbook was available for free online.
While only a minority of students purchased the textbook, a slight majority (53%) did print text materials for the course, with about two-thirds of students reporting that they spent less than $20.00 on their printing costs.In terms of how students accessed the book, 47% used the book online, 23% downloaded a PDF, and 31% used a hard copy.
In answer to the question, "How would you rate the quality of the texts used for this course?"70% said it was about the same as the quality of the texts in their other courses, 23% said it had better quality and 7% said that it was worse.Only five students gave specific responses to the question, "Why did you rate the OER textbooks as being worse than traditional texts" and there was no observable pattern in their responses.Some felt there were errors; others felt it was too difficult to understand. Figure 5 illustrates percent responses from students as to the quality of this text compared to a traditional text.Overall, student responses to using both Collaborative Statistics and the updated Introductory Statistics were positive.For both open textbooks, students overwhelmingly reported the text was the same as, or better than their traditional texts.In response to open-ended questions, both groups of students provided similar comments.The hard copy version of both texts could be purchased for under $30.This cost is significantly less than what it would cost for a student to print out the full pdf of either text as well as purchase a hard copy of a traditional textbook.In general, students who do print can print just the pages that they feel they need in hard copy form.The authors expect that in the future, fewer and fewer students will purchase the hard copy version as online academic reading becomes more prevalent.
One of the limitations of this study, is that while the authors attempted to distinguish between use of a hard copy textbook and In conclusion, we believe this study adds to a growing body of research that indicates that when it comes to textbook costs, students might not "get what they pay for."In other words, students appear to perceive the quality of the open textbook to be as good or better than more expensive commercial textbooks.If this in fact the case, professors and educational administrators should carefully consider adopting open textbooks to reduce the high cost of traditional texts.

Figure 2
Figure2illustrates comparisons between the first two free-response questions: "What made it better?"and "What made it worse?"

Figure 2 :
Figure 2: Contrasting Student Perceptions on Collaborative Statistics (absolute counts)

Figure 4 :
Figure 4: Comparison of Student Complaints Versus Favorite Aspect of Collaborative Statistics (absolute counts)

Figure 6 :
Figure 6: Student Overall Perceptions on Collaborative Statistics (absolute counts) use of an open, online textbook in their research, some students chose to purchase the hard copy of the text, and thus may have given responses that were based on the hard copy version of the open textbook.Future research might focus on cost trade-offs of open versus expensive textbooks, as well as low cost (under $40) versus traditionally priced, expensive textbooks.As most classes that previously used Collaborative Statistics have now adopted Introductory Statistics, and Introductory Statistics is now widely used, future research would also benefit from an expanded survey population of multiple institutions and instructors in order to more adequately generalize the findings of the study.Open Praxis, vol.8 issue 3, July-September 2016, pp.265-276 Papers are licensed under a Creative Commons Attribution 4.0 International License OpenPraxis, vol.8issue 3, July-September 2016, pp.265-276 provided features, narrative, examples and feedback.The final Introductory Statistics is a more community-developed, professional, open textbook than the original version