Engagement of introductory biostatistics students in a novel hybrid course format

Background While online-only courses can be delivered without constraints of geography or synchronization of lecture times, traditional in-person lectures continue to be central to teaching and to student experiences at most universities. Hybrid learning courses fill the gap between, providing convenience and schedule flexibility while maintaining the traditional face-to-face experience. The CUNY Graduate School of Public Health and Health Policy and Hunter College are urban campuses that draw working students from a wide commuting area, leading us to experiment with hybrid course formats that offer a maximum degree of flexibility to students. Methods Lectures and lab sessions were held on campus in a traditional lecture style, but streamed live over YouTube with remote attendees able to type questions and have them answered by the instructor in real time, and later made available for viewing. Results Through a post-course survey we found students to be divided in their preferences for in-person course formats (46%) versus synchronous and asynchronous online formats (35% and 19%, respectively). In the hypothetical scenario of retaking the course, the option of attending each lecture either in-person or online was preferred (69% for this hybrid format, 11% for in-person only and another 11% for online only). Although long commuting times, work and family responsibilities were common, interactivity and learning preference were the most important considerations for students regardless of their decision for online or in-person attendance. Analysis of YouTube viewing data showed a majority of online participation occurring during live lectures, with an additional peak of viewing of all lectures occurring immediately before exams. Conclusions We found this hybrid format to be a viable and popular way to accommodate diverse personal circumstances and learning preferences within the framework of a traditional lecture format. 1 2 1

Hybrid learning courses are defined as courses that deliver 30% to 79% of material via an online medium (Allen & Seaman, 2008).Hybrid courses provide convenience and flexibility through online components while also maintaining the traditional face-to-face experience (Cavanagh, 2011).When designed well, they may also accommodate a wider variety of student learning styles.A metaanalysis conducted by the U.S. Department of Education concluded that students taking hybrid or blended instruction courses had improved outcomes compared to either pure face-to-face learning or pure online learning (Means et al., 2009).
There is little research on hybrid course engagement among adult learners at the graduate level.Adult learners are thought to show greater motivation and learning success based on course metrics like final grades, assignment completion and knowledge retention over time (Coogan, 2009).Further, some researchers posit that modern technology may present some challenges to adult learners that differ from those of younger generations (Coogan, 2009).Thus, the age of students could play a role in hybrid course engagement and outcomes.Further, hybrid course learning may increase accountability as students are expected to endorse self-learning and self-discipline (Coogan, 2009;Hu & Hui, 2012).
Much of the literature on the determinants of course format uptake tests for associations with student learning preferences, self-efficacy and engagement (Hu & Hui, 2012;Shukor et al., 2014).The literature on graduate-level hybrid courses rarely mentions the influence of life circumstances on course options in addition to individual characteristics and preferences.Among adult learners, external influences could potentially be strong indicators of course format preferences.These influences may include travel time, home and work responsibilities.
The present study aims to investigate course format preferences among graduate-level students and possible influential factors for such format preferences.We offered an Introductory Biostatistics course resembling a traditional in-person lecture and lab format, but with added options for synchronous online attendance (while class was occurring) and asynchronous online viewing (at a later time).Students were allowed to choose how to attend or view any class, and we studied their preferences through a post-course survey of habits, life situation, and learning style, and through analysis of YouTube viewing data.

Course format
The master's level introductory biostatistics course was offered in a 12-week semester with classes, two exams, and a final project poster presentation, and is a required course for students pursuing a master of public health (MPH).Each class involved one hour of lecture and two hours of laboratory instruction.In-person attendance was mandatory for the first class, exams, and final project poster presentation; for all other sessions students were given the choice of attendance in-person, online during class (synchronous), or online after class ended (asynchronous).The synchronous offering was provided using Google's "Hangouts on Air," with a live screen share and audio broadcast viewable through YouTube®.The "Q&A Module" allowed viewers to type questions that appeared on the instructor's screen during lecture.These questions were answered verbally as were questions raised by in-person attendees, and became a clickable index of the lecture.Recordings became available immediately afterwards on the course's YouTube channel (https://www.youtube.com/user/ph750spring2014).

Technology required
This format required that the computer used to present material in class be connected to internet and have a working microphone.
In the absence of a dual-screen setup, the "Q&A" module was shown side-by-side with a Microsoft PowerPoint® window used to show lecture slides.A "Lavalier" microphone was used to improve sound quality, but an ordinary webcam microphone would suffice.

Course survey
A total of 47 introductory biostatistics students were given the opportunity to participate in a survey at the end of the semester.An additional 58 introductory epidemiology students whose course followed a more typical hybrid format of alternating in-person and online sessions were also surveyed for comparison.This study was reviewed by the Hunter College (City University of New York) Institutional Review Board (590445-1) and determined exempt according to federal regulations, under 45 CFR 46.101(b).Informed electronic consent was obtained as part of an anonymous online questionnaire.Study participants were given the option and incentive of entering a draw for a $50 Amazon gift certificate upon completion of the questionnaire.
The end of course survey included demographic characteristics such as age, gender, race/ethnicity and travel time to campus.Participants were asked to choose which format was their preferred way to attend lectures.They were subsequently asked an open-ended question, "Why did you prefer this format?"Responses that pertained directly to the question asked were analyzed and assigned primary and secondary topics based on recurring and prevalent themes within the total set of responses.The keyword topics used to categorize responses were as follows: 1) interactivity, 2) convenience, 3) avoiding commute and 4) learning preference.The complete survey and summarized responses are provided in the Supplementary material and Data availability sections, respectively.
Data were collected via the Google Forms® encrypted website and is noted under the Data availability section.

YouTube data collection
Viewership among other metrics are automatically documented by the YouTube® website and can be accessed using the analytics dashboard.The dashboard can be found under "analytics" in the "creator studio" section of the Hunter College PH750 Spring 2014 YouTube® account.Data within the time frame of February 1, 2014 to March 4, 2014 were extracted from this dashboard and saved as a comma separated values file.See the Data availability section.

Statistical power
Statistical power was estimated using the pwr.2p2n.testfunction from the R pwr package.For a sample size of N=26 and a twoproportion test, we estimated 90% power to detect an effect size of between 1.3 and 1.6 for balanced groups and for groups with a 4:1 prevalence ratio respectively.

Data analysis
The survey data were downloaded from the Google Forms® encrypted website and is noted under the Data availability section.Bivariable relationships were only investigated among biostatistics students due to a low response rate from the introductory epidemiology course (19% or 11/58 of these students responded to the survey).Two respondents who were registered for both courses were considered as biostatistics students.
Associations between continuous variables like age and travel time to class and course format preference were analyzed by non-parametric Kruskal-Wallis analysis of variance.Associations between categorical variables and course format preference were assessed by Fisher's Exact Test.YouTube longitudinal viewing data were visualized as stacked line plots.
Data cleaning and all analyses and plotting were performed using R version 3.1.1(R Core Team, 2014).Code and data to reproduce all results in this manuscript are provided at https://github.com/LiNk-NY/H-HybridCE.

Demographics
A total of 26 introductory biostatistics students out of 47 or 55% participated in the anonymous survey.Demographic characteristics of the biostatistics students are shown in Table 1.The majority of these respondents were female (72%), with an average age of 33 years (SD = 8).The median age of the sample was 30 years with a range between 22 and 55 years of age.Of those who answered the survey, 46% of students identified as Non-Hispanic white, 27% as Non-Hispanic black and 15% as Hispanic.The average travel time to the Hunter College Silberman School of Social Work building was 97 minutes (SD = 63 minutes).The median commute time was 90 minutes with a range from 2 to 240 minutes.

Course format preferences
Table 1 also shows self-reported course format preferences and preference reasons.Forty-six percent of respondents preferred the in-person format; thirty-five percent preferred the synchronous course format, and nineteen percent preferred the asynchronous online format.Students were asked, "If you were to take this class again next semester, which format would you choose?"The majority of students (69%) would prefer having the option of both for each class, in-person or online, as was done in this class."Learning preference" was the most endorsed reason with 10 mentions (making up 38% of respondents), followed by "interactivity" with 9 mentions (35%), "convenience" with 7 (27%), and "avoiding commute" with 6 or 23% of biostatistics students.

Analysis of qualitative survey questions
A qualitative analysis of the long response items in the survey extracted primary and secondary reasons for course preferences.Primary and secondary reasons were determined according to order of appearance.Secondary reasons were only documented for those with more than one reason in the same long response item.These preferences were categorized into four salient topics: 1) interactivity, 2) convenience, 3) avoiding commute, and 4) learning preference.Interactivity was defined as the advantage to be able to ask questions and interact with either the professor or classmates during in-person or online lectures.For example, responses similar to "I prefer class in-person because it allows for interaction with classmates and for me to ask questions in real time" were coded as "interactivity".The "convenience" topic was determined in any comments where students indicated learning with less external effort or difficulty.Comments similar to, "I liked to be able to pause the lectures and rewind/repeat statements to better understand difficult topics," were categorized as "convenience" reasons.The "avoiding commute" topic was frequent enough to warrant its own category.Responses that indicated saving time, for example, "I did not have to commute to Harlem," were coded as "avoiding commute".Learning preference is indicated when students' comments explain cognitive advantages to their course preferences.(indicated by a dashed reference line).Smaller stacked shaded areas reflect asynchronous online viewership of lecture videos with a marked increase in views just before the exam.This pattern of most online attendance occurring during the lecture, followed by low but steady rates of asynchronous viewing and a viewing peak immediately before exam was observed among the remaining video lectures and before the second exam.
Figure 1 shows both primary and secondary reasons provided in qualitative survey responses, by course format preference.
Those who preferred the in-person course format endorsed "interactivity" and "learning preference" more often than those who preferred either the synchronous or the asynchronous formats.
Those who preferred the asynchronous format tended to mention "convenience" and "learning preference" as reasons for such choice.Among those who preferred the synchronous format, avoiding the commute was the dominant reason for format preference.
Other factors associated with course format preference Association between life circumstances and course format preference was nearly significant, with students responsible for the care of children and students working full-time more likely than others to prefer asynchronous viewing (p=.09 and p=.08 respectively, Fisher's Exact Test).Age, travel time to campus, gender, race-ethnicity, having a STEM undergraduate degree (Science, Technology, Engineering, Mathematics), having previously taken a hybrid-format course, and self-reported confidence level with statistics were not associated with course format preference.

Synchronous/asynchronous viewing patterns of the lectures
YouTube view frequency patterns of lecture videos were recorded and visualized as stacked line plots.Figure 2 shows the number of views per online video lecture up until the first exam, where the reference line indicates the date of the first exam.The viewing patterns show a high number of views on the date the lecture video is uploaded and an increased number of views for all lecture videos right before the occurrence of the exam.The small increase in viewing between two holidays, Lincoln's birthday (February 12) and President's day (February 17) relative to other non-class days shows that some students used those days off to watch lecture videos.This viewing pattern was consistent during the rest of the course and before the second exam.

Discussion and conclusions
We propose a flexible hybrid course format that allows students to choose between traditional on-campus format, interactive online attendance, or viewing lectures at a later time.This course was offered to mature graduate students (average age over 30), mostly working, spread across a large area within and around New York City with an average commuting time to and from campus of over 90 minutes.This format enabled some students to save long commutes, to attend class while traveling, and to re-watch lectures to solidify difficult concepts, while recognizing that a substantial proportion still prefer to attend class in person.Students in this study were evenly split between preference for in-person or online course format, an observation often qualitatively noted by our faculty, and corresponding closely to the actual proportion of students who attended each class on campus.Among students who preferred to attend online, most preferred to attend while class was occurring, largely for reasons of interactivity and ingrained learning preferences.Having to care for children, employment status, and travel time to campus may also contribute to format preference, but these associations were weaker and not statistically significant likely due to a small sample size and thus, insufficient statistical power (see Statistical Power section of Methods).
Our survey indicated that learning preferences were a stronger indicator of course format preference than commute time, job

Introduction
The authors mention that hybrid courses may accommodate a wider variety of learning styles.I encourage the authors to reconsider their language and further consider the theory behind this statement.There is very little evidence to support that students have learning styles and the theory of learning styles has been deemed as a myth by many in the higher education community.Please provide the grade levels included in the Means et al., 2009 meta-analysis.Please define "adult learners" and reconsider the claim that technology may present challenges to adult learners as this may be specific to a particular age range of students.Someone who is 18 years old in 2017 was likely exposed to technology quite regularly during their education, compared with someone who is 50 years old.
It is unclear from the introduction why these research aims are novel and broadly relevant outside of this particular institution.

Course format
Please explain the difference between lecture and laboratory instruction.
Please explain what a clickable index of lecture means.

Course survey
The aims of this study are exploratory and do not require a research design which includes a The aims of this study are exploratory and do not require a research design which includes a comparison group of students.Thus, it seems irrelevant to mention the epidemiology students.However, if the researchers had a reason for including these data from the epidemiology students, then the readers need much more information about these students in order to interpret the data.Please provide more information about how the four categories were developed.Were qualitative research methodologies used?The references below may be useful: Krippendorff, K.

Data analysis
It is unclear where, if at all, the data from the epidemiology students appear in the paper.Further, if the epidemiology class was used as a comparison group, students who are enrolled in both the biostatistics course and the introductory epidemiology course should be removed from the study.However, it appears that this group of students is not actually begin used as a comparison group.

Demographics
Please explain the difference between the terms travel time and commute time.
Course The "course format preference" question measures whether students prefer in-person, synchronous, or asynchronous courses.However, the "re-take course format preference'" question is measuring whether students want a mandatory in-person class, a mandatory online class or whether they prefer to choose.These questions are measuring very different ideas and this needs to be better explained to the reader.Further, it is unclear which of these questions the open-ended response question is asking students about.Regardless of which question the opened ended question is referring to, open-ended student responses needed to be organized by how students responded to the closed ended question.For example, if the open ended question is asking about student course format preference, all of the open-ended responses from students who identified that they prefer to take a course in person needed to be analyzed together.Analyzing all student responses without taking into account their response to the closed ended question could cause the researchers to miss themes that were specific to a particular preference.For example, I would imagine that students who cite "interactivity" as a reason for attending in person classes may be referring to a different phenomenon than students who cite "interactivity" as a reason for attending synchronous courses.These nuances are important and need to be further explored and explained.The sample size is not large enough for this type of analysis.More data needs to be collected.
Are the percentages of preferred reasons endorsed that are listen in Are the percentages of preferred reasons endorsed that are listen in Table 1 primary, secondary, or both?How many students reported secondary reasons?Did some students list three reasons?
The authors should mention that because this was an open ended question that the percentages are not representative of all students who would possibly agree with a particular reason.I would suggest creating a table with a row per preference category (e.g.interactivity, convenience) and a column which includes a description of the category and a column for a representative student quote that was coded as the particular category.See table 2 in the for an example .Are students really explaining "cognitive advantages" of in-person vs online delivery of material?Please further explain or consider rewording and provide student example quotes.

Synchronous/asynchronous viewing patterns of the lecture
It is unclear from the stated purpose of this paper why the authors report out the viewing pattern of lectures.These data seem unconnected to the rest of the paper.

Figure 1 .
Figure 1.Course format preference reasons by preferred format.Combined primary and secondary self-reported reasons for course format preference were extracted from qualitative topic analysis of an open-ended survey item.Among students who preferred the in-person course format, interactivity was mentioned 7 times in response to the open-ended question, "Why did you prefer this format?"Those who preferred the asynchronous online format mentioned its convenience 5 times in response to the open-ended question.Students who preferred the synchronous online format were likely to mention avoiding the commute as a reason for such preference.
(2004) Strauss, A., & Corbin, J. M. (1997) What percentage of students answered each closed ended question?What percentage of students clearly answered the open-ended question?Of these responses, how many were able to be coded into at least one of the four categories?

Table 1 . Demographic characteristics and course format preferences.
Shaded peaks reflect increased live-online lecture viewership.Stacked areas show cumulative views per lecture video.An increase in online views for all lectures is observed just before the first exam on March 3rd a Multiple preference reasons were possible Figure 2. YouTube lecture viewing patterns.
format preferences To properly analyze what format students preferred, the readers need information about what type of format students experienced.For example, a student who experienced in-person, synchronous, and asynchronous classes has the ability to evaluate the three formats against each other.However, a student who has attended all in-person classes can report what they experienced, but not what they prefer because they have not experienced other modes.Describing what mode students engaged with needs to be separated from measuring what mode a student prefers.