Trialling the use of Google Apps together with online marking to enhance collaborative learning and provide effective feedback

Stage 1 of assignment: collaborative work

Due to the early scheduling of the practical session in the autumn term, a preparatory lecture for the practical sessions was not possible. Therefore, information was disseminated via Moodle (see Supplementary material 6a–Supplementary material 6e).

Documents available on Moodle included: Notifications via News forum, a Guide on how to set up Google Drive, Group lists and the practical handbook, which contained all the relevant information needed for successful completion of the practical sessions and the assignment. During the practical sessions, a presentation was given to guide the students; this was then available from the BS111 Moodle page (see Supplementary material 6c–Supplementary material 6d).

Each group (W,X,Y and Z) went to a different habitat on campus. The habitats used were as follows:

(For more information about these habitats and site numbers, visit the University’s biodiversity web page.)

For the field and laboratory work the Cohort was randomly divided into four equal groups (n = 20 +/-2); each was allocated a grassland habitat on the Wivenhoe campus. The groups were further divided into 7 subgroups at the beginning of the practical, then equipment and data recording sheets were collected and graduate lab assistants took them to the correct location. The site was overviewed to assess different plant species, whether monocot or dicot (each group used a unique set of plant codes, which enabled standardization for the group data collection exercise).

A simple guide was used to help student determine if the plant was a monocot or dicot. The group took a sample of the different plants; labelling it using the unique code, which was attached to the stem. The sample was photographed then put in large plastic bag. A data sheet was filled in with the sample code, name (if known) together with any distinguishing features.

Each subgroup collected data from two randomly-thrown square quadrats (0.5 m²). Samples of soil, plants and invertebrates were taken for further analysis. The following information was collected directly from the quadrats: the number of different types of plant, the percentage cover of these plants, and the percentage of bare ground. A point quadrat was placed over the square quadrat (so it was parallel to the sides and went through the centre of the quadrat). The point quadrat was used to assess the height of the vegetation at 10 cm intervals across the quadrat, and also to determine the number of hits, e.g. the number of leaves contacted before the pin hits the ground when following a vertical path.

Temperature of air, plant (IR temperature gun) and soil (temperature probe) were also recorded. Soil was assessed in the field using feel for dampness, odour to determine type of respiration, and colour noted. A sample was taken from the centre of the quadrat using a trowel to a depth of 10 cm. The shoots (vegetation above ground level) were put in one labelled sample bag and the soil with roots in a second. In the lab, fresh and dry weight measurements were determined for the vegetation (roots had to be separated from the soil sample). A subsample of the mixed soil core was also taken for fresh and dry weight analysis. Fresh weight samples were dried in an oven for over 48 hours.

In the second practical session dry weights of plants and soils were recorded on data sheets or in lab book. Soil samples were analysed for pH using the method described in the soil test kits (Palintest, 1997); texture was determined from feeling wet and dry samples and comparing to descriptions of different soil textures. Soil samples were mixed with water and then allowed to settle to determine sedimentation profiles. A standardised amount of soil was placed in a funnel and standard volume of water was poured through and timed to determine soil porosity. Plant samples collected at the group level were identified within the laboratory using laminated identification sheets and a range of plant guides. Further analysis was carried out independently using the photographs taken in situ. The range and type of invertebrates found in the bagged soil samples were recorded, no special method was used to select for invertebrates.

Instructions were given via Moodle and verbally during the practical sessions, so that each student could create a personal Google Account. In addition, a handwritten record of their account details was collected during the practical sessions. Students were instructed to navigate to https://drive.google.com/ to access the Google Apps service.

The lecturer created a folder for the module (BS111) containing four group folders (labelled W, X, Y and Z). Within each group folder there was one folder that was shared at the group level and seven subfolders, each shared only with the relevant subgroup. The Google Drive desktop application was used to duplicate a template folder/file structure, as this proved to be much more efficient than creating all the folders and files through the web interface (see this support article for more details about the desktop application and how it works).

Each group folder (X to Z) contained two template documents: a Google Docs file called “BS111 Practical 1 Description of Habitat” (see Supplementary materials 1b) and a Google Slides file named “BS111 Photo Submission for Group Work (see Supplementary materials 2). The group folder was shared with all students in the group, who were granted full editing rights over the content of the folder. The seven separate sub-folders contained the Google Sheets document for the data collection task, and were only shared with members of the subgroup. The Google Sheets included pre-formatted tables, with some of the column headers already containing units. The documents at the group and subgroup level also contained inline instructions for the collaborative work (see Supplementary materials 3 for example).

Students worked collaboratively at the subgroup and group level; each student downloaded a copy of the Slides and Docs file that was then uploaded to the University’s online coursework submission service (FASER). The group Slides and Docs files were exported as PDF files and the data collection sheets were exported in Excel format (to make further manipulation possible).

Once this point was reached, the lecturer removed the editing rights of the students in Google Drive, so that further editing was not possible, effectively making all of the collaborative documents “read only”. The documents were then uploaded to Moodle and students used these copies for stages 2 and 3 of the assignment. The lecturer also established a discussion forum site on Moodle to facilitate student discussion of the fieldwork.

Stage 2 of assignment: Peer assessment

Instructions and a pro forma for peer assessment was issued to the students (see Supplementary materials 4). Each student peer assessed the other group work (including the subgroup data collection sheet). This was worth 10% of the overall assignment mark.

The lecturer assessed two aspects of the student peer assessment process:

Stage 3 of assignment: Independent work

Printed instructions, also available in Moodle, were issued. The final part of the assignment was individual analysis of the four habitats for patterns, correlations, and variability. This was an open-ended task (see Supplementary materials 5), facilitated by an Excel guidance file, which was provided on Moodle for download. Students requested further help and support, which was subsequently given.

The assignment files were uploaded to FASER by every individual. Electronic marks and feedback was returned to each student. Group level feedback for Stage 1 was given after the second submission (stages 2 and 3) had been uploaded to FASER. This feedback included information on learning outcomes achieved, advice on areas that needed improvement, and more detailed information on calculations (see Supplementary Information 7). Individual annotated feedback was also provided along with the standard feedback sheet for level 4 students (see Supplementary Information 5).

Students were given access to the group and sub-group datasets via Moodle after the deadline for the first submission, so that Stages 2 and 3 could be completed.

Analysis of student performance

Performance of students at stage 1 was used as a measure of their engagement. Stage 1 included practical and group work. Students achieving a mark higher than the mean mark (65%) were defined as having a good level of engagement (or good performance) in the practical work and group interaction. Students obtaining marks below the average (65%) were defined as having a poorer level of engagement (or below average performance). This threshold (65%) was used at all stages of the assignment, and was used as an indicator of an individual’s level of ecological knowledge and understanding.

Data was ranked according to the University’s degree classmark divisions; students who passed the assignment obtained 40% or greater. Non-engagement is defined as any situation in which a student did not submit work for assessment.

The three stages of the assignment were analysed for student success and for their level of engagement/performance. Pearson’s correlation analysis was used to establish whether there was any link between the marks achieved at different stages of the assignment, and whether group membership or gender had an effect on performance. Analysis of variance was carried out on marks obtained at each stage, comparing groups and degree schemes (Excel vs 2010). Detailed information for this section is available in Supplementary material, Dataset 1.1 – Dataset 1.6.

Stage 1 of assignment: Collaborative work

Three groups successfully collected all of the expected field data; the fourth group collected half the expected amount, i.e. from single quadrats rather than two. The considerable amount of data collected within the two three-hour practical sessions involved a lot of collaboration in the field, as well as in the lab, to ensure data sampling and recording was accurate.

In order to engage in the online group work, students had to create Gmail Accounts and make a handwritten record during the practical sessions (81.6% of students succeeded in doing this). Lists for two of the four groups were almost complete; the other two groups emailed the missing account details to the lecturer.

There was a good level of online collaboration, 92.1% of students were active on Google Drive (Dataset 1.4). Five students (6.5% of the cohort), who were all absent from one of the practical sessions, failed Stage 1 and one of them did not submit any work at all. Qualitative analysis of Google Drive activity, together with the quality of the group work produced (see Figure 1 – Figure 3) provides strong evidence that students successfully engaged in the group work component of the assignment.

Interesting group dynamics were evident from monitoring the Google Drive activity. Members of one group engaged in the process very actively, and some members became natural leaders. These leaders then became a little too controlling and removed the other group members’ editing permissions. The lecturer intervened by sending an email out via Moodle to let them know this was not acceptable behaviour, and permissions were reinstated for all group members. The students also engaged in group discussions using a forum in Moodle and the Google Apps comments feature. The latter was particularly useful when students were refining their documents; the comments stream acted as a private, student-led forum for discussion.

Students successfully accessed the documents on Google Drive to create a written description of the habitat. All groups also managed to curate photographs of the habitat site and plant species found using Google Slides. Self-directed learning clearly took place during the creation of the group documents, which surpassed the expectations of the lecturer in terms of overall quality and attention to detail. The descriptions and figures for the submissions were of a very high standard, especially considering the students had no formal training on expected undergraduate academic standards.

An example of the type of work completed from one group (X) can be seen in Figure 1; it shows the habitat site ‘The Bluebell Wood’ and one of the plants identified, Plantago major (Figure 2); feedback should have been added that this was not a Monocotyledon. This data demonstrates that students did research the Minimum Undergraduate Standards required, as evidenced by the correct use of binomial names and the use and position of figure legends. The students managed to produce group work that met expected standards, all within six weeks of arriving at the University. The task instructions were purposely brief to enable students to make their own contribution. Figure 1 and Figure 2 show that some students used this opportunity to deepen their learning and understanding; the inclusion of additional notes demonstrates that several groups were engaging in high quality research, and managed to identify plants at the species level (Figure 3a) and produced a list of specific plant species present at the site. This also shows an awareness of ecological knowledge.

Figure 1. Example of student generated Google Slide of habitat from Stage 1 of assignment.

Figure 2. Example of student generated Google Slide of plant from Stage 1 of assignment.

Figure 3.

(a) Example of feedback comments on descriptive task of stage 1 of assignment. (b) Lecturer comments added to student descriptive task in Figure 3a.

The group work documents demonstrated that students were more aware of the campus grounds. Each group submitted 10 – 12 photos of habitats/plants and a habitat description (Supplementary materials 8). Group work submissions and individual analysis of the four grassland habitats on the Colchester campus were used as evidence of an increased understanding of ecology (Figure 1–Figure 3). Figure 3 demonstrates that the students in group X gave a good general description of habitat sample and detailed features of habitat boundary; characteristics of different plant forms and presence of different patches within the habitat, and information on weather and soil characteristics.

The Google Sheets data, collected during the subgroup task, highlighted a greater range of student ability and was less successful overall. Raw and calculated data were expected; the submitted information ranged from no data, incomplete data sets, to data sets complying with Minimum Undergraduate Standards. The approach required students to be responsible for keeping data safe until it was inputted online, and to understand the Minimum Undergraduate Standards for tables. Evidence suggests that many students were unaware of these expectations, as more than one of the data sheets had no data submitted. Many others contained data with the units in the body of the table.

The small size of the subgroups meant that there was greater variation in the standard of work. (This is to be expected for normal distribution of data.) This demonstrates that the subgroups were working within the team that they were assigned to. The large number of data sheets generated (28) meant it was difficult to quickly achieve a single, combined dataset in a format that was suitable for the students to access for the latter parts of the assignment. The deadline for the assignment was adjusted to take account of the time needed to collate and disseminate the data to the cohort.

Stage 2 of assignment: Peer assessment

The second part of the assignment was designed to make students aware of the marking process and give them an opportunity to become familiar with the habitats they had not visited. A student who did not take part in the practical sessions (due to a change in degree course) was able to do this and successfully complete the assignment (with a final mark of over 90%).

The peer assessment required students to look at the eight group documents, descriptions and figures of the four habitats, as well as the 28 data sheets. A high proportion of students demonstrated a good level of engagement with the assignment; 78.9% of the cohort obtained over 65% for stage 2. This proves these students reviewed the minimum standards and used their observations to successfully peer assess the work of others (see Figure 4 for an example). Fifty percent of students who obtained a first class mark for the peer assessment also got a first class mark for their independent work. Some students (2.6%) submitted work but had a second class mark for this stage. In these cases the peer assessment had arbitrary marking patterns and over-inflated scores, e.g. 5/5 for everything, and comments were very brief or absent. This demonstrated that they had limited knowledge of expected requirements and types of answers. A larger sub-set of students (14.5%) did not engage in the peer assessment at all (see Dataset 1.2 & Dataset 1.3).

Figure 4. Example of a good individual student peer assessment task, stage 2 of assignment.

Stage 3 of assignment: Independent work

The independent work was an open-ended task. The endpoint of the entire assignment was to carry out correlations to establish if there were any discernible patterns in the data from the different habitats. Students found that the data sheets were not in a suitable format to enable correlation, and had to make difficult decisions on what data to include/exclude and how to standardise the results. As a consequence, the students asked for guidance on sorting and analysing the data. Some individuals became stressed, evidenced by emails sent to the lecturer. An extra tutorial was run in a computer lab to support these students. During the tutorial, the majority of the attendees felt anxious because they had not achieved the ultimate endpoint of the assignment. As a result, the task was modified to include a simpler outcome that they could achieve. Students had put a huge amount of time into the activity; the lecturer feedback indicated that they had done well and listed achievements for figures and descriptions, and highlighted the fact that most had created tables and figures that mainly complied with expected academic standards.

However some students had overcome these issues by that stage, e.g. they had produced correlation tables and figures which they submitted in their individual assignment (see Figure 5 and Figure 6). By the end of this process, 47.4% of students were capable of handling, analysing and interpreting ecological data, enabling them to achieve a first class mark.

Figure 5. Demonstrating how a capable student sorted the class data for stage 3.

Figure 6. Demonstrating how a capable student sorted the class data to produce correlation figure (stage 3).

Overall analysis of the three stages

Over 82% of students were successful in completing all three stages of the assessment; with 85.5 % of students passing stage 1 and 86.8% passing stage 2 (see Dataset 1.5a). A small number (5.3%) did not submit any work in any of the three stages. The peer assessment component (stage 2) had the highest level of non-submission (14.5%), followed by the stage 1 group work (9.2%).

Considering the whole class dataset, 59% of students had a good performance (marks greater than 65%) for the group work, 78.9% for peer assessment and 51.3% for the individual work. For the whole class, there was a positive correlation between the mark obtained by students during group work and the later independent work (stage 2 r = 0.334 p<0.01, stage 3 r = 0.516, n = 76, p<0.001). There was a weaker correlation between the mark obtained in peer assessment and the mark obtained for independent work (r = 0.293, n = 76 p<0.05 – 0.001).

For the set of students with a mark over 65% for the independent work (stages 2 and 3), there was no significant correlation between the marks they obtained in group work and marks achieved at the other two stages of independent work (n = 41, p > 0.05). However, for students who underperformed, with a final mark less than 65%, there was a significant positive correlation between the mark obtained in the group work and the mark obtained during the stage 3 independent work (r 0.597, n =35 p < 0.001). (See Dataset 1.6)

Analysis based on gender indicates female students had a significantly higher mean mark (73.3%) than males (57.8%) for stage 1 (ANOVA: F_1,74 = 6.45, p < 0.05). For stage 3, the female mean mark was also higher at 76.2% compared to a male mean mark of 47.9%; demonstrating a significant difference between male and female performance (ANOVA: F_1,74 22.031, p < 0.0001%). There was no significant difference between the mean marks for male and female students during the stage 2 peer assessment (p > 0.05). For female students there was also a positive correlation between the marks obtained in stage 1 and stage 3 (r = 0.518, n = 34 p<0.01), but no correlation between stages 2 and stages 3 (r = -0.013 p> 0.05). (See Dataset 1.6 & 7).

For males students, there was a positive correlation between all three stages (r = 0.477, r = 0.430 respectively, n = 42 p<0.01). However, in contrast to the female students, there was a weak positive correlation between peer assessment marks for males at stage 2, and the marks obtained for independent analysis during stage 3 (r = 0.349, n = 42 p<0.005).

There were no significant differences in the mean marks of students allocated to the different groups (W, X, Y, Z), or for students studying on different degree schemes (p>0.05).

There were a number of student absences from practical sessions (1.31% of the non-submission data is accounted for by one student, who did not attend either practical session nor submit work during stage 1). Seven students (9.2% of cohort) did not attend practical 2 resulting in a low level of submission for stage 1 (only 2.62% passed this stage), but these students performed well at stages 2 (with marks over 80%) and two students (2.62% of cohort) also performed well at stage 3 (with marks over 90%) (See Dataset 1.8).

Stage 1 of assignment: Collaborative work

The division of the cohort into four smaller learning groups and seven subgroups created an effective and friendly environment for practical work, which provided a good situation to meet the challenge of online collaborative work. The approach of creating small friendly groups and subgroups was a success, in terms of collaboration and passing stage 1 of the assignment. The majority of students succeeded in setting up Google Accounts (92.1%) and were active within Google Drive, which was a major achievement one week into the academic year. Group work also appeared to enhance the performance of underperforming individuals.

It was evident from the practicals and data collection stage that there was too much data to collect and analyse; this should be simplified in the future, e.g. reduce sample collection to one quadrat per sub group. Also, improved management during practical 2, e.g. rotating the groups around the laboratory-based activities, would increase the efficiency and reduce the pressure on the limited equipment available in the laboratory. The class data from this year’s survey will be reviewed to reduce the range of data to collect. This would decrease the time pressure for staff and students during the practical sessions, and would also simplify the class data set.

A better system is needed for subgroup data collection; the method used was cumbersome and not as effective as alternative methods. It hindered the process of looking at the data for ecological analysis. Other options include the use of a single document at the group level; use of electronic voting devices (clickers) to collect data, or the use of an online form to collect class data in a standard format (Google have a tool to facilitate this). This would help team leaders provide class data quickly in a usable form. These alternative approaches would make it easier for students to collect and use the data.

Students collaborated within Google Drive and managed to produce high quality documents (Figure 1 to Figure 3) and there was evidence that they had carried out student-directed learning. They had applied the expected minimum undergraduate standards and carried out descriptive and statistical analysis on their data. This was before formal training had been delivered at Level 4 via a Key Skills Module (BS141). There was a noticeable variation of style between the four groups’ description and slides. This indicates that students worked within their own group and were not using other mechanisms to exchange information with other students outside of their allocated group. Therefore this part of the assignment met its intended learning outcomes.

The group work, together with the use of comments within Google Docs, demonstrates that students engaged in self-directed learning. The open-ended nature of the assignments allowed the students to deepen their learning, which is evident from research notes added to Figure 1 and Figure 2. Students demonstrated emergent behaviour, which included self-discovery of features in Google Apps, e.g. the discussion stream and document permission settings, which suggests student-student learning took place. Therefore, this met the intended outcome of this new approach.

During the group work, it emerged that students were using three methods of electronic communication, i.e. discussing and using comments and emails in Google Drive; using the Moodle forum, and also emailing each other directly via Outlook. The communication tools within Google Drive were most advantageous, as they acted as a private forum for student discussion within their group. The disadvantage of the Moodle forum was that it was accessible to the whole cohort and also, more importantly, all lecturers teaching Level 4 modules in the School of Biological Sciences. Having access to these conversations might colour a lecturer’s view of a particular student when setting and marking their future work. Therefore, it would be wise if the lecturer directed the students to a single method (most likely within the Google ecosystem) as this would avoid such problems.

The success of the three stage assignment required the rapid creation and collection of Google Account details from students. Overall this worked well, but students who didn’t have accounts could not create any evidence of collaboration, hence did not pass stage 1. Reasons for lack of student engagement were varied: Some individuals were reluctant to try something new, others distrusted the online services, and a few didn’t want to put their data in the “Google cloud”. Another probable reason for this poor engagement is the timing of the task; it took place early in the term, so students had to contend with many new things, such as some subgroup members changing degree schemes. Student absences from the practical sessions resulted in 6.5% of them failing Stage 1. Therefore, it appears that the face-to-face interaction with fellow students, and members of staff, is crucial for the success of online collaboration and submission of coursework.

In the future, it would be advantageous to use Google Apps for Education or the Microsoft Office 365 suite of tools, as all members of the University have access to both of these services now. (However, Office 365 would only be used if the collaboration features are as good as those found in Google Drive). Using Google Apps for Education would give the lecturer greater control, e.g. the ability to establish student accounts for Google Drive before the start of the practical session and would make it easier for all students to take part. Google takes the education arm of its business very seriously, which should reassure students that their data is safe. Showing the students a short video revealing the security measures that Google enforces at its data centres would perhaps ease any concern, e.g. Inside a Google data center. Getting students to look at the Google Drive privacy policies and terms of service would also be helpful.

Using Google Drive/Apps did make it a little difficult to see which students had done what, as the activity updates are presented as one continuous stream of information. Using Google Apps for Education could also help improve this situation. It would be possible to provide formative feedback for the stage 1 work rather than wait for formal feedback. This environment is good for self-directed learning, experimentation, and for encouraging informal communication (which will help students engage at a deeper level and gain a more comprehensive understanding of the subject).

Stage 2 of assignment: Peer assessment

The peer assessment provided the link between the group and individual work. It also ensured that the students engaged in data that they were not involved in creating.

Stage 2 enabled students to regain entry into the assignment process if they had failed to complete stage 1, which is a key feature of this multiple-stage process. A student who came onto the course after the practical sessions entered late in stage 1, participated in stage 2 and successfully completed the whole assignment. By engaging students at this stage of the assignment, it was hoped that they would become familiar with the habitats they had not visited and not just rely on the site they sampled. Students did analyse all of the data sets, so the peer assessment activity met its intended aim. However this was the stage that had most non-submissions and also included three students who did not submit any work. It is possible that students were tactical as a larger amount of work was required for a low proportion of marks. Also some students may have found it hard to make judgements.

Stage 3 of assignment: Independent work

For the third stage of the assignment, some students used the tutorial and managed to achieve all of the expected outcomes, i.e. correlation figures and identified plants to the species level through their own endeavours (Figure 5 and Figure 6). It is important to let students know that there is no correct answer expected, and that the assignment is designed to force them to work outside of their comfort zone. This would be made clearer in future.

The initial tasks (e.g. finding averages) should have been accessible to all students. Stage 3 assumed that stages 1 and 2 had been achieved to a high standard. However, the data sheets between subgroups were not consistent—one group had a complete data set missing and other groups had sheets with incomplete data.

The fact that there were three stages to the assignment allowed any student not succeeding at a particular stage to be successful at a later stage. Analysis of gender showed that there was no correlation between stages 2 and 3 marks for females, but a weak positive correlation for male students. This is worth reviewing again in future years. A benefit of this staggered approach was that it required a variety of skills for completion, thus enabling a diverse range of learners to succeed at one or more parts of the assignment.

The overall assignment demanded a high level of engagement, so students who succeeded were clearly involved in a deep learning experience. Many students met this challenge, but some became stressed as they had other assignments to carry out in the same time frame; there was a definite cost versus benefit relationship between the stress created and the learning achieved. For a healthier balance, a simpler data set together with a more directed final assignment (with some scope for an open-ended approach) will be used in the future. A student evaluation of the assignment and self-reflection on skills gained would also be a useful addition.