Using agricultural metadata: a novel investigation of trends in sowing date in on-farm research trials using the Online Farm Trials database

Digital data

The amount of digital data being generated around the world every day is truly massive. More data were generated between 2013 and 2015 than in the whole length of human history before that (Marr, 2015). It is expected that the entire digital universe is expected to reach 44 zettabytes¹ by 2020 and by 2025, it’s estimated that 463 exabytes² of data will be created globally each day (Desjardins, 2019). The sheer volume of data being produced means that excellent data management is essential (Harper et al., 2018). However, it has been estimated that between 80 and 93% of data are held on personal computers or in offline repositories (Babcock, 2015), where they are left in the ‘dark’, and are of limited use (Sadiq, 2016). There are increasing calls for data to be made more widely available for maximum use, as well as the view that research funded by taxpayers should be more readily accessible (Stow et al., 2018). Research data is no longer just ‘nice to have’: such data underpin decisions about health, development of public policy, innovation, profitability and environmental sustainability (Barbour, 2019).

Metadata

For data to be used they need to be brought out of the ‘dark’ and into the ‘light’. That is, they need to be findable. Making data findable is the first step in the ‘FAIR Guiding Principles’ (i.e. Findable, Accessible, Interoperable and Reusable) for scientific data management and stewardship (Wilkinson et al., 2016), and for data to begin to be considered ‘findable’ they must first be available in digital formation in an online platform (i.e. on the internet). Once online, data are made more findable by having rich metadata. The term ‘metadata’ generally refers to ‘information about information’, or ‘data about data’ (Brand et al., 2003), and there are increasing calls for metadata to be treated as equally important as the objects they describe (de Waard & Kircz, 2003). However, metadata records vary greatly in their richness; that is, how much or little of the data is described and captured in the metadata record, where generally the ‘richer the metadata record, the greater the possibilities’ (Brand et al., 2003).

The term ‘metadata’ can mean different things within different settings, and there are many different ways that metadata can be classified. The ‘Metadata, Encoding and Transmission Standard’ (METS) divides metadata into three broad categories: ‘descriptive’, ‘structural’ and ‘administrative’ (Davenhall, 2011). Of these, descriptive metadata elements are the most commonly used in outward-facing online searches. For example, putting ‘keywords’ into a search engine such as Google allows sources of online information to be identified and selected as appropriate. Thus, the richer the metadata applied to a data source, the more findable the data becomes.

The creation and use of meaningful metadata are now recognised as crucial elements in providing value-added services (Simek et al., 2013). Metadata are increasingly being used to detect trends and obtain insights into social, economic and political interactions (Conte et al., 2012; Lazer et al., 2009; Oh & Park, 2018). For example, many scientific publications have reported use of Google Trends to identify changes in people’s search behaviour as indicators of changing interest in a topic (Kampf et al., 2015), measures of public health (e.g. Cook et al., 2011), economics (Kristoufek, 2015) and environmental events (Cha & Stow, 2015). Such studies have typically relied upon metadata from internet usage or high-throughput data; however, trend detection can be conducted on other types of metadata. For example, metadata from weather stations have been used to detect changepoints that indicate events such as gauge changes or station relocation (Li & Lund, 2015).

Accessibility

To maximise use once a data source has been found, the data also need to be accessible. Making data ‘accessible’ is the second step in the FAIR Guiding Principles, meaning that people seeking to use the data can access them at the defined time and by the defined method (Luque, 2019). Further, there are increasing calls to make research data and findings ‘open’, meaning that data can be ‘used, reused and redistributed freely by any person, and that are subject, at most, to the requirement of attribution and to be shared in the same manner in which they appear’ (Dietrich et al., 2015). This is especially the case for projects that are publicly funded (Chugh & Howah, 2019). Thus, the process of and results from experimental research should be open, transparent, reproducible and testable (Davenhall, 2011). Science funders, publishers and governmental agencies now often require data management and stewardship plans for data that are generated in publicly funded research projects (Wilkinson et al., 2016). These typically state that data should be published under an ‘open access’ (OA) model. OA is a set of principles and practices through which research outputs are distributed online, free of cost or other access barriers (Suber, 2004).

Data repositories

Many types of data and information lend themselves well to OA. For example, many scholarly publishers now provide authors with the opportunity to make the research manuscripts available through OA publishing models, and application of licenses such as those by Creative Commons promote sharing of research outputs. For some types of experimental and research data – particularly those from laboratory-based or sensor-driven experiments – there are a number of well-curated, deeply integrated, special-purpose open data repositories such as Genbank (Benson et al., 2013), the Worldwide Protein Data Bank (Berman et al., 2003), and UniProt (The Uniprot Consortium, 2019). A number of ‘general-purpose’ repositories such as Figshare and DataHub, have also been developed, but not all research data or data types can be captured by or submitted to these repositories, and searching repositories that hold such disparate data is often problematic. Indeed, many potentially valuable datasets emerging from traditional, low-throughput research trials don’t fit well into these repositories (Subramaniam, 2004).

Agricultural data

The use of agricultural trial data has enormous potential to improve cropping and management practices (Hyman et al., 2017). Serra da Cruz & do Nascimento (2019) identified a number of difficulties in data-driven research projects in agriculture, including a lack of appropriate infrastructure to store and preserve data and difficulty in sharing datasets. Harper et al. (2018) asserted that ‘the future of agricultural research depends on data’, and that ‘the sheer volume of agricultural biological data being produced today makes excellent data management essential’. These authors also suggest that the ‘value of data increases exponentially when they are properly stored, described, integrated and shared, so they can be easily utilized in future analyses’.

Grains trials

Within the grains and cropping sector in Australia, many thousands of field-based and on-farm research trials have been conducted by grower and farming systems groups, government researchers, universities and private industry groups with the aim of improving the profitability and sustainability of Australian grain production (Wills et al., 2019). However, the results from much of this work is traditionally retained ‘in house’ – on personal computers or institutional and private websites (Serra da Cruz & do Nascimento, 2019) that can be accessed only via subscription or membership status. The data are thus neither findable nor accessible, so the potential value from the re-use of research findings is not being realised. Further, many research topics are being duplicated in both time and space, resulting in wasted time, effort and funding investment (Sexton et al., 2019).

Identification of this urgent need for greater dissemination of research trial data and findings within the Australian grains research community led to the development of Online Farm Trials (OFT) – an open online database that provides open access to on-farm or field-based cropping research trial data and information. Hosting past and present research trials undertaken and contributed by a range of contributors throughout Australia, OFT is a source of knowledge and information to support decision making, practice change and improvements in farm profitability and sustainability.

OFT can be considered as a ‘biocurator’ (Harper et al., 2018), striving to present ‘accessible, accurate and comprehensive representation of biological knowledge’. Biocuration is the process of ‘selecting and integrating biological knowledge, data and metadata within a structured database so that it can be accessible, understandable and reusable by the research community’ (Harper et al., 2018). Data and metadata are taken from trial reports to form the basis of the Trial Projects, which are integrated with other data, including SILO and Bureau of Meteorology weather data and the Soil and Landscape Grid of Australia to deliver a value-added product to database users. OFT Trial Project metadata can be considered as ‘descriptive’, providing information about the basic parameters of each research trial project within the database. The online fields into which mandatory metadata are entered on the OFT website are Trial project code, Trial project title, Growing season year, Trial site, Crop type, Trial type, Trial design and Treatment type. These fields have been defined as the minimum information metadata elements required for the creation of a Trial Project in the OFT database. On-farm crop research trials typically follow the basic scientific procedure whereby experiments are conducted under controlled, documented conditions, and the results are used to determine the best inputs to achieve the desired outputs. However, this may not be the case for demonstration trials, and scientific publishing standards have not generally been applied within the on-farm research activities in the past, so legacy trial reports do not always contain all the required information to generate searchable metadata within OFT (Robinson et al., 2019).

Sowing timing

Sowing time is critical in determining crop yield, so getting the right sowing timing for a crop is one of the most useful ways of maximising grain yield in dryland agriculture (Sharma et al., 2008). It is generally acknowledged within the Australian grains industry that crops are being sown earlier than in the past (Anderson et al., 2016; Flohr et al., 2018; GRDC, 2011), and it could be expected that cropping research trials would be designed to follow the same practices as those being employed within the general industry to ensure results data are relevant to what growers are doing in their paddocks. However, Stephens & Lyons (1998) suggested that is not always the case, and, to the best of our knowledge, investigations into their claim have not yet been reported.

In the first study of its kind in the grains industry, we aimed to determine whether OFT Trial Project metadata could be used to detect trends in sowing dates from on-farm crop research trials across Australia, testing the hypothesis that research trial are being sown earlier in line with local farming practices (i.e. that sowing dates have moved to earlier in the year within the study period).

Western Australia

The sowing date for cropping research trials for WA SDs, with trials in ‘all crops’ moving earlier by around 1.9 days per year between 1998 and 2018; and wheat, barley and canola trials in WA were sown about 1.7, 2.1 and 2.3 days earlier each year, respectively, for the year analysed in each of these crop species (Figure 2; Table 2).

Figure 2. Correlations between mean sowing date and year for state × crop type combinations (>30 trials, R²>0.50).

The category of ‘all crops’ included barley, canola, chickpeas, faba beans, field peas, lentils, linseed, lucerne, lupins, mustard, oats, kaspa peas, triticale, vetch and wheat. Blue line indicates predicted SD, dashed red lines indicate 95% confidence intervals, and green dashed lines indicate 95% prediction intervals.

These findings correspond with multiple reports of earlier sowing of crops in general farming practices in WA. Fletcher et al. (2016a); Fletcher et al. (2016b) reported that field records from seven farms in WA showed sowing of the first cereal crop (wheat or barley) on-farm had advanced markedly in recent years, and was most prominent from 2010 to 2014. The sowing date moved from late May to late April at most sites (although the actual pattern of change was notably different at the seven sites included in the report; see Figure 1 in Fletcher et al. (2016b)) and was likely impacted by changes in management and agronomical practices, including adoption of no-till methods and herbicide resistant crop varieties (Fletcher et al., 2019). This work was based on the report by Stephens & Lyons (1998), who reported that sowing dates of wheat in WA had moved earlier by 1.2 days per year between 1977 and 1990, and confirmed that sowing dates continued to move earlier from around 1995 to 2015. Flohr et al. (2018) also confirmed the general shift, reporting that wheat sowing date records from the Yield Prophet database (the online commercialised version of the crop production models APSIM) in WA show a shift of around 1.3 days/year over the 10-year period from 2008 to 2015. Farre et al. (2019) asserted that trends in earlier sowing in WA over the last decade also apply to canola crops, and used APSIM-canola simulations to establish the optimum sowing window to maximise grain yield for different locations in WA. A report by DPIRD WA (2019) also noted that ‘in the last decade there has been a trend toward earlier sowing of canola by Western Australian growers’.

Results from the OFT metadata analysis show that sowing of crops in WA research trials reflect the trends seen in general practice in this cropping district, and extends the current knowledge to show that the trend is continuing past 2015, at least as far as 2018, and possibly beyond.

Victoria

In Vic., the SD for cropping research trials for ‘all crops’ moved earlier by around 1.3 days per year between 1993 and 2018; wheat, barley and canola trials in Vic. were sown approximately 1.9, 1.6 and 1.6 days earlier each year for the years analysed for each of these crop types (Figure 2; Table 2). This result is similar to the data from the Yield Prophet database showing a rate of change of 2 days/year between 2008 and 2015 for wheat in Victoria (Flohr et al., 2018). However, it differs from findings of Stephens & Lyons (1998), whose survey work showed no change in sowing date in the state between 1977 and 1990. These authors note that their data were based on only five survey responses (the least number of any state), so ‘little confidence can be placed on the results’. The OFT metadata suggests that sowing of research trials in Vic. has moved forwards during the study period, in a similar fashion to WA, and thus reflect general practice in cropping across the state.

Other states

We detected weak trends in SD in three of the state × crop type combinations in these states: NSW barley R² = 0.3771, SA ‘all crops’ R² = 0.3405 and SA barley R² = 0.3497. R² values for all other NSW, Qld, SA and Tas. state × crop type combinations were very weak (< 0.3771), suggesting no clear or consistent relationship between SD and year.

These results differ from those of Stephens & Lyons (1998), who reported that ‘during the 1980s, sowing progressed a day earlier each year at a national scale’. For NSW; however, they note that there were large standard deviations in sowing (wheat) midpoints (see their Figure 4), averaging 21.2 days. Flohr et al. (2018) show that NSW wheat crops were planted 1.1 days/year earlier between 2008 and 2015, but note that southern NSW had the lowest number of fields subscribed to Yield Prophet and that there is a very broad sowing window in this environment. These authors reported sowing of wheat crops in SA has moved 1.3 days/year earlier in the same period. Maitland (2013) reported in the South Australian No Till Farmers Association (SANTFA) newsletter that ‘in recent years, farmers have sown crops earlier in the season’; however, this report contains no data, and thus provides little evidence on which to base further analyses. No published data could be found for Qld or Tas.

Trend detection

There are several possibilities that could explain why we detected no strong trends in SD in OFT metadata for NSW, Qld, SA and Tas. First, reports of earlier sowing of crops in paddocks may be anecdotal or outdated, and crops were not actually sown earlier in these areas during the period included in our analyses. Stephens & Lyons (1998) surveyed wheat farmers undertaken between 1978 and 1990, which is several years before the earliest record used in our analyses, and almost 20 years before the bulk of the data used here. These authors noted that the national trend towards reduced or minimum tillage techniques coincided with their reported earlier sowing dates, so it is possible that once any farmers who were adopting these different management methods had done so, sowing dates ceased to move any further forwards. Other reports providing data regarding sowing trends in these states were from the Yield Prophet database, which is biased towards early adopters of technology (J. Hunt, pers. comm.). Thus, it is possible that sowing dates for crops in NSW, Qld, SA and Tas. have not changed significantly in the years included in our analyses.

A second possible reason why no notable trends between SD and year were detected for NSW, Qld, SA or Tas., is that research trial SDs in these areas may not reflect general practice in the region, so in fact have not been sown earlier across the study period even if farmers were sowing crops earlier. If the main reason why farmers are sowing crops earlier is increased farm size, then the need for earlier sowing is negated in research trials, meaning they are simply not sown earlier.

Third, it may be that trends in SD exist only within smaller geographic regions within each state, and so have been masked by separate agro-ecological zones. Sowing dates are known to be strongly influenced by geographical regions (NSW DPI, 2019), driven by variation in a plethora of environmental variables such as rainfall (particularly in autumn; Bloomfield et al., 2018), spring temperatures and frost risk (Hunt et al., 2019). Large-scale rainfall anomalies have been cited as a driving factor for sowing dates, especially in states with a distinct Mediterranean climate (Stephens & Lyons, 1998). Frost risk was recently reported to vary considerably across the northern grains region, and manipulation of sowing time was identified as one way to minimise yield losses (in chickpeas) due to frost (Chauhan & Ryan, 2020). There are likely many reasons that have contributed to this change but investigations into and discussion of the agronomic factors driving earlier sowing are beyond the scope of the current investigation. However, our work demonstrates that OFT provides a useful source of information, and could be used to investigate trends within, for example, different agro-environmental zones or across different rainfall gradients.

OFT metadata

The OFT database currently provides for the inclusion of exact and accurate geolocation of a trial in the form of latitude and longitude. If entered, this information can be displayed or, for privacy reasons, hidden from the public view at the request of the contributor. Whether hidden or displayed, it can be used to accurately geo-locate a trial site for which climatic variables can be derived for use in analyses. However, few legacy reports contain accurate location information, and even where it may be available, the information is not always entered into the database because it is an optional field. Accurate geo-location (e.g. measurements made via a global positioning system (GPS) could be useful in future analyses, and the location of research trials should be recorded and entered into the database.

The present process of Trial Project creation in OFT is one of manual biocuration, and requires a multidisciplinary effort involving subject area experts, software and technical developers, researchers and project staff. The process of manual biocuration typically involves reading of the trial report and entering data manually into the database. It requires a good understanding of both the research work being entered as well as the functional capacity of the database itself. The original Trial Project entry process for OFT was conducted via a spreadsheet import process, which was managed in-house. Once an upload of projects was completed, the contributor was notified and asked to check that the information had been entered and represented correctly before it was published to the live site (online). However, this process was labour-intensive and slow, and required members of the OFT team to facilitate data entry and publication, so an ‘administration’ centre was developed to allow contributors to enter their data directly to the OFT database without input from the in-house OFT team. This made it easier for contributors to enter data and removed the need for double-handling of trials, however, it simultaneously introduced the problem of quality control. Without the need for Trial Projects to be checked by a member of the OFT team before being published, entry of non-mandatory metadata had not been monitored. Harper et al. (2018) note that manual biocuration is perhaps the best way to curate data, but no database has enough resources to curate all data manually. Investments into the Australian grains industry have been recognised as critical drivers for achieving future productivity gains essential for the sustainability and profitability of cropping enterprises (Walters et al., 2018), so it will be important to evaluate the benefits against the costs of collecting more metadata within the context of ongoing OFT database curation and quality control. There is generally a time investment required to collect metadata, and it is recognised that enriching existing metadata records can be ‘difficult and time consuming’ (Kemp et al., 2018), so recognition of the trade-off remains an important consideration in the collection of metadata for OFT Trial Projects.

For wheat in particular, the trend of earlier sowing dates may have been facilitated by an increase in use of winter wheat varieties investigated in these areas, as the trend towards earlier sowing is reported to have resulted in the planting of more longer-season varieties and less shorter-season varieties (Hamblin & Kyneur, 1993; Stephens, 1995). There is currently no metadata field for variety in OFT, thus, the possible role of varietal-driven differences in sowing date trends could not be accounted for in our analyses. Future development of OFT Trial Project metadata to include variety could be highly beneficial in understanding the role of variety in sowing date trends across the different Australian cropping regions.

Future trends

In our analyses we used simple linear regression, and results suggest that in some areas, research trial crops are continuing to be sown earlier (up to the end of analyses, which was ~2018–19). Simulation studies of wheat in WA suggest that the optimal flowering period (and by extension, sowing date) may move earlier by as much as 29 days under a drier climate (Chen et al., 2020). This raises the question of how much earlier can crops be planted before the advantage is negated, i.e. how many more years will the current trends persist, and what will be the best way to continue to monitor ongoing shifts in sowing dates in the future to allow for the expected effects of ongoing climate change on crop phenology (Kukal & Imrak, 2018)? As Trial Projects from current and future research trials are added to the OFT database, further analyses may show further changes in SD trends, and these could be useful in predicting sowing dates to be used during the planning of future research trials. Further, the question of whether earlier sowing in research trials has led to the expected benefits in terms of crop yield has yet to be investigated. At the present time, there is much information in OFT that is not captured in metadata fields, but future development to improve the richness of the metadata would enable these questions, and many others, to be investigated using Trial Project metadata from the OFT database.