Developing sustainable software solutions for bioinformatics by the “Butterfly” paradigm

Introduction

Typical challenges in bioinformatics in an academic environment include “ad hoc” programming. No maintenance is really possible as scientists such as PhD students and post-doctoral scholars leave after their thesis is completed or after their post-doc contract. These scientists may also have no formal computer science training, and often there is no structured programming¹ and solutions might not be compatible with each other. Furthermore, in an academic environment there are a number of inherent pressures to develop pragmatic and fast (“quick and dirty”) software solutions².

In addition, there are some new and “modern” challenges, which become more and more pressing simply as the technology progresses: big data³, the wave of “omics” data to process, and the problem of interoperability of software tools⁴. Well-known recent solutions for this challenge are Taverna⁵ and Galaxy^6,7. The latter in particular is well suited to dealing with large quantities of data such as new large-scale sequencing data.

Another issue is that the data should be accessible, with uniform syntax and rich semantics for integration. Furthermore, data schemes are prone for change due to rapid advances in the field, so a schema-free representation of data is increasingly important (for scientific data). The UniProt Consortium⁸ for instance has recently shifted from the use of relational databases to the semantic web for flexible data management.

To counter these older and general as well as new challenges, we have now developed a solution of iterative and intertwined development cycles (Butterfly model), which improves the typical aspects of long-term sustainability and maintenance. Furthermore, it features detailed user-requirement analysis, good graphical and simple user interfaces (optimized human-computer interface, HCI) and intuitive software use that also exploits natural language processing. Our model tackles current challenges: first, the interoperability of the software takes into account middleware solutions, so that both database and user interfaces can be used flexibly. Second, the Butterfly approach improves the meticulous database engineering required to build large-scale and “omics” databases.

The Butterfly development cycle is different from previous approaches^9–12. It requires some additional time investment at the start for design and implementation, but this pays off later. The Butterfly model worked well in our hands regarding the above challenges (section: Real time examples using Butterfly). In the following we confirm this by describing concrete software development.

The basic concept is really simple: plan ahead, back-check the critical development steps by a separate sub-cycle and talk with the user. It is important to spend more time on requirement analysis, as well as to invest well in interoperability and maintenance. We do not claim that these problems have not been recognized before nor that no alternative solutions for this are available^2,6–8, but we are confident that with our approach it will be possible to obtain particularly well optimized and high quality bioinformatics software solutions in an academic environment. The initial time investment in the Butterfly paradigm helps to save time later due to the interoperability and easy maintenance of the software solutions achieved.

This paper is organized as follows: this section sets the stage for our agenda (section 1); Current Software Engineering and Development (section 2) highlights the modular phases of software development processes and compares several typical software strategies highlighting the novelty of our approach; next, the Butterfly work flow is explained (section 3), and software examples using the Butterfly approach (section 4) validate the Butterfly design principle by providing concrete examples of software projects from own work. Moreover we discuss some bioinformatics tools based on their type, methodology and usage.

Current Software Engineering and Development

Software Engineering (SE)⁹ is one of the most recognized fields in computer science as it matures and expedites the processes of software development. In particular, it allows a focus on the life cycle of software and sustainable development as an improvement to pragmatic short-lived implementations. SE has introduced many process improvement models and techniques^10–12, and Software Development Life Cycle (SDLC) models¹³, with some variabilities¹⁴ and commonalties^15–19. In general, depending upon the observed commonalties, we state

“Software Engineering is an integrated, cyclic and product line combination of following independent modular approaches: requirements engineering^20–22, design modelling^{23–25,27–35} programming, testing and deployment”.

The five modular SE approaches remain the same when it comes to the software engineering of the scientific software solution development (Figure 1). However, in contrast to a pragmatic and maybe traditional software application development in an academic setting (Figure 2), a major change is the inconsistency in all phases of the SDLC. In the requirement engineering phase (Figure 2; traditional software solution development), all requirements should be provided before the start of design. This is not the case when dealing with most of the scientific software applications, and the requirements continuously change with the passage of time (we have proposed an updated SSE SDLC Model, Figure 1; scientific software solution development). Ultimately, this complicates the process of analysis and filters out functionals. Programming structures become complex (Figure 1), as the possibilities of error proneness (both logical and syntax errors) increase due to the continuous increment of variabilities in the pre-processed source code^15–19.

Figure 1. Scientific Software Engineering (SSE).

SSE integrates and combines in a development cycle the following independent main modular approaches: requirements engineering, design modeling, programming, testing and deployment. Each approach consists of its own sub-modular, integrated and cyclic combination of internal phases: requirement engineering consists of specification, functionals, non-functionals, and analysis; design modeling consists of use cases, system flows, data flow and source code; programming consists of languages, tools and technologies, development, and debugging; testing consists of test cases, modular, integrated and quality; finally, deployment consists of installation, configuration, training, feedback. Iterative cycles lead to continuous improvement. Achievements translate the goals in good software.

Figure 2. Traditional Software Development consisting of integrated and cyclic combination of the following independent modular approaches: requirements engineering, design modeling, programming, testing and deployment.

Testing of integrated and individual modules becomes time consuming (Figure 1), as new test cases have to be continuously rewritten and their application often leads to ‘ripple effects’²⁹: these are unidentified logical or syntax errors in the system which arise while fixing the errors^36,37. Depending upon the nature of the system, many approaches have been proposed to improve software quality control processes^38–50 which improve standard software development and are important in scientific software quality assurance and improvement. Furthermore if the system keeps changing and is inconsistent, then the deployment procedures can also be complex and time consuming, especially for large applications with multiple interfaces and controls providing numerous individual and integrated functionalities.

To further help (SSE, non-computer scientist bioinformaticians etc.) in expediting the processes of adopting the concepts of SDLC, we provide a tabular comparison between different SDLCs, based on their commonalities and variabilities (Table 1). This comparison is based on following twenty four defined comparative SDLC (authors’ initiated) measures: Software Engineering Approach⁹, Initial, Developmental Plan, Software Requirements Engineering, In Depth Requirements Analysis⁵¹, Requirement Validation, Functionals, Risk Analysis, Software Design, Software Architecture Design, In Depth Software Design Modelling, Reusable Designing, Developmental Plan, Tools and Technology Selection and Analysis, Graphical User Interface Design, Preprocessed Source Code Writing^15–18, Integrated Programming, Software Testing, In Depth Software Testing, Customer’s Evaluation, Deployment Procedures, Maintenance, Software Re-Engineering⁵², Cyclic or Repetitive, Easy to learn and Use, and User Training.

We applied these measures to nine different software development life cycle models: Waterfall Model⁵³, V-Model⁵⁴, Spiral Model⁵⁵, Iterative and Incremental Model⁵⁶, Rapid Prototype Model⁵⁷, Agile Development Model⁵⁸, Extreme Programming Model⁵⁸, Evolutionary Model⁵⁹, Code and Fix Model⁶⁰. From this comparative analysis we conclude that there is no such one specific SDLC which can be helpful in all required phases of scientific software solution development, but some which might be more useful: Spiral, Waterfall and V-Model. Moreover the SDLCs famous for the quick development (Rapid Prototype Model, Agile Development Model, Extreme Programming Model, Evolutionary Model, Code and Fix Model), lack in quality software production.

As this is a general and open comparison, depending on the nature of the scientific software application, one can further analyze and pick that which suits best. Furthermore, we considered only the typical effort and strengths for each of these software development paradigms. A meticulous developer can of course take special care and spend more time on any of the features not typically covered by the software paradigm he follows, and turn the “no” for this feature into a “yes” simply by this additional effort during SDLC (for instance, regarding agile programming – for that matter, extreme programming can also be considered as a type of agile development). The goal for our “Butterfly” paradigm is a SDLC paradigm that fulfils all of the features regarding life cycle management of the resulting software.

Butterfly in scientific academia

If we search for “bioinformatics tools” over the web, thousands of entries can be found at one hit. But how many of those are fully designed, developed and tested solutions, used and maintained for a number of years and still in functional use?

At the beginning of a software project in academia, scientific solution development seems very interesting, fascinating and exciting. But with the passage of time, when the levels of complexities increase (due not only to the lack of developmental skills but also to the unavailability of proper designs), the work starts becoming tedious and unfruitful. This causes a lack of interest in software solution development and leads to a preference for wrapping up the work with a working script or small application, which can be published later on.

Here we propose a new science-oriented model (Figure 3), which can help the scientific software solution developers as well as the scientists/end users by generalizing the use of major developmental aspects correlating to the important scientific needs of the target system. The name of our new model is “Butterfly”.

Figure 3. Butterfly model.

It consists of four wings: Scientific Software Engineering (upper left), Human Computer Interaction (lower left), Scientific Methodology (upper right) and Scientific Application (lower right). Moreover it leads to continuous improvement (in yellow). The achievements translate the goals into software.

In accordance with the name of the model, the ‘Butterfly’ represents sustainable and continuous improvements from goals to achievement as the central backbone of development. For larger projects this is further improved by using middleware between user interfaces and accessing the various data and databases involved. This also has the advantage that natural language processing needs to be implemented only once in the middleware and all the other modules have access to it. Finally, the backbone arrangement with a powerful middleware as well as all development elsewhere in the “Butterfly” stresses the interoperability of the software. It is developed by using well-defined and compatible output formats. Furthermore, below the middleware, well adapted, interoperable data schemes boost sustainable development of database structures, including efforts for scheme-less databases and other semantic web developments. The four continuously moving wings (Figure 3) represent the change in the requirements. The upper left wing of the model represents the scientific software engineering principles, and the lower left wing represents the HCI. The upper right wing represents the implementation of the specific methodology (focused on lowering the risk of development ripples) and the lower right wing represents the target scientific application producing the required results.

Butterfly workflow designs

The Butterfly model implementation mechanism has a three-layered architecture: Gray, Yellow and Green (Figure 4).

Figure 4. Butterfly three layer model.

Shown in gray is the abstract layer, in yellow is the basis for design and development, and in green the implementation and testing by the user. The software is released including installation and training.

Gray layer

The gray layer represents the most important phase of scientific software solution development, which involves software designers, developers, testers, graphical interface designers and, most importantly, the users. It consists of four phases: scientific software solution planning, requirements engineering and analysis, conceptual software design and modeling, and user friendly graphical interface. The layer has been named ‘Gray’ because at the beginning of a new scientific software solution development, most of the information seems uncertain.

Scientific software solution planning (Figure 5) is the first step towards a new scientific application development, which requires the introduction to the field itself (e.g. biochemistry, neurobiology, genetics, metabolomics, proteomics etc.) and project related information (e.g. what could be the end product, input to the system, expected output from the system, methodology, ideas, opinions etc.). It is important to know about the user’s information IT background and existing already available (old and recently developed) scientific solutions to the problem. The next important phase is to perform requirements engineering and analysis (Figure 6). During this phase, the most important tasks are to gather the requirements from users (e.g. interviews, brainstorming, documents, publications, running related systems based information etc.) and classify the information in functional and non-functional categories. Here, function requirements are those which can be implemented (based on existing resources, time, budget, labor, tools, technologies and methodologies), and non-functional requirements are those which cannot be implemented. It is very important to clarify with the users what will be the expected end-product because it is possible that the user may not like the output of the system after development. In this case, all the efforts will have been in vain.

Figure 5. Scientific software solution planning.

Abstract planning is the first step of the top, abstract layer (gray, Figure 4) of the Butterfly design, key steps are indicated.

Figure 6. Requirements engineering and analysis.

This is the 2^nd step of the top layer (gray layer) of the three layer model in the Butterfly design. Key tasks are indicated.

The third phase is the conceptual software design and modeling (Figure 7). This is particularly important when there is a team of software developers. Before moving ahead, one should go for some abstract designs based on functional requirements and discuss these with the team. It is crucial to estimate the expected workflow, data sources and data flow in the system. If possible, the abstract design should be discussed with the users as well.

Figure 7. Conceptual software design and modeling.

This is the 3^rd step of the top layer in the butterfly model.

The last phase of the gray layer concerns the design of a user-friendly GUI (Figure 8). First, some mock-ups should be made (hand-made on papers or better to use white board with color markers and later make pictures of finalized GUI designs), then these should be discussed with users in a brain storming session. Finally, based on the perceived designs, the abstract GUI (GUI with no running functionalities) should be created using HCI design patterns.

Figure 8. User Friendly Graphical Interface Designing.

This is the final step of the top, abstract layer (gray) of the three layers in the butterfly design.

Real time examples using Butterfly

A number of new scientific software applications based on the concepts of the Butterfly model have already been proposed, designed, implemented, tested and are currently in use. Some of them have been published and some remain unpublished. These applications are: Least Square MIDA (LS-MIDA), DroLIGHT, Isotopo, Lipid-Pro and App Ant Database.

LS-MIDA^69,70 (Figure 9) is an own published scientific software (Department of Bioinformatics, Biocenter, University of Wuerzburg Germany) which estimates mass isotopomer distribution from the spectral data by analyzing each peak of given mass and each mass atom fragment. It implements a chain of mathematical and statistical algorithms, provides graphical interfaces to help users analyze experimental data and visualize the results, and provides a third party independent experimental data management system. For the most parts the requirement engineering and software design steps are followed until the final solution presented (Figure 9), and details on tested earlier solutions or specific requirements are available from the authors.

Figure 9. Least Square Mass Isotopomers Distribution Analysis’s (LS-MIDA) main graphical user interface.

Scientific software solution towards bioinformatics and biochemistry which estimates mass isotopomers distribution from spectral data by analyzing each peak of given mass and each mass atom fragment. (http://www.tr34.uni-wuerzburg.de/computations/ls_mida/).

DroLIGHT^71–73 (Figure 10) is a published scientific computational solution towards neurobiology and photobiology (Department of Neurobiology and Genetics, Biocenter, University of Wuerzburg, Germany). It is a domain specific, intelligent, distributed, real time, embedded, data management system capable of controlling hardware devices, proficient in producing different colors of lights and monitoring the movements of Drosophila melanogaster. Moreover it also helps users generate circadian rhythms, provides a third party independent experimental data management system and produces experimentation output in two and three dimensional graphics formats. Constant improvement is further boosted by sustainable data structures and engineered robustness of the application. Furthermore, sustainable development and interoperability are considered from the start, support the rapid further development/improvement of software to study and manipulate circadian rhythm by different light cycles and test environments in drosophila.

Figure 10. DroLIGHT’s main graphical user interface.

Scientific software solution towards neurobiology and photobiology, capable of controlling and automating the hardware that produces different colors of lights via Light Emitting Diodes (LEDs). It provides experimental data management system, circadian rhythm generation and 3D visualization of system’s performance and experimentation details. (http://www.neurogenetics.biozentrum.uni-wuerzburg.de/en/project/services/drolight/).

Isotopo software⁷⁴ (Figure 11) is a published scientific software (Department of Bioinformatics, Biocenter, University of Wuerzburg, Germany), a bioinformatics solution with the ability of performing quantitative mass spectrometry in isotope labeling experiments. It is an extended version of the earlier software LS-MIDA with a well-optimized mathematics implementation. It not only provides the graphical interfaces for gas chromatography-mass spectrometry (GC-MS) experimental data analysis, visualization and management, but also provides an intelligent data parser to automatically transform the machine’s pre-processed data into the processable format of the Isotopo software (reducing both time and labor). It also provides a complete database management system as a simple, well sustainable version of a middleware between data storage and GUIs.

Figure 11. Isotopo Data Analyzer’s main graphical user interface.

Scientific software solution towards bioinformatics and biochemistry, with the ability of performing quantitative mass spectrometry to mixtures of materials labeled with stable isotopes. It provides internal database management system, third party independent file based experimental data management system and intelligent data format parser for data extraction and conversion of different data formats. (http://spp1316.uni-wuerzburg.de/bioinformatics/isotopo/).

Lipid-Pro (Figure 12) is an unpublished scientific software (paper is in writing), a computational solution towards lipids and pharmaceutical biology, estimating fragment candidates, pre-parent candidates and most importantly lipids (Department of Pharmaceutical Biology, Biocenter, University of Wuerzburg, Germany). It implements a chain of mathematical operations and experimental data extraction processes. Along with the efficient third party independent data management system, it provides different graphical interfaces for individual and integrated intelligent operations.

Figure 12. Lipid-Pro’s main graphical user interface.

Scientific software solution towards lipids and pharmaceutical biology, which estimates fragments, pre-parent candidates, fragment candidates and most importantly lipids. (http://www.neurogenetics.biozentrum.uni-wuerzburg.de/en/project/services/lipidpro/).

App Ant Database (Figure 13) is an unpublished scientific software (paper is currently in writing), featuring a distributed and embedded database system in the form of a smart phone, tablet and desktop application towards the rotating channel experiment data management during experimentation on desert ants (Department of Behavioral Physiology and Sociobiology, Biocenter, University of Wuerzburg, Germany). It is unique and the first bioinformatics smart phone application to be used in desert ant rotating channel experiments. After extensive requirement engineering, we established an extremely easy to use graphical interface. Furthermore, after studying the user requirements in monitoring desert ant movement and orientation in the desert, the application not only automatically records ant movements, but also estimates and calculates automatically all additional variables required for the project such as azimuth, solar time, equation of the time, time offset, hour angle, altitude, sunrise, sunset and solar noon using astronomical algorithms, recommended by the National Oceanic and Atmospheric Administration (NOAA).

Figure 13. App Ant Database’s smart phone graphical user interface.

Scientific software solution towards the rotating channel experiment data management during experimentation on desert ants. It offers user friendly graphical interfaces for the experimental data entrance, manipulation, management and sharing. (http://www.neurogenetics.biozentrum.uni-wuerzburg.de/en/project/services/ant_app_db/).

Regardless of the individual specifications, development details, technologies used and usage perspectives, the key principles of the Butterfly model were applied in the design of these software solutions. Although the requirements for each of these were not fixed in the beginning, comprehensive requirements gathering and analysis operations were performed using brain storming and interviewing methods. Based on filtered out the functional requirements, the most suitable SDLCs (V-Model⁵⁴ and Spiral⁵⁵) were applied and the software applications were designed using UML (including use cases, class diagrams, sequence diagrams, work flows, activity flows, components diagrams, data flow diagrams etc.). Of those SDLC’s used in scientific software solution development, the Spiral Model proved most helpful and best suited due to its four main pillars: determine objectives, identify and resolve risk, development and test, and plan the next iteration. Another advantage of the spiral model is its risk driven approach, incorporating many useful features and refinements of other software development life cycle models^54,72.

Using the HCI design patterns and principles, the graphical user interfaces of all these applications were designed considering the psychology, scientific and informatics backgrounds of the end users and the deployment environments.

All these applications are easy to deploy and use. We found that users did not require training to install, run and use the applications. As scientific research is a never ending process, these applications are still in development and will be continuously improved with respect to the methods, features, performance and technologies.

Comparing bioinformatics tools

We have performed a short comparative analysis of some bioinformatics software applications (C13⁷⁵, Metatool⁷⁶, BioOpt⁷⁷, FiatFlux⁷⁸ ReMatch⁷⁹, Biolayout⁸⁰, LS-MIDA^69,70, DroLIGHT^71–73, Isotopo⁷⁴), describing their type, methodology, implementation, user friendliness, configuration etc., based on the provided, published information (Table 2).

For our comparison, we chose software applications from fields we are familiar with, yet tried to cover a broad range of different applications and compared Metabolic Flux Analysis (MFA) as well as software metabolic network analysis^81,82, Mass Isotopomers Distribution Analysis (MIDA) including GC-MS data analysis^69,70, and neurobiology applications for behavioral analysis of insects such as desert ants and fly^71–73.

We used the following parameters to classify the chosen bioinformatics software application: SSE type, data management, script or prototype, algorithm type, algorithm/methodology, running mode, publishing, licensing, SDLC information, HCI information, user friendly, easy to configure, easy to train, software re-engineering, cyclic or repetitive, easy to learn and use user training.

From the observed results (Table 2) we conclude that almost all of the applications have good implementations of their methodology (algorithms etc.) but often lack in usage point of views (user interface, documentation etc.), or long term sustainable development of the software (at least regarding the organization of future further developments). These shortcomings are prevented when following the “Butterfly” paradigm.

Conclusions

In the earlier sections of this paper we presented the concepts of usage of the existing scientific software solution design, modeling, implementation, testing and deployment. This helps in resolving conflicts and highlighting valuable differences between the traditional (professional) and scientific applications. This paper also proposes a new approach towards user friendly scientific software solution development, with emphases on the use of proper SDLC, HCI and technologies. The successful implementation of the five applications discussed strongly validates the potential of the Butterfly model.

In conclusion, although the adaptation of SSE principles to the Butterfly model may seem to increase developmental work load in comparison to the current running programming method applications, the Butterfly model will ultimately reduce the work by making the scientific application well designed, flexible, structured, reusable, developed according to a product line, as well as analytical and of high quality. According to its design, the software developed using the Butterfly paradigm is user friendly, easy to learn and deploy.