Powered by OPF
OPF WIKI STATIC ARCHIVE
See the original ↗Page history
2,681 pages · 153 spaces · 776 tags · 4,025 history records · 96.2% of the original wiki recovered
Archived copy. This page was recovered from the Internet Archive snapshot of /display/SP/The SCAPE Functional Review Process taken on 2016-08-18. The original wiki at wiki.opf-labs.org is being decommissioned.

The SCAPE Functional Review Process

Created by Andrew Jackson on Jul 13, 2011 · last edited by Carl Wilson · on Dec 02, 2013 (view history) · 49 versions

Introduction

The aims of of the SCAPE functional review process are to:

This page presents an overview and of the process, and a schedule with accompanying goals.

Overview

What is a Functional Review?

Overall, the aim is to audit all of the software produced by the SCAPE project, with emphasis on examining the following criteria:

  1. Development Process
    • Is the code in a publicly available repository (i.e. Open Planets GitHub site)? (MUST)
    • Is there a continuous build and automated unit testing process in place? (SHOULD)
    • Does the project have an active issue tracking system, e.g. GitHub issues? (SHOULD)
  2. Code Quality
    • Does the project adhere to coding standards?  (MUST)
    • Do unit tests exist? What is the unit test coverage, and is it appropriate? (MUST)
    • Does the project build and run? (MUST)
  3. Documentation
    • Does a basic functional specification exist? (MUST)
    • Does a README file exist? (MUST)
    • Is there adequate software deliverable documentation, covering building, installation, administration and usage? (MUST)
    • Are the development team members documented? (MUST)
    • Are dependencies/licenses clearly identifiable? (MUST)
  4. Functional Evaluation
    • Does the software satisfy its functional specification?
    • How has this been established?
  5. Installation and Deployment
    • Is a downloadable binary package available?
    • Is there an official SCAPE supported installation (e.g. on a central instance)?
    • Have Debian Packages been created of the software?

Functional Review Criteria

The following section outlines the criteria used for functional reviews. In the first instance, this will be an initial set of criteria; it is fully expected that, with input from other developers, these criteria will evolve and expand to suit the project’s needs in future functional reviews. Not all criteria have to be met, and the maturity of the component will be taken into account, for example, new projects will not be expected to have produced installation documentation.

Development Process

Code Availability in Publicly Available Repositories

All project source code MUST be stored in the publicly available Open Planets Foundation's GitHub repository to provide a central repository of code and to ensure its sustainability beyond the project. See the GitHub Project Structure page for details about the appropriate place to store code in the repositories.

Continuous Build/Integration and Automated Unit Testing

Each project SHOULD have a continuous build process, i.e. all new check-ins should trigger a compilation and execution of unit tests. 

The SCAPE and Preservation Watch projects currently use the OPF-Labs Bamboo server for continuous build and reporting. This has suffered from a number of problems surrounding integration with GitHub, often resulting in periodic builds failing. OPF-Labs is not tied in to using Bamboo and the suggestion is to move to using Jenkins (or another CI service). Until an appropriate replacement plan has been defined and agreed, SCAPE will continue to use the OPF Bamboo server.

Issue Tracking

Each project SHOULD have a means to track bugs and other development issues.

If a project does have such a process in place, it is RECOMMENDED that it use the GitHub issues tracking facility associated with the project’s GitHub repository. Utilising this mechanism will ensure the long-term association between the code and the issues log.

Code Quality

Coding Standards

Specific coding standards are described for each language used.  As described in the Technical Implementation Guidelines, Java SHOULD be used for new developments, and is the preferred platform language.

Documentation

Functional Specification

Every project MUST have a basic functional specification describing the functional intention of the code. This MAY be described in a README file for young projects.

README/Getting Started

Every project MUST have a README file describing how to get started with, or use, the software. This MAY include a functional specification.

Software Deliverable Documentation

Official project software deliverables MUST provide documentation covering building, installation, administration and usage of the software.

SCAPE Deployment

If there is an official SCAPE supported installation, e.g. the SCAPE central instance, or a web server provided by one of the partners, then it MUST clearly be documented where it is hosted, who can use it, and what are the support / sustainability arrangements?

Documented Development Team Members

A primary developer contact (at least name and email) MUST be listed in the relevant Maven POM <developers> element. This developer will be the primary point of contact for the software.

Additional developers SHOULD also be listed in the <developers> element of the POM.

Dependencies/Licences

Dependencies and Licensing information MUST be clearly documented and published as described in Technical Implementation Guidelines.

Installation/Deployment

Downloadable Binary Package

All released components MUST provide a downloadable binary package, for example a GitHub binary download that works on at least one OS.

Debian Package/Windows Installer

SCAPE software SHOULD be packaged as Debian packages. This is important for sustainability and for encouraging SCAPE software to be adopted by the wider DP community.

SCAPE Debian packaging instructions are available - Debian Packaging

Schedule

The functional review schedule is presented as a set of SCAPE project milestones, taken from the SCAPE description of work. Each milestone is accompanied by a date and a set of goals which should be met for the milestone.

MS17 - Initial Review

The initial review will be lighter than subsequent ones reflecting the effects of the backlog of effort caused by the changes of Technical Coordinator, the fact that many of the software products are at an early stage of development, and the need to establish the review process itself.  

Of importance to the review process will be the involvement of other project members, particularly those from the TCC group, but ultimately anyone responsible for developing project software. This review will serve as a starting point for encouraging that involvement, providing an initial set of guidelines and infrastructure which can be developed as appropriate by project members. 

The next Functional Review (MS18) is scheduled for January 2013 (M24), with the intervening time seen as a target for engaging other developers in the process.  It is hoped that they will be able to shape and direct the process and guidelines to suit, in particular for developers working with programming languages other than Java.

A selection of projects will be reviewed in January, based on the criteria identified in these wiki pages, with the 2nd Year Review Ramp-Up/Developers workshop providing an opportune time to discuss the process with developers and to review software.

First Review Goals

The primary goal of this first functional review is to establish the initial Functional Review process and criteria, in particular to identify:

  1. A catalogue of SCAPE software components;
  2. Initial automated build and deployment practices;
  3. Initial coding standards and guidelines;
  4. What Continuous Integration platform should be used (remain with Bamboo or move to Jenkins/other?);
  5. How to restructure the SCAPE Github repository.

The above goals form the initial foundations upon which subsequent Functional Reviews will be based. The aim is to put in place initial principles, guidelines and infrastructure to help deliver fully functional and maintainable software. Future reviews will evolve these processes and criteria as appropriate.

MS18 - Functional Review Milestone

The second functional review builds upon the groundwork established by the first review by promoting, encouraging and helping the adoption of the coding guidelines. Specifically:

  1. Promoting the functional review process at the developers and all-staff events
  2. Identifying volunteers to form an initial peer-review team
  3. Restructuring of the SCAPE GitHub repository
  4. Actively encouraging and helping developers to adhere to the guidelines
    1. getting code moved to the OPF/SCAPE GitHub account
    2. ensuring code builds successfully
    3. ensuring that developer's create readme's and specify licenses
  5. Reviewing and encouraging developers to add their project to the SCAPE software components catalogue
  6. Creation of an OPF Jenkins Continuous Integration platform for larger, more mature projects (moving away from Bamboo) and the use of Travis-CI for smaller projects
  7. Identifying targets for the following Milestone MS19, see below.

MS19 - Functional Review Milestone

The following task lists constitute the MS19 goals and the progress made towards them captured on the milestone close.

Many of the tasks are unfinished, although that is because many are ongoing until the project end. Status and where possible a metric for all tasks was added when MS19 was closed. All unfinished tasks were copied into MS20 so that progress can be tracked through to the next milestone.

Continuous Integration for ALL SCAPE software projects

Scheduled Code Quality Builds for Software Deliverables

MS20 - Functional Review Milestone

The task list and goals for MS20 are still been created but will be available for 20/12/2013/

The following lists a set of goals that together constitute MS20.

Ongoing Maintenance

The following tasks were brought up to date for MS19 but now require ongoing checking to make sure new projects comply with project guidelines.

Subsequent Reviews

Subsequent Functional Reviews will need to consider: