Gung Ho Phase 2

Lead Research Organisation: Imperial College London
Department Name: Dept of Computing

Abstract

Historically, major improvements in the accuracy of numerical weather forecasts and climate simulations have come from the increased resolution enabled by the exponential growth in computer power. In order to achieve further gains in accuracy through further increases in resolution, it will be necessary to exploit the massively parallel computer architectures that are becoming available. However, current state-of-the-art operational algorithms are not expected to perform well beyond a few thousand processors: the grid structure of the traditional latitude-longitude grid means that interprocessor communication eventually but inevitably becomes a bottleneck.

The overall aim of the proposed project is to develop a new, three-dimensional, fully compressible dynamical core suitable for operational global and regional weather and climate prediction, as well as for research use, on massively parallel machines, and to demonstrate its accuracy, efficiency, and scalability. The accuracy should be comparable to that of existing state of the art algorithms. The algorithm must be efficient enough to run in the available operational time slots, and it must scale well on 100,000 to 1000,000 processors.

Phase 1 of this project (Feb 2011 - Jan 2013) addressed several of the basic scientific questions that underpin the development, including choice of quasi-uniform horizontal grid, choice of horizontal discretization, choice of transport scheme, time integration scheme, and some of the computer science aspects of the project. Several candidate approaches were tested and evaluated in a simplified two-dimensional fluid system (the Shallow Water Equations), and a small number of promising approaches were identified for further development in Phase 2.

Phase 2 of this project will build on the progress made in Phase 1 in order to develop a three-dimensional, fully compressible dynamical core. The work in Phase 2 falls broadly into three work packages:

* Vertical aspects. The stability and accuracy of the discretization depends crucially on the choice of vertical coordinate, the choice of thermodynamic variables predicted, and the vertical placement of variables relative to each other (`staggering'). It will also depend on the details of how, for example, the pressure gradient term is evaluated, especially near steep mountains, and how the vertical discretization couples with the horizontal discretization. Building on current understanding, candidate schemes will be formulated and tested.

* Code design and development. The code for the three-dimensional dynamical core will be based around a carefully designed software framework. The interface between the numerical discretization and its parallel implementation will be optimized, so that modifications to the former require minimal knowledge of the latter. The software framework will be highly flexible, so that it can easily accommodate future evolution of the dynamical core, such as changes in grid structure.

* Testing. The behaviour of complex numerical algorithms can be difficult to predict theoretically, even when individual components are well understood and tested. It will be vital, therefore, to test comprehensively the proposed formulations at the earliest opportunity, and revise if necessary. Early testing will focus on the shallow water formulation arising out of Phase 1 of the project, and on one-dimensional (column) and two-dimensional (vertical slice) prototypes of the vertical formulation. Testing of the three-dimensional formulation will begin as soon as code is available.

Planned Impact

Society benefits in numerous ways from accurate weather forecasts, via a wide range of weather-sensitive businesses and services (aviation, construction, energy, retail, ... etc.) as well as direct use by the public. Accurate forecasts of extreme weather events are particularly valuable in terms of minimizing risks to property as well as human life and health. Accurate predictions of climate change, particularly at a regional level, are essential for both climate-sensitive businesses and for policy makers, who must evaluate the costs and benefits (both economic and societal) of possible mitigation and adaptation measures. The proposed project will continue the drive towards more accurate weather and climate prediction by providing a key computational tool: a scalable atmospheric dynamical core that can take advantage of future massively parallel computing platforms to achieve higher resolution. Thus, the ultimate beneficiaries are the public, businesses, and policy makers who benefit from operational weather forecasts and climate predictions produced by the Met Office (as well as other users of the Met Office Unified Model around the world).

The immediate beneficiaries are the Met Office themselves. The proposal has been developed in close collaboration with the Met Office, and the project will involve a close partnership with the Met Office, building on the successful relationships and practices developed in Phase 1. UK academics and Met Office staff will be fully integrated in a single project team. There will be frequent project meetings, comprising quarterly plenary workshops interspersed with quarterly topical meetings, with day-to-day communication via a project email list and TWIKI. This close partnership will ensure that the project addresses the needs of the Met Office, is compatible with the other components of their operational system (physical parameterizations, data assimilation), and that the results pull through into their operational activities as rapidly and directly as possible.

The results of this project will be of great interest to the growing number of other groups around the world, both in operational centres and in and academic research, who, driven by the parallel scalability issue, are developing new atmospheric models. They will also be of wider interest to the computational fluid dynamics and high performance computing research communities. Results of this project will be published in the peer reviewed literature, and presented at relevant conferences and workshops, to reach the widest possible audience in both communities.

Publications


10 25 50
Cotter C (2016) Embedded discontinuous Galerkin transport schemes with localised limiters in Journal of Computational Physics
Cotter C (2014) Variational formulations of sound-proof models in Quarterly Journal of the Royal Meteorological Society
Cotter C (2015) Mixed finite elements for global tide models in Numerische Mathematik
Homolya M (2016) A Parallel Edge Orientation Algorithm for Quadrilateral Meshes in SIAM Journal on Scientific Computing
Lange M (2016) Efficient Mesh Management in Firedrake Using PETSc DMPlex in SIAM Journal on Scientific Computing
Luporini F (2015) Cross-Loop Optimization of Arithmetic Intensity for Finite Element Local Assembly in ACM Transactions on Architecture and Code Optimization
McRae A (2016) Automated Generation and Symbolic Manipulation of Tensor Product Finite Elements in SIAM Journal on Scientific Computing
Natale A (2016) Compatible finite element spaces for geophysical fluid dynamics in Dynamics and Statistics of the Climate System
 
Description This grant resulted in numerical methods and prototype implementations which will be employed in the next generation Met Office forecast system.
Exploitation Route The results will be taken forward by the Met Office. In addition, they form the basis of further developments in other simulation fields, most particularly atmosphere and ocean simulation. This work is supported by further NERC and EPSRC grants.
Sectors Aerospace, Defence and Marine,Chemicals,Construction,Environment,Manufacturing, including Industrial Biotechology
 
Description The outcomes of this grant have been taken up by the Met Office and form the design basis for the next generation forecast system currently in development.
First Year Of Impact 2013
Sector Environment
 
Description Standard Grant: Improving Prediction of Fronts
Amount £379,737 (GBP)
Funding ID NE/K012533/1 
Organisation Natural Environment Research Council (NERC) 
Sector Public
Country United Kingdom of Great Britain & Northern Ireland (UK)
Start 01/2014 
End 01/2017
 
Description Standard Grant: Moving meshes for global atmospheric modelling
Amount £128,254 (GBP)
Funding ID NE/M013634/1 
Organisation Natural Environment Research Council (NERC) 
Sector Public
Country United Kingdom of Great Britain & Northern Ireland (UK)
Start 09/2015 
End 08/2018
 
Description Ongoing colloboration with UK Met Office staff 
Organisation Government of the UK
Department Meteorological Office UK
Country United Kingdom of Great Britain & Northern Ireland (UK) 
Sector Public 
PI Contribution We have developed several collaborative interactions with UK Met Office staff
Start Year 2011
 
Title Firedrake 
Description Firedrake is an automated system for the portable solution of partial differential equations using the finite element method (FEM). Firedrake enables users to employ a wide range of discretisations to an infinite variety of PDEs and employ either conventional CPUs or GPUs to obtain the solution. 
Type Of Technology Software 
Year Produced 2013 
Open Source License? Yes  
Impact Firedrake is a principle test platform for the development of Gung Ho, the future UK Met Office dynamical core. 
URL http://www.firedrakeproject.org/
 
Title Gusto 
Description A Python library for compatible finite element dynamical cores 
Type Of Technology Software 
Year Produced 2016 
Open Source License? Yes  
Impact This software is providing a testbed for the development of the Gung Ho dynamical core for the Met Office forecast model. 
URL http://firedrakeproject.org/gusto/
 
Title Slicemodels 
Description This is a code for benchmarking our suite of compatible finite element methods for numerical weather prediction in a vertical slice configuration. 
Type Of Technology Software 
Year Produced 2015 
Open Source License? Yes  
Impact This tool is being used to benchmark numerical schemes for the NERC/Met Office/STFC UK Dynamical Core project ("Gung Ho"). 
URL https://bitbucket.org/colinjcotter/slicemodels
 
Description ICMS Public talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact As part of the International Centre for Mathematical Sciences workshop we organised in Edinburgh, we hosted a public talk on climate uncertainty given by David Stainforth. The idea was to engage the public with the importance of mathematics in climate research, particularly in the combination of climate/weather models and statistics in order to understand and quantify uncertainty.

We received excellent feedback about the talk through the ICMS.
Year(s) Of Engagement Activity 2015
 
Description Maths Foresees workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? Yes
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Workshop resulted in collaborative projects between academics and stakeholders, funded through the Maths Foresees network, and forged new potential collaborations for future project calls.

Amongst the various activities, I started a new engagement with HR Wallingford on a flooding project.
Year(s) Of Engagement Activity 2015
URL http://www1.maths.leeds.ac.uk/mathsforesees/workshopleeds2015.html
 
Description Princes Teaching Trust 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? Yes
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact I had fruitful discussions with teachers at the event who got ideas of how to engage students with numerical analysis topics by discussing our work.

I received very complementary written feedback from teachers via the Prince's Teaching Trust after the event, who said that they would use examples from my talk in their teaching to inspire KS4/5 students.
Year(s) Of Engagement Activity 2015