The Terra-correlator: A computing facility for massive real-time data assimilation in environmental science

Lead Research Organisation: University of Edinburgh
Department Name: Sch of Geosciences


1. Seismic interferometry: Seismologists have traditionally explored the Earth by measuring, analysing and modelling signals generated naturally by earthquakes, or deliberately by man-made sources. Since 2003, several authors derived new methods based on cross-correlation of seismograms that allowed similar information to be obtained in the absence of impulsive events - from the Earth's ambient fluctuations, previously regarded as noise. Current global data sets can be used to produce Earth impulse responses of ~20kB size every two hours, or up to 2TB of output data per day, the bulk of this providing completely new information. This will be done on node 1. The output will then be data-mined continuously and in parallel to extract near-real time information about subsurface changes for forecasting purposes. This requires additional near-real time correlations in time-lapse mode, pattern matching, and other methods of analysis and modelling to be applied on a separate node (node 2), tied to a portal that will ensure such analysis and any forecasts of future behaviour or events (say a volcanic eruption following a seismic velocity change) is verifiably done in advance of the real event time, removing at a stroke perennial problems with retrospective selection bias when analysing forecast quality based on past data.

2. Earth system science: (a) A recurring challenge is to analyse direct Earth observation, satellite and model data with data- and compute-intensive processing for uncertainty analyses and parameter-space exploration. The new facility will be used to produce estimates of carbon stocks and fluxes with confidence intervals over the period 2000-2013. Ultimately our UK runs at 1 km2 may be used to generate near-real time analyses of GHG emissions that are likely to be useful for policy makers. The information content of planned EO missions, such as ESA's BIOMASS mission, will also be explored in observing system simulation experiments. ( b) Many of our typical current analyses of the performance of climate models in comparison with the outcome are constrained by reading in data multiple times, due to lack of memory. We will analyse NOAA data to find extreme events at much higher resolution than before of 25km globally, to research the mechanisms and characteristic signatures of extreme precipitation events.

3. High-resolution real time monitoring of deformation and fluid flow in porous rocks: X-ray C-T (computer-tomography) imaging is computationally intensive in a range of applications, but a very large amount of post-processing is required by the operator to tune the resulting image, notably to separate pore space from the solid phase. Time-lapse measurements open up the possibility of tracking fluid flow in fractures or pores, or observing deformation at unprecedented resolution by tracking (cross-correlating) particle movements. Accordingly the infrastructure overhead used in the first two applications will be used to make such analysis possible in a 'live' experiment, in support of on-going experimental and modelling work in rock physics, initially focussed on carbonates.

Planned Impact



10 25 50
Krause A (2015) dispel4py
Description We have developed, tested and optimised a new facility for extracting new information from continuously monitored data using the mathematical operation of cross-correlation. The method has been tested on data previously regarded as seismic noise - that is the recordings in between earthquake events. Bench-mark tests of the new facility show it is feasible to cross-correlate recordings from the global digital seismic array and update the results once per day, and hence look for subtle small earthquakes that may indicate the imminent nucleation of larger ones, or detect changes in seismic velocity. Next steps are to examine other applications in detecting extreme weather events and tracking deformation and fluid flow in laboratory rock samples by processing high resolution X-ray CT images.
Exploitation Route The long-term aim of the project is to deliver a UK capability for rapid assimilation of 'big data' for characterising and forecasting the behaviour of different aspects of the Earth system.
Sectors Energy,Environment