COMPPACT: Compression of Video using Perceptually Optimised Parametric Coding Techniques

Lead Research Organisation: University of Bristol
Department Name: Electrical and Electronic Engineering

Abstract

It is currently a very exciting and challenging time for video compression. The predicted growth in demand for bandwidth, especially for mobile services is driven largely by video applications and is probably greater now than it has ever been. There are four reasons for this:

(i) Recently introduced formats such as 3D and multiview, coupled with increasing dynamic range, spatial resolution and framerate, all require increased bit-rate to deliver improved immersion;
(ii) Video-based web traffic continues to grow and dominate the internet;
(iii) User expectations coninue to drive flexibility and quality, with a move from linear to non-linear delivery;
(iv) Finally the emergence of new services, in particular mobile delivery through 4G/LTE to smart phones. While advances in network and physical layer technologies will no doubt contribute to the solution, the role of video compression is also of key importance.

This research project is underpinned by the assumption that, in most cases, the target of video compression is to provide good subjective quality rather than to minimise the error between the original and coded pictures. It is thus possible to conceive of a compression scheme where an analysis/synthesis framework replaces the conventional energy minimisation approach. Such a scheme could offer substantially lower bitrates through reduced residual and motion vector coding.

The approach proposed will model scene content using combinations of waveform coding and texture replacement, using computer graphic models to replace target textures at the decoder. These not only offer the potential for dramatic improvements in performance, but they also provide an inherent content-related parameterisation which will be of use in classification and detection tasks as well as facilitating integration with CGI.

This has the potential to create a new content-driven framework for video compression. In this context our aim is to shift the video coding paradigm from rate-distortion optimisation to rate-quality modelling, where region-based parameters are combined with perceptual quality metrics to inform and drive the coding and synthesis processes. However it is clear that a huge amount of research needs to be done in order to fully exploit the method's potential and to yield stable and efficient solutions. For example, mean square error is no longer a valid objective function or measure of quality, and new embedded perceptually driven quality metrics are essential. The choice of texture analysis and synthesis models are also important, as is the exploitation of long-term picture dependencies.

Planned Impact

The investigators and partners have a long track record of delivering impact and pulling through ideas from research into exploitation.

COMPPACT will provide impact through:

- Shaping UK capability in video compression and delivery: It will deliver reference software that enables development
and evaluation. This will provide benefit to industry as well as to the research community, in providing an ability to evaluate and compare codec performance.

- Standardisation: We anticipate that the ideas and methods of COMPPACT will contribute to future video compression standardisation. This will provide new opportunities for video IC design companies in developing superior products. It will also provide benefits to the mobile communications industry through more efficient use of bandwidth, and to application developers and content providers in terms of being able to provide more immersive content at lower bit rates.

- Society: A major impact of this project will be on the wider public, in terms of providing high quality bandwidth-efficient video services, more engaging content representations and enabling new more immersive formats.

- The Bristol Creative Community: will benefit through local events and workshops hosted at the Watershed Arts Centre. We will also provide access by the local creative sector to capture, display and quality assessment facilities in the BVI studio, in particular to SMEs through the Bristol Media Network and the BBC-Anchor Consortium.
 
Description This project has developed a novel means of video compression based on texture warping and synthesis. Instead of encoding whole images or prediction residuals after translational motion estimation, our algorithm employs a perspective motion model to warp static textures and utilises texture synthesis to create dynamic textures. Texture regions are segmented using features derived from the complex wavelet transform and further classified according to their spatial and temporal characteristics.

The project has also developed low complexity perceptual quality metrics that have the potential to be used in the coding loop to prevent warping and synthesis artefacts, as well as outside of it. The new perceptual video quality metric (PVM) out performs all previously published work. This work is now been published in IEEE Trans CSVT.

The proposed algorithm has been integrated into an H.264 video coding framework. The results show significant bitrate savings, of up to 60% compared with H.264 at the same objective quality (based on AVM) and subjective scores. It is currently being integrated into the new HEVC video coding standard.

COMPPACT is currently undertaking further data collection exercises including various textural content acquired at high spatial resolution and high frame rates.

Results of enhanced Rate Quality Optimisation have also been researched that have demonstrated significant improvements over the latest coding standard - HEVC. Content-adaptive Langragian multiplier selection methods have also been generated that improve upon HEVC performance by 3%.
Exploitation Route A number of important collaborations have already arisen from this work where the findings and datasets are already being used. These include:
i) an EU Marie Curie Training Network, collaboratively with Fraunhofer HHI Berlin, Univ. Aachen, Univ. Nantes, BBC, Microsoft, Technicolor, Netflix, Google and Purdue Univ. This project builds on COMPPACT in the area of perceptual video coding and quality metrics and is targeting the next generation of video coding standards.
ii) An EPSRC Platform grant awarded to Bristol Vision Institute (Bull PI), 'Vision for the Future'. A key theme in this is visual immersion where compression techniques will be investigated that achieve compression ratios exceed 1000:1 while preserving the immersive properties of the format. Collaborators include the Academy of Motion Picture Arts and Sciences, Aardman Animations, BBC, and ARRI.
iii) A joint facility, 'The Bristol BBC Immersive Technology Laboratory has been established that is investigating compression for high frame rate and High Dynamic Range content.
Sectors Aerospace, Defence and Marine,Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Electronics,Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Manufacturing, including Industrial Biotechology,Culture, Heritage, Museums and Collections,Transport
URL http://www.bristol.ac.uk/vi-lab/projects/parametricvideocompression/
 
Description The work of this project is likely to only have real impact during the next stage of video standardisation. There is currently a call for evidence being planned in MPEG, leading to a new H.266 standard in 2020. We anticipate contributing to this.
First Year Of Impact 2016
Sector Aerospace, Defence and Marine,Creative Economy,Digital/Communication/Information Technologies (including Software),Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Manufacturing, including Industrial Biotechology,Security and Diplomacy,Transport
Impact Types Societal,Economic,Policy & public services
 
Description EPSRC Platform Grant
Amount £1,362,874 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Academic/University
Country United Kingdom of Great Britain & Northern Ireland (UK)
Start 02/2015 
End 02/2020
 
Description EU Marie Curie ITN
Amount £607,602 (GBP)
Organisation Seventh Framework Programme (FP7) 
Department Marie Curie ITN
Sector Academic/University
Country France, French Republic
Start 09/2013 
End 09/2017
 
Title BV High frame rate database 
Description Collection of high frame rate clips with associated metadata for testing and developing future immersive video formats 
Type Of Material Database/Collection of data 
Year Produced 2015 
Provided To Others? Yes  
Impact None at present 
URL http://data.bris.ac.uk/data/dataset/k8bfn0qsj9fs1rwnc2x75z6t7
 
Title BVI Texture database 
Description Collection of static and dynamic video textures for compression testing 
Type Of Material Database/Collection of data 
Year Produced 2015 
Provided To Others? Yes  
Impact Used by several groups around the world 
URL http://data.bris.ac.uk/datasets/1if54ya4xpph81fbo1gkpk5kk4/
 
Description BBC Immersive Technology Laboratory 
Organisation British Broadcasting Corporation (BBC)
Department BBC Research & Development
Country United Kingdom of Great Britain & Northern Ireland (UK) 
Sector Public 
PI Contribution High Dynamic range coding optimisation for HEVC Perceptual video compression results REDUX database analytics
Collaborator Contribution Provision of REDUX Support for PhD students Collaboration on perceptual quantisation Secondment of BBC employees
Impact New method of perceptual quantisation for HDR HEVC Analysis of BBC archive in terms of feature classification
Start Year 2012
 
Description Keynote: IET ISP 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Keynote Lecture IET ISP- Perceptual Video coding
Year(s) Of Engagement Activity 2015