scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Parametric Video Compression Scheme Using AR Based Texture Synthesis

16 Dec 2008-pp 219-225
TL;DR: A video coding scheme based on parametric compression of texture is proposed that achieves upto 54.52% more compression as compared to the standard H.264/AVC at similar visual quality.
Abstract: In this paper a video coding scheme based on parametric compression of texture is proposed Each macro block is characterized either as an edge block, or as a non edge block containing texture The non edge blocks are coded by modeling them as an auto-regressive process (AR) By applying the AR model in spatio-temporal domain, we ensure both spatial as well as temporal consistency Edge blocks are encoded using the standard H264/AVC The proposed algorithm achieves upto 5452% more compression as compared to the standard H264/AVC at similar visual quality
Citations
More filters
Patent
04 Sep 2009
TL;DR: In this article, a video compression framework based on parametric object and background compression is proposed, where an embodiment detects objects and segments frames into regions corresponding to the foreground object and the background.
Abstract: A video compression framework based on parametric object and background compression is proposed. At the encoder, an embodiment detects objects and segments frames into regions corresponding to the foreground object and the background. The object and the background are individually encoded using separate parametric coding techniques. While the object is encoded using the projection of coefficients to the orthonormal basis of the learnt subspace (used for appearance based object tracking), the background is characterized using an auto-regressive (AR) process model. An advantage of the proposed schemes is that the decoder structure allows for simultaneous reconstruction of object and background, thus making it amenable to the new multi-thread/multi-processor architectures.

27 citations

Journal ArticleDOI
TL;DR: The most significant issues in IAC coding and their relevance for future codec designs are reviewed and the most promising pathways will be analyzed along with limitations and open issues.
Abstract: Perception-oriented video coding based on image analysis and completion (IAC) has received increasing attention over recent years. Several approaches have been proposed that follow diverging strategies: from full compatibility to hybrid block transform coding to alternative codec designs. In this paper, we review the most significant issues in IAC coding and their relevance for future codec designs. The most promising pathways will be analyzed along with limitations and open issues. The challenges in IAC-based video coding will be summarized using methods developed by the authors and others for block-based and region-based IAC coding.

23 citations

Dissertation
22 Jan 2020
TL;DR: C’est a ce niveau that se definit l’objectif du codage video perceptuel qui est d’aller vers une representation of l'information basee sur le contenu percu et qui puisse servir de paradigme pour un codage base qualite perceptuelle des contenus des videos 2D et 3D.
Abstract: On ne peut que constater combien la technologie video est devenue omnipresente en un peu plus d’une decennie. Si les besoins des nouveaux services video ont ete en partie satisfaits par l’efficacite des reseaux et de leurs technologies de couche physique, en parallele les codecs video ont ete primordiaux en offrant compression et nouvelles fonctionnalites. La recherche en codage video se caracterise alors par un fort contexte normatif ou le point de compatibilite se fait au niveau du decodage. Les performances de chaque nouveau standard sont remarquables, car typiquement chacun par rapport au precedent, code avec la meme qualite objective un format video donne mais en divisant par deux le debit necessaire. Il est cependant frappant de constater que le paradigme de la couche de codage n’a pas change depuis l’origine, avec un codec dissymetrique ou les decisions sont prises a l’encodage via un schema hybride integrant les etapes de prediction, transformation, quantification et codage entropique. Il est aussi surprenant de constater la persistance de l’implantation dans les codeurs pour la phase d’optimisation debit-distorsion, d’un calcul de distorsion de type erreur quadratique moyenne, alors que l’on sait la correlation pauvre entre les resultats de cette metrique et le jugement humain. C’est a ce niveau que se definit l’objectif du codage video perceptuel qui est d’aller vers une representation de l'information basee sur le contenu percu et qui puisse servir de paradigme pour un codage base qualite perceptuelle des contenus des videos 2D et 3D. Pour etre en phase avec les approches standards, l’objectif devient de guider les choix de codage de facon a aller vers la fidelite perceptuelle sans perte des videos decodees. Mes travaux se positionnent donc dans cette recherche des outils et des methodes ou les choix pour ameliorer la qualite perceptuelle guident le codage video. Mon memoire retrace mon itineraire scientifique au sein des equipes IVC IRCCyN puis IPI LS2N qui, du fait des recherche qui y sont conduites sur la modelisation du systeme visuel humain, ont offert et offrent encore un cadre collaboratif particulierement propice aux travaux lies au codage video perceptuel. Mes contributions au codage video perceptuel se sont principalement organisees au sein de projets de recherche visant a contribuer aux efforts faits en normalisation, avec le projet ANR ArchiPEG qui avait pour objectif l'optimisation du codage video HD en flux H.264/AVC, le projet ANR PERSEE qui s'interessait au codage perceptuel des formats video 2D et 3D via le codec H.264 et son extension MVC, et le projet europeen ITN PROVISION qui visait le codage perceptuel via HEVC. Mes contributions ont ete produites au travers l’encadrement de masters et de theses (4 soutenues, 1 en cours), et nous verrons que nous avons agi a 2 niveaux de la boucle de codage video hybride. Une premiere possibilite est d’ameliorer directement des unites de la boucle de codage. C’est ainsi que nous avons mis en œuvre une transformation directionnelle tenant compte des orientations geometriques au sein de l’image. Nous avons exploite cette representation de l’information afin de construire des classes de structures, puis adapte la quantification de leur contenu en se basant sur des criteres perceptuels. Nous avons aussi, apres avoir defini la notion de textures spatio-temporelles visuelles, propose un modele mesurant la similarite entre ces textures visuelles. Ce modele computationnel base sur le traitement cortical du SVH, a ensuite ete utilise afin d’ameliorer la boucle de codage de HEVC : en identifiant des classes de textures visuelles et en faisant correspondre a la distorsion objective standard calculee par le codeur sa valeur dans le domaine perceptuel. Une autre approche consiste a mettre en place un schema de pre-analyse avant encodage afin de piloter ensuite efficacement le codage des regions d’interet. La modelisation de l’attention visuelle est donc essentielle au niveau de cette analyse. Nous avons concu un schema de pre-analyse base sur des tubes spatio-temporels. Cet outil realise la segmentation et le suivi des objets constitues de tubes connexes, et il leur associe une saillance visuelle. L’ensemble de ces informations est exploitee afin d’assurer un codage coherent des objets en flux H.264/AVC. Exactement nous avons teste l’adaptation de la structure du GOP, et le guidage de la qualite visuelle. Nous avons aussi developpe le premier modele computationnel de l’attention visuelle 3D s’appliquant a la visualisation d’images stereoscopiques. Ce modele combine deux cartes de saillance visuelle : celle 2D et celle de profondeur. Nous avons utilise ce modele afin d’ameliorer la qualite d’experience de visualisation de la TV3D, par ajout d’un flou de defocalisation sur la region ayant moins d’interet. Dans la derniere partie de mon memoire, un bilan et des perspectives sont donnes. Nous nous interessons en particulier au codage des formats video immersifs representes par des nuages de points 3D. Nos derniers travaux montrent l’interet d’une approche basee quantification vectorielle algebrique et arborescente, permettant de structurer graduellement la geometrie de la scene, et cela en fonction d’un critere debit-distorsion.

14 citations

Book ChapterDOI
TL;DR: This chapter provides an overview of the human visual system, modeling approaches, and perceptual video coding schemes, and issues related to the design of perceptualVideo coding schemes as well as to practical aspects of perceptual video codecs.
Abstract: The volume of data involved in video storage and transmission is rapidly increasing with the introduction of new video formats with higher spatial and temporal resolutions. In order to meet capacities of storage and transmission devices, compression is needed to reduce the amount of data used to represent video. This is achieved using prediction and entropy coding for reducing the redundant information and lossy quantization to further lower the video bitrate. The side effect of quantization is the introduction of coding artifacts which can severely compromise the perceived quality of coded video. To minimize the visibility of coding artifacts, video codecs should be optimized by taking into account the properties of the human visual system which is characterized by a space and time varying sensitivity to coding artifacts. This distortion sensitivity is related to certain masking phenomena, which can be exploited in order to improve the compression efficiency and provide the perception of improved quality. In order to further enhance the compression efficiency of highly optimized video coding standards such as H.264/AVC and the new HEVC, new perceptually tuned tools should be considered. In this context, this chapter provides an overview of the human visual system, modeling approaches, and perceptual video coding schemes. Influences of different human visual system masking phenomena are reviewed with respect to the sensitivity to coding artifacts. Moreover, issues related to the design of perceptual video coding schemes are discussed and related solutions are presented. Finally, the chapter concludes by providing some future research directions addressing the current limitations related to both the human visual system modeling as well as to practical aspects of perceptual video codecs.

13 citations

Proceedings ArticleDOI
01 Oct 2014
TL;DR: The proposed method is deterministic, which solves a key problem for applications such as synthesis-based video coding, and shows potential bitrate savings up to 49% on texture sequences at comparable visual quality.
Abstract: In this paper, an improved 2D+t texture completion framework is proposed, providing high visual quality of completed dynamic textures. A Spatiotemporal Autoregressive model (STAR) is used to propagate the signal of several available frames onto frames containing missing textures. A Gaussian white noise classically drives the model to enable texture innovation. To improve this method, an innovation process is proposed, that uses texture information from available training frames. The proposed method is deterministic, which solves a key problem for applications such as synthesis-based video coding. Compression simulations show potential bitrate savings up to 49% on texture sequences at comparable visual quality. Video results are provided online to allow assessing the visual quality of completed textures.

5 citations


Cites methods from "Parametric Video Compression Scheme..."

  • ...As in [12], squares of size (2cx + 1, 2cy + 1) centered on the position collocated with the current pixel are used in past frames since all their samples are expected to be known....

    [...]

  • ...[12] presents a hybrid video codec based on STAR synthesis....

    [...]

References
More filters
Proceedings ArticleDOI
20 Sep 1999
TL;DR: A non-parametric method for texture synthesis that aims at preserving as much local structure as possible and produces good results for a wide variety of synthetic and real-world textures.
Abstract: A non-parametric method for texture synthesis is proposed. The texture synthesis process grows a new image outward from an initial seed, one pixel at a time. A Markov random field model is assumed, and the conditional distribution of a pixel given all its neighbors synthesized so far is estimated by querying the sample image and finding all similar neighborhoods. The degree of randomness is controlled by a single perceptually intuitive parameter. The method aims at preserving as much local structure as possible and produces good results for a wide variety of synthetic and real-world textures.

2,972 citations

Journal ArticleDOI
TL;DR: The objective is to enhance texture quality as much as possible with a minor sacrifice in efficiency in order to support the conjecture that the pixel-based approach would yield high quality images.

1,462 citations

Proceedings ArticleDOI
01 Dec 2001
TL;DR: A class of automated methods for digital inpainting using ideas from classical fluid dynamics to propagate isophote lines continuously from the exterior into the region to be inpainted is introduced.
Abstract: Image inpainting involves filling in part of an image or video using information from the surrounding area. Applications include the restoration of damaged photographs and movies and the removal of selected objects. We introduce a class of automated methods for digital inpainting. The approach uses ideas from classical fluid dynamics to propagate isophote lines continuously from the exterior into the region to be inpainted. The main idea is to think of the image intensity as a 'stream function for a two-dimensional incompressible flow. The Laplacian of the image intensity plays the role of the vorticity of the fluid; it is transported into the region to be inpainted by a vector field defined by the stream function. The resulting algorithm is designed to continue isophotes while matching gradient vectors at the boundary of the inpainting region. The method is directly based on the Navier-Stokes equations for fluid dynamics, which has the immediate advantage of well-developed theoretical and numerical results. This is a new approach for introducing ideas from computational fluid dynamics into problems in computer vision and image analysis.

1,068 citations


"Parametric Video Compression Scheme..." refers methods in this paper

  • ...The proposed algorithm achieves upto 54.52% more compression as compared to the standard H.264/AVC at similar visual quality....

    [...]

Proceedings ArticleDOI
16 Sep 1996
TL;DR: This work model image sequences of temporal textures using the spatio-temporal autoregressive model (STAR), which expresses each pixel as a linear combination of surrounding pixels lagged both in space and in time.
Abstract: Temporal textures are textures with motion. Examples include wavy water, rising steam and fire. We model image sequences of temporal textures using the spatio-temporal autoregressive model (STAR). This model expresses each pixel as a linear combination of surrounding pixels lagged both in space and in time. The model provides a base for both recognition and synthesis. We show how the least squares method can accurately estimate model parameters for large, causal neighborhoods with more than 1000 parameters. Synthesis results show that the model can adequately capture the spatial and temporal characteristics of many temporal textures. A 95% recognition rate is achieved for a 135 element database with 15 texture classes.

352 citations


"Parametric Video Compression Scheme..." refers background in this paper

  • ...Section 7 contains results and we conclude with section 8....

    [...]

  • ...In section 5 and 6 the proposed encoding scheme is presented and discussed in detail....

    [...]

Proceedings ArticleDOI
24 Nov 2003
TL;DR: This work assumes that the textures in a video scene is classified into two categories: textures with unimportant subjective details and the remainder, and utilizes this assumption for improved video coding using a texture analyzer and a texture synthesizer.
Abstract: We assume that the textures in a video scene is classified into two categories: textures with unimportant subjective details and the remainder We utilize this assumption for improved video coding using a texture analyzer and a texture synthesizer The texture analyzer identifies the texture regions with unimportant subjective details and generates coarse masks as well as side information for the texture synthesizer at the decoder side The texture synthesizer replaces the identified textures by inserting synthetic textures for the identified regions The texture analyzer is based on MPEG-7 descriptors Our approach is integrated into an H264/AVC codec Bit-rate savings up to 194% are shown for a semi-automatic texture analyzer given similar subjective quality as the H264/AVC codec without the presented approach

74 citations