We present a video coding technique based on texture synthesis via stitching small patches of previous images as done in texture transfer. Each macroblock is identified as a texture or non texture block by means of an edge based criterion. The texture blocks are reconstructed using the luminance values as the control block information to guide the texture transfer algorithm. By modifying the texture transfer algorithm we ensure both spatial as well as temporal consistency. The non texture blocks and the luminance values for texture blocks can be coded by any encoder and thus our method can act as a pre-processor to any video codec e.g. H.264 and add to the compression obtained. © 2010 IEEE.