Initialization and Alignment
for Adversarial Texture Optimization

ECCV 2022


Xiaoming Zhao,    Zhizhen Zhao,    Alexander G. Schwing


University of Illinois Urbana-Champaign



While recovery of geometry from image and video data has received a lot of attention in computer vision, methods to capture the texture for a given geometry are less mature. Specifically, classical methods for texture generation often assume clean geometry and reasonably well-aligned image data. While very recent methods, e.g., adversarial texture optimization, better handle lower-quality data obtained from hand-held devices, we find them to still struggle frequently. To improve robustness, particularly of recent adversarial texture optimization, we develop an explicit initialization and an alignment procedure. It deals with complex geometry due to a robust mapping of the geometry to the texture map and a hard-assignment-based initialization. It deals with misalignment of geometry and images by integrating fast image-alignment into the texture refinement optimization. We demonstrate efficacy of our texture generation on a dataset of 11 scenes with a total of 2807 frames, observing 7.8% and 11.1% relative improvements regarding perceptual and sharpness measurements.


Approach Overview


Our approach builds on AdvTex and is composed of two major steps:

  1. TexInit: We initialize the texture using an assignment-based texture generation framework;
  2. TexSmooth: A data-driven adversarial loss is utilized to optimize out artifacts incurred in the assignment step.
Responsive image

Optimized Texture Compared to AdvTex


Please drag the separator to see the texture comparison between AdvTex (left) and ours (right).
Pay attention to the misalignment of AdvTex results. For example, in Scene 01, AdvTex maps the texture of the sofa to the wall.
(The webpage may need to be refreshed twice before this works properly.)

Scene 01
Scene 02
Scene 03
Scene 04
Scene 05
Scene 06
Scene 07
Scene 08
Scene 09
Scene 10
Scene 11

Rendering Comparison


We provide rendering comparisons from camera poses which are part of each scene's test split.
We compare to MVSTex and AdvTex. Please click each scene's image below for corresponding comparison details.


Bibtex

						
			@inproceedings{zhao-tex2022,
				title = {Initialization and Alignment for Adversarial Texture Optimization},
				author = {Xiaoming Zhao and Zhizhen Zhao and Alexander~G. Schwing},
				booktitle = {European Conference on Computer Vision (ECCV)},
				year = {2022},
			}
						
					

Acknowledgements

Supported in part by NSF grants 1718221, 2008387, 2045586, 2106825, MRI #1725729, and NIFA award 2020-67021-32799.