Abstract

Estimation of intrinsic images still remains a challenging task due to weaknesses of ground-truth datasets, which either are too small or present non-realistic issues. On the other hand, end-to-end deep learning architectures start to achieve interesting results that we believe could be improved if important physical hints were not ignored. In this work, we present a twofold framework: (a) a flexible generation of images overcoming some classical dataset problems such as larger size jointly with coherent lighting appearance; and (b) a flexible architecture tying physical properties through intrinsic losses. Our proposal is versatile, presents low computation time, and achieves state-of-the-art results.

© 2019 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Intrinsic decomposition from a single spectral image

Xi Chen, Weixin Zhu, Yang Zhao, Yao Yu, Yu Zhou, Tao Yue, Sidan Du, and Xun Cao
Appl. Opt. 56(20) 5676-5684 (2017)

Deep spectral reflectance and illuminant estimation from self-interreflections

Rada Deeb, Joost Van de Weijer, Damien Muselet, Mathieu Hebert, and Alain Tremeau
J. Opt. Soc. Am. A 36(1) 105-114 (2019)

Improving color constancy by discounting the variation of camera spectral sensitivity

Shao-Bing Gao, Ming Zhang, Chao-Yi Li, and Yong-Jie Li
J. Opt. Soc. Am. A 34(8) 1448-1462 (2017)

References

  • View by:
  • |
  • |
  • |

  1. H. Barrow and J. Tenenbaum, “Recovering intrinsic scene characteristics,” Comput. Vis. Syst. 2, 2 (1978).
  2. R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, “Ground-truth dataset and baseline evaluations for intrinsic image algorithms,” in International Conference on Computer Vision (2009), pp. 2335–2342.
  3. S. Bell, K. Bala, and N. Snavely, “Intrinsic images in the wild,” ACM Trans. Graph. 33, 159 (2014).
    [Crossref]
  4. D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black, “A naturalistic open source movie for optical flow evaluation,” in European Conference on Computer Vision (ECCV) (2012).
  5. J. Shi, Y. Dong, H. Su, and S. X. Yu, “Learning non-Lambertian object intrinsics across ShapeNet categories,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 5844–5853.
  6. A. S. Baslamisli, H. Le, and T. Gevers, “CNN based learning using reflection and Retinex models for intrinsic image decomposition,” in Computer Vision and Pattern Recognition (2018).
  7. A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (ECCV) (2018).
  8. Z. Li and N. Snavely, “CGintrinsics: better intrinsic image decomposition through physically-based rendering,” in European Conference on Computer Vision (ECCV) (2018).
  9. A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).
  10. E. H. Land and J. McCann, “Lightness and Retinex theory,” J. Opt. Soc. Am. 61, 1–11 (1971).
    [Crossref]
  11. B. K. Horn, “Determining lightness from an image,” Comput. Graph. Image Process. 3, 277–299 (1974).
    [Crossref]
  12. Y. Weiss, “Deriving intrinsic images from image sequences,” in International Conference on Computer Vision (2001), pp. 68–75.
  13. M. F. Tappen, W. T. Freeman, and E. H. Adelson, “Recovering intrinsic images from a single image,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005).
    [Crossref]
  14. M. F. Tappen, E. H. Adelson, and W. T. Freeman, “Estimating intrinsic component images using non-linear regression,” in IEEE Conference on Computer Vision and Pattern Recognition (2006), pp. 1992–1999.
  15. B. Funt, M. Drew, and M. Brockington, “Recovering shading from color images,” in European Conference on Computer Vision (1992), pp. 124–132.
  16. P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Neural Information Processing Systems (2011), pp. 765–773.
  17. L. Shen, C. Yeo, and B.-S. Hua, “Intrinsic image decomposition using a sparse representation of reflectance,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 2904–2915 (2013).
    [Crossref]
  18. J. T. Barron and J. Malik, “Shape, illumination, and reflectance from shading,” IEEE Trans. Pattern Anal. Mach. Intell. 37, 1670–1687 (2015).
    [Crossref]
  19. M. Serra, O. Penacchio, R. Benavente, and M. Vanrell, “Names and shades of color for intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (2012), pp. 278–285.
  20. J. T. Barron and J. Malik, “Intrinsic scene properties from a single RGB-D image,” in IEEE Conference on Computer Vision and Pattern Recognition (2013), pp. 17–24.
  21. K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image depth video,” in European Conference on Computer Vision (2012), pp. 327–340.
  22. Q. Chen and V. Koltun, “A simple model for intrinsic image decomposition with depth cues,” in IEEE International Conference on Computer Vision (ICCV) (IEEE, 2013), pp. 241–248.
  23. J. Jeon, S. Cho, X. Tong, and S. Lee, “Intrinsic image decomposition using structure-texture separation and surface normals,” in Computer Vision–ECCV, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, eds. (Springer, 2014), pp. 218–233.
  24. Y. Matsushita, S. Lin, S. B. Kang, and H.-Y. Shum, “Estimating intrinsic images from image sequences with biased illumination,” in Computer Vision–ECCV (Springer, 2004), pp. 274–286.
  25. P. Laffont and J. Bazin, “Intrinsic decomposition of image sequences from local temporal variations,” in IEEE International Conference on Computer Vision (ICCV) (2015), pp. 433–441.
  26. T. Narihira, M. Maire, and S. X. Yu, “Direct intrinsics: learning albedo-shading decomposition by convolutional regression,” in International Conference on Computer Vision (ICCV) (2015).
  27. D. Eigen, C. Puhrsch, and R. Fergus, “Depth map prediction from a single image using a multi-scale deep network,” in NIPS (2014).
  28. T. Zhou, P. Krähenbühl, and A. A. Efros, “Learning data-driven reflectance priors for intrinsic image decomposition,” in IEEE International Conference on Computer Vision (ICCV) (2015), pp. 3469–3477.
  29. E. Shelhamer, J. T. Barron, and T. Darrell, “Scene intrinsics and depth from a single image,” in IEEE International Conference on Computer Vision (ICCV) Workshops (2015).
  30. F. Liu, C. Shen, and G. Lin, “Deep convolutional neural fields for depth estimation from a single image,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), pp. 5162–5170.
  31. S. Kim, K. Park, K. Sohn, and S. Lin, “Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields,” in European Conference on Computer Vision (ECCV) (2016).
  32. T. Nestmeyer and P. V. Gehler, “Reflectance adaptive filtering improves intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 1771–1780.
  33. I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in NIPS (2014).
  34. L. Lettry, K. Vanhoey, and L. V. Gool, “DARN: a deep adversarial residual network for intrinsic image decomposition,” in IEEE Winter Conference on Applications of Computer Vision (WACV) (2018), pp. 1359–1367.
  35. W.-C. Ma, H. Chu, B. Zhou, R. Urtasun, and A. Torralba, “Single image intrinsic decomposition without a single intrinsic image,” in ECCV (2018).
  36. J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, “Signature verification using a Siamese time delay neural network,” Int. J. Pattern Recognit. Artif. Intell. 7, 669–688 (1993).
    [Crossref]
  37. Z. Li and N. Snavely, “Learning intrinsic image decomposition from watching the world,” in CVPR (2018), pp. 9039–9048.
  38. Q. Fan, J. Yang, G. Hua, B. Chen, and D. P. Wipf, “Revisiting deep intrinsic image decompositions,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), pp. 8944–8952.
  39. S. Beigpour, M. L. Ha, S. Kunz, A. Kolb, and V. Blanz, “Multi-view multi-illuminant intrinsic dataset,” in BMVC (2016).
  40. S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser, “Semantic scene completion from a single depth image,” in Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition (2017).
  41. W. Li, S. Saeedi, J. McCormac, R. Clark, D. Tzoumanikas, Q. Ye, Y. Huang, R. Tang, and S. Leutenegger, “Interiornet: mega-scale multi-sensor photo-realistic indoor scenes dataset,” in British Machine Vision Conference (BMVC) (2018).
  42. H. Sial, S. Sancho-Asensio, R. Baldrich, R. Benavente, and M. Vanrell, “Color-based data augmentation for reflectance estimation,” in IS&T Color and Imaging Conference (2018), vol. 2018, pp. 284–289.
  43. “Corel datasets,” http://www.emsps.com/photocd/corelcds.htm .
  44. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 2818–2826.
  45. F. Chollet, “Keras,” 2015, https://github.com/fchollet/keras .
  46. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778.
  47. D. P. Kingma and J. Ba, “Adam: a method for stochastic optimization,” CoRR abs/1412.6980 (2014).
  48. L. Shen and C. Yeo, “Intrinsic images decomposition using a local and global sparse representation of reflectance,” in CVPR (2011), pp. 697–704.
  49. E. Garces, A. Munoz, J. Lopez-Moreno, and D. Gutierrez, “Intrinsic images by clustering,” Comput. Graph. Forum 31, 1415–1424 (2012).
    [Crossref]
  50. Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to Retinex with nonlocal texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
    [Crossref]
  51. S. Bi, X. Han, and Y. Yu, “An l1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition,” ACM Trans. Graph. 34, 78 (2015).
    [Crossref]

2015 (2)

J. T. Barron and J. Malik, “Shape, illumination, and reflectance from shading,” IEEE Trans. Pattern Anal. Mach. Intell. 37, 1670–1687 (2015).
[Crossref]

S. Bi, X. Han, and Y. Yu, “An l1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition,” ACM Trans. Graph. 34, 78 (2015).
[Crossref]

2014 (1)

S. Bell, K. Bala, and N. Snavely, “Intrinsic images in the wild,” ACM Trans. Graph. 33, 159 (2014).
[Crossref]

2013 (1)

L. Shen, C. Yeo, and B.-S. Hua, “Intrinsic image decomposition using a sparse representation of reflectance,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 2904–2915 (2013).
[Crossref]

2012 (2)

E. Garces, A. Munoz, J. Lopez-Moreno, and D. Gutierrez, “Intrinsic images by clustering,” Comput. Graph. Forum 31, 1415–1424 (2012).
[Crossref]

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to Retinex with nonlocal texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

2005 (1)

M. F. Tappen, W. T. Freeman, and E. H. Adelson, “Recovering intrinsic images from a single image,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005).
[Crossref]

1993 (1)

J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, “Signature verification using a Siamese time delay neural network,” Int. J. Pattern Recognit. Artif. Intell. 7, 669–688 (1993).
[Crossref]

1978 (1)

H. Barrow and J. Tenenbaum, “Recovering intrinsic scene characteristics,” Comput. Vis. Syst. 2, 2 (1978).

1974 (1)

B. K. Horn, “Determining lightness from an image,” Comput. Graph. Image Process. 3, 277–299 (1974).
[Crossref]

1971 (1)

Adelson, E. H.

M. F. Tappen, W. T. Freeman, and E. H. Adelson, “Recovering intrinsic images from a single image,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005).
[Crossref]

M. F. Tappen, E. H. Adelson, and W. T. Freeman, “Estimating intrinsic component images using non-linear regression,” in IEEE Conference on Computer Vision and Pattern Recognition (2006), pp. 1992–1999.

R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, “Ground-truth dataset and baseline evaluations for intrinsic image algorithms,” in International Conference on Computer Vision (2009), pp. 2335–2342.

Ba, J.

D. P. Kingma and J. Ba, “Adam: a method for stochastic optimization,” CoRR abs/1412.6980 (2014).

Bala, K.

S. Bell, K. Bala, and N. Snavely, “Intrinsic images in the wild,” ACM Trans. Graph. 33, 159 (2014).
[Crossref]

Baldrich, R.

H. Sial, S. Sancho-Asensio, R. Baldrich, R. Benavente, and M. Vanrell, “Color-based data augmentation for reflectance estimation,” in IS&T Color and Imaging Conference (2018), vol. 2018, pp. 284–289.

Barron, J. T.

J. T. Barron and J. Malik, “Shape, illumination, and reflectance from shading,” IEEE Trans. Pattern Anal. Mach. Intell. 37, 1670–1687 (2015).
[Crossref]

J. T. Barron and J. Malik, “Intrinsic scene properties from a single RGB-D image,” in IEEE Conference on Computer Vision and Pattern Recognition (2013), pp. 17–24.

E. Shelhamer, J. T. Barron, and T. Darrell, “Scene intrinsics and depth from a single image,” in IEEE International Conference on Computer Vision (ICCV) Workshops (2015).

Barrow, H.

H. Barrow and J. Tenenbaum, “Recovering intrinsic scene characteristics,” Comput. Vis. Syst. 2, 2 (1978).

Baslamisli, A. S.

A. S. Baslamisli, H. Le, and T. Gevers, “CNN based learning using reflection and Retinex models for intrinsic image decomposition,” in Computer Vision and Pattern Recognition (2018).

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (ECCV) (2018).

Bazin, J.

P. Laffont and J. Bazin, “Intrinsic decomposition of image sequences from local temporal variations,” in IEEE International Conference on Computer Vision (ICCV) (2015), pp. 433–441.

Beigpour, S.

S. Beigpour, M. L. Ha, S. Kunz, A. Kolb, and V. Blanz, “Multi-view multi-illuminant intrinsic dataset,” in BMVC (2016).

Bell, S.

S. Bell, K. Bala, and N. Snavely, “Intrinsic images in the wild,” ACM Trans. Graph. 33, 159 (2014).
[Crossref]

Benavente, R.

H. Sial, S. Sancho-Asensio, R. Baldrich, R. Benavente, and M. Vanrell, “Color-based data augmentation for reflectance estimation,” in IS&T Color and Imaging Conference (2018), vol. 2018, pp. 284–289.

M. Serra, O. Penacchio, R. Benavente, and M. Vanrell, “Names and shades of color for intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (2012), pp. 278–285.

Bengio, Y.

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in NIPS (2014).

Bi, S.

S. Bi, X. Han, and Y. Yu, “An l1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition,” ACM Trans. Graph. 34, 78 (2015).
[Crossref]

Black, M. J.

D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black, “A naturalistic open source movie for optical flow evaluation,” in European Conference on Computer Vision (ECCV) (2012).

Blanz, V.

S. Beigpour, M. L. Ha, S. Kunz, A. Kolb, and V. Blanz, “Multi-view multi-illuminant intrinsic dataset,” in BMVC (2016).

Brockington, M.

B. Funt, M. Drew, and M. Brockington, “Recovering shading from color images,” in European Conference on Computer Vision (1992), pp. 124–132.

Bromley, J.

J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, “Signature verification using a Siamese time delay neural network,” Int. J. Pattern Recognit. Artif. Intell. 7, 669–688 (1993).
[Crossref]

Butler, D. J.

D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black, “A naturalistic open source movie for optical flow evaluation,” in European Conference on Computer Vision (ECCV) (2012).

Chang, A. X.

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser, “Semantic scene completion from a single depth image,” in Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition (2017).

Chen, B.

Q. Fan, J. Yang, G. Hua, B. Chen, and D. P. Wipf, “Revisiting deep intrinsic image decompositions,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), pp. 8944–8952.

Chen, Q.

Q. Chen and V. Koltun, “A simple model for intrinsic image decomposition with depth cues,” in IEEE International Conference on Computer Vision (ICCV) (IEEE, 2013), pp. 241–248.

Cho, S.

J. Jeon, S. Cho, X. Tong, and S. Lee, “Intrinsic image decomposition using structure-texture separation and surface normals,” in Computer Vision–ECCV, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, eds. (Springer, 2014), pp. 218–233.

Chu, H.

W.-C. Ma, H. Chu, B. Zhou, R. Urtasun, and A. Torralba, “Single image intrinsic decomposition without a single intrinsic image,” in ECCV (2018).

Clark, R.

W. Li, S. Saeedi, J. McCormac, R. Clark, D. Tzoumanikas, Q. Ye, Y. Huang, R. Tang, and S. Leutenegger, “Interiornet: mega-scale multi-sensor photo-realistic indoor scenes dataset,” in British Machine Vision Conference (BMVC) (2018).

Courville, A. C.

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in NIPS (2014).

Dai, Q.

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to Retinex with nonlocal texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

Darrell, T.

E. Shelhamer, J. T. Barron, and T. Darrell, “Scene intrinsics and depth from a single image,” in IEEE International Conference on Computer Vision (ICCV) Workshops (2015).

Das, P.

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (ECCV) (2018).

Dong, Y.

J. Shi, Y. Dong, H. Su, and S. X. Yu, “Learning non-Lambertian object intrinsics across ShapeNet categories,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 5844–5853.

Drew, M.

B. Funt, M. Drew, and M. Brockington, “Recovering shading from color images,” in European Conference on Computer Vision (1992), pp. 124–132.

Efros, A. A.

T. Zhou, P. Krähenbühl, and A. A. Efros, “Learning data-driven reflectance priors for intrinsic image decomposition,” in IEEE International Conference on Computer Vision (ICCV) (2015), pp. 3469–3477.

Eigen, D.

D. Eigen, C. Puhrsch, and R. Fergus, “Depth map prediction from a single image using a multi-scale deep network,” in NIPS (2014).

Fan, Q.

Q. Fan, J. Yang, G. Hua, B. Chen, and D. P. Wipf, “Revisiting deep intrinsic image decompositions,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), pp. 8944–8952.

Fergus, R.

D. Eigen, C. Puhrsch, and R. Fergus, “Depth map prediction from a single image using a multi-scale deep network,” in NIPS (2014).

Freeman, W. T.

M. F. Tappen, W. T. Freeman, and E. H. Adelson, “Recovering intrinsic images from a single image,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005).
[Crossref]

M. F. Tappen, E. H. Adelson, and W. T. Freeman, “Estimating intrinsic component images using non-linear regression,” in IEEE Conference on Computer Vision and Pattern Recognition (2006), pp. 1992–1999.

R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, “Ground-truth dataset and baseline evaluations for intrinsic image algorithms,” in International Conference on Computer Vision (2009), pp. 2335–2342.

Funkhouser, T.

S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser, “Semantic scene completion from a single depth image,” in Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition (2017).

Funkhouser, T. A.

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

Funt, B.

B. Funt, M. Drew, and M. Brockington, “Recovering shading from color images,” in European Conference on Computer Vision (1992), pp. 124–132.

Garces, E.

E. Garces, A. Munoz, J. Lopez-Moreno, and D. Gutierrez, “Intrinsic images by clustering,” Comput. Graph. Forum 31, 1415–1424 (2012).
[Crossref]

Gehler, P. V.

P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Neural Information Processing Systems (2011), pp. 765–773.

T. Nestmeyer and P. V. Gehler, “Reflectance adaptive filtering improves intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 1771–1780.

Gevers, T.

A. S. Baslamisli, H. Le, and T. Gevers, “CNN based learning using reflection and Retinex models for intrinsic image decomposition,” in Computer Vision and Pattern Recognition (2018).

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (ECCV) (2018).

Gong, M.

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image depth video,” in European Conference on Computer Vision (2012), pp. 327–340.

Goodfellow, I. J.

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in NIPS (2014).

Gool, L. V.

L. Lettry, K. Vanhoey, and L. V. Gool, “DARN: a deep adversarial residual network for intrinsic image decomposition,” in IEEE Winter Conference on Applications of Computer Vision (WACV) (2018), pp. 1359–1367.

Groenestege, T. T.

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (ECCV) (2018).

Grosse, R.

R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, “Ground-truth dataset and baseline evaluations for intrinsic image algorithms,” in International Conference on Computer Vision (2009), pp. 2335–2342.

Guibas, L. J.

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

Gutierrez, D.

E. Garces, A. Munoz, J. Lopez-Moreno, and D. Gutierrez, “Intrinsic images by clustering,” Comput. Graph. Forum 31, 1415–1424 (2012).
[Crossref]

Guyon, I.

J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, “Signature verification using a Siamese time delay neural network,” Int. J. Pattern Recognit. Artif. Intell. 7, 669–688 (1993).
[Crossref]

Ha, M. L.

S. Beigpour, M. L. Ha, S. Kunz, A. Kolb, and V. Blanz, “Multi-view multi-illuminant intrinsic dataset,” in BMVC (2016).

Han, X.

S. Bi, X. Han, and Y. Yu, “An l1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition,” ACM Trans. Graph. 34, 78 (2015).
[Crossref]

Hanrahan, P.

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

He, K.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778.

Horn, B. K.

B. K. Horn, “Determining lightness from an image,” Comput. Graph. Image Process. 3, 277–299 (1974).
[Crossref]

Hua, B.-S.

L. Shen, C. Yeo, and B.-S. Hua, “Intrinsic image decomposition using a sparse representation of reflectance,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 2904–2915 (2013).
[Crossref]

Hua, G.

Q. Fan, J. Yang, G. Hua, B. Chen, and D. P. Wipf, “Revisiting deep intrinsic image decompositions,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), pp. 8944–8952.

Huang, Q.

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

Huang, Y.

W. Li, S. Saeedi, J. McCormac, R. Clark, D. Tzoumanikas, Q. Ye, Y. Huang, R. Tang, and S. Leutenegger, “Interiornet: mega-scale multi-sensor photo-realistic indoor scenes dataset,” in British Machine Vision Conference (BMVC) (2018).

Ioffe, S.

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 2818–2826.

Izadi, S.

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image depth video,” in European Conference on Computer Vision (2012), pp. 327–340.

Jeon, J.

J. Jeon, S. Cho, X. Tong, and S. Lee, “Intrinsic image decomposition using structure-texture separation and surface normals,” in Computer Vision–ECCV, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, eds. (Springer, 2014), pp. 218–233.

Johnson, M. K.

R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, “Ground-truth dataset and baseline evaluations for intrinsic image algorithms,” in International Conference on Computer Vision (2009), pp. 2335–2342.

Kang, S. B.

Y. Matsushita, S. Lin, S. B. Kang, and H.-Y. Shum, “Estimating intrinsic images from image sequences with biased illumination,” in Computer Vision–ECCV (Springer, 2004), pp. 274–286.

Karaoglu, S.

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (ECCV) (2018).

Kiefel, M.

P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Neural Information Processing Systems (2011), pp. 765–773.

Kim, S.

S. Kim, K. Park, K. Sohn, and S. Lin, “Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields,” in European Conference on Computer Vision (ECCV) (2016).

Kingma, D. P.

D. P. Kingma and J. Ba, “Adam: a method for stochastic optimization,” CoRR abs/1412.6980 (2014).

Kolb, A.

S. Beigpour, M. L. Ha, S. Kunz, A. Kolb, and V. Blanz, “Multi-view multi-illuminant intrinsic dataset,” in BMVC (2016).

Koltun, V.

Q. Chen and V. Koltun, “A simple model for intrinsic image decomposition with depth cues,” in IEEE International Conference on Computer Vision (ICCV) (IEEE, 2013), pp. 241–248.

Krähenbühl, P.

T. Zhou, P. Krähenbühl, and A. A. Efros, “Learning data-driven reflectance priors for intrinsic image decomposition,” in IEEE International Conference on Computer Vision (ICCV) (2015), pp. 3469–3477.

Kunz, S.

S. Beigpour, M. L. Ha, S. Kunz, A. Kolb, and V. Blanz, “Multi-view multi-illuminant intrinsic dataset,” in BMVC (2016).

Laffont, P.

P. Laffont and J. Bazin, “Intrinsic decomposition of image sequences from local temporal variations,” in IEEE International Conference on Computer Vision (ICCV) (2015), pp. 433–441.

Land, E. H.

Le, H.

A. S. Baslamisli, H. Le, and T. Gevers, “CNN based learning using reflection and Retinex models for intrinsic image decomposition,” in Computer Vision and Pattern Recognition (2018).

Le, H. A.

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (ECCV) (2018).

LeCun, Y.

J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, “Signature verification using a Siamese time delay neural network,” Int. J. Pattern Recognit. Artif. Intell. 7, 669–688 (1993).
[Crossref]

Lee, K. J.

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image depth video,” in European Conference on Computer Vision (2012), pp. 327–340.

Lee, S.

J. Jeon, S. Cho, X. Tong, and S. Lee, “Intrinsic image decomposition using structure-texture separation and surface normals,” in Computer Vision–ECCV, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, eds. (Springer, 2014), pp. 218–233.

Lee, S. U.

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image depth video,” in European Conference on Computer Vision (2012), pp. 327–340.

Lettry, L.

L. Lettry, K. Vanhoey, and L. V. Gool, “DARN: a deep adversarial residual network for intrinsic image decomposition,” in IEEE Winter Conference on Applications of Computer Vision (WACV) (2018), pp. 1359–1367.

Leutenegger, S.

W. Li, S. Saeedi, J. McCormac, R. Clark, D. Tzoumanikas, Q. Ye, Y. Huang, R. Tang, and S. Leutenegger, “Interiornet: mega-scale multi-sensor photo-realistic indoor scenes dataset,” in British Machine Vision Conference (BMVC) (2018).

Li, W.

W. Li, S. Saeedi, J. McCormac, R. Clark, D. Tzoumanikas, Q. Ye, Y. Huang, R. Tang, and S. Leutenegger, “Interiornet: mega-scale multi-sensor photo-realistic indoor scenes dataset,” in British Machine Vision Conference (BMVC) (2018).

Li, Z.

Z. Li and N. Snavely, “Learning intrinsic image decomposition from watching the world,” in CVPR (2018), pp. 9039–9048.

Z. Li and N. Snavely, “CGintrinsics: better intrinsic image decomposition through physically-based rendering,” in European Conference on Computer Vision (ECCV) (2018).

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

Lin, G.

F. Liu, C. Shen, and G. Lin, “Deep convolutional neural fields for depth estimation from a single image,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), pp. 5162–5170.

Lin, S.

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to Retinex with nonlocal texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

S. Kim, K. Park, K. Sohn, and S. Lin, “Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields,” in European Conference on Computer Vision (ECCV) (2016).

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image depth video,” in European Conference on Computer Vision (2012), pp. 327–340.

Y. Matsushita, S. Lin, S. B. Kang, and H.-Y. Shum, “Estimating intrinsic images from image sequences with biased illumination,” in Computer Vision–ECCV (Springer, 2004), pp. 274–286.

Liu, F.

F. Liu, C. Shen, and G. Lin, “Deep convolutional neural fields for depth estimation from a single image,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), pp. 5162–5170.

Lopez-Moreno, J.

E. Garces, A. Munoz, J. Lopez-Moreno, and D. Gutierrez, “Intrinsic images by clustering,” Comput. Graph. Forum 31, 1415–1424 (2012).
[Crossref]

Ma, W.-C.

W.-C. Ma, H. Chu, B. Zhou, R. Urtasun, and A. Torralba, “Single image intrinsic decomposition without a single intrinsic image,” in ECCV (2018).

Maire, M.

T. Narihira, M. Maire, and S. X. Yu, “Direct intrinsics: learning albedo-shading decomposition by convolutional regression,” in International Conference on Computer Vision (ICCV) (2015).

Malik, J.

J. T. Barron and J. Malik, “Shape, illumination, and reflectance from shading,” IEEE Trans. Pattern Anal. Mach. Intell. 37, 1670–1687 (2015).
[Crossref]

J. T. Barron and J. Malik, “Intrinsic scene properties from a single RGB-D image,” in IEEE Conference on Computer Vision and Pattern Recognition (2013), pp. 17–24.

Matsushita, Y.

Y. Matsushita, S. Lin, S. B. Kang, and H.-Y. Shum, “Estimating intrinsic images from image sequences with biased illumination,” in Computer Vision–ECCV (Springer, 2004), pp. 274–286.

McCann, J.

McCormac, J.

W. Li, S. Saeedi, J. McCormac, R. Clark, D. Tzoumanikas, Q. Ye, Y. Huang, R. Tang, and S. Leutenegger, “Interiornet: mega-scale multi-sensor photo-realistic indoor scenes dataset,” in British Machine Vision Conference (BMVC) (2018).

Mirza, M.

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in NIPS (2014).

Munoz, A.

E. Garces, A. Munoz, J. Lopez-Moreno, and D. Gutierrez, “Intrinsic images by clustering,” Comput. Graph. Forum 31, 1415–1424 (2012).
[Crossref]

Narihira, T.

T. Narihira, M. Maire, and S. X. Yu, “Direct intrinsics: learning albedo-shading decomposition by convolutional regression,” in International Conference on Computer Vision (ICCV) (2015).

Nestmeyer, T.

T. Nestmeyer and P. V. Gehler, “Reflectance adaptive filtering improves intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 1771–1780.

Ozair, S.

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in NIPS (2014).

Park, K.

S. Kim, K. Park, K. Sohn, and S. Lin, “Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields,” in European Conference on Computer Vision (ECCV) (2016).

Penacchio, O.

M. Serra, O. Penacchio, R. Benavente, and M. Vanrell, “Names and shades of color for intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (2012), pp. 278–285.

Pouget-Abadie, J.

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in NIPS (2014).

Puhrsch, C.

D. Eigen, C. Puhrsch, and R. Fergus, “Depth map prediction from a single image using a multi-scale deep network,” in NIPS (2014).

Ren, S.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778.

Rother, C.

P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Neural Information Processing Systems (2011), pp. 765–773.

Säckinger, E.

J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, “Signature verification using a Siamese time delay neural network,” Int. J. Pattern Recognit. Artif. Intell. 7, 669–688 (1993).
[Crossref]

Saeedi, S.

W. Li, S. Saeedi, J. McCormac, R. Clark, D. Tzoumanikas, Q. Ye, Y. Huang, R. Tang, and S. Leutenegger, “Interiornet: mega-scale multi-sensor photo-realistic indoor scenes dataset,” in British Machine Vision Conference (BMVC) (2018).

Sancho-Asensio, S.

H. Sial, S. Sancho-Asensio, R. Baldrich, R. Benavente, and M. Vanrell, “Color-based data augmentation for reflectance estimation,” in IS&T Color and Imaging Conference (2018), vol. 2018, pp. 284–289.

Savarese, S.

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

Savva, M.

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser, “Semantic scene completion from a single depth image,” in Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition (2017).

Schölkopf, B.

P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Neural Information Processing Systems (2011), pp. 765–773.

Serra, M.

M. Serra, O. Penacchio, R. Benavente, and M. Vanrell, “Names and shades of color for intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (2012), pp. 278–285.

Shah, R.

J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, “Signature verification using a Siamese time delay neural network,” Int. J. Pattern Recognit. Artif. Intell. 7, 669–688 (1993).
[Crossref]

Shelhamer, E.

E. Shelhamer, J. T. Barron, and T. Darrell, “Scene intrinsics and depth from a single image,” in IEEE International Conference on Computer Vision (ICCV) Workshops (2015).

Shen, C.

F. Liu, C. Shen, and G. Lin, “Deep convolutional neural fields for depth estimation from a single image,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), pp. 5162–5170.

Shen, L.

L. Shen, C. Yeo, and B.-S. Hua, “Intrinsic image decomposition using a sparse representation of reflectance,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 2904–2915 (2013).
[Crossref]

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to Retinex with nonlocal texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

L. Shen and C. Yeo, “Intrinsic images decomposition using a local and global sparse representation of reflectance,” in CVPR (2011), pp. 697–704.

Shi, J.

J. Shi, Y. Dong, H. Su, and S. X. Yu, “Learning non-Lambertian object intrinsics across ShapeNet categories,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 5844–5853.

Shlens, J.

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 2818–2826.

Shum, H.-Y.

Y. Matsushita, S. Lin, S. B. Kang, and H.-Y. Shum, “Estimating intrinsic images from image sequences with biased illumination,” in Computer Vision–ECCV (Springer, 2004), pp. 274–286.

Sial, H.

H. Sial, S. Sancho-Asensio, R. Baldrich, R. Benavente, and M. Vanrell, “Color-based data augmentation for reflectance estimation,” in IS&T Color and Imaging Conference (2018), vol. 2018, pp. 284–289.

Snavely, N.

S. Bell, K. Bala, and N. Snavely, “Intrinsic images in the wild,” ACM Trans. Graph. 33, 159 (2014).
[Crossref]

Z. Li and N. Snavely, “CGintrinsics: better intrinsic image decomposition through physically-based rendering,” in European Conference on Computer Vision (ECCV) (2018).

Z. Li and N. Snavely, “Learning intrinsic image decomposition from watching the world,” in CVPR (2018), pp. 9039–9048.

Sohn, K.

S. Kim, K. Park, K. Sohn, and S. Lin, “Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields,” in European Conference on Computer Vision (ECCV) (2016).

Song, S.

S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser, “Semantic scene completion from a single depth image,” in Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition (2017).

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

Stanley, G. B.

D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black, “A naturalistic open source movie for optical flow evaluation,” in European Conference on Computer Vision (ECCV) (2012).

Su, H.

J. Shi, Y. Dong, H. Su, and S. X. Yu, “Learning non-Lambertian object intrinsics across ShapeNet categories,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 5844–5853.

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

Sun, J.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778.

Szegedy, C.

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 2818–2826.

Tan, P.

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to Retinex with nonlocal texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image depth video,” in European Conference on Computer Vision (2012), pp. 327–340.

Tang, R.

W. Li, S. Saeedi, J. McCormac, R. Clark, D. Tzoumanikas, Q. Ye, Y. Huang, R. Tang, and S. Leutenegger, “Interiornet: mega-scale multi-sensor photo-realistic indoor scenes dataset,” in British Machine Vision Conference (BMVC) (2018).

Tappen, M. F.

M. F. Tappen, W. T. Freeman, and E. H. Adelson, “Recovering intrinsic images from a single image,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005).
[Crossref]

M. F. Tappen, E. H. Adelson, and W. T. Freeman, “Estimating intrinsic component images using non-linear regression,” in IEEE Conference on Computer Vision and Pattern Recognition (2006), pp. 1992–1999.

Tenenbaum, J.

H. Barrow and J. Tenenbaum, “Recovering intrinsic scene characteristics,” Comput. Vis. Syst. 2, 2 (1978).

Tong, X.

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image depth video,” in European Conference on Computer Vision (2012), pp. 327–340.

J. Jeon, S. Cho, X. Tong, and S. Lee, “Intrinsic image decomposition using structure-texture separation and surface normals,” in Computer Vision–ECCV, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, eds. (Springer, 2014), pp. 218–233.

Torralba, A.

W.-C. Ma, H. Chu, B. Zhou, R. Urtasun, and A. Torralba, “Single image intrinsic decomposition without a single intrinsic image,” in ECCV (2018).

Tzoumanikas, D.

W. Li, S. Saeedi, J. McCormac, R. Clark, D. Tzoumanikas, Q. Ye, Y. Huang, R. Tang, and S. Leutenegger, “Interiornet: mega-scale multi-sensor photo-realistic indoor scenes dataset,” in British Machine Vision Conference (BMVC) (2018).

Urtasun, R.

W.-C. Ma, H. Chu, B. Zhou, R. Urtasun, and A. Torralba, “Single image intrinsic decomposition without a single intrinsic image,” in ECCV (2018).

Vanhoey, K.

L. Lettry, K. Vanhoey, and L. V. Gool, “DARN: a deep adversarial residual network for intrinsic image decomposition,” in IEEE Winter Conference on Applications of Computer Vision (WACV) (2018), pp. 1359–1367.

Vanhoucke, V.

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 2818–2826.

Vanrell, M.

H. Sial, S. Sancho-Asensio, R. Baldrich, R. Benavente, and M. Vanrell, “Color-based data augmentation for reflectance estimation,” in IS&T Color and Imaging Conference (2018), vol. 2018, pp. 284–289.

M. Serra, O. Penacchio, R. Benavente, and M. Vanrell, “Names and shades of color for intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (2012), pp. 278–285.

Warde-Farley, D.

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in NIPS (2014).

Weiss, Y.

Y. Weiss, “Deriving intrinsic images from image sequences,” in International Conference on Computer Vision (2001), pp. 68–75.

Wipf, D. P.

Q. Fan, J. Yang, G. Hua, B. Chen, and D. P. Wipf, “Revisiting deep intrinsic image decompositions,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), pp. 8944–8952.

Wojna, Z.

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 2818–2826.

Wu, E.

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to Retinex with nonlocal texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

Wulff, J.

D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black, “A naturalistic open source movie for optical flow evaluation,” in European Conference on Computer Vision (ECCV) (2012).

Xiao, J.

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

Xu, B.

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in NIPS (2014).

Yang, J.

Q. Fan, J. Yang, G. Hua, B. Chen, and D. P. Wipf, “Revisiting deep intrinsic image decompositions,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), pp. 8944–8952.

Ye, Q.

W. Li, S. Saeedi, J. McCormac, R. Clark, D. Tzoumanikas, Q. Ye, Y. Huang, R. Tang, and S. Leutenegger, “Interiornet: mega-scale multi-sensor photo-realistic indoor scenes dataset,” in British Machine Vision Conference (BMVC) (2018).

Yeo, C.

L. Shen, C. Yeo, and B.-S. Hua, “Intrinsic image decomposition using a sparse representation of reflectance,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 2904–2915 (2013).
[Crossref]

L. Shen and C. Yeo, “Intrinsic images decomposition using a local and global sparse representation of reflectance,” in CVPR (2011), pp. 697–704.

Yi, L.

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

Yu, F.

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser, “Semantic scene completion from a single depth image,” in Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition (2017).

Yu, S. X.

T. Narihira, M. Maire, and S. X. Yu, “Direct intrinsics: learning albedo-shading decomposition by convolutional regression,” in International Conference on Computer Vision (ICCV) (2015).

J. Shi, Y. Dong, H. Su, and S. X. Yu, “Learning non-Lambertian object intrinsics across ShapeNet categories,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 5844–5853.

Yu, Y.

S. Bi, X. Han, and Y. Yu, “An l1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition,” ACM Trans. Graph. 34, 78 (2015).
[Crossref]

Zeng, A.

S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser, “Semantic scene completion from a single depth image,” in Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition (2017).

Zhang, L.

P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Neural Information Processing Systems (2011), pp. 765–773.

Zhang, X.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778.

Zhao, Q.

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to Retinex with nonlocal texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image depth video,” in European Conference on Computer Vision (2012), pp. 327–340.

Zhou, B.

W.-C. Ma, H. Chu, B. Zhou, R. Urtasun, and A. Torralba, “Single image intrinsic decomposition without a single intrinsic image,” in ECCV (2018).

Zhou, T.

T. Zhou, P. Krähenbühl, and A. A. Efros, “Learning data-driven reflectance priors for intrinsic image decomposition,” in IEEE International Conference on Computer Vision (ICCV) (2015), pp. 3469–3477.

ACM Trans. Graph. (2)

S. Bell, K. Bala, and N. Snavely, “Intrinsic images in the wild,” ACM Trans. Graph. 33, 159 (2014).
[Crossref]

S. Bi, X. Han, and Y. Yu, “An l1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition,” ACM Trans. Graph. 34, 78 (2015).
[Crossref]

Comput. Graph. Forum (1)

E. Garces, A. Munoz, J. Lopez-Moreno, and D. Gutierrez, “Intrinsic images by clustering,” Comput. Graph. Forum 31, 1415–1424 (2012).
[Crossref]

Comput. Graph. Image Process. (1)

B. K. Horn, “Determining lightness from an image,” Comput. Graph. Image Process. 3, 277–299 (1974).
[Crossref]

Comput. Vis. Syst. (1)

H. Barrow and J. Tenenbaum, “Recovering intrinsic scene characteristics,” Comput. Vis. Syst. 2, 2 (1978).

IEEE Trans. Pattern Anal. Mach. Intell. (4)

L. Shen, C. Yeo, and B.-S. Hua, “Intrinsic image decomposition using a sparse representation of reflectance,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 2904–2915 (2013).
[Crossref]

J. T. Barron and J. Malik, “Shape, illumination, and reflectance from shading,” IEEE Trans. Pattern Anal. Mach. Intell. 37, 1670–1687 (2015).
[Crossref]

Q. Zhao, P. Tan, Q. Dai, L. Shen, E. Wu, and S. Lin, “A closed-form solution to Retinex with nonlocal texture constraints,” IEEE Trans. Pattern Anal. Mach. Intell. 34, 1437–1444 (2012).
[Crossref]

M. F. Tappen, W. T. Freeman, and E. H. Adelson, “Recovering intrinsic images from a single image,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005).
[Crossref]

Int. J. Pattern Recognit. Artif. Intell. (1)

J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, “Signature verification using a Siamese time delay neural network,” Int. J. Pattern Recognit. Artif. Intell. 7, 669–688 (1993).
[Crossref]

J. Opt. Soc. Am. (1)

Other (40)

M. F. Tappen, E. H. Adelson, and W. T. Freeman, “Estimating intrinsic component images using non-linear regression,” in IEEE Conference on Computer Vision and Pattern Recognition (2006), pp. 1992–1999.

B. Funt, M. Drew, and M. Brockington, “Recovering shading from color images,” in European Conference on Computer Vision (1992), pp. 124–132.

P. V. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schölkopf, “Recovering intrinsic images with a global sparsity prior on reflectance,” in Neural Information Processing Systems (2011), pp. 765–773.

Z. Li and N. Snavely, “Learning intrinsic image decomposition from watching the world,” in CVPR (2018), pp. 9039–9048.

Q. Fan, J. Yang, G. Hua, B. Chen, and D. P. Wipf, “Revisiting deep intrinsic image decompositions,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), pp. 8944–8952.

S. Beigpour, M. L. Ha, S. Kunz, A. Kolb, and V. Blanz, “Multi-view multi-illuminant intrinsic dataset,” in BMVC (2016).

S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser, “Semantic scene completion from a single depth image,” in Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition (2017).

W. Li, S. Saeedi, J. McCormac, R. Clark, D. Tzoumanikas, Q. Ye, Y. Huang, R. Tang, and S. Leutenegger, “Interiornet: mega-scale multi-sensor photo-realistic indoor scenes dataset,” in British Machine Vision Conference (BMVC) (2018).

H. Sial, S. Sancho-Asensio, R. Baldrich, R. Benavente, and M. Vanrell, “Color-based data augmentation for reflectance estimation,” in IS&T Color and Imaging Conference (2018), vol. 2018, pp. 284–289.

“Corel datasets,” http://www.emsps.com/photocd/corelcds.htm .

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 2818–2826.

F. Chollet, “Keras,” 2015, https://github.com/fchollet/keras .

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778.

D. P. Kingma and J. Ba, “Adam: a method for stochastic optimization,” CoRR abs/1412.6980 (2014).

L. Shen and C. Yeo, “Intrinsic images decomposition using a local and global sparse representation of reflectance,” in CVPR (2011), pp. 697–704.

R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, “Ground-truth dataset and baseline evaluations for intrinsic image algorithms,” in International Conference on Computer Vision (2009), pp. 2335–2342.

Y. Weiss, “Deriving intrinsic images from image sequences,” in International Conference on Computer Vision (2001), pp. 68–75.

D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black, “A naturalistic open source movie for optical flow evaluation,” in European Conference on Computer Vision (ECCV) (2012).

J. Shi, Y. Dong, H. Su, and S. X. Yu, “Learning non-Lambertian object intrinsics across ShapeNet categories,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 5844–5853.

A. S. Baslamisli, H. Le, and T. Gevers, “CNN based learning using reflection and Retinex models for intrinsic image decomposition,” in Computer Vision and Pattern Recognition (2018).

A. S. Baslamisli, T. T. Groenestege, P. Das, H. A. Le, S. Karaoglu, and T. Gevers, “Joint learning of intrinsic images and semantic segmentation,” in European Conference on Computer Vision (ECCV) (2018).

Z. Li and N. Snavely, “CGintrinsics: better intrinsic image decomposition through physically-based rendering,” in European Conference on Computer Vision (ECCV) (2018).

A. X. Chang, T. A. Funkhouser, L. J. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “ShapeNet: an information-rich 3D model repository,” CoRR abs/1512.03012 (2015).

M. Serra, O. Penacchio, R. Benavente, and M. Vanrell, “Names and shades of color for intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (2012), pp. 278–285.

J. T. Barron and J. Malik, “Intrinsic scene properties from a single RGB-D image,” in IEEE Conference on Computer Vision and Pattern Recognition (2013), pp. 17–24.

K. J. Lee, Q. Zhao, X. Tong, M. Gong, S. Izadi, S. U. Lee, P. Tan, and S. Lin, “Estimation of intrinsic image sequences from image depth video,” in European Conference on Computer Vision (2012), pp. 327–340.

Q. Chen and V. Koltun, “A simple model for intrinsic image decomposition with depth cues,” in IEEE International Conference on Computer Vision (ICCV) (IEEE, 2013), pp. 241–248.

J. Jeon, S. Cho, X. Tong, and S. Lee, “Intrinsic image decomposition using structure-texture separation and surface normals,” in Computer Vision–ECCV, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, eds. (Springer, 2014), pp. 218–233.

Y. Matsushita, S. Lin, S. B. Kang, and H.-Y. Shum, “Estimating intrinsic images from image sequences with biased illumination,” in Computer Vision–ECCV (Springer, 2004), pp. 274–286.

P. Laffont and J. Bazin, “Intrinsic decomposition of image sequences from local temporal variations,” in IEEE International Conference on Computer Vision (ICCV) (2015), pp. 433–441.

T. Narihira, M. Maire, and S. X. Yu, “Direct intrinsics: learning albedo-shading decomposition by convolutional regression,” in International Conference on Computer Vision (ICCV) (2015).

D. Eigen, C. Puhrsch, and R. Fergus, “Depth map prediction from a single image using a multi-scale deep network,” in NIPS (2014).

T. Zhou, P. Krähenbühl, and A. A. Efros, “Learning data-driven reflectance priors for intrinsic image decomposition,” in IEEE International Conference on Computer Vision (ICCV) (2015), pp. 3469–3477.

E. Shelhamer, J. T. Barron, and T. Darrell, “Scene intrinsics and depth from a single image,” in IEEE International Conference on Computer Vision (ICCV) Workshops (2015).

F. Liu, C. Shen, and G. Lin, “Deep convolutional neural fields for depth estimation from a single image,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), pp. 5162–5170.

S. Kim, K. Park, K. Sohn, and S. Lin, “Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields,” in European Conference on Computer Vision (ECCV) (2016).

T. Nestmeyer and P. V. Gehler, “Reflectance adaptive filtering improves intrinsic image estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 1771–1780.

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in NIPS (2014).

L. Lettry, K. Vanhoey, and L. V. Gool, “DARN: a deep adversarial residual network for intrinsic image decomposition,” in IEEE Winter Conference on Applications of Computer Vision (WACV) (2018), pp. 1359–1367.

W.-C. Ma, H. Chu, B. Zhou, R. Urtasun, and A. Torralba, “Single image intrinsic decomposition without a single intrinsic image,” in ECCV (2018).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1.
Fig. 1. Dataset generation setup.
Fig. 2.
Fig. 2. IUI-Network architecture. One encoder and two decoders for reflectance and shading estimation. Three interrelated loss functions. Types of layers are indicated by a color code given at right-bottom of the figure. Scheme of inception modules are given at left-top of the figure.
Fig. 3.
Fig. 3. Training versus loss for different training set sizes. Error stabilizes from 15,000 training samples.
Fig. 4.
Fig. 4. Some examples of our SID dataset. (a) Original images. (b), (d) Reflectance and shading estimation by our IUI network, respectively. (c), (e) GT reflectance and shading, respectively.
Fig. 5.
Fig. 5. Qualitative results on MIT intrinsic image dataset. Compared to other methods, we achieved sharper and better colors and removed shading effects. Our method performed best in bringing reflectance details from dark part of image.
Fig. 6.
Fig. 6. Visual comparison on MPI-Sintel dataset using image split.
Fig. 7.
Fig. 7. Visual comparison on MPI-Sintel using scene split.
Fig. 8.
Fig. 8. Qualitative results on IIW.

Tables (7)

Tables Icon

Table 1. Comparison on Current Available Dataset According to Several Propertiesa,b

Tables Icon

Table 2. Errors for Reflectance and Shading Predictions on Our Dataseta

Tables Icon

Table 3. Estimation Errors on MIT Dataset Reported in Previous Works by Different Methods and for Our IUI Architecture

Tables Icon

Table 4. Results on Sintel Image Split Dataseta

Tables Icon

Table 5. Result on Sintel Scene Split Dataseta

Tables Icon

Table 6. Result on IIW Dataset

Tables Icon

Table 7. Estimation Errors of Different Architectures Trained on ShapeNet-Intrinsic Dataseta

Equations (5)

Equations on this page are rendered with MathJax. Learn more.

I ( x , y ) = R e f ( x , y ) S h a ( x , y ) ,
L I n t ( I , R ^ , S ^ ) = α 1 L R e f ( R , R ^ ) + α 2 L S h a ( S , S ^ ) + α 3 L R S ( I , R ^ S ^ ) ,
M S E ( x , x ¯ ) = i = 1 N x i α ¯ x ¯ i 2 N ,
S S I M ( x , x ¯ ) = ( 2 μ μ ¯ + c 1 ) ( 2 σ x x ¯ + c 2 ) ( μ 2 + μ ¯ 2 + c 1 ) ( σ 2 + σ ^ 2 + c 2 ) ,
D S S I M ( x , x ¯ ) = 1 S S I M ( x , x ¯ ) 2 .

Metrics