Multimodal Vision Research Laboratory

• Role: PI
• Funding Source: Google
• Award Date: 2016

Overview

This project seeks to develop deep convolutional neural networks that can combine ground-level and overhead imagery for remote sensing tasks. We are exploring several approaches. See below for work that was supported by this award.

Related Publication(s)

1. Salem T, Workman S, Jacobs N. 2020. Learning a Dynamic Map of Visual Appearance. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
   bibtex | website

   ×Close

   Bibtex Entry for "Learning a Dynamic Map of Visual Appearance"

   @inproceedings{salem2020dynamic,
     author = {Salem, Tawfiq and Workman, Scott and Jacobs, Nathan},
     booktitle = {{IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}},
     title = {{Learning a Dynamic Map of Visual Appearance}},
     year = {2020},
     annotation = {google_crossview,localization,career,remote_sensing,spotlight,timelapse},
     date-modified = {2020-03-31 16:54:47 -0400},
     project = {https://tsalem.github.io/DynamicMaps/},
     note = {Acceptance rate: 25\%},
     pdf = {http://openaccess.thecvf.com/content_CVPR_2020/papers/Salem_Learning_a_Dynamic_Map_of_Visual_Appearance_CVPR_2020_paper.pdf},
     video = {https://www.youtube.com/watch?v=OVTvgWLlJmM},
     thumbnail = {http://cs.uky.edu/%7ejacobs/thumbnails/dynamic_visual.jpg}
   }
   

2. Workman S, Jacobs N. 2020. Dynamic Traffic Modeling from Overhead Imagery. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
   bibtex | website

   ×Close

   Bibtex Entry for "Dynamic Traffic Modeling from Overhead Imagery"

   @inproceedings{workman2020dynamic,
     annotation = {google_crossview,localization,career,remote_sensing,spotlight},
     author = {Workman, Scott and Jacobs, Nathan},
     booktitle = {{IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}},
     date-modified = {2020-03-31 16:54:51 -0400},
     note = {(oral), Acceptance rate: 5.7\%},
     thumbnail = {http://cs.uky.edu/%7ejacobs/thumbnails/dynamic_road.jpg},
     title = {Dynamic Traffic Modeling from Overhead Imagery},
     year = {2020},
     video = {https://www.youtube.com/watch?v=CNe5EEdyptE},
     project = {http://cs.uky.edu/%7escott/research/traffic/},
     pdf = {http://openaccess.thecvf.com/content_CVPR_2020/papers/Workman_Dynamic_Traffic_Modeling_From_Overhead_Imagery_CVPR_2020_paper.pdf}
   }
   

3. Greenwell C, Workman S, Jacobs N. 2019. Implicit Land Use Mapping Using Social Media Imagery. In: IEEE Applied Imagery and Pattern Recognition (AIPR).
   bibtex

   ×Close

   Bibtex Entry for "Implicit Land Use Mapping Using Social Media Imagery"

   @inproceedings{greenwell2019implicit,
     annotation = {abstract,google_crossview,amazon,nvidia,remote_sensing,social_media},
     author = {Greenwell, Connor and Workman, Scott and Jacobs, Nathan},
     booktitle = {IEEE Applied Imagery and Pattern Recognition (AIPR)},
     title = {Implicit Land Use Mapping Using Social Media Imagery},
     year = {2019}
   }
   

4. Workman S, Zhai M, Crandall D, Jacobs N. 2017. A Unified Model for Near/Remote Sensing. In: IEEE International Conference on Computer Vision (ICCV).
   bibtex | website

   ×Close

   Bibtex Entry for "A Unified Model for Near/Remote Sensing"

   @inproceedings{workman2017nearremote,
     annotation = {google_crossview,career,spotlight,remote_sensing,social_media},
     author = {Workman, Scott and Zhai, Menghua and Crandall, David and Jacobs, Nathan},
     booktitle = {{IEEE International Conference on Computer Vision (ICCV)}},
     project = {http://cs.uky.edu/%7escott/research/unified/},
     thumbnail = {http://cs.uky.edu/%7ejacobs/thumbnails/nearremote.jpg},
     title = {A Unified Model for Near/Remote Sensing},
     year = {2017}
   }
   

5. Workman S, Souvenir R, Jacobs N. 2017. Understanding and Mapping Natural Beauty. In: IEEE International Conference on Computer Vision (ICCV).
   bibtex | website

   ×Close

   Bibtex Entry for "Understanding and Mapping Natural Beauty"

   @inproceedings{workman2017scenic,
     annotation = {google_crossview,career,spotlight,remote_sensing,social_media},
     author = {Workman, Scott and Souvenir, Richard and Jacobs, Nathan},
     booktitle = {{IEEE International Conference on Computer Vision (ICCV)}},
     project = {http://cs.uky.edu/%7escott/research/scenicness/},
     thumbnail = {http://cs.uky.edu/%7ejacobs/thumbnails/beauty.jpg},
     title = {Understanding and Mapping Natural Beauty},
     pdf = {https://arxiv.org/abs/1612.03142},
     year = {2017}
   }
   

6. Zhai M, Bessinger Z, Workman S, Jacobs N. 2017. Predicting Ground-Level Scene Layout from Aerial Imagery. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
   bibtex | code

   ×Close

   Bibtex Entry for "Predicting Ground-Level Scene Layout from Aerial Imagery"

   @inproceedings{zhai2017layout,
     annotation = {google_crossview,localization,spotlight,nvidia,amazon,career,remote_sensing},
     author = {Zhai, Menghua and Bessinger, Zach and Workman, Scott and Jacobs, Nathan},
     booktitle = {{IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}},
     code = {https://github.com/viibridges/crossnet},
     date-modified = {2020-03-31 16:54:39 -0400},
     pdf = {http://arxiv.org/pdf/1612.02709},
     thumbnail = {http://cs.uky.edu/%7ejacobs/thumbnails/predict_layout.jpg},
     title = {Predicting Ground-Level Scene Layout from Aerial Imagery},
     year = {2017}
   }
   

7. Jacobs N, Workman S, Zhai M. 2016. Crossview Convolutional Networks. In: IEEE Applied Imagery and Pattern Recognition (AIPR).
   bibtex

   ×Close

   Bibtex Entry for "Crossview Convolutional Networks"

   @inproceedings{jacobs2016crossview,
     annotation = {google_crossview,amazon,nvidia,remote_sensing,abstract,social_media},
     author = {Jacobs, Nathan and Workman, Scott and Zhai, Menghua},
     booktitle = {IEEE Applied Imagery and Pattern Recognition (AIPR)},
     title = {Crossview Convolutional Networks},
     year = {2016}
   }
   

Acknowledgements

This work was supported by a Google Faculty Research Award.