The manhattan dataset is collected over the downtown and uptown regions in Manhattan, New York City, USA. The very-high-resolution aerial images with a spatial resolution of 0.2m are acquired through the USGS EarthExplorer.The other two datasets are Vancouver in Canada and Stockholm in Europe. Remote sensing images with a spatial resolution of 0.3m were collected through the GoogleMap. Building footprints were collected from the OSM data portal.