The original resolution of in total 40 images is 2048x1536 divided into three datasets (images in set I: 20x, set II: 10x, set III:10x). Each of the three datasets correspond to a specific cooling strategy (i,ii or iii) of a bainitic steel and therefore to a bainitic microstructure with specific characteristics. After cropping the image size of each dataset, sub-datasets are created with image resolution: 768x768, 512x512, 256x256, 192x192, 128x128 and 64x64 (only for dataset II). Each sub-dataset contains a training set ("im_train"), validation set ("im_val") and labels for train and validation set. The appendix provides (i) a dataset consisting of all 40 images in reference size, (ii) the reference images of dataset I which was merged from images of two cooling strategies.