Deepfake Histologic Images for Expanding Colorectal Cancer Histopathology Database Using StyleGAN3

Pathological diagnosis is the basis of cancer treatment. High-resolution pathological images are crucial for model training and routine learning for physicians. However, significant challenges such as the heterogeneity of pathological images, the rarity of some subtypes insufficient pathological image data, uneven data distribution, restricted data sharing and privacy issues hamper the improvement of pathologists’ clinical decision-making abilities. To address these challenges, we proposed using StyleGAN3 to generate colorectal cancer images, aiming at providing clinicians with abundant learning resources and offering sufficient data support for deep learning models that focus on pathological images as their research subjects. Firstly, using the nonlinear mapping network in StyleGAN3 to map random vectors to a latent space, introducing the latent vector at each level of the generator to decouple the learning of different pathological features at different levels, thereby training StyleGAN3. Self-supervised pre-training was performed using equal amounts of synthetic and real images, and subsequent classification and semantic segmentation were performed using the pre-trained model to test the differences between synthetic and real images in downstream tasks. Clinicians in different departments distinguished the synthetic image from the real image to verify the visual credibility of the synthetic image.
The results verify the reliability of model training using synthetic pathological images and ensure the visual rationality of the generated images, concluding that the pathological images generated using StyleGAN3 have significant value in terms of feature extraction and utilization.
The dynamic visualization effects of the generated image are shown as follows.

The weights used for generating pathological images can be downloaded from https://drive.google.com/file/d/1Oms4N_vgLwkMt9RnW4XSCGtDEozCyNA7/view?usp=drive_link.

Then, move to https://github.com/NVlabs/stylegan3, and follow the instructions in the project’s README.md to generate images using gen_images.py. Simply replace the weights with the pkl weights you downloaded.

If you have any questions, feel free to contact us at kikai010310@gmail.com.