Surgery and radiotherapy in breast cancer patients can cause treatment changes and may affect the final breast appearance. In this study, we are trying to evaluate the post treatment breast photographs of the patients and subject these to Artificial Intelligence based program so as to classify into appropriate categories based upon changes from baseline. This automated solution will help in decreasing the time required to achieve this task by physicians in the clinic.

A new algorithm was introduced which is based on deep neural network (DNN) which receives an image as input and returns the coordinates of the breast key points as output. These key points are then given to a shortest-path algorithm that models images as graphs to refine breast key point localization. The algorithm learns, directly from the image, to compute features and to use those features in the analysis of the aesthetic result. This comprises of two main modules: regression and refinement of heatmaps, and regression of key points. To perform the heatmap regression, the U-Net model is used.

The goal of the first module is to generate an intermediate representation consisting on a fuzzy localization for the key points that are to be detected.

The second module receives and refines this fuzzy localization, and through complex calculations, outputting the x and y coordinates of the keypoints, and the data generated from which can be used for disease / image classification.

Inclusion Criteria:

• Confirmed diagnosis of primary breast cancer (invasive or in situ)
• Patient undergone breast conservation / Whole breast reconstruction
• Patient received breast RT
• Already provided written informed consent on earlier projects
• Patient provided photographs of both breasts
• Non-metastatic disease or oligometastatic
• Age > 18 years
• Reconsent given

Exclusion Criteria:

• Mastectomy without whole breast reconstruction
• Bilateral breast cancer
• Partial breast irradiation
• Male patient
• Limited life expectancy due to co-morbidity
• Patients undergoing brachy boost

Only female breast cancer patients will be studied

This is a retrospective analysis of patient photographs that have been acquired after written informed consent as per ethical requirements. The patients accrued in the ongoing prospective study (CTRI/2020/01/022871) have been re-consented for the current study in order to subject their breast photographs for neural network analysis. No photographs are taken separately for the current study. Hence this is essentially a retrospective study of the breast photographs to predict cosmesis.