I am using Keras for boundary/contour detection using a Unet. When I use binary cross-entropy as the loss, the losses decrease over time as expected the predicted boundaries look reasonable
However, I have tried custom losses for Dice, Focal, IOU, with varying LRs, and none of them are working well. I either get NaNs or non-decreasing/barely-decreasing values for the losses. This is regardless of what I use for the LR, whether it be .01 to 1e-6, or whether I vary the ALPHA and GAMMA and other parameters. This doesn’t make sense since for my images, most of the pixels are the background, and the pixels corresponding to boundaries are the minority. For imbalanced datasets, IOU, Dice, and Focal should work better than binary Cross-Entropy
The code I used for the losses are from https://www.kaggle.com/bigironsphere/loss-function-library-keras-pytorch#Jaccard/Intersection-over-Union-(IoU)-Loss
def DiceLoss(targets, inputs, smooth=1e-6): #flatten label and prediction tensors inputs = K.flatten(inputs) targets = K.flatten(targets) intersection = K.sum(K.dot(targets, inputs)) dice = (2*intersection + smooth) / (K.sum(targets) + K.sum(inputs) + smooth) return 1 - dice ALPHA = 0.8 GAMMA = 2 def FocalLoss(targets, inputs, alpha=ALPHA, gamma=GAMMA): inputs = K.flatten(inputs) targets = K.flatten(targets) BCE = K.binary_crossentropy(targets, inputs) BCE_EXP = K.exp(-BCE) focal_loss = K.mean(alpha * K.pow((1-BCE_EXP), gamma) * BCE) return focal_loss def IoULoss(targets, inputs, smooth=1e-6): #flatten label and prediction tensors inputs = K.flatten(inputs) targets = K.flatten(targets) intersection = K.sum(K.dot(targets, inputs)) total = K.sum(targets) + K.sum(inputs) union = total - intersection IoU = (intersection + smooth) / (union + smooth) return 1 - IoU
Even if I try different code for the losses, such as the code below
smooth = 1. def dice_coef(y_true, y_pred): y_true_f = K.flatten(y_true) y_pred_f = K.flatten(y_pred) intersection = K.sum(y_true_f * y_pred_f) return (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth) def dice_coef_loss(y_true, y_pred): return -dice_coef(y_true, y_pred)
the loss values still don’t improve. That is, it will show something like
loss: nan - dice_coef_loss: .9607 - val_loss: nan - val_dice_coef_loss: .9631
and the values won’t change much for each epoch
can anyone help?
