I wondered the same. Seems like it would just make a V-shaped loss around the zero, but abs has that property already!
RELU would have made it flat below zero ( _/ not \/). Adding the abs first just makes RELU do nothing.
RELU would have made it flat below zero ( _/ not \/). Adding the abs first just makes RELU do nothing.