Small instanton transitions for M5 fractions