I was only changing the state of the order. That was not the correct way to do cancellation. I have found out an elegant way to do that here.

I was only changing the state of the order. That was not the correct way to do cancellation. I have found out an elegant way to do that here.

The relevant code from this tutorial is:

if(!$order->canCancel()) {
    throw new Exception('Order cannot be canceled anymore.');
}
$order->cancel();
$order->save();