You can't use that excuse.
It's a minimum time, not exact time. It's been said time and time again but some people still can't understand it. The ref can play as much time as he deems fit.
The penalty was given before the 5 mins were up anyway, so it has to be taken (even if there's a 3 hour delay)