Not a cycling fanatic, but I have done that ride many times.
On the many windy (south westerly) days, you will be virtually grinding to halt cycling home. So cycle home at least one road inland, for example Western Road/Church Road, and your journey time will be cut massively on those windy...