Does it all come under "permitted development rights" ? The council won't give a toss because planning permission isn't needed when the guidelines are adhered to.
If you ever have to sell your property make sure it is described as link detached rather than a semi. Your neighbour has got away with it because they knew it could cost £50k to take it to court for a ruling.