Embrace your desk.
I do both in my job.
Sites are cold, dirty, dusty, noisy and smelly, there's no TEA, and you have to walk three quarters of a mile to take a shit (if you dare enter the site toilets)
Desks are warm and dry and hot tea is on tap.
No brainer.