I would guess that with 20 years of project management under your belt, it is always something you could easily pick up again should you be forced to.
It isn't as if taking a few years out would render your skillset obsolete, so I would say you have nothing to lose by trying something different...