HMS Alliance/The Submarine Museum is in Gosport rather than Portsmouth.
Walking from PHD isn't an option, best bet is likely to be the Gosport Ferry across the harbour and walk from there (although I can't remember how far it is from the Ferry to the Museum) or drive there as a seperate...