It is all relatively cheap if you shop around!
Domain name is important but reasonably cheap to set up for a couple of years. Try fasthosts or godaddy, both reply to queries quite quickly as do Webfusion.
Hosting packages vary depending on what you are after but again, in the big scheme of...