I had this problem in my previous flat. No windows in the bathroom and I believe the extracter-fan was actually just a noise making fan.
Actually discovered a two inch toadstall growing out the side of my bath panel one evening!
Was for ever wiping the mould off ceilings and walls. I painted...