To invest in a buy-to-let you have to be a particular type of outrageous ****.
Houses are unusual commodities: everyone needs one, whether they rent or buy. By adding to demand for houses (which you don't need if you already own a home) you drive the price up for those who are trying to buy...