The government's efforts to speed up housebuilding are sowing division in communities, a Conservative politician has said.