A house in Cardiff will be demolished to make way for a footpath linking a quiet cul-de-sac to planned social housing sparking "ghetto" fears.