They have a tainted relationship because of both Bakugou's mistreatment of Deku and Deku's own acceptance of this and the way he clings to the old 'Kacchan'. Firstly there was the bullying and constant abuse of Bakugou's quirk, then the reveal of Bakugou's superiority complex later in the series.
While we could never know how Bakugou acts towards Deku in a romantic relationship, it's hard to see how they would come together romantically at all, even with evidence posed by BakuDeku fans. Other characters address this, they can see that Bakugou has been a harmful presence in Deku's life and while they don't exactly tell him to stop saying it, it's obvious that Deku is holding on to what they once had. Thus, he still calls Bakuou 'Kacchan' despite everything that's been done to him. He remembers when they were friends, when Bakugou was the leader and he was the content follower. While this seems like evidence of underlying feelings, it's something more like Deku holding onto what once was. Another thing about BakuDeku is that the basis relies on Midoriya's continuing to want to be Bakugou's friend, both when he is being bullied by him and when Bakugou finally gets to a point where he respects Deku.
