When on speaker phone and another call comes in, I lose the current call when I hit reject

Question

When ever I am using the speaker phone and another call comes in. I try to hit "Reject Call" and then the call I am on speaker with disconnects. I Cant let the other call just ring because it then goes over the speakers and I cant hear the speak phone call I'm on.