It's a really odd mechanic that only one person is involved in the conversation. Are the rest of the party just stood back twiddling their thumbs? Very rarely do people converse like that in real life.
It really does diminish the power of tailoring your party to specialties.
If the guy I'm talking to starts being a dick why can't I choose to have my badass character intimidate him? If they're nice why can't I use my charismatic character? Should be able to choose who responds to what whenever you like, it would make dialogue with NPCs so much more involved.
I read somewhere that you can shove companions out of dialogue. I only tried it once but my scrawny warlock couldn't budge Karlach, so I can't confirm.