"Should of" is grammatically incorrect, regardless of whether the user/speaker is aware of its incorrectness. It's a fact, and a fact per se cannot be misleading. It's as simple as that. Linguistic conventions, as you've illustrated, can be formed over time, but that again doesn't take away from the fact that such usage is grammatically incorrect to begin with.