My understanding of the term kafir is that it refers to a person who literally rejects God's authority.
So while even the most blatant polytheist would still be mushrik, he would not (necessarily) be kafir.
It's not until the message has been relayed to him and he refuses to accept it that he would be labelled kafir.
However, it seems the common use of the word kafir, at least nowadays, is to refer to anybody who is non-Muslim, regardless of whether or not they're familiar with God's message and His commands.
What is the actual meaning of this term in the primary sources? As in, when the Qur'an and the hadiths refer to the kuffar (or ayuhal kafireen) which interpretation is more correct?