Your statement, worded differently, seems to read "it's high class in low class circles". Are you implying class is only appropriately defined only by those in high-class? I guess I was looking at it as social prestige regardless of who in the social strata assigns it. In other words, if enough 'low-class' people think you're high-class, you're by definition high-class.
Class is defined by the members of each group. Non-members are by definition mostly ignorant of the groups' culture that they are not a part of. So they can recognize people are different from them, but not which group they belong to.
"low-class" people would not determine who is "high-class" any more than the "high-class" people determine who is "low-class".