People who say "privilege is why" are using this descriptive sense. They mean in the social context some people have been put in difficultly by an unstated material cause, which is producing a society with this relation of privilege. In making this point, they are only describing someone on difficultly in relation to this social order.
In some cases, when we want to identify a cause, people can make a mistake of only giving this description. Sometimes this happens when people are trying to explain the issue. They'll just say "It's privilege" because they already know associated material caused nested with that outcome. Confusing to those outside, who don't know those associated states and causes, but not wrong.
The biggest issue is a lot of people just don't do description of people in the social context. One of the reasons people get confused by notions of privilege is they relate only in terms of a justification or causal state. They take everything about giving a reason for a state, social organisation or event. Description of an event, a person, how someone is treated, how someone understood, is a rejected catergory of inquiry.
The appeal to intentionally is a great example of this tendency. Supposedly, something will only count as discriminationatory if it's intended. Only if someone is rejected
for being black can there be an issue with racism. Social inquiry gets
reduced to reasons for rather than being descriptive of people in social relations.
If use description, intentionally is only one form of an issue. Various issues are going regardless of intention.
Any material cause which produces difficulty for a social group will manifest a relation with respect to that group.
If economic and cultural situations are producing, for example, a society in which black communities have massive rates of incarceration, then the racial social relation is produced regardless of both intention and whether a response is justified.
Thinking in just terms of reasons or intention just doesn't make sense. It leaves out some of the most aspects of social relations. To do so is like trying to think about poverty only in terms of people who we've already employed.