They don’t want you sitting there. They’re hoping you don’t ask them. Your interpretation that it’s hostile is correct, since it’s antisocial behavior.