Not my fave, though I never say never.
I would rather use something like:
"You beat him 4 - 1, you creamed him!
Guy, you defeated him 4 - 1 !
And conversely: guy you was defeated by him 4 - 1 / you was beaten 4 - 1.
I would save "won/lost" to some other sentences e.g. that "you won the championship", you won (earned) 180,000 Dollars when you beat that guy on the final game, etc.
But then, just a matter of personal style/taste here.
Plus, if I say you beat him/you beat Hikaru Nakamura at chess it´s implicit that he is/was your opponent.
Generally you will let the audience/reader know who is your opponent/rival (team). Thus, the sentence with "opponent'' is possible but you can name that guy/team.