First the pattern:With whom did you go to the movies?
Formal, hence grammatical way, some people would say boring way ha ha ha. Who did you go with to the movies?
(more on the speaking side, so it´s accepted in spoken English.)
And back to the crux, to the initial question:Who did you hang out with yesterday?
(again in informal/spoken English - here could be that you went to a place and socialized with a person or a crowd, it doesn´t matter which.)
And other ways, I´ll leave it to others to complement, improve or correct this answer.