It may come with time. Or perhaps you need a teacher, an advanced teacher or some study (read listening) material that fits your level (and your pace as well).
Perhaps some reading, textbooks, training etc would help you, either. With time one will know, almost automatically, that the structure is "I am a lover, not a hater." and this passes onto the listening, say, you listen to a record and you "fill in" words, because you are accostumed to see some expressions.
I hope you make headway, and am equally interested on other testimonies from learners that faced this hurdle before.
I bet there are many interesting stories out there. For instance, from students that went abroad to learn English and from students in Brazil, either. I know listening sometimes is taxing, but as for everything as well, practice makes perfect.
I am far from perfect on that skill, but we will get there!