I could be mistaken, and if so some native speaker will warn everybody. (Thomas, where are you?). We use both in North America, but seems to me 'trousers' is a male garment and 'pants' either male or female.
There's a question you sometimes hear: "Who wears the trousers in that family, anyway?" You hear it with 'pants' too, but see this explanation: http://www.phrases.org.uk/meanings/403100.html