One more to table:
...e não tão mal assim, não tão mal quanto eu pensei.
...e não tão mal quanto se poderia pensar/imaginar.
In fact, I have translated by the "general sense" of it, since in a more "plain English" I think that "wear" could mean "abrasion/scratches, etc" or "from his trouble" (it makes sense as well, as per Thesaurus and my guess work here.)
Wear has several meanings if you search some Thesaurus dictionary, one of them is abrasion. So, I think Marcio nailed it.