Fairly x quite x rather x pretty

An useful tip to define which of them to use in a sentence:

Using nice as an example, imagine a line going from not nice to very nice:

Not - fairly - quite - rather/pretty - very
_
Good - good - good - good - good

Note:
Only rather can be used with comparatives. You can say "rather hotter", but you can't say "fairly hotter".
Still talking about these ones ,only rather can be used with "too".
Example: She's rather too sure of herself.
But you can also use "too" with a little, a bit(informal),a lot, much or far.
In America English, quite isn't very often used to change the meaning of of adjectives or adverbs.
Quite has another meaning when used with certain words,in both British and America English. It mean can completely or absolutely , for example: quite finished; quite alone; quite impossible. So, the meaning of quite depends on the kind of word it is used with.
Both quite and rather can be used before an article, and with a verb, however, pretty and fairly can't be used in the same way,
Example: quiet a nice morning; I rather like them.
Pretty well/much means almost ( One dog looks pretty much like another to me;I pretty well agree with you).

Fonte: Oxford Practical English Usage.