O verbo no infinitivo sem o uso de "to" é o que se chama de "bare infinitive" (pois é, o infinitivo "pelado"), e ocorre em certos casos. Praticamente nenhum falante saberia descrever como e quando isso ocorre, por ser coisa que se aprende convivendo com o idioma. Mas a descrição do fenomeno é esta:
The bare infinitive is not used in as many contexts as the full infinitive, but some of these are quite common:
The bare infinitive is used as the main verb after the dummy auxiliary verb do, or most modal auxiliary verbs (such as will, can, or should). So, "I will/do/can/etc. see it."
Several common verbs of perception, including see, watch, hear, feel, and sense take a direct object and a bare infinitive, where the bare infinitive indicates an action taken by the main verb's direct object. So, "I saw/watched/heard/etc. it happen." (A similar meaning can be effected by using the present participle instead: "I saw/watched/heard/etc. it happening." The difference is that the former implies that the entirety of the event was perceived, while the latter implies that part of the progress of the event was perceived.)
Similarly with several common verbs of permission or causation, including make, bid, let, and have. So, "I made/bade/let/had him do it." (However, make takes a to-infinitive in the passive voice: "I was made to do it.")
After the had better expression. So, "You had better leave now."
With the verb help. So, "He helped them find it." (The use of the to-infinitive with the verb help is also common.)
With the word why. So, "Why reveal it?" (Use of the to-infinitive following why is also common.)
The bare infinitive is the dictionary form of a verb, and is generally the form of a verb that receives a definition; however, the definition itself generally uses a to-infinitive. So, "The word 'amble' means 'to walk slowly.'"
The bare infinitive form coincides with the present subjunctive form as well as the imperative form, but most grammarians do not consider uses of the present subjunctive or imperative to be uses of the bare infinitive."
Felipe, o meu conselho é deixar essas "regras" de lado e procurar se acostumar com as modalidades através da prática. Mas fica ai como referência.