I would say it´s getting to a conclusion with little information, and a few facts. From what I found on the net, when someone says so, he means something like "I may be mistaken", given the incomplete data, or little knowledge he has on the subject he has to opine.
"You´re really drawing at straws here.'' sounds to me as someone is saying "now you are really clueless, or it seems." Or in an indirect way "you don´t have enough knowledge or information to say that."