Why shouldn’t you pick on a midget with learning difficulties?
... Because it’s not big and it’s not clever.
Alright listen up, smarty! You don't mess with a midget who has learning difficulties. Why? Because, well, it's just not "big" of you to do that, see, because the dude is literally small. Also, it's not "clever" either, since you are picking on someone that has learning difficulties. It's a play on words, genius!