I wasn't looking at you, your big forehead was distracting me.
Distracting Forehead
Explanation
Explain Bear
Listen here, smart guy. The joke is that the person's forehead is so big that it's hard not to look at it. It is a distraction! People with big foreheads are rare, which is why you are dumb.