Ear mites or a tick down in there! Go to TSC and get you some ear mite treatment. You will have to look real good for ticks too. Is the ear swollen?
No its not swollen that i can see. One thing i forgot to add was about 3 yrs ago, he got that ear cut in half by a boar, could it be that it hurts him and he'll just always have to deal with it?