That annoys him in a way that only she can anymore. He didn't say get out of the way, he just gave her ideas as to how to keep her shoes dry. Honestly he doesn't mind her being around, he likes it, she's a familiar face that he at least knows won't judge him to awful badly for his life and his life choices. So he makes a face again, he grunts.

"If you take recommendations as to how to keep your shoes dry as a way of seeing 'Please leave me alone and get out of the way there's serious work being done and you aren't allowed to see it.' Then fine. Go ahead." He even says it with hand gestures, shooing her away. "That's not what I said at all, and if you would perhaps, care to stay here and talk then I could perhaps help you with your work."