If you actually do that, you come off as a creep.
Cognitive scientists used video & eye-tracking to figure out what *actually* happens. It's a complicated pattern related to turn-taking.
1. the two people meet each other's eyes only when switching roles (i.e., one person stops talking & the other starts) or when the speaker is emphasizing a point. The difference in the length of pauses.
3. the speaker periodically checks where the listener is looking *without pausing* to gauge attention
(The first book I recommend on this subject is Games People Play, not because of the titular games but because transactional psych gives people a model that lets them avoid seeming needy.)