From my experience with learning languages I'd say that's perfectly normal when you have a conversation in a language that you haven't mastered yet. (Though your English seems fine.) The brain simply needs more time to process the input, whether it's because you're not familiar with a word, or haven't yet gotten used to the speaker's pronunciation or their intonation/melody.