Did you use audio cards? My Japanese listening is pretty good as a result of learning all my vocab through anki with separate image, kanji and audio cards.
I did, but only recently. My listening is improving, but until that Japanese was for me only a written language.
Also, it's easy to create cards from a dictionary, but it is more complicated to find audio resources for specific vocabulary (maybe it is possible to get them from sites like forvo.com).
I wish I had taken a more diversified approach earlier. The first months/years, I almost only used Anki and a grammar book. A better approach maybe would have been to start from simple texts or dialogues.