I share a similar feeling. Remembering last session is easy. Remembering three sessions ago, the one where you got the plot hook and the quest, that gets very difficult. Remembering ten sessions back, outside of an event or awkward mental reconstruction of tone, is insanely difficult.
And ah...