It can't forget everything, but it can and probably does have an effect on how much attention it gives to those particular tokens.