I counteract it with Magnesium Glycinate. Works well for me.

Magnesium has a crazy effect on my sleep, very very noticeably deeper.

Not necessarily more restful, but deep and I don't remember anything. With B6 it's deep but more interspersed with vivid dreams.

I take zinc with them too, my own zma stack basically. I'm not sure the zinc has a direct effect on sleep, I take it for testosterone benefits.

These made me realize that legal vitamins and supplements can have much more than a subtle effect.

Magnesium Taurate is the form of magnesium I settled on. I take 1500mg of it (300mg elemental Mg) every evening before sleep, but I feel that even half of this dose has a noticeable effect on my sleep.

Magnesium Glycinate was destroying my sleep even when I took it in the afternoon. I'd wake up after 4-5h of sleep and would feel completely alert as if it was midday and then tiredness would slowly ascend on me over the next few hours but I'd still be unable to fall asleep.