Are magnets dangerous for lead poisoning in a toddler? Or is it just when they swallow one?

I have never heard of a magnet made of lead. Usually they are made of Iron.

Talk to your Doctor, but I think the only thing you have to worry about is your child developing a tendancy to point North.