Hey - this happened to me when I didn't wash it quickly enough when sanding. I found a had to go over it all lightly again with sandpaper to shift it.
My sanding process now is to get one of those spray bottles that you use in the garden, cover the panel I am sanding in water spray. Get some 1200 wet n dry, wet the paper in a bucket of water and wipe on a piece of soap, then sand the panel, wiping of any residue with a microfibre cloth, squirting clean water on the panel if needed. That seems to keep the residue down, stops it setting and cuts down on the amout of water used to rinse the panel and save me kneeling in puddles of water when sanding!
Thankfully I only have the final sand to do on my van now as 4 coats are ont he main shell. Just the doors need a couple more coats to catch up to 4 each and the roof needs doing.