don't crank the volume on your mp3 player up too much, headphone signal is bigger than line signal (I knew the details years ago iirc line is about 1volt and phones up to about 2.5volts), but I used to do this sort of thing all the time and never damaged anything