usb - Certain thumbdrives keep a system from going out of the boot screen

I have an old r51 which I boot off USB. Apparently, if I have a Toshiba 2GB thumbdrive similar to this plugged in:

... or an old imation 1GB drive plugged in, it won't boot at all - it freezes in the BIOS loading screen until it's unplugged. I need to try it on another system but I'm wondering what could be causing it - since I suppose this is at a lower level than filesystem, and the drives work fine otherwise - even being readable if inserted after the system is booted from another source, with neither drive plugged in.