What Actually Happens If You Remove A USB Stick Without Ejecting It?

Ever yanked out a USB without ejecting it first? The consequences aren't always instant, but what goes on behind the scenes ...