A good starting point is the Arduino Make Your UNO Kit, providing everything you need to build an Arduino Uno microcontroller from scratch and then use it to build your very own synthesizer.

Arduino’s Plug and Make Kit lets your hacking imagination run wild, sans solder Daisy-chain tiny boards into weather stations, game controllers, and way more.