On Linux and other POSIX systems (such as OS X), follow the instructions under "Building PocketSphinx-Python from source" in Notes on using PocketSphinx for installation instructions. Note that the ...

My working microphones are: (You can check this by running python -c "import speech_recognition as sr;print(sr.Microphone.list_working_microphones())".) I installed PocketSphinx from PyPI.