Pocketsphinx + Gstreamer Race Condition? Pocketsphinx Can't Listen To Audio + Record From It At The Same Time In Python Script?

Overview: So this is a follow up to my last problem (here). I will be posting a full answer on that very soon. I'm able to get pocketsphinx to recognize audio input from my PS3 Eye

Solution 1:

So, as a launch-line, you want something like this:

gst-launch alsasrc device=hw:1 ! queue ! audioconvert ! audioresample ! "audio/x-raw-int, rate=16000, width=16, depth=16, channels=1" ! tee name=t ! queue ! audioresample ! "audio/x-raw-int, rate=8000" ! fvader name=vader auto-threshold=true ! pocketsphinx lm=/home/pi/dev/scarlettPi/config/speech/lm/scarlett.lm dict=/home/pi/dev/scarlettPi/config/speech/dict/scarlett.dic hmm=/usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k name=listener ! fakesink dump=1 t. ! valve drop=0 ! queue ! wavenc ! filesink location=test.wav async=0

The idea is that one alsasrc represents your microphone, and since this can be an exclusive access, you should not start another alsasrc to access the same device. This also makes for better code, since you want to record from the same device you are doing speech-detection from (as I understand), and by using the same source you are guaranteed that this will be the case.

Now, in order to do this in python, you should probably set up the pipeline "properly" rather than doing parse-launch, meaning you have to instantiate each element:

self.recording_valve = gst.element_factory_make('valve')

Add them to your pipeline:

self.pipeline.add (self.recording_valve)

And then link them:

self.recording_valve.link (self.next_element)

...which is what parse-launch does internally, however, in your start and stop recording, you can now do:

self.recording_valve.set_property('drop', False)

To start recording, and

self.recording_valve.set_property('drop', True)

To stop.

(Remember to set drop to True initially, since it is False by default)

The last piece you will need to put all this together is to know about the tee, and its request-pads. It can have any number of src-pads, and you can do the following to link it:

self.tee.get_request_pad('src%d').link(self.recording_valve_queue.get_pad('sink'))

Basically, you request a srcpad from the Tee, and link it to the sinkpad of the valve. You will need to do this twice for each of the tee's branches.

Get that all up and running, and you should have something that works.