Pitch predictors¶
NeuraSpeech also allows you to generate pitch/F0 predictors from the stimuli files.
Making pitch predictors¶
The make_pitch_predictors() function in the trf module extracts the pitch contours from the stimuli and convert them into the predictor file format for TRF modeling. Pitch extraction is done using the pitch_ac function from the Parselmouth Python interface to Praat. For details, see pitch_ac in the API reference.
import neuraspeech as ns
# Pitch extraction settings
pitch_cfgs = {
'time_step': 0.005, # Time step in seconds between pitch samples
'pitch_floor': 75, # Minimum pitch frequency in Hz
'pitch_ceiling': 300 # Maximum pitch frequency in Hz
}
ns.trf.make_pitch_predictors(
eeg_sr = 128,
stim_path = 'stim/alice',
predictor_set = 'acou_pred_set',
predictor_name = 'pitch',
pitch_cfgs = pitch_cfgs,
scale = 'mel'
)
Generating pitch predictors using stimuli in: C:\Users\xyc6648\OneDrive - Northwestern University\Desktop\scripts\neuraspeech\neuraspeech\trf\stim\alice Output path: C:\Users\xyc6648\OneDrive - Northwestern University\Desktop\scripts\neuraspeech\neuraspeech\trf\predictors\acou_pred_set\pitch Processing track1.wav... Processing track10.wav... Processing track11.wav... Processing track12.wav... Processing track13.wav... Processing track14.wav... Processing track15.wav... Processing track16.wav... Processing track17.wav... Processing track18.wav... Processing track19.wav... Processing track2.wav... Processing track20.wav... Processing track21.wav... Processing track22.wav... Processing track23.wav... Processing track24.wav... Processing track25.wav... Processing track26.wav... Processing track27.wav... Processing track28.wav... Processing track29.wav... Processing track3.wav... Processing track30.wav... Processing track31.wav... Processing track32.wav... Processing track33.wav... Processing track34.wav... Processing track35.wav... Processing track36.wav... Processing track37.wav... Processing track38.wav... Processing track39.wav... Processing track4.wav... Processing track40.wav... Processing track41.wav... Processing track42.wav... Processing track43.wav... Processing track44.wav... Processing track45.wav... Processing track46.wav... Processing track47.wav... Processing track48.wav... Processing track49.wav... Processing track5.wav... Processing track50.wav... Processing track51.wav... Processing track52.wav... Processing track53.wav... Processing track54.wav... Processing track55.wav... Processing track56.wav... Processing track57.wav... Processing track58.wav... Processing track59.wav... Processing track6.wav... Processing track60.wav... Processing track7.wav... Processing track8.wav... Processing track9.wav... All done.
Key arguments you may want to change or check:
predictor_name: Name of the predictor to generate. In this example, we usepitch.pitch_cfgs: Dictionary containing the pitch extraction parameters. The values here should work well for the current stimuli (the Alice in Wonderland audio tracks). However, you may need to adjustpitch_floorandpitch_ceilingdepending on the speaker. Settingpitch_ceilingtoo high may produce occasional pitch estimates outside the normal human vocal range because of algorithmic estimation errors. These values are usually not a major issue if they are infrequent, but it is still best to set the pitch range to a reasonable range for your stimuli. Thetime_stepparameter controls the interval between consecutive pitch samples. I recommend setting it to a value smaller than the period of your EEG sampling frequency.
scale: Scale of the pitch values. The default is linear. Other options include mel, bark, and erb. See the Praat formulas page for the conversion formulas.
The phoneme predictors will be named like track1~pitch-mel.pickle. The -mel suffix is the scale of the pitch values.
Again, you can do a plot to visualize the pitch predictors:
eeg_path = 'examples/sub-3000_04_ses-1_task-alice.pickle'
stim_path = '../../neuraspeech/trf/stim/alice'
predictor_set_path = '../../neuraspeech/trf/predictors/acou_pred_set'
stim_to_plot = 'track1.wav'
ns.trf.plot_schematic(
to_plot = ['eeg', 'waveform', 'gammatone-8', 'pitch-mel'],
colors = ['Spectral', None, 'inferno', '#D0495B'],
height_ratios = [1, 1, 1, 1.5],
linewidths = [1, None, None, 3.0],
zeros_as_nans = [False, False, False, True], # Treat zeros as NaNs for the pitch-mel plot
figsize = (10, 6),
eeg_path = eeg_path,
stim_to_plot = stim_to_plot,
time_window = (0, 3.0),
stim_path = stim_path,
predictor_set_path = predictor_set_path
)
Then in your TRF analysis, you can specficy an acoustic model like {'acou_pitch': [gammatone-8, gammatone-on-8, pitch-mel]} to include pitch and spectrotemporal representations as predictors. Note that if you set duration_mode = 'concat' during EEG data preparation, you'll need to run the EEG data preparation function again to concatenate the newly genearated predictors.