Model-Based EM Source Separation and Localization Wrappers

This repo contains various wrappers and drivers for running MESSL experiments for the 2015 Jelenik Speech and Language Technologies workshop and for the OSU submission to CHiME3.

The actual MESSL code is in the main MESSL repo.

Examples of running this on OSU's cluster

Step 1: Run MESSL

enhance_wrapper(@(X, fail, fs, file) stubI_messlMc(X, fail, fs, file, 1, 0, 0.12, 1, 'bestMic', ...
  'mrfCompatExpSched', [0 0 0 0 0 0 0 0 .2], 'mrfHardCompatExp', 5 , ...
  'mrfCompatFile', 'messl/ibmNeighborCountsSimple.mat', 'mrfLbpIter', 4), ...
  '/data/corpora/chime3/simulatedMoreMovement/isolated_MESSL', ...
  '~/data1/chime3/out/moreMovement/messlMcMvdrMrf.2Hard5Lbp4', [1 1], 0, 0, 1);

Step 2a: Replay MESSL, version 1: Mask-based noise estimate, MESSL IPD-based steering vector

enhance_wrapper(@(X, fail, fs, file) stubI_replayMessl(X, fail, fs, file, ...
  '~/data1/chime3/out/messlMcMvdrMrf.2Hard5Lbp4Slate/data/', '', 'souden', 'ipd', 'mask', 1, 9), ...
  '/data/corpora/chime3/CHiME3/data/audio/16kHz/isolated/', ...
  '~/data1/chime3/out/replaySlateXcMaskIpd', [1 1], 0, 1, 1);

Step 2b: Replay MESSL, version 2: Mask-based noise estimate, Souden-based "steering vector"

enhance_wrapper(@(X, fail, fs, file) stubI_replayMessl(X, fail, fs, file, ...
  '~/data1/chime3/out/messlMcMvdrMrf.2Hard5Lbp4Slate/data/', '', 'souden', '', 'mask', 1, 9), ...
  '/data/corpora/chime3/CHiME3/data/audio/16kHz/isolated/', ...
  '~/data1/chime3/out/replaySlateXcMaskSouden', [1 1], 0, 1, 1);

Calculate big delay between ch0 and other channels for real dev and test data

cd /homes/3/mandelm/code/matlab/messlJsalt15
enhance_wrapper(@stubI_justXcorr, '/data/corpora/chime3/CHiME3/data/audio/16kHz/isolated/', ...
  '~/data1/chime3/xcorr', [1 1], 0, 0, 2, '[de]t05.*real.*\.CH1\.wav$');

Fix big delay between ch0 and other channels for real dev and test data

conds = {'dt05_bus', 'dt05_caf', 'dt05_ped', 'dt05_str', ...
  'et05_bus', 'et05_caf', 'et05_ped', 'et05_str'}; 
for c=1:length(conds), 
  correctCh0Delay(fullfile('/data/corpora/chime3/CHiME3/data/audio/16kHz/isolated',[conds{c} '_real']), ...
    fullfile('~/data1/chime3/xcorr/data', [conds{c} '_real'])); 
end

Run supervised MVDR using channel 0

enhance_wrapper(@(X, fail, fs, file) stubI_supervisedMvdr(X, fail, fs, file, 0.75, 0.85, 15), ...
  '/data/corpora/chime3/CHiME3/data/audio/16kHz/isolated/', ...
  '~/data1/chime3/out/sup75m85a15db/', [1 1], 0, 0, 2, '[de]t05.*real.*\.CH1\.wav$');

Name		Name	Last commit message	Last commit date
Latest commit History 209 Commits
PEASS_Toolkit_v1.0		PEASS_Toolkit_v1.0
PEASS_Toolkit_v2.0		PEASS_Toolkit_v2.0
kmedoids		kmedoids
messlkeras		messlkeras
pesq		pesq
.gitignore		.gitignore
CHiME3_computeStats.m		CHiME3_computeStats.m
CHiME3_enhance_wrapper.m		CHiME3_enhance_wrapper.m
Generate_mcTrainDataMix2.m		Generate_mcTrainDataMix2.m
Generate_reverbTestMix2.m		Generate_reverbTestMix2.m
OPS_evaluation.m		OPS_evaluation.m
README.md		README.md
analyzeKaldiRes.m		analyzeKaldiRes.m
applyMasksToSeps.m		applyMasksToSeps.m
beamSpaceCluster.m		beamSpaceCluster.m
beamSpaceProject.m		beamSpaceProject.m
bss_eval_sources_nosort.m		bss_eval_sources_nosort.m
c3_predict_mask.py		c3_predict_mask.py
combination2_step1.m		combination2_step1.m
combination2_step2.m		combination2_step2.m
combination3_step1.m		combination3_step1.m
combination3_step2.m		combination3_step2.m
combination4_step1.m		combination4_step1.m
combination4_step2.m		combination4_step2.m
correctCh0Delay.m		correctCh0Delay.m
covsFromIpdParams.m		covsFromIpdParams.m
create_clean_spectrogram.m		create_clean_spectrogram.m
create_masks.m		create_masks.m
create_messl_soft_masks.m		create_messl_soft_masks.m
create_messl_soft_masks.sh		create_messl_soft_masks.sh
create_spectrogram.m		create_spectrogram.m
deepClusteringResynthesis.m		deepClusteringResynthesis.m
enhance_wrapper.m		enhance_wrapper.m
evalMerl.m		evalMerl.m
evaluate_keras_model.sh		evaluate_keras_model.sh
extractMasksFromSeps.m		extractMasksFromSeps.m
generate_audio_from_masks.m		generate_audio_from_masks.m
istft_multi.m		istft_multi.m
keras_exp-mask_cleaner.py		keras_exp-mask_cleaner.py
keras_exp-nspect2mask.py		keras_exp-nspect2mask.py
keras_exp_evaluation.py		keras_exp_evaluation.py
maskDrivenMvdr.m		maskDrivenMvdr.m
maskDrivenMvdrMulti.m		maskDrivenMvdrMulti.m
mixTwoSeps.m		mixTwoSeps.m
mungeMaskForMvdr.m		mungeMaskForMvdr.m
mvdrHendriksLoop.m		mvdrHendriksLoop.m
mvdrHendriksVectorized.m		mvdrHendriksVectorized.m
mvdrLSTM.m		mvdrLSTM.m
mvdrLSTMMulti.m		mvdrLSTMMulti.m
mvdrSouden.m		mvdrSouden.m
mvdrSoudenMulti.m		mvdrSoudenMulti.m
normKernXcorr.m		normKernXcorr.m
pickChanWithBestSnr.m		pickChanWithBestSnr.m
plotMultichanIldIpdStats.m		plotMultichanIldIpdStats.m
plotsJsalt20150710.m		plotsJsalt20150710.m
plotsSpringer16.m		plotsSpringer16.m
plotsWiley16.m		plotsWiley16.m
process_mvdr.m		process_mvdr.m
qsubMatlab.sh		qsubMatlab.sh
queue.pl		queue.pl
sbatchTest.sh		sbatchTest.sh
sepKernXcorr.m		sepKernXcorr.m
simMixMerl.m		simMixMerl.m
slurm.pl		slurm.pl
sph2wav.m		sph2wav.m
srunMatlab.sh		srunMatlab.sh
srunTest.sh		srunTest.sh
srunTest2.sh		srunTest2.sh
srunTest3.sh		srunTest3.sh
stft_multi.m		stft_multi.m
stubI_Clean.m		stubI_Clean.m
stubI_CleanSpectragram.m		stubI_CleanSpectragram.m
stubI_LSTMMessl3.m		stubI_LSTMMessl3.m
stubI_Masks.m		stubI_Masks.m
stubI_MesslKeras.m		stubI_MesslKeras.m
stubI_Spectrogram.m		stubI_Spectrogram.m
stubI_deepClusteringFeats.m		stubI_deepClusteringFeats.m
stubI_justXcorr.m		stubI_justXcorr.m
stubI_messlMc.m		stubI_messlMc.m
stubI_messlMcIld.m		stubI_messlMcIld.m
stubI_pickChan.m		stubI_pickChan.m
stubI_recoverFromLstm.m		stubI_recoverFromLstm.m
stubI_replayMessl.m		stubI_replayMessl.m
stubI_souden11.m		stubI_souden11.m
stubI_supervisedMvdr.m		stubI_supervisedMvdr.m
stub_baselineMvdr.m		stub_baselineMvdr.m
stub_messlMcIld2.m		stub_messlMcIld2.m
stub_messlMcIld2Hard.m		stub_messlMcIld2Hard.m
stub_messlMcTdoa.m		stub_messlMcTdoa.m
stub_messlMcTdoaMrf.m		stub_messlMcTdoaMrf.m
test_mask.py		test_mask.py
tfKernel.m		tfKernel.m
trimMerlRirs.m		trimMerlRirs.m
visualizeAffinity.m		visualizeAffinity.m

speechLabBcCuny/messlJsalt15

Folders and files

Latest commit

History