Currently, in test_libsvm, the sign of the dval is flipped, and in the code there's a reference to a FAQ of the libsvm docs.
However, I noticed that in my case (and also if I clone MVPA-Light (master branch) + libsvm (master branch + compile for linux)) the flip should not be executed.
This can be reproduced by the following snippet of code:
%%
tdir = tempdir;
cd(tdir);
system('git -c http.sslVerify=false clone https://github.com/treder/MVPA-Light'); % the -c is needed to allow the system call to work on our compute cluster
system('git -c http.sslVerify=false clone https://github.com/cjlin1/libsvm');
addpath(fullfile(tdir, 'libsvm', 'matlab'));
make;
addpath(fullfile(tdir, 'MVPA-Light', 'startup'));
startup_MVPA_Light;
%%
% create some data
design = [ones(1,100) ones(1,100)+1];
dat = randn(size(design)) + (design-1);
cfg = [];
cfg.classifier = 'libsvm';
cfg.repeat = 1;
cfg.metric = 'accuracy';
cfg.preprocess = {'zscore'};
cfg.feature_dimension = [];
cfg.k = 5;
rng(42);
res_svm1 = mv_classify(cfg, dat(:), design(:));
% does another classifier also work?
cfg.classifier = 'lda';
rng(42);
res_lda1 = mv_classify(cfg, dat(:), design(:));
% and what if I also want to know about the AUC?
cfg.metric = {'accuracy' 'auc'}; % specifying auc as well, results in the lower-level code to use the dval for the accuracy estimation, rather than clabel
rng(42);
res_lda2 = mv_classify(cfg, dat(:), design(:));
cfg.classifier = 'libsvm';
res_svm2 = mv_classify(cfg, dat(:), design(:));
clearly the res_svm2 returns 1-accuracy, and also the incorrect auc.
Currently, in test_libsvm, the sign of the dval is flipped, and in the code there's a reference to a FAQ of the libsvm docs.
However, I noticed that in my case (and also if I clone MVPA-Light (master branch) + libsvm (master branch + compile for linux)) the flip should not be executed.
This can be reproduced by the following snippet of code:
clearly the res_svm2 returns 1-accuracy, and also the incorrect auc.