Regarding the data of the study (arxiv.org/pdf/1811.08592…): it refers to this (groups.csail.mit.edu/sls/publicatio…) which refers to this (lrec-conf.org/proceedings/lr… - original data) and this (arxiv.org/pdf/1605.01600…). Original data can be obtained here ...
(dcapswoz.ict.usc.edu) and contains 28 persons (!) marked as depressive in 142 (about 2 times the ratio of depressive patients in Germany). Data set was divided in training/test set for some competition (arxiv.org/pdf/1605.01600…) with 107/35 persons, respectively.
To maintain incidence, likely 21 depressive patients were in the training set and 7 (!) in the test set (called "development set"). Thus, a recall of 80% means they found 5-6 of the 7 and a precision of 71% means they labeled 7 to 8 as depressive of which 2 were not depressive.
So, of 33 non-depressive patients they labeled 2 false positively as depressive. In Germany, if the recall is stable, this would mean they would falsely label 4 for finding correctly 5-6.
That is not too bad (I've seen worse), but it is based on a tiny, (thus biased) and peculiar data set. Moreover, the peculiarity of the data set was lost in the reference chain. We definitely need better data bases to build our machine learning on!
@threadreaderapp please unroll!
