Characterizing the supervisory activity of operators in a dynamic environment from electroencephalographic (EEG) data remains difficult due to the complexity and temporality of the cognitive processes involved. We present our team's work highlighting the problems posed by EEG analyses in this context, and proposals for better characterizing supervisory activity, particularly when supervising the performances of automated systems that require few actions from operators and can generate the out-of-the-loop phenomenon, a source of incidents.