Machine learning of electroencephalography signals and eye movements to classify work-in-progress risk at construction sites
Abstract
The construction industry has consistently faced high accident rates and delays in recognizing hazards, posing significant risks to onsite personnel. Traditional hazard detection methods are often reactive rather than proactive, emphasizing a pressing need for innovative solutions. Despite advances in safety technology, a considerable gap remains in real-time, accurate hazard recognition at construction sites. Current technologies do not fully leverage physiological data to predict and mitigate risks. This research introduces a groundbreaking approach by employing machine learning to analyze electroencephalography (EEG) signals and eye movement data, enabling real-time differentiation of safe, warning, and hazardous visual cues. A Random Forest model with an impressive classification accuracy of 99.04% has been developed, marking a significant enhancement in identifying potential hazards. The possible impact of integrating EEG and eye movement analyses into wearable devices or onsite sensors is substantial, as it could revolutionize safety protocols in the construction industry, fostering a safer future.
First published online 31 December 2024
Keyword : construction safety, brain-computer interface, electroencephalography (EEG), eye movement, machine learning, construction site hazard recognition
How to Cite
Chou, J.-S., Liao, P.-C., Liu, C.-Y., & Hou, C.-Y. (2024). Machine learning of electroencephalography signals and eye movements to classify work-in-progress risk at construction sites. Journal of Civil Engineering and Management, 1-16. https://doi.org/10.3846/jcem.2024.22719
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Behzadnia, A., Ghoshuni, M., & Chermahini, S. A. (2017). EEG activities and the sustained attention performance. Neurophysiology, 49(3), 226–233. https://doi.org/10.1007/s11062-017-9675-1
Bui, T., & Das, J. M. (2022). Neuroanatomy, Cerebral hemisphere. StatPearls Publishing.
Cheng, B., Fan, C., Fu, H., Huang, J., Chen, H., & Luo, X. (2022). Measuring and computing cognitive statuses of construction workers based on electroencephalogram: A critical review. IEEE Transactions on Computational Social Systems, 9(6), 1644–1659. https://doi.org/10.1109/TCSS.2022.3158585
Garrett, J. W., & Teizer, J. (2009). Human factors analysis classification system relating to human error awareness taxonomy in construction safety. Journal of Construction Engineering and Management, 135(8), 754–763. https://doi.org/doi:10.1061/(ASCE)CO.1943-7862.0000034
Gong, P., Guo, H., Yuanyue, H., & Guo, S. (2020). Safety risk evaluations of deep foundation construction schemes based on imbalanced data sets. Journal of Civil Engineering and Management, 26(4), 380–395. https://doi.org/10.3846/jcem.2020.12321
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009). The WEKA data mining software: An update. ACM SIGKDD Explorations Newsletter, 11(1), 10–18. https://doi.org/10.1145/1656274.1656278
Hosmer Jr, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (3rd ed.). John Wiley & Sons. https://doi.org/10.1002/9781118548387
Huang, C. X., Shi, N. L., Miao, Y. N., Chen, X. G., Wang, Y. J., & Gao, X. R. (2024). Visual tracking brain-computer interface. Iscience, 27(4), Article 109376. https://doi.org/10.1016/j.isci.2024.109376
İnce, R., Adanır, S. S., & Sevmez, F. (2021). The inventor of electroencephalography (EEG): Hans Berger (1873–1941). Child’s Nervous System, 37, 2723–2724. https://doi.org/10.1007/s00381-020-04564-z
Klem, G. H., LuÈders, H. O., Jasper, H. H., & Elger, C. (1961). The ten twenty electrode system: International Federation of Societies for Electroencephalography and Clinical Neurophysiology. American Journal of EEG Technology, 1(1), 13–19. https://doi.org/10.1080/00029238.1961.11080571
Occupational Safety and Health Administration. (2020). 109 Labor inspection statistics annual report. Ministry of Labor, Taiwan.
Larsen, O. F. P., Tresselt, W. G., Lorenz, E. A., Holt, T., Sandstrak, G., Hansen, T. I., Su, X. M., & Holt, A. (2024). A method for synchronized use of EEG and eye tracking in fully immersive VR. Frontiers in Human Neuroscience, 18. https://doi.org/10.3389/fnhum.2024.1347974
Liao, P.-C., Zhou, X., Chong, H.-Y., Yinan, H., & Zhang, D. (2022). Exploring construction workers’ brain connectivity during hazard recognition: A cognitive psychology perspective. International Journal of Occupational Safety and Ergonomics, 29(1), 207–2015. https://doi.org/10.1080/10803548.2022.2035966
Liu, X., Hu, B. L., Si, Y., & Wang, Q. (2024). The role of eye movement signals in non-invasive brain-computer interface typing system. Medical & Biological Engineering & Computing, 62, 1981–1990. https://doi.org/10.1007/s11517-024-03070-7
Liu, Y., Chen, H., & Wang, X. (2021). Risk prediction and diagnosis of water seepage in operational shield tunnels based on random forest. Journal of Civil Engineering and Management, 27(5), 539–552. https://doi.org/10.3846/jcem.2021.14901
McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5(4), 115–133. https://doi.org/10.1007/BF02478259
Mohamed, W. N. H. W., Salleh, M. N. M., & Omar, A. H. (2012). A comparative study of Reduced Error Pruning method in decision tree algorithms. In 2012 IEEE International Conference on Control System, Computing and Engineering, Penang, Malaysia. https://doi.org/10.1109/ICCSCE.2012.6487177
Moore II, D. H. (1987). Classification and regression trees, by Leo Breiman, Jerome H. Friedman, Richard A. Olshen, and Charles J. Stone. Brooks/Cole Publishing, Monterey, 1984, 358 pages, $27.95. Cytometry, 8(5), 534–535. https://doi.org/10.1002/cyto.990080516
Noghabaei, M., Han, K., & Albert, A. (2021). Feasibility study to identify brain activity and eye-tracking features for assessing hazard recognition using consumer-grade wearables in an immersive virtual environment. Construction Engineering and Management, 147(9), Article 04021104. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002130
Rosen, K. H., & Krithivasan, K. (2012). Discrete mathematics and its applications: with combinatorics and graph theory: Tata. McGraw-Hill Education.
Saedi, S., Fini, A. A. F., Khanzadi, M., Wong, J., Sheikhkhoshkar, M., & Banaei, M. (2022). Applications of electroencephalography in construction. Automation in Construction, 133, Article 103985. https://doi.org/10.1016/j.autcon.2021.103985
Saghafi, A., Tsokos, C. P., Goudarzi, M., & Farhidzadeh, H. (2017). Random eye state change detection in real-time using EEG signals. Expert Systems with Applications, 72, 42–48. https://doi.org/10.1016/j.eswa.2016.12.010
Smola, A., & Vapnik, V. (1997). Support vector regression machines. In NIPS’96: Proceedings of the 9th International Conference on Neural Information Processing Systems (pp. 155–161), Denver, CO, USA. https://dl.acm.org/doi/10.5555/2998981.2999003
Vecchiato, G., Toppi, J., Astolfi, L., De Vico Fallani, F., Cincotti, F., Mattia, D., Bez, F., & Babiloni, F. (2011). Spectral EEG frontal asymmetries correlate with the experienced pleasantness of TV commercial advertisements. Medical & Biological Engineering & Computing, 49(5), 579–583. https://doi.org/10.1007/s11517-011-0747-x.
Vortmann, L.-M., Ceh, S., & Putze, F. (2022). Multimodal EEG and eye tracking feature fusion approaches for attention classification in hybrid BCIs. Frontiers in Computer Science, 4. https://doi.org/10.3389/fcomp.2022.780580
Wei, C.-C. (2021). Collapse warning system using LSTM neural networks for construction disaster prevention in extreme wind weather. Journal of Civil Engineering and Management, 27(4), 230–245. https://doi.org/10.3846/jcem.2021.14649
Bui, T., & Das, J. M. (2022). Neuroanatomy, Cerebral hemisphere. StatPearls Publishing.
Cheng, B., Fan, C., Fu, H., Huang, J., Chen, H., & Luo, X. (2022). Measuring and computing cognitive statuses of construction workers based on electroencephalogram: A critical review. IEEE Transactions on Computational Social Systems, 9(6), 1644–1659. https://doi.org/10.1109/TCSS.2022.3158585
Garrett, J. W., & Teizer, J. (2009). Human factors analysis classification system relating to human error awareness taxonomy in construction safety. Journal of Construction Engineering and Management, 135(8), 754–763. https://doi.org/doi:10.1061/(ASCE)CO.1943-7862.0000034
Gong, P., Guo, H., Yuanyue, H., & Guo, S. (2020). Safety risk evaluations of deep foundation construction schemes based on imbalanced data sets. Journal of Civil Engineering and Management, 26(4), 380–395. https://doi.org/10.3846/jcem.2020.12321
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009). The WEKA data mining software: An update. ACM SIGKDD Explorations Newsletter, 11(1), 10–18. https://doi.org/10.1145/1656274.1656278
Hosmer Jr, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (3rd ed.). John Wiley & Sons. https://doi.org/10.1002/9781118548387
Huang, C. X., Shi, N. L., Miao, Y. N., Chen, X. G., Wang, Y. J., & Gao, X. R. (2024). Visual tracking brain-computer interface. Iscience, 27(4), Article 109376. https://doi.org/10.1016/j.isci.2024.109376
İnce, R., Adanır, S. S., & Sevmez, F. (2021). The inventor of electroencephalography (EEG): Hans Berger (1873–1941). Child’s Nervous System, 37, 2723–2724. https://doi.org/10.1007/s00381-020-04564-z
Klem, G. H., LuÈders, H. O., Jasper, H. H., & Elger, C. (1961). The ten twenty electrode system: International Federation of Societies for Electroencephalography and Clinical Neurophysiology. American Journal of EEG Technology, 1(1), 13–19. https://doi.org/10.1080/00029238.1961.11080571
Occupational Safety and Health Administration. (2020). 109 Labor inspection statistics annual report. Ministry of Labor, Taiwan.
Larsen, O. F. P., Tresselt, W. G., Lorenz, E. A., Holt, T., Sandstrak, G., Hansen, T. I., Su, X. M., & Holt, A. (2024). A method for synchronized use of EEG and eye tracking in fully immersive VR. Frontiers in Human Neuroscience, 18. https://doi.org/10.3389/fnhum.2024.1347974
Liao, P.-C., Zhou, X., Chong, H.-Y., Yinan, H., & Zhang, D. (2022). Exploring construction workers’ brain connectivity during hazard recognition: A cognitive psychology perspective. International Journal of Occupational Safety and Ergonomics, 29(1), 207–2015. https://doi.org/10.1080/10803548.2022.2035966
Liu, X., Hu, B. L., Si, Y., & Wang, Q. (2024). The role of eye movement signals in non-invasive brain-computer interface typing system. Medical & Biological Engineering & Computing, 62, 1981–1990. https://doi.org/10.1007/s11517-024-03070-7
Liu, Y., Chen, H., & Wang, X. (2021). Risk prediction and diagnosis of water seepage in operational shield tunnels based on random forest. Journal of Civil Engineering and Management, 27(5), 539–552. https://doi.org/10.3846/jcem.2021.14901
McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5(4), 115–133. https://doi.org/10.1007/BF02478259
Mohamed, W. N. H. W., Salleh, M. N. M., & Omar, A. H. (2012). A comparative study of Reduced Error Pruning method in decision tree algorithms. In 2012 IEEE International Conference on Control System, Computing and Engineering, Penang, Malaysia. https://doi.org/10.1109/ICCSCE.2012.6487177
Moore II, D. H. (1987). Classification and regression trees, by Leo Breiman, Jerome H. Friedman, Richard A. Olshen, and Charles J. Stone. Brooks/Cole Publishing, Monterey, 1984, 358 pages, $27.95. Cytometry, 8(5), 534–535. https://doi.org/10.1002/cyto.990080516
Noghabaei, M., Han, K., & Albert, A. (2021). Feasibility study to identify brain activity and eye-tracking features for assessing hazard recognition using consumer-grade wearables in an immersive virtual environment. Construction Engineering and Management, 147(9), Article 04021104. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002130
Rosen, K. H., & Krithivasan, K. (2012). Discrete mathematics and its applications: with combinatorics and graph theory: Tata. McGraw-Hill Education.
Saedi, S., Fini, A. A. F., Khanzadi, M., Wong, J., Sheikhkhoshkar, M., & Banaei, M. (2022). Applications of electroencephalography in construction. Automation in Construction, 133, Article 103985. https://doi.org/10.1016/j.autcon.2021.103985
Saghafi, A., Tsokos, C. P., Goudarzi, M., & Farhidzadeh, H. (2017). Random eye state change detection in real-time using EEG signals. Expert Systems with Applications, 72, 42–48. https://doi.org/10.1016/j.eswa.2016.12.010
Smola, A., & Vapnik, V. (1997). Support vector regression machines. In NIPS’96: Proceedings of the 9th International Conference on Neural Information Processing Systems (pp. 155–161), Denver, CO, USA. https://dl.acm.org/doi/10.5555/2998981.2999003
Vecchiato, G., Toppi, J., Astolfi, L., De Vico Fallani, F., Cincotti, F., Mattia, D., Bez, F., & Babiloni, F. (2011). Spectral EEG frontal asymmetries correlate with the experienced pleasantness of TV commercial advertisements. Medical & Biological Engineering & Computing, 49(5), 579–583. https://doi.org/10.1007/s11517-011-0747-x.
Vortmann, L.-M., Ceh, S., & Putze, F. (2022). Multimodal EEG and eye tracking feature fusion approaches for attention classification in hybrid BCIs. Frontiers in Computer Science, 4. https://doi.org/10.3389/fcomp.2022.780580
Wei, C.-C. (2021). Collapse warning system using LSTM neural networks for construction disaster prevention in extreme wind weather. Journal of Civil Engineering and Management, 27(4), 230–245. https://doi.org/10.3846/jcem.2021.14649