[ { "id": "authors:e8r30-qbp54", "collection": "authors", "collection_id": "e8r30-qbp54", "cite_using_url": "https://authors.library.caltech.edu/records/e8r30-qbp54", "type": "article", "title": "Online Moderation in Competitive Action Games: How Intervention Affects Player Behaviors", "author": [ { "family_name": "Kocielnik", "given_name": "Rafal", "orcid": "0000-0001-5602-6056", "clpid": "Kocielnik-Rafal" }, { "family_name": "Li", "given_name": "Zhuofang", "orcid": "0009-0004-0970-9064", "clpid": "Li-Zhuofang" }, { "family_name": "Linegar", "given_name": "Mitchell", "orcid": "0009-0005-0866-405X", "clpid": "Linegar-Mitchell" }, { "family_name": "Sambrano", "given_name": "Deshawn", "orcid": "0000-0001-7709-9799" }, { "family_name": "Soltani", "given_name": "Fereshteh", "orcid": "0000-0002-3952-590X" }, { "family_name": "Kim", "given_name": "Min", "orcid": "0009-0003-0724-1050" }, { "family_name": "Naqvie", "given_name": "Nabiha", "orcid": "0009-0001-8588-2801" }, { "family_name": "Cahill", "given_name": "Grant", "orcid": "0009-0002-4145-8942" }, { "family_name": "Anandkumar", "given_name": "Animashree", "orcid": "0000-0002-6974-6797", "clpid": "Anandkumar-A" }, { "family_name": "Alvarez", "given_name": "R. Michael", "orcid": "0000-0002-8113-4451", "clpid": "Alvarez-R-M" } ], "abstract": "Online competitive action games have flourished as a space for entertainment and social connections, yet they face challenges from a small percentage of players engaging in disruptive behaviors. This study delves into the under-explored realm of understanding the effects of moderation on player behavior within online competitive action games on an example of a popular title - Call of Duty\u00ae:Modern Warfare\u00aeII. We employ a quasi-experimental design and causal inference techniques to examine the impact of moderation in a real-world industry-scale moderation system. We further delve into novel aspects around the impact of delayed moderation, as well as the severity of applied punishment. We examine these effects on a set of four disruptive behaviors including cheating, offensive username, chat, and voice. Our findings uncover the dual impact moderation has on reducing disruptive behavior and discouraging disruptive players from participating. We further uncover differences in the effectiveness of quick and delayed moderation and the varying severity of punishment. Our examination of real-world gaming interactions sets a precedent in understanding the effectiveness of moderation and its impact on player behavior. Our insights offer actionable suggestions for the most promising avenues for improving real-world moderation practices, as well as the heterogeneous impact moderation has on different players.", "doi": "10.1145/3748599", "issn": "2573-0142", "publisher": "Association for Computing Machinery (ACM)", "publication": "Proceedings of the ACM on Human-Computer Interaction", "publication_date": "2025-10-05", "series_number": "6", "volume": "9", "issue": "6", "pages": "96-130" }, { "id": "authors:6rcex-jna67", "collection": "authors", "collection_id": "6rcex-jna67", "cite_using_url": "https://authors.library.caltech.edu/records/6rcex-jna67", "type": "article", "title": "Bandit Algorithms for Efficient Toxicity Detection in Competitive Online Video Games", "author": [ { "family_name": "Morrier", "given_name": "Jacob", "orcid": "0000-0002-1815-7431", "clpid": "Morrier-Jacob" }, { "family_name": "Kocielnik", "given_name": "Rafal", "orcid": "0000-0001-5602-6056", "clpid": "Kocielnik-Rafal" }, { "family_name": "Alvarez", "given_name": "R. Michael", "orcid": "0000-0002-8113-4451", "clpid": "Alvarez-R-M" } ], "abstract": "
\n
\n
\n
This article considers the problem of efficient sampling for toxicity detection in competitive online video games. Video game service operators take proactive measures to detect and address undesirable behavior, seeking to focus these costly efforts where such behavior is most likely. To achieve this objective, service operators need estimates of the likelihood of toxic behavior. When no pre-existing predictive model of toxic behavior is available, one must be estimated in real-time. To this end, we propose a contextual bandit algorithm that uses a small set of variables, selected based on domain expertise, to guide monitoring decisions. This algorithm balances exploration and exploitation to optimize long-term performance and is designed intentionally for easy deployment in production environments. Using data from the popular first-person action game Call of Duty®: Modern Warfare® III, we show that our algorithm consistently outperforms baseline algorithms that rely solely on individual players’ past behavior, achieving improvements in detection rate of up to 24.56 percentage points or 51.5%. These results have substantive implications for the nature of toxicity and illustrate how domain expertise can be harnessed to help video game service operators detect and address toxicity, ultimately fostering a safer and more enjoyable gaming experience.
\n
\n
\n
", "doi": "10.1109/access.2025.3579418", "issn": "2169-3536", "publisher": "IEEE", "publication": "IEEE Access", "publication_date": "2025-06-13", "volume": "13", "pages": "103109-103117" }, { "id": "authors:s8tb7-zwa02", "collection": "authors", "collection_id": "s8tb7-zwa02", "cite_using_url": "https://authors.library.caltech.edu/records/s8tb7-zwa02", "type": "article", "title": "Development of a Classification System for Live Surgical Feedback", "author": [ { "family_name": "Wong", "given_name": "Elyssa Y.", "clpid": "Wong-Elyssa-Y" }, { "family_name": "Chu", "given_name": "Timothy N.", "clpid": "Chu-Timothy-N" }, { "family_name": "Ma", "given_name": "Runzhuo", "orcid": "0000-0001-6381-2661", "clpid": "Ma-Runzhuo" }, { "family_name": "Dalieh", "given_name": "Istabraq S.", "clpid": "Dalieh-Istabraq-S" }, { "family_name": "Yang", "given_name": "Cherine H.", "clpid": "Yang-Cherine-H" }, { "family_name": "Ramaswamy", "given_name": "Ashwin", "orcid": "0000-0002-8816-7838", "clpid": "Ramaswamy-Ashwin" }, { "family_name": "Medina", "given_name": "Luis G.", "orcid": "0000-0002-4188-7096", "clpid": "Medina-Luis-G" }, { "family_name": "Kocielnik", "given_name": "Rafal", "orcid": "0000-0001-5602-6056", "clpid": "Kocielnik-Rafal" }, { "family_name": "Ladi-Seyedian", "given_name": "Seyedeh-Sanam", "orcid": "0000-0002-3606-1556", "clpid": "Ladi-Seyedian-Seyedeh-Sanam" }, { "family_name": "Shtulman", "given_name": "Andrew", "orcid": "0000-0002-4687-3099", "clpid": "Shtulman-Andrew" }, { "family_name": "Cen", "given_name": "Steven Y.", "orcid": "0000-0002-7859-8909", "clpid": "Cen-Steven-Y" }, { "family_name": "Goldenberg", "given_name": "Mitchell G.", "orcid": "0000-0002-4601-5721", "clpid": "Goldenberg-Mitchell-G" }, { "family_name": "Hung", "given_name": "Andrew J.", "orcid": "0000-0002-7201-6736", "clpid": "Hung-Andrew-J" } ], "abstract": "

Importance: Live feedback in the operating room is essential in surgical training. Despite the role this feedback plays in developing surgical skills, an accepted methodology to characterize the salient features of feedback has not been defined.

Objective: To quantify the intraoperative feedback provided to trainees during live surgical cases and propose a standardized deconstruction for feedback.

Design, Setting, and Participants: In this qualitative study using a mixed methods analysis, surgeons at a single academic tertiary care hospital were audio and video recorded in the operating room from April to October 2022. Urological residents, fellows, and faculty attending surgeons involved in robotic teaching cases during which trainees had active control of the robotic console for at least some portion of a surgery were eligible to voluntarily participate. Feedback was time stamped and transcribed verbatim. An iterative coding process was performed using recordings and transcript data until recurring themes emerged.

Exposure: Feedback in audiovisual recorded surgery.

Main Outcomes and Measures: The primary outcomes were the reliability and generalizability of a feedback classification system in characterizing surgical feedback. Secondary outcomes included assessing the utility of our system.

Results: In 29 surgical procedures that were recorded and analyzed, 4 attending surgeons, 6 minimally invasive surgery fellows, and 5 residents (postgraduate years, 3-5) were involved. For the reliability of the system, 3 trained raters achieved moderate to substantial interrater reliability in coding cases using 5 types of triggers, 6 types of feedback, and 9 types of responses (prevalence-adjusted and bias-adjusted \u03ba range: a 0.56 [95% CI, 0.45-0.68] minimum for triggers to a 0.99 [95% CI, 0.97-1.00] maximum for feedback and responses). For the generalizability of the system, 6 types of surgical procedures and 3711 instances of feedback were analyzed and coded with types of triggers, feedback, and responses. Significant differences in triggers, feedback, and responses reflected surgeon experience level and surgical task being performed. For example, as a response, attending surgeons took over for safety concerns more often for fellows than residents (prevalence rate ratio [RR], 3.97 [95% CI, 3.12-4.82]; P\u2009=\u2009.002), and suturing involved more errors that triggered feedback than dissection (RR, 1.65 [95% CI, 1.03-3.33]; P\u2009=\u2009.007). For the utility of the system, different combinations of trainer feedback had associations with rates of different trainee responses. For example, technical feedback with a visual component was associated with an increased rate of trainee behavioral change or verbal acknowledgment responses (RR, 1.11 [95% CI, 1.03-1.20]; P\u2009=\u2009.02).

Conclusions and Relevance: These findings suggest that identifying different types of triggers, feedback, and responses may be a feasible and reliable method for classifying surgical feedback across several robotic procedures. Outcomes suggest that a system that can be generalized across surgical specialties and for trainees of different experience levels may help galvanize novel surgical education strategies.

", "doi": "10.1001/jamanetworkopen.2023.20702", "pmcid": "PMC10308254", "issn": "2574-3805", "publisher": "American Medical Association", "publication": "JAMA Network Open", "publication_date": "2023-06", "series_number": "6", "volume": "6", "issue": "6", "pages": "e2320702" }, { "id": "authors:25yyn-6ch14", "collection": "authors", "collection_id": "25yyn-6ch14", "cite_using_url": "https://resolver.caltech.edu/CaltechAUTHORS:20220930-482429300.5", "type": "article", "title": "Assessing the efficacy of dissection gestures in robotic surgery", "author": [ { "family_name": "Inouye", "given_name": "Daniel A.", "orcid": "0000-0001-7202-4800", "clpid": "Inouye-Daniel-A" }, { "family_name": "Ma", "given_name": "Runzhuo", "orcid": "0000-0001-6381-2661", "clpid": "Ma-Runzhuo" }, { "family_name": "Nguyen", "given_name": "Jessica H.", "orcid": "0000-0003-0454-8463", "clpid": "Nguyen-Jessica-H" }, { "family_name": "Laca", "given_name": "Jasper", "clpid": "Laca-Jasper-A" }, { "family_name": "Kocielnik", "given_name": "Rafal", "orcid": "0000-0001-5602-6056", "clpid": "Kocielnik-Rafal" }, { "family_name": "Anandkumar", "given_name": "Anima", "orcid": "0000-0002-6974-6797", "clpid": "Anandkumar-A" }, { "family_name": "Hung", "given_name": "Andrew J.", "orcid": "0000-0002-7201-6736", "clpid": "Hung-Andrew-J" } ], "abstract": "Our group previously defined a dissection gesture classification system that deconstructs robotic tissue dissection into its most elemental yet meaningful movements. The purpose of this study was to expand upon this framework by adding an assessment of gesture efficacy (ineffective, effective, or erroneous) and analyze dissection patterns between groups of surgeons of varying experience. We defined three possible gesture efficacies as ineffective (no meaningful effect on the tissue), effective (intended effect on the tissue), and erroneous (unintended disruption of the tissue). Novices (0 prior robotic cases), intermediates (1\u201399 cases), and experts (\u2265\u2009100 cases) completed a robotic dissection task in a dry-lab training environment. Video recordings were reviewed to classify each gesture and determine its efficacy, then dissection patterns between groups were analyzed. 23 participants completed the task, with 9 novices, 8 intermediates with median caseload 60 (IQR 41\u201380), and 6 experts with median caseload 525 (IQR 413\u2013900). For gesture selection, we found increasing experience associated with increasing proportion of overall dissection gestures (p\u2009=\u20090.009) and decreasing proportion of retraction gestures (p\u2009=\u20090.009). For gesture efficacy, novices performed the greatest proportion of ineffective gestures (9.8%, p\u2009<\u20090.001), intermediates commit the greatest proportion of erroneous gestures (26.8%, p\u2009<\u20090.001), and the three groups performed similar proportions of overall effective gestures, though experts performed the greatest proportion of effective retraction gestures (85.6%, p\u2009<\u20090.001). Between groups of experience, we found significant differences in gesture selection and gesture efficacy. These relationships may provide insight into further improving surgical training.", "doi": "10.1007/s11701-022-01458-x", "issn": "1863-2491", "publisher": "Springer", "publication": "Journal of Robotic Surgery", "publication_date": "2023-04", "series_number": "2", "volume": "17", "issue": "2", "pages": "597-603" }, { "id": "authors:c9g79-b2898", "collection": "authors", "collection_id": "c9g79-b2898", "cite_using_url": "https://resolver.caltech.edu/CaltechAUTHORS:20221122-564647900.20", "type": "article", "title": "Using Real-time Feedback To Improve Surgical Performance on a Robotic Tissue Dissection Task", "author": [ { "family_name": "Laca", "given_name": "Jasper A.", "clpid": "Laca-Jasper-A" }, { "family_name": "Kocielnik", "given_name": "Rafal", "orcid": "0000-0001-5602-6056", "clpid": "Kocielnik-Rafal" }, { "family_name": "Nguyen", "given_name": "Jessica H.", "orcid": "0000-0003-0454-8463", "clpid": "Nguyen-Jessica-H" }, { "family_name": "You", "given_name": "Jonathan", "clpid": "You-Jonathan" }, { "family_name": "Tsang", "given_name": "Ryan", "clpid": "Tsang-Ryan" }, { "family_name": "Wong", "given_name": "Elyssa Y.", "clpid": "Wong-Elyssa-Y" }, { "family_name": "Shtulman", "given_name": "Andrew", "orcid": "0000-0002-4687-3099", "clpid": "Shtulman-Andrew" }, { "family_name": "Anandkumar", "given_name": "Anima", "orcid": "0000-0002-6974-6797", "clpid": "Anandkumar-A" }, { "family_name": "Hung", "given_name": "Andrew J.", "orcid": "0000-0002-7201-6736", "clpid": "Hung-Andrew-J" } ], "abstract": "Background: There is no standard for the feedback that an attending surgeon provides to a training surgeon, which may lead to variable outcomes in teaching cases. \n\nObjective: To create and administer standardized feedback to medical students in an attempt to improve performance and learning. \n\nDesign, setting, and participants: A cohort of 45 medical students was recruited from a single medical school. Participants were randomly assigned to two groups. Both completed two rounds of a robotic surgical dissection task on a da Vinci Xi surgical system. The first round was the baseline assessment. In the second round, one group received feedback and the other served as the control (no feedback). \n\nOutcome measurements and statistical analysis: Video from each round was retrospectively reviewed by four blinded raters and given a total error tally (primary outcome) and a technical skills score (Global Evaluative Assessment of Robotic Surgery [GEARS]). Generalized linear models were used for statistical modeling. According to their initial performance, each participant was categorized as either an innate performer or an underperformer, depending on whether their error tally was above or below the median. \n\nResults and limitations: In round 2, the intervention group had a larger decrease in error rate than the control group, with a risk ratio (RR) of 1.51 (95% confidence interval [CI] 1.07\u20132.14; p = 0.02). The intervention group also had a greater increase in GEARS score in comparison to the control group, with a mean group difference of 2.15 (95% CI 0.81\u20133.49; p < 0.01). The interaction effect between innate performers versus underperformers and the intervention was statistically significant for the error rates, at F(1,38) = 5.16 (p = 0.03). Specifically, the intervention had a statistically significant effect on the error rate for underperformers (RR 2.23, 95% CI 1.37\u20133.62; p < 0.01) but not for innate performers (RR 1.03, 95% CI 0.63\u20131.68; p = 0.91). \n\nConclusions: Real-time feedback improved performance globally compared to the control. The benefit of real-time feedback was stronger for underperformers than for trainees with innate skill. \n\nPatient summary: We found that real-time feedback during a training task using a surgical robot improved the performance of trainees when the task was repeated. This feedback approach could help in training doctors in robotic surgery.", "doi": "10.1016/j.euros.2022.09.015", "pmcid": "PMC9732447", "issn": "2666-1683", "publisher": "Elsevier", "publication": "European Urology Open Science", "publication_date": "2022-12", "volume": "46", "pages": "15-21" }, { "id": "authors:hjdr0-6jf46", "collection": "authors", "collection_id": "hjdr0-6jf46", "cite_using_url": "https://resolver.caltech.edu/CaltechAUTHORS:20230502-387779700.6", "type": "article", "title": "Design and Evaluation Challenges of Conversational Agents in Health Care and Well-being: Selective Review Study", "author": [ { "family_name": "Kocaballi", "given_name": "Ahmet Baki", "orcid": "0000-0002-8328-5317", "clpid": "Kocaballi-Ahmet-Baki" }, { "family_name": "Sezgin", "given_name": "Emre", "orcid": "0000-0001-8798-9605", "clpid": "Sezgin-Emre" }, { "family_name": "Clark", "given_name": "Leigh", "orcid": "0000-0002-9237-1057", "clpid": "Clark-Leigh" }, { "family_name": "Carroll", "given_name": "John M.", "orcid": "0000-0001-5189-337X", "clpid": "Carroll-John-M" }, { "family_name": "Huang", "given_name": "Yungui", "orcid": "0000-0003-3265-9902", "clpid": "Huang-Yungui" }, { "family_name": "Huh-Yoo", "given_name": "Jina", "orcid": "0000-0001-5811-9256", "clpid": "Huh-Yoo-Jina" }, { "family_name": "Kim", "given_name": "Junhan", "orcid": "0000-0002-9636-2166", "clpid": "Kim-Junhan" }, { "family_name": "Kocielnik", "given_name": "Rafal", "orcid": "0000-0001-5602-6056", "clpid": "Kocielnik-Rafal" }, { "family_name": "Lee", "given_name": "Yi-Chieh", "orcid": "0000-0002-5484-6066", "clpid": "Lee-Yi-Chieh" }, { "family_name": "Mamykina", "given_name": "Lena", "orcid": "0000-0001-5203-274X", "clpid": "Mamykina-Lena" }, { "family_name": "Mitchell", "given_name": "Elliot G.", "orcid": "0000-0001-5480-5021", "clpid": "Mitchell-Elliot-G" }, { "family_name": "Moore", "given_name": "Robert J.", "orcid": "0000-0002-5636-9822", "clpid": "Moore-Robert-J" }, { "family_name": "Murali", "given_name": "Prasanth", "orcid": "0000-0001-8751-5629", "clpid": "Murali-Prasanth" }, { "family_name": "Mynatt", "given_name": "Elizabeth D.", "orcid": "0000-0001-8486-9384", "clpid": "Mynatt-Elizabeth-D" }, { "family_name": "Park", "given_name": "Sun Young", "orcid": "0000-0001-8666-1800", "clpid": "Park-Sun-Young" }, { "family_name": "Pasta", "given_name": "Alessandro", "orcid": "0000-0002-9920-6695", "clpid": "Pasta-Alessandro" }, { "family_name": "Richards", "given_name": "Deborah", "orcid": "0000-0002-7363-1511", "clpid": "Richards-Deborah" }, { "family_name": "Silva", "given_name": "Lucas M.", "orcid": "0000-0001-9795-9071", "clpid": "Silva-Lucas-M" }, { "family_name": "Smriti", "given_name": "Diva", "orcid": "0000-0001-8959-1975", "clpid": "Smriti-Diva" }, { "family_name": "Spillane", "given_name": "Brendan", "orcid": "0000-0001-5893-1340", "clpid": "Spillane-Brendan" }, { "family_name": "Zhang", "given_name": "Zhan", "orcid": "0000-0001-6973-6903", "clpid": "Zhang-Zhan" }, { "family_name": "Zubatiy", "given_name": "Tamara", "orcid": "0000-0002-7902-8115", "clpid": "Zubatiy-Tamara" } ], "abstract": "Background: Health care and well-being are 2 main interconnected application areas of conversational agents (CAs). There is a significant increase in research, development, and commercial implementations in this area. In parallel to the increasing interest, new challenges in designing and evaluating CAs have emerged. \n\nObjective: This study aims to identify key design, development, and evaluation challenges of CAs in health care and well-being research. The focus is on the very recent projects with their emerging challenges. \n \nMethods: A review study was conducted with 17 invited studies, most of which were presented at the ACM (Association for Computing Machinery) CHI 2020 conference workshop on CAs for health and well-being. Eligibility criteria required the studies to involve a CA applied to a health or well-being project (ongoing or recently finished). The participating studies were asked to report on their projects' design and evaluation challenges. We used thematic analysis to review the studies. \n \nResults: The findings include a range of topics from primary care to caring for older adults to health coaching. We identified 4 major themes: (1) Domain Information and Integration, (2) User-System Interaction and Partnership, (3) Evaluation, and (4) Conversational Competence. \n \nConclusions: CAs proved their worth during the pandemic as health screening tools, and are expected to stay to further support various health care domains, especially personal health care. Growth in investment in CAs also shows the value as a personal assistant. Our study shows that while some challenges are shared with other CA application areas, safety and privacy remain the major challenges in the health care and well-being domains. An increased level of collaboration across different institutions and entities may be a promising direction to address some of the major challenges that otherwise would be too complex to be addressed by the projects with their limited scope and budget.", "doi": "10.2196/38525", "pmcid": "PMC9709676", "issn": "1438-8871", "publisher": "JMIR Publications Inc.", "publication": "Journal of Medical Internet Research", "publication_date": "2022-11-15", "series_number": "11", "volume": "24", "issue": "11", "pages": "Art. No. e38525" }, { "id": "authors:96rss-qa524", "collection": "authors", "collection_id": "96rss-qa524", "cite_using_url": "https://resolver.caltech.edu/CaltechAUTHORS:20220912-920381000", "type": "article", "title": "The Relationship Between Technical Skills, Cognitive Workload, and Errors During Robotic Surgical Exercises", "author": [ { "family_name": "Roberts", "given_name": "Sidney I.", "clpid": "Roberts-Sidney-I" }, { "family_name": "Cen", "given_name": "Steven Y.", "orcid": "0000-0002-7859-8909", "clpid": "Cen-Steven-Y" }, { "family_name": "Nguyen", "given_name": "Jessica H.", "orcid": "0000-0003-0454-8463", "clpid": "Nguyen-Jessica-H" }, { "family_name": "Perez", "given_name": "Laura C.", "clpid": "Perez-Laura-C" }, { "family_name": "Medina", "given_name": "Luis G.", "clpid": "Medina-Luis-G" }, { "family_name": "Ma", "given_name": "Runzhuo", "orcid": "0000-0001-6381-2661", "clpid": "Ma-Runzhuo" }, { "family_name": "Marshall", "given_name": "Sandra", "clpid": "Marshall-Sandra" }, { "family_name": "Kocielnik", "given_name": "Rafal", "orcid": "0000-0001-5602-6056", "clpid": "Kocielnik-Rafal" }, { "family_name": "Anandkumar", "given_name": "Anima", "orcid": "0000-0002-6974-6797", "clpid": "Anandkumar-A" }, { "family_name": "Hung", "given_name": "Andrew J.", "orcid": "0000-0002-7201-6736", "clpid": "Hung-Andrew-J" } ], "abstract": "Purpose: We attempt to understand the relationship between surgeon technical skills, cognitive workload, and errors during a simulated robotic dissection task.\n\nMaterials and Methods: Participant surgeons performed a robotic surgery dissection exercise. Participants were grouped based on surgical experience. Technical skills were evaluated utilizing the validated Global Evaluative Assessment of Robotic Skills (GEARS) assessment tool. The dissection task was evaluated for errors during active dissection or passive retraction maneuvers. We quantified cognitive workload of surgeon participants as an index of cognitive activity (ICA), derived from task-evoked pupillary response metrics; ICA ranged 0 to 1, with 1 representing maximum ICA. Generalized estimating equation (GEE) was used for all modelings to establish relationships between surgeon technical skills, cognitive workload, and errors.\n\nResults: We found a strong association between technical skills as measured by multiple GEARS domains (depth perception, force sensitivity, and robotic control) and passive errors, with higher GEARS scores associated with a lower relative risk of errors (all p\u2009<\u20090.01). For novice surgeons, as average GEARS scores increased, the average estimated ICA decreased. In contrast, as average GEARS increased for expert surgeons, the average estimated ICA increased. When exhibiting optimal technical skill (maximal GEARS scores), novices and experts reached a similar range of ICA scores (ICA: 0.47 and 0.42, respectively).\n\nConclusions: This study found that there is an optimal cognitive workload level for surgeons of all experience levels during our robotic surgical exercise. Select technical skill domains were strong predictors of errors. Future research will explore whether an ideal cognitive workload range truly optimizes surgical training and reduces surgical errors.", "doi": "10.1089/end.2021.0790", "pmcid": "PMC9145254", "issn": "0892-7790", "publisher": "Mary Ann Liebert Inc", "publication": "Journal of Endourology", "publication_date": "2022-05", "series_number": "5", "volume": "36", "issue": "5", "pages": "712-720" } ]