Limits...
Process modeling and bottleneck mining in online peer-review systems.

Premchaiswadi W, Porouhan P - Springerplus (2015)

Bottom Line: Finally, we investigated the performance information as well as the total waiting time in order to improve the effectiveness and efficiency of the online submission and peer review system for the prospective conferences and seminars.The results showed that although each paper was initially sent to three different reviewers; it was not always possible to make a decision after the first round of reviewing; therefore, additional reviewers were invited.On the other hand, dealing with the feedback (comments) received from the first and the third reviewers; the conference committee members and the organizers did not attend to those feedback/comments in a timely manner.

View Article: PubMed Central - PubMed

Affiliation: Graduate School of Information Technology, Siam University, Bangkok, Thailand.

ABSTRACT
This paper is divided into three main parts. In the first part of the study, we captured, collected and formatted an event log describing the handling of reviews for proceedings of an international conference in Thailand. In the second part, we used several process mining techniques in order to discover process models, social, organizational, and hierarchical structures from the proceeding's event log. In the third part, we detected the deviations and bottlenecks of the peer review process by comparing the observed events (i.e., authentic dataset) with a pre-defined model (i.e., master map). Finally, we investigated the performance information as well as the total waiting time in order to improve the effectiveness and efficiency of the online submission and peer review system for the prospective conferences and seminars. Consequently, the main goals of the study were as follows: (1) to convert the collected event log into the appropriate format supported by process mining analysis tools, (2) to discover process models and to construct social networks based on the collected event log, and (3) to find deviations, discrepancies and bottlenecks between the collected event log and the master pre-defined model. The results showed that although each paper was initially sent to three different reviewers; it was not always possible to make a decision after the first round of reviewing; therefore, additional reviewers were invited. In total, all the accepted and rejected manuscripts were reviewed by an average of 3.9 and 3.2 expert reviewers, respectively. Moreover, obvious violations of the rules and regulations relating to careless or inappropriate peer review of a manuscript-committed by the editorial board and other staff-were identified. Nine blocks of activity in the authentic dataset were not completely compatible with the activities defined in the master model. Also, five of the activity traces were not correctly enabled, and seven activities were missed within the online submission system. On the other hand, dealing with the feedback (comments) received from the first and the third reviewers; the conference committee members and the organizers did not attend to those feedback/comments in a timely manner.

No MeSH data available.


(Up) Two screenshots of the Social Network Miner (based on the Working Together metric) on the peer-review event log. (Down) A screenshot of the Working Together matrix using ProM Process Mining Framework.
© Copyright Policy - OpenAccess
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4546073&req=5

Fig10: (Up) Two screenshots of the Social Network Miner (based on the Working Together metric) on the peer-review event log. (Down) A screenshot of the Working Together matrix using ProM Process Mining Framework.

Mentions: Likewise, we wanted to know whether people in the same community work together or not. To address this issue, we focused on the cases (instead of the activities) using Social Network Miner technique. Figure 10 shows a social network obtained by the Working Together graph using ProM 5.2. The graph shows active participation and the collectiveness among Mr. A, Mr. C, Ms. B ad Ms. G. The Working Together graph is helpful when there are disjoint teams in the log.Fig. 10


Process modeling and bottleneck mining in online peer-review systems.

Premchaiswadi W, Porouhan P - Springerplus (2015)

(Up) Two screenshots of the Social Network Miner (based on the Working Together metric) on the peer-review event log. (Down) A screenshot of the Working Together matrix using ProM Process Mining Framework.
© Copyright Policy - OpenAccess
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4546073&req=5

Fig10: (Up) Two screenshots of the Social Network Miner (based on the Working Together metric) on the peer-review event log. (Down) A screenshot of the Working Together matrix using ProM Process Mining Framework.
Mentions: Likewise, we wanted to know whether people in the same community work together or not. To address this issue, we focused on the cases (instead of the activities) using Social Network Miner technique. Figure 10 shows a social network obtained by the Working Together graph using ProM 5.2. The graph shows active participation and the collectiveness among Mr. A, Mr. C, Ms. B ad Ms. G. The Working Together graph is helpful when there are disjoint teams in the log.Fig. 10

Bottom Line: Finally, we investigated the performance information as well as the total waiting time in order to improve the effectiveness and efficiency of the online submission and peer review system for the prospective conferences and seminars.The results showed that although each paper was initially sent to three different reviewers; it was not always possible to make a decision after the first round of reviewing; therefore, additional reviewers were invited.On the other hand, dealing with the feedback (comments) received from the first and the third reviewers; the conference committee members and the organizers did not attend to those feedback/comments in a timely manner.

View Article: PubMed Central - PubMed

Affiliation: Graduate School of Information Technology, Siam University, Bangkok, Thailand.

ABSTRACT
This paper is divided into three main parts. In the first part of the study, we captured, collected and formatted an event log describing the handling of reviews for proceedings of an international conference in Thailand. In the second part, we used several process mining techniques in order to discover process models, social, organizational, and hierarchical structures from the proceeding's event log. In the third part, we detected the deviations and bottlenecks of the peer review process by comparing the observed events (i.e., authentic dataset) with a pre-defined model (i.e., master map). Finally, we investigated the performance information as well as the total waiting time in order to improve the effectiveness and efficiency of the online submission and peer review system for the prospective conferences and seminars. Consequently, the main goals of the study were as follows: (1) to convert the collected event log into the appropriate format supported by process mining analysis tools, (2) to discover process models and to construct social networks based on the collected event log, and (3) to find deviations, discrepancies and bottlenecks between the collected event log and the master pre-defined model. The results showed that although each paper was initially sent to three different reviewers; it was not always possible to make a decision after the first round of reviewing; therefore, additional reviewers were invited. In total, all the accepted and rejected manuscripts were reviewed by an average of 3.9 and 3.2 expert reviewers, respectively. Moreover, obvious violations of the rules and regulations relating to careless or inappropriate peer review of a manuscript-committed by the editorial board and other staff-were identified. Nine blocks of activity in the authentic dataset were not completely compatible with the activities defined in the master model. Also, five of the activity traces were not correctly enabled, and seven activities were missed within the online submission system. On the other hand, dealing with the feedback (comments) received from the first and the third reviewers; the conference committee members and the organizers did not attend to those feedback/comments in a timely manner.

No MeSH data available.