Writing Errors and Peer Review in ARCIMSS13th
Short.jpg

Assessment of Writing Errors and Peer Review Process of Accepted Abstracts in the 13th Annual Research Congress of Iran’s Medical Sciences Students

 

Afshin Khani1, Fatemeh Izadpanah1, Amin Zarghami1, Hamid Mahdizadeh1, Soraya Khafri2

 

1Student Research Committee, Babol University of Medical Sciences, Babol, Iran

2Department of Social Medicine, Babol University of Medical Sciences, Babol, Iran

 

 

Abstract

 

Background: Despite more than a decade experience of annual student congresses in Iran, major scientific writing weakness still exists in students’ abstracts submitted to the Annual Research Congress of Iran’s Medical Sciences Students (ARCIMSS). Assessment of these abstracts can provide information on common scientific writing errors and subsequent development of abstract quality for the future congresses. Assessment of writing errors and peer review process of accepted abstracts in ARCIMSS 13th forms the central idea of the present study. Materials and Methods: In this cross-sectional study, from all 505 accepted abstracts, 17 abstracts were excluded and the remaining 488 abstracts were evaluated for affiliation and educational level of the presenter, presentation type, writing errors and the scores of reviewing process for each abstract. Writing errors were compared based on the presenters’ affiliation (Student Research Committee (SRC) affiliated and non-SRC affiliated) and educational level. Also, correlation between students and faculty members’ reviewing scores in peer review process was evaluated.Results: Writing errors were seen in 242 (49.6%) abstracts and the majority of errors were seen in the “author’s affiliation” section (26.6%). The frequency of writing errors was significantly lower in SRC-affiliated abstracts comparing to non-SRC affiliated (P=0.038) that was not significantly different from postgraduate and undergraduate presenters (P=0.34). There was no significant correlation between pre-congress and during-congress peer reviewing scores (P=0.399, r=0.05). Conclusion: There were significant writing errors in accepted abstracts. This issue underlines the necessity of educational interventions performed by SRC members in order to develop students’ skills in abstract writing based on standard guidelines. [GMJ.2014;3(4):245-51]

 

Key Words: Meeting Abstracts; Medical; Peer Review; Students; Writing

Correspondence to:

Hamid Mahdizadeh, Student Research Committee, Babol University of Medical Sciences, Babol, Iran

Tel (Fax): (+98) 9124595861

Email Address: mahdizadehmd@yahoo.com

 

GMJ.2014;3(4):245-51

www.gmj.ir

Introduction

 

Students play an inevitable role in scientific researches in recent years in Iran. Iranian students of biomedical sciences have presented their latest research findings in national congresses for more than a decade. As a matter of fact, Annual Research Congress of Iranian’s Medical Sciences Students (ARCIMSS) is not only a place for presenting abstracts but also a meeting for students to share their research experiences, develop student research qualities and also an opportunity for students to be familiar with research and knowing other young researchers. These congresses have also had a great impact on students’ attitude toward research [1]. While there were always valuable abstracts presented in this congress each year, there are few abstracts that are finally published as a full text article in scientific journals [2]. Serious errors in terms of scientific writing and appropriate research methodology could be among such causes [3]. Appropriate educational interventions in order to develop the methodology and scientific writing skills among students before submitting abstracts to congress, may help enhance the quality of abstracts and increase the chance of acceptance for presentation and subsequent full text publication.

Student Research Committee (SRC), which is the most important supportive organization of students’ researches in Iran, as a student organization; besides holding these educational workshops that are not included in medical schools’ curriculum, can develop students’ knowledge of congress’s instruction for abstract submission, exclusion criteria of abstracts form review process and other specific useful rules of each year’s congress. Furthermore, senior students cooperating with SRC can help beginner students to provide well-written abstracts and avoid common mistakes.

One of the major aspects of ARCIMSS is Peer Review process. Peer review has always been a challengeable issue both in congresses and journals [4]. Students have the opportunity to participate in this process which is a unique feature of ARCIMSS. Students’ involvement in the peer review process can improve their critical appraisal skills and methodological knowledge.

To the best of authors’ knowledge, few studies were carried out about the quality of abstracts in student congress in other countries and no previous study has been conducted in Iran [5]. Hence, the primary aim of the present study was to assess the writing errors and peer-review process of accepted abstracts in the 13th Annual Research Congress of Iran’s Medical Sciences Students that was held by the Student Research Committee of Babol University of Medical Sciences in Babolsar, 9-12 October 2012. The frequency of structural errors leading to exclusion of abstracts in pre-congress screening stage was also assessed as the secondary aim.

 

Materials and Methods

 

Data Collection

This cross-sectional study was carried out on all 505 accepted abstracts of the 13th ARCIMSS. The affiliation and the educational level of the presenting authors, presentation type (oral, poster-discussion and poster), writing errors and the scores of reviewing process were evaluated for each abstract. Writing errors consist of inaccurate or incomplete authors’ names (i.e. writing Dr., Mr., etc for authors or abbreviated authors’ names), not specifying presenter for abstracts, specifying non-student presenter, errors in authors’ affiliation, words within 10% of maximum authorized words count, not mentioning keywords and submitting full text of article with the abstract. Based on the authors’ affiliations, abstracts were divided into two groups; SRC-affiliated and non SRC-affiliated. Also, based on educational level of presenting authors, abstracts were divided into two groups: undergraduates and postgraduates. Seventeen abstracts were excluded from the study, nine of them were from innovations & inventions topic because of dissimilar reviewing process and eight of them were from health system research topic because of lost data.

 

Reviewing Process

Overall 2065 abstracts were submitted to the congress.

All abstracts were primarily screened by the congress scientific team for potential structural errors and those with significant errors were excluded from the review process. After that, 1801 abstracts were blinded and sent for reviewing process to 79 student reviewers. All student reviewers were selected according to curriculum vitae’s score and qualification exam’s score. Each abstract was evaluated by three independent student reviewers and based on the given mean scores. Scholarly abstracts were submitted to second stage of pre-congress review. Altogether, 850 abstracts were selected and sent for faculty members’ evaluation as fourth reviewers. Eventually, 505 abstracts were accepted for presentation. One hundred and one abstracts fit for oral, 42 poster-discussion, and 362 poster presentations. The review process during congress days consisted of three students and one faculty member review scores in oral panels and three independent student reviewers’ scores in poster-talk and poster panels. Peer reviewing was done with standardized checklists which have been shown to have acceptable validity and reliability in previous congresses.The detailed report of the congress and also the process of peer-review was reported previously [6]

Statistical Analysis

Data were analyzed by SPSS software version 12 (Chicago- IL.). Chi-square test was used for comparing writing errors, absence of presenter and presentation type with affiliation and education level of authors. One sample Kolmogorov-Smirnov test was used for determining the normal distribution of quantitative variables. Mann-Whitney U-test and independent T-test was used for comparing affiliation and education level of authors with the reviewing scores. Pearson and Spearman correlation was used for assessment of correlation between faculty members and students’ reviewing score of each article.

 

Results

From all 488 evaluated abstracts, 321 (65.8%) abstracts were presented on congress days, whereas 167 (34.2%) were not. The baseline information of evaluated abstracts was presented in Table 1. The mean score of pre-congress review score was 127.7±10.74 for non-presented and 127.61±11.8 for presented abstracts (P=0.731). Among non-presented abstracts, 72 (43.1%) were SRC-affiliated and 95 (56.9%) were non SRC-affiliated (P=0.004, OR=0.57, 95% CI: 0.39-0.83).

In pre-congress screening stage, non-adherence from formal structure (44%) was the leading cause of excluding abstracts from review process (table 2).

Writing errors were seen in 242 (49.6%) abstracts and the majority of errors were seen in the “author’s affiliation” section (26.6%). The frequency of writing errors was 115 (45.1%) in SRC-affiliated and 127 (54.5%) in non SRC-affiliated abstracts. Writing errors were significantly lower in SRC-affiliated abstracts (P-value=0.038, OR=0.69, 95% CI: 0.48-0.98). The frequency of writing errors was 54 (45.8%) in postgraduates and 188 (50.8%) in undergraduates’ abstracts. There was no significant difference between rate of writing errors and education level of presenters (P=0.34, OR=0.81, 95% CI: 0.54-1.23). Details of different types of writing errors are shown in table 3.

Thirty three (28%) abstracts by postgraduate students were accepted for oral presentation, 13(11%) for poster discussion and 72 (61%) for poster panels. In contrast, 59 (15.9%) abstracts of undergraduate students were accepted for oral presentation, 34 (9.2%) for poster talk and 277 (74.9%) for poster panels (P=0.008). In terms of affiliation, in SRC-affiliated abstracts, 39 (15.3%) were accepted for oral, 24 (9.4%) for poster talk and 192 (75.3%) for poster presentation; whereas in non SRC-affiliated abstracts, 53 (22.7%) were oral, 23 (9.9%) poster talk and 157 (64.7%) poster presentation (P=0.096).

The mean score of peer review in pre-congress was 127.67±11.1 and 43±10.7 during congress. There was no significant correlation between pre-congress and during-congress peer review scores (P-value=0.399, r=0.05). There was a significant positive correlation between students and faculty members’ review scores during congress review process (P<0.001, r=0.533). However, there was no significant correlation between students and faculty members’ review score in pre-congress review process (P=0.178, r=-0.06). The comparison of peer review scores based on affiliation and educational level of presenters in pre-congress and during-congress stage is shown in table 4.

 

Discussion

 

Our study indicated that there were significant errors in writing skills among students especially in “authors’ affiliation” section. Moreover, the majority of excluded abstracts in the screening process had serious errors in the basic structure of the conventional abstract format.

Based on our findings, SRC-affiliation of presenter significantly decreased writing errors and had a protective effect on overall writing errors to occur. Furthermore, our study showed that there was no significant difference in writing errors regarding presenter’s educational level. In the evaluation of peer review process, we found out there was no significant correlation between pre-congress and during-congress scores. Moreover, there was no significant correlation between students’ and faculty members’ scores in pre-congress peer review process. However, there was a positively significant correlation between students and faculty members’ scores during congress peer review process.

Alishiri, et al. reported that significant weakness was seen in “Abstract” section in medical students’ thesis [7]. A weak writing skill among students was also reported in “Reference” and “Methods” sections in other studies [3,8]. Inappropriate level of knowledge in research methodology besides lack of attention to the “instruction for authors” guidelines provided by the scientific team of the congress seems to be the main etiology. As it was expected, the rate of these errors was significantly low among SRC -affiliated abstracts because SRC senior members usually notify other students of submission guidelines and develop their knowledge through regular educational workshops or editing their abstracts. Interestingly, there are research methodology courses in postgraduate students’ formal curriculum. Educational level of presenters had no protective effect on writing errors occurrence. This finding highlights the efficacy of SRC workshops compared to postgraduates’ conventional research education.

The ideal outcome of students’ abstracts is to be published in peer-reviewed journals which can increase the visibility of students’ research. Appropriate criticism and comments from expert students and faculty members as congress reviewers may increase the chance of subsequent publishing in peer-reviewed journals [5]. There was a significantly positive correlation between students’ and faculty members’ review scores during congress review process. This is an interesting finding, however there was no correlation between students and faculty members’ review score in pre-congress review process.

A potential reason is the difference between faculty review checklist and student review checklist. Besides, faculty members usually evaluate the abstracts from their professional perspective while students usually consider methodological aspects. Besides, one of the goals of these congresses is to unify the review process between students and faculty members in order to develop criticism skills among students.

There are limited oral panels in each congress and most of the students are willing to see their work accepted in such panels. So, there is a tough competition for presentation between high quality abstracts in these panels. In this congress, 28% of postgraduates’ abstracts were accepted in oral panels while only 15.9% of undergraduates’ abstracts were accepted in oral panels. Considering the quality of postgraduates’ research projects, the competition between these abstracts and undergraduates’ abstracts seems to be unfair. Hence, it is necessary to review abstracts of postgraduate and undergraduate students independently and consider a certain chance for abstracts of undergraduate students to be accepted as oral presentation. This idea may courage undergraduate students to be involved and submit their abstracts.

The ultimate goal of ARCIMSS is developing students’ research skills. Despite its great annual financial costs, this congress is held by the support of Ministry of Health for enhancing these objectives among biomedical students [2,9]. One of the greatest problems that the executive team encountered was non-presented abstract. There were 167 (34.2%) non-presented abstracts in this congress, the majority of which were non SRC affiliated. This finding revealed that SRC members usually care about the quality and discipline of congress besides their personal gains while other students usually participate in congress for other reasons such as tourism [10]. However, further studies on exact causes of absence of participants on congress days are mandatory to confirm this hypothesis. There were lots of efforts from executive staff for providing an ideal presentation opportunity on congress days; absence of students who have presentation may lead to irregularity in the congress timetable though. On the other hand, it would spoil the chance of those young researchers who are eager for presenting their work. Thus, determining certain penalties for absent presenters such as not devoting certification document of presentation and exclusion from next year congress can force students to present their accepted abstracts.

To the best of authors’ knowledge, this was the first study in which the quality of a student congress abstracts and presentations were assessed. Researchers tried to emphasize the educational aspects which were not previously reported. As a result, we could not compare our findings with other studies and we interpreted the findings according to research circumstances in Iran. Hence, it seems logical to conduct similar studies in different student congress worldwide, Iran is no exception.

In conclusion, there were significant writing errors in the accepted abstracts of the 13th ARCIMSS. There was no correlation between pre-congress and during-congress peer reviewing scores on the one hand, and students and faculty members’ pre-congress peer review score on the other hand.

Moreover, moderate correlation was seen between students and faculty members’ scores during congress peer reviewing.

This issue underlines the necessity of educational interventions performed by SRC members in order to develop students’ skills in abstract writing based on standard guidelines such as STROBE and CONSORT. Moreover, regular national workshops are required for both students and faculty members to unify and improve their peer review process.

 

Acknowledgment

 

Authors would like to thank Mr. Farid Alinezhad (Chairperson of the ARCIMSS 13th) and Mr. Seyed Mohammadreza Banihoseini (Executive Chairperson of the ARCIMSS 13th) for their cooperation in data collection. We also appreciate Dr. Seyed Mohammad Fereshtehnejad for his valuable comments and for editing the manuscript. Finally, we thank deputy of research at the Ministry of Health and Medical Education for the financial support of ARCIMSS 13th.

 

Table 1. Baseline Characteristics of Accepted Abstracts of ARCIMSS 13th

Variables

 

Mean ± SD or N (%)

Affiliation of presenter

SRC affiliated

255(52.3)

Non-SRC affiliated

233 (47.7)

Educational level of presenter

Undergraduate

370 (75.8)

Postgraduate

118 (24.2)

Presentation statues

Presented

321 (65.8)

Non-presented

167 (34.2)

Presentation type

Oral

92 (18.9)

Poster-discussion

47 (9.6)

Poster

349 (71.5)

Study type

Observational

318 (65.2)

Interventional

161 (33)

Review

9 (1.8)

Peer review scores*

Pre-congress

127.67±11.1

During congress

43±10.7

*The Maximum Acquirable Score of Peer Review Process Was 200 for Pre-Congress and 65 for During Congress.

Table 2. The Frequency of Exclusion Criteria in Pre-Congress Screening Stage (N=264)

Structural Error*

N (%)

Non-adherence from formal structure

(background and objective, materials and methods, findings, conclusion)

116 (44)

Non-Adherence from Word Count Limitation

93 (35.3)

Narrative Review Submission for Unauthorized Sections

30 (11.5)

No Findings Reported

2 (0.8)

Unconventional Components (table, figure, reference, etc.)

49 (18.7)

Others (resubmission, blank submission, …)

15 (5.6)

 

*Each Abstract May Have More than One Structural Error.

Table 3. Distribution of Different Writing Errors Among Evaluated Abstracts in ARCIMSS 13th.

Punctuation Error

SRC affiliated (%)

P value

Education level (%)

P value

Total (%)

Yes

(n=255)

 

No

(n=233)

 

Postgraduate

(n=118)

 

Undergraduate

(n=370)

 

Authors’ Name Exactitude

58 (22.7)

72 (30.9)

 

0.042

29 (24.6)

101 (27.3)

0.56

130 (26.6)

Non-Specified Presenter

33 (12.9)

55 (23.6)

 

0.002

17 (14.4)

 

 

71 (19.2)

0.239

88 (18)

Non-student Presenter

28 (11)

 

55 (23.6)

 

<0.001

18 (15.3)

65 (17.6)

0.56

83 (17)

Authors’ Affiliation

141 (55.3)

173 (74.2)

<0.001

81 (68.6)

233 (63)

0.263

314 (64.3)

Over 10% of Word Count Limitation

14 (5.5)

 

24 (10.3)

 

0.048

14 (11.9)

24 (6.5)

0.058

38 (7.8)

Key Word

2 (0.8)

3 (1.3)

0.673

0 (0)

5 (1.4)

0.343

5 (1)

Abstract with full text

3 (1.2)

11 (4.7)

0.019

7 (5.9)

7 (1.9)

0.022

14 (2.9)

Total

115 (45.1)

127 (54.5)

0.038

54 (45.8%)

188 (50.8)

0.34

242 (49.6)

Table 4. Mean Score of Peer Review Process in Evaluated Abstracts of ARCIMSS13th

Review Score

SRC affiliated

P value

Education level

P value

Total

Yes

(n=255)

 

No

(n=233)

 

Postgraduate

(n=118)

 

Undergraduate

(n=370)

 

Pre Congress

 

Students

 

Faculty

 

Total

 

 

 

105.68±12.12

 

34.18±6.69

 

127.87±10.88

 

 

 

105.26±10.59

 

34.41±6.85

 

127.44±11.38

 

 

 

0.438

 

0.708

 

0.608

 

 

 

106.05±14.51

 

34.1±7.23

 

129.28±12.7

 

 

 

105.33±10.42

 

34.34±6.63

 

127.22±10.59

 

 

 

0.267

 

0.748

 

0.268

 

 

 

105.49±11.43

 

34.29±6.76

 

127.67±11.1

 

During Congress

 

Students

Faculty

 

Total

 

 

 

42.66±9.76

 

48.83±8.5

 

43.87±10.25

 

 

 

40.79±12.29

 

45.26±8.83

 

41.84±11.2

 

 

 

0.318

 

0.086

 

0.097

 

 

 

40.99±11.57

 

44.78±9.69

 

42.65±10.34

 

 

 

42.1±10.86

 

48.1±8.06

 

43.1±10.81

 

 

 

0.622

 

0.117

 

0.775

 

 

 

41.8±11

 

46.83±8.81

 

43±10.7

 

References

  1. 1. Hamilton J. The renewal of medical education in Iran: progress and challenge. Med J Islam Repub Iran. 2011;25(2):53-6.
  2. 2. Hosseini SY. Too Many Students’ Congresses, Are They Necessary and Useful? Thrita J Med Sci. 2012;1(2):71-2.
  3. 3. Zarshenas L, Shaghaghian S, Momeni danaei S, Tabatabaei Shr. The Methodological Evaluation of General Dentistry Students’ Theses in Shiraz Dentistry School (2006-2008). Iranian J Med Edu. 2012;12(3):202-9.
  4. 4. Habibzadeh F. Judge the article, not the author. Croat Med J. 2010;51(4):357-8.
  5. 5. Fede AB, Miranda Mda C, Lera AT, Ueda A, Antonangelo DV, Schaffhausser Hde L, et al. Experience with the ABC Foundation School of Medicine undergraduate meeting. Rev Assoc Med Bras. 2010;56(3):313-7.
  6. 6. Khani A, Alinezhad F, Mahdizadeh H, Banihosseini MR, Izadpanah F, Zarghami A, et al. 13th Annual Research Congress of Iran’s Medical Sciences Students. Galen Med J. 2014;3(1):59-62.
  7. 7. Alishri Gh.H, Fakhre Jahani F, Rokhsarizadeh H, Miri SM, Hollisaz MT, Hosseini SM. Evaluation of medical alumni dissertations of one of the medical sciences universities in principles of writing. J Military Med. 2010;12(2):75-9.
  8. 8. Tafreshi S, Tajbakhsh CAA. A survey on rate of using Vancouver style as bibiliographies presented in doctoral theses and post graduate theses of dentistry in shahid behshti university of medical science between 2004-2009. J Dent School Shahid Beheshti Univ Med Sci. 2010;28(1):48-56.
  9. 9. Rostampour F, Jafari M, Seifpanahi-sha’bani H, Moradi Y. The Advantages of Holding Local Medical Students’ Congresses. Thrita J Med Sci. 2012;1(3):109-10.
  10. 10. Gupta D, Mazumdar A, Singh J, Stellini M. Conference tourism or tourists’ conference for students of medicine: a perspective. J Palliat Med. 2013;16(1):7-8.
  11.  

Refbacks

  • There are currently no refbacks.