IEEE Transactions on Power Delivery

Discussion on How to Improve the New Review Scheme

PWRD editorial board welcomes the suggestions, comments, and feedback from the reader, author and reviewer community of PWRD. Please send your comments to EIC or editors. Comments of general interest to the community will be posted here (with the agreement of the writer and with minor editing to hide information that may identify the writer). If applicable, responses will be included as well.

===============

I would like to give you a few very brief remarks about the new IEEE PES TPDel review form. Basically, I am not very keen on it at all. I certainly think that there could be improvements in the review form that we have used for many years, but I am not sold on the new form.

There really is no place to give a coordinated response on the paper under review. That is, there is no place to give an integrated discussion of positives and negatives, or a listing of what is missing, or what is not supported in the paper. Many of these papers seem to be mathematical solutions to non-problems, and many show a very low level of understanding of basic power engineering. In the new form, it might be possible to document those sentiments – but where – and should the comments be repeated in each focused block? I am rather concerned that the scoring is a problem as well because a paper that is simply unsuitable for the Transactions could score rather well in several of the focused blocks, and this could surface as an ‘accept’. The scoring criteria and categories are selectively pedantic.

As we all know, there is probably no fool-proof review form. But I thought that I would give my two cents of comments on the new form. It would be interesting to see what others think – and perhaps how the accept / RR / reject data seem to show any advantages or disadvantages over the old form. I am sure that the editors have seen horribly prepared reviews, and the use of the new form is a reaction to correct this. But I did want to raise the issues above to suggest return to the old review form.

I hope that you will excuse this email – it is not a complaint! You guys are ‘angels’ and no one should complain about volunteers.

EIC Comments: The above comments refer to the early version of the review scheme. The scheme has been improved per the above and other feedbacks.

===============

Thanks for your e-mail. I did just go through the process of entering a revised paper. The process wasn’t too bad. As you mentioned in point 2, it did seem unusual that you were asking the authors to rate their own paper. I agree, that should be taken away and simply support comments encouraged. The organization of the decision letter was okay. Perhaps you could format it a little differently so that things like “Comments from the Editor” are more obvious (we almost missed it). Bolding and possibly underlining the sentences indicating “Comments from the Editor”, “Reviewer 1”, “Reviewer 2”, etc and the various reviewers would help make it stand out more. I hope that helps.

===============

I liked your view and find the review system useful after reviewing a paper recently. It may take more time from the reviewer but the Author’s would be beneficial. The only concern is that to track the total number of paper review request going to individual reviewers from different associate Editors. This number should be reasonable. If this number is not tracked, monitored, and evaluated properly at central level PWRD may find difficulty to get reviewers at some stage.

Inspite of that I sincerely appreciate your efforts in improving the reviewing system which is a novel decision, in my opinion. Other IEEE Transactions should come out something like this or may absorb the same model.

===============

Thank you for the email and also for the genuine care you are demonstrating to improve the PWRD. I in fact used the new system for a recent review and as such I can provide first-hand feedback on its working.

If you recall I wrote to you a couple of years ago when you first announced that an information session for PWRD authors and reviewers was to be held in the PES GM. Since then I have continued to review significantly for the publication and most recently I have been a Guest Editor for the Special Issue on XXXX. Therefore, I believe that I can provide feedback that is hopefully meaningful and of some use.

As you have correctly pointed out the new system is somewhat problematic in its current form. I did find it difficult and rigid to put my comments and as such I had to attach my list of comments so that they can be used by the authors. In general, however, I am not sure if a rubric-based system is a viable approach. The reason for it, in my opinion, is that it attempts to measure the quality of a research publication in the same way a course exam is evaluated. These two entities are, however, vastly different. In the long-run, I can envision that the authors will try to massage their submissions in a way to get past the rubrics if they want to, and that, I am sure, is not the direction you want the publication to go to.

I indicated in my earlier email that one of the main problems with the way reviews are handled is due to the large number of submissions that enter the review system. The number of submissions is an uncontrolled variable, but the number that actually enters the review process is a controlled one, or at least it can become one. I had indicated that the Associate Editors must play a significant role in pre-screening the publication to stop obviously poor publications from entering and clogging the review system. If a smaller number of good quality papers are distributed, then I am confident that the quality of reviews and their time-around time will improve naturally.

Let me give an example of a recent paper that I reviewed. I was asked to review a paper written by a seemingly prominent group of authors. Upon a quick examination of the paper, I realized that the language, grammar, and style in the paper were quite poor. Despite this, I proceeded (with a great deal of difficulty to decipher what the authors had said) only to find out obvious and basic mistakes in the paper starting from the very first equation presented. It took several hours to finish the paper and confirm that it was indeed a poor submission. Upon receipt of the review results I was glad to see that the other reviewers shared the same feeling.

It would have been easy to spot and remove this paper from the list if the AEs were given that mandate and were allowed to make such decision and return the paper to the authors without wasting many reviewers’ time to do so. The current scheme leads to reviewer frustration and many rounds of reviews to fix problems that must not be even an issue in a decent submission.

===============

I would like to give a suggestion: clarify accurately (for yourself) the objective of a change, before to implement it.

For example, in the recent changes, the need of them does not seem to be very clear. From my point of view, the system was working very well, and it was simpler. Maybe, for this type of systems, where many people is involved, it is advisable to follow the well-known phrase: “if something works, don’t change it”.

EIC comment: This is a good suggestion. It has prompted a lot of discussions. The discussions are documented at location: COMMUNITY FEEDBACK – Improving Current Practice.