======================================================================== Plaintext Review Forms - Instructions ======================================================================== Please complete your review using the following guidelines for the scored categories. Appropriateness (1-5) --------------------- Does this paper fit in the conference? Both empirical and theoretical results are welcome. 5 = Certainly. 4 = Probably. 3 = Unsure. 2 = Probably not. 1 = Certainly not. Clarity (1-5) ------------- For the reasonably well-prepared reader, is it clear what was done and why? Is the paper well-written and well-structured? 5 = Very clear. 4 = Understandable by most readers. 3 = Mostly understandable to me with some effort. 2 = Important questions were hard to resolve even with effort. 1 = Much of the paper is confusing. Originality / Innovativeness (1-5) ---------------------------------- How original is the approach? Does this paper break new ground in topic, methodology, or content? How exciting and innovative is the research it describes? Note that a paper could score high for originality even if the results do not show a convincing benefit. 5 = Surprising: Noteworthy new problem, technique, methodology, or insight. 4 = Creative: Relatively few people in our community would have put these ideas together. 3 = Somewhat conventional: A number of people could have come up with this if they thought about it for a while. 2 = Rather boring: Obvious, or a minor improvement on familiar techniques. 1 = Significant portions have actually been done before or done better. Soundness/Correctness (1-5) --------------------------- First, is the technical approach sound and well-chosen? Second, can one trust the claims of the paper -- are they supported by proper experiments and are the results of the experiments correctly interpreted? 5 = The approach is very apt, and the claims are convincingly supported. 4 = Generally solid work, although there are some aspects of the approach or evaluation I am not sure about. 3 = Fairly reasonable work. The approach is not bad, and at least the main claims are probably correct, but I am not entirely ready to accept them (based on the material in the paper). 2 = Troublesome. There are some ideas worth salvaging here, but the work should really have been done or evaluated differently. 1 = Fatally flawed. Meaningful Comparison (1-5) --------------------------- Does the author make clear where the problems and methods sit with respect to existing literature? Are the references adequate? Are the experimental results meaningfully compared with the best prior approaches? 5 = Precise and complete comparison with related work. Good job given the space constraints. 4 = Mostly solid bibliography and comparison, but there are some references missing. 3 = Bibliography and comparison are somewhat helpful, but it could be hard for a reader to determine exactly how this work relates to previous work. 2 = Only partial awareness and understanding of related work, or a flawed empirical comparison. 1 = Little awareness of related work, or lacks necessary empirical comparison. Substance (1-5) --------------- Does this paper have enough substance, or would it benefit from more ideas or results? Note that this question mainly concerns the amount of work; its quality is evaluated in other categories. 5 = Contains more ideas or results than most publications in this conference; goes the extra mile. 4 = Represents an appropriate amount of work for a publication in this conference. (most submissions) 3 = Leaves open one or two natural questions that should have been pursued within the paper. 2 = Work in progress. There are enough good ideas, but perhaps not enough results yet. 1 = Seems thin. Not enough ideas here for a full-length paper. Recommendation (1-5) -------------------- Should the paper be accepted or rejected? 5 = Exciting: I'd fight to get it accepted 4 = Worthy: I would like to see it accepted 3 = Borderline: I'm ambivalent about this one 2 = Mediocre: I'd rather not see it in the conference 1 = Bad: I'd fight to get it rejected Suggested Presentation Type --------------------------- A paper in conference Y can be presented either as poster or as oral presentations. If this paper were selected for presentation, which form of presentation would you find more appropriate? Note that the decisions as to which papers will be presented orally and which as poster presentations will be based on the nature rather than on the quality of the work. An oral paper is potentially to attract more audiences. Reviewer Confidence (1-5) ------------------------- 5 = Positive that my evaluation is correct. I read the paper very carefully and am very familiar with related work. 4 = Quite sure. I tried to check the important points carefully, and I think I know the related work. It's unlikely, though conceivable, that I missed something that should affect my ratings. 3 = Pretty sure, but there's a chance I missed something. I did not carefully check the details, e.g., the math or the experimental design. 2 = Willing to defend evaluation, but it is fairly likely that I missed something or didn't completely understand some central points. 1 = Not my area, or paper is very hard to understand. My evaluation is just an educated guess. ======================================================================== Review Form - Submission #12345 =======================================DZE3CTX9=102===================== Title: The Conference Y Main Topic: Semantics ------------------------------------------------------------------------- SCORED EVALUATION ------------------------------------------------------------------------- ==================================================================== Appropriateness =================================================9YEDC25=S=1======== Insert your evaluation directly following the colon (:) sign. Please select an INTEGER (whole-number) score from within the range specified. The form has been initialized with the minimum score as a default. Replace this default with your own evaluation. Appropriateness (1 - 5): 1 =7XD3914=S=1 ==================================================================== Clarity =================================================9YEDC25=S=2======== Clarity (1 - 5): 1 =7XD3914=S=2 ==================================================================== Originality / Innovativeness =================================================9YEDC25=S=3======== Originality / Innovativeness (1 - 5): 1 =7XD3914=S=3 ==================================================================== Soundness/Correctness =================================================9YEDC25=S=4======== Soundness/Correctness (1 - 5): 1 =7XD3914=S=4 ==================================================================== Meaningful Comparison =================================================9YEDC25=S=5======== Meaningful Comparison (1 - 5): 1 =7XD3914=S=5 ==================================================================== Substance =================================================9YEDC25=S=6======== Substance (1 - 5): 1 =7XD3914=S=6 ==================================================================== Recommendation =================================================9YEDC25=S=7======== Recommendation (1 - 5): 1 =7XD3914=S=7 ==================================================================== Suggested Presentation Type =================================================9YEDC25=S=8======== Place an 'x' mark in the box next to your selection for this category. =X=0 [x]: oral =X=1 [ ]: poster =7XD3914=S=8 ==================================================================== Reviewer Confidence =================================================9YEDC25=S=9======== Reviewer Confidence (1 - 5): 1 =7XD3914=S=9 ------------------------------------------------------------------------- DETAILED COMMENTS ------------------------------------------------------------------------- Please supply detailed comments to back up your rankings. These comments will be forwarded to the authors of the paper. Enter your comments after the "Start" boundary line. Use as many lines as you need, but make sure that the "End" boundary marker directly follows the last line of your detailed comments. =Start==============================================9YEDC25=comments===== =End================================================7XD3914=comments===== ------------------------------------------------------------------------- CONFIDENTIAL COMMENTS FOR COMMITTEE ------------------------------------------------------------------------- You may wish to withhold some comments from the authors, and include them solely for the committee's internal use. You may enter your confidential comments in the area below (using as many lines as you need). If you don't have confidential comments for this submission, leave the area blank. =Start==========================================9YEDC25=confComments===== =End============================================7XD3914=confComments===== ======================================================================== End Review Form - Submission #12345 =======================================CX39VB8D=102=====================