From 0c41f3f140500710094d4bacc89f700d5b380a13 Mon Sep 17 00:00:00 2001 From: Daniel Miessler Date: Tue, 19 Mar 2024 18:55:22 -0700 Subject: [PATCH] Improved analyze_paper. --- patterns/analyze_paper/system.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/patterns/analyze_paper/system.md b/patterns/analyze_paper/system.md index 0087f50..fa94140 100644 --- a/patterns/analyze_paper/system.md +++ b/patterns/analyze_paper/system.md @@ -64,14 +64,25 @@ Known [------7---] Novel Weak: [----5-----] Rigorous Theoretical [--------9-] Empirical -- Total Rating: A 1 - 10 rating for the paper's overall quality, which is the LOWEST of the Novelty and Rigor ratings. +- Final Score: + +- A - F based on the scores above, conflicts of interest, and the overall quality of the paper. Give a 15-word explanation for the grade. + # RATING NOTES - If the paper makes claims and presents stats but doesn't show how it arrived at these stats, then the Methodology Transparency would be low, and the RIGOR score should be lowered as well. +- An A would be a paper that is novel, rigorous, empirical, and has no conflicts of interest. + +- A paper could get an A if it's theoretical but everything else would have to be perfect. + - The stronger the claims the stronger the evidence needs to be, as well as the transparency into the methodology. If the paper makes strong claims, but the evidence or transparency is weak, then the RIGOR score should be lowered. +- Remove up to 1-3 grades for potential conflicts of interest indicated in the report. + + + # OUTPUT INSTRUCTIONS - Output all sections.