Ccf Idea Reviewer

CCF Idea Reviewer

Invocation Controls

CCFA Handoff Mode: PARTIAL (Recommended). Follow metadata.ccf_skill_controls.handoff_question_mode and ../ccf-common/references/handoff-modes.md. Use ../ccf-common/references/routing.md to keep idea scoring separate from idea optimization, manuscript writing, conference scientific review, paper writing review, and rebuttal tasks.

Use this skill only for explicit idea review/scoring/ranking. If the user asks to optimize, develop, concretize, brainstorm, or reshape an idea without asking for scoring or strict review, route to ccf-idea-optimizer or ccf-pipeline-orchestrator by the shared routing rules.

Load ../ccf-common/references/task-modes.md before deciding exploratory, quick, or standard mode. If the user asks for exploration, rescue, brainstorming, or "这个方向还能怎么做" rather than scoring, route to ccf-idea-optimizer. Use quick mode for a short local triage note or one idea's preliminary risk scan. Use standard mode for numeric scoring, multi-idea ranking, novelty-risk diagnosis, target-venue/journal judgment, or investment decisions.

If the user says not to use, disable, skip, or avoid a sibling skill, do not invoke or simulate that skill for the rest of the conversation. Use this skill's local fallback instead: stage-aware verdict, serious-risk list, fixability table, and upgrade/pivot actions without cross-skill execution.

Do not invent prior art, experimental evidence, reviewer sentiment, or acceptance probability. Separate low novelty from unknown novelty, and report confidence independently from score.

Load ../ccf-common/references/review-output-standards.md whenever producing numeric scores, multi-expert panel notes, score-change conditions, or standard-mode reports.

Treat rough ideas, unpublished method details, draft abstracts, and experiment plans as private user data. Load ../ccf-common/references/privacy-and-evidence.md before browsing, using private text in a query, or making evidence/provenance claims.

Core Rule

Review the problem and method only. Do not review prose, section structure, LaTeX, or rebuttal strategy. Act as a strict target-venue or target-journal reviewer: identify the real insight, subtract closest prior work, attack weak mechanisms, and convert every serious deduction into a concrete repair or pivot condition.

No generic reviewer filler is allowed. Every material criticism must name the exact idea claim or mechanism, the evidence or closest work behind the concern, why a strict reviewer would deduct, and what change would alter the score.

Stage awareness is mandatory. A rough seed can score low on current conference readiness while still having good development potential. Do not translate "not ready", "novelty unsearched", "mechanism under-specified", or "overcrowded area" into abandon. Use abandon only when there is no testable central claim and no plausible reformulation after naming at least one attempted rescue route.

Mandatory Review Checklist

In standard mode, complete this checklist before any idea score, ranking, or recommendation. In quick mode, run the local subset and return a compact status.

1. Target venue/journal, venue family, field, and assumptions are explicit. 2. Each idea is normalized into problem, gap, root challenge, insight, method mechanism, expected evidence, and limitation. 3. Literature status is labeled as searched, partially searched, user-provided only, or not searched. 4. Standard mode has a closest-work table from public-safe literature search unless the user forbids browsing or privacy prevents safe queries. 5. Multi-expert roles are applied independently: field expert, method expert, experiment expert, AC/venue expert, and skeptical prior-art expert. Add ethics, systems, theory, user-study, security, or dataset roles only when relevant. 6. All rubric dimensions are scored on 1-5 or marked not applicable with a reason; every score of 3 or below includes deduction, evidence basis, repair condition, and expected score movement. 7. Fatal risks are separated from fixable weaknesses and local refinements. 8. Confidence is reported separately from score. 9. The final recommendation is one of: accept-to-develop, revise, pivot-with-rescue-route, abandon, or needs-literature-search. abandon is reserved for no plausible reformulation, not ordinary weakness. 10. Any optional module transition to ccf-literature-monitor, ccf-literature-searcher, ccf-idea-optimizer, ccf-experiment-designer, or ccf-paper-writer follows the CCFA handoff mode; if denied or disabled, output upgrade actions only.

Load references/strict-idea-review.md in standard mode or whenever the user asks for strict review, novelty diagnosis, or literature-backed scoring. Load references/rubric.md whenever producing numeric scores.

Workflow

1. Identify the user's decision: score one idea, rank several ideas, choose a venue, diagnose rejection risk, check whether the idea is stale, or decide whether to invest. 2. Normalize each idea into a problem-method card. If the idea is too vague to score, infer only fields that are obvious from context and mark the rest as assumptions; ask only when the missing field would change the verdict. 3. Map the venue family and review lens. If the user names a current venue policy, page rule, review form, or target-year criterion, verify the official source before applying it. 4. In standard mode, load references/strict-idea-review.md and search closest literature with public-safe queries. Prefer ccf-literature-searcher when allowed by handoff mode; otherwise use local browsing and the shared source policy. Apply source-quality exclusions. If not searched, cap novelty confidence and state the exact uncertainty. 5. Build the closest-work table and novelty delta before scoring. A novelty claim must say what remains after subtracting the nearest prior work. 6. Load ../ccf-common/references/review-output-standards.md and references/expert-panel.md; produce independent notes from the required expert roles. Each role must include a concrete rejection-grade concern or a reason no such concern exists. Do not force disagreement, praise, or rejection. 7. Load references/rubric.md and score the 10 dimensions. Use references/calibration.md to aggregate, calibrate confidence, apply fatal gates, and choose a recommendation. Include score-change conditions rather than unsupported acceptance probabilities. 8. Convert every major weakness into an upgrade action. Label actions as problem-refinement, method-redesign, novelty-grounding, evidence-design, feasibility-check, venue-switch, pivot, or abandon. For every idea that is not accept-to-develop, include the smallest change that could raise development potential. 9. For multiple ideas, rank by serious-risk-adjusted score after the literature and mechanism scan. Prefer ideas with fewer severe novelty/soundness/evidence risks even if they are less fashionable. If all ideas score low, still identify the best salvageable ingredient in each and the single most promising rescue path overall. 10. If the idea is viable but underdeveloped, follow the CCFA handoff mode before using ccf-idea-optimizer for targeted repair. If the evidence package is the main weakness, follow handoff mode before using ccf-experiment-designer. If a module is denied, stop at verdict, risks, and action queue.

Output Contracts

For one idea in standard mode, return:

Verdict first:
Target venue/journal and assumptions:
Search basis:
Stage and development potential:
Normalized idea:
Closest prior art table:
Novelty delta:
Serious blockers:
Per-expert reviewer notes:
Rubric scores:
Weighted final score:
Score-change conditions:
Confidence:
Strict reviewer comments:
Fixability table:
Upgrade / pivot actions:
Evidence that would change my score:
Recommendation:
Optional next-module decision:
Checklist status:
Output self-check:

For quick mode, return a compact version:

Quick verdict:
Likely strongest point:
Top 3 rejection risks:
Novelty confidence:
Development potential:
What to search next:
Immediate repair actions:

For multiple ideas, return:

Venue lens:
Normalized candidates:
Search basis by candidate:
Ranking:
Score table:
Fatal risks by idea:
Best idea to develop:
Ideas to pivot, rescue, or abandon:
Upgrade plan for the winner:

Reference Files

Load only what is needed:

• references/strict-idea-review.md: Use for standard-mode search, closest-work comparison, anti-filler rules, strict verdicts, and score caps.
• references/expert-panel.md: Use for role-specific reviewer perspectives and comments.
• references/rubric.md: Use for dimensions, weights, and 1-5 scoring anchors.
• references/calibration.md: Use for weighted scores, fatal gates, confidence, recommendations, and multi-idea tournaments.
• references/source-notes.md: Use when explaining provenance, official criteria, or current literature/venue checks.
• ../ccf-common/references/review-output-standards.md: Use for quantitative feedback, independent panel discipline, score-change conditions, and final output self-check.

If the user asks to improve the idea rather than score it, follow the CCFA handoff mode before switching to ccf-idea-optimizer unless explicitly named. If the user asks mainly for recent-paper monitoring or competitor tracking, route to ccf-literature-monitor; if the user asks for deep related work, route to ccf-literature-searcher. If the user provides a full paper draft and asks for paper review, follow the CCFA handoff mode before switching to ccf-paper-reviewer; if not confirmed, provide only idea-stage caveats.

Calibration

Use this file after scoring dimensions.

Weighted Score

Compute:

weighted_score = sum(score_i * weight_i) / 100

Report the result on a 1-5 scale and optionally convert to a 100-point table for readability.

Recommendation Bands

• 4.3-5.0: accept-to-develop. Develop aggressively only after closest-work comparison and evidence planning support the central claim.
• 3.7-4.2: revise. Promising, but one or two named score blockers must be fixed.
• 3.0-3.6: pivot-with-rescue-route. Keep the best ingredient but change problem, method, evidence, or venue.
• 1.0-2.9: high-risk reformulate by default; use abandon only if no testable claim and no plausible rescue route remain.
• Any score with high novelty uncertainty: needs-literature-search if novelty is decision-critical.

Fatal Gates

Apply these after the weighted score:

• Novelty <= 2 with high confidence: cap recommendation at pivot-with-rescue-route.
• Novelty unknown and no closest-work search: cap recommendation at needs-literature-search unless the user asked for quick mode only.
• Method soundness <= 2: cap recommendation at pivot-with-rescue-route.
• Experimental convincibility <= 2 and no alternative proof/study/system evidence: cap at pivot-with-rescue-route.
• Venue fit <= 2: recommend venue switch or pivot.
• Feasibility <= 2 under fixed deadline: cap at pivot-with-rescue-route; use abandon only if constraints make every rescue route infeasible.
• No identifiable insight beyond applying known modules: cap recommendation at pivot-with-rescue-route.
• Internal mechanism conflict: cap recommendation at pivot-with-rescue-route until resolved.

Confidence Labels

Use:

• High: close work, venue fit, and evidence feasibility are well grounded.
• Medium: most inputs are available but literature or resources remain partially uncertain.
• Low: idea is vague, venue is unknown, or closest prior work was not checked.

Do not lower the idea score merely because confidence is low. Lower confidence separately and recommend the next check.

Development Potential

Report development potential separately from the weighted score:

• High: a clear rescue route could make the idea competitive without changing the core problem.
• Medium: the best ingredient is useful, but the problem, mechanism, venue, or evidence path must change.
• Low: only a narrow fragment is reusable, or the user should switch to a different direction after one final check.

Do not use low development potential as a substitute for evidence. Name the exact condition that would make the direction worth another iteration or worth stopping.

Fixability Table

Issue:
Affected dimension:
Severity: high / medium / low
Fix class:
Can fix before writing? yes / partly / no
Required evidence or design change:
Risk-reduction condition:
Development-potential impact:
Owner skill: ccf-literature-searcher / ccf-idea-optimizer / ccf-experiment-designer / ccf-paper-writer / ccf-paper-reviewer

Multi-Idea Tournament

For multiple drafts:

1. Normalize every idea first. 2. Score independently before comparing. 3. Apply fatal gates. 4. Prefer the idea with the strongest fixable path, not just highest average. 5. Keep one backup idea if it has lower novelty risk or better feasibility. 6. If all ideas are weak, still return the best salvageable ingredient and one next search/design test before recommending the user stop.

Fairness Rules

• Do not reward hype without mechanism or evidence.
• Do not punish niche ideas if the target venue values depth and rigor.
• Separate "novelty unknown" from "low novelty".
• Do not require experiments for theory ideas, proofs for empirical ideas, or user studies for non-HCI ideas.
• Penalize unsupported acceptance claims, not ambitious but testable ideas.
• Do not hide a reject-level problem under "future work".
• Do not say "worth pursuing" without naming the exact condition under which it becomes worth pursuing.

Expert Panel

Use this file for role-specific idea review. Keep roles independent before aggregating.

Each expert must write as a strict reviewer, not as a generic coach. Do not use generic praise or generic concern. Every role must name a concrete claim, mechanism, closest-work risk, evidence gap, or venue criterion. After the strict concern, each role must name the smallest repair or evidence test that would change its judgment; if no repair exists, explain why.

Do not force roles to disagree, praise, or reject. A role may say that the idea is plausible on its axis, but it must state the evidence that supports that view. A role may also say insufficient evidence when the idea or search basis is too incomplete for a fair judgment.

Required Roles

Field Expert

Checks:

• Importance of the problem to the target community.
• Relationship to closest known work.
• Whether the claimed gap is real, current, and specific.
• Whether the idea teaches the field something beyond a local improvement.
• Whether the insight would still look interesting after the obvious related-work paragraph is written.

Method Expert

Checks:

• Mechanism clarity.
• Technical soundness.
• Assumptions and failure modes.
• Elegance: whether the method has necessary parts and a coherent insight.
• Whether the contribution is more than combining known modules.
• Whether any components optimize incompatible objectives or make contradictory assumptions.

Experiment Expert

Checks:

• Whether the central claim can be tested.
• Baselines, ablations, metrics, datasets, workloads, proofs, or studies.
• Feasibility under time, compute, data, and implementation constraints.
• Whether negative results or failure cases would still be informative.
• Which single missing comparison would most likely cause rejection.

AC / Venue Expert

Checks:

• Venue fit and audience.
• Whether the idea matches the venue's contribution taste.
• Desk-reject or reviewer-mismatch risk.
• Whether the idea is likely to survive discussion after mixed reviews.
• Whether the contribution would be read as main-track substance, workshop novelty, benchmark engineering, or application-only work.

Skeptical Prior-Art Expert

Checks:

• Novelty collapse risk.
• Obvious close papers, systems, benchmarks, or theory lines to search.
• Whether the new terminology hides a known method.
• Whether the idea should be marked needs-literature-search.
• The strongest "this is already known" objection a reviewer could make.

Optional Roles

Add only when relevant:

• Ethics/reproducibility expert.
• Systems builder.
• User-study expert.
• Theory/proof expert.
• Security threat-model expert.
• Dataset/benchmark curator.
• Domain practitioner.
• Resource and implementation feasibility reviewer.

Per-Expert Output

Role:
Score tendency:
Best possible argument for the idea:
Strict rejection-grade concern:
Anchor:
Why this matters:
Fatal risk if any:
Most valuable repair or pivot:
Development-potential note:
Confidence:

If a role cannot find a rejection-grade concern, say why and identify the evidence that supports that confidence. Do not fill the field with generic encouragement.

Panel Synthesis

After all independent role notes, synthesize without averaging away fatal risks:

Agreement:
Disagreement:
Strongest accept argument:
Strongest reject argument:
Score-relevant uncertainty:
Most valuable next evidence:
Panel-calibrated recommendation:

Idea Review Rubric

Use 1-5 scores for each dimension. Score only the problem and method idea, not manuscript prose.

When novelty, insight, or acceptance potential is decision-critical, apply strict-idea-review.md first. If closest work was not searched, cap novelty at 3 and mark confidence low unless the user supplies credible prior-art coverage.

Score current readiness, not the user's worth or the absolute future of the direction. For rough seeds, include a separate development-potential label. A score of 2-3 often means "not ready without redesign"; it does not automatically mean "do not pursue."

Weighted Dimensions

Weights sum to 100. Adjust only when an official venue review form makes a dimension clearly more important, and state the adjustment.

Quantitative Feedback Output

For standard idea review, output both a dimension scorecard and an aggregate score. Use:

Weighted score (1-5) = sum(dimension score * weight) / 100
Optional overall score (1-10) = weighted score * 2

Do not treat the weighted score as an acceptance probability. The score is a decision aid that must be read with fatal risks, literature-search status, and development potential.

Include this table:

Then report:

Weighted final score:
Current conference readiness:
Development potential:
Confidence:
Score-change conditions:

Every score of 3 or below must name the exact claim, missing mechanism, closest-work risk, missing evidence, or venue criterion that caused the deduction.

Score Anchors

5

Clear CCF-A-level signal. The problem is important, the insight is non-obvious, the method has a defensible mechanism, closest-work comparison leaves a meaningful novelty delta, and the evidence package can decisively test the claim.

4

Promising. The idea has a real contribution and mostly coherent mechanism, but one material gap remains in novelty grounding, method detail, feasibility, or evidence design. The gap is repairable without changing the central idea.

3

Borderline. The idea may become publishable only after substantial refinement. The current problem-method chain leaves important doubts, the novelty delta is thin or unsearched, or the insight is more incremental than the framing suggests.

2

Weak in its current form. The idea is likely incremental, under-motivated, poorly grounded, internally inconsistent, or difficult to validate. A strict reviewer would probably reject the current version unless the problem, mechanism, or differentiation is changed. Still name the most plausible repair route if one exists.

1

Fatal for the current formulation. The problem is not important for the venue, closest work likely already covers the contribution, the method is unsound, the components conflict, or the central claim is untestable. Use abandon only if no credible reformulation remains.

Dimension Guidance

• Problem importance: audience, stakes, bottleneck, and nontriviality. Penalize vague "important application" claims without a named bottleneck.
• Novelty: distance from closest work; mark uncertainty separately from low novelty. Do not score above 3 when closest work was not checked and novelty is central.
• Conceptual innovation: new insight, formulation, mechanism, benchmark, theory, or system design. Penalize pure module recombination unless the combination creates a new capability or explanation.
• Method soundness: assumptions, mechanism, feasibility, and plausible correctness. Penalize architectures whose components optimize incompatible objectives.
• Elegance: simplicity, necessity of components, and explanatory power.
• Feasibility: data, compute, implementation, timeline, and expertise.
• Experimental convincibility: ability to produce evidence that would change reviewer belief. "Run more experiments" is not enough; identify decisive baselines, ablations, datasets, proofs, systems measurements, or user studies.
• Venue fit: match to target community, track, and contribution taste.
• Timeliness: why now, relation to active questions, and risk of saturation.
• Acceptance potential: score after considering fatal risks and likely reviewer disagreement.

Deduction Format

For every score <= 3, include:

Dimension:
Deduction:
Anchor: closest work / missing mechanism / missing evidence / venue criterion / internal contradiction
Why it matters:
Repair condition:
Development-potential effect:

Source Notes

Use this file when explaining the scoring system or when current policy/literature grounding matters.

Shared Registry

The authoritative CCFA source inventory is:

../ccf-common/references/source-registry.yaml

Do not duplicate long URL lists in this skill. Add or update public source records in the shared registry, then run:

python ..\ccf-common\scripts\check_sources.py

Idea-Reviewer Use Rules

• Verify current venue policies from official conference pages when the user asks about latest rules, review forms, tracks, or deadlines.
• Use primary papers, proceedings pages, official project pages, or credible scholar pages for novelty checks.
• In standard mode, record public-safe search queries and closest-work evidence before making strong novelty claims.
• Treat unsearched prior art as uncertainty, not proof of novelty.
• Use public peer-review and research-question frameworks as scaffolds for judgment; do not copy their wording or treat them as venue-specific scoring forms.
• Follow ../ccf-common/references/privacy-and-evidence.md before using private idea text in any search query.
• Apply the shared source-quality policy unless the user explicitly asks to discuss a specific excluded paper separately.

Strict Idea Review Protocol

Use this file when the user asks for idea scoring, idea ranking, novelty risk, investment decisions, or standard mode review. The goal is to prevent generic reviewer prose.

Standard-Mode Search Requirement

In standard mode, search for related literature before giving a strong novelty, insight, or acceptance-potential judgment unless the user explicitly forbids browsing or privacy constraints prevent safe searching.

Search process:

1. Convert the idea into public-safe keywords: problem, setting, method family, claimed mechanism, dataset/benchmark, target venue family, and 2-3 likely synonyms. 2. Do not paste confidential user wording into a search query when the idea is unpublished and specific. Query generic public keywords first. If exact private wording is necessary, ask before using it. 3. Prefer authoritative and high-impact sources: official proceedings, OpenReview, arXiv only when needed for fast-moving fields, ACM/IEEE/USENIX/ACL Anthology/CVF/PMLR/NeurIPS/ICLR/AAAI pages, DBLP, Semantic Scholar, OpenAlex, and major journal publishers. Apply the shared source-quality exclusions. 4. Record 3-8 closest works or state why fewer were found. 5. Separate not searched, searched but weak coverage, and searched with closest-work confidence.

If no search was performed, the final recommendation must either be needs-literature-search or carry low novelty confidence. Do not say "high novelty" without a search-backed nearest-neighbor comparison or strong user-provided prior-art evidence.

Closest-Work Table

Include this table in standard mode:

Closest work:
Venue/year:
Link/source:
What it already does:
Overlap with user idea:
Remaining novelty delta:
Risk to the idea: fatal / high / medium / low
Needed differentiation:

No-Filler Rule

Do not output generic comments such as:

• "有一定创新性"
• "建议进一步完善"
• "需要更多实验"
• "创新性不足"
• "方法描述不够清楚"
• "related work 需要加强"

unless each phrase is immediately followed by:

Exact claim or mechanism under review:
Closest prior art or missing evidence:
Why a strict reviewer would deduct:
Concrete repair or pivot:
What would change the score:

Every major criticism must have at least one anchor:

• a searched paper or known literature line,
• a venue criterion,
• a missing mechanism,
• a missing baseline/evidence path,
• a contradiction inside the proposed idea,
• or a resource/feasibility constraint.

Harsh Reviewer Lens

Act as a strict target-venue or target-journal reviewer. Be professional, but do not soften the verdict with motivational padding. Prefer short, diagnostic sentences.

Strict does not mean terminal. For early ideas, state current conference readiness and development potential separately. A close prior-art hit, missing mechanism, or missing benchmark is a blocker for the current version, not proof that the direction cannot become publishable. Before recommending abandon, name the best possible rescue route and explain why it still fails.

Judge the idea through these axes:

1. Insight: Is there a non-obvious observation that would teach the community something, or is it a wrapper around known components? 2. Problem: Is the bottleneck important, current, and specific enough for the target venue? 3. Method mechanism: Is there a causal or algorithmic reason the method should work? 4. Novelty delta: What exactly remains new after subtracting closest work? 5. Elegance: Are the components necessary, coherent, and mutually compatible? 6. Evidence path: Can decisive experiments, proofs, user studies, benchmarks, or systems measurements test the central claim? 7. Feasibility: Can the user plausibly execute it with available data, compute, engineering effort, and timeline? 8. Venue taste: Would the venue see this as a main-track contribution or as a workshop/application variant?

Stage-Aware Verdict Rule

Use this distinction whenever the idea is a seed, direction, or partial sketch:

Current conference readiness: high / medium / low
Development potential: high / medium / low
Main reason it is not ready:
Best rescue route:
Evidence needed to decide:

Do not mark abandon for:

• novelty not yet searched,
• a crowded topic with a plausible narrower bottleneck,
• missing experiments when a decisive evidence path can be named,
• vague mechanism when a concrete mechanism family can be proposed,
• venue mismatch when another venue family may fit.

Mark abandon only when the idea has no testable central claim, the closest work already covers the same problem and mechanism, and no meaningful problem/method/evidence reframing remains.

Required Verdict Shape

For one idea in standard mode, use:

Verdict first:
Search basis:
Normalized idea:
Closest prior art table:
Novelty delta:
Serious blockers:
Development potential:
Dimension scores:
Strict reviewer comments:
Repair plan:
Evidence that would change my score:
Final recommendation:

For multiple ideas, first run the closest-work and serious-risk scan independently, then rank by serious-risk-adjusted score. Do not rank by buzzword appeal.

Score Guardrails

• Cap novelty at 3 if closest work was not searched and novelty is decision-critical.
• Cap acceptance potential at 3 if the method mechanism is unclear.
• Cap recommendation at pivot if closest prior art already solves the central problem with a similar mechanism. If a different evidence setting, assumption, user group, system constraint, or theory angle could still be meaningful, name that rescue route.
• Cap recommendation at pivot if the only novelty is applying a known method to a new dataset without a venue-valued insight.
• Mark as abandon only when the idea has no testable central claim and no plausible reformulation after the best rescue route has been considered.