
Loki Mode
- 662 installs
- 44k repo stars
- Updated July 27, 2026
- sickn33/antigravity-awesome-skills
loki-mode is a GitHub Actions agent skill that delivers structured Claude code reviews on every pull request for developers who want consistent PR feedback without leaving GitHub.
About
loki-mode packages a Claude Code Review GitHub Actions workflow that runs on pull_request events with types opened and synchronize. The job checks out code on ubuntu-latest with permissions for contents, pull-requests, issues read, and id-token write for Claude authentication. Optional filters can scope reviews to specific path globs such as src/**/*.ts or limit runs to certain PR authors and first-time contributors. Developers reach for loki-mode when they want automated, repeatable PR reviews on every update instead of manual review checklists or inconsistent bot comments. The workflow integrates directly into GitHub so review feedback appears in the PR thread developers already use. Configure path filters when only certain languages need review, or enable author filters for external contributor guardrails. loki-mode suits teams standardizing Claude-assisted review gates in CI.
- Automates Claude-powered PR reviews on open or synchronize events
- Checks code quality, bugs, performance, security, and test coverage
- Respects your CLAUDE.md for project-specific style and conventions
- Configurable to run only on external or first-time contributors
- Outputs constructive feedback directly in the GitHub PR
Loki Mode by the numbers
- 662 all-time installs (skills.sh)
- +2 installs in the week ending Jul 27, 2026 (Skillselion tracking)
- Ranked #194 of 1,382 Code Review & Quality skills by installs in the Skillselion catalog
- Security screen: HIGH risk (skills.sh audit)
- Data as of Jul 28, 2026 (Skillselion catalog sync)
npx skills add https://github.com/sickn33/antigravity-awesome-skills --skill loki-modeAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 662 |
|---|---|
| repo stars | ★ 44k |
| Security audit | 0 / 3 scanners passed |
| Last updated | July 27, 2026 |
| Repository | sickn33/antigravity-awesome-skills ↗ |
How do you automate Claude code reviews on GitHub PRs?
Get structured, consistent code reviews on every pull request using Claude without leaving GitHub.
Who is it for?
Teams wanting consistent Claude PR reviews triggered directly from GitHub Actions on every pull request update.
Skip if: Local-only pre-commit linting, non-GitHub repositories, or security penetration testing unrelated to pull request diffs.
When should I use this skill?
User wants automated Claude code review on GitHub pull requests or needs a PR review GitHub Actions workflow.
What you get
GitHub Actions workflow file, automated PR review comments, and configurable path or author filters for Claude review jobs.
- GitHub Actions workflow YAML
- Automated PR review comments
By the numbers
- Triggers on 2 pull_request types: opened and synchronize
- Example path filters cover 4 src glob patterns
Files
name: Claude Code Review
on: pull_request: types: [opened, synchronize]
Optional: Only run on specific file changes
paths:
- "src/*/.ts"
- "src/*/.tsx"
- "src/*/.js"
- "src/*/.jsx"
jobs: claude-review:
Optional: Filter by PR author
if: |
github.event.pull_request.user.login == 'external-contributor' ||
github.event.pull_request.user.login == 'new-developer' ||
github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR'
runs-on: ubuntu-latest permissions: contents: read pull-requests: read issues: read id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4 with: fetch-depth: 1
- name: Run Claude Code Review
id: claude-review uses: anthropics/claude-code-action@v1 with: claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} prompt: | REPO: ${{ github.repository }} PR NUMBER: ${{ github.event.pull_request.number }}
Please review this pull request and provide feedback on:
- Code quality and best practices
- Potential bugs or issues
- Performance considerations
- Security concerns
- Test coverage
Use the repository's CLAUDE.md for guidance on style and conventions. Be constructive and helpful in your feedback.
Use gh pr comment with your Bash tool to leave your review as a comment on the PR.
See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
or https://code.claude.com/docs/en/cli-reference for available options
claude_args: '--allowed-tools "Bash(gh issue view:),Bash(gh search:),Bash(gh issue list:),Bash(gh pr comment:),Bash(gh pr diff:),Bash(gh pr view:),Bash(gh pr list:*)"'
name: Claude Code
on: issue_comment: types: [created] pull_request_review_comment: types: [created] issues: types: [opened, assigned] pull_request_review: types: [submitted]
jobs: claude: if: | (github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) || (github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) || (github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) || (github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude'))) runs-on: ubuntu-latest permissions: contents: read pull-requests: read issues: read id-token: write actions: read # Required for Claude to read CI results on PRs steps:
- name: Checkout repository
uses: actions/checkout@v4 with: fetch-depth: 1
- name: Run Claude Code
id: claude uses: anthropics/claude-code-action@v1 with: claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
This is an optional setting that allows Claude to read CI results on PRs
additional_permissions: | actions: read
Optional: Give a custom prompt to Claude. If this is not specified, Claude will perform the instructions specified in the comment that tagged it.
prompt: 'Update the pull request description to include a summary of changes.'
Optional: Add claude_args to customize behavior and configuration
See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
or https://code.claude.com/docs/en/cli-reference for available options
claude_args: '--allowed-tools Bash(gh pr:*)'
name: Release
on: push: paths:
- 'VERSION'
branches:
- main
jobs: release: runs-on: ubuntu-latest permissions: content
name: Claude Code Review
on:
pull_request:
types: [opened, synchronize]
# Optional: Only run on specific file changes
# paths:
# - "src/**/*.ts"
# - "src/**/*.tsx"
# - "src/**/*.js"
# - "src/**/*.jsx"
jobs:
claude-review:
# Optional: Filter by PR author
# if: |
# github.event.pull_request.user.login == 'external-contributor' ||
# github.event.pull_request.user.login == 'new-developer' ||
# github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR'
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Code Review
id: claude-review
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
prompt: |
REPO: ${{ github.repository }}
PR NUMBER: ${{ github.event.pull_request.number }}
Please review this pull request and provide feedback on:
- Code quality and best practices
- Potential bugs or issues
- Performance considerations
- Security concerns
- Test coverage
Use the repository's CLAUDE.md for guidance on style and conventions. Be constructive and helpful in your feedback.
Use `gh pr comment` with your Bash tool to leave your review as a comment on the PR.
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://code.claude.com/docs/en/cli-reference for available options
claude_args: '--allowed-tools "Bash(gh issue view:*),Bash(gh search:*),Bash(gh issue list:*),Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr list:*)"'
name: Claude Code
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
issues:
types: [opened, assigned]
pull_request_review:
types: [submitted]
jobs:
claude:
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
actions: read # Required for Claude to read CI results on PRs
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Code
id: claude
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# This is an optional setting that allows Claude to read CI results on PRs
additional_permissions: |
actions: read
# Optional: Give a custom prompt to Claude. If this is not specified, Claude will perform the instructions specified in the comment that tagged it.
# prompt: 'Update the pull request description to include a summary of changes.'
# Optional: Add claude_args to customize behavior and configuration
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://code.claude.com/docs/en/cli-reference for available options
# claude_args: '--allowed-tools Bash(gh pr:*)'
name: Release
on:
push:
paths:
- 'VERSION'
branches:
- main
jobs:
release:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Read version
id: version
run: |
VERSION=$(cat VERSION | tr -d '\n')
echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "tag=v$VERSION" >> $GITHUB_OUTPUT
- name: Check if tag exists
id: check_tag
run: |
if git rev-parse "v${{ steps.version.outputs.version }}" >/dev/null 2>&1; then
echo "exists=true" >> $GITHUB_OUTPUT
else
echo "exists=false" >> $GITHUB_OUTPUT
fi
- name: Create release artifacts
if: steps.check_tag.outputs.exists == 'false'
run: |
mkdir -p release
# ============================================
# Artifact 1: loki-mode.zip (for Claude.ai website)
# SKILL.md at ROOT level for direct upload
# ============================================
mkdir -p release/skill-root
cp SKILL.md release/skill-root/
cp -r references release/skill-root/
cd release/skill-root
zip -r ../loki-mode-${{ steps.version.outputs.version }}.zip .
cd ../..
# Also create .skill file (same as zip, different extension)
cp release/loki-mode-${{ steps.version.outputs.version }}.zip release/loki-mode-${{ steps.version.outputs.version }}.skill
# ============================================
# Artifact 2: loki-mode-api.zip (for console.anthropic.com)
# SKILL.md inside loki-mode/ folder (API requires folder wrapper)
# ============================================
mkdir -p release/api-package/loki-mode
cp SKILL.md release/api-package/loki-mode/
cp -r references release/api-package/loki-mode/
cd release/api-package
zip -r ../loki-mode-api-${{ steps.version.outputs.version }}.zip loki-mode
cd ../..
# ============================================
# Artifact 3: loki-mode-claude-code.zip
# For Claude Code: full package with loki-mode/ folder
# Extract to ~/.claude/skills/
# ============================================
mkdir -p release/loki-mode
cp SKILL.md release/loki-mode/
cp README.md release/loki-mode/
cp LICENSE release/loki-mode/ 2>/dev/null || true
cp VERSION release/loki-mode/
cp CHANGELOG.md release/loki-mode/
cp -r references release/loki-mode/
cp -r examples release/loki-mode/
cp -r tests release/loki-mode/
cp -r scripts release/loki-mode/
cp -r autonomy release/loki-mode/
cd release
zip -r loki-mode-claude-code-${{ steps.version.outputs.version }}.zip loki-mode
tar -czvf loki-mode-claude-code-${{ steps.version.outputs.version }}.tar.gz loki-mode
cd ..
- name: Create Git Tag
if: steps.check_tag.outputs.exists == 'false'
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git tag -a "v${{ steps.version.outputs.version }}" -m "Release v${{ steps.version.outputs.version }}"
git push origin "v${{ steps.version.outputs.version }}"
- name: Extract changelog for this version
if: steps.check_tag.outputs.exists == 'false'
id: changelog
run: |
VERSION="${{ steps.version.outputs.version }}"
CHANGELOG=$(awk "/^## \[$VERSION\]/{flag=1; next} /^## \[/{flag=0} flag" CHANGELOG.md)
if [ -z "$CHANGELOG" ]; then
CHANGELOG="Release v$VERSION"
fi
echo "$CHANGELOG" > changelog_body.txt
- name: Create GitHub Release
if: steps.check_tag.outputs.exists == 'false'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh release create "v${{ steps.version.outputs.version }}" \
release/loki-mode-${{ steps.version.outputs.version }}.zip \
release/loki-mode-${{ steps.version.outputs.version }}.skill \
release/loki-mode-api-${{ steps.version.outputs.version }}.zip \
release/loki-mode-claude-code-${{ steps.version.outputs.version }}.zip \
release/loki-mode-claude-code-${{ steps.version.outputs.version }}.tar.gz \
--title "Loki Mode v${{ steps.version.outputs.version }}" \
--notes-file changelog_body.txt
- name: Skip message
if: steps.check_tag.outputs.exists == 'true'
run: |
echo "Tag v${{ steps.version.outputs.version }} already exists. Skipping release."
.DS_Store
Acknowledgements
Loki Mode stands on the shoulders of giants. This project incorporates research, patterns, and insights from the leading AI labs, academic institutions, and practitioners in the field.
---
Research Labs
Anthropic
Loki Mode is built for Claude and incorporates Anthropic's cutting-edge research on AI safety and agent development.
| Paper/Resource | Contribution to Loki Mode |
|---|---|
| Constitutional AI: Harmlessness from AI Feedback | Self-critique against principles, revision workflow |
| Building Effective Agents | Evaluator-optimizer pattern, parallelization, routing |
| Claude Code Best Practices | Explore-Plan-Code workflow, context management |
| Simple Probes Can Catch Sleeper Agents | Defection probes, anomaly detection patterns |
| Alignment Faking in Large Language Models | Monitoring for strategic compliance |
| Visible Extended Thinking | Thinking levels (think, think hard, ultrathink) |
| Computer Use Safety | Safe autonomous operation patterns |
| Sabotage Evaluations | Safety evaluation methodology |
| Effective Harnesses for Long-Running Agents | One-feature-at-a-time pattern, Playwright MCP for E2E |
| Claude Agent SDK Overview | Task tool, subagents, resume parameter, hooks |
Google DeepMind
DeepMind's research on world models, hierarchical reasoning, and scalable oversight informs Loki Mode's architecture.
| Paper/Resource | Contribution to Loki Mode |
|---|---|
| SIMA 2: Generalist AI Agent | Self-improvement loop, reward model training |
| Gemini Robotics 1.5 | Hierarchical reasoning (planner + executor) |
| Dreamer 4: World Model Training | Simulation-first testing, safe exploration |
| Genie 3: World Models | World model architecture patterns |
| Scalable AI Safety via Doubly-Efficient Debate | Debate-based verification for critical changes |
| Human-AI Complementarity for Amplified Oversight | AI-assisted human supervision |
| Technical AGI Safety Approach | Safety-first agent design |
OpenAI
OpenAI's Agents SDK and deep research patterns provide foundational patterns for agent orchestration.
| Paper/Resource | Contribution to Loki Mode |
|---|---|
| Agents SDK Documentation | Tracing spans, guardrails, tripwires |
| A Practical Guide to Building Agents | Agent architecture best practices |
| Building Agents Track | Development patterns, handoff callbacks |
| AGENTS.md Specification | Standardized agent instructions |
| Introducing Deep Research | Adaptive planning, backtracking |
| Deep Research System Card | Safety considerations for research agents |
| Introducing o3 and o4-mini | Reasoning model guidance |
| Reasoning Best Practices | Extended thinking patterns |
| Chain of Thought Monitoring | Reasoning trace monitoring |
| Agent Builder Safety | Safety patterns for agent builders |
| Computer-Using Agent | Computer use patterns |
| Agentic AI Foundation | Industry standards, interoperability |
Amazon Web Services (AWS)
AWS Bedrock's multi-agent collaboration patterns inform Loki Mode's routing and dispatch strategies.
| Paper/Resource | Contribution to Loki Mode |
|---|---|
| Multi-Agent Orchestration Guidance | Three coordination mechanisms, architectural patterns |
| Bedrock Multi-Agent Collaboration | Supervisor mode, routing mode, 10-agent limit |
| Multi-Agent Collaboration Announcement | Intent classification, selective context sharing |
| AgentCore for SRE | Gateway, Memory, Identity, Observability components |
Key Pattern Adopted: Routing Mode Optimization - Direct dispatch for simple tasks (lower latency), supervisor orchestration for complex tasks (full coordination).
---
Academic Research
Multi-Agent Systems
| Paper | Authors/Source | Contribution |
|---|---|---|
| Multi-Agent Collaboration Mechanisms Survey | arXiv 2501.06322 | Collaboration structures, coopetition |
| CONSENSAGENT: Anti-Sycophancy Framework | ACL 2025 Findings | Blind review, devil's advocate |
| GoalAct: Hierarchical Execution | arXiv 2504.16563 | Global planning, skill decomposition |
| A-Mem: Agentic Memory System | arXiv 2502.12110 | Zettelkasten-style memory linking |
| Multi-Agent Reflexion (MAR) | arXiv 2512.20845 | Structured debate, persona-based critics |
| Iter-VF: Iterative Verification-First | arXiv 2511.21734 | Answer-only verification, Markovian retry |
Evaluation & Safety
| Paper | Authors/Source | Contribution |
|---|---|---|
| Assessment Framework for Agentic AI | arXiv 2512.12791 | Four-pillar evaluation framework |
| Measurement Imbalance in Agentic AI | arXiv 2506.02064 | Multi-dimensional evaluation axes |
| Demo-to-Deployment Gap | Stanford/Harvard | Tool reliability vs tool selection |
---
Industry Resources
Tools & Frameworks
| Resource | Contribution |
|---|---|
| NVIDIA ToolOrchestra | Efficiency metrics, three-reward signal framework, dynamic agent selection |
| LerianStudio/ring | Subagent-driven-development pattern |
| Awesome Agentic Patterns | 105+ production patterns catalog |
Best Practices Guides
| Resource | Contribution |
|---|---|
| Maxim AI: Production Multi-Agent Systems | Correlation IDs, failure handling |
| UiPath: Agent Builder Best Practices | Single-responsibility agents |
| GitHub: Speed Without Control | Static analysis + AI review, guardrails |
---
Hacker News Community
Battle-tested insights from practitioners deploying agents in production.
Discussions
| Thread | Key Insight |
|---|---|
| What Actually Works in Production for Autonomous Agents | "Zero companies without human in the loop" |
| Coding with LLMs in Summer 2025 | Context curation beats automatic RAG |
| Superpowers: How I'm Using Coding Agents | Sub-agents for context isolation (Simon Willison) |
| Claude Code Experience After Two Weeks | Fresh contexts yield better results |
| AI Agent Benchmarks Are Broken | LLM-as-judge has shared blind spots |
| How to Orchestrate Multi-Agent Workflows | Event-driven, decoupled coordination |
| Context Engineering vs Prompt Engineering | Manual context selection principles |
Show HN Projects
| Project | Contribution |
|---|---|
| Self-Evolving Agents Repository | Self-improvement patterns |
| Package Manager for Agent Skills | Skills architecture |
| Wispbit - AI Code Review Agent | Code review patterns |
| Agtrace - Monitoring for AI Coding Agents | Agent monitoring patterns |
---
Individual Contributors
Special thanks to thought leaders whose patterns and insights shaped Loki Mode:
| Contributor | Contribution |
|---|---|
| Boris Cherny (Creator of Claude Code) | Self-verification loop (2-3x quality improvement), extended thinking mode, "Less prompting, more systems" philosophy |
| Ivan Steshov | Centralized constitution, agent lineage tracking, structured artifacts as contracts |
| Addy Osmani | Git checkpoint system, specification-first approach, visual aids (Mermaid diagrams) |
| Simon Willison | Sub-agents for context isolation, skills system, context curation patterns |
---
Production Patterns Summary
Key patterns incorporated from practitioner experience:
| Pattern | Source | Implementation |
|---|---|---|
| Human-in-the-Loop (HITL) | HN Production Discussions | Confidence-based escalation thresholds |
| Narrow Scope (3-5 steps) | Multiple Practitioners | Task scope constraints |
| Deterministic Validation | Production Teams | Rule-based outer loops (not LLM-judged) |
| Context Curation | Simon Willison | Manual selection, focused context |
| Blind Review + Devil's Advocate | CONSENSAGENT | Anti-sycophancy protocol |
| Hierarchical Reasoning | DeepMind Gemini | Orchestrator + specialized executors |
| Constitutional Self-Critique | Anthropic | Principles-based revision |
| Debate Verification | DeepMind | Critical change verification |
| One Feature at a Time | Anthropic Harness | Single feature per iteration, full verification |
| E2E Browser Testing | Anthropic Harness | Playwright MCP for visual verification |
---
License
This acknowledgements file documents the research and resources that influenced Loki Mode's design. All referenced works retain their original licenses and copyrights.
Loki Mode itself is released under the MIT License.
---
Last updated: v2.35.0
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Loki Mode Dashboard</title>
<style>
:root {
--bg-cream: #FDF6E3;
--bg-dark: #1a1a2e;
--coral: #FF6B6B;
--coral-light: #FF8E8E;
--teal: #4ECDC4;
--purple: #A78BFA;
--yellow: #FCD34D;
--green: #10B981;
--red: #EF4444;
--text-dark: #2D3748;
--text-light: #718096;
--card-bg: #FFFFFF;
--border: #E2E8F0;
}
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, sans-serif;
background: var(--bg-cream);
color: var(--text-dark);
min-height: 100vh;
padding: 24px;
}
.header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 32px;
}
.logo {
display: flex;
align-items: center;
gap: 12px;
}
.logo-icon {
width: 48px;
height: 48px;
background: linear-gradient(135deg, var(--coral), var(--coral-light));
border-radius: 12px;
display: flex;
align-items: center;
justify-content: center;
font-size: 24px;
color: white;
font-weight: bold;
}
.logo-text {
font-size: 28px;
font-weight: 700;
color: var(--text-dark);
}
.status-bar {
display: flex;
gap: 16px;
align-items: center;
}
.status-item {
display: flex;
align-items: center;
gap: 8px;
padding: 8px 16px;
background: var(--card-bg);
border-radius: 8px;
border: 1px solid var(--border);
}
.status-dot {
width: 8px;
height: 8px;
border-radius: 50%;
}
.status-dot.active { background: var(--green); }
.status-dot.warning { background: var(--yellow); }
.status-dot.error { background: var(--red); }
.section {
margin-bottom: 32px;
}
.section-title {
font-size: 20px;
font-weight: 600;
margin-bottom: 16px;
color: var(--text-dark);
}
.agents-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(320px, 1fr));
gap: 16px;
}
.agent-card {
background: var(--card-bg);
border-radius: 12px;
padding: 20px;
border: 1px solid var(--border);
box-shadow: 0 2px 8px rgba(0,0,0,0.04);
}
.agent-header {
display: flex;
justify-content: space-between;
align-items: flex-start;
margin-bottom: 12px;
}
.agent-id {
font-weight: 600;
font-size: 14px;
color: var(--text-dark);
}
.agent-type {
font-size: 12px;
color: var(--text-light);
margin-top: 4px;
}
.model-badge {
padding: 4px 10px;
border-radius: 12px;
font-size: 11px;
font-weight: 600;
text-transform: uppercase;
}
.model-badge.sonnet {
background: #E0E7FF;
color: #4338CA;
}
.model-badge.haiku {
background: #D1FAE5;
color: #065F46;
}
.model-badge.opus {
background: #FDE68A;
color: #92400E;
}
.agent-work {
font-size: 13px;
color: var(--text-dark);
line-height: 1.5;
margin-bottom: 16px;
}
.agent-stats {
display: flex;
gap: 16px;
font-size: 12px;
color: var(--text-light);
}
.stat {
display: flex;
align-items: center;
gap: 4px;
}
.agent-status {
display: flex;
align-items: center;
gap: 6px;
font-size: 12px;
margin-top: 12px;
padding-top: 12px;
border-top: 1px solid var(--border);
}
.status-active {
color: var(--green);
}
.status-completed {
color: var(--purple);
}
/* Task Queue Section */
.queue-columns {
display: grid;
grid-template-columns: repeat(4, 1fr);
gap: 16px;
}
.queue-column {
background: var(--card-bg);
border-radius: 12px;
padding: 16px;
border: 1px solid var(--border);
min-height: 300px;
}
.queue-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 16px;
padding-bottom: 12px;
border-bottom: 2px solid var(--border);
}
.queue-title {
font-weight: 600;
font-size: 14px;
}
.queue-count {
background: var(--bg-cream);
padding: 4px 10px;
border-radius: 12px;
font-size: 12px;
font-weight: 600;
}
.queue-column.pending .queue-header { border-color: var(--yellow); }
.queue-column.in-progress .queue-header { border-color: var(--teal); }
.queue-column.completed .queue-header { border-color: var(--green); }
.queue-column.failed .queue-header { border-color: var(--red); }
.task-card {
background: var(--bg-cream);
border-radius: 8px;
padding: 12px;
margin-bottom: 8px;
font-size: 12px;
}
.task-id {
font-weight: 600;
color: var(--text-dark);
margin-bottom: 4px;
}
.task-type {
display: inline-block;
padding: 2px 8px;
background: var(--card-bg);
border-radius: 4px;
font-size: 10px;
color: var(--text-light);
margin-bottom: 8px;
}
.task-desc {
color: var(--text-dark);
line-height: 1.4;
}
.last-updated {
text-align: center;
color: var(--text-light);
font-size: 12px;
margin-top: 24px;
}
@media (max-width: 1024px) {
.queue-columns {
grid-template-columns: repeat(2, 1fr);
}
}
@media (max-width: 640px) {
.queue-columns {
grid-template-columns: 1fr;
}
.agents-grid {
grid-template-columns: 1fr;
}
}
</style>
</head>
<body>
<div class="header">
<div class="logo">
<div class="logo-icon">L</div>
<span class="logo-text">Loki Mode Dashboard</span>
</div>
<div class="status-bar">
<div class="status-item">
<div class="status-dot active"></div>
<span>Phase: DEVELOPMENT</span>
</div>
<div class="status-item">
<span>v2.18.0</span>
</div>
</div>
</div>
<div class="section" id="agents-section">
<h2 class="section-title">Active Agents</h2>
<div class="agents-grid" id="agents-grid">
<!-- Agent cards populated by JS -->
</div>
</div>
<div class="section" id="queue-section">
<h2 class="section-title">Task Queue</h2>
<div class="queue-columns" id="queue-columns">
<div class="queue-column pending">
<div class="queue-header">
<span class="queue-title">Pending</span>
<span class="queue-count" id="pending-count">0</span>
</div>
<div id="pending-tasks"></div>
</div>
<div class="queue-column in-progress">
<div class="queue-header">
<span class="queue-title">In Progress</span>
<span class="queue-count" id="in-progress-count">0</span>
</div>
<div id="in-progress-tasks"></div>
</div>
<div class="queue-column completed">
<div class="queue-header">
<span class="queue-title">Completed</span>
<span class="queue-count" id="completed-count">0</span>
</div>
<div id="completed-tasks"></div>
</div>
<div class="queue-column failed">
<div class="queue-header">
<span class="queue-title">Failed</span>
<span class="queue-count" id="failed-count">0</span>
</div>
<div id="failed-tasks"></div>
</div>
</div>
</div>
<div class="last-updated" id="last-updated">
Last updated: --
</div>
<script>
// Demo data for screenshots
const demoAgents = [
{
id: 'eng-001-backend-api',
type: 'Engineering Backend',
model: 'sonnet',
work: 'Implementing POST /api/todos endpoint with validation and SQLite storage',
runtime: '12m 34s',
tasks: 5,
status: 'active'
},
{
id: 'eng-002-frontend-ui',
type: 'Engineering Frontend',
model: 'sonnet',
work: 'Building React components for todo list with Tailwind styling',
runtime: '8m 21s',
tasks: 3,
status: 'active'
},
{
id: 'qa-001-testing',
type: 'QA Testing',
model: 'haiku',
work: 'Writing unit tests for authentication module',
runtime: '5m 45s',
tasks: 8,
status: 'active'
},
{
id: 'review-security-001',
type: 'Security Review',
model: 'opus',
work: 'Analyzing auth flow for OWASP vulnerabilities',
runtime: '3m 12s',
tasks: 2,
status: 'active'
},
{
id: 'ops-devops-001',
type: 'Operations DevOps',
model: 'sonnet',
work: 'Configuring GitHub Actions CI/CD pipeline',
runtime: '15m 08s',
tasks: 4,
status: 'active'
},
{
id: 'biz-marketing-001',
type: 'Business Marketing',
model: 'haiku',
work: 'Creating landing page copy and SEO meta tags',
runtime: '6m 33s',
tasks: 2,
status: 'completed'
}
];
const demoTasks = {
pending: [
{ id: 'task-015', type: 'eng-database', desc: 'Create migration for user preferences table' },
{ id: 'task-016', type: 'eng-frontend', desc: 'Implement dark mode toggle component' },
{ id: 'task-017', type: 'qa-testing', desc: 'Write E2E tests for checkout flow' }
],
inProgress: [
{ id: 'task-012', type: 'eng-backend', desc: 'Implement JWT refresh token rotation' },
{ id: 'task-013', type: 'eng-frontend', desc: 'Add form validation to signup page' }
],
completed: [
{ id: 'task-001', type: 'eng-backend', desc: 'Setup Express server with TypeScript' },
{ id: 'task-002', type: 'eng-database', desc: 'Create initial SQLite schema' },
{ id: 'task-003', type: 'eng-frontend', desc: 'Initialize React with Vite' },
{ id: 'task-004', type: 'ops-devops', desc: 'Configure ESLint and Prettier' },
{ id: 'task-005', type: 'eng-backend', desc: 'Implement user registration endpoint' }
],
failed: []
};
function renderAgents() {
const grid = document.getElementById('agents-grid');
grid.innerHTML = demoAgents.map(agent => `
<div class="agent-card">
<div class="agent-header">
<div>
<div class="agent-id">${agent.id}</div>
<div class="agent-type">${agent.type}</div>
</div>
<span class="model-badge ${agent.model}">${agent.model}</span>
</div>
<div class="agent-work">${agent.work}</div>
<div class="agent-stats">
<span class="stat">Runtime: ${agent.runtime}</span>
<span class="stat">Tasks: ${agent.tasks}</span>
</div>
<div class="agent-status ${agent.status === 'active' ? 'status-active' : 'status-completed'}">
<div class="status-dot ${agent.status === 'active' ? 'active' : ''}"></div>
${agent.status === 'active' ? 'Active' : 'Completed'}
</div>
</div>
`).join('');
}
function renderTasks() {
const renderTaskList = (tasks, containerId) => {
const container = document.getElementById(containerId);
container.innerHTML = tasks.map(task => `
<div class="task-card">
<div class="task-id">${task.id}</div>
<span class="task-type">${task.type}</span>
<div class="task-desc">${task.desc}</div>
</div>
`).join('');
};
renderTaskList(demoTasks.pending, 'pending-tasks');
renderTaskList(demoTasks.inProgress, 'in-progress-tasks');
renderTaskList(demoTasks.completed.slice(0, 5), 'completed-tasks');
renderTaskList(demoTasks.failed, 'failed-tasks');
document.getElementById('pending-count').textContent = demoTasks.pending.length;
document.getElementById('in-progress-count').textContent = demoTasks.inProgress.length;
document.getElementById('completed-count').textContent = demoTasks.completed.length;
document.getElementById('failed-count').textContent = demoTasks.failed.length;
}
function updateTimestamp() {
const now = new Date().toLocaleString();
document.getElementById('last-updated').textContent = `Last updated: ${now}`;
}
// Initial render
renderAgents();
renderTasks();
updateTimestamp();
// Auto-refresh every 3 seconds (in production, would fetch real data)
setInterval(updateTimestamp, 3000);
</script>
</body>
</html>
Loki Mode Agent Constitution
Machine-Enforceable Behavioral Contract for All Agents
Version 1.0.0 | Immutable Principles | Context-Preserved Lineage
---
Core Principles (Inviolable)
1. Specification-First Development
RULE: No code shall be written before the specification exists.
Enforcement:
IF task.type == "implementation" AND !exists(spec_file):
BLOCK with error: "SPEC_MISSING"
REQUIRE: Create OpenAPI spec firstRationale: Specs are contracts. Code is implementation. Contract before implementation.
2. Git Checkpoint System
RULE: Every completed task MUST create a git checkpoint.
Enforcement:
ON task.status == "completed":
git add <modified_files>
git commit -m "[Loki] Task ${task.id}: ${task.title}"
UPDATE CONTINUITY.md with commit SHARationale: Git history is proof of progress. Every task is a save point.
3. Context Preservation
RULE: All agents MUST inherit and preserve context from their spawning agent.
Enforcement:
ON agent.spawn():
agent.context.parent_id = spawner.agent_id
agent.context.lineage = [...spawner.lineage, spawner.agent_id]
agent.context.inherited_memory = spawner.memory.export()
WRITE .agent/sub-agents/${agent.agent_id}.jsonRationale: Context drift kills multi-agent systems. Lineage is truth.
4. Iterative Specification Questions
RULE: During spec generation, agents MUST ask clarifying questions before assuming.
Enforcement:
WHILE generating_spec:
IF ambiguity_detected OR assumption_required:
questions = generate_clarifying_questions()
IF orchestrator_mode:
answers = infer_from_prd()
ELSE:
answers = ask_user(questions)
UPDATE spec WITH answersRationale: Assumptions create bugs. Questions create clarity.
5. Machine-Readable Rules
RULE: All behavioral rules MUST be represented as structured artifacts, not just prose.
Enforcement:
rules/
├── pre-commit.schema.json # Validation rules
├── quality-gates.yaml # Quality thresholds
├── agent-contracts.json # Agent responsibilities
└── invariants.ts # Runtime assertionsRationale: Humans read markdown. Machines enforce JSON/YAML.
---
Agent Behavioral Contracts
Orchestrator Agent
Responsibilities:
- Initialize .loki/ directory structure
- Maintain CONTINUITY.md (working memory)
- Coordinate task queue (pending → in-progress → completed)
- Enforce quality gates
- Manage git checkpoints
Prohibited Actions:
- Writing implementation code directly
- Skipping spec generation
- Modifying completed tasks without explicit override
Context Obligations:
- MUST read CONTINUITY.md before every action
- MUST update orchestrator.json after phase transitions
- MUST preserve task lineage in completed.json
Engineering Swarm Agents
Responsibilities:
- Implement features per OpenAPI spec
- Write contract tests before implementation
- Create git commits for completed tasks
- Ask clarifying questions when spec is ambiguous
Prohibited Actions:
- Implementing without spec
- Skipping tests
- Ignoring linter/type errors
Context Obligations:
- MUST inherit parent agent's context
- MUST log all decisions to .agent/sub-agents/${agent_id}.md
- MUST reference spec in all implementation commits
QA Swarm Agents
Responsibilities:
- Generate test cases from OpenAPI spec
- Run contract validation tests
- Report discrepancies between code and spec
- Create bug reports in dead-letter queue
Prohibited Actions:
- Modifying implementation code
- Skipping failing tests
- Approving incomplete features
Context Obligations:
- MUST validate against spec as source of truth
- MUST log test results to ledgers/
- MUST create git commits for test additions
DevOps Swarm Agents
Responsibilities:
- Automate deployment pipelines
- Monitor service health
- Configure infrastructure as code
- Manage environment secrets
Prohibited Actions:
- Storing secrets in plaintext
- Deploying without health checks
- Skipping rollback procedures
Context Obligations:
- MUST log all deployments to deployment ledger
- MUST preserve deployment context for rollback
- MUST track infrastructure state in orchestrator.json
---
Quality Gates (Machine-Enforceable)
Pre-Commit Hook (BLOCKING)
quality_gates:
linting:
enabled: true
auto_fix: true
block_on_failure: true
type_checking:
enabled: true
strict_mode: true
block_on_failure: true
contract_tests:
enabled: true
min_coverage: 80%
block_on_failure: true
spec_validation:
enabled: true
validator: spectral
block_on_failure: truePost-Implementation Review (AUTO-FIX)
auto_review:
static_analysis:
tools: [eslint, prettier, tsc]
auto_fix: true
security_scan:
tools: [semgrep, snyk]
severity_threshold: medium
auto_create_issues: true
performance_check:
lighthouse_score: 90
bundle_size_limit: 500kb
warn_only: true---
Memory Hierarchy (Priority Order)
1. CONTINUITY.md (Volatile - Every Turn)
Purpose: What am I doing RIGHT NOW? Update Frequency: Every turn Content: Current task, phase, blockers, next steps
2. CONSTITUTION.md (Immutable - This File)
Purpose: How MUST I behave? Update Frequency: Version bumps only Content: Behavioral contracts, quality gates, invariants
3. CLAUDE.md (Semi-Stable - Significant Changes)
Purpose: What is this project? Update Frequency: Architecture changes Content: Tech stack, patterns, project context
4. Ledgers (Append-Only - Checkpoint)
Purpose: What happened? Update Frequency: After significant events Content: Decisions, deployments, reviews
5. .agent/sub-agents/*.json (Lineage Tracking)
Purpose: Who did what and why? Update Frequency: Agent lifecycle events Content: Agent context, decisions, inherited memory
---
Context Lineage Schema
{
"agent_id": "eng-001-backend-api",
"agent_type": "general-purpose",
"model": "haiku",
"spawned_at": "2026-01-04T05:30:00Z",
"spawned_by": "orchestrator-main",
"lineage": ["orchestrator-main", "eng-001-backend-api"],
"inherited_context": {
"phase": "development",
"current_task": "task-005",
"spec_reference": ".loki/specs/openapi.yaml#/paths/~1api~1todos",
"tech_stack": ["Node.js", "Express", "TypeScript", "SQLite"]
},
"decisions_made": [
{
"timestamp": "2026-01-04T05:31:15Z",
"question": "Should we use Prisma or raw SQL?",
"answer": "Raw SQL with better-sqlite3 for simplicity",
"rationale": "PRD requires minimal dependencies, synchronous ops preferred"
}
],
"tasks_completed": ["task-005"],
"commits_created": ["abc123f", "def456a"],
"status": "completed",
"completed_at": "2026-01-04T05:45:00Z"
}---
Git Checkpoint Protocol
Commit Message Format
[Loki] ${agent_type}-${task_id}: ${task_title}
${detailed_description}
Agent: ${agent_id}
Parent: ${parent_agent_id}
Spec: ${spec_reference}
Tests: ${test_files}Example
[Loki] eng-005-backend: Implement POST /api/todos endpoint
Created todo creation endpoint per OpenAPI spec.
- Input validation for title field
- SQLite insertion with timestamps
- Returns 201 with created todo object
- Contract tests passing
Agent: eng-001-backend-api
Parent: orchestrator-main
Spec: .loki/specs/openapi.yaml#/paths/~1api~1todos/post
Tests: backend/tests/todos.contract.test.ts---
Invariants (Runtime Assertions)
// .loki/rules/invariants.ts
export const INVARIANTS = {
// Spec must exist before implementation
SPEC_BEFORE_CODE: (task: Task) => {
if (task.type === 'implementation') {
assert(exists(task.spec_reference), 'SPEC_MISSING');
}
},
// All tasks must have git commits
TASK_HAS_COMMIT: (task: Task) => {
if (task.status === 'completed') {
assert(task.git_commit_sha, 'COMMIT_MISSING');
}
},
// Agent lineage must be preserved
AGENT_HAS_LINEAGE: (agent: Agent) => {
assert(agent.lineage.length > 0, 'LINEAGE_MISSING');
assert(agent.spawned_by, 'PARENT_MISSING');
},
// CONTINUITY.md must always exist
CONTINUITY_EXISTS: () => {
assert(exists('.loki/CONTINUITY.md'), 'CONTINUITY_MISSING');
},
// Quality gates must pass before merge
QUALITY_GATES_PASSED: (task: Task) => {
if (task.status === 'completed') {
assert(task.quality_checks.all_passed, 'QUALITY_GATE_FAILED');
}
}
};---
Visual Specification Aids
Mermaid Diagram Generation (Required for Complex Features)
RULE: Architecture decisions and complex workflows MUST include Mermaid diagrams.
Example - Authentication Flow:
sequenceDiagram
participant C as Client
participant A as API
participant D as Database
C->>A: POST /api/auth/login
A->>A: Validate credentials
A->>D: Query user
D-->>A: User record
A->>A: Generate JWT token
A-->>C: 200 OK {token}Storage Location: .loki/diagrams/${feature_name}.mmd
When Required:
- Multi-step workflows (3+ steps)
- System architecture changes
- Complex state machines
- Integration points between services
---
Amendment Process
This constitution can only be amended through: 1. Version bump in header 2. Git commit with [CONSTITUTION] prefix 3. Changelog entry documenting what changed and why 4. Re-validation of all existing agents against new rules
Example Amendment Commit:
[CONSTITUTION] v1.1.0: Add visual specification requirement
Added requirement for Mermaid diagrams on complex features to prevent
ambiguity in multi-step workflows. Based on Addy Osmani's insight that
visual aids significantly improve AI-to-AI communication.
Breaking changes: None
New rules: Section "Visual Specification Aids"---
Enforcement
All rules in this constitution are machine-enforceable and MUST be implemented as: 1. Pre-commit hooks (Git) 2. Runtime assertions (TypeScript invariants) 3. Quality gate validators (YAML configs) 4. Agent behavior validators (JSON schemas)
Human guidance is advisory. Machine enforcement is mandatory.
---
"In autonomous systems, trust is built on invariants, not intentions."
Loki Mode - Autonomous Runner
Single script that handles everything: prerequisites, setup, Vibe Kanban monitoring, and autonomous execution with auto-resume.
Quick Start
# Run with a PRD
./autonomy/run.sh ./docs/requirements.md
# Run interactively
./autonomy/run.shThat's it! The script will: 1. Check all prerequisites (Claude CLI, Python, Git, etc.) 2. Verify skill installation 3. Initialize the .loki/ directory 4. Start Vibe Kanban background sync (monitor tasks in real-time) 5. Start Claude Code with live output (no more waiting blindly) 6. Auto-resume on rate limits or interruptions 7. Continue until completion or max retries
Live Output
Claude's output is displayed in real-time - you can see exactly what's happening:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
CLAUDE CODE OUTPUT (live)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[Claude's output appears here in real-time...]Status Monitor (Built-in)
The runner updates .loki/STATUS.txt every 5 seconds with task progress:
╔════════════════════════════════════════════════════════════════╗
║ LOKI MODE STATUS ║
╚════════════════════════════════════════════════════════════════╝
Updated: Sat Dec 28 15:30:00 PST 2025
Phase: DEVELOPMENT
Tasks:
├─ Pending: 10
├─ In Progress: 1
├─ Completed: 5
└─ Failed: 0
Monitor: watch -n 2 cat .loki/STATUS.txtMonitor in Another Terminal
# Watch status updates live
watch -n 2 cat .loki/STATUS.txt
# Or view once
cat .loki/STATUS.txtWhat Gets Checked
| Prerequisite | Required | Notes |
|---|---|---|
| Claude Code CLI | Yes | Install from https://claude.ai/code |
| Python 3 | Yes | For state management |
| Git | Yes | For version control |
| curl | Yes | For web fetches |
| Node.js | No | Needed for some builds |
| jq | No | Helpful for JSON parsing |
Configuration
Environment variables:
# Retry settings
export LOKI_MAX_RETRIES=50 # Max retry attempts (default: 50)
export LOKI_BASE_WAIT=60 # Base wait time in seconds (default: 60)
export LOKI_MAX_WAIT=3600 # Max wait time in seconds (default: 3600)
# Skip prerequisite checks (for CI/CD or repeat runs)
export LOKI_SKIP_PREREQS=true
# Run with custom settings
LOKI_MAX_RETRIES=100 LOKI_BASE_WAIT=120 ./autonomy/run.sh ./docs/prd.mdHow Auto-Resume Works
┌─────────────────────────────────────────────────────────────┐
│ ./autonomy/run.sh prd.md │
└─────────────────────────────────────────────────────────────┘
│
▼
┌───────────────────────┐
│ Check Prerequisites │
└───────────────────────┘
│
▼
┌───────────────────────┐
│ Initialize .loki/ │
└───────────────────────┘
│
▼
┌────────────────────────────────┐
│ Run Claude Code with prompt │◄────────────────┐
└────────────────────────────────┘ │
│ │
▼ │
┌───────────────────────┐ │
│ Claude exits │ │
└───────────────────────┘ │
│ │
┌───────────┴───────────┐ │
▼ ▼ │
┌───────────────┐ ┌───────────────┐ │
│ Completed? │──Yes──│ SUCCESS! │ │
└───────────────┘ └───────────────┘ │
│ No │
▼ │
┌───────────────┐ │
│ Wait (backoff)│─────────────────────────────────────┘
└───────────────┘State Files
The autonomy runner creates:
.loki/
├── autonomy-state.json # Runner state (retry count, status)
├── logs/
│ └── autonomy-*.log # Execution logs
├── state/
│ └── orchestrator.json # Loki Mode phase tracking
└── COMPLETED # Created when doneResuming After Interruption
If you stop the script (Ctrl+C) or it crashes, just run it again:
# State is saved, will resume from last checkpoint
./autonomy/run.sh ./docs/requirements.mdThe script detects the previous state and continues from where it left off.
Differences from Manual Mode
| Feature | Manual Mode | Autonomy Mode |
|---|---|---|
| Start | claude --dangerously-skip-permissions | ./autonomy/run.sh |
| Prereq check | Manual | Automatic |
| Rate limit handling | Manual restart | Auto-resume |
| State persistence | Manual checkpoint | Automatic |
| Logging | Console only | Console + file |
| Max runtime | Session-based | Configurable retries |
Troubleshooting
"Claude Code CLI not found"
npm install -g @anthropic-ai/claude-code
# or visit https://claude.ai/code"SKILL.md not found"
Make sure you're running from the loki-mode directory or have installed the skill:
# Option 1: Run from project directory
cd /path/to/loki-mode
./autonomy/run.sh
# Option 2: Install skill globally
cp -r . ~/.claude/skills/loki-mode/"Max retries exceeded"
The task is taking too long or repeatedly failing. Check:
# View logs
cat .loki/logs/autonomy-*.log | tail -100
# Check orchestrator state
cat .loki/state/orchestrator.json
# Increase retries
LOKI_MAX_RETRIES=200 ./autonomy/run.sh ./docs/prd.md{
"name": "SWE-bench Lite",
"version": "1.0",
"description": "300 real-world GitHub issues for evaluation",
"source": "https://github.com/SWE-bench/SWE-bench",
"problems": 300,
"status": "PLACEHOLDER",
"install_command": "pip install swebench",
"run_command": "python -m swebench.harness.run_evaluation"
}#!/bin/bash
#===============================================================================
# Prepare SWE-bench Submission
# Converts benchmark results to official SWE-bench submission format
#
# Usage:
# ./benchmarks/prepare-submission.sh <results-dir>
# ./benchmarks/prepare-submission.sh benchmarks/results/2026-01-05-10-37-54
#
# Output:
# Creates submission-ready folder at benchmarks/submission/
#===============================================================================
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
CYAN='\033[0;36m'
NC='\033[0m'
log_info() { echo -e "${CYAN}[INFO]${NC} $1"; }
log_success() { echo -e "${GREEN}[PASS]${NC} $1"; }
log_error() { echo -e "${RED}[FAIL]${NC} $1"; }
if [ $# -lt 1 ]; then
echo "Usage: $0 <results-directory>"
echo "Example: $0 benchmarks/results/2026-01-05-10-37-54"
exit 1
fi
RESULTS_DIR="$1"
SUBMISSION_DATE=$(date +%Y%m%d)
SUBMISSION_DIR="$SCRIPT_DIR/submission/${SUBMISSION_DATE}_loki_mode"
log_info "Preparing SWE-bench submission..."
log_info "Results: $RESULTS_DIR"
log_info "Output: $SUBMISSION_DIR"
# Check results directory
if [ ! -d "$RESULTS_DIR" ]; then
log_error "Results directory not found: $RESULTS_DIR"
exit 1
fi
# Check for required files
if [ ! -f "$RESULTS_DIR/swebench-loki-predictions.json" ]; then
log_error "Predictions file not found: $RESULTS_DIR/swebench-loki-predictions.json"
exit 1
fi
# Create submission directory
mkdir -p "$SUBMISSION_DIR"
# Copy template files
log_info "Copying template files..."
cp "$SCRIPT_DIR/submission-template/README.md" "$SUBMISSION_DIR/"
cp "$SCRIPT_DIR/submission-template/metadata.yaml" "$SUBMISSION_DIR/"
# Convert predictions to JSONL format
log_info "Converting predictions to JSONL format..."
python3 << CONVERT_PREDS
import json
with open("$RESULTS_DIR/swebench-loki-predictions.json", 'r') as f:
predictions = json.load(f)
with open("$SUBMISSION_DIR/all_preds.jsonl", 'w') as f:
for pred in predictions:
# Format required by SWE-bench
entry = {
"instance_id": pred["instance_id"],
"model_patch": pred["model_patch"],
"model_name_or_path": pred.get("model_name_or_path", "loki-mode")
}
f.write(json.dumps(entry) + '\n')
print(f"Converted {len(predictions)} predictions to JSONL format")
CONVERT_PREDS
# Copy trajectories if they exist
if [ -d "$RESULTS_DIR/trajs" ]; then
log_info "Copying trajectory files..."
cp -r "$RESULTS_DIR/trajs" "$SUBMISSION_DIR/"
TRAJ_COUNT=$(ls -1 "$SUBMISSION_DIR/trajs" 2>/dev/null | wc -l | tr -d ' ')
log_success "Copied $TRAJ_COUNT trajectory files"
else
log_info "No trajectory files found (run benchmark with --loki for trajectory logging)"
mkdir -p "$SUBMISSION_DIR/trajs"
fi
# Copy logs if they exist
if [ -d "$RESULTS_DIR/logs" ]; then
log_info "Copying log files..."
cp -r "$RESULTS_DIR/logs" "$SUBMISSION_DIR/"
LOG_COUNT=$(ls -1 "$SUBMISSION_DIR/logs" 2>/dev/null | wc -l | tr -d ' ')
log_success "Copied $LOG_COUNT log directories"
else
log_info "No log files found (run benchmark with --loki for log capture)"
mkdir -p "$SUBMISSION_DIR/logs"
fi
# Update metadata with actual results
log_info "Updating metadata with actual results..."
python3 << UPDATE_META
import json
import yaml
from datetime import datetime
# Load results
with open("$RESULTS_DIR/swebench-loki-results.json", 'r') as f:
results = json.load(f)
# Load metadata template
with open("$SUBMISSION_DIR/metadata.yaml", 'r') as f:
metadata = yaml.safe_load(f)
# Update with actual results
metadata['results'] = {
'patch_generation_rate': round((results.get('generated', 0) / results.get('total_problems', 1)) * 100, 2),
'problems_solved': results.get('generated', 0),
'problems_total': results.get('total_problems', 0),
'fixed_by_rarv': results.get('fixed_by_rarv', 0),
'avg_attempts': round(results.get('avg_attempts', 1.0), 2),
'total_time_seconds': round(results.get('elapsed_time', 0)),
'avg_time_per_problem_seconds': round(results.get('elapsed_time', 0) / max(results.get('total_problems', 1), 1))
}
metadata['submission']['date'] = datetime.now().strftime('%Y-%m-%d')
# Save updated metadata
with open("$SUBMISSION_DIR/metadata.yaml", 'w') as f:
yaml.dump(metadata, f, default_flow_style=False, sort_keys=False)
print("Metadata updated with actual results")
CONVERT_PREDS
# Generate submission summary
log_info "Generating submission summary..."
cat > "$SUBMISSION_DIR/SUBMISSION_CHECKLIST.md" << 'CHECKLIST'
# SWE-bench Submission Checklist
## Required Files
- [x] all_preds.jsonl - Predictions in JSONL format
- [x] README.md - Description of the system
- [x] metadata.yaml - Submission metadata
## Optional but Recommended
- [ ] trajs/ - Reasoning trajectories (required for some leaderboards)
- [ ] logs/ - Execution logs
## Pre-Submission Steps
1. **Verify predictions format:**
```bash
head -1 all_preds.jsonl | python -m json.tool
```
2. **Run SWE-bench evaluator (optional but recommended):**
```bash
python -m swebench.harness.run_evaluation \
--predictions all_preds.jsonl \
--max_workers 4 \
--run_id loki_mode_v2.25.0
```
3. **Fork and create PR:**
```bash
# Fork https://github.com/SWE-bench/experiments
# Clone your fork
git clone https://github.com/YOUR_USERNAME/experiments.git
cd experiments
# Copy submission
cp -r /path/to/submission evaluation/lite/20260105_loki_mode
# Create PR
git checkout -b loki-mode-submission
git add .
git commit -m "Add Loki Mode submission"
git push origin loki-mode-submission
```
4. **Submit PR with:**
- Link to this repository
- Brief description of the system
- Any relevant benchmark methodology notes
## Contact
For questions about this submission, open an issue at:
https://github.com/asklokesh/loki-mode/issues
CHECKLIST
# Final summary
echo ""
echo "======================================================================"
echo " SUBMISSION PREPARED"
echo "======================================================================"
echo " Location: $SUBMISSION_DIR"
echo ""
echo " Files:"
ls -la "$SUBMISSION_DIR/"
echo ""
echo " Next Steps:"
echo " 1. Review all_preds.jsonl format"
echo " 2. Run SWE-bench evaluator (optional)"
echo " 3. Fork SWE-bench/experiments"
echo " 4. Copy submission folder to evaluation/lite/"
echo " 5. Create pull request"
echo "======================================================================"
log_success "Submission preparation complete!"
{
"benchmark": "HumanEval",
"version": "1.0",
"timestamp": "2026-01-05T00:24:04.904083",
"total_problems": 164,
"status": "INFRASTRUCTURE_READY",
"note": "Benchmark infrastructure created. Run with --execute to run actual tests.",
"sample_problems": [
"HumanEval/0",
"HumanEval/1",
"HumanEval/2",
"HumanEval/3",
"HumanEval/4"
]
}Loki Mode Benchmark Results
Overview
This directory contains benchmark results for Loki Mode multi-agent system.
Benchmarks Available
HumanEval
- Problems: 164 Python programming problems
- Metric: Pass@1 (percentage of problems solved on first attempt)
- Competitor Baseline: MetaGPT achieves 85.9-87.7%
SWE-bench Lite
- Problems: 300 real-world GitHub issues
- Metric: Resolution rate
- Competitor Baseline: Top agents achieve 45-77%
Running Benchmarks
# Run all benchmarks
./benchmarks/run-benchmarks.sh all
# Run specific benchmark
./benchmarks/run-benchmarks.sh humaneval --execute
./benchmarks/run-benchmarks.sh swebench --executeResults Format
Results are saved as JSON files with:
- Timestamp
- Problem count
- Pass rate
- Individual problem results
- Token usage
- Execution time
Methodology
Loki Mode uses its multi-agent architecture to solve each problem: 1. Architect Agent analyzes the problem 2. Engineer Agent implements the solution 3. QA Agent validates with test cases 4. Review Agent checks code quality
This mirrors real-world software development more accurately than single-agent approaches.
{
"benchmark": "SWE-bench Lite",
"version": "1.0",
"timestamp": "2026-01-05T00:24:04.950779",
"total_problems": 300,
"status": "INFRASTRUCTURE_READY",
"note": "Benchmark infrastructure created. Install swebench package for full evaluation.",
"install": "pip install swebench",
"evaluation": "python -m swebench.harness.run_evaluation --predictions predictions.json"
}{
"benchmark": "HumanEval",
"version": "1.0",
"timestamp": "2026-01-05T00:49:17.745476",
"model": "opus",
"timeout_per_problem": 300,
"total_problems": 164,
"status": "COMPLETED",
"problems": [
{
"task_id": "HumanEval/0",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/0.py"
},
{
"task_id": "HumanEval/1",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/1.py"
},
{
"task_id": "HumanEval/2",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/2.py"
},
{
"task_id": "HumanEval/3",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/3.py"
},
{
"task_id": "HumanEval/4",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/4.py"
},
{
"task_id": "HumanEval/5",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/5.py"
},
{
"task_id": "HumanEval/6",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/6.py"
},
{
"task_id": "HumanEval/7",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/7.py"
},
{
"task_id": "HumanEval/8",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/8.py"
},
{
"task_id": "HumanEval/9",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/9.py"
},
{
"task_id": "HumanEval/10",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/10.py"
},
{
"task_id": "HumanEval/11",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/11.py"
},
{
"task_id": "HumanEval/12",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/12.py"
},
{
"task_id": "HumanEval/13",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/13.py"
},
{
"task_id": "HumanEval/14",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/14.py"
},
{
"task_id": "HumanEval/15",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/15.py"
},
{
"task_id": "HumanEval/16",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/16.py"
},
{
"task_id": "HumanEval/17",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/17.py"
},
{
"task_id": "HumanEval/18",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/18.py"
},
{
"task_id": "HumanEval/19",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/19.py"
},
{
"task_id": "HumanEval/20",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/20.py"
},
{
"task_id": "HumanEval/21",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/21.py"
},
{
"task_id": "HumanEval/22",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/22.py"
},
{
"task_id": "HumanEval/23",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/23.py"
},
{
"task_id": "HumanEval/24",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/24.py"
},
{
"task_id": "HumanEval/25",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/25.py"
},
{
"task_id": "HumanEval/26",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/26.py"
},
{
"task_id": "HumanEval/27",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/27.py"
},
{
"task_id": "HumanEval/28",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/28.py"
},
{
"task_id": "HumanEval/29",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/29.py"
},
{
"task_id": "HumanEval/30",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/30.py"
},
{
"task_id": "HumanEval/31",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/31.py"
},
{
"task_id": "HumanEval/32",
"passed": false,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/32.py"
},
{
"task_id": "HumanEval/33",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/33.py"
},
{
"task_id": "HumanEval/34",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/34.py"
},
{
"task_id": "HumanEval/35",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/35.py"
},
{
"task_id": "HumanEval/36",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/36.py"
},
{
"task_id": "HumanEval/37",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/37.py"
},
{
"task_id": "HumanEval/38",
"passed": false,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/38.py"
},
{
"task_id": "HumanEval/39",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/39.py"
},
{
"task_id": "HumanEval/40",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/40.py"
},
{
"task_id": "HumanEval/41",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/41.py"
},
{
"task_id": "HumanEval/42",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/42.py"
},
{
"task_id": "HumanEval/43",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/43.py"
},
{
"task_id": "HumanEval/44",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/44.py"
},
{
"task_id": "HumanEval/45",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/45.py"
},
{
"task_id": "HumanEval/46",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/46.py"
},
{
"task_id": "HumanEval/47",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/47.py"
},
{
"task_id": "HumanEval/48",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/48.py"
},
{
"task_id": "HumanEval/49",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/49.py"
},
{
"task_id": "HumanEval/50",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/50.py"
},
{
"task_id": "HumanEval/51",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/51.py"
},
{
"task_id": "HumanEval/52",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/52.py"
},
{
"task_id": "HumanEval/53",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/53.py"
},
{
"task_id": "HumanEval/54",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/54.py"
},
{
"task_id": "HumanEval/55",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/55.py"
},
{
"task_id": "HumanEval/56",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/56.py"
},
{
"task_id": "HumanEval/57",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/57.py"
},
{
"task_id": "HumanEval/58",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/58.py"
},
{
"task_id": "HumanEval/59",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/59.py"
},
{
"task_id": "HumanEval/60",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/60.py"
},
{
"task_id": "HumanEval/61",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/61.py"
},
{
"task_id": "HumanEval/62",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/62.py"
},
{
"task_id": "HumanEval/63",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/63.py"
},
{
"task_id": "HumanEval/64",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/64.py"
},
{
"task_id": "HumanEval/65",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/65.py"
},
{
"task_id": "HumanEval/66",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/66.py"
},
{
"task_id": "HumanEval/67",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/67.py"
},
{
"task_id": "HumanEval/68",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/68.py"
},
{
"task_id": "HumanEval/69",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/69.py"
},
{
"task_id": "HumanEval/70",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/70.py"
},
{
"task_id": "HumanEval/71",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/71.py"
},
{
"task_id": "HumanEval/72",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/72.py"
},
{
"task_id": "HumanEval/73",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/73.py"
},
{
"task_id": "HumanEval/74",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/74.py"
},
{
"task_id": "HumanEval/75",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/75.py"
},
{
"task_id": "HumanEval/76",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/76.py"
},
{
"task_id": "HumanEval/77",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/77.py"
},
{
"task_id": "HumanEval/78",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/78.py"
},
{
"task_id": "HumanEval/79",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/79.py"
},
{
"task_id": "HumanEval/80",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/80.py"
},
{
"task_id": "HumanEval/81",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/81.py"
},
{
"task_id": "HumanEval/82",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/82.py"
},
{
"task_id": "HumanEval/83",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/83.py"
},
{
"task_id": "HumanEval/84",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/84.py"
},
{
"task_id": "HumanEval/85",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/85.py"
},
{
"task_id": "HumanEval/86",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/86.py"
},
{
"task_id": "HumanEval/87",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/87.py"
},
{
"task_id": "HumanEval/88",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/88.py"
},
{
"task_id": "HumanEval/89",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/89.py"
},
{
"task_id": "HumanEval/90",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/90.py"
},
{
"task_id": "HumanEval/91",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/91.py"
},
{
"task_id": "HumanEval/92",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/92.py"
},
{
"task_id": "HumanEval/93",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/93.py"
},
{
"task_id": "HumanEval/94",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/94.py"
},
{
"task_id": "HumanEval/95",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/95.py"
},
{
"task_id": "HumanEval/96",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/96.py"
},
{
"task_id": "HumanEval/97",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/97.py"
},
{
"task_id": "HumanEval/98",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/98.py"
},
{
"task_id": "HumanEval/99",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/99.py"
},
{
"task_id": "HumanEval/100",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/100.py"
},
{
"task_id": "HumanEval/101",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/101.py"
},
{
"task_id": "HumanEval/102",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/102.py"
},
{
"task_id": "HumanEval/103",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/103.py"
},
{
"task_id": "HumanEval/104",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/104.py"
},
{
"task_id": "HumanEval/105",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/105.py"
},
{
"task_id": "HumanEval/106",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/106.py"
},
{
"task_id": "HumanEval/107",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/107.py"
},
{
"task_id": "HumanEval/108",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/108.py"
},
{
"task_id": "HumanEval/109",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/109.py"
},
{
"task_id": "HumanEval/110",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/110.py"
},
{
"task_id": "HumanEval/111",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/111.py"
},
{
"task_id": "HumanEval/112",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/112.py"
},
{
"task_id": "HumanEval/113",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/113.py"
},
{
"task_id": "HumanEval/114",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/114.py"
},
{
"task_id": "HumanEval/115",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/115.py"
},
{
"task_id": "HumanEval/116",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/116.py"
},
{
"task_id": "HumanEval/117",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/117.py"
},
{
"task_id": "HumanEval/118",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/118.py"
},
{
"task_id": "HumanEval/119",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/119.py"
},
{
"task_id": "HumanEval/120",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/120.py"
},
{
"task_id": "HumanEval/121",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/121.py"
},
{
"task_id": "HumanEval/122",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/122.py"
},
{
"task_id": "HumanEval/123",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/123.py"
},
{
"task_id": "HumanEval/124",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/124.py"
},
{
"task_id": "HumanEval/125",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/125.py"
},
{
"task_id": "HumanEval/126",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/126.py"
},
{
"task_id": "HumanEval/127",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/127.py"
},
{
"task_id": "HumanEval/128",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/128.py"
},
{
"task_id": "HumanEval/129",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/129.py"
},
{
"task_id": "HumanEval/130",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/130.py"
},
{
"task_id": "HumanEval/131",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/131.py"
},
{
"task_id": "HumanEval/132",
"passed": false,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/132.py"
},
{
"task_id": "HumanEval/133",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/133.py"
},
{
"task_id": "HumanEval/134",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/134.py"
},
{
"task_id": "HumanEval/135",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/135.py"
},
{
"task_id": "HumanEval/136",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/136.py"
},
{
"task_id": "HumanEval/137",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/137.py"
},
{
"task_id": "HumanEval/138",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/138.py"
},
{
"task_id": "HumanEval/139",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/139.py"
},
{
"task_id": "HumanEval/140",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/140.py"
},
{
"task_id": "HumanEval/141",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/141.py"
},
{
"task_id": "HumanEval/142",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/142.py"
},
{
"task_id": "HumanEval/143",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/143.py"
},
{
"task_id": "HumanEval/144",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/144.py"
},
{
"task_id": "HumanEval/145",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/145.py"
},
{
"task_id": "HumanEval/146",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/146.py"
},
{
"task_id": "HumanEval/147",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/147.py"
},
{
"task_id": "HumanEval/148",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/148.py"
},
{
"task_id": "HumanEval/149",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/149.py"
},
{
"task_id": "HumanEval/150",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/150.py"
},
{
"task_id": "HumanEval/151",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/151.py"
},
{
"task_id": "HumanEval/152",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/152.py"
},
{
"task_id": "HumanEval/153",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/153.py"
},
{
"task_id": "HumanEval/154",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/154.py"
},
{
"task_id": "HumanEval/155",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/155.py"
},
{
"task_id": "HumanEval/156",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/156.py"
},
{
"task_id": "HumanEval/157",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/157.py"
},
{
"task_id": "HumanEval/158",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/158.py"
},
{
"task_id": "HumanEval/159",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/159.py"
},
{
"task_id": "HumanEval/160",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/160.py"
},
{
"task_id": "HumanEval/161",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/161.py"
},
{
"task_id": "HumanEval/162",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/162.py"
},
{
"task_id": "HumanEval/163",
"passed": true,
"error": null,
"solution_file": "/Users/lokesh/git/loki-mode/benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/163.py"
}
],
"passed": 161,
"failed": 3,
"errors": 0,
"pass_rate": 98.17,
"elapsed_seconds": 1263.46
}from typing import List
def has_close_elements(numbers: List[float], threshold: float) -> bool:
""" Check if in given list of numbers, are any two numbers closer to each other than
given threshold.
>>> has_close_elements([1.0, 2.0, 3.0], 0.5)
False
>>> has_close_elements([1.0, 2.8, 3.0, 4.0, 5.0, 2.0], 0.3)
True
"""
for i in range(len(numbers)):
for j in range(i + 1, len(numbers)):
if abs(numbers[i] - numbers[j]) < threshold:
return True
return Falsefrom typing import List
def separate_paren_groups(paren_string: str) -> List[str]:
""" Input to this function is a string containing multiple groups of nested parentheses. Your goal is to
separate those group into separate strings and return the list of those.
Separate groups are balanced (each open brace is properly closed) and not nested within each other
Ignore any spaces in the input string.
>>> separate_paren_groups('( ) (( )) (( )( ))')
['()', '(())', '(()())']
"""
paren_string = paren_string.replace(' ', '')
result = []
current_group = ''
depth = 0
for char in paren_string:
if char == '(':
depth += 1
current_group += char
elif char == ')':
depth -= 1
current_group += char
if depth == 0:
result.append(current_group)
current_group = ''
return resultdef is_palindrome(string: str) -> bool:
""" Test if given string is a palindrome """
return string == string[::-1]
def make_palindrome(string: str) -> str:
""" Find the shortest palindrome that begins with a supplied string.
Algorithm idea is simple:
- Find the longest postfix of supplied string that is a palindrome.
- Append to the end of the string reverse of a string prefix that comes before the palindromic suffix.
>>> make_palindrome('')
''
>>> make_palindrome('cat')
'catac'
>>> make_palindrome('cata')
'catac'
"""
if not string:
return ''
for i in range(len(string)):
if is_palindrome(string[i:]):
return string + string[:i][::-1]
return stringdef make_a_pile(n):
"""
Given a positive integer n, you have to make a pile of n levels of stones.
The first level has n stones.
The number of stones in the next level is:
- the next odd number if n is odd.
- the next even number if n is even.
Return the number of stones in each level in a list, where element at index
i represents the number of stones in the level (i+1).
Examples:
>>> make_a_pile(3)
[3, 5, 7]
"""
result = []
current = n
for _ in range(n):
result.append(current)
current += 2
return resultdef words_string(s):
"""
You will be given a string of words separated by commas or spaces. Your task is
to split the string into words and return an array of the words.
For example:
words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
"""
if not s:
return []
# Replace commas with spaces, then split on whitespace
s = s.replace(',', ' ')
return s.split()def choose_num(x, y):
"""This function takes two positive numbers x and y and returns the
biggest even integer number that is in the range [x, y] inclusive. If
there's no such number, then the function should return -1.
For example:
choose_num(12, 15) = 14
choose_num(13, 12) = -1
"""
if x > y:
return -1
if y % 2 == 0:
return y
if y - 1 >= x:
return y - 1
return -1def rounded_avg(n, m):
"""You are given two positive integers n and m, and your task is to compute the
average of the integers from n through m (including n and m).
Round the answer to the nearest integer and convert that to binary.
If n is greater than m, return -1.
Example:
rounded_avg(1, 5) => "0b11"
rounded_avg(7, 5) => -1
rounded_avg(10, 20) => "0b1111"
rounded_avg(20, 33) => "0b11010"
"""
if n > m:
return -1
avg = round((n + m) / 2)
return bin(avg)def unique_digits(x):
"""Given a list of positive integers x. return a sorted list of all
elements that hasn't any even digit.
Note: Returned list should be sorted in increasing order.
For example:
>>> unique_digits([15, 33, 1422, 1])
[1, 15, 33]
>>> unique_digits([152, 323, 1422, 10])
[]
"""
def has_even_digit(n):
while n > 0:
digit = n % 10
if digit % 2 == 0:
return True
n //= 10
return False
result = [num for num in x if not has_even_digit(num)]
return sorted(result)def by_length(arr):
"""
Given an array of integers, sort the integers that are between 1 and 9 inclusive,
reverse the resulting array, and then replace each digit by its corresponding name from
"One", "Two", "Three", "Four", "Five", "Six", "Seven", "Eight", "Nine".
For example:
arr = [2, 1, 1, 4, 5, 8, 2, 3]
-> sort arr -> [1, 1, 2, 2, 3, 4, 5, 8]
-> reverse arr -> [8, 5, 4, 3, 2, 2, 1, 1]
return ["Eight", "Five", "Four", "Three", "Two", "Two", "One", "One"]
If the array is empty, return an empty array:
arr = []
return []
If the array has any strange number ignore it:
arr = [1, -1 , 55]
-> sort arr -> [-1, 1, 55]
-> reverse arr -> [55, 1, -1]
return = ['One']
"""
names = {
1: "One",
2: "Two",
3: "Three",
4: "Four",
5: "Five",
6: "Six",
7: "Seven",
8: "Eight",
9: "Nine"
}
filtered = [x for x in arr if 1 <= x <= 9]
filtered.sort()
filtered.reverse()
return [names[x] for x in filtered]def f(n):
""" Implement the function f that takes n as a parameter,
and returns a list of size n, such that the value of the element at index i is the factorial of i if i is even
or the sum of numbers from 1 to i otherwise.
i starts from 1.
the factorial of i is the multiplication of the numbers from 1 to i (1 * 2 * ... * i).
Example:
f(5) == [1, 2, 6, 24, 15]
"""
result = []
for i in range(1, n + 1):
if i % 2 == 0:
# factorial of i
factorial = 1
for j in range(1, i + 1):
factorial *= j
result.append(factorial)
else:
# sum of numbers from 1 to i
result.append(sum(range(1, i + 1)))
return resultdef even_odd_palindrome(n):
"""
Given a positive integer n, return a tuple that has the number of even and odd
integer palindromes that fall within the range(1, n), inclusive.
Example 1:
Input: 3
Output: (1, 2)
Explanation:
Integer palindrome are 1, 2, 3. one of them is even, and two of them are odd.
Example 2:
Input: 12
Output: (4, 6)
Explanation:
Integer palindrome are 1, 2, 3, 4, 5, 6, 7, 8, 9, 11. four of them are even, and 6 of them are odd.
Note:
1. 1 <= n <= 10^3
2. returned tuple has the number of even and odd integer palindromes respectively.
"""
even_count = 0
odd_count = 0
for i in range(1, n + 1):
s = str(i)
if s == s[::-1]:
if i % 2 == 0:
even_count += 1
else:
odd_count += 1
return (even_count, odd_count)def count_nums(arr):
"""
Write a function count_nums which takes an array of integers and returns
the number of elements which has a sum of digits > 0.
If a number is negative, then its first signed digit will be negative:
e.g. -123 has signed digits -1, 2, and 3.
>>> count_nums([]) == 0
>>> count_nums([-1, 11, -11]) == 1
>>> count_nums([1, 1, 2]) == 3
"""
count = 0
for num in arr:
if num == 0:
continue
elif num > 0:
digit_sum = sum(int(d) for d in str(num))
else:
s = str(abs(num))
digit_sum = -int(s[0]) + sum(int(d) for d in s[1:])
if digit_sum > 0:
count += 1
return countdef move_one_ball(arr):
"""We have an array 'arr' of N integers arr[1], arr[2], ..., arr[N].The
numbers in the array will be randomly ordered. Your task is to determine if
it is possible to get an array sorted in non-decreasing order by performing
the following operation on the given array:
You are allowed to perform right shift operation any number of times.
One right shift operation means shifting all elements of the array by one
position in the right direction. The last element of the array will be moved to
the starting position in the array i.e. 0th index.
If it is possible to obtain the sorted array by performing the above operation
then return True else return False.
If the given array is empty then return True.
Note: The given list is guaranteed to have unique elements.
For Example:
move_one_ball([3, 4, 5, 1, 2])==>True
Explanation: By performin 2 right shift operations, non-decreasing order can
be achieved for the given array.
move_one_ball([3, 5, 4, 1, 2])==>False
Explanation:It is not possible to get non-decreasing order for the given
array by performing any number of right shift operations.
"""
if len(arr) == 0:
return True
# Count the number of "breaks" where arr[i] > arr[i+1]
# For a rotated sorted array, there should be at most one such break
breaks = 0
n = len(arr)
for i in range(n):
if arr[i] > arr[(i + 1) % n]:
breaks += 1
# If there's at most one break, the array can be sorted by rotation
return breaks <= 1from typing import List
def string_xor(a: str, b: str) -> str:
""" Input are two strings a and b consisting only of 1s and 0s.
Perform binary XOR on these inputs and return result also as a string.
>>> string_xor('010', '110')
'100'
"""
result = []
for char_a, char_b in zip(a, b):
if char_a == char_b:
result.append('0')
else:
result.append('1')
return ''.join(result)def exchange(lst1, lst2):
"""In this problem, you will implement a function that takes two lists of numbers,
and determines whether it is possible to perform an exchange of elements
between them to make lst1 a list of only even numbers.
There is no limit on the number of exchanged elements between lst1 and lst2.
If it is possible to exchange elements between the lst1 and lst2 to make
all the elements of lst1 to be even, return "YES".
Otherwise, return "NO".
For example:
exchange([1, 2, 3, 4], [1, 2, 3, 4]) => "YES"
exchange([1, 2, 3, 4], [1, 5, 3, 4]) => "NO"
It is assumed that the input lists will be non-empty.
"""
odd_count_lst1 = sum(1 for x in lst1 if x % 2 != 0)
even_count_lst2 = sum(1 for x in lst2 if x % 2 == 0)
if even_count_lst2 >= odd_count_lst1:
return "YES"
else:
return "NO"def histogram(test):
"""Given a string representing a space separated lowercase letters, return a dictionary
of the letter with the most repetition and containing the corresponding count.
If several letters have the same occurrence, return all of them.
Example:
histogram('a b c') == {'a': 1, 'b': 1, 'c': 1}
histogram('a b b a') == {'a': 2, 'b': 2}
histogram('a b c a b') == {'a': 2, 'b': 2}
histogram('b b b b a') == {'b': 4}
histogram('') == {}
"""
if not test or test.strip() == '':
return {}
letters = test.split()
counts = {}
for letter in letters:
counts[letter] = counts.get(letter, 0) + 1
if not counts:
return {}
max_count = max(counts.values())
return {letter: count for letter, count in counts.items() if count == max_count}def reverse_delete(s,c):
"""Task
We are given two strings s and c, you have to deleted all the characters in s that are equal to any character in c
then check if the result string is palindrome.
A string is called palindrome if it reads the same backward as forward.
You should return a tuple containing the result string and True/False for the check.
Example
For s = "abcde", c = "ae", the result should be ('bcd',False)
For s = "abcdef", c = "b" the result should be ('acdef',False)
For s = "abcdedcba", c = "ab", the result should be ('cdedc',True)
"""
result = ''.join(char for char in s if char not in c)
is_palindrome = result == result[::-1]
return (result, is_palindrome)def odd_count(lst):
"""Given a list of strings, where each string consists of only digits, return a list.
Each element i of the output should be "the number of odd elements in the
string i of the input." where all the i's should be replaced by the number
of odd digits in the i'th string of the input.
>>> odd_count(['1234567'])
["the number of odd elements 4n the str4ng 4 of the 4nput."]
>>> odd_count(['3',"11111111"])
["the number of odd elements 1n the str1ng 1 of the 1nput.",
"the number of odd elements 8n the str8ng 8 of the 8nput."]
"""
result = []
for s in lst:
count = sum(1 for c in s if c in '13579')
template = "the number of odd elements in the string i of the input."
replaced = template.replace('i', str(count))
result.append(replaced)
return resultdef minSubArraySum(nums):
"""
Given an array of integers nums, find the minimum sum of any non-empty sub-array
of nums.
Example
minSubArraySum([2, 3, 4, 1, 2, 4]) == 1
minSubArraySum([-1, -2, -3]) == -6
"""
min_sum = nums[0]
current_sum = nums[0]
for i in range(1, len(nums)):
current_sum = min(nums[i], current_sum + nums[i])
min_sum = min(min_sum, current_sum)
return min_sumdef max_fill(grid, capacity):
import math
"""
You are given a rectangular grid of wells. Each row represents a single well,
and each 1 in a row represents a single unit of water.
Each well has a corresponding bucket that can be used to extract water from it,
and all buckets have the same capacity.
Your task is to use the buckets to empty the wells.
Output the number of times you need to lower the buckets.
Example 1:
Input:
grid : [[0,0,1,0], [0,1,0,0], [1,1,1,1]]
bucket_capacity : 1
Output: 6
Example 2:
Input:
grid : [[0,0,1,1], [0,0,0,0], [1,1,1,1], [0,1,1,1]]
bucket_capacity : 2
Output: 5
Example 3:
Input:
grid : [[0,0,0], [0,0,0]]
bucket_capacity : 5
Output: 0
Constraints:
* all wells have the same length
* 1 <= grid.length <= 10^2
* 1 <= grid[:,1].length <= 10^2
* grid[i][j] -> 0 | 1
* 1 <= capacity <= 10
"""
total_lowerings = 0
for row in grid:
water_units = sum(row)
if water_units > 0:
total_lowerings += math.ceil(water_units / capacity)
return total_loweringsdef sort_array(arr):
"""
In this Kata, you have to sort an array of non-negative integers according to
number of ones in their binary representation in ascending order.
For similar number of ones, sort based on decimal value.
It must be implemented like this:
>>> sort_array([1, 5, 2, 3, 4]) == [1, 2, 3, 4, 5]
>>> sort_array([-2, -3, -4, -5, -6]) == [-6, -5, -4, -3, -2]
>>> sort_array([1, 0, 2, 3, 4]) [0, 1, 2, 3, 4]
"""
return sorted(arr, key=lambda x: (bin(x).count('1'), x))def select_words(s, n):
"""Given a string s and a natural number n, you have been tasked to implement
a function that returns a list of all words from string s that contain exactly
n consonants, in order these words appear in the string s.
If the string s is empty then the function should return an empty list.
Note: you may assume the input string contains only letters and spaces.
Examples:
select_words("Mary had a little lamb", 4) ==> ["little"]
select_words("Mary had a little lamb", 3) ==> ["Mary", "lamb"]
select_words("simple white space", 2) ==> []
select_words("Hello world", 4) ==> ["world"]
select_words("Uncle sam", 3) ==> ["Uncle"]
"""
if not s:
return []
vowels = set('aeiouAEIOU')
result = []
for word in s.split():
consonant_count = sum(1 for char in word if char.isalpha() and char not in vowels)
if consonant_count == n:
result.append(word)
return resultdef get_closest_vowel(word):
"""You are given a word. Your task is to find the closest vowel that stands between
two consonants from the right side of the word (case sensitive).
Vowels in the beginning and ending doesn't count. Return empty string if you didn't
find any vowel met the above condition.
You may assume that the given string contains English letter only.
Example:
get_closest_vowel("yogurt") ==> "u"
get_closest_vowel("FULL") ==> "U"
get_closest_vowel("quick") ==> ""
get_closest_vowel("ab") ==> ""
"""
if len(word) < 3:
return ""
vowels = "aeiouAEIOU"
for i in range(len(word) - 2, 0, -1):
if word[i] in vowels:
if word[i-1] not in vowels and word[i+1] not in vowels:
return word[i]
return ""def match_parens(lst):
'''
You are given a list of two strings, both strings consist of open
parentheses '(' or close parentheses ')' only.
Your job is to check if it is possible to concatenate the two strings in
some order, that the resulting string will be good.
A string S is considered to be good if and only if all parentheses in S
are balanced. For example: the string '(())()' is good, while the string
'())' is not.
Return 'Yes' if there's a way to make a good string, and return 'No' otherwise.
Examples:
match_parens(['()(', ')']) == 'Yes'
match_parens([')', ')']) == 'No'
'''
def is_balanced(s):
count = 0
for c in s:
if c == '(':
count += 1
else:
count -= 1
if count < 0:
return False
return count == 0
s1, s2 = lst[0], lst[1]
if is_balanced(s1 + s2) or is_balanced(s2 + s1):
return 'Yes'
return 'No'from typing import List, Optional
def longest(strings: List[str]) -> Optional[str]:
""" Out of list of strings, return the longest one. Return the first one in case of multiple
strings of the same length. Return None in case the input list is empty.
>>> longest([])
>>> longest(['a', 'b', 'c'])
'a'
>>> longest(['a', 'bb', 'ccc'])
'ccc'
"""
if not strings:
return None
longest_str = strings[0]
for s in strings[1:]:
if len(s) > len(longest_str):
longest_str = s
return longest_strdef maximum(arr, k):
"""
Given an array arr of integers and a positive integer k, return a sorted list
of length k with the maximum k numbers in arr.
Example 1:
Input: arr = [-3, -4, 5], k = 3
Output: [-4, -3, 5]
Example 2:
Input: arr = [4, -4, 4], k = 2
Output: [4, 4]
Example 3:
Input: arr = [-3, 2, 1, 2, -1, -2, 1], k = 1
Output: [2]
Note:
1. The length of the array will be in the range of [1, 1000].
2. The elements in the array will be in the range of [-1000, 1000].
3. 0 <= k <= len(arr)
"""
sorted_arr = sorted(arr, reverse=True)
top_k = sorted_arr[:k]
return sorted(top_k)def solution(lst):
"""Given a non-empty list of integers, return the sum of all of the odd elements that are in even positions.
Examples
solution([5, 8, 7, 1]) ==> 12
solution([3, 3, 3, 3, 3]) ==> 9
solution([30, 13, 24, 321]) ==>0
"""
return sum(x for i, x in enumerate(lst) if i % 2 == 0 and x % 2 == 1)def add_elements(arr, k):
"""
Given a non-empty array of integers arr and an integer k, return
the sum of the elements with at most two digits from the first k elements of arr.
Example:
Input: arr = [111,21,3,4000,5,6,7,8,9], k = 4
Output: 24 # sum of 21 + 3
Constraints:
1. 1 <= len(arr) <= 100
2. 1 <= k <= len(arr)
"""
total = 0
for i in range(k):
if -99 <= arr[i] <= 99:
total += arr[i]
return totaldef get_odd_collatz(n):
"""
Given a positive integer n, return a sorted list that has the odd numbers in collatz sequence.
The Collatz conjecture is a conjecture in mathematics that concerns a sequence defined
as follows: start with any positive integer n. Then each term is obtained from the
previous term as follows: if the previous term is even, the next term is one half of
the previous term. If the previous term is odd, the next term is 3 times the previous
term plus 1. The conjecture is that no matter what value of n, the sequence will always reach 1.
Note:
1. Collatz(1) is [1].
2. returned list sorted in increasing order.
For example:
get_odd_collatz(5) returns [1, 5] # The collatz sequence for 5 is [5, 16, 8, 4, 2, 1], so the odd numbers are only 1, and 5.
"""
odd_numbers = []
current = n
while current != 1:
if current % 2 == 1:
odd_numbers.append(current)
if current % 2 == 0:
current = current // 2
else:
current = 3 * current + 1
odd_numbers.append(1)
return sorted(odd_numbers)def valid_date(date):
"""You have to write a function which validates a given date string and
returns True if the date is valid otherwise False.
The date is valid if all of the following rules are satisfied:
1. The date string is not empty.
2. The number of days is not less than 1 or higher than 31 days for months 1,3,5,7,8,10,12. And the number of days is not less than 1 or higher than 30 days for months 4,6,9,11. And, the number of days is not less than 1 or higher than 29 for the month 2.
3. The months should not be less than 1 or higher than 12.
4. The date should be in the format: mm-dd-yyyy
for example:
valid_date('03-11-2000') => True
valid_date('15-01-2012') => False
valid_date('04-0-2040') => False
valid_date('06-04-2020') => True
valid_date('06/04/2020') => False
"""
if not date:
return False
try:
parts = date.split('-')
if len(parts) != 3:
return False
mm, dd, yyyy = parts
if len(mm) != 2 or len(dd) != 2 or len(yyyy) != 4:
return False
month = int(mm)
day = int(dd)
year = int(yyyy)
if month < 1 or month > 12:
return False
if day < 1:
return False
if month in [1, 3, 5, 7, 8, 10, 12]:
if day > 31:
return False
elif month in [4, 6, 9, 11]:
if day > 30:
return False
elif month == 2:
if day > 29:
return False
return True
except:
return Falsedef split_words(txt):
'''
Given a string of words, return a list of words split on whitespace, if no whitespaces exists in the text you
should split on commas ',' if no commas exists you should return the number of lower-case letters with odd order in the
alphabet, ord('a') = 0, ord('b') = 1, ... ord('z') = 25
Examples
split_words("Hello world!") ➞ ["Hello", "world!"]
split_words("Hello,world!") ➞ ["Hello", "world!"]
split_words("abcdef") == 3
'''
if ' ' in txt:
return txt.split()
elif ',' in txt:
return txt.split(',')
else:
count = 0
for char in txt:
if char.islower() and (ord(char) - ord('a')) % 2 == 1:
count += 1
return countdef is_sorted(lst):
'''
Given a list of numbers, return whether or not they are sorted
in ascending order. If list has more than 1 duplicate of the same
number, return False. Assume no negative numbers and only integers.
Examples
is_sorted([5]) ➞ True
is_sorted([1, 2, 3, 4, 5]) ➞ True
is_sorted([1, 3, 2, 4, 5]) ➞ False
is_sorted([1, 2, 3, 4, 5, 6]) ➞ True
is_sorted([1, 2, 3, 4, 5, 6, 7]) ➞ True
is_sorted([1, 3, 2, 4, 5, 6, 7]) ➞ False
is_sorted([1, 2, 2, 3, 3, 4]) ➞ True
is_sorted([1, 2, 2, 2, 3, 4]) ➞ False
'''
# Check if sorted in ascending order
for i in range(len(lst) - 1):
if lst[i] > lst[i + 1]:
return False
# Check for more than 1 duplicate of any number
from collections import Counter
counts = Counter(lst)
for count in counts.values():
if count > 2:
return False
return Truedef intersection(interval1, interval2):
"""You are given two intervals,
where each interval is a pair of integers. For example, interval = (start, end) = (1, 2).
The given intervals are closed which means that the interval (start, end)
includes both start and end.
For each given interval, it is assumed that its start is less or equal its end.
Your task is to determine whether the length of intersection of these two
intervals is a prime number.
Example, the intersection of the intervals (1, 3), (2, 4) is (2, 3)
which its length is 1, which not a prime number.
If the length of the intersection is a prime number, return "YES",
otherwise, return "NO".
If the two intervals don't intersect, return "NO".
[input/output] samples:
intersection((1, 2), (2, 3)) ==> "NO"
intersection((-1, 1), (0, 4)) ==> "NO"
intersection((-3, -1), (-5, 5)) ==> "YES"
"""
def is_prime(n):
if n < 2:
return False
if n == 2:
return True
if n % 2 == 0:
return False
for i in range(3, int(n**0.5) + 1, 2):
if n % i == 0:
return False
return True
start1, end1 = interval1
start2, end2 = interval2
intersection_start = max(start1, start2)
intersection_end = min(end1, end2)
if intersection_start > intersection_end:
return "NO"
length = intersection_end - intersection_start
if is_prime(length):
return "YES"
else:
return "NO"def prod_signs(arr):
"""
You are given an array arr of integers and you need to return
sum of magnitudes of integers multiplied by product of all signs
of each number in the array, represented by 1, -1 or 0.
Note: return None for empty arr.
Example:
>>> prod_signs([1, 2, 2, -4]) == -9
>>> prod_signs([0, 1]) == 0
>>> prod_signs([]) == None
"""
if not arr:
return None
sum_magnitudes = sum(abs(x) for x in arr)
sign_product = 1
for x in arr:
if x == 0:
return 0
elif x < 0:
sign_product *= -1
return sum_magnitudes * sign_productdef greatest_common_divisor(a: int, b: int) -> int:
""" Return a greatest common divisor of two integers a and b
>>> greatest_common_divisor(3, 5)
1
>>> greatest_common_divisor(25, 15)
5
"""
while b:
a, b = b, a % b
return adef tri(n):
"""Everyone knows Fibonacci sequence, it was studied deeply by mathematicians in
the last couple centuries. However, what people don't know is Tribonacci sequence.
Tribonacci sequence is defined by the recurrence:
tri(1) = 3
tri(n) = 1 + n / 2, if n is even.
tri(n) = tri(n - 1) + tri(n - 2) + tri(n + 1), if n is odd.
For example:
tri(2) = 1 + (2 / 2) = 2
tri(4) = 3
tri(3) = tri(2) + tri(1) + tri(4)
= 2 + 3 + 3 = 8
You are given a non-negative integer number n, you have to a return a list of the
first n + 1 numbers of the Tribonacci sequence.
Examples:
tri(3) = [1, 3, 2, 8]
"""
if n == 0:
return [1]
result = [1, 3]
for i in range(2, n + 1):
if i % 2 == 0:
result.append(1 + i // 2)
else:
result.append(result[i - 1] + result[i - 2] + (1 + (i + 1) // 2))
return resultdef strlen(string: str) -> int:
""" Return length of given string
>>> strlen('')
0
>>> strlen('abc')
3
"""
return len(string)Related skills
How it compares
Pick loki-mode for native GitHub PR review automation instead of local-only review prompts that never post CI feedback.
FAQ
Which GitHub events trigger loki-mode reviews?
loki-mode triggers on pull_request events with types opened and synchronize. The claude-review job runs on ubuntu-latest and can optionally filter by file paths or PR author association.
What permissions does the loki-mode workflow need?
loki-mode requires GitHub Actions permissions for contents read, pull-requests read, issues read, and id-token write. These enable Claude to authenticate and post structured review feedback on pull requests.
Is Loki Mode safe to install?
skills.sh reports 0 of 3 security scanners passed. Review the Security Audits panel on this page before installing in production.