Devtu Github

DevTU GitHub Workflow

Safely push ToolUniverse code to GitHub by enforcing pre-push cleanup, pre-commit hooks, and test validation.

Instructions

When the user wants to push code, fix CI, or prepare a commit, follow this workflow:

Phase 1: Pre-Push Cleanup

1. Move temp files out of root - session docs and ad-hoc test scripts must NOT be pushed:

# Move session markdown files to temp_docs_and_tests/
for f in $(ls *.md 2>/dev/null | grep -v README.md | grep -v CHANGELOG.md | grep -v LICENSE.md); do
  mv "$f" temp_docs_and_tests/
done

# Move root-level test scripts to temp_docs_and_tests/
for f in $(ls test_*.py 2>/dev/null); do
  mv "$f" temp_docs_and_tests/
done

2. Verify nothing unwanted is staged:

git status --short

Red flags - these should NEVER be staged:

• *_SUMMARY.md, *_REPORT.md, SESSION_*.md in root
• test_*.py in root (these are ad-hoc scripts, not real tests)
• .env or credential files
• temp_docs_and_tests/ contents

Phase 2: Activate Pre-Commit Hooks

3. Ensure pre-commit is installed and active:

pre-commit install

This enables automatic checks on every git commit:

• ruff check --fix - Python linting with auto-fix
• ruff format - Code formatting
• YAML/TOML validation
• Trailing whitespace removal
• End of file fixes

4. Verify hooks are active:

ls -la .git/hooks/pre-commit

Phase 3: Run Tests

5. Run the full test suite locally:

python -m pytest tests/ -x --tb=short -q

6. If tests fail, diagnose using the error patterns below and fix before proceeding.

Phase 4: Commit and Push

7. Stage only specific files (never use git add . or git add -A):

git add src/tooluniverse/specific_file.py tests/specific_test.py

8. Commit (pre-commit hooks run automatically):

git commit -m "Clear, descriptive message"

9. Rebase onto latest main BEFORE pushing (CRITICAL — prevents PR conflicts):

git fetch origin
git stash            # stash any uncommitted work
git rebase origin/main
git stash pop        # restore uncommitted work

If rebase conflicts arise, resolve them (keep our newer changes), then:

git add <conflicted-file>
git rebase --continue

10. Push (force-with-lease after a rebase):

git push --force-with-lease origin <branch-name>

After pushing, verify the PR is conflict-free:

gh pr view <PR-number> --json mergeable,mergeStateStatus
# Must show: "mergeable":"MERGEABLE"

Files That Must NEVER Be Pushed

Temp Session Documents (Root-Level .md)

These are session notes created during development. Move to temp_docs_and_tests/ before committing:

Exception: README.md, CHANGELOG.md, LICENSE.md are real docs and MUST stay.

Root-Level Test Scripts

Ad-hoc test scripts like test_*.py in root are NOT part of the test suite (tests/ directory is). Move them to temp_docs_and_tests/:

Other Excluded Files

• .env - Environment variables with secrets
• temp_docs_and_tests/ - Already in .gitignore
• .claude/ - Claude Code configuration
• __pycache__/, *.pyc - Python bytecode
• .DS_Store - macOS metadata

Common Test Failure Patterns

Pattern 1: KeyError: 'role'

Symptom: KeyError: 'role' when accessing message dicts

Fix: Add return_message=True to tu.run() and use .get():

messages = tu.run(calls, use_cache=True, return_message=True)
if msg.get("role") == "tool":

Pattern 2: Mock Not Subscriptable

Symptom: TypeError: 'Mock' object is not subscriptable

Fix: Use real dicts for all_tool_dict and add _get_tool_instance:

mock_tu.all_tool_dict = {"Tool": mock_tool}
mock_tu._get_tool_instance = lambda name, cache=True: mock_tu.all_tool_dict.get(name)

Pattern 3: Linting Errors (F841, E731)

Fix F841 (unused variable): Use _ prefix or _ = func() Fix E731 (lambda assignment): Replace with def

Pattern 4: Temp Files Tracked by Git

Symptom: git status shows temp files as modified/staged

Fix:

git rm -r --cached temp_docs_and_tests/
git rm --cached API_DISCOVERY_SESSION_SUMMARY.md
git commit -m "Remove temp files from tracking"

Pre-Commit Hook Configuration

The project uses .pre-commit-config.yaml:

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    hooks: [end-of-file-fixer, trailing-whitespace, check-yaml, check-toml]
  - repo: https://github.com/astral-sh/ruff-pre-commit
    hooks: [ruff-check --fix, ruff-format]

Scope: Only files matching ^(ToolUniverse/)?src/tooluniverse/

Quick Reference

Pre-Push Checklist

Before every push, verify:

• [ ] Temp markdown files moved from root to temp_docs_and_tests/
• [ ] Root-level test_*.py scripts moved to temp_docs_and_tests/
• [ ] Pre-commit hooks installed (pre-commit install)
• [ ] All tests pass locally (pytest tests/ -x)
• [ ] No linting errors
• [ ] Only relevant files staged (no .env, no temp files)
• [ ] Commit message is clear and descriptive
• [ ] Correct branch selected

Git Commit Guidelines

• Never include AI attribution in commits
• Never commit session documentation markdown files
• Use git add <specific-files> instead of git add .
• Write clean, professional commit messages
• One logical change per commit

DevTU GitHub CI & Testing Skill

You are an expert at debugging and fixing GitHub CI failures, test issues, and pre-commit hook problems in ToolUniverse development.

Mission

When the user reports GitHub CI failures, test failures, or wants to push changes to GitHub: 1. Activate pre-commit hooks if not already active 2. Run tests locally to catch issues before pushing 3. Fix test failures systematically 4. Ensure temp files are not pushed to GitHub 5. Commit and push fixes properly

Pre-Commit Hook Setup

Step 1: Check if Pre-Commit is Installed

pre-commit --version

If not installed:

pip install pre-commit

Step 2: Activate Pre-Commit Hooks

pre-commit install

This installs hooks that run automatically on git commit:

• ruff check - Python linting
• ruff format - Code formatting
• YAML/TOML validation
• Trailing whitespace removal
• End of file fixes

Step 3: Verify Installation

ls -la .git/hooks/pre-commit

Should show an executable pre-commit file.

Running Tests Locally

Before Every Push

CRITICAL: Always run tests locally before pushing to avoid CI failures.

Full Test Suite (Recommended)

python -m pytest tests/ -x --tb=short -q

Options:

• -x: Stop at first failure
• --tb=short: Short traceback format
• -q: Quiet mode (less verbose)

Expected time: ~6 minutes for full suite

Quick Test (Specific Files)

python -m pytest tests/test_task_manager.py -v
python -m pytest tests/test_tooluniverse_cache_integration.py -v
python -m pytest tests/unit/test_run_parameters.py -v

Test a Single Failing Test

python -m pytest tests/path/to/test.py::TestClass::test_method -xvs

Common Test Failure Patterns

Pattern 1: KeyError: 'role' in Message Tests

Symptom:

KeyError: 'role'
# in code like: if msg["role"] == "tool"

Root Cause: The test is calling tu.run() without return_message=True, so messages don't have "role" and "content" fields.

How tu.run() Works:

• return_message=False (default): Returns raw results as list
• return_message=True: Returns formatted messages with "role" and "content"

Fix:

# WRONG:
messages = tu.run(batch_calls, use_cache=True)
if msg["role"] == "tool":  # ❌ KeyError!

# CORRECT:
messages = tu.run(batch_calls, use_cache=True, return_message=True)
if msg.get("role") == "tool":  # ✅ Safe access

Files Affected Today:

• tests/test_tooluniverse_cache_integration.py
• tests/unit/test_run_parameters.py

Pattern 2: Mock Object Not Subscriptable

Symptom:

TypeError: 'Mock' object is not subscriptable
# in code like: result["data"]["value"]

Root Cause: The mock is returning a Mock object instead of actual data because: 1. Mock methods aren't configured properly 2. Missing _get_tool_instance method on mock ToolUniverse 3. Shared mock instances between tests

Fix for Mock ToolUniverse:

@pytest.fixture
def mock_tool_universe():
    mock_tu = Mock()

    # Create SEPARATE mock tools (avoid shared state)
    mock_tool1 = Mock()
    mock_tool1.run = AsyncMock(return_value={"data": {"result": "success"}})

    mock_tool2 = Mock()
    mock_tool2.run = AsyncMock(return_value={"data": {"result": "success"}})

    # Use real dict (not mock)
    mock_tu.all_tool_dict = {"TestTool": mock_tool1, "OtherTool": mock_tool2}

    # Add _get_tool_instance method (TaskManager needs this!)
    def get_tool_instance(tool_name, cache=True):
        return mock_tu.all_tool_dict.get(tool_name)

    mock_tu._get_tool_instance = get_tool_instance

    return mock_tu

Fix for Async Tool Mocks:

# WRONG:
mock_tool.run = async_function  # Direct assignment

# CORRECT:
mock_tool.run = AsyncMock(side_effect=async_function)  # Proper async mock

Files Affected Today:

• tests/test_task_manager.py

Pattern 3: Linting Errors (F841, E731)

Symptom:

F841 Local variable assigned but never used
E731 Do not assign a lambda expression, use a def

Common F841 Fixes:

# Unused variable
result = some_function()  # ❌ F841 if not used

# Fix 1: Use underscore
_ = some_function()  # ✅ Indicates intentionally unused

# Fix 2: Actually use it
result = some_function()
assert result is not None  # ✅ Now it's used

Common E731 Fixes:

# Lambda assignment
get_value = lambda x: x * 2  # ❌ E731

# Fix: Use def
def get_value(x):  # ✅
    return x * 2

Pattern 4: Temp Files Being Pushed to GitHub

Symptom:

User: "do not push temp folder into github!"

Root Cause: Files were added to git using git mv before being added to .gitignore, so they're tracked even though .gitignore lists them.

Fix:

# 1. Remove from git tracking (keeps local files)
git rm -r --cached temp_docs_and_tests/

# 2. Verify .gitignore has the entry
grep "temp_docs_and_tests" .gitignore
# Should show: temp_docs_and_tests/

# 3. Commit the removal
git commit -m "Remove temp_docs_and_tests/ from git tracking"

# 4. Push
git push origin auto

# 5. Verify (local files exist, git doesn't track)
ls temp_docs_and_tests/ | wc -l  # Should show files
git ls-files temp_docs_and_tests/ | wc -l  # Should show 0

Prevention: Always add folders to .gitignore BEFORE creating/moving files:

echo "temp_docs_and_tests/" >> .gitignore
git add .gitignore
git commit -m "Add temp folder to gitignore"

Systematic Debugging Workflow

Step 1: Activate Pre-Commit Hook

pre-commit install

Step 2: Run Tests Locally

python -m pytest tests/ -x --tb=short -q 2>&1 | tail -50

Look for:

• FAILED tests/... - Which test failed
• Error message - KeyError, TypeError, AssertionError, etc.
• Line number - Where the failure occurred

Step 3: Reproduce the Specific Failure

python -m pytest tests/path/to/test.py::TestClass::test_method -xvs

Step 4: Read the Test File

# Read the failing test to understand what it's testing
cat tests/path/to/test.py | grep -A 20 "def test_method"

Step 5: Apply Pattern-Based Fix

• KeyError 'role' → Add return_message=True and use .get()
• Mock not subscriptable → Fix mock configuration
• F841/E731 → Fix linting issues
• Temp files pushed → Remove from git tracking

Step 6: Verify Fix Locally

python -m pytest tests/path/to/test.py -xvs

Should see: 1 passed

Step 7: Run Full Test Suite

python -m pytest tests/ -x --tb=short -q

Ensure no regressions were introduced.

Step 8: Commit with Pre-Commit Hook

git add <fixed_files>
git commit -m "Fix test: <brief description>"

The pre-commit hook will run automatically and check:

• ✅ Ruff linting
• ✅ Ruff formatting
• ✅ YAML/TOML validity
• ✅ Trailing whitespace

Step 9: Push to GitHub

git push origin auto

Quick Reference Commands

Pre-Commit

pre-commit install              # Activate hooks
pre-commit run --all-files      # Run manually on all files
pre-commit autoupdate           # Update hook versions

Testing

# All tests
pytest tests/ -x --tb=short -q

# Specific test
pytest tests/test_file.py::TestClass::test_method -xvs

# With coverage
pytest tests/ --cov=src/tooluniverse --cov-report=term-missing

# Stop after N failures
pytest tests/ --maxfail=3

Git

# Check what will be committed
git status --short

# Unstage files
git restore --staged <file>

# Remove from tracking but keep local
git rm --cached <file>

# Show what changed in last commit
git show HEAD

# Amend last commit (use carefully!)
git commit --amend --no-edit

Common Mistakes to Avoid

❌ Don't: Push Without Running Tests

git add .
git commit -m "Fix"
git push  # ❌ Might fail CI!

✅ Do: Test Before Push

python -m pytest tests/ -x --tb=short -q  # Run tests first
git add <specific_files>
git commit -m "Fix test_something: add return_message=True"
git push

❌ Don't: Modify Multiple Unrelated Things

# Commit mixes test fixes with new features
git commit -m "Fix tests and add new feature"  # ❌ Hard to review

✅ Do: Commit Fixes Separately

git add tests/test_cache.py
git commit -m "Fix test_cache: add return_message=True"

git add src/feature.py
git commit -m "Add new feature X"

❌ Don't: Use git add . Blindly

git add .  # ❌ Might include temp files, logs, etc.

✅ Do: Add Specific Files

git add tests/test_file.py src/module.py
# Or review with: git add -p

What to Push and What NOT to Push

✅ ALWAYS Push (Production Code)

Source Code:

• src/tooluniverse/*.py - Core library code
• tests/*.py - Test files
• examples/*.py - Example scripts
• skills/*/ - Skill files (use git add -f if in .gitignore)

Configuration:

• pyproject.toml - Project configuration
• setup.py - Package setup
• .pre-commit-config.yaml - Pre-commit configuration
• pytest.ini - Test configuration
• .gitignore - Git ignore rules

Documentation:

• README.md - Main documentation
• docs/**/*.rst - Sphinx documentation
• docs/**/*.md - Markdown documentation (NOT temp docs!)
• CHANGELOG.md - Version history
• LICENSE - License file

❌ NEVER Push (Temporary/Local Files)

Temp Folders:

• temp_docs_and_tests/ - Temporary documentation and test files
• temp/, tmp/ - Any temporary directories
• .temp/, ._temp/ - Hidden temp directories

Build Artifacts:

• build/, dist/ - Package build outputs
• *.egg-info/ - Python package metadata
• __pycache__/ - Python bytecode cache
• *.pyc, *.pyo - Compiled Python files

IDE and Editor Files:

• .vscode/ - VS Code settings (usually)
• .idea/ - PyCharm settings
• *.swp, *.swo - Vim swap files
• .DS_Store - macOS finder metadata

Logs and Data:

• *.log - Log files
• *.sqlite, *.db - Database files (unless example data)
• cache/ - Cache directories
• *.tmp - Temporary files

Environment Files:

• .env - Environment variables (contains secrets!)
• .env.local - Local environment config
• venv/, env/ - Virtual environments
• .python-version - Local Python version

Personal Configuration:

• .claude/ - Claude Code configuration
• *.local - Personal config files

⚠️ MAYBE Push (Check First)

Skills:

• skills/devtu-*/ - Development skills (check .gitignore)
• If creating new devtu skill: use git add -f skills/devtu-skillname/

Data Files:

• Small example data: ✅ Push
• Large datasets: ❌ Don't push (use Git LFS or external storage)

Configuration:

• .vscode/settings.json - Only if team shares settings
• Makefile - ✅ Push if used for automation

How to Check What Will Be Pushed

Before committing:

# See what files are staged
git status --short

# See detailed diff of what will be committed
git diff --cached

# Check if a file is ignored
git check-ignore -v filename

After committing, before pushing:

# See what commits will be pushed
git log origin/auto..HEAD

# See files changed in commits that will be pushed
git diff origin/auto..HEAD --name-status

Emergency: Accidentally Staged Wrong Files

Unstage specific file:

git restore --staged filename

Unstage all files:

git restore --staged .

Undo last commit (keeps changes):

git reset --soft HEAD~1

Remove file from git but keep local:

git rm --cached filename

Verifying .gitignore Works

Check if files are ignored:

# Check specific file
git check-ignore -v temp_docs_and_tests/somefile.md

# Should output the .gitignore rule that matches
# Example: .gitignore:152:temp_docs_and_tests/

List all tracked files (should NOT include temp):

git ls-files | grep temp_docs_and_tests
# Should return nothing (0 lines)

List all ignored files:

git status --ignored

Success Checklist

Before pushing to GitHub, ensure:

• [ ] Pre-commit hook is installed (pre-commit install)
• [ ] All tests pass locally (pytest tests/ -x)
• [ ] No linting errors (pre-commit runs on commit)
• [ ] Temp files are in .gitignore and not tracked
• [ ] Commit message is clear and descriptive
• [ ] Only relevant files are committed (not temp/logs)
• [ ] Changes are on the correct branch

Example Session

User: "GitHub CI is failing with test errors"

Your Response:

I'll help you fix the CI failures. Let me follow the systematic approach:

**Step 1: Activate pre-commit hooks**

pre-commit install

**Step 2: Run tests locally to identify failures**

python -m pytest tests/ -x --tb=short -q 2>&1 | tail -50

*[Wait for results and analyze the failure pattern]*

**Step 3: Fix the specific failure**
*[Apply pattern-based fix - e.g., add return_message=True for KeyError 'role']*

**Step 4: Verify the fix**

python -m pytest tests/test_file.py -xvs

**Step 5: Run full test suite**

python -m pytest tests/ -x --tb=short -q

**Step 6: Commit and push**

git add tests/test_file.py git commit -m "Fix test: add return_message=True" git push origin auto

All tests should now pass in CI! ✅

Files Changed Today (Real Examples)

Commit 1: fca22e2 - Fix test_task_manager.py

Issue: Mock not subscriptable Fix:

• Create separate mock tool instances
• Add _get_tool_instance method
• Use AsyncMock(side_effect=...) for async tools

Commit 2: 890cb11 - Fix test_tooluniverse_cache_integration.py

Issue: KeyError: 'role' Fix:

• Add return_message=True
• Change msg["role"] to msg.get("role")

Commit 3: f775c6f - Fix test_run_parameters.py

Issue: KeyError: 'role' in batch test Fix:

• Add return_message=True
• Use .get() for safe access

Commit 4: 1d9222a - Remove temp_docs_and_tests/

Issue: Temp folder being pushed to GitHub Fix:

• git rm -r --cached temp_docs_and_tests/
• Commit removal (keeps local files)

Memory Markers

When you see these patterns:

• KeyError: 'role' → Missing return_message=True
• Mock object is not subscriptable → Fix mock configuration
• F841 or E731 → Linting errors to fix
• User says "don't push temp folder" → Check git ls-files and remove from tracking

Always remember: 1. Pre-commit hook MUST be active 2. Test locally BEFORE pushing 3. Fix one test at a time 4. Verify full suite before pushing 5. Check that temp files aren't tracked

End of Instructions

Follow this systematic approach every time there are CI failures or test issues. The patterns are proven to work - we fixed 40 tests today using these exact techniques!