
Tribe V2 Bci Applied
- 1 installs
- 3 repo stars
- Updated August 5, 2026
- broomva/skills
tribe-v2-bci-applied is a Claude skill that uses Meta's TRIBE v2 brain encoder to predict neural responses to media and optimize stimuli for applied BCI research.
About
tribe-v2-bci-applied is a Claude skill for applied BCI research and neuro-informed content optimization using Meta's TRIBE v2 brain encoder. It predicts cortical responses to media without a scanner, ranks content by predicted engagement, and optimizes stimuli toward a target brain region. A developer uses it for A/B testing creative, accessibility research, and non-invasive BCI groundwork. The model is CC BY-NC 4.0, so it is licensed for non-commercial research only.
- Predicts fMRI cortical responses to video, audio, and text without a scanner
- Ranks content by predicted neural engagement per cortical region
- Optimizes stimuli to maximize activation in a target region
Tribe V2 Bci Applied by the numbers
- 1 all-time installs (skills.sh)
- Ranked #1,803 of 2,064 Data Science & ML skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
tribe-v2-bci-applied capabilities & compatibility
- Capabilities
- neural prediction · content ranking · stimulus optimization
- Use cases
- research · data analysis · video generation
- Runs
- Runs locally
- Pricing
- Free
What tribe-v2-bci-applied says it does
Predicts neural responses to media, UI, and content without brain scanners — enabling stimulus
This skill is for non-commercial research only.
rank them by predicted neural engagement — which one will drive more visual attention, emotional resonance, or language processing.
npx skills add https://github.com/broomva/skills --skill tribe-v2-bci-appliedAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 1 |
|---|---|
| repo stars | ★ 3 |
| Last updated | August 5, 2026 |
| Repository | broomva/skills ↗ |
What it does
Predict and rank neural responses to media for BCI research and stimulus optimization.
Who is it for?
Non-commercial research ranking or optimizing media by predicted brain engagement.
Skip if: Commercial neuromarketing or profit-driven audience profiling, which the CC BY-NC license forbids.
When should I use this skill?
Predicting neural responses to content, ranking stimuli by engagement, or optimizing stimuli for a target region.
What you get
Per-region neural engagement scores and optimized stimulus variants without a scanner.
- Engagement rankings CSV
- Per-region activation scores
- Optimized stimulus variants
By the numbers
- Predicts ~20,000 cortical vertices on fsaverage5
- Model weights ~several GB on first run
- Licensed CC BY-NC 4.0
Files
TRIBE v2 Applied BCI Skill
Agentic skill for applied BCI research and neuro-informed content optimization — from predicting fMRI cortical responses to media without brain scanners, through stimulus optimization and attention ranking, to generating cortical priors for non-invasive BCI decoding research.
License constraint: TRIBE v2 is CC BY-NC 4.0. This skill is for non-commercial research only. Commercial neuromarketing, advertising optimization, or audience profiling for profit requires a separate license from Meta. Read references/ethics-privacy.md before any applied use.
---
Quick Start
1. Install TRIBE v2
# Python 3.11+ required
git clone https://github.com/facebookresearch/tribev2
cd tribev2
pip install -e .2. Load Model and Run First Prediction
from tribev2 import TribeModel
# Load model — downloads weights on first run (~several GB)
model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")
# Build events dataframe from your stimulus
df = model.get_events_dataframe(video_path="path/to/video.mp4")
# Predict cortical responses
preds, segments = model.predict(events=df)
# preds.shape = (n_timesteps, n_vertices)
# n_vertices ~20,000 on fsaverage5 surface mesh
print(f"Predicted response shape: {preds.shape}")
print(f"Mean activation across all cortex: {preds.mean():.4f}")3. Supported Input Modalities
# Video (extracts visual + auditory + motion features)
df = model.get_events_dataframe(video_path="clip.mp4")
# Text only (activates language network)
df = model.get_events_dataframe(text_path="script.txt")
# Audio only (activates auditory + language regions)
df = model.get_events_dataframe(audio_path="voiceover.wav")4. Extract Region Activation
import numpy as np
# Visual cortex — approximate fsaverage5 vertex range
visual_vertices = list(range(1000, 7000))
visual_activation = preds[:, visual_vertices].mean()
print(f"Visual cortex mean activation: {visual_activation:.4f}")---
Workflow A: Content Engagement Ranking
Trigger: You have N media files (videos, audio clips, or text variants) and want to rank them by predicted neural engagement — which one will drive more visual attention, emotional resonance, or language processing.
When to use: A/B testing ad creative before production, ranking tutorial formats, comparing voiceover styles, testing UI motion animations.
Tool: scripts/content_tester.py
Input: folder of files (all same modality), target regions
1. Load TRIBE v2 once
→ Single model load amortized across all files
2. For each file:
→ model.get_events_dataframe(...)
→ model.predict(events=df)
→ Compute mean activation per requested region
→ Record per-file scores
3. Compute overall_engagement_score = mean(all region scores)
4. Rank files descending by overall_engagement_score
Output: CSV with per-region scores + overall rank, top-3 printed to consoleRun it:
python scripts/content_tester.py \
--input-dir ./ad_variants/ \
--modality video \
--regions visual,auditory,language \
--output engagement_rankings.csvInterpreting results:
| Score range | Interpretation |
|---|---|
| > 0.6 | High predicted engagement — stimulus strongly activates target networks |
| 0.3 – 0.6 | Moderate engagement — typical for well-produced content |
| < 0.3 | Low engagement — consider redesigning stimulus elements |
Design considerations:
- Scores are relative within your batch, not absolute fMRI values
- Compare within modality for best results (don't rank videos vs audio directly)
- High visual + low language = visually engaging but not verbally memorable
- High language + low visual = good for information retention tasks
---
Workflow B: Stimulus Optimization
Trigger: You have a base stimulus and want to find variants that maximize predicted activation in a specific cortical region — e.g., maximize visual cortex response for a display ad, or maximize language network response for a tutorial narration.
When to use: Iterative content refinement, creative optimization loops, accessibility improvements (maximize auditory processing for hearing-impaired content), BCI stimulus design.
Tool: scripts/optimize_stimulus.py
Input: base stimulus file, target region, modality, number of variants
1. Predict baseline activation on original file
→ Establish baseline score for target region
2. Generate N perturbation variants
Video: brightness (0.7x–1.4x), contrast (0.8x–1.3x), saturation, playback speed
Audio: speed (0.85x–1.15x), pitch shift, volume normalization variants
Text: model prints paraphrase suggestions for human review (cannot auto-perturb text)
3. Predict activation on each variant
→ model.predict(events=variant_df)
→ Score target_region_vertices.mean()
4. Rank variants by target region mean activation
→ Output CSV: variant_file, target_region_mean, rank
→ Print top variant's delta vs baseline
Output: Ranked CSV, best variant path, improvement percentageRun it:
python scripts/optimize_stimulus.py \
--input ./base_ad.mp4 \
--target-region visual \
--modality video \
--n-variants 10 \
--output-dir ./optimized/Target region options:
| Region flag | Cortical target | Applied goal |
|---|---|---|
visual | V1–V4, MT (vertices 1000–7000) | Visual attention, saliency |
auditory | A1 + belt + STS (vertices 8000–13000) | Voice quality, audio engagement |
language | Broca's + Wernicke's (vertices 15000–18500, LH) | Comprehension, verbally memorable |
motion | MT/V5 (vertices 5500–7000) | Motion perception, dynamic content |
default_mode | mPFC/PCC/AG (vertices 19000–20000) | Mind-wandering, narrative immersion |
Greedy optimization loop (advanced — run in a shell loop):
BEST="base_ad.mp4"
for ITER in 1 2 3 4 5; do
python scripts/optimize_stimulus.py \
--input "$BEST" \
--target-region visual \
--modality video \
--n-variants 8 \
--output-dir ./iter_${ITER}/
BEST=$(python -c "
import csv
with open('iter_${ITER}/rankings.csv') as f:
rows = list(csv.DictReader(f))
print(rows[0]['variant_file'])
")
echo "Iter $ITER best: $BEST"
done---
Workflow C: Attention Proxy Analysis
Trigger: You want to quantify predicted attentional engagement — not just one region, but a composite proxy combining visual and social/multisensory processing.
Rationale: True attentional engagement in fMRI correlates with simultaneous activation of:
- Early visual cortex (V1–V4): processing visual input at all
- MT/V5: tracking motion — moving stimuli attract attention
- STS (superior temporal sulcus): social signals, face motion, voice prosody
High combined score = stimulus is predicted to capture and hold attention.
import numpy as np
from tribev2 import TribeModel
model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")
def attention_proxy(model, file_path, modality="video"):
"""Compute attention proxy score from TRIBE v2 predictions."""
if modality == "video":
df = model.get_events_dataframe(video_path=file_path)
elif modality == "audio":
df = model.get_events_dataframe(audio_path=file_path)
else:
df = model.get_events_dataframe(text_path=file_path)
preds, segments = model.predict(events=df)
# Component regions
early_visual = preds[:, 1000:5500].mean(axis=1) # V1–V4
motion_region = preds[:, 5500:7000].mean(axis=1) # MT/V5
sts_region = preds[:, 11500:13000].mean(axis=1) # STS
# Attention proxy: weighted combination
attention_ts = 0.4 * early_visual + 0.3 * motion_region + 0.3 * sts_region
return {
"attention_proxy_mean": float(attention_ts.mean()),
"attention_proxy_peak": float(attention_ts.max()),
"attention_proxy_std": float(attention_ts.std()),
"peak_timestep": int(attention_ts.argmax()),
"early_visual_mean": float(early_visual.mean()),
"motion_mean": float(motion_region.mean()),
"sts_mean": float(sts_region.mean()),
}
# Usage
result = attention_proxy(model, "campaign_video.mp4", modality="video")
print(f"Attention proxy: {result['attention_proxy_mean']:.4f}")
print(f"Peak engagement at timestep: {result['peak_timestep']}")Interpreting the attention proxy timeseries:
import matplotlib.pyplot as plt
# Visualize attention dynamics over time
preds, segments = model.predict(events=df)
early_v = preds[:, 1000:5500].mean(axis=1)
motion = preds[:, 5500:7000].mean(axis=1)
sts = preds[:, 11500:13000].mean(axis=1)
proxy = 0.4 * early_v + 0.3 * motion + 0.3 * sts
plt.figure(figsize=(12, 4))
plt.plot(proxy, label="Attention proxy", linewidth=2)
plt.plot(early_v, alpha=0.5, label="Visual (V1-V4)")
plt.plot(motion, alpha=0.5, label="Motion (MT)")
plt.plot(sts, alpha=0.5, label="STS (social)")
plt.xlabel("Timestep")
plt.ylabel("Predicted activation")
plt.title("Attention proxy across stimulus duration")
plt.legend()
plt.tight_layout()
plt.savefig("attention_dynamics.png", dpi=150)Use this to:
- Find the moment in a video where attention is predicted to drop (cut or restructure that segment)
- Compare opening hooks: which 10-second intro scores highest on attention proxy?
- Identify which audio/visual elements drive the peaks
---
Workflow D: BCI Prior Generation
Trigger: You are working on a non-invasive BCI (EEG, fMEG, or fNIRS) decoding project and need population-average cortical activation priors — e.g., to localize imagined speech, visual imagery, or auditory perception without running a full fMRI study.
Rationale: TRIBE v2 was trained on large-scale fMRI data. Its predictions represent population-average expected activations for a stimulus class. These priors can seed:
- Spatial filters for EEG source localization (beamforming, eLORETA)
- Region-of-interest masks for constrained decoding
- Expected activation patterns for cross-modal transfer learning
import numpy as np
from tribev2 import TribeModel
model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")
def generate_cortical_prior(model, stimulus_class_files: list, modality: str = "audio") -> np.ndarray:
"""
Generate a population-average cortical activation prior for a stimulus class.
Args:
stimulus_class_files: list of file paths for stimuli in this class
modality: 'video', 'audio', or 'text'
Returns:
prior: (n_vertices,) mean activation map across stimuli and time
"""
all_preds = []
for fpath in stimulus_class_files:
if modality == "audio":
df = model.get_events_dataframe(audio_path=fpath)
elif modality == "video":
df = model.get_events_dataframe(video_path=fpath)
else:
df = model.get_events_dataframe(text_path=fpath)
preds, _ = model.predict(events=df)
# Average over time for this stimulus
all_preds.append(preds.mean(axis=0))
# Average over stimulus class
prior = np.stack(all_preds).mean(axis=0)
return prior
def save_prior_as_nifti_compatible(prior: np.ndarray, output_path: str):
"""
Save prior as numpy array for downstream BCI toolchain use.
Compatible with MNE-Python, nibabel, and FSL workflows.
"""
np.save(output_path, prior)
print(f"Saved prior shape {prior.shape} to {output_path}")
print("Load with: import numpy as np; prior = np.load('prior.npy')")
# Example: generate speech vs. non-speech priors for EEG decoding
speech_files = ["speech_1.wav", "speech_2.wav", "speech_3.wav"]
non_speech_files = ["music_1.wav", "noise_1.wav", "tone_1.wav"]
speech_prior = generate_cortical_prior(model, speech_files, modality="audio")
non_speech_prior = generate_cortical_prior(model, non_speech_files, modality="audio")
# Differential contrast prior (speech - non-speech)
contrast_prior = speech_prior - non_speech_prior
save_prior_as_nifti_compatible(contrast_prior, "speech_contrast_prior.npy")
# Identify top vertices (most discriminative regions)
top_vertices = np.argsort(np.abs(contrast_prior))[-500:]
print(f"Top 500 discriminative vertices: {top_vertices}")
print(f"Language network (Broca's ~15000-17000): {sum(15000 <= v <= 17000 for v in top_vertices)} vertices in range")Integrating with MNE-Python (EEG source modeling):
import mne
import numpy as np
# Load your TRIBE v2 prior
prior = np.load("speech_contrast_prior.npy") # shape: (n_vertices_fsaverage5,)
# Use as initial weights for minimum norm estimate (MNE)
# Prior needs to be projected to source space matching your EEG setup
# See MNE docs: mne.minimum_norm.make_inverse_operator with depth weighting
# The prior defines which regions you expect to be active —
# feeds into beamformer spatial filter initialization or
# constrains the solution space for sparse inverse methodsImportant disclosures for BCI use: TRIBE v2 priors are population-average predictions. Individual subject brains differ in activation patterns. When using these priors in real BCI pipelines, disclose this assumption to end users. Do not present population-average predictions as personalized neural decoding. See references/ethics-privacy.md for full BCI-specific risk disclosure.
---
Ethical Guardrails
This skill operates under CC BY-NC 4.0 restrictions and ethical norms for brain simulation research.
This skill MUST NOT be used for:
- Commercial advertising optimization or neuromarketing for profit
- Building psychological profiles for commercial audience targeting
- Generating "neural dark patterns" — stimuli designed to bypass conscious decision-making
- Any profiling of individuals without explicit informed consent
- Any use involving minors without a guardian consent framework
- Surveillance or monitoring applications
This skill MAY be used for:
- Academic research and publication (non-commercial)
- Accessibility improvement research
- Clinical hypothesis generation (not diagnosis)
- UX research with full participant disclosure
- Non-invasive BCI research with appropriate consent frameworks
Before any applied use, read the full ethics and licensing reference: references/ethics-privacy.md
For commercial licensing, contact Meta Research: https://research.facebook.com
---
Tool Reference
TRIBE v2 API
| Method | Input | Output | Notes |
|---|---|---|---|
TribeModel.from_pretrained(model_id) | HuggingFace model ID | TribeModel instance | Downloads ~GB of weights on first call |
model.get_events_dataframe(video_path=) | Video file path | pd.DataFrame | Extracts visual, auditory, motion features |
model.get_events_dataframe(audio_path=) | Audio file path | pd.DataFrame | Extracts auditory + language features |
model.get_events_dataframe(text_path=) | Text file path | pd.DataFrame | Extracts language + semantic features |
model.predict(events=df) | Events DataFrame | (preds, segments) | preds.shape = (timesteps, vertices) |
Key Output Properties
| Property | Value | Description |
|---|---|---|
preds.shape[0] | Varies with stimulus duration | Number of predicted timepoints |
preds.shape[1] | ~20,000 | Vertices on fsaverage5 surface mesh |
preds.mean() | Float | Overall cortical activation mean |
preds[:, v_start:v_end].mean() | Float | Region mean activation |
Supported Formats
| Modality | Formats | Notes |
|---|---|---|
| Video | .mp4, .avi, .mov | Extracts visual + audio features jointly |
| Audio | .wav, .mp3, .flac | Pure auditory feature extraction |
| Text | .txt | Language model feature extraction |
---
Cortical Region Reference
For full region atlas with vertex ranges, activation profiles, and applied BCI use cases, see references/cortical-region-atlas.md.
Quick vertex range cheatsheet:
| Region | Vertices (approx) | Key activators |
|---|---|---|
| V1/V2 (primary visual) | 1000–4000 | Edges, contrast, spatial frequency |
| V4 (color/form) | 4000–5500 | Color, shape, object form |
| MT/V5 (motion) | 5500–7000 | Optical flow, motion direction |
| IPS/FEF (attention) | 7000–8500 | Top-down attention, gaze control |
| A1 (primary auditory) | 8000–10000 | Tone, pitch, onset |
| Belt regions (auditory) | 10000–11500 | Voice, timbre, melody |
| STS (social/voice) | 11500–13000 | Speaker identity, prosody, face motion |
| FFA (faces) | 12000–14000 | Face identity, expression |
| PPA (places/scenes) | 14000–16000 | Spatial layout, architecture |
| Broca's area (LH) | 15000–17000 | Syntax, speech production |
| Wernicke's area (LH) | 17000–18500 | Speech comprehension |
| VWFA (reading) | 18500–19500 | Visual words, orthography |
| vmPFC (reward/value) | 18000–19000 | Emotional valence, reward expectation |
| mPFC/PCC/AG (DMN) | 19000–20000 | Narrative, self-referential, mind-wandering |
---
References
- [references/cortical-region-atlas.md](references/cortical-region-atlas.md) — Full applied cortical atlas: region properties, vertex ranges, activation profiles, and BCI/neuromarketing use cases
- [references/ethics-privacy.md](references/ethics-privacy.md) — CC BY-NC license constraints, consent frameworks, prohibited uses, and BCI-specific risk disclosures
- TRIBE v2 paper: Benchetrit et al. (2025) — "Brain-wide visual responses to natural stimuli" — Meta AI Research
- TRIBE v2 demo: https://aidemos.atmeta.com/tribev2
- TRIBE v2 repo: https://github.com/facebookresearch/tribev2
- fsaverage5 surface: FreeSurfer fsaverage5 — 20,484 vertices per hemisphere; TRIBE v2 uses this as its prediction target
Cortical Region Atlas for TRIBE v2 Applied Work
Applied reference for BCI research, neuromarketing, and content optimization using TRIBE v2 predictions on the fsaverage5 surface mesh (~20,484 vertices per hemisphere).
Note on vertex ranges: All ranges listed here are approximate, derived from published parcellations (Glasser 2016 HCP MMP1.0, Wang 2015 probabilistic retinotopy, Fedorenko 2010 language localizer). For production BCI use, replace with exact parcellation indices registered to your fsaverage5 space. These ranges are sufficient for content ranking and relative comparisons.
---
1. Visual Processing Hierarchy
The visual system is organized hierarchically from primary cortex (V1) through increasingly complex feature representations. TRIBE v2 was trained on visual stimuli with high fidelity — this is the best-characterized region for applied use.
| Region | Hemisphere | Approx fsaverage5 vertex range | What activates it | Applied BCI/neuromarketing use |
|---|---|---|---|---|
| V1 / V2 (primary visual cortex) | Bilateral | 1000–4000 | Edges, contrast, spatial frequency, luminance gradients | Baseline visual response — any on-screen content; contrast/sharpness testing |
| V4 (color/form area) | Bilateral | 4000–5500 | Color saturation, shape contours, object form | Color palette testing, logo recognition, brand color impact |
| MT / V5 (motion area) | Bilateral | 5500–7000 | Optical flow, motion direction, speed, biological motion | Dynamic ad elements, video transitions, animation speed testing |
| FFA (fusiform face area) | Bilateral (RH dominant) | 12000–14000 | Face identity, expression, eye contact | Spokesperson effectiveness, avatar design, product-with-person ads |
| PPA (parahippocampal place area) | Bilateral | 14000–16000 | Spatial layouts, architecture, scene geometry, outdoor environments | Location/setting imagery in ads, spatial UI design, real estate content |
| EBA (extrastriate body area) | Bilateral | Near MT, ~6500–8000 | Body shape, posture, gesture | Fitness content, fashion, gesture-based UI |
Applied notes (visual hierarchy):
- V1/V2 activation is essentially always present for visual stimuli — use as normalization baseline
- High MT activation = content has strong motion/dynamics; correlates with perceived energy and liveliness
- High FFA + high STS = face + social processing; strong predictor of emotional engagement with on-screen people
- PPA activation matters for setting-driven narratives (travel, real estate, outdoor lifestyle)
---
2. Auditory Processing
Auditory cortex is organized from primary (tonotopic) through increasingly abstract speech and social representations. TRIBE v2 captures these well when video or audio input includes natural soundtracks.
| Region | Hemisphere | Approx fsaverage5 vertex range | What activates it | Applied BCI/neuromarketing use |
|---|---|---|---|---|
| A1 (primary auditory cortex) | Bilateral | 8000–10000 | Pure tones, pitch, onset transients, amplitude modulation | Music beat timing, audio onset design, alert sound design |
| Auditory belt regions | Bilateral | 10000–11500 | Voice identity, timbre, music melody, pitch patterns | Voiceover voice selection, music genre testing, podcast audio quality |
| STS (superior temporal sulcus) | Bilateral | 11500–13000 | Speaker identity, prosody, emotional tone, lip movement, audiovisual integration | Voiceover effectiveness, emotional tone of narration, talking-head video engagement |
Applied notes (auditory):
- STS is a hub for social audio — it activates for emotionally expressive speech, charming voices, and audiovisual synchrony
- High STS + high FFA = strong predicted engagement with on-screen speakers
- Low auditory activation despite audio content = flat prosody or generic background music with no salience
- Use auditory belt to compare different music tracks or voiceover artists
---
3. Language Network
The language network is left-hemisphere dominant and distributed across frontal and temporal cortex. TRIBE v2 was trained with text and speech inputs that engage this network.
| Region | Hemisphere | Approx fsaverage5 vertex range | What activates it | Applied BCI/neuromarketing use |
|---|---|---|---|---|
| Broca's area (IFG pars triangularis + opercularis) | Left | 15000–17000 | Syntactic processing, speech production planning, semantic working memory | Caption/subtitle complexity, script structure testing, syntax simplicity A/B |
| Wernicke's area (posterior STG / STS) | Left | 17000–18500 | Speech comprehension, phonological decoding, semantic integration | Spoken content clarity, voiceover comprehension, word choice optimization |
| VWFA (visual word form area, fusiform gyrus) | Left | 18500–19500 | Reading, letter recognition, word-level orthographic processing | On-screen text readability, caption font/size testing, reading level assessment |
Applied notes (language):
- High Broca's activation = content requires more active linguistic processing — good for complex narratives, can be fatigue-inducing for simple ads
- High Wernicke's = speech is being actively decoded — indicates comprehensible but engaging spoken content
- High VWFA = strong on-screen text processing; matters when captions are critical to comprehension (e.g., silent video)
- For accessibility: compare VWFA activation for different caption styles to find the most readable format
---
4. Emotional and Social Processing
These regions mediate reward valuation, emotional salience, and social interpretation. Note: vmPFC is cortical and within TRIBE v2's fsaverage5 coverage; amygdala is subcortical and has limited coverage.
| Region | Hemisphere | Approx fsaverage5 vertex range | What activates it | Applied BCI/neuromarketing use |
|---|---|---|---|---|
| vmPFC (ventromedial prefrontal cortex) | Bilateral (medial) | 18000–19000 | Reward value, emotional valence, self-relevance, preference encoding | Emotional resonance testing, brand value perception, preference-driven content design |
| OFC (orbitofrontal cortex) | Bilateral | Near vmPFC, ~17500–18500 | Expected reward, pleasantness, sensory value | Hedonic appeal of visual/taste/luxury content |
| Amygdala | Bilateral (subcortical) | Not well-covered in fsaverage5 | Emotional salience, threat detection, arousal | Limited TRIBE v2 coverage; use vmPFC as cortical proxy for valence |
| TPJ (temporoparietal junction) | Bilateral (RH dominant) | ~13000–15000 | Theory of mind, social attribution, agency | Social narrative content, character-driven storytelling, empathy-inducing ads |
Applied notes (emotional/social):
- vmPFC activation is a key marker for content that feels personally relevant or rewarding
- TPJ activation correlates with understanding character intentions and social dynamics in narratives
- Combine vmPFC + FFA + STS for a "social-emotional engagement" composite score
- Avoid confusing vmPFC activation with DMN activation — they partially overlap; check context
---
5. Attention and Default Mode Network
These regions govern top-down attentional control and the mind-wandering / narrative immersion state. The DMN typically deactivates during external task engagement — high DMN activation during content viewing can indicate mind-wandering or self-referential processing.
| Region | Hemisphere | Approx fsaverage5 vertex range | What activates it | Applied BCI/neuromarketing use |
|---|---|---|---|---|
| IPS (intraparietal sulcus) | Bilateral | 7000–8000 | Spatial attention, visual salience-driven orienting, numerical processing | Attentional guidance design, UI layout scanning behavior, infographic comprehension |
| FEF (frontal eye fields) | Bilateral | ~8000–8500 | Volitional gaze direction, covert attention | Eye-tracking prediction, UI element prominence, call-to-action placement |
| mPFC (medial prefrontal cortex — DMN node) | Bilateral (medial) | 19000–19500 | Self-referential thought, prospective memory, mind-wandering | Detect narrative immersion vs. distraction; low mPFC during task = high focus |
| PCC (posterior cingulate cortex — DMN node) | Bilateral (medial) | ~19200–19700 | Default mode hub, autobiographical memory, internal narrative | Narrative resonance, story-driven engagement; PCC activation = deep immersion |
| Angular gyrus (DMN + language overlap) | Bilateral | ~19500–20000 | Semantic integration, narrative comprehension, conceptual metaphor | Abstract concept understanding, brand storytelling, metaphor in advertising |
Applied notes (attention and DMN):
- IPS + FEF high activation = stimulus is actively directing visual attention; useful for UI design testing
- DMN (mPFC + PCC + angular gyrus) often deactivates during externally demanding tasks
- High DMN during content = either mind-wandering (bad) or deep narrative immersion (good) — context matters
- For content designed to be immersive/story-driven, moderate DMN activation is expected and desirable
- For instructional or attention-critical content (safety videos, tutorial walkthroughs), low DMN + high IPS is ideal
---
Composite Attention Proxy
For applied engagement scoring, a weighted combination of visual + motion + social regions provides a robust attention proxy that correlates with behavioral attention measures:
Attention proxy = 0.4 * V1–V4 + 0.3 * MT/V5 + 0.3 * STSThis captures: Are they looking? (V1–V4), Is it dynamic? (MT), Are social signals present? (STS)
---
Cross-Region Interaction Patterns
Common multi-region combinations and their interpretations:
| Pattern | Interpretation | Applied use case |
|---|---|---|
| High visual + High STS + High FFA | Strong social-visual engagement; face+voice together | Talking-head videos, spokesperson ads |
| High visual + Low language + Low STS | Visually engaging but not verbally or socially memorable | Silent visual ads, abstract motion graphics |
| High language + Low visual | Verbally driven; good for podcasts, radio ads, text-heavy content | Audio content, documentary narration |
| High language + High VWFA | Reading-heavy engagement; captions are load-bearing | Tutorial content with heavy text |
| High vmPFC + High FFA | Emotional + face engagement; personal resonance | Testimonial content, empathy-driven ads |
| High DMN + Low visual | Possible mind-wandering; content may not be sustaining attention | Flag for content revision |
| High MT + Low FFA + Low STS | Action/motion without social engagement | Product-demo videos without people |
---
fsaverage5 Technical Notes
- Total vertices: 20,484 per hemisphere (40,968 bilateral); TRIBE v2 likely uses the combined surface
- Vertex numbering: Not anatomically contiguous — parcellation lookup required for exact ROIs
- Resolution: ~3mm average inter-vertex distance (coarser than fsaverage with 163,842 vertices)
- Registration: All individual brains registered to this surface via FreeSurfer spherical registration
- Atlases to use for exact parcellations:
- Glasser 2016 HCP MMP1.0 (360 parcels bilateral) — best parcellation for applied work
- Wang 2015 probabilistic retinotopy — best for early visual cortex
- Fedorenko 2010 functional localizer — language network
- Yeo 2011 7-network / 17-network — coarse functional networks
Getting exact parcellation indices (Python):
import nibabel as nib
import numpy as np
# Download HCP MMP1.0 parcellation for fsaverage5
# From: https://github.com/ThomasYeoLab/CBIG/tree/master/stable_projects/brain_parcellation
label_img = nib.load("hcp_mmp1_fsaverage5_lh.label.gii")
labels = label_img.darrays[0].data # (n_vertices,) array of parcel IDs
parcel_names = label_img.labeltable.labels # map ID → name
# Get exact V4 vertices
v4_vertices = np.where(labels == parcel_id_for_V4)[0].tolist()---
Further Reading
- Glasser et al. (2016) "A multi-modal parcellation of human cerebral cortex" — Nature
- Wang et al. (2015) "Probabilistic maps of visual topography in human cortex" — Cerebral Cortex
- Benchetrit et al. (2025) "Brain-wide visual responses to natural stimuli" — Meta AI Research (TRIBE v2 paper)
- Huth et al. (2016) "Natural speech reveals the semantic maps that tile human cerebral cortex" — Nature
- Regev et al. (2019) "Selective responses to video stimuli in neural systems across the human brain" — Cerebral Cortex
Ethics and Privacy Reference for TRIBE v2 Applied Use
This document governs the acceptable and prohibited uses of this skill. Read this before any applied use of TRIBE v2 predictions. These constraints are not advisory — they are enforced by the skill's Ethical Guardrails section and by CC BY-NC 4.0 licensing law.
---
1. License Constraint: CC BY-NC 4.0
TRIBE v2 is released under the Creative Commons Attribution-NonCommercial 4.0 International License.
What this means:
| Permitted | Blocked |
|---|---|
| Academic research and publication | Optimizing content for commercial advertising campaigns |
| Accessibility and assistive technology research | Audience profiling for commercial targeting or revenue generation |
| Clinical hypothesis generation | Licensing or selling TRIBE v2 predictions as a product or service |
| Educational use | Neuromarketing research conducted on behalf of a for-profit client |
| Non-commercial UX research with full disclosure | Any use where the primary purpose is to increase commercial revenue |
| BCI research (non-profit, academic) | Building a commercial product whose core value derives from TRIBE v2 predictions |
For commercial use: Contact Meta Research to negotiate a separate commercial license.
- Meta AI Research contact: https://research.facebook.com
- License inquiries: reach out via the TRIBE v2 GitHub repository (https://github.com/facebookresearch/tribev2)
Attribution: Any publication or product using TRIBE v2 must attribute:
Benchetrit et al. (2025). "Brain-wide visual responses to natural stimuli." Meta AI Research.
---
2. Consent Framework
Brain response simulation using TRIBE v2 does not involve any real brain scanning. No actual fMRI data is collected. However, predictions about neural responses to content raise important privacy and consent considerations.
Core principle: Treat TRIBE v2 predictions like real fMRI data for consent purposes.
Why: Even though predictions are model outputs (not measured from real people), they characterize how human brains respond to stimuli. Using these predictions to optimize content for exploitation — without the knowledge of the audience — is equivalent to conducting covert neuroimaging on them.
Consent tiers by use case:
| Use case | Minimum consent requirement |
|---|---|
| Academic research (published) | IRB/ethics board approval; participant disclosure of computational methods used |
| Internal UX research | Inform participants that neural prediction modeling was used in content design |
| Accessibility research | Full disclosure to participants; results shared back where appropriate |
| Clinical hypothesis generation | IRB approval; predictions treated as supporting evidence only, not diagnosis |
| BCI research | Full informed consent; disclose that population-average priors are used (see section 5) |
What does NOT require individual consent: Batch predictions on media you created, for internal research purposes, with no deployment to target audiences. Example: comparing two product videos you own to select the more neurally engaging one for internal review.
What DOES require disclosure: Any case where the predictions influence content that will be served to an audience without their knowledge that neural prediction was used in content design.
---
3. Prohibited Uses
The following uses of this skill are explicitly prohibited under CC BY-NC 4.0, ethical research norms, and the intended use policy of TRIBE v2.
3.1 Exploitative Optimization
Prohibited: Optimizing content to exploit emotional vulnerabilities in target audiences.
This includes: designing stimuli that maximize predicted amygdala/vmPFC activation specifically to trigger fear, anxiety, or impulsive decision-making in a commercial context. Neural optimization is acceptable for positive engagement; it is not acceptable when directed at vulnerabilities.
3.2 Commercial Audience Profiling
Prohibited: Building psychological or neural profiles of audience segments for the purpose of commercial advertising targeting, political micro-targeting, or personalized influence at scale.
This includes: generating TRIBE v2 predictions for a corpus of content consumed by a demographic and using those predictions to infer what that demographic is "neurologically susceptible to."
3.3 Neural Dark Patterns
Prohibited: Generating stimuli specifically designed to bypass conscious decision-making — colloquially called "neural dark patterns."
This includes:
- Content optimized to trigger automatic emotional responses before conscious evaluation can occur
- Interfaces designed to maximize neural engagement specifically to override deliberate choice
- Attention capture mechanisms designed to make disengagement neurologically costly
3.4 Minor-Specific Optimization Without Guardian Framework
Prohibited: Optimizing content targeting minors (under 18) without a formal guardian consent and oversight framework.
This includes: using TRIBE v2 to A/B test children's content for engagement optimization without parental consent and independent ethics review.
3.5 Non-Consensual Individual Profiling
Prohibited: Using TRIBE v2 to generate individual-level neural response predictions for real people without their explicit informed consent.
Note: TRIBE v2 is a population-average model — it does not predict a specific individual's responses. But inference about how content affects specific groups requires the same consent frameworks as group-level neuroimaging studies.
3.6 Surveillance and Monitoring
Prohibited: Using TRIBE v2 predictions as inputs to surveillance, attention monitoring, or behavioral monitoring systems without explicit opt-in consent from subjects.
---
4. Acceptable Uses
The following uses are explicitly acceptable within CC BY-NC 4.0 and ethical norms:
| Use case | Requirements |
|---|---|
| Academic research on media perception | Non-commercial; cite TRIBE v2; follow institutional ethics procedures |
| Accessibility improvement research | Full participant disclosure; results shared back to community |
| Clinical hypothesis generation | Treat as hypothesis-generating only; no diagnostic use; IRB required |
| UX research with full participant disclosure | Inform participants that neural prediction modeling informed design |
| Non-invasive BCI research (EEG/fMEG) | Disclose use of population-average priors (see section 5); IRB required |
| Educational demonstrations | Must make clear predictions are population averages, not individual measurements |
| Internal creative optimization (non-commercial) | For research institutions or non-profits evaluating content they own |
---
5. BCI-Specific Risk Disclosures
When TRIBE v2 priors feed into a real BCI decoding pipeline (e.g., EEG source localization, imagined speech decoding), the following must be disclosed to end users and documented in any publication:
5.1 Population-Average Assumption
TRIBE v2 generates predictions based on patterns learned from a population of participants. Individual brains differ substantially in:
- Exact anatomical location of functional regions
- Magnitude of activation responses
- Individual processing strategies
Required disclosure: "The cortical priors used in this BCI system are derived from a population-average neural encoder (TRIBE v2, Meta AI Research). These priors represent expected population-level activation patterns and may not accurately reflect any individual user's neural responses."
5.2 No Diagnostic Use
TRIBE v2 predictions and the priors derived from them must not be used as diagnostic tools for neurological or psychiatric conditions. They are research and research-support tools only.
5.3 No Clinical Decision Support
TRIBE v2 predictions must not be used to make or inform clinical decisions about individual patients without explicit regulatory approval and clinical validation.
5.4 Data Security for BCI Predictions
Predicted neural response data should be treated with the same security standards as actual fMRI data:
- Do not transmit over unencrypted channels
- Do not store without consent
- Apply the same retention and deletion policies as clinical imaging data
---
6. Reporting Issues and Misuse
If you believe TRIBE v2 or this skill is being misused, or if you discover a vulnerability or ethical concern:
- TRIBE v2 issues: Open an issue on https://github.com/facebookresearch/tribev2
- Meta Responsible AI: https://about.meta.com/actions/safety/topics/safety/responsible-ai/
- Institutional concerns: Report to your institution's IRB or ethics board
If you are uncertain whether a use case is permitted under CC BY-NC 4.0: 1. Assume it is prohibited until confirmed otherwise 2. Consult the CC BY-NC 4.0 license text 3. For commercial edge cases, contact Meta Research directly
---
Summary Card
Quick reference for field decisions:
| Question | Answer |
|---|---|
| Can I use this for a commercial ad campaign? | No — CC BY-NC 4.0 prohibits commercial use |
| Can I publish academic research using this? | Yes — with attribution and IRB compliance |
| Can I use this to improve accessibility tools? | Yes — with participant disclosure |
| Can I build a SaaS product on TRIBE v2 predictions? | No — commercial use; contact Meta for license |
| Do I need consent to predict responses to my own videos? | No, for internal non-commercial research |
| Do I need consent to deploy neural-optimized content to an audience? | Yes — disclose that neural prediction was used in content design |
| Can I use TRIBE v2 priors in an EEG BCI? | Yes — with population-average disclosure to users and IRB approval |
| Can I use this for children's content optimization? | Only with formal guardian consent framework and ethics review |
#!/usr/bin/env python3
"""
content_tester.py — Batch content testing and engagement ranking using TRIBE v2
Tests a folder of media files and ranks them by predicted neural engagement
across specified cortical regions. Outputs a CSV with per-region scores.
Usage:
python content_tester.py \
--input-dir ./ad_variants/ \
--modality video \
--regions visual,auditory,language \
--output engagement_rankings.csv
Supported modalities: video, audio, text
Supported regions: visual, auditory, language, motion, default_mode
(any combination as comma-separated list)
"""
import argparse
import csv
import sys
from pathlib import Path
from typing import List, Dict, Optional
import numpy as np
# ---------------------------------------------------------------------------
# Cortical region vertex map (fsaverage5, ~20k vertices)
# Approximate ranges from published parcellations.
# ---------------------------------------------------------------------------
REGION_VERTEX_MAP: Dict[str, List[int]] = {
"visual": list(range(1000, 7000)), # V1–V4 + MT/V5
"auditory": list(range(8000, 13000)), # A1 + belt + STS
"language": list(range(15000, 18500)), # Broca's + Wernicke's (LH approx)
"motion": list(range(5500, 7000)), # MT/V5 specifically
"default_mode": list(range(19000, 20000)), # mPFC/PCC/angular gyrus
}
VALID_REGIONS = list(REGION_VERTEX_MAP.keys())
VALID_MODALITIES = ["video", "audio", "text"]
# File extensions per modality
MODALITY_EXTENSIONS: Dict[str, List[str]] = {
"video": [".mp4", ".avi", ".mov", ".mkv", ".webm"],
"audio": [".wav", ".mp3", ".flac", ".aac", ".ogg", ".m4a"],
"text": [".txt", ".md", ".rst"],
}
def load_model(cache_folder: str = "./cache"):
"""Load TRIBE v2 model, downloading weights on first call."""
try:
from tribev2 import TribeModel # type: ignore[import-untyped]
except ImportError:
print("ERROR: tribev2 not installed.")
print("Install with: git clone https://github.com/facebookresearch/tribev2 && cd tribev2 && pip install -e .")
sys.exit(1)
print(f"Loading TRIBE v2 model (cache: {cache_folder})...")
model = TribeModel.from_pretrained("facebook/tribev2", cache_folder=cache_folder)
print("Model loaded.\n")
return model
def get_valid_files(input_dir: str, modality: str) -> List[str]:
"""
Collect all files in input_dir with extensions matching the modality.
Args:
input_dir: Directory path to scan
modality: 'video', 'audio', or 'text'
Returns:
Sorted list of absolute file paths
"""
input_path = Path(input_dir)
if not input_path.exists():
print(f"ERROR: Input directory not found: {input_path}")
sys.exit(1)
if not input_path.is_dir():
print(f"ERROR: Path is not a directory: {input_path}")
sys.exit(1)
valid_exts = set(MODALITY_EXTENSIONS.get(modality, []))
files = [
str(f.resolve())
for f in sorted(input_path.iterdir())
if f.is_file() and f.suffix.lower() in valid_exts
]
return files
def predict_file_activations(
model,
file_path: str,
modality: str,
regions: List[str],
) -> Optional[Dict[str, float]]:
"""
Run TRIBE v2 prediction for a single file and compute mean activation
per requested region.
Args:
model: Loaded TribeModel instance
file_path: Path to stimulus file
modality: 'video', 'audio', or 'text'
regions: List of region names to score
Returns:
Dict mapping region name -> mean activation float,
or None if prediction failed
"""
try:
if modality == "video":
df = model.get_events_dataframe(video_path=file_path)
elif modality == "audio":
df = model.get_events_dataframe(audio_path=file_path)
elif modality == "text":
df = model.get_events_dataframe(text_path=file_path)
else:
raise ValueError(f"Unknown modality: {modality}")
preds, _ = model.predict(events=df)
except Exception as e:
print(f" ERROR predicting {Path(file_path).name}: {e}")
return None
n_vertices = preds.shape[1]
region_scores = {}
for region in regions:
if region not in REGION_VERTEX_MAP:
print(f" WARNING: Unknown region '{region}', skipping.")
region_scores[region] = float("nan")
continue
raw_verts = REGION_VERTEX_MAP[region]
valid_verts = [v for v in raw_verts if v < n_vertices]
if not valid_verts:
print(f" WARNING: No valid vertices for region '{region}' "
f"(model has {n_vertices} vertices, region starts at {min(raw_verts)}).")
region_scores[region] = float("nan")
else:
region_scores[region] = float(preds[:, valid_verts].mean())
return region_scores
def compute_engagement_score(region_scores: Dict[str, float]) -> float:
"""
Compute overall engagement score as mean across all non-NaN region scores.
Args:
region_scores: Dict mapping region name -> activation float
Returns:
Mean activation across all valid regions (float)
"""
valid_scores = [v for v in region_scores.values() if not (isinstance(v, float) and v != v)] # filter NaN
if not valid_scores:
return float("nan")
return float(np.mean(valid_scores))
def save_results_csv(results: List[Dict], regions: List[str], output_path: str) -> None:
"""
Save ranked results to CSV.
Args:
results: List of result dicts (already sorted descending by engagement)
regions: List of region names (determines CSV columns)
output_path: Output CSV file path
"""
fieldnames = ["rank", "filename"] + [f"{r}_mean" for r in regions] + ["overall_engagement_score"]
with open(output_path, "w", newline="") as f:
writer = csv.DictWriter(f, fieldnames=fieldnames)
writer.writeheader()
for row in results:
csv_row = {
"rank": row["rank"],
"filename": row["filename"],
"overall_engagement_score": f"{row['overall_engagement_score']:.6f}",
}
for r in regions:
val = row.get(f"{r}_mean", float("nan"))
csv_row[f"{r}_mean"] = f"{val:.6f}" if val == val else "nan" # nan check
writer.writerow(csv_row)
print(f"\nResults saved to: {output_path}")
def print_top_results(results: List[Dict], regions: List[str], top_n: int = 3) -> None:
"""Print the top N most engaging files to console."""
print(f"\n{'=' * 60}")
print(f"TOP {min(top_n, len(results))} MOST ENGAGING FILES")
print(f"{'=' * 60}")
for row in results[:top_n]:
print(f"\n #{row['rank']}: {row['filename']}")
print(f" Overall engagement score: {row['overall_engagement_score']:.6f}")
for r in regions:
val = row.get(f"{r}_mean", float("nan"))
val_str = f"{val:.6f}" if val == val else "nan"
print(f" {r:20s}: {val_str}")
print(f"{'=' * 60}")
def main():
parser = argparse.ArgumentParser(
description="Batch content engagement testing using TRIBE v2 neural predictions.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Rank all videos by visual + auditory + language engagement
python content_tester.py --input-dir ./videos/ --modality video --regions visual,auditory,language --output results.csv
# Rank audio files by auditory and language activation only
python content_tester.py --input-dir ./audio_clips/ --modality audio --regions auditory,language --output audio_results.csv
# Rank text variants by language and default_mode activation
python content_tester.py --input-dir ./text_variants/ --modality text --regions language,default_mode --output text_results.csv
Available regions: visual, auditory, language, motion, default_mode
""",
)
parser.add_argument(
"--input-dir",
required=True,
help="Directory containing media files to test",
)
parser.add_argument(
"--modality",
required=True,
choices=VALID_MODALITIES,
help=f"Input modality for all files. Options: {', '.join(VALID_MODALITIES)}",
)
parser.add_argument(
"--regions",
default="visual,auditory,language",
help=(
"Comma-separated list of cortical regions to score. "
f"Options: {', '.join(VALID_REGIONS)}. "
"Default: visual,auditory,language"
),
)
parser.add_argument(
"--output",
default="engagement_rankings.csv",
help="Output CSV file path (default: engagement_rankings.csv)",
)
parser.add_argument(
"--cache-folder",
default="./cache",
help="Cache folder for TRIBE v2 model weights (default: ./cache)",
)
parser.add_argument(
"--top-n",
type=int,
default=3,
help="Number of top results to print to console (default: 3)",
)
args = parser.parse_args()
# Parse and validate regions
requested_regions = [r.strip().lower() for r in args.regions.split(",") if r.strip()]
unknown_regions = [r for r in requested_regions if r not in VALID_REGIONS]
if unknown_regions:
print(f"ERROR: Unknown regions: {', '.join(unknown_regions)}")
print(f"Valid options: {', '.join(VALID_REGIONS)}")
sys.exit(1)
if not requested_regions:
print("ERROR: No valid regions specified.")
sys.exit(1)
# Collect input files
files = get_valid_files(args.input_dir, args.modality)
if not files:
valid_exts = MODALITY_EXTENSIONS.get(args.modality, [])
print(f"ERROR: No {args.modality} files found in {args.input_dir}")
print(f"Expected extensions: {', '.join(valid_exts)}")
sys.exit(1)
print(f"\nTRIBE v2 Content Tester")
print(f"{'=' * 50}")
print(f"Input directory: {args.input_dir}")
print(f"Modality: {args.modality}")
print(f"Regions: {', '.join(requested_regions)}")
print(f"Files found: {len(files)}")
print(f"Output CSV: {args.output}")
print(f"{'=' * 50}\n")
# Load model once
model = load_model(cache_folder=args.cache_folder)
# Process each file
results = []
for i, file_path in enumerate(files):
filename = Path(file_path).name
print(f"[{i+1}/{len(files)}] Processing: {filename}")
region_scores = predict_file_activations(model, file_path, args.modality, requested_regions)
if region_scores is None:
print(f" SKIPPED (prediction failed)")
continue
engagement_score = compute_engagement_score(region_scores)
row = {
"filename": filename,
"overall_engagement_score": engagement_score,
}
for r in requested_regions:
row[f"{r}_mean"] = region_scores.get(r, float("nan"))
results.append(row)
# Print inline summary
scores_str = " | ".join(
f"{r}={region_scores.get(r, float('nan')):.4f}" for r in requested_regions
)
print(f" Engagement={engagement_score:.4f} | {scores_str}")
if not results:
print("\nERROR: No files were successfully processed.")
sys.exit(1)
# Sort by overall engagement descending
results.sort(key=lambda x: x["overall_engagement_score"], reverse=True)
# Add rank
for rank, row in enumerate(results, start=1):
row["rank"] = rank
# Print top results
print_top_results(results, requested_regions, top_n=args.top_n)
# Save to CSV
save_results_csv(results, requested_regions, args.output)
print(f"\nProcessed {len(results)}/{len(files)} files successfully.")
print(f"Top file: {results[0]['filename']} (score={results[0]['overall_engagement_score']:.4f})")
if len(results) < len(files):
n_failed = len(files) - len(results)
print(f"WARNING: {n_failed} file(s) failed to process. Check error messages above.")
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""
optimize_stimulus.py — Greedy stimulus optimization using TRIBE v2
Maximizes predicted activation in a target cortical region by iteratively
generating and evaluating stimulus perturbations.
Usage:
python optimize_stimulus.py \
--input path/to/base_stimulus.mp4 \
--target-region visual \
--modality video \
--n-variants 10 \
--output-dir ./optimized/
Supported target regions: visual, auditory, language, motion, default_mode
Supported modalities: video, audio, text
"""
import argparse
import csv
import os
import sys
from pathlib import Path
from typing import List, Dict, Tuple
import numpy as np
# ---------------------------------------------------------------------------
# Cortical region vertex map (fsaverage5, ~20k vertices)
# These are approximate ranges derived from published parcellations.
# For production use, replace with exact parcellation indices from
# a registered fsaverage5 atlas (e.g., HCP MMP1.0, Glasser 2016).
# ---------------------------------------------------------------------------
REGION_VERTEX_MAP: Dict[str, List[int]] = {
"visual": list(range(1000, 7000)), # V1–V4 + MT/V5
"auditory": list(range(8000, 13000)), # A1 + belt + STS
"language": list(range(15000, 18500)), # Broca's + Wernicke's (LH approx)
"motion": list(range(5500, 7000)), # MT/V5 specifically
"default_mode": list(range(19000, 20000)), # mPFC/PCC/angular gyrus
}
VALID_REGIONS = list(REGION_VERTEX_MAP.keys())
VALID_MODALITIES = ["video", "audio", "text"]
def get_region_vertices(region: str) -> List[int]:
"""
Return approximate fsaverage5 vertex indices for a named cortical region.
Args:
region: One of 'visual', 'auditory', 'language', 'motion', 'default_mode'
Returns:
List of vertex indices (integers) for that region
Raises:
ValueError: If region name is not recognized
"""
region = region.lower().strip()
if region not in REGION_VERTEX_MAP:
raise ValueError(
f"Unknown region '{region}'. "
f"Valid options: {', '.join(VALID_REGIONS)}"
)
return REGION_VERTEX_MAP[region]
def load_model(cache_folder: str = "./cache"):
"""Load TRIBE v2 model, downloading weights on first call."""
try:
from tribev2 import TribeModel # type: ignore[import-untyped]
except ImportError:
print("ERROR: tribev2 not installed.")
print("Install with: git clone https://github.com/facebookresearch/tribev2 && cd tribev2 && pip install -e .")
sys.exit(1)
print(f"Loading TRIBE v2 model (cache: {cache_folder})...")
model = TribeModel.from_pretrained("facebook/tribev2", cache_folder=cache_folder)
print("Model loaded.")
return model
def predict_region_activation(
model,
file_path: str,
modality: str,
region_vertices: List[int],
) -> Tuple[float, np.ndarray]:
"""
Run TRIBE v2 prediction and compute mean activation for a cortical region.
Args:
model: Loaded TribeModel instance
file_path: Path to stimulus file
modality: 'video', 'audio', or 'text'
region_vertices: List of vertex indices to average over
Returns:
(region_mean_activation, full_preds_array)
"""
if modality == "video":
df = model.get_events_dataframe(video_path=file_path)
elif modality == "audio":
df = model.get_events_dataframe(audio_path=file_path)
elif modality == "text":
df = model.get_events_dataframe(text_path=file_path)
else:
raise ValueError(f"Unknown modality: {modality}")
preds, _ = model.predict(events=df)
# Guard against vertex indices exceeding prediction dimensionality
n_vertices = preds.shape[1]
valid_verts = [v for v in region_vertices if v < n_vertices]
if len(valid_verts) == 0:
raise ValueError(
f"No valid vertices found. Prediction has {n_vertices} vertices "
f"but requested range starts at {min(region_vertices)}."
)
region_act = float(preds[:, valid_verts].mean())
return region_act, preds
def generate_video_variants(
input_path: str,
output_dir: str,
n_variants: int,
) -> List[str]:
"""
Generate video variants via brightness/contrast/saturation/speed perturbations.
Requires ffmpeg to be installed.
Args:
input_path: Path to input video file
output_dir: Directory to write variant files
n_variants: Number of variants to generate
Returns:
List of paths to generated variant files
"""
try:
import subprocess
result = subprocess.run(["ffmpeg", "-version"], capture_output=True, timeout=5)
if result.returncode != 0:
raise RuntimeError("ffmpeg not found")
except (FileNotFoundError, RuntimeError):
print("WARNING: ffmpeg not found. Cannot generate video variants automatically.")
print("Install ffmpeg: brew install ffmpeg (macOS) or apt install ffmpeg (Linux)")
return []
import subprocess
# Define perturbation parameter grid
perturbation_params = []
# Brightness variations: eq=brightness=-0.2 to +0.2
for brightness in np.linspace(-0.2, 0.2, max(3, n_variants // 3)):
perturbation_params.append({
"type": "brightness",
"value": round(float(brightness), 2),
"filter": f"eq=brightness={brightness:.2f}",
})
# Contrast variations: eq=contrast=0.8 to 1.4
for contrast in np.linspace(0.8, 1.4, max(3, n_variants // 3)):
perturbation_params.append({
"type": "contrast",
"value": round(float(contrast), 2),
"filter": f"eq=contrast={contrast:.2f}",
})
# Speed variations: setpts for video speed 0.85x to 1.15x
for speed in [0.85, 0.9, 1.0, 1.1, 1.15]:
pts_factor = round(1.0 / speed, 3)
perturbation_params.append({
"type": "speed",
"value": speed,
"filter": f"setpts={pts_factor}*PTS",
})
# Trim to requested number
perturbation_params = perturbation_params[:n_variants]
input_stem = Path(input_path).stem
input_suffix = Path(input_path).suffix
variant_paths = []
for i, params in enumerate(perturbation_params):
variant_name = f"{input_stem}_variant_{i:02d}_{params['type']}_{params['value']}{input_suffix}"
variant_path = os.path.join(output_dir, variant_name)
cmd = [
"ffmpeg", "-y",
"-i", input_path,
"-vf", params["filter"],
"-c:a", "copy",
variant_path,
]
result = subprocess.run(cmd, capture_output=True, timeout=120)
if result.returncode == 0:
variant_paths.append(variant_path)
print(f" Generated variant {i+1}/{len(perturbation_params)}: {params['type']}={params['value']}")
else:
print(f" WARNING: Failed to generate variant {i+1}: {result.stderr.decode()[:200]}")
return variant_paths
def generate_audio_variants(
input_path: str,
output_dir: str,
n_variants: int,
) -> List[str]:
"""
Generate audio variants via speed and pitch perturbations.
Uses pydub if available, falls back to ffmpeg.
Args:
input_path: Path to input audio file
output_dir: Directory to write variant files
n_variants: Number of variants to generate
Returns:
List of paths to generated variant files
"""
try:
import subprocess
result = subprocess.run(["ffmpeg", "-version"], capture_output=True, timeout=5)
if result.returncode != 0:
raise RuntimeError("ffmpeg not found")
except (FileNotFoundError, RuntimeError):
print("WARNING: ffmpeg not found. Cannot generate audio variants automatically.")
return []
import subprocess
input_stem = Path(input_path).stem
input_suffix = Path(input_path).suffix or ".wav"
variant_paths = []
# Speed variants: atempo filter (0.85x to 1.15x)
speed_values = np.linspace(0.85, 1.15, min(n_variants, 7)).tolist()
# Volume normalization variants
volume_values = [0.8, 0.9, 1.0, 1.1, 1.2]
all_params = (
[{"type": "speed", "value": round(s, 2), "filter": f"atempo={s:.2f}"} for s in speed_values]
+ [{"type": "volume", "value": v, "filter": f"volume={v}"} for v in volume_values]
)[:n_variants]
for i, params in enumerate(all_params):
variant_name = f"{input_stem}_variant_{i:02d}_{params['type']}_{params['value']}{input_suffix}"
variant_path = os.path.join(output_dir, variant_name)
cmd = [
"ffmpeg", "-y",
"-i", input_path,
"-af", params["filter"],
variant_path,
]
result = subprocess.run(cmd, capture_output=True, timeout=60)
if result.returncode == 0:
variant_paths.append(variant_path)
print(f" Generated variant {i+1}/{len(all_params)}: {params['type']}={params['value']}")
else:
print(f" WARNING: Failed to generate variant {i+1}: {result.stderr.decode()[:200]}")
return variant_paths
def generate_text_suggestions(input_path: str, n_variants: int) -> None:
"""
For text modality, TRIBE v2 cannot automatically mutate semantic content
without risking unintended meaning changes. Print paraphrase suggestions
for human review instead.
Args:
input_path: Path to input text file
n_variants: Number of variants requested
"""
with open(input_path, "r") as f:
text = f.read()
print("\n" + "=" * 60)
print("TEXT MODALITY: Manual paraphrase required")
print("=" * 60)
print(f"Original text ({len(text)} chars):")
print(text[:500] + ("..." if len(text) > 500 else ""))
print()
print(f"To optimize text for language network activation ({n_variants} variants requested),")
print("consider these paraphrase dimensions:")
print()
print(" 1. SYNTAX COMPLEXITY: Vary sentence length (short/medium/complex)")
print(" 2. VOCABULARY REGISTER: Formal vs. conversational tone")
print(" 3. ACTIVE VS PASSIVE: More active voice → stronger language network response")
print(" 4. CONCRETE VS ABSTRACT: Concrete nouns activate more cortical surface")
print(" 5. READING LEVEL: Flesch-Kincaid grade 6 vs 12 → different processing load")
print(" 6. NARRATIVE STRUCTURE: First-person vs third-person perspective")
print()
print("After creating variants, save each as a .txt file and re-run this script")
print("with --input pointing to each variant file.")
print("=" * 60 + "\n")
def score_variants(
model,
variant_paths: List[str],
modality: str,
region_vertices: List[int],
target_region: str,
) -> List[Dict]:
"""
Run TRIBE v2 prediction on all variants and return scored results.
Args:
model: Loaded TribeModel instance
variant_paths: List of variant file paths
modality: 'video', 'audio', or 'text'
region_vertices: Vertex indices for target region
target_region: Region name (for reporting)
Returns:
List of dicts with variant_file, target_region_mean_activation, rank
"""
results = []
for i, vpath in enumerate(variant_paths):
print(f" Scoring variant {i+1}/{len(variant_paths)}: {Path(vpath).name}")
try:
activation, _ = predict_region_activation(model, vpath, modality, region_vertices)
results.append({
"variant_file": vpath,
"target_region": target_region,
"target_region_mean_activation": activation,
})
except Exception as e:
print(f" WARNING: Failed to score {vpath}: {e}")
# Sort by activation descending
results.sort(key=lambda x: x["target_region_mean_activation"], reverse=True)
# Add rank
for rank, row in enumerate(results, start=1):
row["rank"] = rank
return results
def save_results_csv(results: List[Dict], output_path: str) -> None:
"""Save ranked results to CSV."""
if not results:
print("No results to save.")
return
fieldnames = ["rank", "variant_file", "target_region", "target_region_mean_activation"]
with open(output_path, "w", newline="") as f:
writer = csv.DictWriter(f, fieldnames=fieldnames)
writer.writeheader()
for row in results:
writer.writerow({k: row[k] for k in fieldnames})
print(f"Saved rankings CSV: {output_path}")
def main():
parser = argparse.ArgumentParser(
description="Greedy stimulus optimization using TRIBE v2 cortical predictions.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Optimize a video for visual cortex activation
python optimize_stimulus.py --input ad.mp4 --target-region visual --modality video --n-variants 10 --output-dir ./optimized/
# Optimize audio for language network activation
python optimize_stimulus.py --input narration.wav --target-region language --modality audio --n-variants 8 --output-dir ./optimized/
# Text optimization (prints paraphrase suggestions)
python optimize_stimulus.py --input script.txt --target-region language --modality text --n-variants 6 --output-dir ./optimized/
Available target regions: visual, auditory, language, motion, default_mode
""",
)
parser.add_argument(
"--input",
required=True,
help="Path to the base stimulus file (video, audio, or text)",
)
parser.add_argument(
"--target-region",
required=True,
choices=VALID_REGIONS,
help=f"Target cortical region to maximize. Options: {', '.join(VALID_REGIONS)}",
)
parser.add_argument(
"--modality",
required=True,
choices=VALID_MODALITIES,
help=f"Input modality. Options: {', '.join(VALID_MODALITIES)}",
)
parser.add_argument(
"--n-variants",
type=int,
default=10,
help="Number of stimulus variants to generate and evaluate (default: 10)",
)
parser.add_argument(
"--output-dir",
default="./optimized",
help="Directory to save variants and results (default: ./optimized)",
)
parser.add_argument(
"--cache-folder",
default="./cache",
help="Cache folder for TRIBE v2 model weights (default: ./cache)",
)
args = parser.parse_args()
# Validate input file
input_path = Path(args.input).resolve()
if not input_path.exists():
print(f"ERROR: Input file not found: {input_path}")
sys.exit(1)
# Create output directory
output_dir = Path(args.output_dir).resolve()
output_dir.mkdir(parents=True, exist_ok=True)
print(f"\nTRIBE v2 Stimulus Optimizer")
print(f"=" * 50)
print(f"Input: {input_path}")
print(f"Target region: {args.target_region}")
print(f"Modality: {args.modality}")
print(f"N variants: {args.n_variants}")
print(f"Output dir: {output_dir}")
print(f"=" * 50 + "\n")
# Handle text modality separately (no auto-perturbation)
if args.modality == "text":
generate_text_suggestions(str(input_path), args.n_variants)
print("For text modality, create variants manually and re-run this script.")
print("No predictions will be run on unmodified text.")
sys.exit(0)
# Load model
model = load_model(cache_folder=args.cache_folder)
# Get target region vertices
region_vertices = get_region_vertices(args.target_region)
print(f"Target region '{args.target_region}': {len(region_vertices)} vertices")
# Predict baseline activation
print(f"\nPredicting baseline activation for: {input_path.name}")
try:
baseline_activation, _ = predict_region_activation(
model, str(input_path), args.modality, region_vertices
)
print(f"Baseline {args.target_region} activation: {baseline_activation:.6f}")
except Exception as e:
print(f"ERROR: Failed to predict baseline: {e}")
sys.exit(1)
# Generate variants
print(f"\nGenerating {args.n_variants} stimulus variants...")
if args.modality == "video":
variant_paths = generate_video_variants(
str(input_path), str(output_dir), args.n_variants
)
elif args.modality == "audio":
variant_paths = generate_audio_variants(
str(input_path), str(output_dir), args.n_variants
)
else:
variant_paths = []
if not variant_paths:
print("WARNING: No variants were generated.")
print("Check that ffmpeg is installed (brew install ffmpeg) and try again.")
sys.exit(1)
print(f"Generated {len(variant_paths)} variants.")
# Score all variants
print(f"\nScoring variants on '{args.target_region}' region...")
results = score_variants(
model, variant_paths, args.modality, region_vertices, args.target_region
)
if not results:
print("ERROR: No variants could be scored.")
sys.exit(1)
# Save results
csv_path = output_dir / "rankings.csv"
save_results_csv(results, str(csv_path))
# Print summary
best = results[0]
best_activation = best["target_region_mean_activation"]
improvement_pct = ((best_activation - baseline_activation) / abs(baseline_activation) * 100
if baseline_activation != 0 else 0.0)
print(f"\n{'=' * 50}")
print(f"OPTIMIZATION RESULTS")
print(f"{'=' * 50}")
print(f"Baseline activation: {baseline_activation:.6f}")
print(f"Best variant activation: {best_activation:.6f}")
print(f"Improvement: {improvement_pct:+.1f}%")
print(f"Best variant: {Path(best['variant_file']).name}")
print(f"")
print(f"Top 3 variants by {args.target_region} activation:")
for row in results[:3]:
print(f" Rank {row['rank']}: {Path(row['variant_file']).name} "
f"(activation={row['target_region_mean_activation']:.6f})")
print(f"\nFull rankings saved to: {csv_path}")
if improvement_pct > 5:
print(f"\nRECOMMENDATION: Use {Path(best['variant_file']).name}")
print(f" +{improvement_pct:.1f}% predicted {args.target_region} activation vs baseline.")
elif improvement_pct > 0:
print(f"\nNote: Marginal improvement ({improvement_pct:+.1f}%). Consider wider perturbation range.")
else:
print(f"\nNote: No variants outperformed baseline. Original stimulus may already be optimal.")
print(f"Try with more variants (--n-variants 20) or different target region.")
if __name__ == "__main__":
main()
Related skills
FAQ
Can I use it commercially?
No. TRIBE v2 is CC BY-NC 4.0, so this skill is for non-commercial research only; commercial use needs a separate license from Meta.
Does it need a brain scanner?
No. It predicts fMRI cortical responses to media without any scanner.