Tribe V2 Neuroscience

TRIBE v2 Neuroscience

In-silico neuroscience using Meta FAIR's TRIBE v2 — predict fMRI cortical responses to video, audio, or text using a single pretrained transformer. Run experiments on any hardware, without a scanner.

What TRIBE v2 Is

TRansformer for In-silico Brain Experiments (v2) is a brain encoding model released by Meta FAIR on March 26, 2026. It is not a language model — it predicts fMRI BOLD responses on the fsaverage5 cortical surface (~20,000 vertices per hemisphere) given multimodal sensory input.

Architecture:

Video  → V-JEPA2 (video encoder)
Audio  → Wav2Vec-BERT 2.0 (audio encoder)
Text   → LLaMA 3.2-3B (text encoder)
         ↓
   Unified Transformer
         ↓
   fsaverage5 mesh (~20k vertices)
   (n_timesteps × n_vertices)

Key properties:

• 70x resolution improvement over TRIBE v1
• 2-3x accuracy improvement, zero-shot generalization to new subjects
• 5-second temporal offset built in — accounts for hemodynamic lag
• Log-linear scaling with fMRI training data (like LLMs with tokens)
• License: CC BY-NC 4.0 (non-commercial research only)
• HuggingFace: facebook/tribev2
• Demo: https://aidemos.atmeta.com/tribev2

---

Quick Start

1. Install TRIBE v2

# Requires Python 3.11+
git clone https://github.com/facebookresearch/tribev2
cd tribev2
pip install -e .

2. Load the Model

from tribev2 import TribeModel

model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")

The first load downloads model weights (~7GB). Subsequent loads use the cache.

3. Run Your First Prediction

# Video input (returns DataFrame of events)
df = model.get_events_dataframe(video_path="stimulus.mp4")

# Text input
df = model.get_events_dataframe(text_path="transcript.txt")

# Audio input
df = model.get_events_dataframe(audio_path="audio.wav")

# Predict cortical responses
preds, segments = model.predict(events=df)

# Output shape: (n_timesteps, n_vertices)
# n_vertices ≈ 20,484 on fsaverage5
print(preds.shape)   # e.g., (142, 20484)
print(segments)      # list of segment boundaries in seconds

---

Workflow A: Single Stimulus Prediction

Trigger: You have one video, audio clip, or transcript and want to know which brain regions activate.

from tribev2 import TribeModel
import numpy as np
import pandas as pd

# Load model
model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")

# Load stimulus (choose one modality)
df = model.get_events_dataframe(video_path="face_stimulus.mp4")
# df = model.get_events_dataframe(audio_path="speech.wav")
# df = model.get_events_dataframe(text_path="story.txt")

# Predict
preds, segments = model.predict(events=df)
# preds: numpy array (n_timesteps, n_vertices)

# Find peak activation timestep
peak_t = np.argmax(preds.mean(axis=1))
peak_activations = preds[peak_t, :]

# Top-10 most activated vertices at peak
top_verts = np.argsort(peak_activations)[-10:][::-1]
print(f"Peak timestep: {peak_t} (~{peak_t * 1.5:.1f}s)")
print(f"Top vertices: {top_verts}")
print(f"Peak activation values: {peak_activations[top_verts]}")

# Regional mean activations (using known fsaverage5 ranges)
REGIONS = {
    "V1_left":  (0, 1500),
    "FFA_right": (9900, 10400),
    "A1_left":  (3500, 4200),
    "Broca_left": (6200, 6800),
    "DMN_mPFC": (14000, 15000),
}

for region, (v_start, v_end) in REGIONS.items():
    region_mean = preds[:, v_start:v_end].mean()
    print(f"  {region}: mean activation = {region_mean:.4f}")

Use the Cortical Atlas to interpret which vertices correspond to which regions.

---

Workflow B: Virtual Experiment

Trigger: You want to compare brain responses across multiple stimuli — e.g., faces vs. objects vs. scenes.

from tribev2 import TribeModel
import numpy as np
import pandas as pd
from pathlib import Path

model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")

# Define your stimulus set
stimuli = {
    "faces":   "stimuli/faces.mp4",
    "scenes":  "stimuli/scenes.mp4",
    "objects": "stimuli/objects.mp4",
    "baseline": "stimuli/scrambled.mp4",
}

# Define regions of interest (vertex ranges on fsaverage5)
ROIS = {
    "FFA_right": (9900, 10400),    # Fusiform Face Area
    "PPA_right": (10400, 11000),   # Parahippocampal Place Area
    "LOC_right": (8800, 9500),     # Lateral Occipital Complex
    "EBA_right": (9500, 9900),     # Extrastriate Body Area
}

results = []

for condition, path in stimuli.items():
    df = model.get_events_dataframe(video_path=path)
    preds, _ = model.predict(events=df)
    
    for roi_name, (v_start, v_end) in ROIS.items():
        roi_activation = preds[:, v_start:v_end].mean()
        results.append({
            "condition": condition,
            "roi": roi_name,
            "mean_activation": roi_activation,
            "peak_activation": preds[:, v_start:v_end].max(),
        })

# Analyze
df_results = pd.DataFrame(results)
pivot = df_results.pivot(index="condition", columns="roi", values="mean_activation")
print(pivot)

# Expected result for face selectivity:
# FFA_right should be highest for "faces" condition
# PPA_right should be highest for "scenes" condition

Use scripts/run_experiment.py to automate this over a directory of stimuli.

---

Workflow C: Paradigm Replication

Trigger: You want to replicate a classic neuroscience finding in-silico before running a real fMRI study.

Example: Replicating face selectivity in the Fusiform Face Area (Kanwisher 1997).

from tribev2 import TribeModel
import numpy as np

model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")

# Kanwisher 1997: faces >> objects in FFA
# FFA is right-lateralized, ~vertices 9900-10400 (fsaverage5)
FFA_RIGHT = (9900, 10400)
FFA_LEFT  = (1100, 1600)   # smaller response expected

face_df   = model.get_events_dataframe(video_path="faces_stimulus.mp4")
object_df = model.get_events_dataframe(video_path="objects_stimulus.mp4")

face_preds,   _ = model.predict(events=face_df)
object_preds, _ = model.predict(events=object_df)

# Compute selectivity index
def roi_mean(preds, vertex_range):
    v_start, v_end = vertex_range
    return preds[:, v_start:v_end].mean()

face_ffa_r   = roi_mean(face_preds, FFA_RIGHT)
object_ffa_r = roi_mean(object_preds, FFA_RIGHT)
face_ffa_l   = roi_mean(face_preds, FFA_LEFT)

selectivity_index = (face_ffa_r - object_ffa_r) / (face_ffa_r + object_ffa_r + 1e-8)

print(f"FFA-right (faces):   {face_ffa_r:.4f}")
print(f"FFA-right (objects): {object_ffa_r:.4f}")
print(f"FFA-left  (faces):   {face_ffa_l:.4f}")
print(f"Selectivity index:   {selectivity_index:.4f}")

# Positive selectivity_index = FFA prefers faces
# Right > Left = expected right lateralization
# This replicates Kanwisher 1997 in-silico

See paradigm-library.md for 8 fully documented paradigms with expected results and vertex ranges.

---

Workflow D: Hypothesis Testing

Trigger: You have a hypothesis like "visual cortex responds to motion but not to static images" and want to test it computationally.

Step 1: Define your hypothesis formally

H0: mean_activation(MT_right, motion_video) == mean_activation(MT_right, static_image_video)
H1: mean_activation(MT_right, motion_video) > mean_activation(MT_right, static_image_video)
Region: MT/V5 right hemisphere, vertices 7800-8200 (approx fsaverage5)

Step 2: Generate contrasting stimuli

The stimuli must differ only on the dimension you're testing. For motion vs. static:

• Motion: videos with global optic flow (dot fields, moving gratings)
• Static: same scene photographed repeatedly (no temporal change)

Step 3: Run predictions and compute the contrast

from tribev2 import TribeModel
import numpy as np
from scipy import stats

model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")

MT_RIGHT = (7800, 8200)

# Multiple clips per condition for effect size estimation
motion_clips  = ["motion_01.mp4", "motion_02.mp4", "motion_03.mp4"]
static_clips  = ["static_01.mp4", "static_02.mp4", "static_03.mp4"]

def get_roi_activation(clips, roi):
    activations = []
    for clip in clips:
        df = model.get_events_dataframe(video_path=clip)
        preds, _ = model.predict(events=df)
        v_start, v_end = roi
        activations.append(preds[:, v_start:v_end].mean())
    return np.array(activations)

motion_acts = get_roi_activation(motion_clips, MT_RIGHT)
static_acts = get_roi_activation(static_clips, MT_RIGHT)

# One-tailed t-test
t_stat, p_val = stats.ttest_ind(motion_acts, static_acts, alternative='greater')
effect_size   = (motion_acts.mean() - static_acts.mean()) / np.std(np.concatenate([motion_acts, static_acts]))

print(f"Motion MT activation:  {motion_acts.mean():.4f} ± {motion_acts.std():.4f}")
print(f"Static MT activation:  {static_acts.mean():.4f} ± {static_acts.std():.4f}")
print(f"t = {t_stat:.3f}, p = {p_val:.4f}")
print(f"Cohen's d = {effect_size:.3f}")
print(f"H1 supported: {p_val < 0.05 and t_stat > 0}")

Step 4: Interpret and decide whether to proceed to real fMRI

If the in-silico result supports H1 with d > 0.5, the effect size is large enough to power a real study. Use TRIBE v2 output to:

• Estimate required sample size (TRIBE v2 predictions correlate with real fMRI at r~0.6-0.8)
• Identify best ROIs to measure in-scanner
• Pre-register your analysis plan

---

Output Interpretation

Shape: (n_timesteps, n_vertices)

preds, segments = model.predict(events=df)

# preds.shape[0] = number of TRs (fMRI volumes)
#   Each TR ≈ 1.5 seconds (typical fMRI repetition time)
#   Total duration covered = n_timesteps × 1.5s

# preds.shape[1] = 20,484 vertices (fsaverage5 surface)
#   Vertices 0–10,241      = Left hemisphere
#   Vertices 10,242–20,483 = Right hemisphere

What the values mean

TRIBE v2 outputs z-scored BOLD signal predictions in arbitrary units:

• 0.0 = mean response for this brain region (no activation above baseline)
• > 0 = above-average activation
• < 0 = below-average (suppression or deactivation)
• Typical range: [-3.0, 3.0]

The 5-second temporal offset

TRIBE v2 automatically applies a +5s hemodynamic lag. The prediction at timestep t reflects neural processing that occurred at t - 5s in the stimulus. You do not need to manually shift; the model handles this.

Vertex-to-region mapping

fsaverage5 has 10,242 vertices per hemisphere. Key landmarks:

Left hemisphere (vertices 0–10,241):
  V1/V2 primary visual:    0–1,500
  V3/V4 ventral stream:   1,500–3,000
  MT/V5 motion:           2,800–3,200
  A1 primary auditory:    3,500–4,200
  Broca's area (44/45):   6,200–6,800
  VWFA (word form):       7,100–7,500

Right hemisphere (vertices 10,242–20,483):
  V1/V2 primary visual:  10,242–11,742
  FFA (face area):        9,900–10,400  ← note: RH vertex numbers
  PPA (place area):      10,400–11,000
  A1 primary auditory:   13,742–14,442

See references/brain-regions.md for the full atlas with all regions, hemispheres, and vertex ranges.

Segments

preds, segments = model.predict(events=df)
# segments: list of (start_sec, end_sec) tuples
# Corresponds to natural scene/speech boundaries the model detected
# Useful for aligning predictions to stimulus timing

---

Using the Companion Scripts

Single Prediction

python scripts/predict_brain.py \
  --input stimulus.mp4 \
  --modality video \
  --output results/predictions.csv \
  --cache-dir ./model-cache

Output CSV columns: timestep, vertex_id, predicted_activation

Also prints top-5 vertices at peak timestep to stdout.

Batch Experiment

python scripts/run_experiment.py \
  --stimuli-dir stimuli/faces/ \
  --modality video \
  --output-dir results/face_experiment/ \
  --region FFA_right

Output CSV: stimulus_file, region, mean_activation, peak_timestep

---

Common Pitfalls

---

Detailed References

• [references/brain-regions.md](references/brain-regions.md) — Full cortical atlas: every region, hemisphere, fsaverage5 vertex range, and what activates it
• [references/paradigm-library.md](references/paradigm-library.md) — 8 classic paradigms with TRIBE v2 replication protocols and expected results
• [scripts/predict_brain.py](scripts/predict_brain.py) — CLI for single-stimulus prediction with CSV output
• [scripts/run_experiment.py](scripts/run_experiment.py) — Batch multi-stimulus experiment runner with region-averaged output

Cortical Atlas — fsaverage5 Brain Regions for TRIBE v2

TRIBE v2 outputs predictions on the fsaverage5 surface mesh, which has 10,242 vertices per hemisphere (20,484 total). This reference maps named brain regions to approximate vertex ranges in that combined array.

Important caveats:

• Vertex boundaries are approximate. Individual functional regions vary across subjects.
• The ranges below reflect typical functional boundaries from FreeSurfer parcellations and published fMRI atlases.
• Left hemisphere: vertices 0–10,241. Right hemisphere: vertices 10,242–20,483.
• TRIBE v2 is trained on naturalistic stimuli; very short (< 5s) or highly artificial stimuli may produce weaker signals.

---

Visual Cortex

Lateralization notes (visual):

• FFA is strongly right-lateralized for faces.
• PPA is bilateral with slight right dominance for scenes.
• MT is bilateral; both hemispheres required for full motion processing.
• LOC is bilateral for object recognition.

---

Auditory Cortex

Lateralization notes (auditory):

• Speech: left-lateralized (left A1 + Wernicke's area stronger for phonemes and words).
• Music and prosody: right-lateralized (right belt auditory + right STS).
• STS is a multimodal convergence zone — activates for combined audio-visual speech.

---

Language Network

Lateralization notes (language):

• Language is strongly left-lateralized in ~95% of right-handers and ~70% of left-handers.
• Lateralization index (LI) = (left - right) / (left + right); expect LI > 0.2 for any language stimulus.
• VWFA is exclusively left hemisphere; no right homolog shows significant activation for text.

---

Motor and Somatosensory Cortex

Notes:

• M1 and S1 can activate during observation of actions (action observation network).
• For TRIBE v2 with video input, expect M1/S1 activation when stimuli show people interacting physically (sports, dance, handcraft).
• Somatotopy: face representation at inferior end, leg at superior-medial end of both M1 and S1.

---

Default Mode Network (DMN)

Deactivation pattern:

• The key DMN signature is negative BOLD (deactivation below resting baseline) during externally directed, cognitively demanding tasks.
• In TRIBE v2, this appears as negative predicted activation values in DMN regions for demanding stimuli.
• Conversely, slow narrative content or rest-like stimuli should produce positive activation in DMN.

---

Prefrontal Cortex

Notes:

• DLPFC activates during narrative content with high cognitive load (complex grammar, working memory demands in the story).
• OFC is harder to drive with purely sensory stimuli; best activated by emotionally valenced or reward-predictive content.

---

Quick Lookup: Region by Stimulus Type

---

Notes on Precision

These vertex ranges are best estimates derived from:

• FreeSurfer's Desikan-Killiany and Destrieux cortical parcellations mapped to fsaverage5
• Published fMRI coordinates converted to surface vertices (MNI → fsaverage5 via spherical registration)
• The TRIBE v2 paper's reported region-of-interest analyses

Exact activation peaks will vary with stimulus content and quality. For high-precision region localization, use a full parcellation atlas (e.g., mne.datasets.fetch_fsaverage, then map Glasser HCP 360-region parcellation to fsaverage5 vertices).

For exploratory work, the ranges above are sufficient to detect the canonical activations described in the paradigm library.

Paradigm Library — Classic Neuroscience Findings Replicable In-Silico with TRIBE v2

Each entry covers: the original finding, the stimulus type required, the expected TRIBE v2 vertex range to check, and what a positive replication looks like.

---

1. Face Selectivity in the Fusiform Face Area (FFA)

Original finding: Kanwisher, McDermott & Chun (1997, J. Neuroscience) A region in the right fusiform gyrus (FFA) responds significantly more to upright faces than to objects, houses, or scrambled images. This was the founding paper for the "face patch" system.

Stimulus type needed

• Faces condition: video or images of frontal/profile human faces (no bodies visible, neutral expression acceptable)
• Control condition: matched images/video of common objects (chairs, tools) at same visual complexity

Expected TRIBE v2 vertex range to check

How to interpret a positive replication

• mean_activation(FFA_right, faces) > mean_activation(FFA_right, objects) with a selectivity index > 0.15
• Right FFA should show at least 1.5x higher response than left FFA for faces
• Objects and scrambled stimuli should produce activation near 0 in FFA

selectivity_index = (face_ffa - object_ffa) / (face_ffa + object_ffa + 1e-8)
# Positive replication: selectivity_index > 0.15

---

2. Scene Selectivity in the Parahippocampal Place Area (PPA)

Original finding: Epstein & Kanwisher (1998, Nature) A region in the parahippocampal cortex (PPA) responds maximally to images of places and scenes (indoor rooms, outdoor landscapes) compared to faces or objects.

Stimulus type needed

• Scenes condition: video of outdoor landscapes, cityscapes, indoor environments (no people in foreground)
• Control condition: close-up object or face videos without spatial context

Expected TRIBE v2 vertex range to check

How to interpret a positive replication

• PPA shows higher activation for scenes than for faces or isolated objects
• PPA and FFA should show a double dissociation: scenes > faces in PPA, faces > scenes in FFA
• The double dissociation is the strongest replication signal

---

3. Object Selectivity in the Lateral Occipital Complex (LOC)

Original finding: Malach et al. (1995, PNAS) A lateral occipital region (LOC) responds to intact objects regardless of exact size, viewpoint, or illumination — tuned to object shape rather than low-level features.

Stimulus type needed

• Intact objects condition: video or images of everyday objects in various viewpoints
• Scrambled control: pixel-scrambled versions of the same images (same spatial frequency, no object structure)

Expected TRIBE v2 vertex range to check

How to interpret a positive replication

• mean_activation(LOC, intact_objects) > mean_activation(LOC, scrambled) by at least 0.2 units
• LOC activation should be bilateral (both hemispheres), unlike FFA which is right-dominant
• Should be robust across object category (tools, animals, vehicles)

---

4. Body Selectivity in the Extrastriate Body Area (EBA)

Original finding: Downing et al. (2001, Science) A region in lateral occipitotemporal cortex (EBA) responds selectively to images of human bodies and body parts compared to objects, scrambled bodies, or faces.

Stimulus type needed

• Body condition: video of people walking, silhouettes, body parts (hands, arms) — faces cropped out
• Control condition: matched objects or scrambled body images

Expected TRIBE v2 vertex range to check

How to interpret a positive replication

• EBA activation > baseline for body stimuli
• Face stimuli should not drive EBA (helps dissociate EBA from FFA)
• Both hemispheres should activate; slight right lateralization expected

---

5. Language Lateralization in Broca's Area

Original finding: Broca (1861) (clinical); modern fMRI confirmations by Binder et al. (1997, JCMS) Language production and comprehension are strongly left-lateralized in most right-handed individuals. Broca's area (IFG pars triangularis, BA44/45) activates during sentence comprehension, verbal working memory, and syntactic processing.

Stimulus type needed

• Language condition: spoken or written sentences — narrative, syntactically complex preferred
• Control condition: non-linguistic auditory tone sequences or visual patterns of the same duration

Expected TRIBE v2 vertex range to check

How to interpret a positive replication

• Left Broca's shows higher activation for sentences than for matched non-linguistic stimuli
• Compute lateralization index (LI):

LI = (left_broca - right_broca) / (left_broca + right_broca + 1e-8)
# LI > 0 → left-lateralized (expected)
# LI > 0.2 → strong lateralization (replication)

---

6. Visual Word Form Area (VWFA) Selectivity

Original finding: Cohen et al. (2000, Science) A region in the left fusiform gyrus (VWFA) responds selectively to written words and letter strings compared to non-orthographic visual stimuli such as faces, objects, or symbol strings.

Stimulus type needed

• Words condition: video or images of printed words, sentences, or letter strings in various fonts and cases
• Control condition: false fonts (letter-like symbols with no linguistic value), objects, or faces

Expected TRIBE v2 vertex range to check

How to interpret a positive replication

• VWFA activation significantly higher for real words vs. false-font strings
• Left-lateralized: right hemisphere homolog (17,300–17,700) should show minimal response
• Selectivity should be invariant to font and case (uppercase vs. lowercase both drive VWFA)

---

7. Motion Selectivity in MT/V5

Original finding: Zeki (1974, Brain); Zeki et al. (1991, fMRI) Area MT (middle temporal) / V5 responds selectively to visual motion — coherent moving dot fields, optic flow, moving gratings — compared to static images.

Stimulus type needed

• Motion condition: video with coherent optic flow (moving dot fields, walking humans, flowing water, camera dolly shots)
• Static condition: same content photographed with no camera or object motion (freeze frames replayed)

Expected TRIBE v2 vertex range to check

How to interpret a positive replication

• MT activation significantly higher for motion than static stimuli
• Bilateral: both hemispheres should show the effect
• Effect should be robust across motion types (translational, radial, rotational)
• V1 activation should also increase but less specifically (distinguishes MT from V1 response)

---

8. Default Mode Network Deactivation During Task

Original finding: Raichle et al. (2001, PNAS); Buckner et al. (2008, review) The Default Mode Network (DMN) — comprising mPFC, PCC, angular gyrus, and hippocampus — deactivates during externally-directed cognitive tasks but is active during rest, mind-wandering, and self-referential thought.

Stimulus type needed

• Task condition: cognitively demanding content — rapid arithmetic narration, spatial navigation instructions, fast-changing unfamiliar stimuli that demand attention
• Rest/narrative condition: slow, familiar naturalistic narrative (a familiar story told at relaxed pace)

Expected TRIBE v2 vertex range to check

How to interpret a positive replication

• DMN regions show negative activation (below-baseline suppression) during demanding-task stimuli
• DMN regions show positive activation during naturalistic narrative
• Task-negative contrast = mean_activation(DMN, demanding) - mean_activation(DMN, narrative) should be negative

dmn_contrast = (demanding_mPFC + demanding_PCC) / 2 - (narrative_mPFC + narrative_PCC) / 2
# Replication: dmn_contrast < -0.1

This is the most counterintuitive paradigm to replicate: showing that brain regions deactivate with demanding stimuli.

---

Using These Paradigms in TRIBE v2

All paradigms above require the same basic workflow:

from tribev2 import TribeModel
import numpy as np

model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")

# Load condition stimuli
df_a = model.get_events_dataframe(video_path="condition_A.mp4")
df_b = model.get_events_dataframe(video_path="condition_B.mp4")

preds_a, _ = model.predict(events=df_a)
preds_b, _ = model.predict(events=df_b)

# Extract ROI
v_start, v_end = 9900, 10400  # e.g., FFA_right
roi_a = preds_a[:, v_start:v_end].mean()
roi_b = preds_b[:, v_start:v_end].mean()

contrast = roi_a - roi_b
print(f"Contrast A - B = {contrast:.4f}")

See scripts/run_experiment.py for automated multi-condition batch processing.