
Jax Development
- 46 installs
- 3 repo stars
- Updated June 29, 2026
- tristanmanchester/agent-skills
Helps with ai & agent building tasks during AI-assisted development.
About
jax-development is a Claude Code skill for ai & agent building. It helps solo builders move faster with AI-assisted coding.
- jax-development
- AI & Agent Building
- AI-coding skill
Jax Development by the numbers
- 46 all-time installs (skills.sh)
- Ranked #7,629 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/tristanmanchester/agent-skills --skill jax-developmentAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 46 |
|---|---|
| repo stars | ★ 3 |
| Last updated | June 29, 2026 |
| Repository | tristanmanchester/agent-skills ↗ |
What it does
Helps with ai & agent building tasks during AI-assisted development.
Files
JAX Development
Use this skill for substantial JAX work. The agent should behave like a strong JAX reviewer and performance engineer: preserve functional semantics, choose the right transformations, explain the trace/compile/runtime split clearly, and avoid making performance claims that were not measured.
This version is designed to be unusually agent-friendly. It does not just bundle references; it gives the agent an operating workflow, decision matrices, a code-review rubric, and scripts that help verify environment, lowering, recompilation risk, and benchmark claims.
Core promise
When this skill is active, the default standard is:
1. produce runnable JAX code, not generic advice 2. explain why the change works in JAX terms 3. call out likely sharp bits even if the user did not ask 4. verify claims with the bundled scripts when possible 5. separate compile-time, run-time, transfer, and sharding issues instead of mixing them together
When this skill should own the task
Use this skill when the difficult part of the request is any of the following:
- translating NumPy, SciPy, TensorFlow, or PyTorch code into idiomatic JAX
- fixing tracer, control-flow, PRNG, shape, dtype, or side-effect bugs
- choosing between
jit,vmap,scan,fori_loop,while_loop,cond,grad,jacrev,jacfwd,remat,shard_map, or export - removing recompiles, host-device round trips, Python overhead, or dishonest benchmarking
- reasoning about
jax.Array, meshes,PartitionSpec,NamedSharding, explicit sharding,pmapmigration, multi-host semantics, or collectives - using
jax.debug.print,checkify,make_jaxpr, lowering, compiler IR, profiler traces, or memory profiling - using custom derivatives, export, AOT lowering, custom partitioning, Pallas, or the JAX source tree
Compose this skill with framework-specific skills when needed, but let this one own the JAX-specific reasoning.
Do not over-apply the skill
Do not force JAX when the real problem is one of these instead:
- pure NumPy optimisation where JAX is explicitly out of scope
- generic CUDA, Triton, NCCL, or driver debugging with no meaningful JAX component
- framework-only design questions whose hard part is not JAX
- irregular dynamic object-heavy Python where the right answer is probably to keep the hot path outside JAX
When in doubt, ask: “Is the root of the problem tracing, transformations, array semantics, compilation, sharding, or the JAX runtime?” If yes, use this skill.
First-response workflow
1. Classify the task
Put the request into one or more lanes immediately:
- code design or porting
- debugging or correctness
- performance or compilation
- sharding or distributed execution
- advanced extension points
- JAX repo navigation or source-level questions
Then open the matching reference file:
references/EXPERT-WORKFLOW.mdfor the overall workflowreferences/MENTAL-MODEL.mdfor tracing and staging semanticsreferences/TRANSFORM-DECISION-MATRIX.mdfor choosing primitivesreferences/PORTING-PATTERNS.mdfor NumPy or PyTorch rewritesreferences/CODE-REVIEW-RUBRIC.mdfor self-review before replyingreferences/DEBUGGING-TRIAGE.mdfor error diagnosisreferences/PERFORMANCE-PLAYBOOK.mdfor speed, memory, and compile-time workreferences/SHARDING-PLAYBOOK.mdfor distributed and multi-device designreferences/ADVANCED-EXTENSIONS.mdfor custom autodiff, export, Pallas, FFI, and internalsreferences/REPO-MAP.mdfor local source-tree navigationreferences/SOURCES.mdfor provenance and maintenance notes
2. Inspect before guessing
If the problem could be environment-, backend-, or project-specific, inspect first.
Environment:
python3 scripts/jax_env_report.py --format jsonStatic project scan:
python3 scripts/jax_project_scan.py PATH --format jsonBenchmark a callable honestly:
python3 scripts/jax_benchmark_harness.py --helpInspect jaxpr, lowering, and IR:
python3 scripts/jax_compile_probe.py --helpCheck likely recompile behaviour across cases:
python3 scripts/jax_recompile_explorer.py --helpSearch a local JAX checkout:
python3 scripts/jax_repo_locator.py --help3. Reduce to a minimal reproducer
Prefer the smallest function that still exhibits the behaviour. JAX problems get much easier once shapes, dtypes, batching axes, randomness, and transformation boundaries are explicit.
4. Choose the least powerful mechanism that solves the problem
Default ordering:
- pure eager
jax.numpyfirst - then
jitorvalue_and_grad - then
vmaporscan - then explicit sharding
- then
shard_map - then custom derivative, export, custom partitioning, or Pallas
- then FFI or JAX internals
Escalate only with evidence.
5. End with a high-signal answer
Unless the user asked for something else, the reply should end with:
- diagnosis or design choice
- corrected code or patch
- why it works in JAX terms
- how to verify it
- remaining risks, backend caveats, or performance unknowns
Expert operating rules
1. Treat JAX functions as pure. Inputs in, outputs out. Hidden mutation, global state, or implicit randomness are usually design bugs once transforms enter the picture. 2. Make randomness explicit. Thread keys through the program, split once per consumer, and return updated keys when state continues. 3. Keep the hot path in JAX space. Host conversion inside transformed code is almost always a bug or a sync point. 4. Separate static and dynamic values. Shapes, dtypes, Python objects, and some configuration values influence tracing and compilation. 5. Use structured control flow. If a branch or loop depends on array values, use JAX control-flow primitives instead of Python. 6. Benchmark honestly. Warm up, block, and distinguish transfer cost, compile cost, and steady-state execution. 7. Optimise after evidence. Use scans, compile probes, profiler traces, or lowering inspection before proposing deep rewrites. 8. Prefer current JAX idioms. Typed keys, jax.Array, and modern sharding APIs are the default unless the codebase is intentionally legacy. 9. Think globally for sharding first. Start with global-view code and explicit placement before dropping to per-device manual code. 10. Never bluff backend-specific behaviour. CPU, GPU, TPU, and multi-host runs differ materially. Say what was verified and what was inferred.
Default red flags to proactively check
Always scan for these, even if the user did not mention them:
np.asarray,.item(),.tolist(),jax.device_get, or printing arrays in a hot path- Python
if,for, orwhileinside transformed code - shape construction or indexing based on traced values
- global or reused PRNG keys
- repeated creation of jitted callables inside loops
- changing shapes, dtypes, or static arguments causing compile storms
- very large Python loops that should be
scanorfori_loop pmapcode that may be better expressed with modern sharding APIs- unexplained precision assumptions or implicit
x64expectations - replicated-versus-sharded confusion in distributed code
Available scripts
scripts/jax_env_report.py— report versions, backend, devices, config, env vars, and an optional smoke test.scripts/jax_project_scan.py— AST-based scan for common JAX sharp bits and migration targets.scripts/jax_benchmark_harness.py— benchmark a callable with warm-up, blocking, optionaljit, and optional donation.scripts/jax_compile_probe.py— inspecteval_shape, jaxpr, lowering, and compiler IR; optionally write artefacts to disk.scripts/jax_recompile_explorer.py— run several input cases through a jitted function and flag likely recompiles or signature drift.scripts/jax_repo_locator.py— search a local JAX checkout for relevant docs, tests, or source files by topic.
All scripts are non-interactive, support --help, and default to structured JSON output.
Available assets
assets/mre_template.py— minimal reproducible example templateassets/training_step_template.py— idiomatic compiled training step with explicit key plumbingassets/scan_template.py— carry-state loop usinglax.scanassets/sharding_template.py— mesh plusNamedShardingstarterassets/shard_map_template.py— manual SPMD starter usingjax.shard_mapassets/benchmark_template.py— honest timing pattern with warm-up and blockingassets/profile_template.py— trace and memory-profile starterassets/checkify_template.py— runtime checks that survivejitassets/custom_vjp_template.py— custom reverse-mode rule starterassets/export_template.py— export and serialisation starterassets/pallas_kernel_skeleton.py— kernel-level starting pointassets/issue_report_template.md— compact bug report / investigation template
Output quality bar
Before sending a final answer, mentally run the code or design through references/CODE-REVIEW-RUBRIC.md. The answer should usually satisfy all of the following:
- runnable or patch-ready code
- correct transformation and sharding semantics
- explicit discussion of compile and runtime consequences
- no accidental host round trips in the claimed hot path
- no hidden PRNG or state bugs
- an honest verification method
If the task is exploratory research code
Prefer a staged plan:
1. get a correct eager version in jax.numpy 2. add tests or invariants 3. add transformations one at a time 4. benchmark and profile 5. only then attempt aggressive sharding or kernel work
This workflow beats premature jit/pmap/Pallas every time.
Skill maintenance
When updating this skill, refresh the JAX facts most likely to drift:
- installation guidance
- sharding APIs and
pmapmigration status - randomness recommendations
- profiler and memory-tooling guidance
- export / AOT APIs
- Pallas and custom extension interfaces
#!/usr/bin/env python3
"""Honest JAX benchmarking starter."""
import time
import jax
import jax.numpy as jnp
def workload(x):
return jnp.tanh(x @ x.T).sum()
def main():
x = jax.random.normal(jax.random.key(0), (2048, 256))
workload_jit = jax.jit(workload)
# Warm-up / compile
workload_jit(x).block_until_ready()
times = []
for _ in range(10):
t0 = time.perf_counter()
y = workload_jit(x)
y.block_until_ready()
times.append((time.perf_counter() - t0) * 1e3)
print("steady-state times_ms:", times)
print("mean_ms:", sum(times) / len(times))
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""Runtime checks that survive jit via checkify."""
from jax.experimental import checkify
import jax
import jax.numpy as jnp
def f(x, i):
checkify.check(i >= 0, "index must be non-negative: {i}", i=i)
checkify.check(i < x.shape[0], "index out of bounds: {i}", i=i)
y = x[i]
checkify.check(jnp.isfinite(y).all(), "non-finite output")
return y
def main():
x = jnp.arange(8, dtype=jnp.float32)
checked = checkify.checkify(f, errors=checkify.user_checks | checkify.index_checks)
err, y = jax.jit(checked)(x, 3)
err.throw()
print("y:", y)
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""Custom VJP starter."""
import jax
import jax.numpy as jnp
@jax.custom_vjp
def clipped_log1p(x):
return jnp.log1p(jnp.maximum(x, -0.999999))
def clipped_log1p_fwd(x):
y = clipped_log1p(x)
return y, x
def clipped_log1p_bwd(residual, g):
(x,) = (residual,)
grad = 1.0 / (1.0 + jnp.maximum(x, -0.999999))
return (g * grad,)
clipped_log1p.defvjp(clipped_log1p_fwd, clipped_log1p_bwd)
def main():
x = jnp.array([0.0, 1.0, 2.0], dtype=jnp.float32)
print(clipped_log1p(x))
print(jax.grad(lambda t: jnp.sum(clipped_log1p(t)))(x))
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""Export / serialisation starter."""
import numpy as np
import jax
import jax.numpy as jnp
from jax import export
def f(x):
return jnp.sin(x) + 2.0 * x
def main():
sig = jax.ShapeDtypeStruct((4,), np.float32)
exported = export.export(jax.jit(f))(sig)
print("platforms:", exported.platforms)
print("in_avals:", exported.in_avals)
print("out_avals:", exported.out_avals)
if __name__ == "__main__":
main()
JAX issue / investigation template
Summary
What failed, and under which transform or backend?
Minimal reproducer
# paste the smallest runnable example hereExpected behaviour
What should have happened?
Actual behaviour
What happened instead? Include the exact exception if any.
Environment
Paste:
python3 scripts/jax_env_report.py --format textNotes
- shapes and dtypes:
- backend and device count:
- whether the problem reproduces eagerly:
- whether it reproduces on CPU:
- whether shapes vary across calls:
- whether randomness is involved:
#!/usr/bin/env python3
"""Minimal reproducible example template for JAX bugs."""
import jax
import jax.numpy as jnp
def f(x):
# Replace with the smallest function that still reproduces the problem.
return x + 1
def main():
x = jnp.arange(4, dtype=jnp.float32)
print("eager:", f(x))
print("jit:", jax.jit(f)(x))
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""Minimal Pallas skeleton.
This is only a starting point for real kernel work after higher-level JAX
optimisation has been exhausted.
"""
import jax
import jax.numpy as jnp
from jax.experimental import pallas as pl
def add_one_kernel(x_ref, y_ref):
idx = pl.program_id(0)
y_ref[idx] = x_ref[idx] + 1.0
def main():
x = jnp.arange(16, dtype=jnp.float32)
y = pl.pallas_call(
add_one_kernel,
out_shape=jax.ShapeDtypeStruct(x.shape, x.dtype),
grid=(x.shape[0],),
)(x)
print(y)
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""Trace and memory-profile starter."""
import jax
import jax.numpy as jnp
import jax.profiler
@jax.jit
def step(x):
return jnp.tanh(x @ x.T)
def main():
x = jax.random.normal(jax.random.key(0), (2048, 256))
with jax.profiler.trace("/tmp/jax-trace"):
for _ in range(5):
x = step(x[:, :256])
x.block_until_ready()
# Optional: capture device memory profile for OOM / leak work.
jax.profiler.save_device_memory_profile("/tmp/jax-memory.prof")
print("Wrote /tmp/jax-trace and /tmp/jax-memory.prof")
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""Carry-state loop starter using lax.scan."""
import jax
import jax.numpy as jnp
def step(carry, x_t):
state = carry
new_state = 0.95 * state + x_t
y_t = jnp.tanh(new_state)
return new_state, y_t
@jax.jit
def run(init_state, xs):
final_state, ys = jax.lax.scan(step, init_state, xs)
return final_state, ys
def main():
xs = jnp.linspace(0.0, 1.0, 10)
final_state, ys = run(jnp.array(0.0), xs)
print("final_state:", final_state)
print("ys:", ys)
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""Starter for manual SPMD code with jax.shard_map."""
from __future__ import annotations
import numpy as np
import jax
import jax.numpy as jnp
from jax.sharding import Mesh, PartitionSpec as P
def main():
devices = np.array(jax.devices())
if devices.size < 2:
raise SystemExit("This template expects at least 2 devices.")
mesh = Mesh(devices.reshape((devices.size,)), ("data",))
@jax.shard_map(mesh=mesh, in_specs=P("data"), out_specs=P("data"))
def per_shard_scale(x):
# `x` is the local shard view. Add collectives here if needed.
return 2.0 * x
x = jnp.arange(devices.size * 4, dtype=jnp.float32).reshape(devices.size, 4)
y = per_shard_scale(x)
print(y)
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""Starter for global-view sharding with Mesh and NamedSharding."""
from __future__ import annotations
import numpy as np
import jax
import jax.numpy as jnp
from jax.sharding import Mesh, NamedSharding, PartitionSpec as P
def main():
devices = np.array(jax.devices())
if devices.size < 2:
raise SystemExit("This template expects at least 2 devices.")
mesh = Mesh(devices.reshape((devices.size,)), ("data",))
data_sharding = NamedSharding(mesh, P("data", None))
repl_sharding = NamedSharding(mesh, P())
x = jax.device_put(jnp.arange(devices.size * 8, dtype=jnp.float32).reshape(devices.size, 8), data_sharding)
w = jax.device_put(jnp.eye(8, dtype=jnp.float32), repl_sharding)
@jax.jit
def f(x, w):
return x @ w
y = f(x, w)
print("input sharding:", x.sharding)
print("output sharding:", y.sharding)
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""Compiled training-step starter with explicit PRNG plumbing."""
from __future__ import annotations
import functools
import jax
import jax.numpy as jnp
def init_params(key, in_dim: int, out_dim: int):
k1, k2 = jax.random.split(key)
w = 0.01 * jax.random.normal(k1, (in_dim, out_dim))
b = jnp.zeros((out_dim,), dtype=w.dtype)
return {"w": w, "b": b}, k2
def model(params, x):
return x @ params["w"] + params["b"]
def loss_fn(params, batch, key):
x, y = batch
key, dropout_key = jax.random.split(key)
logits = model(params, x)
keep = jax.random.bernoulli(dropout_key, p=0.9, shape=logits.shape)
logits = jnp.where(keep, logits / 0.9, 0.0)
loss = jnp.mean((logits - y) ** 2)
metrics = {"loss": loss}
return (loss, metrics), key
@functools.partial(jax.jit, donate_argnums=(0,))
def train_step(params, batch, key, lr=1e-2):
def wrapped_loss(p):
(loss, metrics), new_key = loss_fn(p, batch, key)
return loss, (metrics, new_key)
(loss, (metrics, new_key)), grads = jax.value_and_grad(wrapped_loss, has_aux=True)(params)
new_params = jax.tree.map(lambda p, g: p - lr * g, params, grads)
sq_norm = sum((jnp.sum(g * g) for g in jax.tree.leaves(grads)), start=jnp.array(0.0, dtype=loss.dtype))
metrics = {**metrics, "grad_norm": jnp.sqrt(sq_norm)}
return new_params, new_key, metrics
def main():
key = jax.random.key(0)
params, key = init_params(key, in_dim=8, out_dim=4)
x = jax.random.normal(key, (16, 8))
y = jax.random.normal(key, (16, 4))
params, key, metrics = train_step(params, (x, y), key)
print(jax.device_get(metrics))
if __name__ == "__main__":
main()
{
"skill_name": "jax-development",
"evals": [
{
"id": "control-flow-cond",
"prompt": "Refactor evals/files/bad_control_flow.py into idiomatic JAX that works under jit, and explain why the original version fails.",
"expected_output": "A corrected version that replaces data-dependent Python control flow with a JAX control-flow primitive or array-wise selection, plus a clear explanation of tracer/concretisation semantics.",
"files": [
"evals/files/bad_control_flow.py"
],
"assertions": [
"The solution replaces Python data-dependent control flow inside jitted code with `lax.cond`, `lax.switch`, or `jnp.where`.",
"The response explains that Python tried to use a traced value as a concrete boolean or equivalent control-flow guard.",
"The final code is runnable and remains pure."
]
},
{
"id": "prng-threading",
"prompt": "Fix the PRNG handling in evals/files/prng_reuse.py. I want explicit key threading and a short note on how to verify we are not reusing keys.",
"expected_output": "A rewritten version that accepts and returns keys explicitly, splits keys correctly, and explains why reusing a key gives repeated random behaviour.",
"files": [
"evals/files/prng_reuse.py"
],
"assertions": [
"The solution removes the hidden module-level key from the hot path or clearly avoids using it.",
"The rewritten code uses `split` or equivalent explicit key management.",
"The response explains how to verify the fix, for example by repeated calls or key-reuse debugging."
]
},
{
"id": "benchmark-honesty",
"prompt": "Review evals/files/naive_benchmark.py and show me how to benchmark it properly on JAX.",
"expected_output": "An answer that distinguishes compile time from steady-state execution, uses blocking, and avoids timing only dispatch.",
"files": [
"evals/files/naive_benchmark.py"
],
"assertions": [
"The response mentions warm-up or first-call compilation separately from repeated-call timing.",
"The timing method uses `.block_until_ready()` or `jax.block_until_ready(...)`.",
"The answer warns about asynchronous dispatch or host timing pitfalls."
]
},
{
"id": "pmap-migration",
"prompt": "Look at evals/files/legacy_pmap.py and outline a sensible migration path to newer sharding APIs without changing semantics prematurely.",
"expected_output": "A measured migration plan that preserves semantics first, explains the current `pmap` role, and suggests modern sharding APIs deliberately.",
"files": [
"evals/files/legacy_pmap.py"
],
"assertions": [
"The answer does not recommend a blind rewrite without discussing semantics.",
"The response recognises `pmap` as legacy-oriented relative to newer sharding APIs.",
"The migration plan mentions meshes, named sharding, shard_map, or equivalent modern concepts."
]
},
{
"id": "host-roundtrip",
"prompt": "Diagnose evals/files/host_roundtrip.py. It works sometimes, but I suspect there's an accidental host round-trip inside jit.",
"expected_output": "A diagnosis that identifies the host conversion inside the compiled region and rewrites the code to stay in JAX space until the edge.",
"files": [
"evals/files/host_roundtrip.py"
],
"assertions": [
"The response identifies the NumPy conversion inside `jit` as a problem.",
"The rewritten code returns a JAX value from the jitted function and moves host conversion outside if needed.",
"The explanation mentions performance or tracer implications of host conversion."
]
},
{
"id": "closure-recompile",
"prompt": "Review evals/files/closure_recompile.py and tell me why it is likely to trigger repeated tracing or compilation. Then rewrite it cleanly.",
"expected_output": "A diagnosis of jitted function creation inside a loop or changing closure capture, plus a cleaner design with hoisted compilation.",
"files": [
"evals/files/closure_recompile.py"
],
"assertions": [
"The answer points out that `jax.jit` is being created inside a loop or repeatedly around a closure.",
"The rewritten version hoists `jit` or otherwise stabilises the compilation boundary.",
"The response mentions compile-cache churn or repeated tracing."
]
},
{
"id": "scan-rewrite",
"prompt": "Rewrite evals/files/python_loop_blowup.py in a way that is more JAX-friendly for compile time and explain why.",
"expected_output": "A version using `lax.scan` or `lax.fori_loop` with an explanation that large Python loops under jit can inflate the traced program.",
"files": [
"evals/files/python_loop_blowup.py"
],
"assertions": [
"The answer replaces the large Python loop with a structured JAX loop primitive.",
"The response explains why the original loop could blow up compile size or trace time.",
"The rewritten code preserves the intended accumulation semantics."
]
},
{
"id": "dynamic-mask",
"prompt": "Fix evals/files/dynamic_mask.py for use under jit. I still want something that preserves the semantics as much as possible.",
"expected_output": "An answer that explains why boolean masking creates a dynamic-size result and proposes a fixed-shape alternative such as `where`, padding, or a mask/value pair.",
"files": [
"evals/files/dynamic_mask.py"
],
"assertions": [
"The response explains the dynamic-shape issue with boolean masking under jit.",
"The answer proposes a fixed-shape rewrite or an explicit alternative representation.",
"The answer does not pretend that `x[x > 0]` is directly safe inside jit."
]
}
]
}import jax
import jax.numpy as jnp
@jax.jit
def clip_or_negate(x):
if x.mean() > 0:
return jnp.clip(x, 0.0, 1.0)
return -x
import jax
import jax.numpy as jnp
def run(xs, scale):
outputs = []
for x in xs:
f = jax.jit(lambda y: scale * y + 1.0)
outputs.append(f(x))
return outputs
import jax
import jax.numpy as jnp
@jax.jit
def positives_only(x):
return x[x > 0]
import numpy as np
import jax
import jax.numpy as jnp
@jax.jit
def f(x):
y = jnp.sin(x)
return np.asarray(y).sum()
import jax
import jax.numpy as jnp
@jax.pmap(axis_name="data")
def step(x):
y = x - jax.lax.pmean(x, "data")
return y / (1e-6 + jnp.linalg.norm(y, axis=-1, keepdims=True))
import jax
import jax.numpy as jnp
def matmul_step(x, w):
return jnp.tanh(x @ w)
def benchmark_once():
x = jax.random.normal(jax.random.key(0), (2048, 512))
w = jax.random.normal(jax.random.key(1), (512, 512))
fast = jax.jit(matmul_step)
return fast(x, w) # caller times this without blocking
import jax
import jax.numpy as jnp
GLOBAL_KEY = jax.random.key(0)
@jax.jit
def sample_pair():
a = jax.random.normal(GLOBAL_KEY, (4,))
b = jax.random.uniform(GLOBAL_KEY, (4,))
return a, b
import jax
import jax.numpy as jnp
@jax.jit
def accumulate(x):
acc = jnp.zeros_like(x[0])
for i in range(1000):
acc = acc + jnp.sin(x[i])
return acc
[
{
"query": "Can you turn this NumPy finite-difference update into JAX and batch it over 64 parameter sets with vmap?",
"should_trigger": true
},
{
"query": "why does jit throw ConcretizationTypeError when I branch on an array value?",
"should_trigger": true
},
{
"query": "I think my dropout mask repeats every step in JAX. Please inspect the key plumbing.",
"should_trigger": true
},
{
"query": "Benchmark this JAX training step properly on GPU and tell me whether compile or runtime dominates.",
"should_trigger": true
},
{
"query": "Migrate this pmap loop to the newer sharding style without breaking semantics.",
"should_trigger": true
},
{
"query": "Show me where custom_vjp and batching rules live in the JAX repo.",
"should_trigger": true
},
{
"query": "Export this jitted function as StableHLO and explain the shape assumptions.",
"should_trigger": true
},
{
"query": "Why does jax.debug.print show values in a weird order inside my compiled function?",
"should_trigger": true
},
{
"query": "I need to update formulas in an Excel workbook and fix the conditional formatting.",
"should_trigger": false
},
{
"query": "Optimise this pure NumPy code with Numba; JAX is not allowed here.",
"should_trigger": false
},
{
"query": "Write a CUDA kernel for a custom NCCL all-reduce without using any JAX code.",
"should_trigger": false
},
{
"query": "Can you review my PyTorch Lightning callback setup for checkpoint naming?",
"should_trigger": false
}
][
{
"query": "Port this PyTorch forward pass to functional JAX and make the recurrent loop compile efficiently.",
"should_trigger": true
},
{
"query": "I need help understanding Mesh, PartitionSpec, and NamedSharding for a 2x4 device setup.",
"should_trigger": true
},
{
"query": "This JAX program gets slower every time shapes change. Can you check for recompiles?",
"should_trigger": true
},
{
"query": "Please write a tiny Pallas kernel skeleton for a hotspot after ordinary jit did not help.",
"should_trigger": true
},
{
"query": "Could you clean up my pandas DataFrame and make a chart from the CSV?",
"should_trigger": false
},
{
"query": "Recommend a Linux driver version for CUDA generally; there is no JAX involved.",
"should_trigger": false
},
{
"query": "Explain autograd in general, not JAX specifically.",
"should_trigger": false
},
{
"query": "Help me refactor this plain Python state machine with lots of dictionaries and strings.",
"should_trigger": false
}
]Advanced extensions
Use this file only after ordinary JAX design, transforms, and sharding have been exhausted or ruled out.
Escalation ladder
Prefer the simplest option that solves the real problem:
1. rewrite in plain JAX 2. use a different transform or control-flow primitive 3. add checkpoint / donation / explicit sharding 4. use custom_jvp or custom_vjp 5. use export / AOT lowering 6. use custom_partitioning 7. use Pallas 8. use FFI 9. modify JAX internals directly
custom_jvp / custom_vjp
Use when:
- default autodiff is mathematically wrong for the desired abstraction
- a stable forward formula and stable backward formula differ
- the primitive is too expensive to differentiate naively
Ask:
- is the derivative truly custom, or does the forward pass just need stabilising?
- can the tangent/cotangent rules be tested independently?
- does the custom rule preserve batching / compilation expectations?
Use templates:
assets/custom_vjp_template.py
jax.export and AOT lowering
Use when:
- tracing/lowering and execution must be decoupled
- you need a serialisable staged computation
- you need shape-polymorphic export
- you need to inspect lowered programs without executing them immediately
Use assets/export_template.py as a starter.
custom_partitioning
Use when:
- a function needs specialised sharding-aware lowering
- compiler suggestions are not enough
- the function should participate in distributed compilation with custom rules
This is already advanced. Only recommend it when the sharding semantics are clearly understood.
Pallas
Use when:
- profiling points to a real kernel-level bottleneck
- plain JAX and XLA optimisations are not enough
- the target backend and kernel semantics are understood well enough
Do not recommend Pallas just because code is “slow”. It is the right answer only when:
- the hotspot is real
- the algorithm is stable
- higher-level rewrites have been exhausted
- the user can tolerate backend-specific complexity
Use:
assets/pallas_kernel_skeleton.py
FFI
Use when:
- a custom kernel or external library is already available
- Pallas and other extension points are insufficient
- integration cost is justified by the performance or functionality gain
FFI is powerful but expensive in complexity, portability, and maintenance.
JAX internals
Use source-level work only when:
- the user is modifying JAX itself
- the public API is insufficient
- the issue is truly an internal bug or design change
When reasoning about internals, think in terms of:
- primitives
- tracers
- jaxpr
- abstract evaluation
- batching rules
- autodiff rules
- MLIR / XLA lowering
Use references/REPO-MAP.md and scripts/jax_repo_locator.py.
Advanced-work checklist
Before proposing an advanced extension, verify:
- the simpler alternatives were considered and rejected for a concrete reason
- the user actually needs the extra power
- the backend and portability trade-offs were stated
- the testing plan is clear
- the performance or correctness case is specific, not hand-wavy
Code review rubric
Use this file before finalising an answer. It helps the agent self-review JAX code the way an experienced maintainer would.
Must-fix categories
1. Purity and state
Check:
- any hidden global state?
- any in-place mutation that should be a functional update?
- any traced values leaking into globals or object fields?
If yes, fix before anything else.
2. Randomness
Check:
- is the key explicit?
- is it split exactly where needed?
- is the updated key returned when the state continues?
- is the same key accidentally reused?
If key handling is muddy, the code is not production-ready.
3. Static vs dynamic boundary
Check:
- are Python decisions being made on traced values?
- are array shapes or sizes created from runtime data?
- are large Python objects or lambdas changing across calls?
If yes, expect tracer errors or compile storms.
4. Host-device boundary
Check:
np.asarray.item().tolist()device_get- frequent printing or callbacks
- Python control flow around array values
If any of these sit in the claimed hot path, call them out.
Performance review categories
5. Compilation structure
Check:
- is
jithoisted out of loops? - are many tiny compiled regions being created?
- is the whole step compiled instead of many subpieces when that makes sense?
- are shapes, dtypes, and static args stable across calls?
6. Loop and batch structure
Check:
- Python loop that should be
scan? - Python per-example loop that should be
vmap? - huge unrolled jaxpr likely causing compile blowup?
7. Memory and numerics
Check:
- dtype expectations explicit?
- unnecessary materialisation?
- remat or donation worth considering?
- stable algebra around divisions, logs, exps, masks, and softmax-like code?
8. Benchmark honesty
Check:
- warm-up?
- blocking?
- compile time separated from steady-state?
- host transfer excluded or included intentionally?
- device/backend stated?
No benchmark claim should survive without these.
Distributed review categories
9. Sharding semantics
Check:
- what is the logical global shape?
- which axes are sharded and over what mesh axes?
- which values are replicated?
- are collectives aligned with the right axis names?
- are local vs global semantics explicit?
10. Legacy API choices
Check:
- is
pmapbeing used because it is genuinely appropriate, or just because the code is old? - would modern sharding APIs clarify the logic?
Response-quality categories
11. Explanation quality
A strong answer explains:
- what was wrong
- why the fix works under tracing / compilation / sharding semantics
- what to measure or test next
12. Verification quality
A strong answer gives:
- a small correctness check
- a timing / profiler plan when performance is discussed
- backend caveats if behaviour may differ on GPU/TPU
Rapid scorecard
For each category, grade the draft answer mentally as:
- green: solid
- amber: acceptable but mention caveat
- red: fix before sending
If purity, randomness, static/dynamic boundaries, or benchmark honesty are red, the answer should not be sent yet.
Debugging triage
Use this file for tracer errors, NaNs, shape bugs, mysterious slowdowns, or “works eagerly, fails under jit” reports.
Fast triage order
1. Is the function pure? 2. Is randomness explicit? 3. Is Python control flow touching traced values? 4. Are shapes/dtypes changing between calls? 5. Is there a host boundary in the hot path? 6. Is the failure really a runtime/backend issue instead?
Minimal debugging toolbox
jax.debug.print
Use when you need runtime values inside staged code.
Pattern:
jax.debug.print("loss={loss}", loss=loss)Use this instead of plain print for traced values.
checkify
Use when you need assertions or runtime checks that survive compilation.
Good uses:
- bounds checks
- finiteness checks
- user invariants
Template: assets/checkify_template.py
jax.make_jaxpr
Use when you need to see what program JAX is tracing. Helpful for:
- giant unrolled loops
- unexpected primitives
- captured constants or shape logic
Lowering / compiler IR
Use scripts/jax_compile_probe.py when the issue may be:
- compile storm
- unexpected lowering
- huge jaxpr / IR
- sharding or export confusion
Debug flags
Temporarily consider:
jax_disable_jitjax_debug_nansjax_debug_infsjax_debug_key_reuse
These are debugging tools, not production fixes.
Error families and likely causes
Concretisation / tracer conversion
Symptoms:
ConcretizationTypeErrorTracerBoolConversionErrorTracerArrayConversionErrorNonConcreteBooleanIndexError
Think:
- Python asked for a concrete value too early
- boolean masking is changing shape
- a NumPy helper touched a traced value
- a Python branch depends on data
Default fixes:
cond,switch,scan,while_loop,fori_loop,where- keep fixed shapes
- move host work out of the transform boundary
Unexpected tracer leaks
Symptoms:
- values stored in globals or object fields
UnexpectedTracerError- transforms fail after refactoring stateful code
Think:
- a transformed function is not actually pure
- traced values escaped through mutation or closures
Default fix:
- return values explicitly
- store persistent state as pytrees, not hidden side-effects
Numerical failures
Symptoms:
- NaNs only after
jit - gradients become zero or explode
- eager and jitted outputs diverge too much
Think:
- fused algebra changed evaluation order
- dtype assumptions differ by backend
- masked expressions still evaluate unstable branches
- division, log, exp, norm, or softmax math is unstable
Default fixes:
- stabilise algebra
- inspect dtypes
- use debug flags to catch NaNs early
- compare with tolerances, not exact equality
PRNG bugs
Symptoms:
- same dropout mask every step
- suspiciously identical random samples
- nondeterminism when code “should” be reproducible
Think:
- key reuse
- hidden global key
- missing
fold_infor step or process index - mixing legacy and typed key conventions poorly
Default fixes:
- make key lifetime explicit
- split once per consumer
- return the updated key
- enable key-reuse checking when useful
Runtime/backend failures
Symptoms:
JaxRuntimeError- device OOM
- backend-specific crashes
- only fails on multi-device or multi-host
Think:
- wrong installation/runtime pairing
- sharding mismatch
- collective mismatch
- memory pressure
- backend limitation
Default steps:
- run
scripts/jax_env_report.py - reduce to CPU or single-device if possible
- make sharding explicit
- profile memory if OOM is involved
Debugging patterns that often work
“Disable, isolate, reintroduce”
1. run eagerly 2. add jit 3. add grad 4. add vmap 5. add sharding
The step that breaks tells you what semantics changed.
“Inspect shape before value”
Many JAX bugs are about shape, dtype, or sharding, not the actual numbers. Check those first.
“Rewrite, don’t patch around”
If the root cause is:
- Python branch on array
- Python loop in the hot path
- dynamic result shape
- hidden global RNG
- host round-trip
then structural rewrite beats another debug print.
Useful commands
Environment:
python3 scripts/jax_env_report.py --format textProject scan:
python3 scripts/jax_project_scan.py PATH --format textLowering:
python3 scripts/jax_compile_probe.py --helpWhat to include in the final answer
- exact failure class or root cause
- a minimal fixed example
- why the bug appears only under tracing / compilation if relevant
- how to re-test after the fix
Expert workflow
This file is the control tower for the whole skill. Use it when you need to decide what to do first, what evidence to gather, and what a strong JAX answer should contain.
The five-step loop
1. Model the program before touching the code
Ask these questions immediately, even if only implicitly:
- What are the true inputs and outputs?
- What state is being threaded through the computation?
- Which values are arrays, and which are Python configuration?
- Which axes are batch, time, feature, or device axes?
- Where does randomness enter, and how long should it live?
- Where is the intended compilation boundary?
- Is the user describing a compile problem, a runtime problem, or a scaling problem?
If you cannot answer those, do not start “optimising” yet.
2. Make a tiny reproducible slice
For JAX, a tiny reproducer should expose:
- explicit shapes and dtypes
- explicit key handling
- a single transformation boundary when possible
- the smallest loop or branch that still fails
- backend assumptions if they matter
Use assets/mre_template.py if you need to create one quickly.
3. Gather the cheapest useful evidence
Use this ladder. Climb only as high as needed.
1. Static reasoning
- Does the code violate JAX rules about purity, control flow, or host/device boundaries?
2. Environment evidence
python3 scripts/jax_env_report.py --format json
3. Static project scan
python3 scripts/jax_project_scan.py PATH --format json
4. Lowering evidence
python3 scripts/jax_compile_probe.py ...
5. Timing evidence
python3 scripts/jax_benchmark_harness.py ...
6. Cross-case compile evidence
python3 scripts/jax_recompile_explorer.py ...
7. Profiler / memory evidence
- use the profile and memory templates from
assets/
Do not jump straight to “use Pallas” or “donate buffers” unless the evidence points there.
4. Apply the minimum structural fix
Prefer the smallest change that resolves the real issue.
Examples:
- tracer bool error -> replace data-dependent Python
ifwithlax.condorjnp.where - compile storm -> stabilise shapes / static args / closure capture before touching kernels
- long Python loop ->
scanorfori_loop - repeated RNG -> explicit key plumbing
- slow multi-device code -> make sharding explicit before rewriting the algorithm
5. Prove the fix
A strong answer does not stop at “here is the patch”. It explains how to verify:
- what should now compile once instead of many times
- which timings should be compared
- which values or shapes should stay constant
- where to look in profiler traces
- which caveats still depend on backend or scale
Default response shape
Unless the user asked for something very different, structure the answer like this:
1. Root cause or recommendation 2. Runnable code / patch 3. Why it works 4. How to verify 5. Risks or caveats
Common lane-specific workflows
Porting or refactoring
1. Write pure eager jax.numpy. 2. Remove mutation and hidden state. 3. Thread keys explicitly. 4. Add jit. 5. Add batching (vmap) or loop primitives (scan) if needed. 6. Only then consider sharding.
Debugging
1. Reproduce outside the full training loop. 2. Identify whether the failure is:
- tracing
- control flow
- shape/dtype
- randomness
- side-effects
- sharding / runtime
- numerics
3. Use the appropriate debug tool:
jax.debug.printcheckifymake_jaxpr- lowering inspection
- debug flags
4. Rewrite the offending structure rather than piling on print statements.
Performance
1. Confirm there is a real bottleneck. 2. Separate:
- transfer
- first-call compile
- steady-state execution
- synchronisation / materialisation
3. Look for:
- compile storms
- Python loops
- host round trips
- unintentional replication
- missing donation
- poor sharding
4. Benchmark and profile before any major redesign.
Sharding or distributed work
1. Make the single-device version correct. 2. Express the logical global array shape. 3. Choose sharding mode:
- automatic via
jit - explicit sharding
- manual
shard_map
4. Check local versus global semantics in multi-host code. 5. Only drop to manual collectives when global-view code is insufficient.
What “expert JAX style” usually means
- pure functions
- explicit pytrees
- explicit randomness
- shapes and dtypes treated as first-class design constraints
- minimal host interaction
- loop and branch primitives chosen intentionally
- performance claims backed by timing or IR evidence
- distributed semantics stated clearly rather than implied
When to say “don’t do this in JAX”
Be willing to push back when the design is a bad match:
- extremely irregular variable-length outputs in the hot path
- heavy Python object mutation at every step
- host callbacks every iteration
- tiny scalar-heavy logic where compilation cost dominates
- requests for low-level accelerator tuning before basic JAX issues are fixed
In those cases, offer a hybrid design instead of blindly forcing everything through jit.
Mental model
This file is about how JAX actually executes programs. Use it whenever the user is confused about tracing, staging, compilation, or why JAX behaves differently from NumPy.
The big idea
JAX is not just “NumPy on accelerators”. It is a system for tracing pure numerical Python functions into a simpler intermediate program, transforming that program, and then lowering it for execution.
The three most important phases to keep separate are:
1. Tracing / specialisation
- Python runs once with tracer objects instead of ordinary arrays.
- JAX learns the abstract program: shapes, dtypes, control-flow structure, constants, and primitive operations.
2. Compilation / lowering
- The abstract program is lowered to backend IR and compiled.
3. Execution
- Compiled code runs on CPU, GPU, or TPU, often asynchronously with respect to Python.
Confusion comes from mixing these phases.
What Python sees vs what JAX sees
Python sees:
- objects
- control flow
- lists, dicts, classes
- side-effects
- concrete integers and booleans
JAX sees, during tracing:
- abstract arrays with shape/dtype information
- pure primitive operations
- structured control flow
- pytrees as containers
If Python asks a traced value for something concrete, you get tracer/concretisation errors.
Static versus dynamic
Treat these as potentially part of the compile signature:
- input shapes
- input dtypes
- sharding
- static arguments
- captured Python objects
Treat these as dynamic runtime values:
- array contents
- per-example data
- values carried through
scanorwhile_loop
When a value is truly compile-time configuration, make it static or keep it on the Python side. When it is data, keep the logic in JAX space.
Purity is not optional
The reliable mental model is:
out = f(inputs)not:
f(mutates_globals, consumes_hidden_rng, appends_to_list, logs_everything)Side-effects can appear to work in eager mode, but become misleading or broken under transforms because tracing sees the program structure, not ordinary step-by-step imperative execution.
Why Python control flow breaks
Python if, while, and short-circuiting and/or operate on concrete truth values. Inside jit, array-dependent branches need JAX control flow:
lax.condlax.switchlax.fori_looplax.while_looplax.scan
Elementwise selection often wants jnp.where, not a branch.
Why shapes matter so much
Compilation usually specialises to shapes and dtypes. That means:
- changing shape often means a new compile
- dynamically sized outputs are difficult in compiled JAX
- boolean masking that changes result size is a common anti-pattern
- padded fixed-shape representations are often the right solution
If compile time is exploding, inspect shape variation first.
Async dispatch
JAX frequently returns control to Python before device execution has completed. That means:
- timing without blocking is misleading
- printing or converting arrays can introduce hidden synchronisation
- host-side inspection changes the program’s performance behaviour
This is why honest benchmarks call .block_until_ready() or jax.block_until_ready(...).
Pytrees are the right structured-data model
A pytree is JAX’s way of handling nested structured inputs and outputs. It is usually the right abstraction for:
- model parameters
- optimiser state
- training state
- nested batches or metadata
Use pytrees instead of custom mutation-heavy object graphs whenever possible.
Randomness is explicit by design
JAX’s PRNG model is functional:
- a key is an input value
- using the same key twice gives the same result
- new randomness comes from
splitorfold_in - keys should be threaded through the computation
This design supports reproducibility and parallelism, but it punishes hidden global RNG state.
jax.Array and sharding
Modern JAX uses jax.Array as the unified array abstraction, including arrays that span multiple devices. For many workflows, you should think in terms of a logical global array and then specify or inspect its sharding.
This leads to three increasingly explicit modes:
- automatic parallelism with
jit - explicit sharding in the array type
- fully manual per-device code with
shard_map
JAX code review questions
When reviewing any JAX function, ask:
- Which values are static?
- Which values are dynamic?
- Is randomness explicit?
- Where is the compile boundary?
- Are any Python decisions being made on traced values?
- Are there host-device conversions in the hot path?
- Could the program be expressed with fewer, larger compiled regions?
If you can answer these clearly, most JAX bugs become much easier to fix.
Performance playbook
Use this file for compile-time blowups, slow steady-state execution, hidden synchronisation, memory pressure, or distributed performance work.
First rule: separate the costs
Never talk about “JAX performance” as one number. Separate:
- host-to-device transfer
- first-call trace and compile
- steady-state device execution
- synchronisation / materialisation
- communication or resharding
- memory pressure / OOM behaviour
Most bad optimisation advice comes from mixing these.
Honest benchmark pattern
Use assets/benchmark_template.py or scripts/jax_benchmark_harness.py.
Checklist:
- warm up first
- block before stopping the timer
- report first-call and steady-state separately
- say whether data transfer is inside or outside the timer
- state backend and key shapes/dtypes
Compile-time problems
Symptoms:
- first call is extremely slow
- every call looks like a first call
make_jaxpris huge- CPU-side time dominates
Check:
- changing shape or dtype
- changing sharding
- static arguments changing every call
- creating new lambdas/partials/jitted functions inside loops
- long Python loops inside
jit - giant captured constants / closures
Tools:
scripts/jax_compile_probe.pyscripts/jax_recompile_explorer.py
Typical fixes:
- stabilise shapes
- hoist
jit - use
scanorfori_loop - make static args explicit and small
- avoid rebuilding objects each call
Steady-state execution problems
Symptoms:
- first call is fine, repeated calls are still slow
- GPU/TPU utilisation is poor
- code is fast in theory but not in practice
Check:
- host round-trips
- tiny compiled kernels separated by Python
- poor batching
- accidental replication or resharding
- heavy callbacks or printing
- slow data pipeline starving the device
Typical fixes:
- larger compiled regions
vmap/scan- keep arrays on device
- explicit sharding
- reduce logging / callbacks in the hot path
Memory problems
Symptoms:
- OOM
- high peak memory
- code only works with tiny batch sizes
Check:
- large intermediates being materialised
- duplication across branches or batches
- unnecessary outputs retained
- no donation where it would help
- sharding causing replication
- activations that could be recomputed
Typical fixes:
- buffer donation
- rematerialisation (
jax.checkpoint) - better sharding
- smaller live ranges
- structured loops instead of unrolled Python
Do not suggest donation or remat automatically; tie them to evidence.
Donation
Donation is useful when:
- an input buffer is dead after the call
- the output can reuse its storage
- memory pressure is real
Donation is not a magic speed-up knob. Use it primarily for memory and only after correctness and API semantics are clear.
Persistent compilation cache
Consider it when:
- the same program is compiled repeatedly across runs
- compile time is a real user pain point
- the environment is stable enough for cache reuse to matter
This is especially relevant for development loops and repeated workloads, not as a first response to every slowdown.
Profiling strategy
Use a profiler before deep optimisation when:
- the user wants serious speed work
- compile time is not obviously the whole story
- memory or communication may dominate
Start with:
- trace collection (
assets/profile_template.py) - device memory profiling for OOM or leaks
- lowering inspection if compile structure seems wrong
Performance review questions
Ask these in order:
1. Is the code timing dispatch or actual execution? 2. Is the slow path compile, execute, transfer, or communication? 3. Are shapes/dtypes/shardings stable? 4. Is there a Python loop or callback in the hot path? 5. Is the data already on device? 6. Is sharding aligned with the algorithm? 7. Is the memory footprint forcing a bad design?
“Fast JAX code” defaults
These defaults are often right:
- compile coarse-grained steps, not every small helper
- batch independent work with
vmap - express long loops with
scan - keep hot arrays on device
- measure with blocking
- reduce shape churn
- prefer global-view sharding before manual per-device code
- use profiler traces rather than intuition for serious tuning
What to report back to the user
When performance is discussed, try to report:
- backend and device count
- input shapes and dtypes
- first-call time
- repeated-call summary
- what was inside the timer
- the most likely remaining bottleneck
That level of honesty is more useful than a vague “should be faster”.
Porting patterns
Use this file when rewriting NumPy, SciPy, TensorFlow, or PyTorch-style code into idiomatic JAX.
The safe order
1. make it correct in eager jax.numpy 2. remove mutation and hidden state 3. make randomness explicit 4. add jit 5. add batching or loop primitives 6. add sharding only after single-device correctness is clear
Pattern: mutation to functional updates
NumPy / PyTorch style:
x[i] += yJAX style:
x = x.at[i].add(y)If the update pattern is large and performance-sensitive, re-think the algorithm rather than translating mutation mechanically.
Pattern: module state to pytrees
Object-heavy code often wants to become a pytree:
- parameters
- optimiser state
- model buffers
- recurrent carry
- RNG state
Avoid burying arrays inside objects that are mutated in place.
Pattern: global RNG to explicit keys
Bad:
GLOBAL_KEY = jax.random.key(0)Better:
def step(state, key, batch):
key, subkey = jax.random.split(key)
...
return new_state, keyIf code is legacy and expects PRNGKey, preserve compatibility, but prefer typed keys in new code.
Pattern: Python data-dependent branch to JAX control flow
Bad:
@jax.jit
def f(x):
if x.sum() > 0:
return x
return -xBetter:
@jax.jit
def f(x):
return jax.lax.cond(x.sum() > 0, lambda y: y, lambda y: -y, x)For elementwise choice, prefer jnp.where.
Pattern: long Python loop to scan
Bad:
@jax.jit
def run(state, xs):
for x in xs:
state = step(state, x)
return stateBetter:
@jax.jit
def run(state, xs):
state, ys = jax.lax.scan(step, state, xs)
return state, ysThis often improves compile time dramatically.
Pattern: per-example Python loop to vmap
Bad:
outs = [f(x) for x in batch]Better:
outs = jax.vmap(f)(batch)Pattern: host logging in the hot path
Bad:
@jax.jit
def step(...):
print(loss)Better:
- use
jax.debug.printwhen debugging - aggregate metrics and log outside the hot path in production
Pattern: boolean masking that changes shape
Bad inside jit:
x = x[x > 0]Better:
- keep the full tensor and mask with
where - or pad to fixed shape and track a validity mask
Pattern: NumPy helpers inside transformed code
Bad:
np.asarray(x)
float(x)
len(x)Better:
- stay in
jax.numpy - use array ops for shape-aware logic
- keep Python-only work outside transforms
Pattern: training step design
Strong default:
- params and opt state are pytrees
- batch is a pytree
- key is explicit
- one
value_and_gradcall inside onejit - metrics are returned as aux
- donation is considered only after correctness and measurement
Pattern: shape-polymorphic ambitions
Before trying shape polymorphism or export, first ask:
- can I bucket or pad shapes?
- can I separate compile-time configuration from runtime values?
- is the real issue a compile storm caused by accidental shape churn?
Often a simpler fixed-shape design beats a complicated polymorphic one.
Porting checklist
Before calling the port “done”, verify:
- no hidden mutation
- no global RNG dependence
- no host round-trips in the hot path
- data-dependent loops/branches use JAX primitives
- tests compare with tolerances rather than exact floating-point equality
- timings exclude first-call compilation unless compile time is part of the question
Repository map
This file assumes a local checkout similar to the provided jax-main.zip snapshot.
High-value documentation paths
Start with docs before implementation.
docs/installation.mddocs/benchmarking.mddocs/async_dispatch.rstdocs/control-flow.mddocs/random-numbers.mddocs/debugging.mddocs/debugging/checkify_guide.mddocs/debugging/print_breakpoint.mddocs/device_memory_profiling.mddocs/profiling.mddocs/persistent_compilation_cache.mddocs/buffer_donation.mddocs/sharded-computation.mddocs/notebooks/explicit-sharding.mddocs/migrate_pmap.mddocs/aot.mddocs/export/export.mddocs/export/shape_poly.mddocs/gpu_performance_tips.mddocs/faq.rstdocs/changelog.mddocs/notebooks/shard_map.mddocs/pallas/anddocs/jax.experimental.pallas*.rst
High-value source modules
These are the first places to inspect in the Python sources:
jax/_src/api.pyjax/_src/api_util.pyjax/_src/array.pyjax/_src/random.pyjax/_src/debugging.pyjax/_src/checkify.pyjax/_src/errors.pyjax/_src/custom_derivatives.pyjax/_src/ad_checkpoint.pyjax/_src/stages.pyjax/_src/export/jax/_src/pjit.pyjax/_src/sharding.pyjax/_src/sharding_impls.pyjax/_src/mesh.pyjax/_src/mesh_utils.pyjax/_src/pallas/
Tests worth checking early
Tests are often the fastest truth source for current behaviour.
tests/api_test.pytests/errors_test.pytests/lax_control_flow_test.pytests/random_test.pytests/checkify_test.pytests/debugging_primitives_test.pytests/pjit_test.pytests/pmap_test.pytests/shard_map_test.pytests/profiler_test.pytests/export_test.py
Search recipes
Control-flow and tracer errors:
rg "ConcretizationTypeError|TracerBoolConversionError|NonConcreteBooleanIndexError" docs tests jaxRandomness:
rg "random.key|PRNGKey|key reuse|fold_in|split" docs tests jaxSharding and migration:
rg "NamedSharding|PartitionSpec|Mesh|shard_map|pmap|migrate_pmap" docs tests jaxDebugging and profiling:
rg "debug.print|checkify|block_until_ready|profiler|compiler_ir" docs tests jaxCustom autodiff / export:
rg "custom_vjp|custom_jvp|export|ShapeDtypeStruct|shape polymorphism" docs tests jaxPallas:
rg "pallas|mosaic|triton" docs tests jaxRecommended protocol for repo questions
1. read the relevant docs page 2. inspect the nearest _src implementation 3. confirm with a nearby test 4. only then make a behaviour claim
If docs and code look out of sync, prefer current tests plus changelog or migration docs.
Sharding playbook
Use this file for multi-device, multi-host, NamedSharding, PartitionSpec, Mesh, shard_map, or pmap migration work.
Start with the three modes
Think about JAX parallelism in this order:
1. Automatic parallelism with jit
You write global-view code for one logical array. The compiler chooses a partitioned execution strategy.
Good when:
- you want the simplest high-level design
- the compiler can infer a good strategy
- you do not need explicit collectives
2. Explicit sharding
You still write global-view code, but sharding becomes part of the array/type-level story. This is the right level for many serious multi-device programs.
Good when:
- placement matters
- you want predictable data layout
- you want the compiler constrained by your sharding choices
3. Manual parallelism with shard_map
You write per-device code and explicit collectives.
Good when:
- you need manual SPMD control
- compiler-driven partitioning is not enough
- you need exact collective semantics inside the mapped function
Default design sequence
1. make single-device code correct 2. state the logical global shape 3. define the mesh 4. define PartitionSpec 5. construct NamedSharding 6. place inputs explicitly if needed 7. benchmark and inspect resharding 8. only then consider shard_map
Core abstractions
Mesh
A mesh names device axes. The names matter because they are how you express sharding and collectives.
PartitionSpec
Maps logical array axes onto mesh axes. Mentioning a mesh axis means sharding along that array dimension; omitting it means replication along that mesh axis.
NamedSharding
Combines a mesh and a PartitionSpec into an explicit placement object.
Review questions for sharded code
- What is the logical global array shape?
- Which array axes are sharded?
- Over which mesh axes?
- Which values are replicated?
- Are collectives using the correct axis names?
- Are local vs global semantics explicit?
Common mistakes
Accidental resharding
Symptoms:
- unnecessary communication
- unexpected slowdown
- outputs or intermediates moving between layouts
Fix:
- inspect input and output shardings
- make placement explicit
- align adjacent computations on compatible shardings
Confusing local and global data
In multi-host code, remember:
- local addressable shards are not the full global array
- indexing a global array can trigger unexpected movement or resharding
- when you truly want local data, use local-shard APIs intentionally
Rank-reduction confusion in pmap
Legacy pmap habits can lead to wrong assumptions during migration. Modern sharding APIs are more explicit and usually easier to reason about.
pmap migration stance
For new code:
- start with modern sharding APIs
- use
pmaponly if compatibility or migration cost strongly argues for it
For existing code:
- preserve semantics first
- migrate incrementally
- check:
- implicit mapped axis assumptions
- collectives
- local vs global views
- donation behaviour
- indexing semantics
Multi-host checklist
When jax.process_count() > 1, make these explicit:
- process-local devices vs global device set
- which data is loaded on which host
- whether a value is local, global, or replicated
- whether all processes execute the same collectives in the same order
Memory and host offload notes
Sharding interacts with memory. If the task involves host offloading or memory kinds:
- be explicit about placement
- do not hide data movement
- confirm whether the problem is true device memory pressure or accidental replication
Practical advice
If the user asks for “make it multi-GPU” and the current code is still impure or shape-unstable, do not jump to sharding. Fix the single-device design first. Distributed JAX amplifies weak assumptions.
Sources and maintenance notes
This skill was rebuilt from two inputs:
1. the provided agent-skill authoring guides 2. current JAX documentation plus the provided jax-main.zip source snapshot
JAX topics explicitly refreshed for this version
- installation and platform guidance
- asynchronous dispatch and honest benchmarking
- typed PRNG keys and key-reuse considerations
- control-flow primitives and
scan/fori_loop - modern sharding APIs and
pmapmigration - export / serialisation and AOT lowering
- profiling and memory-tooling guidance
- Pallas and advanced extension points
Maintenance checklist
When updating the skill for a newer JAX release:
1. re-check:
docs/changelog.mddocs/installation.mddocs/random-numbers.mddocs/debugging.mddocs/benchmarking.mddocs/sharded-computation.mddocs/migrate_pmap.mddocs/export/export.mddocs/device_memory_profiling.md
2. revisit:
jax/_src/api.pyjax/_src/random.pyjax/_src/debugging.pyjax/_src/pjit.pyjax/_src/sharding.pyjax/_src/pallas/
3. refresh the eval prompts if terminology or recommended APIs shift
Notes for future editors
- keep
SKILL.mdfocused on workflow and escalation logic - push deep detail into the reference files
- prefer scripts that produce structured output and avoid interactive prompts
- keep claims about performance and backend behaviour tied to evidence
- treat
pmap, export, and Pallas guidance as likely to drift over time
Transform decision matrix
Use this file when choosing the right primitive or transformation.
First choose the base representation
Start with a pure function over arrays and pytrees. Do not add transformations until the eager version is conceptually clean.
Choose the transform by intent
jax.jit
Use when:
- the same computation will run repeatedly
- Python overhead matters
- shapes/dtypes/shardings are reasonably stable
Do not use first when:
- the function still contains hidden side-effects
- the data-dependent control flow has not been rewritten
- the user is still trying to understand basic correctness
jax.grad / jax.value_and_grad
Use when:
- the function returns a scalar loss
- reverse-mode is appropriate
- you want end-to-end training-step compilation
Prefer value_and_grad(..., has_aux=True) when the forward pass should also return metrics or updated non-differentiated state.
jax.vmap
Use when:
- the computation is the same for many independent examples
- you currently have a Python loop over batch elements
- you want batched Jacobians / per-example gradients
Red flags:
- the body is not truly independent across the batched axis
- memory blows up because batching the whole computation is too large
lax.scan
Use when:
- you have many iterations with a fixed-shape carry
- the loop body is uniform
- compile time or jaxpr size is large because of Python unrolling
Typical wins:
- RNNs
- time stepping
- optimisation loops
- sequential simulation
lax.fori_loop
Use when:
- you want a counted loop primitive
- bounds are known or simple
- you do not need to materialise all intermediate outputs as with
scan
Important detail:
- static trip counts can lower to
scan, which improves reverse-mode support - dynamic trip counts behave more like
while_loop
lax.while_loop
Use when:
- the number of iterations is data-dependent
- you need loop semantics inside compiled code
- the carry has a fixed pytree structure and fixed leaf shapes/dtypes
lax.cond / lax.switch
Use when:
- the branch depends on an array value
- Python branching would try to concretise a tracer
jnp.where
Use when:
- the choice is elementwise
- both branches are arrays with compatible shapes
jax.checkpoint / jax.remat
Use when:
- peak memory is the bottleneck
- recomputation is cheaper than storing intermediates
Do not assume this is a free win. Measure wall time and memory after applying it.
jit + scan
This is the default solution for “I have a long compiled loop”. Prefer:
@jax.jit
def run(carry, xs):
return jax.lax.scan(step, carry, xs)over:
@jax.jit
def run(...):
for ...:
...jit + value_and_grad
This is the default solution for “I need a fast training step”. Put the whole step under one compilation boundary when shapes are stable.
jit + vmap
This is the default solution for “same computation over a batch”. Usually prefer:
fast_batched = jax.jit(jax.vmap(fn))or compile the full caller that contains the vmap.
shard_map
Use when:
- you need explicit per-device code
- you need explicit collectives
- automatic / explicit sharding in global-view code is not enough
Think of it as the manual-transmission option. Powerful, but more demanding.
pmap
Use when:
- the codebase already uses it heavily and migration churn would be high
- you need compatibility with existing patterns
For new work, prefer modern sharding APIs unless there is a strong reason not to.
jax.export / AOT lowering
Use when:
- staging and lowering must be separated from execution
- you need a serialisable compiled representation
- you need shape-polymorphic exported programs
custom_jvp / custom_vjp
Use when:
- default autodiff is wrong, unstable, or too expensive
- you need to hide a numerically stabilised forward pass behind a custom derivative
Quick decision rules
- Independent examples?
vmap - Long sequential loop with fixed carry?
scan - Data-dependent branch?
cond - Data-dependent loop count?
while_loop - Scalar loss gradient?
value_and_grad - Memory issue, compute is cheap?
checkpoint - Multi-device, compiler should decide?
jitwith sharding - Multi-device, you want explicit collectives?
shard_map - Need serialisation / offline compile?
export
Anti-pattern replacements
- Python loop in
jit->scan/fori_loop - Python
ifon array ->cond/where - Global RNG -> explicit key threading
- Small jitted functions created in a loop -> hoist
jit - Huge per-example Python loop ->
vmap pmapfor new global-view code -> modern sharding APIs first
Questions to answer before picking a primitive
- Is the iteration count static or dynamic?
- Are the batched computations independent?
- Is the carry shape fixed?
- Does the branch depend on array data?
- Do I need outputs from every step or only the final state?
- Is this about code clarity, compile time, runtime, memory, or distributed semantics?
Getting those right is usually more important than the specific primitive name.
\
#!/usr/bin/env python3
"""Benchmark a Python callable with optional JAX JIT and proper blocking."""
from __future__ import annotations
import argparse
import importlib
import importlib.util
import json
import statistics
import sys
import time
from pathlib import Path
from typing import Any
def load_module(module_name: str | None, file_path: str | None) -> Any:
if bool(module_name) == bool(file_path):
raise ValueError("Exactly one of --module or --file is required.")
if module_name:
return importlib.import_module(module_name)
path = Path(file_path or "")
if not path.exists():
raise FileNotFoundError(f"Module file not found: {path}")
spec = importlib.util.spec_from_file_location(path.stem, path)
if spec is None or spec.loader is None:
raise ImportError(f"Could not load module spec from: {path}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def resolve_attr(obj: Any, dotted: str) -> Any:
current = obj
for part in dotted.split("."):
current = getattr(current, part)
return current
def load_json_arg(raw: str | None, file_path: str | None, default: Any) -> Any:
if raw is not None and file_path is not None:
raise ValueError("Choose either the inline JSON form or the file form, not both.")
if raw is not None:
return json.loads(raw)
if file_path is not None:
return json.loads(Path(file_path).read_text(encoding="utf-8"))
return default
def numeric_tree(value: Any) -> bool:
if isinstance(value, (int, float, bool)):
return True
if isinstance(value, list):
return all(numeric_tree(v) for v in value)
return False
def maybe_import_jax():
try:
jax = importlib.import_module("jax")
jnp = importlib.import_module("jax.numpy")
return jax, jnp
except Exception:
return None, None
def tree_arrayify(value: Any, jnp_module: Any) -> Any:
if isinstance(value, dict):
return {k: tree_arrayify(v, jnp_module) for k, v in value.items()}
if isinstance(value, list):
if numeric_tree(value):
return jnp_module.array(value)
return [tree_arrayify(v, jnp_module) for v in value]
return value
def tree_device_put(value: Any, jax_module: Any) -> Any:
if isinstance(value, dict):
return {k: tree_device_put(v, jax_module) for k, v in value.items()}
if isinstance(value, list):
return [tree_device_put(v, jax_module) for v in value]
try:
return jax_module.device_put(value)
except Exception:
return value
def parse_int_tuple(raw: str | None) -> tuple[int, ...] | None:
if raw is None or raw == "":
return None
return tuple(int(part.strip()) for part in raw.split(",") if part.strip())
def block_until_ready(value: Any, jax_module: Any | None) -> Any:
if hasattr(value, "block_until_ready"):
return value.block_until_ready()
if jax_module is not None:
try:
return jax_module.block_until_ready(value)
except Exception:
pass
return value
def run_once(fn: Any, args: list[Any], kwargs: dict[str, Any], jax_module: Any | None) -> Any:
out = fn(*args, **kwargs)
block_until_ready(out, jax_module)
return out
def summary(times_ms: list[float]) -> dict[str, float]:
return {
"mean_ms": statistics.mean(times_ms),
"median_ms": statistics.median(times_ms),
"min_ms": min(times_ms),
"max_ms": max(times_ms),
"stdev_ms": statistics.pstdev(times_ms) if len(times_ms) > 1 else 0.0,
}
def benchmark(fn: Any, args: list[Any], kwargs: dict[str, Any], repeat: int, jax_module: Any | None) -> list[float]:
times_ms = []
for _ in range(repeat):
t0 = time.perf_counter()
run_once(fn, args, kwargs, jax_module)
times_ms.append((time.perf_counter() - t0) * 1e3)
return times_ms
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Benchmark a Python callable with optional JAX JIT and proper blocking.",
formatter_class=argparse.RawTextHelpFormatter,
epilog="""Exit codes:
0 success
2 operational error
Examples:
python3 scripts/jax_benchmark_harness.py --file evals/files/naive_benchmark.py --function matmul_step \\
--args-json '[[[1.0, 2.0], [3.0, 4.0]], [[1.0], [2.0]]]' --arrayify --jit --compare-eager
python3 scripts/jax_benchmark_harness.py --module mypkg.train --function step \\
--args-file args.json --kwargs-file kwargs.json --jit --repeat 20
python3 scripts/jax_benchmark_harness.py --file train.py --function step \\
--args-file args.json --jit --static-argnums 2 --donate-argnums 0,1
""",
)
source = parser.add_mutually_exclusive_group(required=True)
source.add_argument("--module", help="Import path for the module containing the callable.")
source.add_argument("--file", help="Path to a Python file containing the callable.")
parser.add_argument("--function", required=True, help="Callable name or dotted attribute path.")
parser.add_argument("--args-json", help="JSON list of positional arguments.")
parser.add_argument("--args-file", help="Path to a JSON file containing positional arguments.")
parser.add_argument("--kwargs-json", help="JSON object of keyword arguments.")
parser.add_argument("--kwargs-file", help="Path to a JSON file containing keyword arguments.")
parser.add_argument("--arrayify", action="store_true", help="Convert numeric JSON lists to `jax.numpy.array` when JAX is available.")
parser.add_argument("--device-put", action="store_true", help="Apply `jax.device_put` to arguments before timing when JAX is available.")
parser.add_argument("--jit", action="store_true", help="Wrap the callable with `jax.jit`.")
parser.add_argument("--static-argnums", help="Comma-separated positional indices to mark static when using --jit.")
parser.add_argument("--donate-argnums", help="Comma-separated positional indices to donate when using --jit.")
parser.add_argument("--compare-eager", action="store_true", help="Also benchmark the original eager callable.")
parser.add_argument("--repeat", type=int, default=10, help="Number of timed steady-state repetitions. Default: 10")
parser.add_argument("--warmup", type=int, default=1, help="Warm-up calls before timed loops. Default: 1")
parser.add_argument("--format", choices=("json", "text"), default="json", help="Output format. Default: json")
parser.add_argument("--output", help="Write the report to this file instead of stdout.")
return parser.parse_args()
def main() -> int:
args = parse_args()
try:
module = load_module(args.module, args.file)
fn = resolve_attr(module, args.function)
if not callable(fn):
raise TypeError(f"Resolved object is not callable: {args.function}")
raw_args = load_json_arg(args.args_json, args.args_file, [])
raw_kwargs = load_json_arg(args.kwargs_json, args.kwargs_file, {})
if not isinstance(raw_args, list):
raise TypeError("Positional arguments JSON must decode to a list.")
if not isinstance(raw_kwargs, dict):
raise TypeError("Keyword arguments JSON must decode to an object.")
jax, jnp = maybe_import_jax()
proc_args = list(raw_args)
proc_kwargs = dict(raw_kwargs)
if args.arrayify:
if jax is None or jnp is None:
raise RuntimeError("--arrayify requires JAX to be importable.")
proc_args = [tree_arrayify(v, jnp) for v in proc_args]
proc_kwargs = {k: tree_arrayify(v, jnp) for k, v in proc_kwargs.items()}
if args.device_put:
if jax is None:
raise RuntimeError("--device-put requires JAX to be importable.")
proc_args = [tree_device_put(v, jax) for v in proc_args]
proc_kwargs = {k: tree_device_put(v, jax) for k, v in proc_kwargs.items()}
static_argnums = parse_int_tuple(args.static_argnums)
donate_argnums = parse_int_tuple(args.donate_argnums)
report: dict[str, Any] = {
"source": args.module or args.file,
"callable": args.function,
"repeat": args.repeat,
"warmup": args.warmup,
"jax_available": jax is not None,
"jit_requested": args.jit,
"compare_eager": args.compare_eager,
"static_argnums": static_argnums,
"donate_argnums": donate_argnums,
}
if args.compare_eager or not args.jit:
for _ in range(args.warmup):
run_once(fn, proc_args, proc_kwargs, jax)
eager_times = benchmark(fn, proc_args, proc_kwargs, args.repeat, jax)
report["eager"] = {
"times_ms": eager_times,
"summary": summary(eager_times),
}
if args.jit:
if jax is None:
raise RuntimeError("--jit requires JAX to be importable.")
jit_kwargs = {}
if static_argnums is not None:
jit_kwargs["static_argnums"] = static_argnums
if donate_argnums is not None:
jit_kwargs["donate_argnums"] = donate_argnums
fn_jit = jax.jit(fn, **jit_kwargs)
t0 = time.perf_counter()
run_once(fn_jit, proc_args, proc_kwargs, jax)
first_call_ms = (time.perf_counter() - t0) * 1e3
for _ in range(max(args.warmup - 1, 0)):
run_once(fn_jit, proc_args, proc_kwargs, jax)
jit_times = benchmark(fn_jit, proc_args, proc_kwargs, args.repeat, jax)
report["jit"] = {
"first_call_ms": first_call_ms,
"times_ms": jit_times,
"summary": summary(jit_times),
}
if "eager" in report and "jit" in report:
eager_mean = report["eager"]["summary"]["mean_ms"]
jit_mean = report["jit"]["summary"]["mean_ms"]
report["speedup_vs_eager_mean"] = (eager_mean / jit_mean) if jit_mean else None
except Exception as exc:
sys.stderr.write(f"Error: {type(exc).__name__}: {exc}\n")
return 2
if args.format == "json":
text = json.dumps(report, indent=2, sort_keys=True)
else:
lines = [
f"Callable: {report['source']}::{report['callable']}",
f"Repeat: {report['repeat']}",
f"Warmup: {report['warmup']}",
f"JAX available: {report['jax_available']}",
f"JIT requested: {report['jit_requested']}",
f"Static argnums: {report['static_argnums']}",
f"Donate argnums: {report['donate_argnums']}",
"",
]
if "eager" in report:
lines.append("Eager")
for key, value in report["eager"]["summary"].items():
lines.append(f" {key}: {value:.3f}")
lines.append("")
if "jit" in report:
lines.append("JIT")
lines.append(f" first_call_ms: {report['jit']['first_call_ms']:.3f}")
for key, value in report["jit"]["summary"].items():
lines.append(f" {key}: {value:.3f}")
lines.append("")
if "speedup_vs_eager_mean" in report:
lines.append(f"speedup_vs_eager_mean: {report['speedup_vs_eager_mean']:.3f}")
text = "\n".join(lines)
if args.output:
Path(args.output).write_text(text + ("" if text.endswith("\n") else "\n"), encoding="utf-8")
else:
sys.stdout.write(text)
if not text.endswith("\n"):
sys.stdout.write("\n")
return 0
if __name__ == "__main__":
raise SystemExit(main())
\
#!/usr/bin/env python3
"""Inspect JAX tracing/lowering artefacts for a callable."""
from __future__ import annotations
import argparse
import hashlib
import importlib
import importlib.util
import json
import sys
from pathlib import Path
from typing import Any
def load_module(module_name: str | None, file_path: str | None) -> Any:
if bool(module_name) == bool(file_path):
raise ValueError("Exactly one of --module or --file is required.")
if module_name:
return importlib.import_module(module_name)
path = Path(file_path or "")
if not path.exists():
raise FileNotFoundError(f"Module file not found: {path}")
spec = importlib.util.spec_from_file_location(path.stem, path)
if spec is None or spec.loader is None:
raise ImportError(f"Could not load module spec from: {path}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def resolve_attr(obj: Any, dotted: str) -> Any:
current = obj
for part in dotted.split("."):
current = getattr(current, part)
return current
def load_json_arg(raw: str | None, file_path: str | None, default: Any) -> Any:
if raw is not None and file_path is not None:
raise ValueError("Choose either the inline JSON form or the file form, not both.")
if raw is not None:
return json.loads(raw)
if file_path is not None:
return json.loads(Path(file_path).read_text(encoding="utf-8"))
return default
def numeric_tree(value: Any) -> bool:
if isinstance(value, (int, float, bool)):
return True
if isinstance(value, list):
return all(numeric_tree(v) for v in value)
return False
def maybe_import_jax():
try:
jax = importlib.import_module("jax")
jnp = importlib.import_module("jax.numpy")
return jax, jnp
except Exception:
return None, None
def tree_arrayify(value: Any, jnp_module: Any) -> Any:
if isinstance(value, dict):
return {k: tree_arrayify(v, jnp_module) for k, v in value.items()}
if isinstance(value, list):
if numeric_tree(value):
return jnp_module.array(value)
return [tree_arrayify(v, jnp_module) for v in value]
return value
def parse_int_tuple(raw: str | None) -> tuple[int, ...] | None:
if raw is None or raw == "":
return None
return tuple(int(part.strip()) for part in raw.split(",") if part.strip())
def sha256_text(text: str | None) -> str | None:
if text is None:
return None
return hashlib.sha256(text.encode("utf-8")).hexdigest()
def to_text(obj: Any) -> str:
if obj is None:
return ""
if isinstance(obj, str):
return obj
for getter in (
lambda: obj.as_text(),
lambda: obj.operation.get_asm(enable_debug_info=True),
lambda: obj.operation.get_asm(),
):
try:
return getter()
except Exception:
pass
return str(obj)
def preview_text(text: str, max_lines: int, max_chars: int) -> str:
lines = text.splitlines()
clipped = "\n".join(lines[:max_lines])
if len(clipped) > max_chars:
clipped = clipped[:max_chars]
return clipped
def write_artifact(path: Path, text: str) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(text, encoding="utf-8")
def safe_repr(value: Any) -> str:
try:
return repr(value)
except Exception as exc: # pragma: no cover - best effort only
return f"<repr failed: {type(exc).__name__}: {exc}>"
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Inspect `eval_shape`, jaxpr, lowering, and compiler IR for a callable.",
formatter_class=argparse.RawTextHelpFormatter,
epilog="""Exit codes:
0 success
2 operational error
Examples:
python3 scripts/jax_compile_probe.py --file my_module.py --function step --args-file args.json --arrayify
python3 scripts/jax_compile_probe.py --module pkg.train --function step --args-file args.json --jit --write-dir probe_out
python3 scripts/jax_compile_probe.py --file my_module.py --function fn --args-json '[[1, 2, 3]]' --arrayify --dialects stablehlo,hlo
""",
)
source = parser.add_mutually_exclusive_group(required=True)
source.add_argument("--module", help="Import path for the module containing the callable.")
source.add_argument("--file", help="Path to a Python file containing the callable.")
parser.add_argument("--function", required=True, help="Callable name or dotted attribute path.")
parser.add_argument("--args-json", help="JSON list of positional arguments.")
parser.add_argument("--args-file", help="Path to a JSON file containing positional arguments.")
parser.add_argument("--kwargs-json", help="JSON object of keyword arguments.")
parser.add_argument("--kwargs-file", help="Path to a JSON file containing keyword arguments.")
parser.add_argument("--arrayify", action="store_true", help="Convert numeric JSON lists to `jax.numpy.array` when JAX is available.")
parser.add_argument("--jit", action="store_true", help="Wrap the callable with `jax.jit` before lowering.")
parser.add_argument("--static-argnums", help="Comma-separated positional indices to mark static when using --jit.")
parser.add_argument("--donate-argnums", help="Comma-separated positional indices to donate when using --jit.")
parser.add_argument("--compile", action="store_true", help="Compile the lowered program and report compile timing when possible.")
parser.add_argument("--dialects", default="stablehlo", help="Comma-separated compiler IR dialects to attempt. Default: stablehlo")
parser.add_argument("--max-preview-lines", type=int, default=80, help="Maximum preview lines per artefact. Default: 80")
parser.add_argument("--max-preview-chars", type=int, default=12000, help="Maximum preview characters per artefact. Default: 12000")
parser.add_argument("--write-dir", help="Optional directory for full artefacts such as jaxpr and IR text files.")
parser.add_argument("--format", choices=("json", "text"), default="json", help="Output format. Default: json")
parser.add_argument("--output", help="Write the report to a file instead of stdout.")
return parser.parse_args()
def main() -> int:
args = parse_args()
try:
jax, jnp = maybe_import_jax()
if jax is None or jnp is None:
raise RuntimeError("JAX is not importable in this environment.")
module = load_module(args.module, args.file)
fn = resolve_attr(module, args.function)
if not callable(fn):
raise TypeError(f"Resolved object is not callable: {args.function}")
raw_args = load_json_arg(args.args_json, args.args_file, [])
raw_kwargs = load_json_arg(args.kwargs_json, args.kwargs_file, {})
if not isinstance(raw_args, list):
raise TypeError("Positional arguments JSON must decode to a list.")
if not isinstance(raw_kwargs, dict):
raise TypeError("Keyword arguments JSON must decode to an object.")
proc_args = list(raw_args)
proc_kwargs = dict(raw_kwargs)
if args.arrayify:
proc_args = [tree_arrayify(v, jnp) for v in proc_args]
proc_kwargs = {k: tree_arrayify(v, jnp) for k, v in proc_kwargs.items()}
static_argnums = parse_int_tuple(args.static_argnums)
donate_argnums = parse_int_tuple(args.donate_argnums)
report: dict[str, Any] = {
"source": args.module or args.file,
"callable": args.function,
"jit_requested": args.jit,
"static_argnums": static_argnums,
"donate_argnums": donate_argnums,
"dialects_requested": [d.strip() for d in args.dialects.split(",") if d.strip()],
}
try:
eval_shape = jax.eval_shape(fn, *proc_args, **proc_kwargs)
report["eval_shape_repr"] = safe_repr(eval_shape)
except Exception as exc:
report["eval_shape_error"] = f"{type(exc).__name__}: {exc}"
try:
jaxpr_obj = jax.make_jaxpr(fn)(*proc_args, **proc_kwargs)
jaxpr_text = str(jaxpr_obj)
report["jaxpr"] = {
"lines": len(jaxpr_text.splitlines()),
"sha256": sha256_text(jaxpr_text),
"preview": preview_text(jaxpr_text, args.max_preview_lines, args.max_preview_chars),
}
if args.write_dir:
write_artifact(Path(args.write_dir) / "jaxpr.txt", jaxpr_text)
except Exception as exc:
report["jaxpr_error"] = f"{type(exc).__name__}: {exc}"
lowered_fn = fn
if args.jit:
jit_kwargs = {}
if static_argnums is not None:
jit_kwargs["static_argnums"] = static_argnums
if donate_argnums is not None:
jit_kwargs["donate_argnums"] = donate_argnums
lowered_fn = jax.jit(fn, **jit_kwargs)
try:
if not hasattr(lowered_fn, "lower"):
raise TypeError("Target does not support `.lower(...)`; try using --jit.")
lowered = lowered_fn.lower(*proc_args, **proc_kwargs)
lowered_text = to_text(lowered)
report["lowering"] = {
"sha256": sha256_text(lowered_text),
"preview": preview_text(lowered_text, args.max_preview_lines, args.max_preview_chars),
}
if args.write_dir:
write_artifact(Path(args.write_dir) / "lowering.txt", lowered_text)
ir_reports = {}
for dialect in report["dialects_requested"]:
try:
ir_obj = lowered.compiler_ir(dialect=dialect)
ir_text = to_text(ir_obj)
ir_reports[dialect] = {
"sha256": sha256_text(ir_text),
"preview": preview_text(ir_text, args.max_preview_lines, args.max_preview_chars),
}
if args.write_dir:
suffix = "mlir" if dialect in {"stablehlo", "mhlo"} else "txt"
write_artifact(Path(args.write_dir) / f"{dialect}.{suffix}", ir_text)
except Exception as exc:
ir_reports[dialect] = {"error": f"{type(exc).__name__}: {exc}"}
report["compiler_ir"] = ir_reports
if args.compile:
import time
t0 = time.perf_counter()
compiled = lowered.compile()
compile_ms = (time.perf_counter() - t0) * 1e3
report["compile"] = {
"ok": compiled is not None,
"elapsed_ms": compile_ms,
}
except Exception as exc:
report["lowering_error"] = f"{type(exc).__name__}: {exc}"
except Exception as exc:
sys.stderr.write(f"Error: {type(exc).__name__}: {exc}\n")
return 2
if args.format == "json":
text = json.dumps(report, indent=2, sort_keys=True)
else:
lines = [
f"Callable: {report['source']}::{report['callable']}",
f"JIT requested: {report['jit_requested']}",
f"Static argnums: {report['static_argnums']}",
f"Donate argnums: {report['donate_argnums']}",
"",
]
if "eval_shape_repr" in report:
lines.append("eval_shape:")
lines.append(report["eval_shape_repr"])
lines.append("")
if "eval_shape_error" in report:
lines.append(f"eval_shape_error: {report['eval_shape_error']}")
lines.append("")
if "jaxpr" in report:
lines.append(f"jaxpr lines: {report['jaxpr']['lines']}")
lines.append(report["jaxpr"]["preview"])
lines.append("")
if "lowering" in report:
lines.append("lowering preview:")
lines.append(report["lowering"]["preview"])
lines.append("")
if "compiler_ir" in report:
for dialect, info in report["compiler_ir"].items():
lines.append(f"{dialect}:")
if "error" in info:
lines.append(f" error: {info['error']}")
else:
lines.append(info["preview"])
lines.append("")
if "compile" in report:
lines.append(f"compile elapsed_ms: {report['compile']['elapsed_ms']:.3f}")
text = "\n".join(lines)
if args.output:
Path(args.output).write_text(text + ("" if text.endswith("\n") else "\n"), encoding="utf-8")
else:
sys.stdout.write(text)
if not text.endswith("\n"):
sys.stdout.write("\n")
return 0
if __name__ == "__main__":
raise SystemExit(main())
\
#!/usr/bin/env python3
"""Emit a structured report about the local JAX environment.
Design goals:
- non-interactive
- JSON by default
- standard-library only
- useful even when JAX is not importable
"""
from __future__ import annotations
import argparse
import importlib
import importlib.metadata
import json
import os
import platform
import sys
import time
from typing import Any
PACKAGE_NAMES = (
"jax",
"jaxlib",
"numpy",
"scipy",
"flax",
"optax",
"equinox",
"orbax-checkpoint",
)
CONFIG_KEYS = (
"jax_enable_x64",
"jax_default_matmul_precision",
"jax_debug_nans",
"jax_debug_infs",
"jax_debug_key_reuse",
"jax_platform_name",
"jax_default_prng_impl",
"jax_transfer_guard",
"jax_compilation_cache_dir",
)
ENV_PREFIXES = (
"JAX_",
"XLA_",
"CUDA_",
"NVIDIA_",
"ROCM",
"HIP_",
"TPU",
"NCCL_",
)
ENV_NAMES = {
"CUDA_VISIBLE_DEVICES",
"NVIDIA_VISIBLE_DEVICES",
"XLA_FLAGS",
"PYTHONPATH",
"LD_LIBRARY_PATH",
"PATH",
}
def package_info(name: str) -> dict[str, Any]:
out: dict[str, Any] = {"installed": False}
try:
out["version"] = importlib.metadata.version(name)
out["installed"] = True
except importlib.metadata.PackageNotFoundError:
out["version"] = None
except Exception as exc: # pragma: no cover - best effort only
out["version"] = None
out["error"] = f"{type(exc).__name__}: {exc}"
return out
def selected_environment() -> dict[str, str]:
env = {}
for key, value in os.environ.items():
if key in ENV_NAMES or any(key.startswith(prefix) for prefix in ENV_PREFIXES):
env[key] = value
return dict(sorted(env.items()))
def read_config_value(jax_module: Any, key: str) -> Any:
cfg = getattr(jax_module, "config", None)
if cfg is None:
return None
# Try the most stable public-ish access patterns first.
for getter in (
lambda: getattr(cfg, "values", {}).get(key),
lambda: cfg.read(key), # type: ignore[attr-defined]
lambda: getattr(cfg, key),
):
try:
value = getter()
if value is not None:
return value
except Exception:
pass
return None
def maybe_key(jax_module: Any, seed: int):
try:
return jax_module.random.key(seed)
except Exception:
return jax_module.random.PRNGKey(seed)
def smoke_test(jax_module: Any, jnp_module: Any) -> dict[str, Any]:
report: dict[str, Any] = {"ok": False}
t0 = time.perf_counter()
key = maybe_key(jax_module, 0)
x = jax_module.random.normal(key, (256, 256), dtype=jnp_module.float32)
y = (x @ x.T).block_until_ready()
@jax_module.jit
def loss_fn(z):
return jnp_module.sum(jnp_module.tanh(z @ z.T))
loss = loss_fn(x)
loss.block_until_ready()
grad = jax_module.grad(lambda z: jnp_module.sum(jnp_module.sin(z)))(x)
jax_module.block_until_ready(grad)
report["ok"] = True
report["elapsed_ms"] = (time.perf_counter() - t0) * 1e3
report["matmul_shape"] = tuple(int(v) for v in y.shape)
report["loss_dtype"] = str(loss.dtype)
return report
def load_jax_report(run_smoke_test: bool) -> dict[str, Any]:
report: dict[str, Any] = {"imported": False}
try:
jax = importlib.import_module("jax")
jnp = importlib.import_module("jax.numpy")
except Exception as exc:
report["import_error"] = f"{type(exc).__name__}: {exc}"
return report
report["imported"] = True
report["version"] = getattr(jax, "__version__", None)
try:
report["default_backend"] = jax.default_backend()
except Exception:
report["default_backend"] = None
try:
report["process_count"] = int(jax.process_count())
report["process_index"] = int(jax.process_index())
report["device_count"] = int(jax.device_count())
report["local_device_count"] = int(jax.local_device_count())
except Exception as exc:
report["process_error"] = f"{type(exc).__name__}: {exc}"
devices = []
try:
for dev in jax.devices():
devices.append(
{
"id": getattr(dev, "id", None),
"platform": getattr(dev, "platform", None),
"device_kind": getattr(dev, "device_kind", None),
"process_index": getattr(dev, "process_index", None),
"memory_limit": getattr(dev, "memory_limit", None),
}
)
except Exception as exc:
report["devices_error"] = f"{type(exc).__name__}: {exc}"
report["devices"] = devices
cfg = {}
for key in CONFIG_KEYS:
cfg[key] = read_config_value(jax, key)
report["config"] = cfg
if run_smoke_test:
try:
report["smoke_test"] = smoke_test(jax, jnp)
except Exception as exc:
report["smoke_test"] = {"ok": False, "error": f"{type(exc).__name__}: {exc}"}
return report
def build_report(run_smoke_test: bool) -> dict[str, Any]:
return {
"python": {
"version": sys.version.split()[0],
"executable": sys.executable,
},
"platform": {
"system": platform.system(),
"release": platform.release(),
"machine": platform.machine(),
"platform": platform.platform(),
},
"packages": {name: package_info(name) for name in PACKAGE_NAMES},
"environment": selected_environment(),
"jax": load_jax_report(run_smoke_test),
}
def format_text(report: dict[str, Any]) -> str:
lines = []
lines.append("Python")
lines.append(f" version: {report['python']['version']}")
lines.append(f" executable: {report['python']['executable']}")
lines.append("")
lines.append("Platform")
for key, value in report["platform"].items():
lines.append(f" {key}: {value}")
lines.append("")
lines.append("Packages")
for name, info in report["packages"].items():
status = info.get("version") if info.get("installed") else "not installed"
lines.append(f" {name}: {status}")
lines.append("")
lines.append("Environment")
for key, value in report["environment"].items():
lines.append(f" {key}={value}")
lines.append("")
jax_info = report["jax"]
lines.append("JAX")
if not jax_info.get("imported"):
lines.append(f" import_failed: {jax_info.get('import_error')}")
return "\n".join(lines)
lines.append(f" version: {jax_info.get('version')}")
lines.append(f" default_backend: {jax_info.get('default_backend')}")
lines.append(f" process_count: {jax_info.get('process_count')}")
lines.append(f" process_index: {jax_info.get('process_index')}")
lines.append(f" device_count: {jax_info.get('device_count')}")
lines.append(f" local_device_count: {jax_info.get('local_device_count')}")
lines.append("")
lines.append("Devices")
for device in jax_info.get("devices", []):
lines.append(
" - id={id} platform={platform} kind={device_kind} process_index={process_index} memory_limit={memory_limit}".format(
**device
)
)
lines.append("")
lines.append("Config")
for key, value in jax_info.get("config", {}).items():
lines.append(f" {key}: {value}")
smoke = jax_info.get("smoke_test")
if smoke:
lines.append("")
lines.append("Smoke test")
for key, value in smoke.items():
lines.append(f" {key}: {value}")
return "\n".join(lines)
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Emit a structured report about the local JAX environment.",
formatter_class=argparse.RawTextHelpFormatter,
epilog="""Exit codes:
0 success
2 operational error
Examples:
python3 scripts/jax_env_report.py
python3 scripts/jax_env_report.py --smoke-test --format text
python3 scripts/jax_env_report.py --output env.json
""",
)
parser.add_argument("--smoke-test", action="store_true", help="Run a small JAX smoke test if JAX is importable.")
parser.add_argument("--format", choices=("json", "text"), default="json", help="Output format. Default: json")
parser.add_argument("--output", help="Write output to a file instead of stdout.")
return parser.parse_args()
def main() -> int:
args = parse_args()
try:
report = build_report(args.smoke_test)
text = json.dumps(report, indent=2, sort_keys=True) if args.format == "json" else format_text(report)
if args.output:
with open(args.output, "w", encoding="utf-8") as f:
f.write(text)
if not text.endswith("\n"):
f.write("\n")
else:
sys.stdout.write(text)
if not text.endswith("\n"):
sys.stdout.write("\n")
return 0
except Exception as exc:
sys.stderr.write(f"Error: {type(exc).__name__}: {exc}\n")
return 2
if __name__ == "__main__":
raise SystemExit(main())
\
#!/usr/bin/env python3
"""Static scanner for common JAX sharp bits.
The scanner is intentionally conservative:
- it reports likely review targets
- it does not claim to prove a bug
- it is useful even without JAX installed
"""
from __future__ import annotations
import argparse
import ast
import json
import sys
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Any
DEFAULT_EXCLUDES = {
".git",
"__pycache__",
".venv",
"venv",
"build",
"dist",
".mypy_cache",
".pytest_cache",
".ruff_cache",
}
@dataclass
class Finding:
file: str
line: int
column: int
severity: str
kind: str
message: str
snippet: str | None
def dotted_name(node: ast.AST | None) -> str | None:
if node is None:
return None
if isinstance(node, ast.Name):
return node.id
if isinstance(node, ast.Attribute):
head = dotted_name(node.value)
return f"{head}.{node.attr}" if head else node.attr
if isinstance(node, ast.Call):
return dotted_name(node.func)
return None
def parse_int_literal(node: ast.AST | None) -> int | None:
if isinstance(node, ast.Constant) and isinstance(node.value, int):
return node.value
return None
class JaxScanner(ast.NodeVisitor):
def __init__(self, filename: Path, source: str):
self.filename = filename
self.lines = source.splitlines()
self.findings: list[Finding] = []
self.loop_depth = 0
self.module_level = True
self.jit_aliases = {"jax.jit", "jit"}
self.transform_aliases = {
"jax.jit",
"jax.pmap",
"jax.vmap",
"jax.shard_map",
"jax.smap",
"jax.experimental.pjit.pjit",
"jit",
"pmap",
"vmap",
"shard_map",
"smap",
"pjit",
}
self.numpy_aliases = {"np", "numpy"}
self.random_key_aliases = {
"jax.random.key",
"jax.random.PRNGKey",
"random.key",
"random.PRNGKey",
"key",
"PRNGKey",
}
self.in_transformed_function_stack: list[str] = []
def add(self, node: ast.AST, severity: str, kind: str, message: str) -> None:
lineno = getattr(node, "lineno", 0)
col = getattr(node, "col_offset", 0)
snippet = None
if 1 <= lineno <= len(self.lines):
snippet = self.lines[lineno - 1].strip()
self.findings.append(
Finding(
file=str(self.filename),
line=lineno,
column=col,
severity=severity,
kind=kind,
message=message,
snippet=snippet,
)
)
def _is_transform_decorator(self, node: ast.AST) -> bool:
name = dotted_name(node)
if name in self.jit_aliases or name in self.transform_aliases:
return True
if isinstance(node, ast.Call):
func_name = dotted_name(node.func)
if func_name in {"functools.partial", "partial"} and node.args:
first = dotted_name(node.args[0])
return first in self.jit_aliases or first in self.transform_aliases
return False
def visit_Import(self, node: ast.Import) -> None:
for alias in node.names:
name = alias.name
asname = alias.asname or name
if name == "jax":
self.jit_aliases.add(f"{asname}.jit")
self.transform_aliases.update(
{f"{asname}.jit", f"{asname}.vmap", f"{asname}.pmap", f"{asname}.shard_map", f"{asname}.smap"}
)
if name == "numpy":
self.numpy_aliases.add(asname)
self.generic_visit(node)
def visit_ImportFrom(self, node: ast.ImportFrom) -> None:
mod = node.module or ""
for alias in node.names:
target = alias.asname or alias.name
full = f"{mod}.{alias.name}" if mod else alias.name
if full == "jax.jit":
self.jit_aliases.add(target)
self.transform_aliases.add(target)
elif full in {
"jax.vmap",
"jax.pmap",
"jax.shard_map",
"jax.smap",
"jax.experimental.pjit.pjit",
}:
self.transform_aliases.add(target)
elif full in {"jax.random.key", "jax.random.PRNGKey"}:
self.random_key_aliases.add(target)
elif full == "numpy":
self.numpy_aliases.add(target)
self.generic_visit(node)
def visit_Assign(self, node: ast.Assign) -> None:
if self.module_level:
rhs = dotted_name(node.value)
if rhs in self.random_key_aliases:
self.add(
node,
"warning",
"global-prng-key",
"Module-level PRNG key detected. Prefer explicit key threading rather than persistent hidden RNG state.",
)
self.generic_visit(node)
def visit_FunctionDef(self, node: ast.FunctionDef) -> None:
transformed = any(self._is_transform_decorator(d) for d in node.decorator_list)
prev_module_level = self.module_level
self.module_level = False
if transformed:
self.in_transformed_function_stack.append(node.name)
self.generic_visit(node)
if transformed:
self.in_transformed_function_stack.pop()
self.module_level = prev_module_level
visit_AsyncFunctionDef = visit_FunctionDef
def visit_If(self, node: ast.If) -> None:
if self.in_transformed_function_stack:
self.add(
node,
"warning",
"python-if-in-transformed-function",
"Python `if` inside a transformed function may fail if the condition depends on traced values. Consider `jax.lax.cond` or `jnp.where`.",
)
self.generic_visit(node)
def visit_For(self, node: ast.For) -> None:
self.loop_depth += 1
if self.in_transformed_function_stack:
msg = "Python `for` loop inside a transformed function will execute at trace time or be unrolled. Consider `lax.scan` or `lax.fori_loop`."
trip_count = None
if isinstance(node.iter, ast.Call) and dotted_name(node.iter.func) == "range":
if node.iter.args:
trip_count = parse_int_literal(node.iter.args[0])
if trip_count is not None and trip_count <= 4:
msg += " This loop is tiny, so it may be acceptable; still review it intentionally."
self.add(node, "info", "python-for-in-transformed-function", msg)
self.generic_visit(node)
self.loop_depth -= 1
def visit_While(self, node: ast.While) -> None:
self.loop_depth += 1
if self.in_transformed_function_stack:
self.add(
node,
"warning",
"python-while-in-transformed-function",
"Python `while` loop inside transformed code often needs `jax.lax.while_loop`.",
)
self.generic_visit(node)
self.loop_depth -= 1
def visit_ListComp(self, node: ast.ListComp) -> None:
if self.in_transformed_function_stack:
self.add(
node,
"info",
"list-comprehension-in-transformed-function",
"List comprehension inside transformed code may signal Python-side work. Review whether this should be array programming or `vmap`.",
)
self.generic_visit(node)
def visit_Call(self, node: ast.Call) -> None:
name = dotted_name(node.func)
if self.loop_depth > 0 and name in self.jit_aliases:
self.add(
node,
"warning",
"jit-created-inside-loop",
"A jitted callable is being created inside a loop. Hoist `jax.jit(...)` out of the loop to avoid repeated tracing and confusing cache behaviour.",
)
if name == "jax.random.PRNGKey" or name == "random.PRNGKey" or name == "PRNGKey":
self.add(
node,
"info",
"legacy-key-api",
"Legacy `PRNGKey` API detected. New code often prefers typed keys via `jax.random.key(...)`, unless compatibility requires legacy keys.",
)
if name == "jax.pmap" or name == "pmap":
self.add(
node,
"info",
"pmap-usage",
"`pmap` detected. Review whether modern sharding APIs would be clearer for new code or major refactors.",
)
if self.in_transformed_function_stack:
if name == "print":
self.add(
node,
"warning",
"print-in-transformed-function",
"Use `jax.debug.print` for traced runtime values. Plain `print` only sees trace-time information.",
)
if name in {"open", "logging.info", "logging.warning", "logging.debug"}:
self.add(
node,
"warning",
"side-effect-in-transformed-function",
"Host-side side effects inside transformed functions are a common source of confusion and tracer leaks.",
)
if name in {"np.asarray", "numpy.asarray", "np.array", "numpy.array"} or any(
name == f"{alias}.asarray" or name == f"{alias}.array" for alias in self.numpy_aliases
):
self.add(
node,
"warning",
"numpy-conversion-in-transformed-function",
"NumPy conversion inside transformed code can force host conversion or fail on tracers. Stay in `jax.numpy` inside the compiled region.",
)
if name in {"jax.device_get", "device_get"}:
self.add(
node,
"warning",
"device-get-in-transformed-function",
"Device-to-host transfer inside transformed code is usually a performance or correctness smell.",
)
if isinstance(node.func, ast.Attribute) and node.func.attr in {"item", "tolist"}:
self.add(
node,
"warning" if self.in_transformed_function_stack else "info",
"host-conversion",
f"Array conversion via `.{node.func.attr}()` may force synchronisation or host conversion. Review whether this belongs outside the hot path.",
)
self.generic_visit(node)
def visit_Attribute(self, node: ast.Attribute) -> None:
name = dotted_name(node)
if name in {"jax.device_get", "jax.device_put", "jax.block_until_ready"}:
# Merely using these is not wrong, but it often matters during review.
self.add(
node,
"info",
"runtime-boundary-api",
f"`{name}` appears in the code. Review whether it represents an intentional device boundary.",
)
self.generic_visit(node)
def iter_python_files(root: Path, excludes: set[str]) -> list[Path]:
if root.is_file():
return [root] if root.suffix == ".py" else []
files = []
for path in root.rglob("*.py"):
if any(part in excludes for part in path.parts):
continue
files.append(path)
return sorted(files)
def scan_file(path: Path) -> tuple[list[Finding], str | None]:
try:
source = path.read_text(encoding="utf-8")
except UnicodeDecodeError:
source = path.read_text(encoding="latin-1")
try:
tree = ast.parse(source, filename=str(path))
except SyntaxError as exc:
return [], f"SyntaxError: {exc}"
scanner = JaxScanner(path, source)
scanner.visit(tree)
return scanner.findings, None
def build_suggestions(findings: list[Finding]) -> list[str]:
kinds = {f.kind for f in findings}
suggestions = []
if "python-if-in-transformed-function" in kinds or "python-while-in-transformed-function" in kinds:
suggestions.append("Review data-dependent Python control flow and replace it with `lax.cond`, `lax.while_loop`, or `jnp.where` where appropriate.")
if "python-for-in-transformed-function" in kinds:
suggestions.append("Check long Python loops under `jit`; many should be `lax.scan` or `lax.fori_loop`.")
if "global-prng-key" in kinds or "legacy-key-api" in kinds:
suggestions.append("Review PRNG handling. Prefer explicit key threading and typed keys for new code unless compatibility requires legacy keys.")
if "numpy-conversion-in-transformed-function" in kinds or "host-conversion" in kinds:
suggestions.append("Review host/device boundaries. Keep hot-path computations in `jax.numpy` and move host conversions to the program edge.")
if "jit-created-inside-loop" in kinds:
suggestions.append("Hoist `jax.jit` construction out of loops to reduce repeated tracing and cache churn.")
if "pmap-usage" in kinds:
suggestions.append("For major refactors, compare current `pmap` usage against modern sharding APIs and `shard_map` migration guidance.")
return suggestions
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Scan Python files for common JAX sharp bits and migration targets.",
formatter_class=argparse.RawTextHelpFormatter,
epilog="""Exit codes:
0 success
2 operational error
Examples:
python3 scripts/jax_project_scan.py .
python3 scripts/jax_project_scan.py src/ --format text
python3 scripts/jax_project_scan.py my_file.py --max-findings 50
""",
)
parser.add_argument("path", help="File or directory to scan.")
parser.add_argument("--exclude-dir", action="append", default=[], help="Directory names to exclude. May be passed multiple times.")
parser.add_argument("--format", choices=("json", "text"), default="json", help="Output format. Default: json")
parser.add_argument("--max-findings", type=int, default=200, help="Maximum number of findings to include. Default: 200")
parser.add_argument("--output", help="Write output to a file instead of stdout.")
return parser.parse_args()
def main() -> int:
args = parse_args()
root = Path(args.path)
if not root.exists():
sys.stderr.write(f"Error: path not found: {root}\n")
return 2
excludes = set(DEFAULT_EXCLUDES) | set(args.exclude_dir)
files = iter_python_files(root, excludes)
all_findings: list[Finding] = []
parse_errors: list[dict[str, str]] = []
for path in files:
findings, parse_error = scan_file(path)
if parse_error is not None:
parse_errors.append({"file": str(path), "error": parse_error})
all_findings.extend(findings)
severity_counts: dict[str, int] = {}
kind_counts: dict[str, int] = {}
for finding in all_findings:
severity_counts[finding.severity] = severity_counts.get(finding.severity, 0) + 1
kind_counts[finding.kind] = kind_counts.get(finding.kind, 0) + 1
all_findings_sorted = sorted(all_findings, key=lambda f: (f.file, f.line, f.column, f.kind))
limited_findings = all_findings_sorted[: max(args.max_findings, 0)]
report = {
"path": str(root),
"files_scanned": len(files),
"parse_errors": parse_errors,
"summary": {
"total_findings": len(all_findings),
"shown_findings": len(limited_findings),
"severity_counts": severity_counts,
"kind_counts": dict(sorted(kind_counts.items())),
},
"suggestions": build_suggestions(all_findings),
"findings": [asdict(f) for f in limited_findings],
}
if args.format == "json":
text = json.dumps(report, indent=2, sort_keys=True)
else:
lines = [
f"Path: {report['path']}",
f"Files scanned: {report['files_scanned']}",
f"Parse errors: {len(report['parse_errors'])}",
f"Total findings: {report['summary']['total_findings']}",
"",
"Severity counts:",
]
for sev, count in sorted(report["summary"]["severity_counts"].items()):
lines.append(f" {sev}: {count}")
lines.append("")
lines.append("Suggestions:")
if report["suggestions"]:
for item in report["suggestions"]:
lines.append(f" - {item}")
else:
lines.append(" - No specific suggestions.")
lines.append("")
lines.append("Findings:")
if limited_findings:
for f in limited_findings:
lines.append(
f" - {f.file}:{f.line}:{f.column} [{f.severity}] {f.kind}: {f.message}"
)
if f.snippet:
lines.append(f" {f.snippet}")
else:
lines.append(" (none)")
if parse_errors:
lines.append("")
lines.append("Parse errors:")
for err in parse_errors:
lines.append(f" - {err['file']}: {err['error']}")
text = "\n".join(lines)
if args.output:
Path(args.output).write_text(text + ("" if text.endswith("\n") else "\n"), encoding="utf-8")
else:
sys.stdout.write(text)
if not text.endswith("\n"):
sys.stdout.write("\n")
return 0
if __name__ == "__main__":
raise SystemExit(main())
\
#!/usr/bin/env python3
"""Search a local JAX checkout for relevant docs, tests, and source files."""
from __future__ import annotations
import argparse
import json
import os
import re
import sys
from pathlib import Path
from typing import Any
DEFAULT_DIRS = ("docs", "jax", "tests", "benchmarks")
TEXT_EXTS = {".py", ".md", ".rst", ".txt", ".bzl", ".yaml", ".yml"}
def tokenize_query(query: str) -> list[str]:
return [tok for tok in re.split(r"[^a-zA-Z0-9_]+", query.lower()) if len(tok) >= 2]
def iter_candidate_files(repo: Path, include_dirs: tuple[str, ...], kind: str) -> list[Path]:
dirs = []
if kind == "all":
dirs = include_dirs
elif kind == "docs":
dirs = ("docs",)
elif kind == "source":
dirs = ("jax",)
elif kind == "tests":
dirs = ("tests",)
elif kind == "benchmarks":
dirs = ("benchmarks",)
out = []
for d in dirs:
root = repo / d
if not root.exists():
continue
for path in root.rglob("*"):
if path.is_dir():
continue
if path.suffix.lower() in TEXT_EXTS:
out.append(path)
return sorted(out)
def score_file(path: Path, repo: Path, query_tokens: list[str], phrase: str, max_bytes: int) -> dict[str, Any] | None:
rel = path.relative_to(repo)
rel_str = rel.as_posix().lower()
try:
content = path.read_text(encoding="utf-8", errors="ignore")
except Exception:
return None
if len(content) > max_bytes:
content = content[:max_bytes]
content_lower = content.lower()
path_hits = {tok: rel_str.count(tok) for tok in query_tokens if tok in rel_str}
content_hits = {tok: content_lower.count(tok) for tok in query_tokens if tok in content_lower}
score = 0
score += sum(min(count, 3) * 5 for count in path_hits.values())
score += sum(min(count, 5) * 1 for count in content_hits.values())
if phrase and phrase in rel_str:
score += 20
if phrase and phrase in content_lower:
score += 10
if score == 0:
return None
preview_lines = []
if query_tokens:
line_matches = 0
for lineno, line in enumerate(content.splitlines(), start=1):
line_lower = line.lower()
if any(tok in line_lower for tok in query_tokens):
preview_lines.append({"line": lineno, "text": line.strip()[:240]})
line_matches += 1
if line_matches >= 5:
break
return {
"path": rel.as_posix(),
"score": score,
"path_hits": path_hits,
"content_hits": content_hits,
"preview_lines": preview_lines,
}
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Search a local JAX checkout for relevant docs, tests, or source files by topic.",
formatter_class=argparse.RawTextHelpFormatter,
epilog="""Exit codes:
0 success
2 operational error
Examples:
python3 scripts/jax_repo_locator.py --repo /path/to/jax --query "custom vjp batching"
python3 scripts/jax_repo_locator.py --repo . --query "shard_map pmap migration" --kind docs
python3 scripts/jax_repo_locator.py --repo . --query "debug.print compiler_ir" --kind all --format text
""",
)
parser.add_argument("--repo", required=True, help="Path to a local JAX checkout.")
parser.add_argument("--query", required=True, help="Search query.")
parser.add_argument("--kind", choices=("all", "docs", "source", "tests", "benchmarks"), default="all", help="Subset of the repo to search. Default: all")
parser.add_argument("--limit", type=int, default=20, help="Maximum number of results to return. Default: 20")
parser.add_argument("--max-bytes", type=int, default=200000, help="Maximum bytes of each file to score. Default: 200000")
parser.add_argument("--format", choices=("json", "text"), default="json", help="Output format. Default: json")
parser.add_argument("--output", help="Write output to a file instead of stdout.")
return parser.parse_args()
def main() -> int:
args = parse_args()
repo = Path(args.repo)
if not repo.exists():
sys.stderr.write(f"Error: repo path not found: {repo}\n")
return 2
query_tokens = tokenize_query(args.query)
phrase = args.query.lower().strip()
files = iter_candidate_files(repo, DEFAULT_DIRS, args.kind)
scored = []
for path in files:
result = score_file(path, repo, query_tokens, phrase, args.max_bytes)
if result is not None:
scored.append(result)
scored.sort(key=lambda item: (-item["score"], item["path"]))
scored = scored[: max(args.limit, 0)]
report = {
"repo": str(repo),
"query": args.query,
"kind": args.kind,
"query_tokens": query_tokens,
"results": scored,
}
if args.format == "json":
text = json.dumps(report, indent=2, sort_keys=True)
else:
lines = [
f"Repo: {report['repo']}",
f"Query: {report['query']}",
f"Kind: {report['kind']}",
"",
]
if not scored:
lines.append("No matches.")
else:
for item in scored:
lines.append(f"- {item['path']} (score={item['score']})")
if item["path_hits"]:
lines.append(f" path_hits: {item['path_hits']}")
if item["content_hits"]:
lines.append(f" content_hits: {item['content_hits']}")
for line in item["preview_lines"]:
lines.append(f" L{line['line']}: {line['text']}")
lines.append("")
text = "\n".join(lines)
if args.output:
Path(args.output).write_text(text + ("" if text.endswith("\n") else "\n"), encoding="utf-8")
else:
sys.stdout.write(text)
if not text.endswith("\n"):
sys.stdout.write("\n")
return 0
if __name__ == "__main__":
raise SystemExit(main())