
Hugging Science
- 797 installs
- 32.7k repo stars
- Updated August 3, 2026
- k-dense-ai/scientific-agent-skills
hugging-science is a version 1.1 agent skill that surfaces curated scientific datasets, models, papers, and Hugging Face Spaces for developers building domain ML agents in biology, chemistry, and related fields.
About
hugging-science is a K-Dense Inc. skill (metadata version 1.1) that indexes scientific resources for AI and ML work across biology, chemistry, physics, astronomy, climate, genomics, materials, medicine, ecology, energy, engineering, math, drug discovery, protein design, weather modeling, theorem proving, single-cell analysis, and PDE solving. The curated, LLM-friendly catalog helps agents discover datasets, models, blog posts, and interactive Spaces, then use them via Hugging Face datasets, transformers, the HF Inference API, gradio_client, and methodology citations. Reach for hugging-science when scientific domain questions need vetted HF assets instead of generic web search. It suits ML engineers and research coders building biology or chemistry agents who want structured discovery paths and reproducible citations rather than ad-hoc model hunting.
- Curated index of scientific resources across 17 domains including genomics, protein design, PDE solving, weather modelin
- Delivers parseable llms.txt, llms-full.txt and per-domain markdown files optimized for LLM consumption
- Connects directly to Hugging Face datasets, transformers, Inference API, gradio_client and community Spaces
- Provides methodology citations alongside resources for reproducible scientific ML workflows
- Higher signal-to-noise than generic web search for scientific AI/ML questions
Hugging Science by the numbers
- 797 all-time installs (skills.sh)
- +46 installs in the week ending Aug 2, 2026 (Skillselion tracking)
- Ranked #1,328 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/k-dense-ai/scientific-agent-skills --skill hugging-scienceAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 797 |
|---|---|
| repo stars | ★ 32.7k |
| Last updated | August 3, 2026 |
| Repository | k-dense-ai/scientific-agent-skills ↗ |
How do you find scientific Hugging Face models and datasets?
Quickly surface high-quality, domain-specific scientific datasets, models, papers, and interactive demos when building ML-powered agents for biology, chemistry, p
Who is it for?
ML engineers building scientific domain agents who need curated Hugging Face datasets, models, and Spaces with citation-ready references.
Skip if: General-purpose coding or non-scientific app features with no HF datasets, models, or domain literature needs.
When should I use this skill?
User works on biology, chemistry, genomics, drug discovery, climate ML, or asks for scientific HF datasets, models, or Spaces.
What you get
Curated dataset and model shortlists, Space demos, methodology citations, and integration snippets for HF APIs.
- curated resource shortlists
- HF integration snippets
- methodology citations
By the numbers
- Skill metadata version 1.1
- Covers 15+ scientific domains including genomics, drug discovery, and PDE solving
Files
Hugging Science
Hugging Science is a curated, LLM-friendly index of scientific datasets, models, blog posts, and interactive demos for ML researchers. Use it when a scientific ML question lands in front of you — it's much higher signal than generic search and the entries are pre-filtered for quality and openness.
There are two related surfaces, and you should use both:
- The catalog at `huggingscience.co` — a static, parseable index of resources across 17 scientific domains. It exposes
llms.txt(compact),llms-full.txt(full content), andtopics/<slug>.md(per-domain). These are markdown files designed to be fetched and read. - The `hugging-science` Hugging Face organization —
huggingface.co/hugging-science— community-submitted datasets, a few models, and ~27 interactive Spaces (notably BoltzGen for protein/binder design, Dataset Quest for submissions, and Science Release Heatmap for ecosystem visualization).
The catalog points to resources hosted on the broader Hugging Face Hub. So an entry like arcinstitute/opengenome2 is a regular HF dataset that you load with the datasets library; an entry like facebook/esm2_t33_650M_UR50D is a regular HF model you load with transformers. The catalog's job is curation and discovery; usage goes through standard Hugging Face APIs.
When to use this skill
Engage this skill when the user's task involves AI/ML applied to science. Common signals:
- Names a scientific domain (protein, genome, molecule, crystal, weather, climate, galaxy, EEG, microbiome, pathology, plasma, …)
- Asks "is there a dataset/model for X" where X is scientific
- Wants to fine-tune on scientific data, evaluate on scientific benchmarks, or reproduce a scientific ML paper
- Asks about specific known scientific models (Evo-2, ESM2, BoltzGen, Nucleotide Transformer, AlphaFold-derived, etc.)
- Needs an interactive demo for a scientific task (binder design, theorem proving, etc.)
If the task is generic ML (recommendation systems, chatbot RAG, vision on cats and dogs), this skill is not the right tool — defer to general HF Hub knowledge instead.
Core workflow
Most invocations follow this five-step loop. Don't skip discovery — the value of Hugging Science is that it has already filtered hundreds of resources down to high-signal picks per domain.
1. Identify the domain(s)
Map the user's task to one or more of the 17 topic slugs:
astronomy · benchmark · biology · biotechnology · chemistry · climate · conservation · earth-science · ecology · energy · engineering · genomics · materials-science · mathematics · medicine · physics · scientific-reasoning
Some tasks span multiple topics (e.g., drug discovery → chemistry + biology + medicine). Fetch each relevant topic.
2. Fetch the relevant catalog content
Use the bundled script for clean, structured access:
python scripts/fetch_catalog.py topic biology
python scripts/fetch_catalog.py topic materials-science --filter models
python scripts/fetch_catalog.py search "protein language model"
python scripts/fetch_catalog.py all # full llms-full.txtYou can also fetch the raw markdown directly:
https://huggingscience.co/llms.txt— compact indexhttps://huggingscience.co/llms-full.txt— every entry, every domainhttps://huggingscience.co/topics/<slug>.md— one domain (slug is hyphenated, e.g.materials-science.md,earth-science.md,scientific-reasoning.md)
Each entry is a markdown block with Type, Tags, HuggingFace URL (or Link for blogs), and a one-line description. See references/topics-and-slugs.md for the entry schema and slug list.
3. Pick the right resource(s)
Read the descriptions and tags. Match to the user's task with judgment, not keyword overlap. Things to weigh:
- Scale fit — Evo-2 40B is overkill for a quick sequence classification on a laptop; ESM2 35M might be perfect.
- License and access — most are open, but check the underlying HF model card.
- Modality alignment — DNA vs. protein vs. SMILES vs. crystal structure; many "biology" models are not interchangeable.
- Recency / supersession — if both an older and newer entry cover the same task, prefer newer unless there's a reason not to.
If you're not sure which resource to pick, briefly present the top 2–3 candidates to the user with their tradeoffs, then proceed once they choose. Don't pick silently when the choice materially changes the work.
For domain-specific go-to picks (the "if in doubt, start here" entries), see references/flagship-resources.md.
4. Use the resource
The mechanics depend on resource type. Read the matching reference file before writing code:
- Datasets →
references/using-datasets.md— loading viadatasets, streaming for huge corpora, common columns, splits - Models →
references/using-models.md— localtransformers, Hugging Face Inference API, Inference Providers for very large models, GPU sizing - Spaces (interactive demos) →
references/using-spaces.md—gradio_clientpattern with a worked BoltzGen example
The reference files are short and focused. If you're already fluent in the relevant API, skim; if not, read fully before writing code. The patterns are different from generic HF usage in a few important places (e.g., trust_remote_code requirements, scientific-data dtype gotchas).
5. Cite the methodology
When the catalog has a blog post matching the task (Type: blog or in the Blog Posts section of a topic file), include its URL when you explain your approach to the user. Methodology blogs are written by the dataset/model authors and answer "why this design" questions that model cards usually skip. Treat them like citations — a one-line "see <link> for the methodology behind X" is plenty.
Authentication: HF_TOKEN
Many catalog resources are gated (clinical data, large foundation models, private Spaces). Authenticate via the HF_TOKEN environment variable.
Load `HF_TOKEN` from a `.env` file when available — that's where the user keeps secrets. Use python-dotenv at the top of any script that hits the HF API:
from dotenv import load_dotenv
load_dotenv() # picks up HF_TOKEN from .env in cwd or any parent dirIf .env doesn't exist or doesn't define HF_TOKEN, fall back gracefully — many resources are public and work without it. Don't hard-code tokens, don't echo them, and don't suggest huggingface-cli login as the primary path; the user prefers .env.
The .env file should contain a line like:
HF_TOKEN=hf_...If you're creating a new project, also add .env to .gitignore if it isn't already there.
A few important things to remember
The catalog is curated, not exhaustive. If a user needs a specific resource and Hugging Science doesn't list it, that doesn't mean it doesn't exist on HF Hub. Search HF Hub directly as a fallback. But always start with the catalog when the domain matches — the curation is the value.
The entries are pointers. Don't try to "use Hugging Science" as if it were an API. There is no Hugging Science inference endpoint. Every actionable resource lives on HF Hub or as a HF Space, and you use it via the standard HF tooling.
Many scientific models require `trust_remote_code=True`. Custom architectures (Evo-2, many genomics/materials models) ship custom modeling code. This is normal in this ecosystem. Pass the flag and inform the user.
Scientific datasets are often large and weirdly-shaped. Genomics corpora can be billions of tokens; cosmology images can be hundreds of GB; materials datasets contain non-standard objects (crystal structures, graphs). Use streaming (streaming=True on load_dataset) by default for anything claimed to be over a few GB, and inspect schema before assuming columns.
Spaces are great for one-off scientific generations. If the user wants to design a binder for a target protein or run inference on a hosted model demo, calling the Space via gradio_client is faster and cheaper than spinning up the model locally. Check references/using-spaces.md first — huggingface.co/hugging-science has ~27 of these.
The catalog itself may evolve. Entries get added regularly; occasionally entries change slugs. If a URL 404s, refetch the topic file or llms.txt to get the current state — don't paper over the failure.
Bundled resources
scripts/fetch_catalog.py— fetch and filter catalog content. Run with--helpfor full usage. Use this in preference to ad-hoc WebFetch calls when you need structured access.references/topics-and-slugs.md— exact topic slugs, what each covers, and the entry schema.references/using-datasets.md— patterns and gotchas for loading scientific datasets.references/using-models.md— running scientific models locally, via Inference API, or via Inference Providers.references/using-spaces.md— calling HF Spaces (notably BoltzGen) programmatically withgradio_client.references/flagship-resources.md— go-to dataset/model picks per domain when the user wants a sensible default.
Flagship resources by domain
Sensible "if in doubt, start here" picks per scientific domain. These are the resources users most often want when they describe a task in plain language. Always confirm with the live catalog (fetch_catalog.py topic <slug>) before final recommendation — the catalog evolves and there may be something newer.
The catalog is the source of truth. Treat this file as a fast cheatsheet, not a directory.
Biology / Genomics
DNA foundation models
arcinstitute/evo2_40b— 40B-param DNA LM, 9.3T nucleotide pretraining; zero-shot variant effect prediction, sequence generation. Huge — use Inference Providers or a Space.arcinstitute/evo2_7b— 7B instruction-tuned variant; runs on a single 24 GB GPU.
Protein language models
facebook/esm2_t33_650M_UR50D— 650M ESM2; standard for embeddings, structure prediction, mutation scoring. Strong default.- Smaller ESM2 variants (
*_t30_150M_*,*_t12_35M_*) for laptop/CPU use.
Single-cell / transcriptomics
arcinstitute/Stack-Large(STACK) — single-cell foundation model with in-context learning across cell types.Merck/TEDDY— 116M single-cell foundation models for genomics + drug discovery.
Antibodies
opig/OAS— Observed Antibody Space, ~1B antibody sequences. The standard antibody ML dataset.
Bioacoustics / ecology
EarthSpeciesProject/NatureLM-audio— first audio-LM for animal vocalizations.EarthSpeciesProject/esp-aves2-sl-beats-all— self-supervised bioacoustic encoder.
Genomic corpora
arcinstitute/opengenome2— curated prokaryotic + eukaryotic sequences for foundation-model pretraining.
Chemistry / Drug discovery
SandboxAQ/AQAffinity— drug-target affinity prediction.- SAIR dataset — 1M+ protein-ligand co-structures (see SandboxAQ blog post in catalog).
- For SMILES/molecular tasks: check the catalog
chemistrytopic for current best — molecular foundation models change frequently.
Materials science
- LeMaterial — large open materials database (see catalog blog post 2024-12-10).
- Crystal structure foundation models, perovskite datasets — fetch
topic materials-sciencefor the live list. Many shippymatgen.Structureobjects rather than tensors.
Physics
- PDE-solver datasets (
topic physics --filter datasets) — magnetohydrodynamics, fluid dynamics, plasma. - Physics-Informed Neural Networks (PINN) — methodology covered in catalog blog posts.
Climate / Earth science / Weather
- Weather-foundation-model entries under
topic climate— these are often huge multi-modal models with structured atmospheric inputs (geopotential, temperature on multiple pressure levels). - Satellite imagery / remote sensing models under
topic earth-science.
Medicine / Pathology
hugging-science/breast-cancer-detector-2— image classification baseline from the org itself.- Pathology + radiology foundation models — fetch
topic medicine. Many are gated; check the model card for access requirements.
Mathematics / Scientific reasoning
- Kimina-Prover family — large theorem-proving models. Likely needs Inference Providers.
topic scientific-reasoningfor LLM-as-scientific-assistant work, paper QA, multi-step reasoning evals.
Astronomy
- Galaxy survey datasets (e.g.,
hugging-science/mmu_legacysurvey_dr10_south_21) — image-heavy, FITS format, may needastropy. - Astronomical foundation models — fetch
topic astronomy.
Cross-domain interactive demos (Spaces)
hugging-science/boltzgen-demo— protein/peptide/nanobody binder design (the marquee demo).hugging-science/dataset-quest— discover and submit scientific datasets.hugging-science/science-release-heatmap— visualize who's publishing AI4Science resources.
How to use this list
1. User describes a task → match to a domain row above. 2. Fetch the live topic file with fetch_catalog.py topic <slug> and confirm the recommended resource still exists / is current. 3. Read the resource's HF card for input format, license, access requirements. 4. Follow using-datasets.md / using-models.md / using-spaces.md for the actual code. 5. If a related blog post is listed in the catalog, cite it when explaining methodology.
If nothing in this cheatsheet fits, run fetch_catalog.py search "<keyword>" against the full index. The catalog has hundreds of entries this file doesn't enumerate.
Topic slugs and entry schema
The Hugging Science catalog organizes scientific resources across 17 topics. Each has a markdown file at https://huggingscience.co/topics/<slug>.md. Slugs are lowercase and hyphenated.
Topic slugs and what each covers
| Slug | What's in here |
|---|---|
astronomy | Galaxy/stellar surveys, cosmology, exoplanets, telescope imagery, foundation models for astronomical data |
benchmark | Cross-domain evaluation suites — useful when comparing methods or running standard tests |
biology | Protein/DNA/single-cell data and models, antibodies, bioacoustics, microbiome — broad biology umbrella |
biotechnology | Synthetic biology, fermentation, applied genetic engineering data |
chemistry | Molecules, reactions, drug discovery, SMILES corpora, DFT data, ligand-protein interactions |
climate | Weather forecasting, climate models, atmospheric data, storm/flood prediction |
conservation | Wildlife monitoring, biodiversity, camera-trap and bioacoustic models |
earth-science | Remote sensing, satellite imagery, geospatial foundation models |
ecology | Species distribution, ecosystem dynamics, biogeography (overlaps with conservation/biology) |
energy | Battery materials, fusion plasma, grid simulation, renewables modeling |
engineering | CAD, mechanical/structural simulation, robotics datasets |
genomics | DNA language models, variant effect, single-cell, phylogenetics (overlaps heavily with biology) |
materials-science | Crystal structures, band gaps, catalysts, perovskites, alloys, materials foundation models |
mathematics | Theorem proving, formal math, mathematical reasoning datasets |
medicine | Pathology, radiology, clinical NLP, drug-disease, EHR (overlaps with biology/chemistry) |
physics | PDE solvers, fluid/plasma simulation, particle physics, physics-informed ML |
scientific-reasoning | LLMs for scientific QA, paper understanding, multi-step scientific reasoning |
Cross-domain reality
Most real tasks span multiple slugs. Pull all relevant ones rather than guessing:
- "Drug discovery" →
chemistry,biology,medicine - "Protein structure prediction" →
biology,chemistry - "Weather forecasting model" →
climate,earth-science,physics - "Single-cell foundation model" →
biology,genomics,medicine - "Battery electrolyte design" →
materials-science,chemistry,energy - "Bioacoustic species ID" →
biology,ecology,conservation
When in doubt, fall back to python scripts/fetch_catalog.py search "<keyword>" against the full catalog.
Entry schema
Each catalog entry is an H3 block with bulleted metadata followed by a description. Three flavors:
Datasets
### org/dataset-name
- **Type**: <category, e.g. "Genomics", "Pathology", "PDE Simulation">
- **Tags**: <comma-separated topic tags>
- **HuggingFace**: https://huggingface.co/datasets/org/dataset-name
<one-line description>Models
### Model Display Name
- **Type**: <category, e.g. "Protein Language Model", "Materials Foundation Model">
- **Tags**: <comma-separated topic tags>
- **HuggingFace**: https://huggingface.co/org/model-id
<one-line description>Blog posts
### Post Title
- **Author**: <username>
- **Date**: <YYYY-MM-DD>
- **Tags**: <comma-separated>
- **Link**: <URL — usually huggingface.co/blog/...>
<one-line description>Endpoints
https://huggingscience.co/llms.txt— compact site indexhttps://huggingscience.co/llms-full.txt— every entry, every domain (this is the file to fetch when you want to grep across the whole catalog)https://huggingscience.co/topics/<slug>.md— one domainhttps://huggingscience.co/feed.xml— RSS for new entries
The fetch_catalog.py script wraps these and adds parsing, filtering, and JSON output. Prefer the script for structured access; use raw WebFetch/curl only if the script fails.
Using scientific datasets from the catalog
Hugging Science dataset entries always link to a Hugging Face Hub dataset (huggingface.co/datasets/<org>/<name>). You load them with the standard datasets library. The interesting part is what makes scientific datasets different from typical NLP/vision datasets — that's what this file is about.
Install
Use uv for all installs:
uv pip install datasets huggingface_hub # in an active venv
# or, project-style:
uv add datasets huggingface_hub
# one-off:
uv run --with datasets python my_script.pyFor private/gated datasets, authenticate via HF_TOKEN. Prefer loading from `.env`:
# .env (in project root, gitignored)
HF_TOKEN=hf_...from dotenv import load_dotenv
load_dotenv() # picks up HF_TOKEN before any HF call
from datasets import load_dataset
ds = load_dataset("opig/OAS")If python-dotenv isn't installed: uv add python-dotenv (or uv pip install python-dotenv).
A surprising number of biomedical datasets are gated (clinical PHI proxies, antibody repertoires from named patients). Check the dataset card before assuming open access.
Default loading pattern
from datasets import load_dataset
ds = load_dataset("arcinstitute/opengenome2")
print(ds) # see splits and columns
print(ds["train"][0]) # peek at one rowUse streaming for large datasets — by default
Many scientific corpora are 10 GB to many TB. load_dataset(..., streaming=True) returns an IterableDataset that pulls shards on demand instead of materializing the whole thing on disk:
ds = load_dataset("arcinstitute/opengenome2", split="train", streaming=True)
for example in ds.take(10):
...Rule of thumb: if the dataset card mentions billions of tokens, millions of images, or "TB", default to streaming and only switch to full download when the user explicitly wants offline reproducibility.
Inspect schema before assuming columns
Generic datasets have predictable columns (text, label, image). Scientific datasets often don't. Before writing preprocessing code, look at one example:
sample = next(iter(load_dataset("opig/OAS", split="train", streaming=True)))
print(sample.keys())Common surprises:
- Genomics: columns can be
sequence,species,taxonomy,accessionrather thantext. - Materials: rows may contain serialized
pymatgenStructureobjects or CIF strings — not numeric tensors. - Imaging: medical/astronomy images can be FITS, DICOM, or NIfTI rather than PNG/JPEG. The
imagecolumn may be raw bytes that need a domain-specific decoder. - Time series / signals: EEG, audio, weather often have variable-length arrays under a column like
signalorarray; the dtype matters (float16vsfloat32) for memory.
Splits and subsets
- Many scientific datasets ship multiple configs (e.g.,
load_dataset("Merck/TEDDY", "single_cell")). Ifload_dataseterrors with "Please pick a config", read the dataset card or runget_dataset_config_names("..."). - Some have non-standard split names (
pretrain,held_out_species,test_chr1). Don't assumetrain/validation/test.
Filtering and subsetting
For very large datasets, prefer filter on a streaming iterator over downloading and slicing:
ds = load_dataset("opig/OAS", split="train", streaming=True)
human_only = ds.filter(lambda ex: ex.get("species") == "human")To convert a streaming subset into an in-memory dataset for training:
from datasets import Dataset
subset = Dataset.from_list(list(human_only.take(10_000)))Train/eval handoff to transformers
Once shaped correctly, scientific datasets feed Trainer/SFTTrainer like any other. The bridge is usually a tokenizer or feature extractor that's specific to the domain:
- DNA: tokenizer from the matching DNA model (e.g.,
AutoTokenizer.from_pretrained("arcinstitute/evo2_7b", trust_remote_code=True)). - Proteins:
AutoTokenizer.from_pretrained("facebook/esm2_t33_650M_UR50D"). - SMILES: usually a character-level or BPE tokenizer; check the model card.
If a model and dataset come from the same org, their tokenizers/preprocessors are usually compatible by design — that's a strong signal to pair them.
Caveats specific to scientific data
- License: Some datasets are CC-BY-NC (research only). Check before any commercial deployment suggestion.
- Versioning: Major scientific datasets revise their splits over time. Pin a
revision=if reproducibility matters. - Preprocessing must match training: For foundation models, the catalog's blog posts often document the exact preprocessing used in pretraining (tokenizer config, normalization). When fine-tuning, replicate it — small mismatches (e.g., reverse complement augmentation for DNA) can wreck downstream performance.
Using scientific models from the catalog
Hugging Science model entries link to standard Hugging Face Hub repos. There are three sensible execution paths. Pick based on model size and whether the user is doing one-off inference or a long batch job.
Decision: where to run the model
| Path | When to use | What it costs |
|---|---|---|
| Local with `transformers` | Models ≤ ~7B params, user has a GPU or wants offline use, or doing fine-tuning | Disk + VRAM; free |
| HF Inference API (serverless) | Quick one-off inference on smaller hosted models, no GPU needed | Free tier exists, then pay-per-call |
| HF Inference Providers | Very large models (Evo-2 40B, Kimina-Prover 72B), or when you need throughput | Pay-per-token; routed to third-party providers |
| HF Space (gradio_client) | The model has an interactive demo and you want easy structured I/O without managing weights | Free if Space is public; see using-spaces.md |
Always check the model card first — some entries are only available as Spaces (no public weights), and some are gated and require approval before download.
Local with transformers
Use uv for installs:
uv pip install transformers torch accelerate python-dotenv # in an active venv
# or project-style:
uv add transformers torch accelerate python-dotenvFor gated models, put the token in .env rather than running huggingface-cli login:
# .env (gitignored)
HF_TOKEN=hf_...from dotenv import load_dotenv
load_dotenv() # reads HF_TOKEN before any HF call
from transformers import AutoModel, AutoTokenizer
model_id = "facebook/esm2_t33_650M_UR50D"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModel.from_pretrained(model_id)
inputs = tok("MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQ", return_tensors="pt")
embeddings = model(**inputs).last_hidden_statetrust_remote_code=True is normal here
A large fraction of scientific models — Evo-2, many Nucleotide Transformer variants, single-cell foundation models, several materials models — ship custom modeling code in their repo. transformers will refuse to load them without trust_remote_code=True:
model = AutoModel.from_pretrained("arcinstitute/evo2_7b", trust_remote_code=True)Tell the user when you set this flag and why. It executes Python from the repo, so the user should trust the org. The catalog only lists reputable orgs (Arc Institute, Meta/Facebook AI, EleutherAI, SandboxAQ, Merck, etc.), but the user gets to make the call.
Sizing the GPU
Rough memory for inference at fp16 (very approximate — quantization changes this):
- 35M–650M params (most ESM2 variants): runs on a laptop GPU or even CPU.
- 1B–7B (Evo-2 7B, Nucleotide Transformer 2.5B, STACK Large): single 24 GB GPU is fine.
- 40B+ (Evo-2 40B, Kimina-Prover): needs multi-GPU or A100/H100; almost always better via Inference Providers unless the user has the hardware.
For training/fine-tuning, multiply by ~3–4× for activations and optimizer state.
HF Inference API (serverless)
Fast for tiny one-off jobs without setting up a GPU. The model has to be supported on the serverless tier (smaller models, popular pipelines).
uv pip install huggingface_hub python-dotenv # or: uv add huggingface_hub python-dotenvPut your token in .env rather than exporting per-shell:
# .env (gitignored)
HF_TOKEN=hf_...from dotenv import load_dotenv
load_dotenv() # InferenceClient reads HF_TOKEN from env
from huggingface_hub import InferenceClient
client = InferenceClient(model="facebook/esm2_t33_650M_UR50D")
result = client.feature_extraction("MKTAYIAKQR")The serverless API supports tasks like feature_extraction, text_generation, image_classification, token_classification. For non-standard scientific tasks (e.g., DNA sequence generation), you may need Inference Providers or local execution instead.
HF Inference Providers
For very large models or when you want production throughput. Inference Providers route requests to vetted backends (Together, Fireworks, Replicate, Sambanova, etc.) that host frontier models.
from huggingface_hub import InferenceClient
client = InferenceClient(provider="together", model="arcinstitute/evo2_40b")
output = client.text_generation("ATCGGCTA", max_new_tokens=64)Check the model card for which providers host it. Not every catalog model is available — many are research-only and only hosted by their authors as a Space.
After loading: standard pipelines apply
Once the model is loaded, scientific models behave like any other transformers model — you embed sequences, generate, classify, or fine-tune. The unique steps are:
1. Use the matching tokenizer/feature extractor. Don't try to feed protein sequences to a DNA tokenizer; the alphabets are different and the model will silently produce garbage. 2. Match the preprocessing from pretraining. For fine-tuning, the catalog's blog posts often spell out exact preprocessing recipes (special tokens, normalization, augmentation). Read them before training. 3. Mind the output head. Many scientific foundation models are masked-LM by default; classification or regression downstream tasks usually need an extra head layered on model.last_hidden_state.
When you can't run a model anywhere
Some catalog models are demo-only — the authors host a Space but never published weights. In that case:
- See
using-spaces.mdand call the Space viagradio_client. - Or surface this constraint to the user and offer the next-best fully-open alternative from the same topic file.
Using Hugging Science Spaces (interactive demos)
A Hugging Face Space is a hosted web app — usually Gradio or Streamlit — that wraps a model behind a UI. The hugging-science org maintains ~27 of these, and many catalog entries point to a Space rather than (or in addition to) raw weights. Spaces are the fastest way to get scientific output without managing models or GPUs yourself.
Spaces are not just web pages — every Gradio Space exposes a programmatic API. You can call them from Python with gradio_client and parse the result as a normal Python value.
When to call a Space (vs. running the model locally)
Reach for a Space when:
- The user wants a one-shot result, not a fine-tuning loop.
- The model is huge (40B+) and the user has no GPU.
- The model has private weights and the Space is the only public interface.
- The Space already implements complex orchestration (tokenization, sampling, post-processing) you'd otherwise reimplement.
Reach for local execution instead when:
- You'll call it many times in a loop (Spaces have rate limits and queues).
- You're fine-tuning, batching at scale, or need offline reproducibility.
- The Space is private/gated and you can't get access.
Setup
uv pip install gradio_client python-dotenv # or: uv add gradio_client python-dotenvFor private/gated Spaces, store the token in .env and load it at startup:
# .env (gitignored)
HF_TOKEN=hf_...from dotenv import load_dotenv
load_dotenv() # gradio_client picks up HF_TOKEN automaticallyThe general pattern
from gradio_client import Client
client = Client("hugging-science/<space-name>")
# Find the API endpoints exposed by this Space:
print(client.view_api())
# Call the endpoint named in view_api(), e.g. "/predict":
result = client.predict(
"argument_one",
42,
api_name="/predict",
)
print(result)view_api() is the discovery step — it prints every exposed endpoint with parameter names and types. Always run it once when wrapping a new Space; the function signature varies between Spaces and isn't always obvious from the UI.
Worked example: BoltzGen (protein/peptide/nanobody binder design)
BoltzGen is one of the flagship Spaces in the hugging-science org. It generates designed binders against a target protein.
from gradio_client import Client, file
client = Client("hugging-science/boltzgen-demo")
print(client.view_api()) # inspect first
# Typical call shape (verify against view_api() — endpoint names evolve):
result = client.predict(
target_pdb=file("/path/to/target.pdb"),
binder_type="protein", # or "peptide", "nanobody"
n_designs=8,
api_name="/generate",
)
# result is usually a list of generated sequences/structures or a path
# to a downloadable file inside the Space's tmp dir.When the Space returns a file path, gradio_client downloads the file to a local temp location and returns the path — handy for pipelines that need the actual output (.pdb, .fasta).
File inputs
Many scientific Spaces take structured file inputs (PDB, CIF, FASTA, NIfTI, FITS). Wrap them with gradio_client.file(...):
from gradio_client import file
result = client.predict(file("target.pdb"), api_name="/predict")Don't pass raw paths as strings — Gradio uploads files differently from text and the type wrapper signals which is which.
Other notable Spaces in hugging-science
These are good defaults to know about. Always check view_api() for the current signature.
| Space | Purpose |
|---|---|
hugging-science/boltzgen-demo | Protein / peptide / nanobody binder design |
hugging-science/anatomy-of-boltzgen | Educational walkthrough of BoltzGen architecture |
hugging-science/dataset-quest | Browse and submit community scientific datasets |
hugging-science/science-release-heatmap | Visualize AI4Science contributors across orgs and domains |
hugging-science/HuggingMod | Community moderation tooling |
The full live list lives at huggingface.co/hugging-science (Spaces tab). If a Space name 404s, the org may have renamed it — search the org page or check the catalog entry.
Rate limits and queue behavior
Free Spaces share a community GPU queue. For interactive use this is fine; for any kind of batching:
- Expect occasional
queue is fullor timeout errors. Add retry-with-backoff. - For large workloads, duplicate the Space into your own account (the "Duplicate" button on the Space page) to get private compute.
- Or: run the underlying model locally if weights are public — usually preferable for >10s of calls.
When the Space has no API
A small minority of Spaces disable the API or are Streamlit-based without a clean programmatic interface. In that case, fall back to local model execution (using-models.md) or surface the limitation to the user — don't try to scrape the UI.
#!/usr/bin/env python3
"""
Fetch and parse content from the Hugging Science catalog (huggingscience.co).
The catalog ships LLM-friendly markdown at three endpoints:
- https://huggingscience.co/llms.txt (compact index)
- https://huggingscience.co/llms-full.txt (every entry, every domain)
- https://huggingscience.co/topics/<slug>.md (one domain at a time)
Each entry in a topic file looks like:
### org/name-or-title
- **Type**: <category>
- **Tags**: <comma-separated>
- **HuggingFace**: <url> (or **Link**: <url> for blog posts)
- **Author**: <username> (blog posts only)
- **Date**: <YYYY-MM-DD> (blog posts only)
<one-line description>
Usage examples:
fetch_catalog.py topics # list all topic slugs
fetch_catalog.py topic biology # fetch and pretty-print a topic
fetch_catalog.py topic materials-science --filter models
fetch_catalog.py topic chemistry --tag "drug discovery"
fetch_catalog.py all # dump llms-full.txt
fetch_catalog.py search "protein language" # substring search across llms-full.txt
fetch_catalog.py json topic biology # structured JSON output
Stdlib only — no external deps required.
"""
from __future__ import annotations
import argparse
import json
import re
import sys
import urllib.error
import urllib.request
from dataclasses import dataclass, asdict, field
from typing import Iterable
BASE = "https://huggingscience.co"
KNOWN_TOPICS = [
"astronomy",
"benchmark",
"biology",
"biotechnology",
"chemistry",
"climate",
"conservation",
"earth-science",
"ecology",
"energy",
"engineering",
"genomics",
"materials-science",
"mathematics",
"medicine",
"physics",
"scientific-reasoning",
]
@dataclass
class Entry:
title: str
section: str # "datasets" | "models" | "blog posts" | "unknown"
type: str = ""
tags: list[str] = field(default_factory=list)
url: str = ""
author: str = ""
date: str = ""
description: str = ""
def matches_filter(self, kind: str | None, tag: str | None) -> bool:
if kind:
section_aliases = {
"datasets": {"datasets", "dataset"},
"models": {"models", "model"},
"blogs": {"blog posts", "blog", "blogs"},
}
wanted = section_aliases.get(kind.lower(), {kind.lower()})
if self.section.lower() not in wanted:
return False
if tag:
t = tag.lower()
if not any(t in tg.lower() for tg in self.tags) and t not in self.type.lower():
return False
return True
def fetch(url: str) -> str:
req = urllib.request.Request(url, headers={"User-Agent": "hugging-science-skill/1.0"})
try:
with urllib.request.urlopen(req, timeout=30) as resp:
return resp.read().decode("utf-8", errors="replace")
except urllib.error.HTTPError as e:
sys.exit(f"HTTP {e.code} fetching {url}: {e.reason}")
except urllib.error.URLError as e:
sys.exit(f"Network error fetching {url}: {e.reason}")
def parse_markdown(md: str) -> list[Entry]:
"""Parse a topic file or llms-full.txt into a list of Entry records.
Strategy: track the most recent H2 (## Section) as the section label;
each H3 (### Title) starts a new entry, with bulleted metadata and a
free-text description until the next H3 or H2."""
entries: list[Entry] = []
current_section = "unknown"
current: Entry | None = None
desc_lines: list[str] = []
def finalize(e: Entry | None, desc: list[str]) -> None:
if e is None:
return
e.description = " ".join(line.strip() for line in desc if line.strip()).strip()
entries.append(e)
for raw in md.splitlines():
line = raw.rstrip()
# Section header (H2)
m2 = re.match(r"^##\s+(.+?)\s*$", line)
if m2 and not line.startswith("###"):
finalize(current, desc_lines)
current, desc_lines = None, []
section = m2.group(1).strip().lower()
section = section.split(" (")[0] # strip "(N)" suffix if present
current_section = section
continue
# Entry header (H3)
m3 = re.match(r"^###\s+(.+?)\s*$", line)
if m3:
finalize(current, desc_lines)
current = Entry(title=m3.group(1).strip(), section=current_section)
desc_lines = []
continue
if current is None:
continue
# Metadata bullet
mb = re.match(r"^\s*[-*]\s+\*\*(\w+)\*\*\s*:\s*(.+?)\s*$", line)
if mb:
key, value = mb.group(1).lower(), mb.group(2).strip()
if key == "type":
current.type = value
elif key == "tags":
current.tags = [t.strip() for t in value.split(",") if t.strip()]
elif key in ("huggingface", "link", "url"):
current.url = value
elif key == "author":
current.author = value
elif key == "date":
current.date = value
continue
# Otherwise treat as description
desc_lines.append(line)
finalize(current, desc_lines)
return entries
def render_entry(e: Entry) -> str:
lines = [f"### {e.title}"]
if e.type:
lines.append(f"- Type: {e.type}")
if e.tags:
lines.append(f"- Tags: {', '.join(e.tags)}")
if e.url:
lines.append(f"- URL: {e.url}")
if e.author:
lines.append(f"- Author: {e.author}")
if e.date:
lines.append(f"- Date: {e.date}")
if e.description:
lines.append("")
lines.append(e.description)
return "\n".join(lines)
def render_entries(entries: Iterable[Entry], group_by_section: bool = True) -> str:
entries = list(entries)
if not entries:
return "(no entries matched)"
if not group_by_section:
return "\n\n".join(render_entry(e) for e in entries)
by_section: dict[str, list[Entry]] = {}
for e in entries:
by_section.setdefault(e.section, []).append(e)
out = []
for section, items in by_section.items():
out.append(f"## {section.title()} ({len(items)})")
out.append("")
for e in items:
out.append(render_entry(e))
out.append("")
return "\n".join(out).rstrip() + "\n"
def cmd_topics(_: argparse.Namespace) -> None:
print("Known Hugging Science topic slugs (use as `topic <slug>`):\n")
for t in KNOWN_TOPICS:
print(f" {t}")
print(
"\nSlugs are best-effort; if a topic 404s, fetch llms.txt directly to see the "
"live list:\n python fetch_catalog.py raw llms"
)
def cmd_topic(args: argparse.Namespace) -> None:
slug = args.slug.strip().lower().replace("_", "-").replace(" ", "-")
md = fetch(f"{BASE}/topics/{slug}.md")
entries = parse_markdown(md)
entries = [e for e in entries if e.matches_filter(args.filter, args.tag)]
if args.format == "json":
print(json.dumps([asdict(e) for e in entries], indent=2))
else:
print(f"# Hugging Science: {slug} ({len(entries)} entries)\n")
print(render_entries(entries))
def cmd_all(args: argparse.Namespace) -> None:
md = fetch(f"{BASE}/llms-full.txt")
if args.raw:
print(md)
return
entries = parse_markdown(md)
entries = [e for e in entries if e.matches_filter(args.filter, args.tag)]
if args.format == "json":
print(json.dumps([asdict(e) for e in entries], indent=2))
else:
print(f"# Hugging Science: full catalog ({len(entries)} entries)\n")
print(render_entries(entries))
def cmd_search(args: argparse.Namespace) -> None:
md = fetch(f"{BASE}/llms-full.txt")
entries = parse_markdown(md)
needle = args.query.lower()
matched = [
e
for e in entries
if needle in e.title.lower()
or needle in e.description.lower()
or needle in e.type.lower()
or any(needle in t.lower() for t in e.tags)
]
matched = [e for e in matched if e.matches_filter(args.filter, args.tag)]
if args.format == "json":
print(json.dumps([asdict(e) for e in matched], indent=2))
else:
print(f"# Search '{args.query}' — {len(matched)} match(es)\n")
print(render_entries(matched))
def cmd_raw(args: argparse.Namespace) -> None:
name = args.name.lower()
if name in ("llms", "index"):
print(fetch(f"{BASE}/llms.txt"))
elif name in ("full", "llms-full"):
print(fetch(f"{BASE}/llms-full.txt"))
else:
sys.exit(f"unknown raw target: {name} (try 'llms' or 'full')")
def main() -> None:
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
sub = p.add_subparsers(dest="cmd", required=True)
sp = sub.add_parser("topics", help="list known topic slugs")
sp.set_defaults(func=cmd_topics)
sp = sub.add_parser("topic", help="fetch a single topic file")
sp.add_argument("slug", help="topic slug, e.g. biology, materials-science")
sp.add_argument("--filter", choices=["datasets", "models", "blogs"], help="restrict to one section")
sp.add_argument("--tag", help="restrict to entries with a tag substring (case-insensitive)")
sp.add_argument("--format", choices=["markdown", "json"], default="markdown")
sp.set_defaults(func=cmd_topic)
sp = sub.add_parser("all", help="fetch the full llms-full.txt")
sp.add_argument("--raw", action="store_true", help="print the raw file untouched")
sp.add_argument("--filter", choices=["datasets", "models", "blogs"], help="restrict to one section")
sp.add_argument("--tag", help="restrict to entries with a tag substring")
sp.add_argument("--format", choices=["markdown", "json"], default="markdown")
sp.set_defaults(func=cmd_all)
sp = sub.add_parser("search", help="substring search across the full catalog")
sp.add_argument("query")
sp.add_argument("--filter", choices=["datasets", "models", "blogs"], help="restrict to one section")
sp.add_argument("--tag", help="restrict to entries with a tag substring")
sp.add_argument("--format", choices=["markdown", "json"], default="markdown")
sp.set_defaults(func=cmd_search)
sp = sub.add_parser("raw", help="dump a raw catalog file")
sp.add_argument("name", help="'llms' for llms.txt, 'full' for llms-full.txt")
sp.set_defaults(func=cmd_raw)
args = p.parse_args()
args.func(args)
if __name__ == "__main__":
main()
Related skills
How it compares
Choose hugging-science for domain-scientific HF discovery; use generic HF docs when the task is routine NLP unrelated to science catalogs.
FAQ
What scientific domains does hugging-science cover?
hugging-science covers biology, chemistry, physics, astronomy, climate, genomics, materials, medicine, ecology, energy, engineering, math, drug discovery, protein design, weather modeling, theorem proving, single-cell work, and PDE solving per the skill description.
Which Hugging Face libraries does hugging-science use?
hugging-science guides agents through Hugging Face datasets, transformers, the HF Inference API, and gradio_client when loading models, running inference, or opening interactive Spaces, plus methodology citations for reproducibility.
What version is the hugging-science skill?
hugging-science metadata lists version 1.1 with skill-author K-Dense Inc. in the scientific-agent-skills repository, marking the current catalog index generation referenced by the skill file.