Building With Llms

# Building with LLMs - All Guest Insights

*60 guests, 110 mentions*

---

## Albert Cheng
*Albert Cheng*

> "We're working on training some of these Slack bots to essentially be the first party provider of a lot of these answers [SQL queries], which makes the company as a whole lot more data informed."

**Insight:** Using LLMs for text-to-SQL can democratize data access and reduce the burden on data analysts for ad-hoc questions.

**Tactical advice:**
- Implement a Slack bot that translates natural language questions into SQL queries for the team.

*Timestamp: 00:17:07*

---

> "We've invested a bit in at least carving out the main screens of our product experience... and building essentially AI prototypes of those using tools like a V0 or a Lovable. And when you have those foundational pieces, you can then share them with the rest of the company and they can use that as a starting point."

**Insight:** AI prototyping tools can dramatically accelerate the 'explore' phase of product development by making ideas clickable and discussable instantly.

**Tactical advice:**
- Use tools like V0 or Lovable to create functional prototypes of core product screens for faster feedback.

*Timestamp: 00:18:52*


## Alexander Embiricos
*Alexander Embiricos*

> "For a model to work continuously for that amount of time, it's going to exceed its context window. And so we have a solution for that, which we call compaction. But compaction is actually a feature that uses all three layers of that stack. So you need to have a model that has a concept of compaction... at the API layer, you need an API that understands this concept... and at the harness layer, you need a harness that can prepare the payload."

**Insight:** Enabling long-running agent tasks requires a 'compaction' strategy coordinated across the model, API, and application harness to manage context window limits.

**Tactical advice:**
- Optimize the full stack (model, API, and harness) in parallel rather than treating the model as a black box
- Implement compaction to allow agents to maintain state over long durations

*Timestamp: 00:23:28*


## Aishwarya Naresh Reganti + Kiriti Badam
*Aishwarya Naresh Reganti + Kiriti Badam*

> "LLMs are pretty sensitive to prompt phrasings and they're pretty much black boxes. So you don't even know how the output surface will look like. So you don't know how the user might behave with your product, and you also don't know how the LLM might respond to that."

**Insight:** The black-box nature of LLMs makes predicting the output surface difficult, requiring builders to anticipate a wide range of non-deterministic behaviors.

**Tactical advice:**
- Design for a fluid interface where user intent can be communicated in infinite ways.
- Prepare for sensitivity in prompt phrasing that can significantly alter outputs.

*Timestamp: 00:08:01*

---

> "I feel like kind of misunderstood is the concept of multi-agents. People have this notion of, 'I have this incredibly complex problem. Now I'm going to break it down into, hey, you are this agent. Take care of this. You're this agent. Take care of this.' And now if I somehow connect all of these agents, they think they're the agent utopia and it's never the case... letting the agents communicate in terms of peer-to-peer kind of protocol... is incredibly hard to control."

**Insight:** Peer-to-peer multi-agent systems are often less effective and harder to control than a single supervisor agent orchestrating sub-tasks.

**Tactical advice:**
- Use a supervisor agent model to manage sub-agents rather than a decentralized 'gossip protocol.'
- Limit the ways a multi-agent system can go off-track by centralizing orchestration.

*Timestamp: 01:01:34*


## Alex Komoroske
*Alex Komoroske*

> "I use it to think through problems. And so like when I'm trying to name a concept or get a handle on a few different ways of looking at something, just saying, 'Here's what's in my brain about this topic right now. Here's some releva