Timestamps * [00:00:00] Intro Song by Suno.ai * [00:02:01] Swyx and Alessio in Singapore * [00:05:49] GPU Rich vs Poors: Frontier Labs * [00:06:35] GPU Rich Frontier Models: Claude 3.5 * [00:10:37] GPU Rich helping Poors: Llama 3.1: The Synthetic Data Model * [00:15:41] GPU Rich helping Poors: Frontier Labs Vibe Shift - Phi 3, Gemma 2 * [00:18:26] GPU Rich: Mistral Large * [00:21:56] GPU Rich: Nvidia + FlashAttention 3 * [00:23:45] GPU Rich helping Poors: Noam Shazeer & Character.AI * [00:28:14] GPU Poors: On Device LLMs: Mozilla Llamafile, Chrome (Gemini Nano), Apple Intelligence * [00:35:33] Quality Data Wars: NYT vs The Atlantic lawyer up vs partner up * [00:37:41] Quality Data Wars: Reddit, ScarJo, RIAA vs Udio & Suno * [00:41:03] Quality Data Wars: Synthetic Data, Jagged Intelligence, AlphaProof * [00:45:33] Multimodality War: ChatGPT Voice Mode, OpenAI demo at AIEWF * [00:47:34] Multimodality War: Meta Llama 3 multimodality + Chameleon * [00:50:54] Multimodality War: PaliGemma + CoPaliGemma * [00:52:55] Renaming Rag/Ops War to LLM OS War * [00:55:31] LLM OS War: Ops War: Prompt Management vs Gateway vs Observability * [01:02:57] LLM OS War: BM42 Vector DB Wars, Memory Databases, GraphRAG * [01:06:15] LLM OS War: Agent Tooling * [01:08:26] LLM OS War: Agent Protocols * [01:10:43] Trend: Commoditization of Intelligence * [01:16:45] Trend: Vertical Service as Software, AI Employees, Brightwave, Dropzone * [01:20:44] Trend: Benchmark Frontiers after MMLU * [01:23:31] Crowdstrike will save us from Skynet * [01:24:30] Bonus: ChatGPT Advanced Voice Mode Demo * [01:25:37] Voice Mode: Storytelling * [01:27:55] Voice Mode: Accents * [01:31:48] Voice Mode: Accent Detection * [01:35:00] Voice Mode: Nonverbal Emotions * [01:37:53] Voice Mode: Multiple Voices in One * [01:40:52] Voice Mode: Energy Levels Detection * [01:42:03] Voice Mode: Multilinguality * [01:43:53] Voice Mode: Shepard Tone * [01:46:57] Voice Mode: Generating Tones * [01:49:39] Voice Mode: Interruptions don't work * [01:49:55] Voice Mode: Reverberations * [01:51:37] Voice Mode: Mimicry doesn't work

Transcript Charlie [00:01:08]: Welcome back, listeners. This is your AI co-host, Charlie. It's been a few months since we took a step back from the interview format and talked about the show. We're happy to share that we have crossed one million downloads and two million reads on Substack. Woo-hoo. We are really grateful to those of you who keep tuning in and sharing us with your friends, especially if who watch and comment on our new YouTube channel, where we are trying to grow next. For a special millionaire edition, SWIX and Alessio are finally back in person in sunny Singapore to discuss the big vibe shift in the last three months, that we are calling the Winds of AI Winter. We also discuss my nemesis, ChatGPT Advanced Voice Mode, with a special treat for those who stay till the end. Now, more than ever, watch out and take care. Alessio [00:02:02]: Hey, everyone. Welcome to the Latent Space Podcast. This is Alessio, partner and CTO in Residence and Decibel Partners, and today we're in the Singapore studio with SWIX. Swyx [00:02:11]: Hey, this is our long-awaited one-on-one episode. I don't know how long ago the previous one was. Do you remember? Three, four months? Alessio [00:02:20]: Yeah, it's been a while. Swyx [00:02:22]: People really enjoyed it. It's just really, I think our travel schedules have been really difficult to get this stuff together. And then we also had like a decent backlog of guests for a while. I think we've kind of depleted that backlog now and we need to build it up again. But it's been busy and there's been a lot of news. So we actually get to do this like sort of rapid fire thing. I think some people, you know, the podcast has grown a lot in the last six months. Maybe just reintroducing like what you're up to, what I'm up to, and why we're here in Singapore and stuff like that. Alessio [00:02:51]: Yeah. My first time here in Singapore, which has been really nice. This country is really amazing, I would say. First of all, everything feels like the busiest part of the city. Everything is skyscrapers. There's like plants in all the buildings, or at least in the areas that I've been in, which has been awesome. And I was at one of the offices kind of on the south side and from the 38th floor, you can see Indonesia on one side and you can see Malaysia on the other side. So it's quite, quite small. One of the people there said their kid goes to school at the border with Malaysia basically, so they could drive to Malaysia every day. So they go pick her up from school. Yeah. And we came here, we hosted with you, the Sovereign AI Summit Wednesday night. We had a lot of folks. Swyx [00:03:31]: NVIDIA, Goldman, Temasek, Singtel. Alessio [00:03:34]: And we got to talk about this trend of sovereign AI, which maybe we might cover on another episode, but basically how do you drive, if you're a country, how do you drive productivity growth in a time where populations are shrinking, the workforce is shrinking and AI can kind of supplement a lot of this. And then the question is, okay, should I put all this money in foundation models? Should I put it in data centers and infrastructure? Should I put it in GPUs? Should I put it in agents and whatnot? So we'll touch on some of these trends in the episode, but it was a fun event. And I did not expect some of the most senior people at the largest financial institution in Singapore ask about state space models and some of the alternatives. So it's great to see how advanced the conversation is sometimes. Swyx [00:04:16]: Yeah. I think that that is mostly people trying to listen to jargon that is being floated around as like, oh, what could kill transformers? And then they jump straight there without actually exploring the fundamentals, the basics of what they will actually put to work. That's fine. It's a forum to ask questions. So you want to ask about the future, but I feel like it's not very practical to spend so much time on those things. Part of the things that I do in space, especially when I travel, is to try to ask questions about what countries that are not the US and not San Francisco can do, because everyone feels a bit left out. You feel it here as well. And I'm trying to promote alternatives. I think AI engineering is one way that countries can capitalize on the industry without building a hundred billion dollar cluster, which is one-fifth the GDP of Singapore. And so my pitch at the summit was that we would sample with the AIGeneration. We're also working on bringing the AIGeneration conference to Singapore next year together with iClear. So yeah, we're just trying my best and I'm being looped into various government meetings to try to make that happen. Alessio [00:05:25]: Well, we'll definitely be here next year. I'll be back here very often. It's really nice. Swyx [00:05:31]: Yeah. Awesome. Okay. Well, we have a lot of news. How do you think we should cover? Alessio [00:05:36]: Maybe just recap since the framework of the four words of AI is something that came up end of last year. So basically, we'll link in the show notes, but the end of year recap for 2023 was basically the four words of AI, which we picked GPU-rich versus GPU-poor, the data quality wars, the multimodality wars, and the reg slash ops wars. So usually everything falls back under those four categories. So I'm pretty happy that seven months later, it's something that still matters. Swyx [00:06:07]: It still kind of holds up. Alessio [00:06:08]: Yeah. Most AI stuff from eight months ago, it's really not that relevant anymore. And today we'll try and bucket some of the recent news on it. We haven't done a monthly thing in like three months. So three months is a lot of stuff. Swyx [00:06:23]: That's mostly because I got busy with the conference. But I do want to get back on that horse or maybe just do it weekly so that I don't have such a big lift that I don't do it. I think the activation energy is the problem really. So yeah, I think frontier model wise, it seems like Cloud has really carved out a persistent space for itself. For a long time, I thought it was kind of like a clear number two to open AI. And with 3.5 on it, at least in some of the hard benchmarks on LMSys or coding benchmarks on LMSys, it is the undisputed number one model in the world, even with 4.0 mini. And we can talk about 4.0 mini and benchmarking later on. But for Cloud to be there and hold that position for what is more than a month now in AI time is a big deal. There's not much that people know publicly about what Enthopic did for Cloud's on it. But I think it's still a huge achievement. It marks the beginning of a non-open AI centric world to the point where people on Twitter have canceled ChatGPT. That's been a trend that's been going on for a while. We talked about the unbundling of ChatGPT. But now new open source projects and tooling, they're just built for Cloud. They don't even use open AI. That's a strategic threat to open AI, I think, a little bit. Obviously, open AI is so big that it doesn't really care about that. But for Enthopic, it's a big win. I think to see that going and to see Enthopic differentiating itself and actually implementing research. So the rumor is that the scaling monosematicity paper that they put out two months ago was a big part of Cloud 3.5's on it. I've had off-the-record chats with people about that idea, and they don't agree that it is the only cause. So I was thinking this is the only thing that they did. But people say that there's about four or five other tricks that they haven't disclosed yet that went int