Transcript [00:00] - The important is, is your code good? We care about [00:03] excellent code. We don't care who you are. Like maybe you're a dog. I don't [00:07] care, right? I don't care where you come from. I need to look at your code. [00:11] Oh, yeah, but I'm an engineer at this very large company in [00:15] Italy, in Germany, in the US. We don't care. We care about the [00:19] quality of your code because this is what defines our community [00:23] and which means that we have a lot of people who contribute who are some very different [00:27] backgrounds and very introverted. Sure. But that's okay, right? [00:31] - FFmpeg is probably one of the biggest CPU users in the world. Everything we've just [00:35] said in the past couple of minutes, every sentence is someone's [00:39] lifetime's work. There are books about every sentence. So the level of complexity [00:44] in many cases is inordinate. [00:45] - FFmpeg has one hundred thousand lines of assembly for all the codecs. [00:50] - For all codecs. Mm-hmm. [00:50] - And just this one has two hundred and forty thousand. Every cycle [00:55] matters. We are talking about probably three billion [00:58] devices which are going to decode video nonstop because, for [01:02] example, thirty percent of the video from Netflix is now in AV1, [01:06] fifty percent of YouTube. [01:07] - This is what peak video codecs should look like. [01:11] Seventy-nine point nine percent assembly, nineteen point six percent C, [01:15] and zero point five percent other. [01:18] - And what's incredible is with those tweets, which is factual, people get crazy. [01:25] - For the last two years, they go crazy. No, intrinsics is fine. The compiler- [01:28] - You can optimize your compiler. Auto-vectorization, it's your fault. You don't [01:32] understand. And we've tried that forever, right? [01:35] - For two years, and two years later, showing hundreds of examples [01:40] of handwritten assembly. No, no, no, you're doing it wrong. The compiler can do this. [01:43] The intelligence agencies tried to, like, say, "Can you put a backdoor in VLC?" [01:48] - Yes. Two of them. [01:50] - Well, what did you say? [01:51] - No. Well, I was a lot less polite. [01:54] - Basically saying, "Hell no." [01:56] - Like, if we had to compromise our software, we would shut it down. This is clear. [02:00] - Any tweets Kieran, you regret? [02:04] - Tweets I regret? [02:05] - Or is it like that, how does the French song go? Regret nothing. [02:08] - Don't regret anything. No, it's because regrets are attacks on your mind. [02:17] - The following is a conversation all about FFmpeg and VLC [02:21] with Jean-Baptiste Kempf and Kieran Kunhya. [02:25] FFmpeg is an open source software system that is [02:30] the invisible backbone behind YouTube, Netflix, Chrome, [02:33] VLC, Discord, and basically every platform that [02:37] touches video or audio on the internet. [02:41] It can decode, encode, transcode, stream, and [02:45] play almost any video or audio format ever [02:49] created. To me, it is one of the most incredible software [02:53] systems ever developed, and it's all done by volunteers. [02:59] VLC is also a legendary piece of software. It is an [03:03] open source media player that plays basically anything you throw at [03:07] it, any format, any platform, no ads, no tracking. [03:11] It has been downloaded over six billion times, and [03:15] again, for me, it has been one of my favorite pieces of [03:18] software ever, with the most legendary logo, which I, [03:22] of course, had to honor in this conversation by wearing the [03:27] VLC traffic cone hat the whole time. [03:31] So again, above all else, thank you to the incredible volunteer [03:35] engineers who put their heart and soul into this code that has been [03:39] used and loved by billions of people. Thank you. [03:43] And about the two great engineers and human beings I'm talking to in this episode, [03:50] Jean-Baptiste is the president of VideoLAN and [03:54] is a key figure behind VLC and FFmpeg. Kieran is a longtime codec engineer, FFmpeg [04:01] contributor, and the man behind the now infamous [04:05] FFmpeg account on Twitter/X that I recommend [04:09] everybody follow for the memes and for the [04:13] unapologetic celebration of open source and great low-level software engineering. [04:20] Let me also say that it's inspiring and humbling that [04:24] so much of modern civilization rests on software built by [04:28] people who are not chasing fame or money, but are [04:31] obsessed with the craft of engineering. [04:34] We live in a world where billions of people consume video every day without [04:38] ever thinking about the invisible machinery underneath it. But that [04:42] machinery matters. Open source infrastructure matters. [04:46] It is one of the great examples of human beings quietly collaborating [04:50] across borders to build something useful, durable, and elegant for the rest of us. [04:57] And so this conversation is not just about codecs and [05:01] media pipelines. It is also about the deeper spirit of [05:05] engineering and generosity that makes projects like FFmpeg possible. [05:11] Again, I can never say it enough. Thank you. [05:16] This is the Lex Fridman Podcast. To support it, please check out our [05:20] sponsors in the description, where you can also find links [05:23] to contact me, ask questions, give feedback, and so on. [05:28] And now, dear friends, here's Jean-Baptiste Kempf and Kieran Kunhya. [05:35] So the legend goes VLC can open everything. What's the [05:39] weirdest thing that you know that it can open? [05:43] - You know, there is a ton of people who are using VLC to record VHS [05:46] videos, right? Like, it's just like you plug it with a capture card and you [05:50] can basically record VHS video. [05:52] - Well, how does that work? [05:53] - Basically, it's, you know, those type of capture card where you can put a Peritel [05:57] in or- ... or RCA, and you put that, and actually VLC can play those type of cards, [06:02] and there is a module which allows to control directly some of those [06:06] VCR camcorders. We support DVD audios lately, right? We spent [06:10] the summer working on DVD-Audio support, and [06:14] like there is no, no one's making any DVD audio support. There is a custom [06:17] encryption schemes. [06:19] - What about Lucasfilm? [06:20] - Oh, yeah, and there is of course all the weird codecs support, game codecs supported [06:23] by FFmpeg. [06:25] - The one Star Wars video game, the first ten- second opening sequence, someone has gone and [06:29] implemented that and made sure that's bit exact on one disc that existed at [06:33] one time of one little sequence in the game. [06:36] - And then funnily, there was a... At one VideoLAN conference, we made a [06:40] competition to make the weirdest and most horrible file ever [06:44] ... and see if VLC could play it. [06:46] - What did it end up being? What's the file? [06:48] - It was an MKV file made by Derek- [06:52] ... which each of the frame was changing resolution, aspect [06:56] ratio- ... rotation and it was like- [06:59] - Did it work? [07:00] - Yes. And there was another one where [07:03] the whole video was actually animated subtitles, right? SSA, right? So- [07:08] - Yeah. I remember that, yeah [07:08] - ... each, this one was- [07:10] And so each frame was a black frame, but on top of that there was a, [07:14] a subtitle that was animated for each frame. [07:16] - There was a file that's a valid ZIP and a valid MP3 at the same time or something like that, [07:20] so. [07:20] - So yeah, we'd made a competition of stupid files. [07:23] - And it worked. It opened all of the stupid files. [07:27] - Yes. [07:27] - By the way, [07:28] For people who are not familiar, I am wearing a hat. Would it be fair to say this [07:32] is the best worst logo of all time, the cone? [07:36] - Yeah, by far, right? The logo of VLC is so iconic, right? [07:40] Like we are a team with a small number of people and the icon [07:44] is known everywhere. I go to middle of nowhere in India or in [07:47] China, people know the cone, right? And 25% of [07:51] the website traffic that comes to our main website is [07:55] cone player, right? So, so many people don't know VLC, right? They know the cone player. [07:59] - That's the thing they Google for is cone player. [08:01] - Yeah. They go on Google and they put cone player and they download VLC, right? [08:05] So that's iconic. And once we tried to change it as a joke, right? We said [08:09] it was going to be a type of uh, [08:12] caterpillar construction and we said that during April 1st- [08:18] ... and we had around 10,000 emails saying, "No, don't change the logo," and so on, right? [08:21] So it's so iconic, right? It's so distinctive, right? If you want to do a [08:25] video player, you're going to put a play button on a TV, right? And that's a YouTube, [08:29] YouTube logo, right? It's unoriginal. This one is orange, right? [08:33] - Yeah. [08:33] - It's very bright and it's weird. [08:37] - And it's ridiculous and it's absurd and it's hilarious. It becomes meme and meme becomes [08:41] culture. Yeah. [08:41] - And you keep it and you know about it and you know that in 20 years, like you [08:45] still have, going to have the cones and remember, oh yeah, that was a video player. [08:49] - Yeah. And we'll talk about, you know, the mission of FFmpeg being [08:53] a kinda the archival aspect of it. So you can think about 1,000 years from [08:57] now we'll have all these videos that only VLC can open. Humans, [09:01] human civilization has already destroyed itself multiple times and the [09:05] only thing that will remain is this like, you know, the cockroaches will be [09:09] crawling around and it'll be the VLC logo- ... with some of the archival [09:13] footage that VLC can open. And the aliens will show up and they'll press [09:17] play and they'll get to see it all 'cause- [09:18] - Well, really, really hope so, right? But there is also so many memes where people say, "Well, I'm sure I can put [09:22] a pancake inside my DVD drive and VLC will play it." Like- [09:25] - Can they? [09:26] - No, we tried. It doesn't. Um- [09:27] - Doesn't. [09:28] - ... but we actually have a video of us trying that. Didn't work. [09:31] - A codec for physical reality, I don't know what that would even look like. [09:34] - There was a guy who did that, right? He printed a small cone, right? [09:38] Like the ones we distribute as goodies and inside he put an [09:41] RFID chip which was his way of playing a movie, right? And so he- [09:46] ... put this on a RFID player and when he put that it was playing like The [09:50] Last Star Wars and so on. So instead of having like DVD boxes, he had like [09:54] VLC cones all around and he plugged that and that was like physical objects. [09:59] - So the thing that we're talking about is everything around video [10:03] codecs, video encoding, video decoding, video streaming, video [10:07] player client that I'm wearing on my head, the entire [10:10] ecosystem enabling free media. We'll talk about FFmpeg, we'll talk [10:14] about VideoLAN, VLC and all the other incredible video [10:18] technology that is used probably by billions of people. So JB, [10:25] you're the lead developer behind the legendary VLC player. [10:29] Kieran, amongst many other things, you're lead developer behind the legendary [10:33] FFmpeg handle on Twitter. And both of you have spicy opinions I would say. [10:40] So today we wanna talk about FFmpeg and VLC. [10:45] For context for people who are not aware and I'm sure [10:49] basically everybody listening to this have used these two [10:53] technologies probably regularly without knowing it. [10:57] So FFmpeg underlies basically most video on the internet [11:01] including YouTube, Netflix, Chrome, Firefox, of course [11:05] VLC and countless other video platforms. It [11:08] is estimated that over 90% of video processing workflows online and [11:12] offline involve FFmpeg. VLC has been downloaded at least 6.5 billion times. But [11:22] likely that number, 'cause it's impossible to really count the number [11:26] is much higher than that. Virtually any operating system [11:30] supports virtually any media format. [11:35] The limitation being it can't open pancakes. So, [11:39] Can we just lay out some of the basics to help people [11:43] understand what's involved in all of this? So when we press [11:47] play on a video player like VLC, what happens? What-- [11:52] How does it go from the file or the stream to the [11:56] pixels on the screen and the sound on the speaker? What are the big [11:59] stages to be aware of? [12:01] - So there are several stages, right? The first stage is to get from [12:05] an address, right, which is the type of URL, to [12:09] give you a byte of streams, right? So this would be, for example, HTTP, [12:14] file, DVD, right? You give the path to the media, and it gives [12:17] you a stream of data. [12:19] - The stream needs to be cut up by what's known as the container, the demultiplexer or [12:23] demux. We'll try and keep the jargon light throughout this, but it [12:27] needs to go and start demarcating video and audio frames. So it just gets data from the [12:31] operating system blocks at a time and needs to start cutting these frames up [12:35] into compressed data. It then needs to start doing simple [12:39] parsing of the video frames- ... mainly to figure out whether that codec is GPU [12:44] decodable or needs to fall back to software. We're very sort [12:48] of used to assuming the GPU will play all of these things. There'll be hardware [12:52] acceleration. I think it's up to forty- five percent of files are not GPU [12:55] decodable. So these need to be probed. They need to be detected. There can [12:59] be variants of a given codec, some of which are decodable on the [13:03] GPU. Different vendors of GPU might have different capabilities, [13:07] so those need to be detected. So if it's GPU capable, you pass it through [13:11] to the GPU black box. So now if there's a software fallback, [13:15] that means in the beginning is to first do deentropy [13:19] coding, so removing the mathematical coding of the bit stream. So this [13:22] uses capabilities such as Huffman coding or arithmetic [13:25] coding to actually decompress the mathematical layer of the bit stream. [13:30] We then need to start reading the syntax elements for intra prediction. So intra [13:34] prediction are like still images of the video, so your I-frames. [13:40] So this works and operates in the spatial domain. So you do your intra prediction in spatial [13:44] domain. You have a residual because your prediction isn't quite [13:47] matching that of reality. So you've made a prediction, but then there's a little bit [13:51] left, and that's what's known as the residual. This is stored in the frequency [13:55] domain, and these are quantized to decompact their space. [13:59] We then need to do the inverse transform to bring them back to the [14:03] spatial domain and apply these residuals. [14:05] - So a lot of the process of the decoding is this thing is compressed. [14:10] And you have to predict the highest quality thing that's [14:14] supposed to go there. I-frame- ... is the best representation you have spatially. [14:19] And then there's a lot of temporal compression that can happen [14:23] depending on the codec, and then you're predicting. You're predicting what [14:26] the reality that was captured in this rawest form. [14:29] - Yeah, because what people don't realize is that the compression on [14:33] video and audio is [14:36] one hundred times, right? Like, people don't realize how compressed we, we [14:40] do, right? For audio, you move, you compress by, when you go from [14:44] normal audio to MP3, you compress by ten times, right? When, when you [14:48] move to video, you need one hundred times, two hundred times, right? So you need [14:51] to remove all the details, but that you don't care about [14:56] because all the compressions that we do, and that's very important, people forget about [14:59] that, is to be viewed by humans, right? [15:02] So all the codecs, either for audio, mimic basically how your [15:06] ear works, right? And a lot of things about, like, the [15:09] response on the ear and same for your eyes, right? And so, for [15:14] example, on video, we don't work on RGB, right? Everyone expects [15:18] to work in RGB. We don't, right? We move to YUV, which is [15:22] basically one is luminance, brightness, and the other are colors. [15:26] And this matches your eyes, where inside your eyes you have the cones and the buttons, right? [15:30] With some of them look on brightness and more on, on the other on colors, right? So [15:34] we need to compress a lot, and so we need to degrade. But in order to [15:38] degrade, we need to match the human perception, and this is why it's so [15:42] difficult. And then we need to use the maximum power, [15:46] mathematical power, very complex technologies. We move to the [15:49] frequency domain, as Kieran said. We do a ton of [15:53] dequantizing, in order to get the best compression, but it still looks good. [15:58] - You're trying to compress in order to maximize the highest [16:02] quality thing for human perception. [16:04] - That is correct. And this is very important, right? Compression is not like [16:08] a ZIP, right? A ZIP, you have data in, you get data out, [16:12] right? And you try with all the ZIP compression to arrive with [16:17] the limit. Here we are degrading the signal, right? And so we [16:20] need to degrade both the audio and the video signal in the best way [16:24] possible. And we can do that, but it involves, first, [16:28] a lot of theoretical knowledge about how the eye works, but [16:32] it, a lot of mathematical change, a lot of mathematical tricks, [16:36] right? For example, when you move to RGB and you do go to [16:40] YUV, for example, what we do very often is that [16:44] we scale down the resolution of the color compared to the brightness. [16:48] And most of the time, just this without compression, it [16:51] divides the size by two, but most people don't see it, [16:55] right? And so on and so on, right? And then you go [16:59] to very complex mathematical change. So of course [17:04] Fourier transforms, which de facto are not Fourier transforms, they are like [17:07] discrete cosine transform, but that's the same idea. So frequency domain [17:11] we split the video by blocks, right? So that's why when it's [17:15] wrongly decoded, you see those blocks and badly encoded, you see those blocks, [17:19] and so on, to arrive to compression states that are [17:23] insanely high, right? And each generation of the codec is like thirty percent less- [17:29] ... for the same quality, right? And this requires amount of [17:32] power of computational power that is huge. [17:35] - No, no, but you should elaborate. It's thirty percent better, but an order of [17:39] magnitude, perhaps, perhaps even two orders of magnitude more [17:42] compression power. That's the big difference. [17:45] - What do you mean by compression power? [17:47] - Sorry, CPU power to achieve that level of compression. [17:49] - Oh, yeah. So you have to be able to leverage the CPU and sometimes GPU, like [17:53] you mentioned. And then we should mention that a lot of this [17:56] programming is done at the lowest possible- [18:01] ... stack, whether it's C and of course, as the legendary— [18:05] ... Twitter handle re-emphasizes over and over, a lot of assembly. [18:10] - So what happens globally is that you have an address, right? Which gives you [18:14] with the operating system, a stream of bytes, a stream of data, right? And this is the first step. [18:18] And the second step arises with demuxing, where you're going to separate audio, [18:22] video, subtitle in type of different tracks. And then on each of those [18:26] tracks, you're going to decompress them, decode them, either audio with an [18:30] audio codec, video to video codec, and subtitle to subtitle codec. [18:34] And once you've decompressed those type of things, you have raw images, [18:38] raw, and then you're going to talk with your graphics card and your screen and [18:42] display that. And same for the audio, you're going to talk to your audio card, which [18:46] then is going to go in analog to your audio speakers. [18:50] - And everything we've just said in the past couple of minutes, [18:53] every sentence is someone's lifetime's work. There are books about- [18:56] ... every sentence. So the level of complexity [18:59] in many cases is inordinate. You know, it's, it's... Every sentence [19:03] has thousands of people working on this- [19:06] ... in industry as a whole, books written about [19:10] it. So there's a lot of detail, there's a lot of subtleties, there's a [19:14] lot of both academic and practical realities, both of which matter. [19:20] - Uh, we mentioned codecs, but I don't think you mentioned containers. [19:24] So what, what are the actual containers [19:28] for some of the stuff we're talking about? So people are familiar with MP4, [19:33] uh, MOV, MKV. So anyway, what are containers versus the thing that goes inside? [19:41] - So the container is what we call also the muxer, right? When I say demuxing, it [19:45] means decontainerizing, right? So actually, if you look, [19:49] mux means multiplexer and demultiplexer, right? [19:53] Mux and demux are those. And same, a codec is actually coder, decoder, right? [19:58] Um, and so containers are this collection [20:02] of multiple tracks, right? So it's a, what normal people call the file [20:06] format, but it's a bit more, um, subtle than that. [20:10] But the most known one, of course, is MP4, but when I started, [20:14] it was AVI, right? AVI was the, the video format from- [20:17] ... from Microsoft, and MOV, M-O-V, which became [20:21] MP4, was a format from Apple. In the open source community [20:25] one of the person that is still active on VideoLAN is called Steve Lhomme and started, [20:29] This Matroska format, which is, like, a bit more complex and [20:33] more future-proof. And there are so many others. [20:38] - So, I mean, there's a, it's a pretty common thing, and maybe it'll even happen in [20:42] this conversation, that people confuse container [20:45] and the codec, right? So confuse MP4 and H.264, for [20:49] example. Is that a horrible violation? [20:51] - No, it's not, because technically the name of H.264 is MPEG-4 Part 10. Because [20:59] MPEG-4 is actually a meta specification which has [21:03] several things in it, right? There is the Part 2. [21:07] so there is, like, audio codecs, right? AAC de facto is MP4 audio- [21:11] ... something. There are actually several video codecs, right, inside the [21:15] MPEG-4 specification. One of them is MPEG-4 Part 10, [21:19] called also AVC, called also H.264. Right? So [21:24] it's completely the fault of the industry to make things difficult [21:28] to understand. So that's very difficult so that people [21:31] then don't understand why sometimes you talk about MPEG-4 Part 10, [21:35] where you mean H.264, and why it's not MP4. [21:38] - So you can technically shove in all kinds of different codecs inside [21:42] containers and horribly so. [21:44] - But broadly speaking, though, MP4 is understood to generally [21:48] be H.264 plus AAC audio. 99% of the time that's that, and that, [21:55] the rest are de minimis, the small effects, you know, edge effects really compared to that. [21:58] So it's not the end of the world. There, there are people [22:02] who do get annoyed by that. But also in reality, something like VLC, just to point out, [22:06] the file may say .MP4, but it may be something completely different, and that's [22:10] one of the challenges both FFmpeg and VLC have is the real world is a [22:14] completely different place to a three- letter file format. [22:17] - And this is very important to say, right? Like, for example, in VLC and in FFmpeg, [22:21] we discard the file format, right? We look [22:25] into the file to understand what's in it because so many [22:28] people, like, they say, "Oh, it's a video, it should must be MP4," but technically it's [22:32] an MOV or maybe it's a MKV, right? So we [22:36] analyze in real time everything that we have, and we don't trust- ... the format. [22:42] - So what information does the fact that it's .MP4 give you? [22:45] - It helps, right? It gives you a hint, right? Just like, oh, it's [22:49] finished by .MP4. I will start first by opening, [22:54] probing it with the MP4 container demuxer [22:58] to see, well, it should be that. But I don't trust it, and if I'm lost, I say, "Okay, [23:02] maybe I'm going to try it." So it bumps the priority of the module. [23:06] - So how do you get to... just to take a bit of a tangent there. [23:11] You know, the dumb thing is if you try the MP4, [23:15] but it turns out it's a different codec than you would have expected, [23:19] Most players just break there. [23:21] - Yes. [23:22] - Yes. [23:22] - And so how do you not break? There's just philosophically, I'm sure [23:26] there's a bunch of stumbling blocks along the way where it's [23:30] easy to just break and stop, freak out. That's it. How does VLC not? [23:34] - This is why VLC is popular. But the reason is because [23:38] actually VLC was, is just a client of a streaming [23:42] solution called VideoLAN from a very long time [23:46] ago, from the late '90s. And when you're playing video [23:50] which are on UDP, right, in network, they might be damaged, right? [23:54] So you don't trust your inputs, and this is very important into the security is that you [23:58] don't trust your inputs. So everything in VLC is prepared to work with broken files. [24:06] And it's a philosophical idea from the beginning, and everything [24:10] is engineered into that. And it's a culture, right? And so, [24:14] for example... And VLC became very popular on that because a long time ago when people [24:18] were pirating content which they do a lot less today- [24:23] - And none of us ever have- [24:24] - No, of course not. Um— ... the metadata to play some files like AVI is at, [24:30] at the end of the file, right? And when you're downloading, you don't have that, right? [24:34] So VLC was just like, "Hey, this file is broken, but I'm still going to try to [24:38] interpret it," and this was very useful. [24:41] - We hinted at the awesomeness of the various different stages. [24:45] We hinted at the awesomeness of codecs, the depth and the richness and the complexity [24:49] of everything involved there. What— Let's try to define [24:53] what is a video codec? What's involved there? What does it mean to [24:57] compress something? You already started to hint at it— [24:59] ... but can we elaborate a little bit more? [25:01] - So there's a huge amount of redundancy in any video both [25:05] spatial and temporal, and the point of any video codec is to remove [25:09] this redundant data, use mathematical properties as part of this reduction [25:13] process. So more often than not, using several orders of magnitude more [25:17] compute to compress because that's more costly versus both costly [25:21] both financially and in CPU resources— [25:23] ... versus the decompression. So it's asymmetric in that respect. Uh, [25:27] often the case because compression is done once, but there could be lots of viewers of [25:31] another file. So to take that information and compress it by [25:36] 100x, 200x, removing redundant information and [25:40] using mathematical properties to make that small, but also have properties [25:44] such as error resilience. So as, as JB suggested, [25:48] VLC in the beginning was, was used to play UDP network feeds, and UDP [25:51] network feeds lose packets. And so some of the design goals of a [25:55] codec is also to be recoverable. [25:58] You need to actually be able to join a stream. It's not necessarily a file. You need to join, [26:03] get on the decoding process, and start decoding. [26:06] - And, and to give a more image to people [26:10] who are not familiar, right? Like, when you're going to see any type of movie, right? [26:14] You're going to see the camera is going to pan, right, and travel. And [26:18] you realize that, for example, all the background is the same from, for, like, [26:22] a minute, right? Or— [26:23] ... thirty seconds, right? So you can reuse the cloud that you see uh, on [26:27] the background, you can reuse that from a frame to another, right? [26:31] And so it's, gets the more, the more [26:35] memory you have, the more power, the more comparisons you can make, right? And so [26:39] the more compressed you can be. And most of the modern codecs are [26:43] basically doing that. [26:44] - So just to make it even more explicit. So what is video? Video is a [26:50] bunch of pixels often RGB. [26:54] You have three values, and you have a grid of pixels, and you have, [26:58] let's say, twenty-four or thirty or sixty, [27:02] frames a second, and you just have all these pixels [27:05] repeating and showing different stuff- [27:08] ... thirty times a second. And so the question, the philosophical, [27:12] the technical question is, how can I compress all of that, [27:17] store all of that at 100x? [27:20] - Yep. Or 1,000x, right? [27:22] - 1,000x. [27:22] - The target is 1,000x, right? [27:24] - And the goal is when you say redundancy, what is redundant? Meaning stuff at best [27:32] that humans wouldn't notice if it was missing. [27:35] - So for example, you have a picture of a cloud, right? And from the [27:39] next frame, they're still going to be the same cloud, so it's redundant. You could just put it once and [27:43] not do it, right? Or you have a black background behind me, for [27:47] example. The black is the same on the whole picture, right? So you can say, "Well, you know, in [27:51] this picture, take the pixels that you have on the top left and the one [27:55] on the top right. I'm not going to give the value. I'm just going to tell you it's the same at the top [27:58] left." And then you can say for frame one reuse [28:02] something from the previous frame or the previous, previous frame, and so on and [28:06] so on, right? So you could... Basically, it's [28:10] unlimited, but then it's limited in terms of memory or in terms [28:14] of compute power. Because, for example, if you need to compare pixels [28:17] on two hundred frames in the past on 4K resolutions, it's a huge amount of compute. [28:26] - And then when you're showing it, you have to do the decompress of all of that. [28:30] So is it the codec, the, has the [28:33] encoding and the decoding is a coupled process that you're developing? [28:37] - Yes, exactly, right. And those are two different [28:41] trade-offs, right? Are you going to compress more? But then it might be [28:45] more difficult to decode. Are you going to [28:49] comp- to make it a codec that is more complex to encode and easier to [28:53] decode? Are you going to make a codec that is easier to encode because you need to be [28:56] fast, but then the, the client side, the, the player is going to spend more [29:00] time? That's why you have so many different type of codecs, is that it's not [29:05] always easy. And to make it even more complex, modern codecs [29:09] like AV1, AV2, or VVC are actually not codecs. They are [29:12] a collection of tools, right? There are multiple tools, multiple [29:16] codecs in the same codec to, depending on the image, get the more compression. [29:21] - So just to elaborate, codecs like AV1, VVC [29:26] have a much wider, have a wide audience. It could be a screen share [29:30] content, it could be video, it could be animation. All of these [29:34] require different coding tools. So [29:38] what happens these days is a collection of tools are put in and called [29:42] AV1 and called AV2, called VVC to allow for different use [29:45] cases. So you may be on Zoom and sharing your PowerPoint, [29:49] and then you need to show the audience a video. That codec needs to [29:53] start changing its tool set depending on the content to compress in a different way. [29:59] - And like you said, there's a bunch of incredible engineers behind each part of [30:03] that, each part of the tools that make up AV1, for example. [30:06] Uh, so we've kind of danced around it. We talked about VLC, [30:10] the logo, the hat. Let's talk about FFmpeg. What, what is FFmpeg exactly? [30:17] - FFmpeg is basically the low-level libraries for codecs, so compressions [30:26] and decompression, muxers and demuxers, and filters. It's— [30:30] The core is this, and then you have several tools which allow you to create a [30:34] type of pipeline to process any type of video files. [30:38] And it's used as a library absolutely inside [30:41] everything from VLC to Chrome to your smart TVs, to [30:46] basically any video that you see online you usually use [30:49] FFmpeg. And FFmpeg in it has all those [30:53] type of tools, and sometimes depend on other libraries like [30:57] x264, libvpx, and others, right? So it's really now the de facto tool to process [31:05] images. [31:06] - From a philosophical level, I think it's incredible that your home [31:10] videos, your grandmother's home videos and trillion-dollar corporations [31:15] effectively are on a level playing field using the same technology stack. [31:21] It's— it wouldn't be a surprise, you know, these big companies just have three [31:24] thousand-line FFmpeg commands. [31:27] There are some that use the API, but there are some that just have long command lines. [31:31] - So yeah, there's a bunch of tools, like literally command line tool, [31:35] FFmpeg, of course, FFprobe. There's libraries, libavcodec, libavformat, libavfilter. [31:43] But the FFmpeg on the command line— [31:47] is, like, legendary because you can cut— Like, there's so many [31:51] parameters. You can customize everything to hell. [31:53] - It's a language. It's an actual language. [31:55] - It's an actual— yeah, you could think of it as a programming language. [31:57] - Yeah, of course, I'm sure. Because— so most of the people, they're going to take FFmpeg, [32:02] file in, file out, and specify the format, right? But you can-- We've seen [32:07] thousands of characters, and we've seen also, like, people, like, doing [32:11] programming generation of command lines [32:14] to make FFmpeg. There is a ton of people who are using AI to generate [32:18] command lines for FFmpeg because you have no idea what it is. But you can do- specify so [32:22] many filters right on the command line, right? So [32:26] FFmpeg is this collection of toolbox for multimedia processing that everyone uses. [32:34] And everyone that is watching your videos is also using it, right? You're on [32:38] YouTube. Well, it's FFmpeg on the client side. Well, the your [32:42] server side, on the server side. The client side is probably Chrome. Well, you're using [32:45] FFmpeg also. [32:46] And you're using OBS to record. Well, it's FFmpeg, right? You're using a ton [32:50] of important, like, big box, professional boxes. Well, it's very [32:54] possible that inside some part of FFmpeg is running. [32:57] - I mean, there's like so many, just to give people an idea, like I use [33:01] FFmpeg a lot on, on everything. Just trivial stuff like, [33:06] Take a video, add an intro video and an outro [33:09] video, and fade one into the other like what is it [33:13] called? Dip to black, like where it dips and then shows [33:17] the next video and does the same thing with audio. There's [33:21] like a cross dissolve of the audio. It's quiet, it quiets the audio and [33:25] makes it loud again. And then there's a bunch of stuff like [33:29] showing the captions on screen card, like baking the [33:33] captions in. You can customize the font. [33:36] You can do all kinds of layering of audio and video. There's a [33:40] million things and of course, all of that works [33:45] like magically with basically any codec. [33:49] Like anything you can shove in on the audio and the video side, it works. [33:53] - But it's like if you look at, for example, [33:56] you can do things that you would do with Adobe After Effects- [33:59] ... in command line on FFmpeg, right? It's, and it's very interesting because, for [34:03] example, for imaging, there is not such tool. [34:07] There is a few tools, but not with the breadth of FFmpeg. [34:10] - So ImageMagick has a similar kind of- [34:13] - Yes, but you will not- [34:14] - ... spirit, but it- [34:14] - ... do some filters, complex filters. You don't have the equivalent of Photoshop- [34:18] in command line, right? But for video, you have FFmpeg in command line. [34:22] - Yeah. It's incredible. I mean, it's like an example of a thing when a [34:26] bunch of great people get together and they get a vision, and they [34:29] stick by that vision for many years, which is incredible. [34:33] - And the vision behind, and the same for VLC and FFmpeg, is that [34:38] we make everything that is very complex easy to use for the normal [34:44] people, for everyone. [34:45] Right? Our goal is to make something that is insanely complex technically [34:49] and make it easy to use, right? And people, they use VLC, they [34:53] drop a file. They don't realize how complex the file is, but they [34:57] play it. Or, or people put any type of thing inside FFmpeg with [35:01] complex filters, and it just works like magically, right? And people- [35:05] And this is our mission, right? Make very complex things. [35:08] - We wouldn't be here and you wouldn't be here if this [35:11] required, you know, a traditional television studio setup. [35:16] It's tools like FFmpeg that democratize this. The podcast and [35:19] streaming revolution, the YouTube revolution- [35:22] was caused. You know, FFmpeg was a big player in that because it [35:26] democratized this technology that was once in the nineties, for example, you needed [35:32] equipment that cost hundreds of thousands of dollars to do compression. It was the size of a car, [35:37] and now everybody has that at almost an exact level playing field, and [35:41] that's something that's so remarkable. [35:42] - It gave voice to a lot of people. And just to clarify, we say you, [35:46] you wouldn't be here, not the human, but the podcast. [35:49] - The podcast. Oh, sorry. You as a... Sorry. [35:50] - I would still... VLC did not have anything to do on a biological [35:55] level- ... at creating me as a human. [35:56] - But, but it's like you realize also everything moved from text to images [36:00] and images to video, right? [36:02] Look at social networks. Video is everywhere. It's the most [36:05] powerful medium there is, right? And when you see [36:09] shorts and, and in Reels and in TikTok, right? It's [36:13] amazingly powerful to give... Video is amazing for [36:17] that, right? But the complexity is important. [36:20] - This is what people don't realize. I mean, this is really it [36:24] gave power to the individual all across the world. That's real [36:28] freedom. And I think, I can't believe it, but we still haven't mentioned the [36:32] actual obvious thing for people who are not familiar, which [36:35] it's open source, and there's a open source [36:39] community of users and developers behind it. So [36:43] it's really, it's a movement. So, like, we'll talk a bunch in a [36:47] bunch of different ways about the community behind it. But can you speak to the open [36:51] source element? So when we say what is FFmpeg, it's an open source project. [36:57] - Yeah. So FFmpeg, VLC, x264, VideoLAN, everything [37:01] we do is fully open source. And for the people who don't understand how open [37:05] source is, my usual analogy is about a chocolate cheesecake, right? Um, [37:09] usually for you, when you want to buy your cheesecake, you go to a bakery, they give you the [37:12] cheesecake. The other one way of having a cheesecake is have your [37:16] grandma give you a recipe of how to make that. When we do open source, [37:20] we give you the chocolate cake, and we give you the recipe to [37:24] actually remake the same cake, but at the same time tell you how [37:28] to build the oven and also how you're allowed to modify [37:32] the recipe and resell it to someone else. [37:35] And this is because software is just a very long recipe of small [37:39] instruction. Computers are not very clever. They go very, very [37:43] fast. So a normal program has tens of billions of instructions [37:46] instead of the tens when you have your chocolate recipe. So [37:50] a lot of the software industry was about selling software, like where you [37:54] just have like the final cheesecake. In open source, we [37:58] give you everything, and that managed to get a [38:02] lot of people work together, right? Because then you decide that you're going to [38:06] make the best program, the best recipe for video, and you create [38:10] communities. In FFmpeg, since the beginning of FFmpeg, probably [38:15] two thousand to three thousand- [38:17] - In the thousands, yeah [38:17] - ... people have contributed from the beginning, right? And then it's exactly like the [38:21] Linux kernel, right? The Linux kernel has probably ten thousand people [38:24] contributing everywhere, and they get together, well, [38:28] mostly online, right? So they virtually get together to create the [38:32] best tool for something. And on FFmpeg and VLC, it's just like, [38:38] well, this codec doesn't work, so I'm going to work on the codec, and I'm going [38:42] to add the support for this file inside FFmpeg, so it will be [38:46] beneficial to everyone. Because again, we work for the greater good. We work for [38:51] everyone, and that is what open source is. [38:54] - And we should mention, depending on the licensing, [38:57] You could probably build a billion-dollar, maybe even a trillion-dollar company [39:04] around basic... as a wrapper to... [39:07] - Well, yes- ... people do. [39:09] People do, right? There was a lot of problems with mostly, [39:13] Cloud providers who are basically running some open source tools, [39:17] In the cloud and just give you the API to [39:21] access to that. And there was a lot of um, [39:25] databases like Mongo or Elastic who changed their license in order [39:29] to avoid those type of scenarios. [39:31] - This is a question we get a lot in FFmpeg is, "Why don't you do that?" [39:35] And you can't. We have, we have thousands of contributors, some of whom aren't even alive anymore. [39:40] It would need all of their agreement to do that, and JB will go maybe a bit later [39:44] and talk about how challenging that process was in VLC to do the re-licensing. [39:49] - The license is a social contract in terms of Rousseau de [39:53] facto of the community. The community does [39:57] not agree on much beside the license. People go [40:01] around, discuss around because of the license, and that [40:05] also allow those license fork, right? Sometimes the community splits, but [40:09] it's possible because of the license and to merge back. [40:12] And we've seen that so many times, right? GCC and GC, [40:15] And EGCS in the past. We have seen, for example, all the web [40:19] browsers, right? They started as web, like KHTML, which becomes WebKit and [40:23] then which becomes Blink, right? So open source [40:26] license is like the core of the community [40:30] and people are coming from all around the world, very different type of religion, [40:37] Political borders. They work [40:41] in the same way on a project to solve a specific problem, [40:45] and the specific problem we're working on is to make multimedia easy for everyone. [40:50] - Uh, looking it up on Perplexity here, looking at the different open source licenses. [40:54] Most major open source licenses fall into two buckets: [40:58] permissive, very few conditions, and copyleft, share-alike [41:02] requirements for derivatives. Below is a brief practical summary of the [41:06] main ones you'll see in the wild. MIT license, BSD, ISC, Apache, GNU GPL, GNU AGPL. [41:20] Where's LGPL? Yeah, LGPL. Let's see. [41:23] There's the Mozilla Public License. There's Eclipse Public License. It [41:27] goes on. There's a lot of variety. I mean, I think the really popular [41:31] ones is MIT, GPL, LGPL- [41:34] - Yeah. And BSD. BSD [41:36] - ... and BSD, Apache. Sometimes you'll see- [41:38] - Apache as well [41:38] - ... Apache. Unlicense, that's an option. Attempts to dedicate code to the public domain with a fallback [41:42] permissive license. [41:43] - There are many licenses for many different things. What people don't understand [41:47] that public domain is something that doesn't exist worldwide, right? [41:51] So it's all the open source licensing [41:55] use the copyright law, right, the international copyright law, in order [41:59] to give rights on how you use the software or how you [42:03] modify. It's de facto a copyright license [42:06] contract that you give to the end user or to the developer. And [42:10] so you have like the first one, which are basically very permissive, MIT, BSD. [42:14] You give the code and basically you do whatever you want, right? [42:19] You take it, you want, you modify, you do what you want. And this [42:23] is popular for JavaScript and the type of BSD operating system. [42:29] - So some of them, one of the parameters is whether they require attribution, [42:33] meaning if you use the code, you have to say- [42:35] - Yes. So in those types of permissive licenses, some you need to say if you [42:39] use it, which is called attribution, and some you don't. And then there is [42:43] the other part of license which are copyleft, [42:46] where you need to give back to the community your [42:50] modifications and with different strings attached. [42:53] some weak copyleft license, like the Mozilla Public [42:57] License, to some which are a bit stronger like a [43:01] GPL, or even very strong like AGPL. So all of [43:05] those are different types of licensing that depends on what [43:09] your goals are and how you want to structure your community, which is why I spoke about [43:12] social contract, because this is very important to understand. [43:16] FFmpeg and VLC are mostly GPL or LGPL. The Linux kernel is GPL [43:24] but Android is Apache. A ton of JavaScript frameworks that are using are mostly MIT. [43:32] All the BSD kernels, OpenBSD, NetBSD are of course BSD. And so it's [43:39] philosophical change on how you want people to contribute back- ... basically. [43:44] - So I think you talked about that, you've moved at one point from GPL to [43:52] LGPL on certain parts of the project. What... Can you describe the difference [43:55] between the two, and what does it take to move to, I guess, a more permissive... [44:00] So that direction is more permissive. LGPL is more permissive than GPL. [44:04] - Yeah. So you have to realize that you can always go from [44:07] more permissive to less permissive, right? Because of course, [44:12] those licenses are basically statements, and so if you restrict, [44:16] you can always restrict more, right? So in a GPL project, you can take [44:20] MIT code, but you cannot do the opposite, right? Because they are more [44:24] constrained to match. Indeed, in fact, I changed the core [44:29] of libVLC, which is the engine of VLC- ... from GPL to LGPL. [44:37] And there were two reasons to do that. The first one is that so people can [44:41] use the VLC engine, libVLC, into [44:45] third-party applications. So a lot of applications which are playing video [44:49] on your phone or on your tablet are actually VLC engine in it- [44:52] ... which is calling FFmpeg in it. Um, so that was [44:57] one of the ways to create one of the companies I created, which is doing consulting [45:01] and integration of those types of applications where you integrate VLC [45:05] into third-party solutions like inside game engines or [45:09] stuff like that. With GPL, you couldn't do that because that means you needed to open source [45:13] everything, and those are for a lot of, like, commercial companies who don't want [45:17] that. [45:18] - So you can create a company with LGPL, you can create a company around it. [45:22] - Yes. [45:22] - You can do a commercial thing. You don't have to open source it. [45:24] So that's a big, big leap. [45:25] - So you could play video in your game. [45:27] - Yes. [45:27] - The problem is I'm a game developer, and I want to play some videos- [45:32] ... and I don't want to be forced to open source the entire game just to play those videos. [45:36] So that's where the consulting business, the libVLC LGPL- [45:40] ... allows you to do that. The LGPL, the library GPL as it used to be known, [45:43] allows you to do that. [45:44] - And FFmpeg is exactly the same. It force... LGPL [45:48] forces you to give back what you change on this component, this- [45:52] ... library, which is why it's library GPL. [45:55] And so you can use FFmpeg as LGPL into, like, [45:59] any type of application, even non-open source, but you need to give back the [46:03] modification you did on FFmpeg. Same on libVLC. [46:06] - Is it limiting from an open source perspective to go GPL? [46:10] Because if you-- if your library, if your code is GPL, it means you're not... [46:17] You're basically discouraging companies from building a business- [46:22] - Yes [46:22] - ... around it, right? Is that, is that fair to say? [46:23] - It depends on the company, but the company whose business model requires [46:27] the source, the application to be closed source, yes, it's limited. [46:32] So that's why, for example, I moved to LGPL. The second reason is a, [46:35] a bit more obscure, is that the terms and conditions of the, [46:39] App Store, the Apple App Store for iOS makes it very [46:43] complex to have GPL application on it, while it's easier to have [46:47] LGPL applications on it. So VLC on Windows and on Mac, [46:52] And on Linux is GPL. The core is LGPL. [46:55] But on iOS the iPhone version and the Apple TV [46:59] version is a type of different license called the MPL. [47:03] And yes, I went and changed the license and it was a long story. [47:08] - Yeah. So I think basically to change the license you have to contact all the [47:12] contributors. [47:12] - Yes. It's very important to understand that open source [47:16] projects are what we call in the US copyright law [47:20] joint work, or in civil law collective [47:23] works or collaborative works, is that you work all [47:27] together in terms of the same goal, and then you create one software, which is one release. [47:32] But the copyright is kept by all the individuals. [47:36] Some open source projects don't do that. They force copyright assignment, but this is not what we do. [47:40] We're communities. So everyone has basically copyright on what [47:44] they changed. And this copyright stays even [47:48] if at the end your contribution was deleted because the new [47:51] contribution was based on your previous one, right? So if you want to properly [47:55] re-license, you need to find all the contributors. And at that time, I [47:59] had to contact more than three hundred and fifty people. And [48:03] sometimes, well, they're just an email, right? So it's... you need to actually track down. [48:07] I actually, like, travel to some place to go someone that I [48:10] was like, sorry, that I'd found online to see a-- to [48:14] go to their job and say, "Well, you licensed that. Can you-- do you [48:18] want to change from GPL to LGPL?" Most of the times they don't even care. They wanted to [48:22] help VLC. But also it brought me to very complex situation. I [48:26] arrived to the work of a person who was a factory worker. [48:30] Um, and I said, "Well, I need to you to sign that," [48:35] because it was his son who died who actually wrote the code, right? [48:40] So I had to explain all those type of open source meaning, [48:43] and no, I was not a company trying to rip out the two line or five [48:47] line that that guy did- [48:49] ... but was useful, and the whole community agreed on that, and he had [48:53] no idea I was a factory worker. This com-- And I was a lot younger, right? Like [48:57] it was fourteen years ago, and like, like I was [49:01] almost in tears, right? It's very difficult, right? We are talking about lives of people and [49:05] he explaining, and we went talk about the photo of this guy, right? [49:09] So it's important to do it right and to do it correctly. [49:13] But yes, that means tracking down everything because every contribution works. [49:18] There are some project who don't respect that, and we do re-licensing a bit, like, [49:22] aggressively. [49:23] But as I said, it destroyed the whole heart of the community because it's-- we [49:27] only agree on the, on the license, so that's important. [49:31] - I would emphasize the community is such a wide-ranging group of people. [49:35] There's people in the Syrian war zone with electricity part-time. There's, [49:41] there's all people from all walks of life- [49:44] ... rich, poor, young, old. So it's quite remarkable to get, [49:50] you know, a group of people aligned on something. And that's an [49:54] achievement in itself. [49:55] - Yeah. It's incredible. And a lot of them are introverts, so you [50:00] coming to find them and getting them and getting them to answer an email might [50:03] be quite difficult. [50:05] - Most of us are introverts, right? You need to be more precise. You have [50:09] extremely introverts, extremely, extremely introverts and introverts, right? [50:12] It's just like a whole spectrum of different people. It doesn't matter. The [50:16] important is, is your code good? [50:19] Is your code great? Is your technology great? We care about excellent code. [50:23] We don't care who you are. Sorry, it's just like we have no idea to check. [50:28] We cannot check, right? Like, maybe you're a dog. I don't care, right? [50:32] I don't care where you come from. I need to look at your code. And this is [50:35] important because people don't understand that, and they come to the community and send them some [50:39] patches, and they get rejected, and they don't like that because, [50:44] I mean, you're just like, "Sorry, it's not up to our standards." "Oh, yeah, but I'm [50:48] engineer at this very large company in Italy, in Germany, [50:52] in the US." We don't care. We care about the quality of your code [50:56] because this is what defines our community, and which means that we have a [51:00] lot of people who contribute who are some very different backgrounds and, [51:04] and very introverts, sure. But that's okay, right? [51:07] - So one of the legends of the community is of course, [51:11] Linus Torvalds, who created Linux and is [51:15] a longtime maintainer of the Linux kernel. As the [51:19] legend goes, he can be pretty harsh on this meritocratic process of [51:23] reviewing the code and saying it's not good enough. Can you just speak [51:27] to the legend of Linus Torvalds? [51:29] - Linus is one of a kind, right? And [51:33] I would even go and say that what he did on Git is more interesting than what he [51:37] did on the Linux kernel. [51:39] He's very harsh, but what people don't see is usually when he's [51:43] harsh to, it's people who are maintainer of part of the [51:47] kernel, right? So they know him, right? So he's not very harsh like [51:51] that to everyone. The thing is, what he created in his room [51:55] is basically powering every server online, right? Even [51:58] at Microsoft cloud called Azure, I'm quite sure seventy, eighty [52:02] percent of the servers are running Linux. All your Android phones are running [52:06] Linux. What he did with the power of open source, sure, [52:12] is amazing. And yes, the quality of the Linux [52:15] kernel is very high, and yes, it's difficult, but [52:20] we cannot compromise on that. We cannot compromise on quality [52:24] because in the end, and you have to understand that, is the core [52:28] community of VLC is five people. The core community of FFmpeg is ten to fifteen, [52:34] and we are the ones who are going to maintain your code, right? [52:38] Because one thousand contributors in the timeline and just ten staying, it's [52:42] one percent chance that someone comes and stays. One percent. [52:46] So you will have change of job, change of wives, you have children, you [52:50] have accident in life. You're going to change jobs, whatever. You're not going to [52:54] come back. It's most likely. So we are the one going to maintain your code. [53:00] It needs to be maintainable. It needs to be excellent. [53:05] And yes, sometimes that means that you need to rework your work because it was good, but [53:09] it's not excellent, and we need excellence because we are very [53:12] few to maintain something that is critical for the whole. [53:15] - But we should also mention that there is some spiciness, some harshness to the [53:19] language that's sometimes used when you're keeping this high bar of excellence. [53:25] Is there something to say to that? [53:27] - It's, it's true, right? It's also the fact that, for example, what we're doing [53:31] is low level. It's extremely technical. You get into this [53:35] community. The tone gets very like [53:39] a type of-- It's a subculture, right? So people who arrive from the external are [53:43] basically not known to the subculture. Most of those people [53:47] around FFmpeg and VLC, we do VideoLAN DevDays, VDD every, [53:51] every year. They are so fun in real life, and they love it. [53:54] But it's true that you're online and sometimes, like, the tone, you [53:58] don't realize how it is. But that's okay. [54:01] - It's a culture. I mean, you get this in the gaming culture. There's pretty harsh, [54:04] intense, the way people communicate, and it's-- everyone [54:08] understands that the way you show love and respect just looks different in [54:11] different communities. Sometimes people... It depends. If it's a [54:15] book club, usually people are going to be much sweeter. If it's an open source [54:21] project that's very high stakes and used by millions of people- [54:24] - But it's very not often insults that you see, for example, [54:28] in the gaming, right? And so Linus' tone is a bit unusual [54:32] even for the open source community. It's more like it's more harsh on the results, [54:36] saying, "No, this is not good. This is crap." Those type of things that you will see. [54:40] - Try not to make it about the person, make it about the code. [54:42] - Yes. [54:43] - It's very, very matter of fact, and I think you've got to look at it in terms of, you know, [54:47] the famous FFmpeg is developed almost entirely by volunteers, and that's true, and you've got [54:51] to imagine someone's done a hard day's work at their day job. They come home. [54:56] You know, terseness might be a thing, you know, it... And that's not something to take [54:59] personally. [55:01] - You're tired, you're busy, but you still care about this open source stuff. [55:05] But you may not be able to explain and handhold someone on every subtle detail. [55:10] - And also you have to [55:12] realize that most people don't speak English as native language. [55:16] And this is especially for open source projects like [55:20] FFmpeg and VLC, which are mostly centered out of Europe. Sometimes like [55:26] people who are from the US or, or just like are very [55:30] not happy about the tone, but most of the time it's also like they don't know better, right? [55:34] It's difficult. The language is-- English is a difficult language. There is so many [55:37] subtleties and tone and so on that you don't have, right? [55:40] So often it's also difficult in those type of community about like [55:44] different cultures and languages. [55:46] - So as the legend goes, JB, you repeatedly turned down millions of dollars to keep [55:54] VLC open source free for everyone without ads. [56:00] So take me through the reasoning behind that decision of [56:03] leaving millions of dollars on the table. [56:06] - Yeah, that's like almost a meme, right, on Reddit or- [56:09] - There literally is a meme on Reddit. [56:11] - 9GAG and yeah, yeah. See, there's- [56:14] - You looking like a wizard in the, in the VLC hat on Reddit. This [56:20] is JB, the creator of VLC media player. He refused [56:24] tens of millions of dollars in order to keep VLC ads free. [56:28] Thanks, Jean-Baptiste Kempf. You can even summon him on Reddit. [56:33] - Yeah. And usually if you see, right, it's usually like people tag me, right? And, [56:37] and then there is me, and then like I say, "Good morning." [56:40] I got twenty-four K upvotes, which is great, right? My karma on Reddit is amazing, [56:44] at least on that account. So the question is, [56:49] needs to be answered first, what is the story about VLC, right? Because [56:54] yes, this is true, I refuse [56:57] dozens of millions of dollars, yes, several times. Yes, I could be a [57:00] multimillionaire and be somewhere on the beach. Um, [57:03] but I did not do it because I thought it was not [57:07] moral and it was not the right thing to do. And this is very important for [57:11] myself, is to be like, I work for the greater [57:14] good, I work for people, and I don't want-- It's not just by myself. [57:18] But the reason is also because I did not feel that I'm [57:22] completely legitimate to do that, and let me explain you why. VLC [57:26] story is a very weird story. In France, [57:30] we have university and we have a type of top colleges [57:34] and those top of excellency schools are engineering schools, [57:38] business schools, and basically lawyers and medical, [57:42] right? But they're outside of university, and in order to enter those, [57:46] you spend two years working like crazy math, physics to [57:50] enter those best engineering schools. [57:53] One of the schools is called the École Centrale Paris. It has changed name since, but it was [57:57] called the École Centrale Paris. And because it was Centrale, they [58:01] had to move it because it was too small after the World War II and, and they moved [58:05] it, they wanted to move it to the center of France in a place called Clermont-Ferrand. [58:09] And the alumni decided that this was not okay, right? It is [58:12] a, the school that Eiffel, right, the, the one who did the Eiffel [58:16] Tower, attended to, right? So they said, "No, no, we are amazing, great school. We cannot [58:20] do that." And so they bought a piece of land south of Paris very near [58:24] Paris. And it was a campus managed by a nonprofit of the alumni, okay? [58:32] Because of that, everything on the campus was managed by students. The [58:36] university did nothing, right? So radio, TV, supermarket, library defining who was [58:44] going into which rooms. Everything was managed by the students. [58:48] - That's amazing. That's an amazing experiment, that it all, all [58:52] didn't go to hell quickly. It somehow flourished. [58:55] - It worked great, and I learned so much in my life [58:59] doing those side activities, right? Because you're twenty-two and you need to run your campus, [59:03] else you don't have electricity, right? [59:05] So you care about that, right? But anyway, in the '80s they [59:09] did a full experiment of deploying a network mostly sponsored [59:13] by IBM and 3Com, which was a token ring network. [59:17] So token ring is something that probably almost no one [59:21] knows about anymore. It's a networking technology where [59:25] you don't have routers, right? [59:26] Everyone is linked. It's type, like really a ring, and when you want to [59:30] send a message, you talk to your neighbor who's going to put the message to the next one, [59:34] who's going to put the things to the next one, [59:37] in terms of ring. The issue with token ring is, of course, is that it's [59:41] very slow because every computer on the network needs to [59:45] open the message, see if it's okay. Is it for me? No, it's not, and [59:48] then send it back, like a token which is traveling around the ring. [59:53] In the '80s, you're doing some Telnet and sending mails as [59:57] university. That's okay, right? But starts the '90s, [60:01] and the '90s and start video games, and when you have high [60:05] latency in video games, basically you die, right? So in nineteen ninety-four, nineteen [60:09] ninety-five, around Doom and Duke Nukem coming around, they want a faster [60:12] network. So the students go and see the university and say, "You know [60:16] what? We want a faster network. We need to work," which, and also play [60:20] video games. And the university tells them that basically, [60:24] "Oh, I'm sorry, we cannot help you because you understand the [60:28] campus is not ours. You manage it, so [60:31] do something. And you should see some basically partners of this [60:35] university and basically go away." And they go, and they [60:39] actually go and see the CIO of [60:43] Bouygues, which is a large French company and who's doing some [60:46] TVs in France. And he says, "Well, you know what? The future [60:50] of video is satellite." Well, today we know it's not, but at least it was [60:54] a good idea. In nineteen ninety-five, the first satellite dish, and he says that [60:59] instead of having like one satellite dish and a big decoder for [61:03] each of the students, which are one thousand and five hundred, what about [61:07] you build, like you put an enormous dish and [61:11] only one decoder, and you send the video directly on the network. [61:15] And that required a very fast network. Today, it's obvious, but at [61:19] the time was, like, the first to do video streaming. So they built this project, [61:23] which was called Network 2000. [61:26] Of course, right, we are in the '90s, right? Everything is- ... futuristic is called 2000, [61:30] like— [61:30] - Yeah, 2000, yeah. [61:32] - And so they do the Network 2000 project. It's completely hacked. [61:36] It crashes after 45 seconds. That's okay. The demo is 40 seconds. [61:40] It leaks memory. That's okay. They put 64 megabytes of RAM instead of the [61:44] 8 or 16 you have, and the demo should have stopped there. And that was the Network [61:48] 2000 project by the students. [61:49] - What was the format of the video that they had to work with? [61:52] - MPEG-2 because satellite is MPEG-2 TS for transport, [61:55] MPEG-2 video, and MPEG-2 audio at that time. [61:59] And the project should have stopped there. Everyone was happy. They had, like, [62:03] amazing ATM network at 155 megabits per [62:07] second. They had probably one of the best network in Europe at that time, and [62:11] they stopped the project. Six months or a year later, two students arrive and say, [62:15] "Well, you know what? Maybe other people care about video streamed on a [62:19] local network," and they create the VideoLAN project, [62:23] VideoLAN. And one of them is called Christophe Massiot, that is a good [62:27] friend of both Kieran and me, and they start the project. It's not [62:30] even open source yet, and they spend around three years [62:34] to get the school to agree to make it open source. Because the [62:38] university wanted to get some-- because of the IP and copyright of [62:42] the students, wanted to basically monetize these MPEG-2 decoders. [62:47] - Just to be clear, so what was the main application, streaming on a local network? [62:50] - It was streaming on a local network. [62:52] - By the way, that's just, like, to state the obvious. This is before YouTube. This is before- [62:56] - Ten years before YouTube. You have a Pentium 60 or 75, right? [63:00] You, the main machine was 486DX at 33 megahertz, right? [63:04] - Bear in mind, television was the main form of video at the time. You could [63:08] get new channels. In the '90s, having even one new channel when you grew up with four [63:12] channels, having a fifth or a sixth was a big deal, and so [63:15] having this satellite service with, you know, [63:19] dozens, even hundreds of channels was so groundbreaking. [63:22] - Especially because this is university where you had a ton of different nationalities, right? [63:26] So there was a ton of people who wanted... So the-- in the end, they had, like, [63:30] several dishes on different types of satellites, right? Because, for example, a lot of people were [63:33] coming from the Maghreb or the Middle East and they, so they went [63:37] to different types of satellites. Anyway, the solution worked great, [63:43] and they started the VideoLAN project. The VideoLAN project has several [63:47] and some are completely crazy solutions, like one how to [63:51] create multicast on a unicast network, but let's not come [63:55] to that. It's too, too complex. But VideoLAN client part is what became VLC. [64:03] Actually, they basically strong-armed the university to force it to open source [64:07] because university did not understand that. And in 2001, it's still early. [64:11] But basically, yes, the university agreed early 2001 [64:15] to make it open source. I joined the project in 2003 because that's [64:18] when I joined the university. So the first thing is [64:22] I'm not the one who created VLC because actually no one did, right? [64:26] - Just kind of naturally emerged from the VideoLAN project. And we should mention that, like, [64:30] again, you said it just, but to make it clear, VideoLAN, [64:35] As what it became was at the time was a set of technologies [64:39] around video, and the VLC, what you called the client, that's the thing [64:43] that most normies, uh- [64:47] - That is correct, and [64:48] - ... think of, like, as the thing, which is, like, the thing that pops up when you click on a [64:52] video and you play it. [64:53] - So I arrive in 2003, and then I will create the [64:57] open source nonprofit organization called VideoLAN, [65:01] and I took everything out of the university to create a nonprofit [65:05] project and something sustainable. It's, yes, it's true that I spent [65:09] more time than anyone on VLC and VideoLAN. That is sure. [65:13] but it's a continuity of a previous project, VideoLAN, the [65:17] student project, which is a continuity of the Network 2000 project, which is a [65:21] continuity of that and that. [65:22] - I'm sure there's moments along the way there you were thinking of, like, [65:26] what is the future of this from an open source perspective? 'Cause as, [65:30] as the internet is blowing up, and there is companies... I [65:34] mean, for people who don't remember, like, there's companies making [65:37] huge amounts of money. [65:39] - And I can tell you that in 2005, the project should have died, [65:44] And I made it to continue the [65:48] project. At some point, we were only two active developers. [65:52] and I thought it was great technology and was useful, [65:56] and it will be useful and I made that my life and my, [66:00] my time. And I made that grow from a [66:04] few hundreds of thousands of users, millions of users to what we have [66:08] now, which is probably billions of version of VLC around the [66:12] world and used everywhere. So [66:15] that's a bit the story of VLC. There is ton of very funny stories around [66:19] that. Many people from around the world working [66:23] on it, like you said, in Syria or middle of nowhere in India. But [66:27] along the way, I got several offers which were either to [66:31] bundle toolbars, right? You remember those horrible toolbars- [66:35] ... which were basically spyware, or changing your web [66:39] browser or your search engine or even, like, [66:43] advertisement inside VLC. And I didn't like that, [66:47] right? I am-- and people don't understand that. It's not-- I'm not [66:51] against money, right? I'm very happy to make money. I created several [66:54] startups and one I hope that is going to work very well. [66:58] It's the fact that I believe that you need to win money [67:02] ethically. There is a right way of doing that, and doing sneaky [67:06] advertisement or stealing data is not the correct way, right? [67:10] For example, if Netflix arrived at some point and say, "Well, we want to put Netflix inside [67:14] VLC," probably the story would have been different, right? But they didn't. The only [67:18] people who came to us were shady ads company. [67:22] And if I do that, right, I would have a ton of money, right? And then three years [67:26] later, project is gone, right? Someone forks it and something else happens. [67:31] - So it's not even necessarily ads or any of that, it's the shadiness of the- [67:35] ... dishonesty of the-- So you had a good radar, you had a good [67:39] threshold of like, "No, this compromises the [67:43] spirit of what this is supposed to represent." [67:46] - But also it's for me, right? I'm like very selfishly, I [67:49] need to go to bed at night and be happy about what I've done, right? Maybe it's [67:53] my upbringing, maybe it's my parents' fault or whatever, right? But [67:57] I believe there is right and wrong, right? [68:01] And this was the right decision at the time. [68:05] It still is. I want to be proud of what I've been doing. [68:08] And like, if I had sold out, I would have [68:12] betrayed so many other people who work here. [68:14] - Yeah, well, I should say me and most of the internet [68:17] thank you for that decision. It's inspiring for others, [68:23] I think that are pushing the open source movement forward, [68:28] that it's okay to do these kinds of huge sacrifices [68:32] if you believe it's right. And I think in that case it was right and it was the reason that VLC [68:36] became as successful as it was, 'cause it's an embodiment, it's a symbol of [68:43] like, you know, freedom and what the open source community can create. [68:46] - Yeah, and be a service for so many people around the world, and this is important. [68:50] - We should emphasize in the 2000s it was really normal to download a program and it [68:54] secretly installs some spyware. It was buried in very faint text or in the [69:00] license text box that nobody reads that at the bottom- [69:03] ... "Oh, I will be installing this toolbar- [69:04] ... and changing all these things," and it was very common to have to, you [69:08] know, you install a program to do something at the time of any sort. [69:11] - To put yourself in the mind of a developer at that time, I think it's [69:15] very easy, to everybody listening to this, it's very easy at that [69:19] time to convince yourself to take a few thousand dollars- [69:24] ... a few thousand dollars to do it. To say no to much more money— [69:30] ... takes guts and takes vision. [69:34] - The last offer I had was obscene, [69:37] and they say, "Yeah, but imagine with all that money you could build [69:41] something new, open source," right? It was like the mind trick was, [69:46] it was difficult. [69:47] But for me it was just like, "No, this doesn't work like that or this is not the right thing, [69:51] so I don't do it." [69:54] and again, right, it's not that I don't like money or whatever. It's just like [69:58] it wasn't right. [70:01] - Well, once again, thank you from me and from the rest of the internet. [70:04] Let me talk a little bit more about the open source movement, about the [70:08] fact that, as you say over and over and over and over, FFmpeg, [70:13] is and many open source projects are built by [70:16] volunteers. So there's a bit of drama recently, [70:20] uh, Kieran, on the interwebs, on Twitter. [70:25] You have a spicy style on Twitter that I think [70:28] articulates and celebrates all the incredible developers and [70:32] development and the code, especially [70:36] assembly that's involved in building some of these codecs and building some of this [70:40] incredible technology. But that brings us to the, a bit of a [70:44] debacle that happened. Tell me the full saga of what happened with [70:48] the Google security engineers. [70:50] - Just to be clear, Google are one of the biggest supporters of open source out there. [70:54] They have been for a long time. It's just I think some things kind [70:57] of went a bit overboard this time. So FFmpeg [71:01] itself, and this is not like a secret, it's on the homepage, you know, the, [71:05] it processes untrusted data. There can be security issues when you parse [71:09] untrusted data. That's very normal. But recently what changed was Google started [71:13] using AI to create security reports on an open source project, FFmpeg. [71:19] Volunteers had to deal with that. They did, they provided very limited [71:22] funding, and they even went to the media first announcing how good their AI was [71:27] before the issues could be fixed. [71:29] - And this is in the public forum. [71:31] - Yeah, this is all public. [71:32] - So report, reporting an issue, using AI to find an issue in the code, [71:36] which is a security vulnerability, and then reporting that publicly before you're able to fix [71:40] it. [71:40] - Yeah. It's announcing how good their AI is, that they provided a standard [71:45] 90-day industry deadline without [71:48] without really understanding the nature of volunteer-driven [71:52] development. In addition, this vulnerability was on an obscure 1990s game codec. [72:00] the way-- And let's look at it from their standpoint to begin with. Let's you know— [72:04] - Yeah. Can you steer me in their case? [72:06] - Yeah, sure. They have substantial resources working on the security of open source [72:10] projects [72:11] that, you know, are ubiquitous, and they've used, you know, a lot of compute to do [72:15] that and very expensive and very capable security researchers, [72:19] to do that. And that's their viewpoint is they are contributing by [72:23] doing that. But I think that's where opinions differ. [72:30] it opened up a lot of interesting fissures I would say. [72:36] it does seem that there's a portion of the security community that [72:41] look at themselves a bit like building architects that never have to go to site. [72:44] You know, going to site is something that is a little bit beneath them, the actual [72:48] day-to-day construction. They're there to do their security things and it's someone else's [72:52] problem. The security industry also kind of has [72:57] a very aggressive tone towards things. The, the language they use is extremely [73:01] aggressive. They use very strong language like, "You will get popped." So and to, [73:05] Joe Public, get popped, [73:07] you know, means something quite bad. For them it means to get hacked. [73:11] The way I would look at it personally is a little bit like the padlock on your home. [73:15] The padlock on your home or, you know, the lock on your home [73:21] is there to protect against the capabilities of what it's there to protect. [73:28] It's not there to protect nuclear secrets. It's not there to protect Fort Knox. And [73:33] it could be looked at that they're using AI at a level of scale to go [73:37] and pick those locks and then say, "Hey, your [73:41] lock's not secure. You need to deal with this." Whereas actually they're the ones with [73:44] resources to be able to [73:48] fix this. But that seems to not be something either they'll contribute to in terms of [73:52] patches or in terms of financially. And the scale of AI is kind of the issue. [73:56] The bug reports are very wordy. They're very, [74:00] very-- It's almost a denial of service by AI-generated bug [74:04] reports on very niche codecs. [74:08] and the other issue the security community has is everything is marked high priority. [74:12] You're going to, you know, "This is the most important thing in the world, and you need to deal with this. [74:15] High, high, high, vulnerable, scary, scary, scary," [74:18] on a game codec used on one disk in 1993. [74:24] And that's where the dichotomy lies. Going around [74:28] telling everyone that their padlock's not safe, well, that's a hobby project of [74:32] somebody. The safety of that codec is consummate to [74:36] what that person thinks. It's their hobby. It's good that they're security analyzing it, but it [74:40] doesn't need a big scary warning, "This is a critical vulnerability." [74:45] We also may recently also see that there was another [74:48] quote-unquote vulnerability. It wasn't at Google in this case, but [74:53] a filter could overflow and have an integer overflow, and one of your [74:57] pixels could be the wrong color. And this was marked high, 7.5 severity in red. [75:05] And at some point, the security industry needs to realize you can't keep crying wolf [75:09] like this because this just leads to people, you know, the equivalent thereof of putting [75:14] password stickers on their PC. You know, you can't just keep crying wolf every day. [75:17] And I appreciate, you know, that's their modus operandi is to create as much [75:23] scare and fear. But from the Google standpoint, at the end of the day, [75:29] they need to contribute either financially or with patches. Google [75:32] uses FFmpeg at a scale probably you or I couldn't even contemplate, [75:38] millions of CPU cores. [75:40] And yes, they contribute in areas mostly regarding their own products, so VP9, AV1. [75:47] But in a wider sense, [75:51] there's a disproportionate level of contribution. Yes, they fund students. Yes, they fund Summer [75:55] of Code. And I think so Alex Strange is a former FFmpeg developer I think [76:01] posting in a personal capacity. [76:02] - So he posted about security engineers on [76:07] Hacker News. His post reads, "The problem with security [76:10] reports in general is security people are rampant [76:13] self-promoters, in parentheses, Linus [76:17] once called them something worse. Imagine you're a humble [76:21] volunteer open source developer. If a security researcher finds a bug in your code, [76:28] they're going to make up a cute name for it, start a website with a logo. [76:33] Google is going to give them a million-dollar bounty. They're going to [76:37] go to DEF CON and get a prize, and I assume go to some [76:41] kind of secret security people orgy where everyone is dressed [76:45] like they're in The Matrix. Nobody is going to do any of [76:49] this for you when you fix it." basically commenting on the sort of the incentives [76:58] for the different people involved and misaligned. [77:01] - The problem here is the disproportion of means on discovery [77:07] compared to patching it, right? And this is the biggest issue, right? [77:12] And after that debacle, Google did some changes. [77:14] - They are now starting to send patches, which is- [77:17] - And they also now have reward tools for fixing issues. So [77:21] it has changed a bit because of that debacle. So it's good, right? [77:25] But we've seen, and we talk about Google, but we have seen like some [77:29] other large companies saying, "Oh, you need to fix this bug because it's critical in our [77:33] product." [77:33] - Can you explain the XZ fiasco? The FFmpeg tweet reads, [77:39] "The XZ fiasco has shown how a dependence on [77:43] unpaid volunteers can cause major problems. Trillion-dollar [77:47] corporations expect free and urgent support from [77:51] volunteers. Microsoft, Microsoft Teams posted on a [77:55] bug tracker full of volunteers that their issue is high priority. [78:02] After politely requesting a support contract from Microsoft for long-term [78:05] maintenance, they offered a one-time payment of a few thousand dollars [78:09] instead. This is unacceptable. We didn't make it up. This is what [78:13] Microsoft Teams actually did." And then you give [78:17] the image and the details and all that kind of stuff, showing that these trillion-dollar [78:21] companies are not giving much money, not giving much support. [78:24] - They think an open source project is a traditional vendor that they have an [78:28] SLA. They think a public bug tracker is actually, [78:32] you know, a third-party vendor's Jira where you can do all of these things. It's [78:36] not. It is there to report bugs. [78:38] I think the thing that made this particularly heinous was [78:42] the name-dropping of Microsoft, the name-dropping that this is a visible product. [78:46] If this was just a general bug report, I think that would have made it a lot [78:50] better. [78:51] - Yeah, so they literally said, like, "This is a big deal because a lot of [78:55] people are using it in Microsoft." I wonder what [78:59] happens psychologically. So I think what happens in these companies, maybe you can [79:03] correct me, is they— You're right. They just think of FFmpeg as like a vendor that [79:10] Microsoft surely is paying a huge amount of money to. [79:14] They kind of assume that in their interaction, and [79:18] nobody anywhere on the stack is going like, "Wait a minute. [79:22] Shouldn't we be giving like millions of dollars to FFmpeg?" [79:25] - And this is a very big problem in large— Like we're talking about some [79:29] companies, but it's the same everywhere, right? A lot of those [79:33] companies. Like the, when we talk to that person, right, [79:37] he was just like a manager on one project in Microsoft Teams, right? He [79:41] had never really discussed with open source community. He [79:45] had no idea, right? It was like—but the problem [79:49] is that usually there is what we call OSPOs, right? Open source program [79:53] offices in those type of companies, and they are the ones who are supposed to [79:57] discuss with open source vendors. Um, or open source communities. But like [80:03] they often don't explain that correctly internally, right? And here it's just like [80:08] we are not your supplier. If you want me to be your supplier, I'm very [80:12] happy, right? I will send you a contract and SLAs. Like I created [80:16] five companies who are doing that around open source projects, so that's okay. [80:19] - We should say that some of the spicy tweets that Kieran, you're [80:23] behind, and some of the debacle produced results. [80:28] - Yes. [80:28] - Positive results. [80:29] - Donations have increased substantially. They're still not enough to [80:33] cover even a single full-time developer, but [80:36] on both a, you know, awareness level and a technical level, [80:41] there's substantially more technical awareness and sort of awareness of the importance [80:45] of FFmpeg as a result, as a result of X and what's [80:48] happened. I can say, you know, it solved its purpose. People [80:52] realize the level of importance FFmpeg has. [80:55] - And on VideoLAN it's the same, right? Like for example, a, a very simple example. [81:01] For more than a year, we couldn't update VLC on [81:05] Android because of a bug on the Play Store, on Android Play [81:09] Store, right? The only way we got someone to answer [81:13] was to put a very spicy, as you say [81:17] tweet saying that we are going to stop distributing VLC [81:21] for Android, right? And we have around 100 million people using that. And now then [81:28] someone from Android actually came and discussed to us, right? We had the [81:32] same issue with Microsoft or, or like saying that we were [81:36] going to stop distributing VLC on the Windows Store. And [81:39] unfortunately, we are so small that the [81:44] only very strong power we have to solve those issues [81:48] is blaming on social network because it snowballs and [81:52] now they listen to us. But so as large companies often have difficulty [81:57] talking to us. Like for example, VLC, right, is probably one of the top [82:01] 10 software used on Windows. I am not part of Microsoft ISV [82:08] programs, right? I don't have a point of contact at Microsoft, right? [82:12] While I'm sure any other software, Adobe, Spotify, has a [82:16] point of contact. I don't have that, right? So [82:20] raising awareness works. It's sometimes very spicy, lot of drama. Well, [82:26] X and Twitter are okay for that, but it's efficient. [82:30] - Uh, so everybody listening to this should go follow FFmpeg on Twitter, on X, follow [82:38] VideoLAN on Twitter, on X. Go donate. Donate ... to FFmpeg. [82:45] - And thank you, Lex. Over the years, several years you've been a supporter of, you [82:49] know, FFmpeg and VideoLAN on X. You know, giving us shout-outs, appreciating, [82:54] you know, what we do. [82:55] - FFmpeg for life. [82:57] - And for example, like Tim Sweeney, Carmack, and a few [83:00] others, like very high-level people have raised also the awareness [83:04] on our X accounts, and that helped a lot also. [83:09] - Karpathy as well. [83:09] - Karpathy, yes. [83:10] - Karpathy as well, yeah. [83:12] - Yeah. I mean, also, you know, outside of the fact that so many people use it, it's [83:15] so impactful on the world, it's also a great representation of a great open [83:19] source project. Like the value of assembly and C and making sure that like you take [83:27] programming seriously for real world systems. [83:30] - It's not just that. We'll talk about assembly later I'm sure, 'cause that's its whole topic in itself, but it's [83:34] also celebrating people like Andreas Rheinhardt who do maintenance. It is, I [83:40] believe unpaid, as I believe as a volunteer. He's doing massive [83:43] refactorings. Uh, Andreas Rheinhardt and Anton Khirnov [83:47] rewriting ffmpeg.c with threading. Celebrating those guys, [83:51] celebrating the untold labor that's gone into this [83:55] that actually doesn't change anything from the user standpoint. The files are exactly the [83:59] same, but wow, the, the, the airplane has been rebuilt whilst it's in the air. [84:03] - Christian Garcia said, "As a teenager running this account," referring to the FFmpeg [84:07] ... account, and you responded, "Teenagers have written more assembly [84:11] in FFmpeg than Google engineers." But also [84:15] just pointing out that there's a lot of incredible contributors who are teenagers. [84:19] - Like JB said, we don't care who you are, where you're from, what you do. [84:24] Teenagers have written thousands of lines of assembly, [84:28] Over the years. Give a shout-out back in the days to Daniel Kang. [84:33] So also highlighting the work of people like Ruikai Peng. This is a 16-year-old, some of his [84:37] first contributions to FFmpeg, [84:40] actually doing and putting some of these quote unquote security researchers to shame by [84:44] by actually finding issues and fixing them and being [84:47] 16. There's no barriers. There's no barriers to you have to [84:51] study on, at college under this person and understand these. It's you can [84:55] learn C, and let's be honest, it's from, it's from the K&R book. Learn C. [85:00] You can learn assembly. We'll talk about that maybe a bit later. [85:03] You can contribute to world-class technologies. [85:06] - In VLC one of the oldest contributors called [85:10] Felix, he's the one doing everything on Mac and iOS. He's starting [85:14] working on VLC. He was 16. We had a guy called [85:18] Edward Wong, who used to be a Google Summer of Code student who [85:22] stayed for three years around VideoLAN. He was 14, [85:26] right? And, and part of Google Summer of Code and Google [85:29] Code-in, which were programs where basically we have students or high school, [85:34] We wrote a ton of assembly for x264 and for VLC [85:38] and for FFmpeg, right? So everyone can contribute. [85:41] - And he also did a good job because he didn't play the alarmist CVE [85:45] heist, create a CVE, which is like, a [85:49] public exposure of security and do these big scary [85:53] red 7.5 high priority. He just fixed an issue in Git [85:57] after three days and just fixed it. He didn't need to go and play a big [86:00] security drama about it. And I think [86:03] I posted, you know, the kids are all right. Whereas- there's, you know, [86:07] there is a por- I'm not saying all security people do this, but there is a portion of the [86:10] security community, as Alex said, that likes to hype themselves up by [86:14] creating drama. They would have happily raised, "This is a high [86:18] priority CVE 8.0" or whatever on a [86:22] issue that actually was in Git. It wasn't even in a release, it was in development, and three [86:26] days later was fixed. [86:27] - Well, I just want to put a little bit of love out there, even to the bigger [86:34] much love and respect to Google engineers. Like you said, they're [86:39] Some of the, the best software engineers in the world, and they do contribute a lot- [86:43] ... even on the security front. And also, you know, I'm a big fan of [86:46] Theo. Much love to Theo. He was part of this, [86:50] Debacle and drama a little bit. I think when you just zoom [86:53] out on the grand arc of human history, [86:57] the drama contributed positively to everybody involved. Donations went up. [87:02] It brought more attention to the topic, allowed [87:06] everybody to bicker in a way that ultimately got them to figure out [87:10] what FFmpeg is all about. [87:11] - So the way we looked at this is like it's a rap battle at the end of the day, you know? [87:16] No, but it is. We say stuff, we say stuff- [87:19] ... but we can, we can leave it on. X is a perfect place for, you know, international rap [87:23] battle. You say stuff. I say stuff about your mama, but it doesn't mean, you know, I have [87:27] an actual personal issue with her. [87:29] Uh, and that's what it looks like. The Theo situation, you know, JB can maybe expand, went a [87:33] little bit too far and there was a little... But, you know, it's just a bit of fun. [87:37] It's just a bit of rap battle. It's a bit, it's WWE. You know, [87:40] everyone's having a bit of fun on X. [87:42] It doesn't need to be taken seriously. You know, the teenagers thing, you know, [87:46] that... So that guy was a Google employee saying, "Hey, you know, there are other ways to run [87:50] an open source business." You know, go and there's like, oh, man, just have a bit of fun, you know? [87:54] That's what the point of this account is. And, and furthermore, if you can teach people [87:58] about the ways of open source projects, assembly, et cetera, by doing [88:02] that, I think there's a lot to be offered here. It's not dunking on people for dunking's sake. [88:06] It's showing actually the story that I think X learnt is these are not big [88:10] corporate open source projects. This is not Kubernetes where there's, [88:14] you know, hundreds, maybe thousands of people- [88:15] ... paid to develop this stuff. These are just people in their basements in their spare time, [88:20] and if you can address that topic in a fun and entertaining way- [88:23] ... I think that's the good thing and that's, that's the value of X and [88:27] then the reach we have. [88:28] - And to be honest, right, like even at Google, Google is [88:34] one entity, but so many different people, right? And there is a ton [88:38] of Google engineers we work with [88:42] all the time, and even like Google from YouTube to Chrome to [88:46] Chrome Media to the rest of Google, those are very different types of entities. But [88:50] what we do is efficient. And, for example for, [88:54] for Theo, right? It went a bit too far. I had him... Like I calmed everyone [88:58] down. I had him on the phone. We said, "Okay, like this goes too far," and so on. [89:02] But in the end yeah, it's a rap battle, but it's positive for the [89:06] project. It, like the awareness we have on open source and, [89:10] and I mean true open source from communities right now is [89:14] increased dramatically in the last two years, and this is useful. [89:19] - Uh, what do you think motivates all the incredible contributors that we've been [89:23] talking about? Like, what's the engine? It's so interesting to see. [89:26] - So- [89:26] - Like you said, they're sitting in the basement. What's the driver? What's the engine there? [89:29] - There are many drivers, but weirdly the main one [89:33] is that what we do in multimedia plays videos, and video is [89:37] cool, right? And, and for example, we have so many [89:41] people in the community who arrive because they loved watching [89:45] anime, right? And this is like the advice when people ask me, [89:50] "What should I work on in open source? How do I start?" And my answer is always the same: [89:54] work on something you love. [89:56] I am working on VLC because I love movies, right? And I love [90:00] watching the same movies over and over, even if my wife hates me when I do [90:04] that, right? But because it's interesting, right? Because it's a topic that [90:08] you like, right? The first, that's the first thing where people come to [90:12] usually to VLC and FFmpeg. The second thing is that [90:15] technically we, because we search for excellence, this is [90:19] the best school ever, right? This is the best school [90:23] ever of programming. If you're good in C, in [90:27] FFmpeg, if you know how to write assembly, I assure you you're going to [90:31] be one of the best programmers ever, even if you're working on writing [90:34] TypeScript, because this is the most amazing thing to [90:38] do. And you will, like, have to get reviews by some of [90:42] the most seasoned programmers ever who are going to look at [90:46] every part of your code and tell you why it's not great. It's like we are the [90:50] best teachers that you've ever had in programming, right? [90:52] - Andrew Kelley started Zig. He was an FFmpeg developer and started Zig after [90:56] his FFmpeg school. I mean, [90:59] it's the place to learn so many aspects of programming in the [91:03] real world, in a thing used by billions of people. You have nowhere to [91:07] hide. You have to be open and honest about your flaws and how you can learn [91:11] and be better. [91:12] - And what is also interesting in multimedia is [91:15] that you have 16 milliseconds to display a frame. It's not like [91:19] a game engine where you can basically slow down and wait a frame. [91:23] Like, so it's, you need to be good, right? There is no choice, else you don't have your [91:27] video. And because of how codecs, if you miss a frame, you're going [91:31] to destroy the look of the video, right? So you need to be good. [91:35] You need to be perfect to have the right thing. But also [91:39] is that it's not just pure programming in the mathematical sense, right? [91:43] A lot of people don't understand, but [91:47] in order to program correctly on the open source multimedia community, [91:51] you need to understand how computers work. And when you write assembly, [91:55] you need to understand about CPU pipelining, right? You need to [91:59] understand how SIMD works, how the ALU works, right? You need to [92:03] understand what, how IO works, right? And this is what I [92:07] think that is missing to a lot of engineers and software engineers today, is [92:11] understanding what we call computer architecture. And, [92:15] like, seriously, like some of the debates is like, should we use this assembly call or this [92:19] one? And people say, "Well, no, it's going to be like three cycles on this [92:23] type of CPU and this one," and has massive impact on the output, right? [92:27] - We should expand. FFmpeg is probably one of the biggest CPU users in the world. There's [92:31] it's probably running— ... as we speak
... [transcript truncated due to length]
