Building mulVid 0.2.0 (Part 1: Pre-amble)

If your reading this then mulVid 0.2.0 has been released (and hopefully, you've used it!). This has been the most challenging project of my technical career and to say it took everything I had to complete it would be an understatement. 

I'm writing this for those who care about what happened behind the scenes and I'll also cover some of the technical details. Think of this like a summary of what happened with all the boring bits removed. Like most things that end being extremely hard, I assumed this would be an easy project that I could hack on over a few weekends, show off to my buddies and then add to my portfolio. It ended up taking close to 1.5 years, so a bit more than a few weekends.

Before I decided to even embark on this, I had done some basic research and didn't find anything that fit what I needed in terms of features. Seeing as I couldn't find anything I needed right away, I decided to build my own. Coming from the world of open source, I (incorrectly) assumed that I could find a project that was 90% of the way there and then tweak it to do what I needed. When that ended up not being the case, the merchant in me became intrigued. 

I kept asking myself, "Why hasn't anyone built this?", "Does anyone actually want this?" and the most dangerous question of all, "How hard could it be?". 

Long story short, most of the answers to the first two questions boiled down to either "It's impossible" or "No one would want it". Saying "It's impossible" would be a perfectly valid answer if this was still 2020 but it isn't and a lot has changed since then. As for those saying "No one would want it", I knew that was just an uninformed opinion from someone who had never worked with videos.

As for the answer to "How hard could it be?", the answer is "very hard, very hard indeed". I used AI to hack an initial prototype but I kept running into roadblocks because I fundamentally didn't understand how video data worked. I really did try to avoid studying up on how videos worked at a fundamental level but I kept coming back to the conclusion that if I was gonna be able to build something that worked the way I needed it to, I couldn't cut any corners. Having accepted my lack of knowledge, I set out to rectify those gaps. This was when I discovered something very interesting. Low-level video internals is a very niche topic.

I can't remember the last time a day had gone by where I didn't watch a video but up until now I had never really cared about how the actual footage was handled and processed. Even when I edited videos, I had a very, very limited understanding of what was going on behind the scenes. As I tried to fill in my knowledge gaps, I was very surprised at how little info there was out there. I was used to googling "{name of tech} + tutorials" and be spoilt for options in terms of tutorials and even books but in this case I was drawing (mostly) blanks. For most people, this would be discouraging but at this piont, I had been thoroughly nerd-sniped so I was in it for the long-haul if for no other reason than gaining the knowledge. I did end up finding some books and reading through them. I went from not knowing a damn thing to knowing a little w/ an understanding of what was left to learn. The obvious next step was how to transfer this new knowledge into code.

It was around this time that I stumbled into FFmpeg. I can't say enough good things about that project. If software projects could be designated as world heritage sites, FFmpeg would be at the top of that list. Most of the video capabilities on the modern internet are powered directly or indirectly by it and the craziest part is that it's maintained entirely but volunteers. It looked promising and so I started looking through the code. What I saw convinced me that this would become a huge part of mulVid.

I now had a Swiss army tool kit that let me manipulate any video in any format and I knew enough to get going and pick up the rest along the way. I returned to the earlier prototype that I had vibe-coded and after playing with it for a few days, tossed it aside and decided to build from scratch. I initially wanted to get going on building the video editor immediately but I knew that it would be better to build something related with a smaller feature scope to make sure I had the fundamentals mastered. Thinking along this line, the obvious option was to build a multimedia player. Most video editors had one so it wouldn't be wasted work since I would just merge the code into the large code base for the editor. In hindsight, that was one of the best decisions I made because what I thought were now small gaps in my knowledge were still actually large chasms.

Thanks for reading. I'm splitting this story into multiple parts, the next part will be linked at the bottom of each article.

Brought to you by mulVid.