Open The Black Box (The Solution To My DOM Diffing Woes)

Oct 03 09:59

Sometimes, you just have to do the hard thing, put your thinking cap on, and open the black box. In my case, I had to really sit down and understand how Livewire's DOM-diffing algorithm works, and THEN I was able to clear my head and move forward.


So last time we chatted well two times ago talked about my dom diffing woes. Why dom diffing matters in Livewire why it's not perfect why it's super frustrating. Well, I have good news. We're through the woods a little bit and I feel a lot better about where where I'm at with the whole situation in Livewire.

So I want to talk about how I got there. Where to begin so I had mentioned that there's a fork in the road right now. I'm using a package called morph Dom. It's the same package that Phoenix live view uses for Dom diffing and it works. Okay out of the box. I had to do some tweaking I found some bugs right away, but I've encountered a few nebulous bugs that just kind of reveal that morphdom isn't the most advanced Dom differ.

Maybe another way of putting this it I guess the more specific way of putting this it doesn't it doesn't look ahead. So it does Dom diffing but it doesn't predict the future like other Dom differs do and they can basically more intelligently figure out what the actual diff is. It's kind of hard to describe.

But anyway, I don't want to get into like implementation details here. I want to talk about the big picture. So I. Morph time sort of because it's the thing that I've been using and I've been hacking on it. Like I have like duct tape around it and things like that. I've actually, you know pulled it into the project and change specific lines in the code base.

So I kind of own it in my project and their gives you there's like a certain feeling about. About a package that you haven't written about really any code that you haven't written. It doesn't always feel good and your judgements usually clouded. I think a lot of there's like a lot of biases there.

A lot of people think that the code is more poorly written than it actually is if they didn't write it there's all that sort of stuff and it really is hard to reason about there's like one giant while loop with a while loop nested inside of it and a ton of conditionals like probably like eight layers of nesting really hard to follow.

So I definitely feel that way about this code base. I don't understand it makes me feel bad. So of course, there's always the rewrite in the sky well for talk about the rewrite in the sky. There's always new plugins that you can try and I've tried them before I wasted a couple weekends on you trying to switch to a virtual Dom diffing Library.

So I went back to the internet and. Try to see you know, what's out there. What are the alternatives to morphdom morphdom is specific in some specific ways for for a tool like Livewire that I can't just use any Dom differ because a lot of them work with virtual dumbs which live where doesn't use and if that doesn't make sense to you don't worry about it, but there's kind of this niche of Dom differs that work with actual Dom in the browser, which is what Livewire needs so I found an alternative that I don't know if I have some wood across before which is kind of crazy.

Decently popular, you know, the first thing I look at is GitHub Stars. It's funny. It's kind of an arbitrary metric but really it's like it's kind of like on Amazon when you look at. Like sometimes I just look at the amount of reviews and that is that like Swayze my decision, you know. Yeah, so I look at GitHub stars and I go all right.

Is this something that is just Niche somebody started a year ago or two years ago and they're not going to continue keeping up or whatever. So I look at GitHub stars and I look at the last commit date. Those are two things I look at when I open up a GitHub repository. The last commit date was an August that's not bad.

And there's like 500 something GitHub Stars. That's also not bad. I think morph Tom has like 2,000 now, but when I started using it, it was more like a thousand so. So this all looks good to me. I'm like, okay this could work. And so I pull it down. I open a new branch and I pull it down just to do a proof of concept.

This is really important as well as like to basically for any stuff like this. I mean this is kind of obvious stuff, but but it is a good reminder that don't don't start rewriting something like it's it doesn't take that much effort to try something. The problem is when you don't know when to stop trying and we'll talk about that in a minute, but it doesn't take that much effort to try something.

It's good to try. Things near the next time you're in like a meeting or whatever like a Sprint planning or something and there's people talking Ad nauseam about what Solutions might work and what effort they might take take, you know an hour and pull the freaking thing down and actually use it for 5 minutes and it'll actually inform your decision in a huge way.

So I decided to just pull it down and start just hacking on it seeing if I could retrofit into Live Wire and make it work. So that part didn't take me too long. But like I said the risk in that is that you think it's going to work and so you you go down the road and it's it can be a long road and you have to decide when enough is enough.

If you keep going you push forward you fix the bugs you make it match the old behavior and give it and get the new Behavior or you punt on it. And you say you know. But this is too involved. There's too many unknowns here. This is going to take too long. I'm not going to chase this right now.

That's a really really hard decision to make potentially the most difficult decision to make in development. But one of the most important decisions to make so. I did what I wanted it to do right away. Like the thing that morphdom didn't do if Dom did like it's a smart dom differ. Oh, it's more advanced than morphdom in some ways, but there was some other Hang-Ups and so I just started fixing them like one by one and a basically I've read the whole source code.

I've grabbed everything, you know, I went really deep down the whole probably took me a day and a half two days. And I came up and was basically frustrated because I couldn't get it to work. Right? So I went and found another library and did sort of the same thing and then I contemplated. Well, maybe I just get an actual virtual Dom differ like the one vue.js uses pull that in that way.

My basically my virtual Dom diffing will be Rock Solid because it's views core, but but I'll have to do some weird tinkering to get it to work with dhamma in the browser. So I thought about that and basically it all left me in this place. We're just feeling down this this is you know, it's all like a if you picture it like like a hike or something.

It's this is the well maybe a Hikes not the right word. But you're at the bottom of the pit. This is the part where you're you're just you're alone and you're at the bottom of the pit and you don't have any good solution in front of you you feel bad about what's happening. What exists right? Now you start to get that imposter syndrome you start to look at other people and think everybody's better than you everybody's projects better than you and just not feel good about yourself.

And that's pretty much where I was for a handful of days and I didn't know the right way forward. So yesterday I thought you know what I think what I need to do is just do the hard work do the hard work of really owning morph dumb like understand it. Understand every line of it. I've walked it line by line before but I mean really like I mean sit down and read through morphed them until I feel like I know how it works completely to a point that I can change it, you know, like sit down and do that.

So I did and guess what it didn't take me that long. It like it's crazy how that happens. But I'm sure you can relate to this experience where you wear something seems sort of Out Of Reach so you avoid it but when you look it in the eye it's you know, 10 minutes away. I'm thinking of another story where I tried to write a recursive function to solve some really complicated math problem spent hours and hours and hours on it.

I eventually realized that I was too far down the hole and there the whole would continue to be too far I punted on it and I solved it manually by hand with a pen and paper and it took me I think 15 minutes. There's so many so many stories like that. When sometimes the right thing to do is just to do the hard laborious work.

Yeah, so that that's what. Decided to do and I sat down and I dropped morphdom and then after that because I understood exactly what was happening. I was able to basically solve my the first original problem that sent me on this Rabbit Trail in 10 minutes. I added four lines of code to the core of morphdom.

And it fixed my problem and then so I then I looked at another issue that I heading to haven't even looked at a GitHub issue dealing with dom diffing hold it down. It's a different problem that's core to morphdom and I solved it in another probably 15 minutes and I have tests for. It's perfect now.

So now I feel good about it. I feel good about the project. I feel like I own this piece of technology. It's still an open source project. I didn't write it myself. So I still get the benefit of all all of it the open-source NE sandpeople hardening it by fire over time. But I feel good about it. I feel like I own it and the answer was opening the Black Box morphed on was a black box to me.

Dom diffing was a black box. It was the last black box and black boxes feel they're intimidating their mountains to climb. You think you're not smart enough. You think that it's out of your a range of expertise whatever at the end of the day. I mean if it's a if it's a code base its code, you know, you know code you can read code I can read while loops and for statements and whatever and I can sit there.

And figure out how it works and I underestimated myself and it's not that difficult to understand and I feel like I fully understand it now. So the answer was opening the black box go right into the challenge do the hard thing. It makes the rest of the things easier. I'm happy now. My time different woes are over for the meantime, and when I encounter new problems, I now have a lasting solution because I understand how the thing works so I can change it and make it behave the way I want it to behave.

So let this little Saga be an inspiration to you. Thanks for following along. Enjoy your day.