Sometimes, code threatens to blow up in your face. I don’t mean it crashes, that’s no disaster for a programmer (unless it starts happening after you release it). I mean several apparently simple elements of the design, when put together, cause the complexity of the program to increase exponentially.

You can start frantically coding to try to patch up the holes, then realise the new code doesn’t work either and is actually even more broken. The best approach I find is to sit back. Take a walk. Sleep on it. Maybe you just missed something. Adding complexity is never good, because when the next problem comes along it will get even worse. If all else fails, you will have to re-think the original design.

In my case, undo/redo, instances and actor links were the explosive scenario.

Now, the scene code knows nothing about undo/redo. There are many functions to modify it, and they just operate as if there was no such thing. I like this. It keeps the classes simple. This works because operations on a tree structure are inherently reversible. If I need to modify a node I copy it and then swap. So the command list doesn’t need to know much about the scene either. Create links between nodes though, and that’s not quite true anymore.

Problem one: if a node is instanced, and it is deleted, the instances should disappear too. But they didn’t. The problem was that the undo buffer was keeping a pointer to the original node. These are reference-counted pointers. The extra pointer meant that the object was not being deleted, so the instances (with weak pointers) were pointing to a valid object. Simple enough, you might think. Just copy the object and keep the copy in the undo buffer instead. But then how do you redo? All the instances will be pointing at nothing. I was contemplating having to maintain a list of all references with each node, or hacking the smart pointers to replace a deleted object.

Luckily, I didn’t go too far with either of those ideas. There’s a simpler solution. Every node knows the scene it belongs to. If a node is removed from the scene, for example by being deleted, the scene pointer is set to null. So all the instances have to do is check whether the instanced node is part of the scene, returning null if it isn’t. One line of code.

Problem two (even worse): This trick doesn’t work with actor links. It’s not possible through the actor interface to hide the broken links. However, in this case the links are already two-way. So it’s easy to delete and restore a link, and from there create a command list to unlink and relink a hierarchy of nodes. It’s a still a slight explosion, because the code to manage links for cut/paste and undo/redo is larger than the original link code, but it’s manageable.

Which leads to problem three: deleting an instance unlinks the original node. The linking/unlinking code has to ignore instances. This is also why the instances have to disappear when the original is deleted (I considered using a strong reference so they would stay around) – the links are gone so they don’t work anymore.

Any more lit fuses lying around? I hope not, I’ve just got all this stuff working..