Linear/Binary Search Methods
Just as with anything else, there are methods that make identifying a problematic mod/plugin easier for everyone involved. We’ll be going over two methods that are made with the intent to provide a structured search method for mod/plugin lists of almost any type, regardless of game or modloader.
The Principle
You can normally infer what mod/plugin is causing a bug from the logs or the bug itself, but in the absence of that you’re forced to test each mod/plugin as the culprit. By using a standard searching method you grantee that you’ve tested each mod in the same way, benefit from saving time in repetition, and working from a documented set of issues and solutions that could occur while searching.
The flow chart depicted on the right shows the two methods we’ll be covering here, but they both have the same goal. Remove each mod/plugin, test and repeat until the bug is gone, and then add mods/plugins back until you’re out of mods/plugins to add back or you get the bug again.
The sections below will cover those ideas in more detail and provide insight into issues that you may run into while testing.
Loading...
Identify and Replicate
Before you can figure out what mod/plugin is causing an issue, you need to make sure you can reliably cause the bug to occur, or replicate it. So you need to identify the minimum number of elements you need to trigger it and make a setup that reliably causes it.
You should use as few elements as possible, because the more complex your replication method is, the less likely you are to actually have found the root cause and the harder it may be to replicate with a dwindling mod/plugin list. So do your best to figure out the interaction itself instead of the blocks, items, or events involved.
For example, your server may crash when a player is pushed off of a cliff by a modded mob, consider the replication methods below
- pushing the player off of the same cliff with the same mob
- pushing the player off of the same cliff with any mob
- the player jumping manually off of the same cliff
- the player jumping manually from any height resulting in death
You’ll notice that as we go down the list it gets less and less specific compared to the original event. This is the process of narrowing down the bugs replication method by removing elements that may not be necessary. If you do this well, you may even be able to guess at what mod/plugin is causing your problem before you even start removing them.
It’s also important to note that just with the information provided there are a number of other factors at play that I didn’t bring up in the example replication methods that I’ve personally run into causing crashes, such as the specific player, their inventory contents at the time of death, the dimension, or the state of the mob. All of these things can make finding a replication method extremely difficult, so when you encounter a bug multiple times make sure to take notes on the circumstances.
Removal/Disabling
In order to test without some of your mods/plugins, you’ll need to remove or disable them. If you’re just disabling one mod/plugin there may be easier options for disabling individual mods depending on your modloader, but in this case we’ll be doing that with a large amount of the mod/plugin list, so there’s a better, and more universal, method.
You should move the mods/plugins to a separate folder from what your modloader uses to load its mods/plugins. This allows you to move them in bulk and have a working list of what you’ve removed from your environment without having to look at each file.
dependencies/dependents
A lot of mods/plugins make use of other mods/plugins to work. These are called dependencies and the mod/plugin will generally refuse to load without it.
When removing a mod/plugin that acts as a dependent, that being the mod/plugin that is relied on by another plugin/mod to load, your environment may crash on startup or the mod/plugin that depends on it may just fail to start. In either case you’ll need to remove that mod/plugin and add it to your “quarantine” pile.
It may also be beneficial to note the dependency relationship for when you add that mod/plugin back. But the mod/plugin will likely note the required and optional dependencies on it’s download page, so you can double check there.
Done in Halves (Binary)
Binary searching is geared towards searching a large list of mods/plugins with a minimal amount of re-testing the bug. By searching the list in halves you can quickly identify which mods do and don’t have any relation to the bug.
After replicating the bug you can start searching through your mod/plugin list by using the binary search method. This is done by halving your mod/plugin list until the bug can’t be replicated anymore. Once that happens you can add mods/plugins back, either individually or in halves of the removed mods/plugins, until the bug can be replicated again.
If you add your mods/plugins back in halves then when you can replicated the bug again you should set those mods/plugins aside and only add those mods/plugins back. This is how you reduce the pool of mods/plugins you have to test.
The one caveat to this method is that some mods/plugins will have dependencies or dependents, meaning that there are other mods/plugins that either require another mod to run or exist only to be run by another mod. The latter of which is generally known as a library mod. In those cases you’ll want to remove those mods in a way that still allows your mod/plugin list to function while removing the library mods.
See the section on mod/plugin removal for more information about this.
One by One (Linear)
Linear searching is a much, much slower process than binary and is comprised of removing and testing each mod individually instead of working in halves. You may need to do this if you have a complex ecosystem or a bug that is particularly troublesome. As a general rule, binary searches are preferred and should be attempted first.
Now that you can replicate the bug, you can use the linear search method to remove each mod/plugin, and it’s dependencies/dependents, individually and re-testing the bug. This may be preferred over the binary search method when you have a small mod/plugin list, you have a large number of dependencies/dependents, your mod/plugin ecosystem is tightly integrated, or when the binary search method hasn’t identified the mod(s)/plugins(s) in question.
The major downside is that checking each mod/plugin and it’s dependencies/dependents individually is an incredibly slow and tedious process, especially as your mod/plugin list grows. But that may be worth the detail you get over each mods/plugins removal.
Test in Isolation
It’s often that a bug is caused by the interaction of multiple mods/plugins, and in those cases it can be somewhat painful to make sure that you’ve identified the right culprits. The bulk of the processes outlined above has been made to ensure that if this is the case, it’s far easier to catch, but it’s still very good practice to test the mod(s)/plugin(s) you’ve identified on their own before reporting them.
This ensures without a shadow of a doubt that they are the cause of the bug, or at the very least that they make it apparent. It also provides the developer with a clean environment for them to start with when you file a bug report.
All you need to do is create a server/client with the modloader you used and install only the mod(s)/plugin(s) you identified. After you’ve done that and make sure they work, test the bug exactly as you did before. If it still happens, file your bug report with the author(s) and they may be able to help you get it fixed!