In VersionPress 2.0, apart from sync / staging and revamped UI, we also took a look at a long-standing issue we have with the db.php drop-in. This blog post will be a bit technical but the TL;DR is that we will now be able to run side-by-side with some popular plugins like W3 Total Cache or Query Monitor and generally work on sites that need to use the database drop-in for some reason.

The problem

VersionPress needs to observe database operations quite closely because whatever goes there, also needs to be potentially stored in the Git repository. Some other plugins like VaultPress depend solely on WP hooks & filters but that was not good enough for us for two main reasons:

1. Hooks don't cover everything. While they might cover a lot, maybe something like 95%, we need to track the site as a whole, i.e., 100%.
2. Third-party plugins are a problem here because they usually don't provide hooks, or not many of them.

So we need to be quite close to the database and observe the traffic that goes into it. Unfortunately, while WordPress provides many extensibility points at a higher level, there are very few of them at the low, database level. In fact, there are only two, both added way back in 2007 as part of WordPress 2.1 (ticket #2721):

1. The 'query' filter
2. The db.php drop-in

The query filter isn't really useful for VersionPress because when it is invoked, the query was already transformed from PHP arrays to just a string and perhaps more importantly, the hook is executed before the actual query is run so for example, if the query inserted a new post, the filter isn't given the ID of this new post. Simply put, the query filter doesn't provide enough data for us.

So we were left with the db.php drop-in option. Drop-ins are an interesting part of the WordPress extensibility story and we were easily able to adjust the wpdb class to do what we wanted. However, there is one major downside to drop-ins: only one plugin can use them. In our specific case, if VersionPress v1 was installed, the site couldn't use other popular plugins like W3 Total Cache or Query Monitor that also utilize db.php. And vice versa: if one of those plugins was installed, VersionPress couldn't.

That was bad.

Looking for solutions

I should start by saying that we knew from the beginning that utilizing db.php is not really right. We are not a database driver after all, but similarly other plugins who “abuse” db.php, we do so for the lack of better extensibility points in WordPress.

So about a year ago, when VersionPress was in an early alpha stages, we started a discussion in Trac ticket #29710. The suggestion was basically to add a couple of new hooks / filters into the wpdb class but kind of as expected, the core team wasn't too keen to add them – after all, our use case is pretty rare and they need to serve a broad range of plugins. Fair enough. (BTW, there was a similar proposal in 2012, #21412, but that also led nowhere.)

What was very useful, though, was the discussion around it. It made us understand the problem deeper.

There were a couple of suggestions on how to avoid db.php and still extend the wpdb class, like for example replacing the $wpdb instance in the plugin code. Each approach had certain problems but it would still be an improvement over v1.

As we played with some of these workaround, we realized something important: the drop-ins are DROP-INs for a reason. Literally, you just drop a file on a disk and WordPress loads it instead of the default one. This means that the plugin loading mechanism is completely avoided, there are no worries about plugin ordering, filter priorities etc. Specifically, the database drop-in will simply execute its content as soon as WordPress needs to talk to the database, and that happens sooner than plugins are loaded.

So we actually had a bigger problem than we initially thought. Even if the wpdb class provided all the hooks we ever wanted, we would still miss some of the queries until VersionPress was loaded. We could do a couple of things to make our plugin load sooner, like renaming the project to “aaaVersionPress” 🙂 , but really, the only 100% reliable way to capture all queries is to be the drop-in.

The solution in v2

So we needed to utilize db.php and avoid it at the same time. Nice.

What follows is an ugly hack. Please turn off your computer if you don't like seeing drastic things done to code.

.

.

.

.

In v2, we monkey-patch the built-in wpdb class. We drop in our code directly into the WordPress source file. Yes we do.

Of course this is tricky because that WordPress file can change slightly between releases (so we cannot just apply a standard patch to it), the file will be manipulated during WordPress updates and other operation so we need to detect that and re-apply our changes as soon as possible, etc. It is a hack but in our tests, it works, and that's what matters. After all, VersionPress as a whole is a tale of hacks and creative solutions on top of the WordPress platform that are not always nice but provide value to the users. That's what we care about.

The solution has probably these two disadvantages:

• If there is a db.php drop-in that doesn't call the parent implementation of insert(), update() and delete() methods, our code will not be invoked. This is a natural limitation that we cannot do much about but fortunately, it is really, really rare to encounter such drop-ins.
• Some security plugins might issue a warning when they see a changed wp-db.php file. We didn't have a time to fully test this yet but I can imagine there will be some issues with this. But this is solvable.

So there you go. VersionPress 2.0 will no longer occupy the db.php file which improves its compatibility with other plugins. The solution is not perfect but we couldn't find anything better. If you are a bright mind and can think of a solution that would improve on our v2 implementation, please let us know and we will incorporate it in the next EAP release. And send you some cookies.

Thanks for reading!