I created and maintain a two Middleman extensions and always planned on upgrading them to work with Middleman v4 (which was released more than a year ago!). Since I hadn't upgraded any of my Middleman powered websites to v4, there was never an urgend need though, but I finally found the time.
There's a great guide on how to create an extension with the new v4 syntax, but unfortunately no official guide (yet!) on how to upgrade an extension to work with Middleman v4. So I simply looked at a few other extensions that had been upgraded already and also checked the Middleman source code for what had actually changed.
If you have more tips or experienced other difficulties in upgrading an extension, please leave a comment below! I'd be happy to extend this post to make it more comprehensive, and maybe it can become the source for an Extension Upgrade Guide in the official Middleman docs.