Last week WordPress meta contributors implemented a “Live Preview” button for plugins in the official directory, with the intention of allowing users to safely test any plugin in one click. The button went live across all of WordPress.org’s 59,000+ plugins but took plugin developers by surprise as it was pushed through without any communication or input from stakeholders.
The implementation was premature and failed to take into consideration the many different types of plugins that appear to be broken due to inadequate support in the Playground testing environment.
Five weeks ago, Automattic-sponsored Meta team contributor Steve Dufresne commented on the ticket, “Adding that it’s likely still the case (someone else can confirm), not all plugins work in the Playground so we should build in an opt-out mechanism.”
This suggestion was roundly ignored by other participants on the ticket and the Playground previews went live. It became immediately apparent that this was done without thorough testing as many plugin authors reported the previews created an unfavorable, broken experience for users.
“Who decided to release Preview without posting on make.wordpress.org/plugins/ with some advanced warning to plugin devs?” WordPress developer Alan Fuller asked, starting a discussion in the #meta Slack channel. “Was that a #meta decision? Can it be reverted and due notice given?”
Retired Plugins team rep Mika Epstein identified three major use cases that were missed, which she estimates will impact 30-40% of plugins not working in the Playground environment:
- It won’t work for add-on plugins (ie. anything for Woo) because we have no way to identify plugin dependancies, and the sandbox won’t know to install the ‘parent’ plugin
- It won’t work (well) for anything that requires a lot of customization (WooCommerce itself)
- It won’t work AT ALL for anything that’s a server integration (Memcached, Redis, etc).
- Multisite
Participants noted that DEBUG is also set to True, allowing unrelated warnings and notices to be displayed to the visitor.
“It stinks to work really hard on a plugin and then have some preview show up that makes it look totally broken when it’s not,” WordPress developer Ben Sibley said.
“This feature is a neat idea, but it needs a lot more work. We’ve gone decades without live previews; why was there suddenly a rush to launch this today when it’s demonstrably unreliable?
“As others have stated, this should be rolled back immediately and switched to an opt-in feature. Once it’s rolled back, work on giving plugin devs information about how the preview works so we can decide if it’s right for us or not. There is no rush to release this without proper communication and testing!”
Newsletter Glue co-founder Lesley Sim requested the feature be opt-in, contending that the average user won’t have patience if something appears broken and will assume there is a problem with the plugin, not the directory or the playground.
“So it ends up reflecting badly on the plugin developer, which can be really stressful for them if it means a loss in (potential) revenue/installs (yes, I understand that many people think this shouldn’t be a key concern, but it is the reality for many small plugin devs) or if they have additional support burden as a result of this feature, which is completely out of their hands,” Sim said.
After others echoed these concerns, Automattic-sponsored contributor Alex Shiels, who implemented the feature, said he didn’t expect it would be controversial and said he was “over-optimistic about how smoothly it would work.” He deployed a commit that added an opt-out toggle so plugin committers could disable the Live Preview button.
“The reason I didn’t communicate prior to deploy is, there was discussion on the ticket for a month prior; and because Playground has been live for several months now,” Shiels said. “Every published plugin in the directory has already been available for running in the Playground since well before this ticket. All I did was make it easy to get there with a single click. Apologies for catching you all off-guard.”
Others requested WordPress.org implement a customizable Demo link url in the readme file, instead of turning Playground previews on for all plugins, along with many more suggestions for making the environment better for showcasing plugins.
After continued pushback urging Shiels to make the feature opt-in instead of opt-out, he removed the button on Friday, October 6.
“I do want to emphasize that a lot of the worry and concern wasn’t about the fact that a plugin was broken in Playground,” plugin developer Aurooba Ahmed said. “Most of us know if our plugin works in playground or not, it was that a very apparent feature was pushed to the plugin repo that affects how users evaluate plugins, without discussion and feedback from enough of the key stakeholder audiences.
“I look forward to seeing how the feature is iterated upon (because ultimately it’s a fantastic concept) so that it can be useful in all the right ways for all stakeholders.”
In the meantime, users who enjoy having quick access to Playground may want to check out the Chrome browser extension created by LUBUS, a development agency. It adds a Playground” button to theme and plugin pages on WordPress.org so users can test drive extensions with one click.
When adding the opt-out toggle, Shiels commented on the ticket that plugins broken in Playground were broken before the ticket was opened and will remain that way even if the plugin does not opt into the Live Preview button.
“I know the Playground team is hard at work on addressing bugs and compatibility issues there,” Shiels said. “And I intend to further improve the Live Preview support in the plugin directory to make things better for users and plugin developers alike. Many of your concerns can be addressed using Blueprints which will allow configuring and installing dependencies, importing demo content, and other neat things. I’ll work on making Blueprint support available as soon as I’ve confirmed some engineering details with the Playground team.”
There is more work to be done before this feature is ready for rollout. The live preview button is currently disabled while contributors iron out compatibility issues.
