Before you release a plugin, you should exercise it in ways that you probably didn't try while you were developing. Walking through these steps yourself can save you a first round of bug reports from your users, and thus make your plugin higher-quality and lower cost-to-deploy.
Test with Slaves
Be sure to test the plugin with slave nodes running on remote machines before you try to deploy for real.
Watch for exceptions in logs
Watch the Jenkins console (not just the build output) for exceptions. Run maven with exception traces turned on, lest exceptions triggered by jelly can get lost in
giant logs, because sometimes you don't see any errors in the html page.
mvn -e hudson-dev:run
and
mvn -e hpi:run
Or grep the logs for the Big Three of jelly exceptions:
InvocationTargetException
NullPointerException
NoSuchMethodError
Test with java -jar jenkins.war (not just hpi:run)
Just testing a plugin with mvn hpi:run is insufficient. It will not expose classloader issues. At the very least, install your plugin into a test HUDSON_HOME and then run that jenkins with java -jar jenkins.war. Even better is to deploy your plugin to the container that will eventually hold a real deployment.
Test with real builds in a non-production server
Test a new plugin in a non-production clone of a production server, with the actual projects that the plugin will run on in production. Often plugin bugs are revealed by real data that weren't revealed by test data. Beware hubris! When you are testing a plugin and trying to simulate real-world usage, you have to restart jenkins whenever you think you've got a release candidate. Doing this on a production server will aggravate users and make Jenkins look flaky.