Selenium IDE was a great way to handle automated web app testing like the Store Locator Plus plugins for WordPress.    Selenium IDE is a simple script recorder and playback too that runs on Firefox.    Or, I should say, it used to run on Firefox.  That broke in 2017 when Firefox 52 came out.

After a lot of research I finally found a viable alternative to Selenium IDE that will work with modern browsers.  It is also free, locally installed, and open source. All winning attributes.  Paying for services is not much of an issue so the free part is not a requirement just a “that’s nice” feature.

Web app testing services

I tried several paid alternatives including Browserstack — a paid monthly service that runs virtual desktops and mobile device simulations hosting various browsers. Having to connect to a remote server via proxies or tunnels is a pain.    It also means no testing when offline or when the network is unreliable.    Having multiple browsers is great but 90% of the testing that needs to happen is base functionality which is the same across browsers.    Modern browser are also very good at testing mobile with browser like Safari going beyond simple screen sizing in their mimic of IOS, for example.

Other alternatives included several locally installed proprietary test frameworks.   Nearly every one of them ranges from mediocre to downright horrid.    This is clearly an industry stuck in the 1990s mindset of application development — from the start where you have to fill out a form with all your contact info to be “allowed” to demo the product (and be later harassed by sales people) to the 1980s desktop-centric interfaces.    Many did not work on MacOS.   Those that worked were heavy, bloated, and had a steep learning curve.    Does nobody integrate with my phpStorm, my web app IDE?

It just so happens that the best local testing suite today happens to be free.

The winner?   Selenium Webdriver with a few libraries like WebDriverIO + Mocha + Chai to make things easy.

A WordPress development and testing environment

It took years of learning and many false starts to settle on what many consider the “standards” for plugin development.    Varying Vagrant Vagrants (VVV) as a virtual machine manager with simple scripts to configure multiple WordPress installs with their own plugin/theme and even PHP version stacks.    phpStorm with great CSS, JavaScript, and of course PHP support including self-documenting code features and WordPress-aware autocompletion.   Not too mention Node, Grunt, and many other modern frameworks supported “out of the box”.    The ever-reliable MacOS and the ready-for-remote-automation Safari browser.

After more than couple of false starts the next couple of paragraphs are my mental notes about what I believe, for now at least, to be a solid setup for having WordPress plugin development and solid user-interface driven or “Automated Web” based testing for the apps that will surpass what Selenium IDE brought to the table.

Setting up the main phpStorm project folder

It took years of using phpStorm to figure it out the right project setup.    Doing so can save a lot of time and aggravation and avoid frequent trips to the command line.     Start by isolating your code as much as possible into separate phpStorm project specific folders.  Do not use an existing folder with code in it. Start with a new empty folder.

This can be a bit of a trick if you have complex projects that share components as the Store Locator Plus WordPress plugins (WPSLP) and SaaS Service (MySLP) do.    They share common plugins but run different WordPress installs on different hosts (local/virtual hosts are different as well), and use different themes.   The trick here is to use the VVV Customfile to map shared paths on a per-plugin/theme basis or, if you are lucky, on a wp-content/plugins or wp-content/themes basis if you can get away with activating/de-activing plugins to set the right environment.

Note: sharing all of wp-content/plugins in a VVV host is great if keeping different plugins deactivated works, but keep in mind your code searches in phpStorm will find code in deactivated plugins as well.   You may want/need to exclude those unused plugin folders from the project after it is setup if it creates “search clutter”.

Under the phpStorm project folder will reside the .idea hidden folder with all the phpStorm project-specific settings like VVV server path to local (VM host) paths for live debugging, your file watchers for SCSS compilation and Uglify JS and CSS compression that runs automatically.   Under this folder is allow where you want to place your TEST folders , creating them if they are new or pulling them in with a git pull if the test suites already exist.    Placing your node-dependent code under the main phpStorm folder for your project is important for the next few elements to work.

Adding directories to phpStorm projects

Since this is a WordPress development project we need more than an empty phpStorm directory with a test folder in it.    First add the WordPress code from the VVV install.    You’ll need to have provisioned the VVV box first with vagrant up after you configured it.  There are other articles about setting up VVV.

Go to phpStorm > Preferences > Directories

Click Add Content Root on the right panel and add your site’s VVV public_html directory as a content root.  This is typically ~/vagrant-local/www/<sitename>/public_html.    Doing so allows phpStorm to find the WordPress base code for autocomplete.    It will also likely auto-detect this is a WordPress install and ask if you want to setup as a WordPress project which adds the autocomplete path as well as sets the Code Style to match WordPress Core standards.

Using a shared wp-content directory

Keeping a “clean” WordPress directory in the VVV structure is a good idea.    Placing custom code such as your plugins in a different location, ~/WordPress/wp-content for example, is one way to do that.

Map the virtual server using the VVV Customfile to that directory so ~/WordPress/wp-content replaces the ~/vagrant-local/www/<sitename>/public_html/wp-content directory.

Now you need to tell phpStorm it is part of your project as well.   Using the same method as above add the ~/WordPress/wp-content/ directory as a Content Root as well.    Now you an edit your php plugin and theme files as part of the project.

Adding Selenium+Webdriver directories

If you did not do so already under the main project folder create sub-folders for tests.   If you already have repositories setup for this pull them into sub-directories under the main phpStorm directory.

By placing NodeJS based projects like Selenium Webdriver under the main phpStorm project folder you get the benefit of the built-in NodeJS + NPM package manager in phpStorm.  This not only makes it easy to install project-specific modules without corrupting your system-wide (global) node space but also makes autocomplete and code inspection a lot nicer in phpStorm. You’ll want both to rapidly develop quality code and test suites.

When you are done you will have a phpStorm project that looks similar to the following.

 

 

• WPSLP_new is my phpStorm Project name and folder name.
• node_modules is installed by phpStorm as needed, more on that in a few.
• selenium_for_wordpress is the git repository with my old-school Selenium IDE (for posterity), native Selenium Webdriver, and new WebDriverIO (wdio) test scripts.
• wp-dev-kit is my Grunt task manager
• public_html is the VVV installed WordPress app the local test server runs on.
• scss is a Foundation library (not part of this discussion)
• wp-content is the custom theme and plugin code in my ~/WordPress/wp-content folder.

Setting up a better Selenium Webdriver

Selenium Webdriver itself works fine for automating web tests using JavaScript.   Combined with the built-in Safari Enable Remote Automation setting you can get JavaScript to remote-control he browser and do things like click buttons, check text is on a page, and other base automation and test scripts.   You’ll also find with vanilla Selenium Webdriver you end up writing a LOT of repeated code.

Soon you realize the Selenium Webdriver is standard JavaScript with a nice library conveniently installed by NodeJS.  You are running a node project.   You start creating shared config files and scripts to eliminate redundant code as you follow the DRY standard for programming.    If you are like me, light will dawn and you’ll realize “someone else must have already done this… there MUST be a Selenium Webdriver framework”. Yes there is.

A description of what we need…

WebDriver.IO adds a layer on top of Selenium Webdriver that gives you a shared configuration file to tell your Selenium Webdriver app which browser to use, how long to wait to time out a page, and other things you likely put in your custom shared config file (I did).   It also comes with a full library of simplified “Selenium Webdriver common tasks” with things like “Click a button labelled X” or “Check a cookie value”.   Things you likely started adding to your shared JavaScript library.   This makes it a LOT easier and faster to write web automation scripts in JavaScript without having to maintain your own ever-expanding interface library.

Weddriver.IO Selenium Server Standalone is the “glue” that allows WebdriverIO to talk to Selenium clients like the built-in Safari Remote Automation listener. If  you use Firefox of Chrome you’ll need to install those web drivers separateley and restart those browser afterwards.

Mocha is one of many NodeJS test suite helpers available that happen to work great with WebDriver.IO.   This is what manages your tests.  What is run and what is reported back to the console.    It provides some helpers and syntactic sugar to organize your test scripts, both within individual test and for groups of tests.   It is going to help manage your tests and is a good place to start.    Add it to your project.

Chai is another useful NodeJS library.  It is an “Assertion Library”.   It is a set of well defined tests that you run against your JavaScript app.  In the case of Selenium Webdriver automation it ensures that various elements of the page has what you expect and handles reporting any issues in a graceful manner. We ‘re going to add that  as well.

Getting test drivers installed in phpStorm

You setup your project directory and put the test folders within.  You mapped your php code from the VVV install and your custom code to the project.  Now it is time to setup your automated tests.

While the instructions for Selenium WebDriver, Mocha, and Chai all give you command line options the easiest way to do this is to use the built-in phpStorm NPM system. You want phpStorm to be aware of the Node module for autocomplete and code inspection.

First make sure you have NodeJS installed.  This goes on your VVV host box not in the VVV install itself.

Go to phpStorm > Preferences > Languages & Frameworks > Node.js and NPM

If node is installed properly on your system you should see the Node interpreter is set in the top box. On MacOS it typically lives in /usr/local/bin/node for a global install, which is the preferred location.

You will see an empty Packages list below.

Click the + at the bottom of the window and start adding your modules:

Search selenium-webdriver , when it comes up click Install Package on the bottom of the search list.

Do the same for webdriverio , wdio-selenium-server-standalone, wdio-mocha-framework, chai.

This is the equivalent of doing npm install on each of these.

This will create the proper node_modules directory in the main project folder.   All sub-folders that contain NodeJS code will be able to find these modules without explicit requires with paths due to the way NodeJS discovers modules.     The modules you install this way are shared throughout this specific project and can include your Grunt task automation modules as well (not covered in this article).

You are now ready to create your first test.

Setup your WebdriverIO configuration

Open Terminal in phpStorm.

You should been in your main project root folder.   ~/phpStorm Projects/WPSLP_new/ for example.

Check wdio is ready for you:

# ./node_modules/.bin/wdio --help

WebdriverIO CLI runner

Usage: wdio [options] [configFile]
Usage: wdio config
Usage: wdio repl [browserName]
config file defaults to wdio.conf.js
The [options] object will override values from the config file.
An optional list of spec files can be piped to wdio that will override configured specs.

It should com back with some notice that indicates WebdriverIO is ready.

Now configure it.   Use the defaults for most of the options that come up but select the options for Selenium Standalone, Mocha, and Chai when available from the options list.    Since I want my config file to be part of my git repo and I don’t want it polluting the main directory I move to my tests subdirectory and run the command there.

# cd tests
# ../node_modules/.bin/wdio config

The WebdriverIO Testrunner page shows an example of how this works.

This will create a wdio.conf.js file that is used to run your tests and configure the environment.

Writing the first test

When you ran the configuration it asked for a place to put test spec files and error screen shots.   If you chose the defaults you’ll need to create those directories under your tests folder.    The test spec folder is where the first test “base_site_online.js” will go.

Create a file in your projects ~/tests/test/specs folder called “base_site_online.js”.  Change the wpslp.test references to match the URL to your local VVV box.   Make sure that box is up with the #vagrant up before running any tests.

describe('WPSLP Base Test', function() {
    it('Check wpslp.test is online', function() {
        browser.url('http://wpslp.test');
        browser.getTitle().should.be.equal('SLP Dev – Map Locations With WordPress');
    });});

This test script is writing in JavaScript using NodeJS and the Mocha + Chai libraries build on top of Selenium Webdriver and Webdriver.IO.    it is a lot cleaner than “long hand” Selenium Webdriver.

This test is now ready to run from the wdio executor.

Running the test

Since the test relies on Webdriver.IO we need to run it from that app not from the standard NodeJS binary or from the default JavaScript engine.   If you right-click and select “run” on your test it will fail because the wdio libraries are not loaded.

Instead you want to use the WebDriver.IO Testrunner to manage this for you. Test runner will, by default, run all test files in the test/specs directory.    It will even run multiple tests at the same time to speed things up, opening multiple windows at once.    Configuring and managing test suites, synchronous versus asynchronous and other options are for another article.

Go to Run > Edit Configurations > NodeJS

Configuring WDIO Tests in phpStormClick the Add [+] while Node.js is highlighted to add a new NodeJS script configuration.

Name it WDIO Tests.

Change the Node interpreter to your project’s wdio binary ~/phpStorm Projects//node_modules/.bin/wdio.

You will likely need to click the more button […] and add this as a new Node interpreter. When you do, click the Add button [+] and then specify the .bin/wdio file in your local project install. You can leave the global NPM libraries intact.

Note: The way NodeJS works is it will look there first, then look in your local directory where the script runs for a ./node_modules subfolder, then the PARENT of your directory for ./node_modules and work its way from the script directory back to the system root. Since you put your test folder UNDER the main phpStorm project folder the scripts will work their way up the folder tree eventually finding your phpStorm-installed ./node_modules that live in the root folder of your project including Mocha, Chai, and all the other goodies.

Now run all your WebdriverIO tests you created:

Go to Run > Run… > WDIO Tests

A console window will open running all the tests and reporting results.

What is reported back depends on which logging tool you selected during configuration (I use dots) and logging level (I use ‘error’ or ‘verbose’). You can edit your wdio.conf.js at any time to change this.

 

While there is a lot to digest here it is not as complicated as it seems. With a  little effort you can start writing tests for your web apps to ensure a quality stable product or automate repetitive data entry or configuration tasks.     The JavaScript syntax used to write the tests is simple and well-documented on the WebdriverIO API , Chai, and Mocha pages.     You can also move forward with testing your apps even after the demise of Selenium IDE.