Cypress vs Selenium

In the past Selenium was considered the ideal test automation tooling for you, if you needed real browser based tests for your web application. With Selenium you could focus on real user experience (UX) tests for unit and regression testing.

For a while now you may have read a lot about Cypress. Cypress is the new kid on the block and supposedly the new fancy way of web UI application test automation as well as the only and ideal alternative to Selenium. Many Cypress developers and users are predicting the end of Selenium coming soon. We can not read the crystal bowl. Instead we prefer to let the facts decide. Having said that, we have a slightly different opinion compared to most Cypress promoters. Let us explain why by diving deeper into both frameworks.

1. Why Selenium?

Selenium WebDriver, we hate and we love it, depending on how deep you need to go into the framework for your tests. The more you focus on UX (real end user regression testing), the more you get into the internals of the WebDriver. However, from our experience in about 98% of the cases you'll never face any restrictions considering the abstraction over all possible browsers.

1.1. Possiblities of using Selenium

To consider the possibilities of Selenium, we will get familiarized with the benefits of Selenium.

1.1.1. Ease of learning

It is said that developers have a steep learning curve at the beginning of using Selenium. This is true if the developers are not aware that they are performing remote controlling of a browser. And in simple terms Selenium could be considered as a remote control of a running browser. What increases the complexity of Selenium is the required interprocess communication between your test code, your browser and sometimes by an additional native WebDriver process. The native WebDriver typically gets automatically executed by Selenium and you may have seen it already (e.g. with the processes geckodriver or chromedriver).

1.1.2. Browser abstraction

In many UI testing frameworks, Selenium WebDriver is incorporated to enable test automation on browsers. This means that all major browsers natively support Selenium WebDriver for testing, which makes Selenium quite interesting for developers. Nowadays you can use WebDriver for any major browser, including:

• Firefox
• Chrome
• Chrome (headless mode)
• Internet Explorer
• Safari
• Opera
• HtmlUnit (headless mode)

This means you can easily do a multi-browser testing with your Selenium based tests by running the same tests with multiple different WebDrivers. Of couse, mostly the main issues arrise with the browsers which do not focus on standards, such as the IE.
But now you have an easy and fast way to identify browser issues in your application.

1.1.3. Operating system

Selenium supports many operating systems, enabling you to develop once and run the code on multiple operating systems. You can run your tests on:

• Windows
• Mac
• Linux
• Android
• iOS (partially)

1.1.4. Programming languages

One major benefit of Selenium is the support for many different languages. You can use nearly any of the major languages with a relevant market share. Some of the supported languages are:

• Java
• JavaScript
• C#
• Python
• Ruby
• Perl
• PHP

This wide support for programming languages makes Selenium quite interesting for many developer communities using different languages.

1.1.5. The Selenium license

Selenium is open source and released under the Apache 2.0 license.

1.1.6. Available documentation

Wide support of browsers, programming languages and operating systems makes Selenium a very well documented testing framework.

1.1.7. Maturity of Selenium

Selenium was developed in 2004. Selenium 2 was introduced in 2009 and Selenium 3 in 2016. Although Selenium is already rather old considering the year of the first release, the project has been regularly developed and updated.

1.1.8. Using for load and remote testing

Selenium brings in the RemoteWebDriver, which even makes it possible to run remote and load tests with real browsers and real browser behaviour.

1.1.9. Testing framework and libs

Selenium does not force you to use any specific testing framework. You can use JUnit 3, 4, 5, TestNG, Cucumber, Mocha, Phantom, and many more since Selenium is "only" abstracting the browser layer for the developers. This makes Selenium very handy for any tech lead, as you can stay with your preferred programming language and testing framework.

1.1.10. Basic Architecture

Selenium runs outside of the browser and communicates with the browser through the network. You have your test code, which is typically running in one process; often you have a custom driver, which is sometimes natively implemented in Selenium or in a custom addition process like the geckodriver, which runs outside of the browser and outside of the test code. This means you have the indirection of inter-process communications through the network. This communication is well abstracted, but you need to know this to understand odd issues that sometimes arise when using Selenium.

Synthetic vs native events

What are synthetic and what are native events in terms of UI testing?

Native events

As a user you either use the keyboard with keystokes, the mouse with moving or clicking or a touch device with touching or gestures. In all cases the operating system is passing these human events to the browser. The browser itself translates these "native" events into the JavaScript events. This means all human interactions end in a JavaScript "native" event. Selenium's default behaviour is to always prefer the usage of native events. This means that if you tell Selenium in a test to click on a button, Selenium will use the browser's native variant to fire a real native event in the JavaScript runtime.
The benefit of native events is the fact that the real user interaction can be simulated like the user would have really clicked on a button.
But sadly there is also a drawback. Native events are not platform independent. This means the abstraction layer of Selenium needs to cope with all different operating systems, to make platform dependency as irrelevant as possible for the developers.

1.1.11. Synthetic events

As an opposite to native events, synthetic events are only JavaScript events, which correspond to its native variants. This means that instead of telling the browser "hey browser, you received a mouse click from the OS, please translate it into the JavaScript event mouse click", you can shorten the path by telling the browser's JavaScript runtime "hey JS-runtime, the browser received a mouse-click event".

Of course synthetic events evict the need of abstracting native events for different operating systems, but that also brings in a new fact - that you cannot be sure if you get the same behaviour as when a real user is firing real (native) events in the browser.

To give you a good example which drawbacks synthetic events have:

Let's assume you need to simulate a mouse move and a click after hovering over the right WebElement, in our example a button. Now assume this button is hidden by a layer on top of the button layer. If you now fire a native mouse click event, the browser would get a native click event and would try to click on the area where the button should be. But since the button is hidden by the layer on top of the button, the button click will not be executed.

Also, consider the TAB-Key behaviour of web applications. When you press the 'tab' key your focus jumps from the current WebElement to the next element. This is done by getting the native TAB-Key event from the operating system and passing it to the browser. With synthetic events the browser does not get a 'tab' key is pressed event and therefore it will not change the focus in your application.

2. Why Cypress?

Cypress is gaining a lot of attention, but what makes Cypress so special during the development?

2.1. Possiblities of using Cypress

What are the benefits of Cypress?

2.1.1. Ease of learning

The Cypress users are of opinion that it is easier to use Cypress compared to Selenium. This means you can easily test your web application with Cypress. You need to install an exe and everythings gets automatically done for you. Also, the developer friendly documentation helps a lot.

2.1.2. Browser abstraction

Even if the developers of Cypress had cross-browser testing in mind, they have not implemented any other browser support apart from Chrome untill February 2019. 

This means you can not do real cross-browser testing, which is crucial for environments with non unique browser environments.

2.1.3. Operating system

Just like Selenium, Cypress also supports various operating systems. This means you develop once and run the code on multiple operating systems. You can run the tests on:

• Windows
• Mac
• Linux

2.1.4. Programming languages

The simplicity of Cypress to support only one programming language (JavaScript) is also its major drawback. There is no other option to develop browser based tests but with JavaScript.

2.1.5. The Cypress license

Cypress was also released as open source under the MIT license. However, compared to Selenium not all parts are open source, e.g. the dashboard in Cypress is closed source.

2.1.6. Available documentation

The documentation is good, not as good when compared to Selenium, but you can find all relevant details for development with Cypress.

2.1.7. Maturity of Cypress

Cypress is a very young framework compared to Selenium. It was initially started in 2015 and the first 1.0.0 version was released in 2017. The quality of the issues getting reported and fixed already looks quite mature. Also, the focus on only one browser is reducing the complexity for the Cypress developers compared to Selenium.

2.1.8. Using for load and remote testing

Cypress can be used with Mocha and Chai for testing. Whatever they support, Cypress will also support it. By design neither is implemented to do load testing.

2.1.9. Testing framework and libs

Since Cypress is running only on JavaScript, you are already limited to testing frameworks in the JavaScript cosmos. In addition to this, Cypress tests are written using Mocha with Chai. This means if you are already using a different framework, you have to switch it or find a custom integration. Compared to the freedom you have with Selenium this looks like a big restriction on your decisions as an architect.

2.1.10. Basic Architecture

Unlike Selenium, Cypress runs inside the browser in the JavaScript runtime environment. The architectural complexity is simpler compared to Selenium, which makes the use of Cypress much easier for young developers. Also, the steep learning curve is not noticeable (at least until Cypress starts to support multi-browser and native events).

Cypress is built on top of the Electron app, and since Cypress runs IN the browser, it allows the developers to do very unique DOM manipulations directly in the browser and during testing. Also, Cypress does not need any network communication, which brings it to often cited "developer friendly test automation".

2.2. Synthetic vs native events

Cypress has plans to support native events, but as of February 2019 there are no specific roadmaps to get native events implemented.

2.2.1. Native events

Not supported as of February 2019.

2.2.2. Synthetic events

Cypress only supports synthetic events. Since it is running in the browser, Cypress allows the testing developers to test all the functions you can execute in the browser's JavaScript runtime.

3. Conclusion for Cypress vs Selenium

As long as your main programming language is JavaScript, you should consider to use Cypress. Keep in mind that Cypress does not support you on cross-browser testing or really simulates real user events. If you are also not bothered by the restrictions on Mocha and Chai, using Cypress makes sense.

If you are already using a different main language than JavaScript, you should double check if you want to restrict yourself with Cypress. You would most likely restrict yourself with the need for using another testing framework such as JUnit or TestNG. Also, if you need to simulate real user interactions you need the native event support, which means you should not use Cypress until they find a native event support solution. Additionally, if you consider the cross-browser testing support, you currently have only one choice: to use Selenium.