forked from SeleniumHQ/docs
-
Notifications
You must be signed in to change notification settings - Fork 0
/
guidelines.html
283 lines (215 loc) · 11.6 KB
/
guidelines.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
<!doctype html>
<meta charset=utf-8>
<title>Guidelines and Recommendations</title>
<link rel="icon" href="favicon.ico" type="image/vnd.microsoft.icon">
<link rel="shortcut icon" href="favicon.ico" type="image/vnd.microsoft.icon">
<link rel=stylesheet href=se.css>
<link rel=prev href=remote.html title="Remote WebDriver">
<link rel=next href=worst.html title="Worst Practices">
<script src=docs.js></script>
<h1>Guidelines and Recommendations</h1>
<p>A note on "Best Practices": We've intentionally avoided the phrase "Best
Practices" in this documentation. No one approach works for all situations.
We prefer the idea of "Guidelines and Recommendations." We encourage
you to read through these and thoughtfully decide what approaches
will work for you in your particular environment.
<p>Functional testing is difficult to get right for many reasons.
As if application state, complexity, and dependencies don't make testing difficult enough,
dealing with browsers (especially with cross-browser incompatibilities)
makes writing good tests a challenge.
<p>Selenium provides tools to make functional user interaction easier,
but doesn't help you write well-architected test suites.
In this chapter we offer advice, guidelines, and recommendations.
on how to approach functional web page automation.
<p>This chapter records software design patterns popular
amongst many of the users of Selenium
that have proven successful over the years.
<h2>Page object models</h2>
<p>Page Object is a Design Pattern which has become popular in test
automation for enhancing test maintenance and reducing code
duplication. A page object is an object-oriented class that serves as
an interface to a page of your AUT. The tests then use the methods of
this page object class whenever they need to interact with that page
of the UI. The benefit is that if the UI changes for the page, the
tests themselves don’t need to change; only the code within the page
object needs to change. Subsequently, all changes to support that new
UI are located in one place.
<p>The Page Object Design Pattern provides the following advantage:
there is clean separation between test code and page specific code
such as locators (or their use if you’re using a UI map) and layout.
<h3>Page object methods should return a value</h3>
<ul>
<li>If you submit a page and are redirected,
it should return the new page object.
<li>If you click submit on login
and you want to check to see if a user is logged in,
it should return True or False in a method.
</ul>
<h2>Domain Specific Language</h2>
<p>A domain specific language (DSL) is a system which provides the user with
an expressive means of solving a problem. It allows a user to
interact with the system on their terms – not just programmer-speak.
<p>Your users, in general, don't care how your site looks. They don't
care about the decoration, animations, or graphics. They
want to use your system to push their new employees through the
process with minimal difficulty. They want to book travel to Alaska.
They want to configure and buy unicorns at a discount. Your job as the
tester is to come as close as you can to “capturing” this mind-set.
With that in mind, we set about “modeling” the application you're
working on, such that the test scripts (the user's only pre-release
proxy) “speak” for and represent the user.
<p>With Selenium, DSL is usually represented by methods, written to make
the API simple and readable – they enable a report between the
developers and the stakeholders (users, product owners, business
intelligence specialists, etc.).
<h3>Benefits</h3>
<ul>
<li><strong>Readable:</strong> Business stakeholders can understand it.
<li><strong>Writable:</strong> Easy to write, avoids unnecessary duplication.
<li><strong>Extensible:</strong> Functionality can (reasonably) be added
without breaking contracts and existing functionality.
<li><strong>Maintainable:</strong> By leaving the implementation details out of test
cases, you are well-insulated against changes to the AUT.
</ul>
<h3>Java</h3>
<p>Here is an example of a reasonable DSL method in Java.
For brevity's sake, it assumes the `driver` object is pre-defined
and available to the method.
<pre><code class=java>/**
* Takes a username and password, fills out the fields, and clicks "login".
* @return An instance of the AccountPage
*/
public AccountPage loginAsUser(String username, String password) {
WebElement loginField = driver.findElement(By.id("loginField"));
loginField.clear();
loginField.sendKeys(username);
// Fill out the password field. The locator we're using is "By.id", and we should
// have it defined elsewhere in the class.
WebElement passwordField = driver.findElement(By.id("password"));
passwordField.clear();
passwordField.sendKeys(password);
// Click the login button, which happens to have the id "submit".
driver.findElement(By.id("submit")).click();
// Create and return a new instance of the AccountPage (via the built-in Selenium
// PageFactory).
return PageFactory.newInstance(AccountPage.class);
}</code></pre>
<p>This method completely abstracts the concepts of input fields,
buttons, clicking, and even pages from your test code. Using this
approach, all your tester has to do is call this method. This gives
you a maintenance advantage: if the login fields ever changed, you
would only ever have to change this method - not your tests.
<pre><code class=java>public void loginTest() {
loginAsUser("cbrown", "cl0wn3");
// Now that we're logged in, do some other stuff--since we used a DSL to support
// our testers, it's as easy as choosing from available methods.
do.something();
do.somethingElse();
Assert.assertTrue("Something should have been done!", something.wasDone());
// Note that we still haven't referred to a button or web control anywhere in this
// script...
}</code></pre>
<p>It bears repeating: one of your primary goals should be writing an
API that allows your tests to address <strong>the problem at hand, and NOT
the problem of the UI</strong>. The UI is a secondary concern for your
users – they don't care about the UI, they just want to get their job
done. Your test scripts should read like a laundry list of things
the user wants to DO, and the things they want to KNOW. The tests
should not concern themselves with HOW the UI requires you to go
about it.
<h2>Generating application state</h2>
<p>Selenium should not be used to prepare a test case. All repetitive
actions, and preparation for a test case, should be done through other
methods. For example, most web UIs have authentication (e.g. a login
form). Eliminating logging in via web browser before every test will
improve both the speed and stability of the test. A method should be
created to gain access to the AUT (e.g. using an API to login and set a
cookie). Also, creating methods to pre-load data for
testing should not be done using Selenium. As mentioned previously,
existing APIs should be leveraged to create data for the AUT.
<h2>Mock external services</h2>
<p>Eliminating the dependencies on external services will greatly improve
the speed and stability of your tests.
<h2>Improved reporting</h2>
<p>Selenium is not designed to report on the status of test cases
run. Taking advantage of the built-in reporting capabilities of unit
test frameworks is a good start. Most unit test frameworks have
reports that can generate xUnit or HTML formatted reports. xUnit
reports are popular for importing results to a Continuous Integration
(CI) server like Jenkins, Travis, Bamboo, etc. Here are some links
for more information regarding report outputs for several languages.
<!-- TODO: Add links.-->
<h2>Avoid sharing state</h2>
<p>Although mentioned in several places it is worth mentioning again. Ensure tests are isolated from one another.
<p>Don't share test data. Imagine several tests that each query the database for valid orders before picking one to perform an action on. Should two tests pick up the same order you are likely to get unexpected behaviour.
<p>Clean up stale data in the application that might be picked up by another test e.g. invalid order records.
<p>Create a new WebDriver instance per test. This helps ensure test isolation and makes parallelisation simpler.
<h2>Test independency</h2>
<p>Write each test as its own unit. Write the tests in a way that won't be reliant on other tests to complete:
<p>Let's say there is a content management system with which you can create some custom content which then appears on your website as a module after publishing, and it may take some time to sync between the CMS and the application.
<p>A wrong way of testing your module is that the content is created and published in one test, and then checking the module in another test. This is not feasible as the content may not be available immediately for the other test after publishing.
<p>Instead, you can create a stub content which can be turned on and off within the affected test, and use that for validating the module. However, for content creation, you can still have a separate test.
<h2>Consider using a fluent API</h2>
<p>Martin Fowler coined the term "Fluent API". Selenium already
implements something like this in their <code>FluentWait</code> class which is
meant as an alternative to the standard <code>Wait</code> class. You could
enable the Fluent API design pattern in your page object and then
query the Google search page with a code snippet like this one:
<pre><code class=java>driver.get( "http://www.google.com/webhp?hl=en&tab=ww" );
GoogleSearchPage gsp = new GoogleSearchPage();
gsp.withFluent().setSearchString().clickSearchButton();</code></pre>
<p>The Google page object class with this fluent behavior
might look like this:
<pre><code class=java>public class GoogleSearchPage extends LoadableComponent<GoogleSearchPage> {
private final WebDriver driver;
private GSPFluentInterface gspfi;
public class GSPFluentInterface {
private GoogleSearchPage gsp;
public GSPFluentInterface(GoogleSearchPage googleSearchPage) {
gsp = googleSearchPage;
}
public GSPFluentInterface clickSearchButton() {
gsp.searchButton.click();
return this;
}
public GSPFluentInterface setSearchString( String sstr ) {
clearAndType( gsp.searchField, sstr );
return this;
}
}
@FindBy(id = "gbqfq") private WebElement searchField;
@FindBy(id = "gbqfb") private WebElement searchButton;
public GoogleSearchPage(WebDriver driver) {
gspfi = new GSPFluentInterface( this );
this.get(); // If load() fails, calls isLoaded() until page is finished loading
PageFactory.initElements(driver, this); // Initialize WebElements on page
}
public GSPFluentInterface withFluent() {
return gspfi;
}
public void clickSearchButton() {
searchButton.click();
}
public void setSearchString( String sstr ) {
clearAndType( searchField, sstr );
}
@Override
protected void isLoaded() throws Error {
Assert.assertTrue("Google search page is not yet loaded.", isSearchFieldVisible() );
}
@Override
protected void load() {
if ( isSFieldPresent ) {
Wait<WebDriver> wait = new WebDriverWait( driver, 3 );
wait.until( visibilityOfElementLocated( By.id("gbqfq") ) ).click();
}
}
}</code></pre>
<h2>Fresh browser per test</h2>
<p>Start each test from a clean known state.
Ideally, spin up a new virtual machine for each test.
If spinning up a new virtual machine is not practical,
at least start a new WebDriver for each test.
For Firefox, start a WebDriver with your known profile.
<pre><code class=java>FirefoxProfile profile = new FirefoxProfile(new File("pathToFirefoxProfile"));
WebDriver driver = new FirefoxDriver(profile);</code></pre>