Automated browser testing, explained in code

Now, when you're running in CI, this is going to be a headless Chrome instance. You don't want anything popping up. But in development and testing, you're going to want it to pop up so that you can see what's happening and interact with it for debugging. Next you're going to be issuing commands. And that's exactly what browser tests do, is you're actually sending commands to the browser telling it to click, telling it to send keys, telling it to move in a certain way or hover or drag and drop. These commands will then return data. This data you'll use to either drive more commands or maybe do some assertions. Now when you finish your interactions, your probably next step is going to be able to run SQL queries because you're going to want to see if the interactions that you meant to do in the remote system were actually done. And so you're going to access the data and the database to verify that.

And that's an important step. Again, the more assertions you can do by hitting the database directly and avoiding driving the browser, the less flaky your tests are going to be and the more fast they're going to be. And finally you do assertions based on the data. So this could be data based from the database. This could be data retrieved from elements, whatever. And that's pretty much it. That is a test from beginning to end. You set up the initial data, again, as little test driving the browser as you can. Then you do your tests where you're accessing data. Then you get the data from either the web or from the database and you do your assertions. Let's take a look at how browser tests work in the real world, explained through code. And to do that, I'm going to show you how we do testing in my production application, Sleuth.

This application is built on Python and Django and uses selenium for the web browser. But the concepts that I'm talking about could be useful in any sort of framework. So to show this working, I'm going to dig into a test, this test notifications. This is going to be testing a notifications UI. Let's see what it looks like. Here is the class, test my notifications. Notice there's not a lot to it. It should be pretty self-explanatory. You can almost read exactly what's happening. We're logging in, we're connecting, we're doing some assertions, and we're performing some actions. So let's walk through it line by line. The first step is we need to log in. And notice we're calling this login page. I'm not going to URL. We're not touching anything in the webpage. We're calling this page object. This is what a page object is. It is an object that encapsulates all browser interactions so that you can work with this at a high level. And then as your UI changes, your tests continue to work at it at a high level. Let's dig into login page and see what it's like.

So I put my pages in a separate file. This allows me to reuse them amongst different tests. And here's what it looks like. We have our login page up here. And we're going to do some setup, and then we're going to actually perform the log in. Let's dig into that log in method. So the first thing we're going to do is we're going to go to a URL, go to the accounts login. We're going to find an element called username. We're going to type in that element. We're going to click a button. Type in another element, click a button, and wait until everything is done. And that's it. Notice, this is actually the part of your code that interacts with the browser. It is driving the browser, clicking, typing, doing things. You want to isolate all that code inside of the page object for a couple reasons. First, if this structure of the log in page changes, you don't need to change your test.

And in fact, this just happened to us in Sleuth. A couple weeks ago, we decided to go through our login process and update it with new screens, a new flow, whatever. Our tests, however, didn't have to change because people used this login method. Our tests created the login page, used the login method, and nothing changed for the tests. We had to change one place, this function, to use the new structure. The second thing this does is. It makes your test very readable. So let's go back into our test and I'll show you what I mean in more detail. So now we're back in test notifications, and you can see that here's the log in page. The second thing after we've logged in is, we create another page object. This page object is pretty specific to this test. It's testing the my identities page. And notice, we go to it and then we run a connection.

We perform an action on this page. So not only do we create page objects that are reusable amongst many tests, but we also create page objects that are used even just for one test because it keeps the test code readable. When you're creating an automated browser test, what takes the time isn't writing the test. You think that, oh, I need to make test creation really easy because it's such a pain in the ass. True. But if you look at the actual lifetime of a test, maintaining the test is the hardest part. Keeping it up to date as your application changes and evolves, making sure it's not flaky, where you might have different things that run at different times and sometimes the test passes, sometimes it doesn't. Huge problem to debug and maintain because you need to be able to trust your test. If you're using tests as a gate between you, your code, and delivering the production that test needs to be bulletproof.

And so you want to keep the test as simple and as readable as possible so they can be maintained in the long term. After we create this page object, we do some assertions. So down here we're asserting that the personal slack checkbox is true, that the personal deploy checkbox is true. Now, again, notice we're not accessing any elements on the page. We're dealing with it at that higher level of abstraction. And this enables a hidden benefit that you wouldn't think about using page objects in this style and that is, you can start doing test driven development with your selenium tests. So what I mean by that is, if you're going to design a new page, you can, at this high level using this kind of code, you can write the high level interactions, the main pieces that should be on the page, the main type of actions that should happen.

Write this test. It will fail, of course. Because you haven't implemented the page. Implement the page and continuously run the test, implement the page until the test pass. And at that point, you know you have implemented the functionality. Test driven development is a separate topic, but I will say I've had a lot of success using test driven development with selenium pages to help me design and implement new features. As much as you try, you're going to get flaky tests. You're going to get tests that work sometimes and fail other times. And when that happens, it's usually failing in CI, not in your system, which means it's hard to reproduce. So here's a little tip that I've picked up that will save you a ton of time when you're trying to debug these tests. And that is this little bit of code. What this is doing is we're currently using Django's test framework so we can hook into a cleanup process.

But conceptually, after each run of a test, if there is any errors like you see up here then it creates a screenshot. So what you're doing is you're capturing the state of the browser at the point of failure so that you can go into the test run and see exactly what was happening. It's not perfect. A perfect one would record a whole video and there are some test frameworks that help you with that. But at a very minimum, you need to get a screenshot of what the browser was doing so that if you know it was on the wrong page or what was happening so that you can debug it and hopefully reproduce it and solve it quickly. So that's just a little bit of code that I write. Sometimes you have test frameworks that have this built in. Sometimes you need to build it like in the case of selenium. But once you have this, it'll help you a lot in debugging your tests.

There's a lot more to browser based testing, but that's how we do it in my company, Sleuth. I also stream open source and Sleuth development on Twitch, Tuesdays and Thursdays. Information down below. Feel free to come by with questions or just to hang out. If you're interested in continuous delivery, continuous deployment, CICD, testing, or anything related, please let me know down on the comments if you'd like to see a video on those topics. If you're interested in continuous delivery, we have this video that I made that walks through how I use continuous deployment inside of Sleuth itself. Or if you're interested in mistakes that you want to avoid when you're adopting continuous deployment, check out this video down below. Happy deploying.