More Testing is More Better
- 65 Topics
- 20 Replies
Hey folks! A couple of months ago, at Sauce Con Online, I got the chance to dig around in the source code of the Sauce Labs Slack plugin and refactor the tests. It needed that, because it’s currently in a broken state… And fixing it is one of the things the Community team will be tackling sometime soon.We’ve got scope to add a liiiitle bit of additional functionality, and what we add is up to you. Currently, the Slack plugin can:Retrieve and embed details about a job Retrieve and embed details about a build Retrieve and embed details about a userSo what’s it gonna be?
Hello folks! We’d like some help with terminology here at Sauce Labs. Specifically, we want to talk about Builds. We introduced a Build reporting system several years ago, with the intent of letting customers group a series of tests together, to indicate they all ran at the same time. This gave customers easier reporting and matched how testing is usually done in a CI process. This feature works great; The problem is the name. See, it’s pretty common for Build to refer to a specific version of the code under test, but not the tests ran against that code. For instance, if you have a quarterly release with a pre-determined release number, you might refer to that release number as the Build. Or, you might have several suites of independent tests, running against the same checkin, and only consider that checkin to be a Build. So, we’re looking for a better, clearer term for: A single execution of a group of tests against a specific version of a system ran at a single point in time No
Does this sound familiar to you?“My manager never lets me attend conferences”We understand; It can be difficult to convince managers to spare you the time and money you need for training and development. While SauceCon Online should be an easier sell (As an online event, tickets range from Free to Not Very Expensive), you’ll still need to convince your manager it’s time well spent. That means writing a business case. Step One - Gather DataCheck out the SauceCon Online Schedule to see what sessions you’d like to attend. Tying sessions back to your company’s needs is your main weapon. Look for sessions whose content either supports an upcoming project, or talk about overcoming a challenge your company is facing. Step Two - Summarise ValueGiving your manager a series of takeaways can help show what direct advantages they’ll get from your attendance. Some of the takeaways for SauceCon Online include:API Testing Practices An intro to Accessibility Testing Overcoming challenges in Mobil
About SauceCon Online 2021SauceCon brings together the global community of Sauce Labs users and automated testing experts. Join us online April 20-22, as teams from around the world will come together to learn from each other and level up their automated testing and continuous delivery skills. What can I expect?Two days of talks 33 world class speakers Access to Sauce Labs staff Best Practises for Testers, Developers, Coders, DevOps Leaders and teams like yours 6 In-Depth half-day workshops on API testing, Automation, WebDriverIO, Docker, Cypress and Test FrameworksYou can watch all the SauceCon Online 2020 sessions for free, right here to get a better idea of what SauceCon Online is like. When is it?April 20-22 What are the workshops?Check them out here. How Much Are Tickets?Tickets. Are. FREE.Tickets are tiered, with the “Free” tier getting you access to all Keynote Sessions and Industry Technical Presentations.Basic TicketsGive you access to everything in the Free Tier, plus breakou
You may have heard about the Sauce Labs Testrunner Toolkit, our new containerized solution for running Cypress (or Playwright, or Puppeteer, or...) in Sauce Labs.We started working on the Testrunner Toolkit because we wanted to make it easy for customers to make use of these newer frameworks, and bring the advantages of our insights and analytics tools to teams using them. Something I’ve been hearing from customers and a couple of my colleagues, though, is the question:Why wouldn’t you just use Selenium? I think it’s great that Cypress and its ilk have been able to learn from the evolution of Selenium and WebDriver and make something new, and I’m curious; are you thinking of switching, or have you switched already?What makes you and your team excited about them, and what makes them nervous?
Over on Stack Overflow, user sam78 asked a question about starting a testing department from scratch.Although it’s a very broad question, it is an interesting one… Is there any general advice you could offer people who are creating a formal testing situation for their employer?I’m going to put my own answer below, but I’d love to hear your thoughts on what a new testing department should aim to do, how they should do it, and how to demonstrate value and success.
Psoriasis is a condition that can cause breaks in the skin by causing severe dryness that can even lead to bleeding sometimes. It is important to get the best ayurvedic treatment for psoriasis to keep yourself from the severity of this condition. The best ayurvedic treatment for psoriasis ensures the removal of psoriasis roots and reverses the whole condition, and the skin looks as if psoriasis was never there in the first place. Name -: Dr. Amit VermaMobile -: 8303000777Add -: E 1/12, 1st floor, Main Market Rd, Malviya Nagar, New Delhi, Delhi 110017Website Mail -: email@example.comWebsite -: https://www.vrikshakalpaayurveda.com/treatments/psoriasis-treatment/ Social -: 1. https://www.facebook.com/vrikshakalpaayurveda/2. https://twitter.com/vrikshakalpa3. https://www.instagram.com/vrikshakalpa_ayurveda/
To me, (history is) a great source of systems evolutions and dynamics examples.Writes Michael Dubakov in his article about hypertext tools from the 80's.He goes over a few different systems that use Hypertext, and some of them have pretty advanced features, even today.For instance, Warm Linking, which gives you the ability to push and pull content across links.Most notable is that the internet didn't evolve this way; It's less connected than it was before. Tools like Notion are trying to provide a universal hyperlink content document editor workspace... thing... But they're limited, and most of us spend time in multiple webapps instead.It's well worth a read.
Adding #TODO: Refactor this! to your code gives you fuzzy little feelings of happiness that you intend to improve things "later".You won't, though.Not unless you install Todo or Die. This Ruby gem (also Rust) does something mere comments could) never do: It actually reminds you to do the thing.TodoOrDie("Remove after TestObject fully integrated", by: "2022-01-01")As author Justin Searls puts it:So I did what any programmer would do in the face of an intractable social problem: I wrote code in the vain hope of solving things without needing to talk to anyone. And now this gem exists.Available now where-ever good Rubygems are installed.
Every year the hosting/CDN/Modern Web Enablement company Netlify conducts a survey of the state of the Jamstack community.This year's results have some interesting insights into the industry.Everyone (OK, 80% of people) agree with Sauce Labs that Mobile is critically important to their business... And since responses came across all industries, this probably applies to you and your work, as well. Hate tiny screens? People are still targeting desktop, as well.A quarter of Jamstack developers are building sites for Enterprise software, with a third also building B2B software solutions. These sites serve anywhere between 1 and millions of users, too. The more you work on bigger sites, the more likely you are to specialise (say, as a Front End developer). These sites are becoming more sophisticated; almost half of all respondents use microservices, functions-as-a-service, or containers to power their backend. "Static" isn't so static anymore.Respondents rated Performance as their mos
Do you use use Maven as the dependency manager for your projects? If so, this article will give you some insight into how it works, along with some useful commands. You will find these especially helpful if you are using a private repository as the main repo to source the dependencies for your project. By using the dependency list generation methods described in this article, you can greatly reduce the workload needed to make dependencies available in your private repository, or locally if you intend to run your project offline.How Does Maven Manage Dependencies?You add dependencies for your project to your Maven configuration file (also known as the pom.xml file, for Project Object Model). As you build your project using Maven, it resolves these dependencies and downloads the dependencies to your local repository folder. This folder is usually located in your user’s home folder and is named .m2. Each dependency downloaded from the repository is a project itself, and has its own depen
There are several ways to handle authentication and security dialogs during testing. You might, for example, need to log a user in, or bypass a browser warning. The topics in this section cover cookie injection with Selenium, basic HTTP authentication, and other options.Basic HTTP AuthenticationBasic HTTP authentication supplies a username and password via URL, such as https://admin:firstname.lastname@example.org/basic_auth. This request sends the credentials in the standard HTTP "Authorization" header.Because browser support for basic HTTP authentication is limited, we recommend Injecting Cookies to Bypass Authentication Dialogs and Running an AutoIt Script as a Pre-run Executable to Handle Windows Security Authentication Dialogs as solutions for authentication while testing. Browser HTTP Authentication Support Chrome Supported Firefox Supported, but Firefox will display a prompt asking you to confirm. Safari Unsupported. See the Support KB article Setting Basi
Most testers experience having to deal with flaky tests whether or not they realize it. Learn how to deal with flaky Java tests to improve your testing.In the world of user interface (UI) testing we all have come across tests and UI behavior that is not 100% repeatable. In other words, sometimes tests are “flaky.” The challenge with these tests is in trying to determine what causes this flakiness, and finding ways to resolve it, so that we are able to test the UI in a reliable fashion without raising false alarms and creating test gaps.Failure ModesThe two most common situations that lead to flakiness are external factors over which one has no control, and the application or website you are testing not being in the expected state when you test it. Among the external factors, network conditions, different client types, and test infrastructure limitations are among the most common causes of flakiness.The way to diagnose these factors usually boils down to comparing the physical testing e
WebDriver provides a number of locator strategies for accessing elements on a webpage. It's tempting to use complex XPath expressions like //body/div/div/*[@class="someClass"] or CSS selectors like #content .wrapper .main. While these might work when you are developing your tests, they will almost certainly break when you make unrelated refactoring changes to your HTML output.Instead, use sensible semantics for CSS IDs and form element names, and try to restrict yourself to using these semantic identifiers. For example, in Java you could designate elements with driver.findElement(By.id("someId")); or driver.findElement(By.name("someName")); or, in the example of PHP, you could use $this->byId() or $this->byName() . This makes it much less likely that you'll inadvertently break your page by shuffling around some lines of code.
As you move into fully automated testing and builds, with tests running in parallel and against multiple device/browser/platform/operating system combinations, be aware of the load that this will place on both your CI/CD server and the site under test. A best practice in this situation is make sure that both the CI/CD server and the site under test are on machines that are not running additional processes or open to additional network traffic, and can handle the additional number of simultaneous jobs and tests.
It's a reality that tests run on Sauce Labs will always have some latency, compared to running locally, due to there being an Internet in the middle between your tests and our browsers. So, rather than running in the same machine, each Selenium request is sent through the wire to our VMs and receives a response back through the same channels. These are some other common factors that contribute to test duration, and a few tips for using your minutes efficiently. Identify Time SinksIf you're testing against a local application server, you will see some additional latency due to the distance from our browsers to your server. The same applies if you're using Sauce Connect to test against a firewalled server. When you're running Sauce Connect, it sends all requests from Sauce's browsers back through the machine running Sauce Connect, such that they appear to originate from that machine. This means that all data sent between the Sauce cloud and the site you're testing must travel first to y
Imperative v. declarative test scenarios is a concept that is often discussed in the context of Cucumber and Behavior Driven Development (BDD), but it is applicable to all languages and test runners. A great post by Aslak, the author of Cucumber, gives a great description of his intentions along with a number of additional links at the bottom for further reading.Imperative testing or programming is essentially spelling out with as much detail as necessary how to accomplish something. Declarative testing or programming is only specifying (or declaring) what needs to be accomplished.This is seen acutely in BDD circles because the goal of BDD is to get all of the interested parties (Project, Dev, Test, Business, etc) to collaborate on the requirements of a feature before anyone begins working on the implementation. Many testers have latched on to BDD tools as glorified test runners rather than a way to actually facilitate BDD practices. This results in features that include actual code an
Sauce Labs test logs are securely stored, protecting them from external access. However, there are still some groups that can see test logs, including Sauce Support, your parent account, and other accounts in your company (depending on test privacy settings).Solution - Don't use real credentialsThe best way to avoid this is to avoid using "real" credentials in tests, through the creation of temporary accounts.Workaround - Transmit session tokens onlyYou can also avoid sensitive credentials using Selenium's ability to extract and inject cookies into accounts:Create a session in your environment, either directly in the application engine, or by using a local Selenium session or headless browser. Extract the session tokens (local storage objects, credentials, cookies, etc.). Use Selenium to push these objects and tokens into the browser under Sauce Labs' controlThis technique avoids sending plain text passwords, however, the sent tokens and cookies are still logged. If your session tokens
Reusing your test accounts is an antipatternReusing an account between test runs can lead to:Problems with account state before testing starts Failures when account setup code has changed Failures that only show on other accounts (like your production customers) Parallelisation problems between tests using the same accountWhen should you create a new account?Roughly speaking, whenever your tests aren't interacting with previous or future tests:Running a test on different platforms Running a test that doesn't depend on other test state (tests with shared state are also an antipattern) Every time you run a test suite
There will always be flaky tests, and tests that once breezed through with no problem can fail for what seems like no reason. The trick is figuring out whether a test that fails does so because it found a real problem in your app functionality, or because there was an issue with the test itself. The best way to handle this problem is to log your failing tests into a database and then analyze them. Even tests that fail intermittently with no apparent cause may turn out to have a pattern when you are able to analyze them in detail and as a larger data set. If this is beyond the scope of your testing setup, the next best strategy is to log your failing cases into a log file that records the browser, version, and operating system for those tests, and then retry those tests. If they continue to fail after a second or third retry, chances are that the issue is with the functionality you're testing, rather than the test itself. This isn't a total solution for dealing with flakes, but it shoul
Already have an account? Login
Login to the community
No account yet? Create an account
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.