Troubleshooting and Asking Good Questions: A VERY Comprehensive Guide.

  • 30 March 2021
  • 0 replies

Userlevel 1
Badge +2

The Best Way To Get An Issue Resolved

...is to troubleshoot yourself, and if that fails, ask a good question.  And it turns out those are often the same thing.

Good questions contain lots of useful information about the problem.  The more useful information a community member or support engineer has when resolving your question, the faster it’ll get resolved… And gathering that information basically is troubleshooting.

Doing troubleshooting lets you ask good questions.  Good questions lead to faster resolutions.

(See this article about why asking good questions leads to faster resolutions)

Do you want your issue resolve quickly?  Of course you do!  So let’s talk about how to make that happen.


Firstly, Search The Forum

Your problem may have occurred to other users in the past, in which case your solution might already exist!


Secondly, Gather Information (eg, Troubleshooting)

Good questions usually have:

  • A description of what is happening (including what you intend to happen)
  • Details about when it’s happening
  • The conditions it occurs under
  • What’s been ruled out (and how)
  • Anything suspicious you’ve found

This might sound overwhelming, but they’re usually quite easy for you to answer (and might be impossible for someone else to, which is why getting the answers upfront is so important!)


What is Happening?

Demonstrating the problem helps explain to others who are unfamiliar with your system

Depending on your question or issue, you might want to send:

  • Screenshots
  • A link to a Sauce Labs test
  • Test Runner or Selenium logs

Be precise!  If it’s a long test, link to a specific command or give a video timestamp.  If it’s a visual problem, highlight the impacted area.

Help people understand why the behaviour is problematic by describing what correct behaviour is.  If this was previously working, include the same information as you sent for the failure (screenshots, logs, etc)


When is it Happening?

Knowing when and how often problems occur helps isolate changes that may be causing them

It’s not hard to forget (or never be told!) about a change that might impact your system.  Knowing when it occurs helps identify patterns and causes that might be responsible.

It’s helpful to describe:

  • If the problem was previously successful, when the problem started occurring
  • How frequent the problem is.  Is it every test run?  Only on Thursday?  If it’s only happened once, what happens if you try to replicate it?


Under what conditions does it occur?

Knowing what conditions have to happen for the bug to occur helps narrow down the likely culprits

A problem that only happens to a specific tester suggests a problem with their specific environment.  A problem only Firefox?  Probably a Firefox bug!

  • Who’s running the test?  Does it only occur on CI, or does it happen to a specific colleague?  Can you run it from the command line but not your IDE?
  • What platform is impacted?  Does it occur on any version of iOS?  All versions of Chrome above 73?  Only on Window 10, not Windows 8?  In Firefox AND Safari?
  • What version/s of software are you running?  What plugins?
  • What else might have changed on your side?  Software versions, network arrangement, code changes?


Rule Things Out.

Trying solutions, and telling us what you’ve tried means we won’t suggest the same thing.  It also helps narrow down the culprit

This one is pretty easy but can involve some searching and some knowledge.  Check what might have caused the problem and, if you can, rule it out to save others from suggesting it.

  • If the problem’s been resolved previously, try the same solution again (eg, restarting a server)
  • Try changing something about the environment; If you know it happens on CI, try it from a developer’s machine.  See if libraries need updating, roll back any changes made (such as increased test concurrency), check alternative browser targets
  • Check any test runner, browser, or tool logs to see if there are exceptions, crashes, or errors… See if a search of this forum (Or Stack Overflow, or Google!) suggests anything relevant.
  • Check if there were outages when the problem occurred; Maybe a vendor was down, or something was re-starting?
  • Anything that doesn’t solve the problem, include it!


Anything suspicious you’ve found

Even if you don’t know how it’s related, things that look wrong can help others figure out problems

Your subconscious is pretty smart.  If you think something is a weird co-incidence, you don’t see how it might cause a problem but you’ve got a hunch, let us know!

  • Did another team increase their concurrency?
  • Did a vendor have an outage the day before the problem?
  • Have you just started using a new tool?
  • Was there anything in the logs you thing might be relevant?


Third, Ask Us!

The Community is the fastest, most accurate source for assistance

Specifically, ask the Community.

If you can ask a question in the Community, you should.  Why?  Speed and Experience.  The Community can often get you more in-depth answers, faster, with almost no downside:

  1. Sauce Support engineers are experts on Sauce Labs.  Our Community contains experts on everything else (like test runners, weird Appium behaviour, API Definition languages...)
  2. Sauce Support engineers answer questions in the Community!  If your question can be answered publicly, that helps make things better for everyone (Including letting others at your company see the results, without adding them to another email group thread!)
  3. Our Moderator team reads all new questions and escalates questions into Support tickets where appropriate


That’s it!

Congratulations on making it through all that text!  Sauce Labs Support and the Community are here to help solve your problems and help you get things delivered.  Asking good questions helps us do that!


Happy Testing!

0 replies

Be the first to reply!