Latest Entries »

In the last couple of years I’ve gone through such amazing changes as a person, and along the way I’ve discovered who I am, what I’m capable of doing and what I want to do next. I need a new challenge and a new city. What could be better than the technology centre of the world?

So, I’ve decided to move to San Francisco.

This is all well and good, but Campaign Monitor is a Sydney-based company. So in order to move, I’ll have to sadly say goodbye to the awesome people at Campaign Monitor and look for a new employment in the Bay Area. That’s right folks, I’m looking for a job.

What kind of job, you ask? Well, I’d like to be doing stuff that’s new and interesting, with people who are really passionate about what they are making. I want to be hands-on, doing testing, but my version of “doing testing” means that I use all skills at my disposal to test the product in the best way possible. So it doesn’t just mean “writing automation” or “doing exploratory testing”. It means, figure out the best way to test it, and then do that. I’m more interested in seeing what people need me to do, rather than describing a silo that I want to fit into.

I have a great big heap of skills, wrapped up in a likeable tester exterior. Here’s a sample of what I can offer:

Test automation experience. I’ve written test frameworks and test suites from scratch using Java, C#, WatiN, Rational Functional Tester (Robot), WebAii and Selenium. For several clients, I’ve analyzed, maintained and refactored existing automated test suites to make them more efficient and effective. I’ve integrated test suites with FitNesse, TestLink, Team Foundation Server and Team City (continuous integration). In addition to creating traditional test suites, I have also developed tools that have helped testers and developers to test effectively. Being a developer-skilled tester is about more than just automating test cases; it’s about using technology to make the whole testing process more efficient.

“Trish has a knack of building some incredibly simple, yet deviously smart testing tools, with ‘clicky thing‘ being a classic example.”
- Bruce McLeod, Automation Test Manager

Software development experience. I’ve worked as both a PHP Developer and a .NET Developer in the past, and I’m currently learning Ruby on Rails by writing a Rails application from the ground up. I can’t emphasise enough how much these experiences have helped me in testing web applications. There is so much more happening in a web application beyond the GUI. Understanding how web applications work gives you a greater understanding of how web applications *don’t* work.

“Trish is logical, methodical, and has an excellent memory for detail. She has demonstrated she can both think like an end-user, and at the same time have sound technical knowledge; vital, as all but the most trivial testing methods are tool- and technology-intensive. Her experience has clearly given her insight into the likeliest places to find bugs. Lastly, she also takes the time to build good relations with developers and designers. We will miss her!”
- Mark Langsworth, Senior Software Developer

Solid testing experience. I’ve invested a lot of time in improving my manual testing skills so that I can find important information as quickly as possible. I believe in a context-driven test approach – tailor the solution to the problem. As part of my training, I have been coached by Anne-Marie Charrett and I have taken James Bach’s Rapid Software Testing course. Good testing, at its core, is about exploring, learning, analyzing and reporting.

“Trish has always been a pleasure to work with at Campaign Monitor. Her attention to detail always shines through, every step of the way through a project. She’s an invaluable resource to a user-experience designer; especially in the early stages of wireframing, where it’s easy to forget about the many edge cases that are sure to arise.

Bugs — no matter how small — never seem to get past Trish. I often moan and groan having to fix them all, but it always makes for better experience for the end user; and that’s what matters.”
- Jesse Dodds, Lead UI/UX Designer

Leadership. Despite my continued insistence on being as hands-on in the testing process as possible, I still gravitate towards leadership roles. I like to provide as much guidance as is necessary to enable teams to be productive, while still encouraging them to take a proactive approach to their tasks. I have achieved amazing results when I have involved the whole test team in shaping a test approach. Other testers’ brains are the most valuable testing tools that money can buy.

“A far cry from the command and control mentality that plagues much of the software management landscape, Trish enables and encourages testers to do the most valuable thing possible: Ask questions and think creatively.

My time working at Campaign Monitor was one of the most challenging and rewarding experiences of my testing career, thanks largely to Trish’s formidable ability as both a tester and a manager. Trish removed roadblocks and bureaucracy, motivated and inspired us with creative solutions to problems but knew when to get out the way so we could do our best work as a team.

I would work with Trish again in a heart beat.”
- James Martin, Software Tester

Want to know more?
Listen to my podcast at Testcast.net.
Read some of my most popular blog posts and publications.
Follow me on Twitter.
Check out Campaign Monitor, the web application I’ve been testing for the last three years.
Or just contact me via email at trishkhoo@gmail.com.

Oh, and if you’d like to meet me in person, I’ll be at SF Agile Conference from 2 – 8 June this year.

See you on the other side of the world. :)

Recently I found myself in a familiar testing situation – a developer had made a change to a feature that required testing by visual comparison. Hours of tedious, repetitive manual testing awaited me.

Campaign Monitor is a web app that allows users to import an HTML page and send it as an email to a list of subscribers. It runs the email through a conversion process so that the HTML page will work properly with our system. As you can imagine, this means there is a massive amount of variation possible for this kind of input, and the correctness of the output depends as much on the layout and visual elements of the page as the page content itself. In other words, it’s not enough to just verify the same elements exist on the page – the way they look is also important.

In this case, the change was an improvement to the conversion of foreign character sets, so the developer had sent me a large list of HTML pages with foreign characters to verify that the emails still looked correct.

So I needed to import campaigns in both the test environment and the production environment and visually confirm that they still looked the same. We had encountered a similar situation in the past. The results was hours and hours of tedious manual testing. But this is a task that can only be done manually, right?

To hell with that!

It was a pretty straightforward process to set up an automated script that would import a campaign and show the preview screen. Then I just needed a way to capture the previews in each environment as a screenshot for easy comparison. Luckily WatiN has a method called CaptureWebPageToFile() (part of the IE object) which captures the entire web page in the browser window and saves it as an image. So even if part of the web page is outside the visible part of the browser window, the entire page will still be captured in the image.

So that allowed me to take a list of web based campaigns as input and automatically generate screenshots of each campaign in two different environments for easy visual comparison, saving hours of tedious clicking. As a test of the tool, I imported two copies of the same email campaign and changed one of them to display different text in the title of the first paragraph.

But wait! I can do better.

Visually comparing two screenshots was still pretty annoying and time-consuming. So I added a step to automatically compare the two images pixel by pixel, and tell me if they’re identical. If they’re not identical, then I want a new screenshot showing me how they’re different.

TestApi has a tool for doing exactly what I wanted. So I made a method that would take two images, compare them, and generate a diff image if they weren’t identical. The diff image isn’t fantastic, but it gives a rough guide to where the problem areas are. In the example diff image below, any parts of the image that are the same are coloured black, and any parts that are different are coloured blue. So for this test, any characters that were not properly encoded would show up quite obviously. This way we know exactly which campaigns are problematic and all we have to do is follow those up, instead of sifting through potentially hundreds of campaigns manually.

After I wrote the method for comparing the screenshots, I just needed to speed up the process a bit. Logging into each environment and importing the campaign took a lot of time, so it was easier just to import once and reimport for each new campaign. But processing one environment at a time meant that I had to wait for it to get through the whole list before I could see any results. So I compromised and processed the campaigns in batches of ten.

Unfortunately, I can’t share the code with you because it’s so tightly woven into our product-specific automation library that it wouldn’t be usable as an independent tool. But with the concepts and links above, it should be easy enough for you to adapt it for your own project.

Now, is your finger itching to hit the comment button and tell me there’s an even easier way for me to solve the problem I had? You’d be absolutely right. I was about ten minutes into writing this script when I realised all I had to do was import this page and I had a handy list of incorrectly-converted characters to send to the developer. But the visual comparison tool was still handy to make sure nothing else had gone wrong as a result of the change.

I ended up finding that some of the foreign character sets were incorrectly encoded, and I already had screenshots saved by the tool to send to the developer. Not only did I find a bug, but now that it’s written we can easily adapt this script to regression test changes to this feature and other similar features in the application in future. So that means less repetitive testing and more exploratory testing!

One useful robot

Last year, I was feeling a bit overwhelmed by having to track multiple streams of information all day long. Luckily, I was able to merge two of those streams by combining Team City, our continuous integration tool, with Campfire, our chat room tool.

Using the handy APIs for each of these tools, fellow tester Adrian and I put together our very own Campfire bot. We did already have a Campfire bot implemented in Ruby, but as our team is more familiar with C#, we started from scratch and made a .NET application. We did keep the original name and thus, ‘freshbot’ was reborn.

When we develop new features at Campaign Monitor, there are a few suites of tests that we like to run against each build. These can take up to a total of 10 minutes to complete.

Previously, we’d repeatedly check Team City to see if the tests had completed, then notify the rest of the team in Campfire when we were starting, or stopping test suites. Now we have freshbot monitoring certain builds for us automatically and logging the results in Campfire. Freshbot also lets us know when other environments are building, so we end up with a nice time log of what builds were happening at what time in our Campfire chat room.

Here are a few other useful things we’ve programmed freshbot to do for us:
- Remind us to order our lunch before the cutoff time, and suggest a random sandwich (a randwich) for us.
- Provide us with wisdom.
- Advise us on how to crush our enemies.
- Cheer us up when we’re sad.
- Entertain us with random quotes and images from the movie Zoolander.
- Dazzle us with pointless facts.
- Retrieve random images for us from Bing search, based on keywords.

From time to time, freshbot’s nemesis stalebot has been known to launch a verbal attack on freshbot, mostly consisting of unpleasant sandwich suggestions. We don’t actually know for sure who stalebot’s secret identity is, but we have our suspicions.

After a few days of running and refining freshbot, I noticed that it was crashing overnight. So I added some exception handling and logging to see what the problem was. I found out that Team City undergoes maintenance at the same time every night, so any calls made to the server during this period would return errors. Having discovered this, I decided to check on some of the “random” failures that we’d been encountering in our nightly automated test runs and, as I’d suspected, some of them were coinciding with this maintenance time. By rescheduling the test runs, we were able to avoid these failures.

As freshbot would say, “Our greatest glory is not in never falling, but in rising every time we fall.” Thanks, freshbot.

>> Want to work in a place like this? Campaign Monitor is hiring!

I remember three things that made me think testing was cool before I was a tester.

The first one was when I was a high school student and I was reading an article online that a tester for LucasArts games wrote. It was a funny “day in the life” piece, and described how he got to play the game before it was finished and found bugs that would make Guybrush Threepwood’s head fly across the screen. He described the glee he felt in waiting for the optimum time to tell the developer about the one flashing white pixel in the bottom of every frame, and watching that developer burst into tears. He sounded like a man who loved his job.

The second one was when I was a university student and I was studying human-computer interaction as a subject. The textbook had a piece describing how Microsoft conducted usability testing. They had usability labs, and brought in real users and observed them while they used the software. It sounded like an interesting way to gain insight into how people used software, and a way to help make software easier to use.

The third one was when I was fresh out of university and working for Microsoft as a developer. My boss Gary had been working there for about thirteen years and he loved to tell us stories about Microsoft. One story he told us was about a group of testers in Microsoft who would walk around in packs wearing black trench coats, striking fear and awe into the hearts of developers as they passed by. They’d walk up to a workstation, click a few buttons, crash the system and coolly walk on their way, while developers scrambled to fix the error. They sounded like badasses.

Conversely, I remember the first time I met a software developer. I was in my first year of university and had told my uncle that I was studying IT and wanted to be a programmer. My uncle was a manager in a bank. He was shocked. “What? You want to be a programmer? Aiyoh. Come with me, I’ll show you a programmer.” So he took me to the bank and showed me his huge, beautiful office. It had a separate lounge area and an ensuite bathroom. Then he took me down a corridor to a black unmarked door, punched in a combination into a keypad and there, in the middle of a windowless room, sitting in a pod of cubicles, sat four men who looked like they’d lost the will to live. “Look at these guys” said my uncle, gesturing to a despondent-looking programmer. “Look at their faces. Is this the kind of job you want?”

When someone meets you for the first time, having never met a software tester before, what will they think of software testing when they walk away?

A couple of weeks ago, the Sydney testers meet up group got together to talk about how to introduce testing to an organisation. Drinks were had (thanks SoftEd!), stories were shared, debates were had and friendships were formed. Here are a few of the points that I took away from the discussion.

Introducing testing is not just about introducing testers
Introducing testing to a new team means changing the way the team develops software. It’s not enough to simply add testers and expect testing to happen. Integrating testing into the software development lifecycle is something that involves the way every team member works.

Authority is important
If you are to be a catalyst for change, you need enough authority to be able to instigate the change. Often this kind of authority can come with being an outside consultant. It was noted that sometimes the fact that you are being paid a huge sum of money in order to introduce new practices can lead to you having a stronger voice in the organisation. Being a contractor can have similar advantages. It was discussed that, as a contractor, you’re less affected by the politics of the organisation and this enables you to introduce change more easily.

Developers can test too
There was some debate about the difference between testers and developers, and how to tell if someone is “destined” to be a tester versus a developer. Testers can program, and developers can test, and it’s beneficial for both testers and developers to have these skills. The difference is the individual’s personal preference, and level of skill in these areas.

Self-marketing
The value in testing isn’t always readily apparent, especially to high-level managers. Demonstrating the value in testing activities is an important part of introducing testing to an organisation. Metrics such as bug counts can be used to help show value, but they can also be misused. For example, if testers help developers to improve their own testing skills then the amount of reported bugs may decrease, and this would look bad for a team that uses bug reports as a metric to show test team effectiveness. For this reason it’s important to demonstrate that testing is more about preventing bugs than finding bugs.

To automate or not to automate
Introducing automated GUI tests is a project risk, and is an activity that is often underestimated in terms of time cost and skill requirement. There is always a maintenance cost that needs to be considered when introducing automated tests. Automation metrics can also be easily misused. On the other hand, if considered well, automated testing can be a valuable asset to many software development teams.

The tester mindset
Testers do need a negative mindset to perform some kinds of testing activities. But good testers need to be able to switch out of this mindset at will or they’ll just become unpleasant to work with, and this is not good self-marketing for the testing team.

A few follow-up topics were suggested, including “how to recruit testers” and “how blurred should the line be between testers and developers?”

You can find information on upcoming Sydney Tester Meetups on the meetup homepage.

Recently, a link came my way about Aspiritech, a testing company that only hires testers with Asperger’s Syndrome or high-functioning Autism. I’d heard of companies like this before, and was curious about the advantages and limitations of this testing approach, and about what made software testing so suited to this demographic. So I expressed some interest on Twitter, and was subsequently introduced to Michael Drejer. Michael has Asperger’s Syndrome, and is a software tester working at BOAS Specialister in Denmark. So he was able to answer a few of my questions over Twitter.

Michael:
Oh yeah, @Aspiritech they’re good guys. I work for the company that started the whole asperger-software tester thing. Or well, we are a separate company now from the original Specialisterne. We are called BOAS Specialister. Specialisterne is Danish for The Specialists (and BOAS is a contraction of words Live on your own Asperger).

Trish:
Cool. How does the test approach of these orgs differ from other orgs?

Michael:
They don’t really. Except we’re better and more thorough. We follow test cases/plans when we have them, or write them then execute. The main difference is that you have to have an asperger diagnosis to be hired. And the work environment is more asperger friendly.

Trish:
Is all of the testing done with test cases, or are there also exploratory sessions done without test cases?

Michael:
We do bug hunts as well. Some things are easy enough to do without test cases. But it’s good to have a structured process to work from, that’s one of the important things.

Trish:
Interesting. Do you work directly with developers, or does your team work externally from the main dev team?

Michael:
We work only as external testers. We usually report the bugs in a bug report software, depending on what the client uses. Over the years, we have done a lot of testing for Lego on their website, shop and flash games for example.

Trish:
Cool. What makes the structure of the testing so important?

Michael:
It’s the asperger thing. We really don’t like when things aren’t structured. It’s what makes it difficult for us to find work, because a lot of jobs and places of work don’t have that kind of structure we need. That’s why QA testing is so good for us. And because we have an extraordinary ability to notice details, we find more bugs than most other testers do.

Ah I see. Do you think your preference for structure helps you spot software defects more easily?

Michael:
Structure prevents me from losing my mind in frustration, allowing me to focus on the testing. The work environment is also an important factor. I don’t like a lot of light, and I don’t like having my back towards the door. My co-workers have other issues, mostly sensory-related.

Trish:
Interesting. I know a few test approaches that aren’t as structured though. I guess that wouldn’t work as well for you guys? Like for example, working without specification documents, and/or not using test cases.

Michael:
When testing websites we rarely use test cases (I basically make them in my head and remember for testing again). BTW, a couple of us are ISEB certified. I think it depends on the scope of the software. Big software, like the one we’re working on now, needs test cases.

Trish:
Ah sorry, when you said structured, I thought you meant test cases. What did you mean by structured?

Michael:
I do mean test cases. But when it’s smaller jobs, it’s okay, we don’t need test cases for that. If it’s a complex software program, then we need to know what to do. It is just how we work most optimal, and it can stress us out very easily if we don’t have test cases, or at least some sort of recipe for how to do the testing. We just don’t always have the privilege of having test cases.

If you’d like to learn more, Michael wrote this guest blog post about working as a software tester with Asperger’s Syndrome. It was recently published in the book “The Thinking Person’s Guide to Autism”.

There’s a plague upon the testing craft, and it’s called “fake testing”. You could also call it “ineffective testing”. Here’s James Bach’s guide to faking a test project.

I believe fake testing is generally practiced by testers who just don’t know any better. We can’t really blame them. Most testers “fall in” to the profession. There’s a widely held belief that testing is an unskilled profession that can be done by anyone. So when it comes to adding testers to a project, often “anyone” is hired.

We could try to educate those “anyone” testers, but there’s not much point. They are the symptom, and not the problem. The problem is a lack of understanding of testing by the people who are hiring these testers in the first place. The problem is that testing is easy to fake, and its effectiveness is difficult to measure.

This is why we end up with testing certification. Hiring managers don’t know enough about testing to recognise good testers, so they’re just turning to some kind of authority to tell them what a good tester looks like.

If we want to attract more skilled professionals to software testing, then we need to work on educating the developers, the project managers, the product managers, the recruiters and the other people involved in hiring testers in the first place. If they recognise good testing, then they’ll know how to recognise good testers. Understanding how to effectively test software is something that the whole software development team should know about to some extent anyway.

So how do we do it? Well, for starters, maybe we should stop siloing ourselves as a community. We should be networking with the rest of the IT community, doing talks at developer and project management conferences, submitting articles to a broader audience. I’m sure some of us already are. It would be great to introduce more testing-focused coursework in universities. I know it was an area that was definitely lacking in my computer science coursework.

It’s been so good to see non-testers coming along to the Sydney Tester Meetups. I think we need more online resources, so that it’s easy to direct people to good sources of information about software testing. Something like “good testing starts here”. Dot com.

What do you think? How can we help educate the greater IT community about software testing?

I’ve talked to a lot of unhappy testers – smart, talented testers who always feel like they are working at odds with their teams in order to advocate quality. In this kind of situation, it’s easy to believe that quality is dead. In my last post, I speculated that perhaps it just isn’t cost-effective anymore for companies to have a primary quality focus. In which case, as an advocate for quality, how could a tester ever have real job satisfaction?

Software isn’t just a “see who can make the most money” game. There are people out there who believe in making good products.

The quality of the product is limited by the quality goal of the company. Testers, developers, designers – we all help the company achieve that goal. If the goal doesn’t align with our own personal ideas of quality, then we will not have much job satisfaction.

Whether you like software to look beautiful, or produce huge profits, or be driven by beautiful code, or make your customers love you, you want to build something that makes you proud.

Jerry Weinberg famously said that quality is value to someone at some time. What is quality to you, at this time? Are you working at a place that has the same quality definition as you? If not, then you have two choices – adjust your personal definition of quality to match the company’s, or find a company that has the same values as you do.

Every successful company is making quality software, by some person’s definition of quality. If we want to have real satisfaction in what we help create, we need to find or create a company with quality values that align with our own, and then work out how we can best serve to help achieve that goal.

“Quality is dead,” declared Goranka Bjedov. And I thought to myself, I knew it.

For some time now I’ve been concerned about this. I’d always seen specialised testing as a way to turn a mediocre-quality product into a high-quality product. But the evidence before my own eyes is that the market is full of buggy software. The worst part is, people accept buggy software as the norm and learn to live with it.

We say that nobody can ever be “done” testing, because there is always more than can be tested. There comes a point in any project where the cost of continuing to test outweighs the benefit in releasing earlier, but with unknown bugs.

“The truth is, bad software makes more money” – Goranka Bjedov, STANZ 2011.

I recently watched a project going through a fairly difficult stabilization phase. Too much change towards the end of the development cycle and not enough testing at the start meant that most of the bugs were being found at the end, so the team was looking at a few more weeks of catch up time to find and fix all of the bugs. But, for reasons that I still don’t completely understand, all of the bugs were postponed and it was released anyway. The next two weeks were a mad scramble to fix production bugs, followed by several weeks of less frantic fixing. I was ready to label the project a colossal failure. But the product was actually very well-received by most customers. By the end of the first week, customers were singing praises for the new feature and the project was hailed as a success.

It left me wondering what the point was of having testers on that project at all. I was told that, without testers, it would have been a whole lot worse. Is that rewarding though? Working to make something “not as bad as it could be”?

After STANZ, I stayed with my friend Richard for a few days. Richard is a medical doctor, training in radiation oncology. I asked him if it was depressing being in a job treating cancer, which is a condition that’s difficult to treat and not completely curable.

“Oh not at all” he said. “There are some practical things you can do, to improve quality of life”.
“But,” I pressed, “when people think cancer, they usually think that’s it for them. It seems like you’re just fighting an uphill battle.”
“Everyone dies eventually, of something,” he explained. “But I know that there’s things I can do that means they’re not going to die today. And the quality of life they have in the meantime can be made a lot better.”
I nodded. “I guess that’s true.”
“Once you accept mortality, medicine’s not that hard. It’s actually quite rewarding.”

Perhaps we need a mentality shift, and a different way of seeing our role. If we see ourselves as the cure for bad software, then we’re going to be repeatedly disappointed.

I emailed my friend James Martin for his perspective, and he offered this one:

How about working to make something ‘better’. What if all any of us did, regardless of job title, was try as hard as we could to make the best stuff possible, accepting the fact that even the best of us has only a fragile grasp of what it takes to make something that another person will be delighted by.

Personally, it helps me to think that my whole job description is ‘to make things better’. That way I can solve problems using whatever tools feel right for the task at hand and I don’t feel constrained by someone else’s opinion of who I should be and what I should do.

As usual, the people setting the ‘standards’ for our ‘profession’ do so without knowing our context. But we can set our own standards for ourselves and play by our own rules. I like to think that you and I did that last year. I still think of what we did at Campaign Monitor as my best work.

If quality is dead, what are we doing here?
If we are adding value, what is it and how do we measure it?

When WebAii encounters a modal dialog of any kind, its default response is to sit there staring at it for eternity, causing your entire test suite to hang. It’s not exactly ideal. So, here is some hackery I put together to prevent this from happening.

Stuff that might be important to know:
- I’m using WebAii version 2.0 beta, which might not be the latest version but is pretty close to it. I did make some effort to try to work out what the latest version is, but it was far too hard to work it out from the website. So if the latest version handles this better, we’re all in luck I suppose.
- We’re converting all our tests to WatiN. Try it today!

1. Start an unexpected dialog monitor before each click action, based on the assumption that dialogs generally appear as the result of a click.
2. Set up a second click action for when you’re *expecting* a dialog, so that your unexpected dialog monitor doesn’t handle the expected stuff.

public void ClickElement(Element element, bool isDialogExpected)
{
    Assert.IsNotNull(element, "Error: element to be clicked was null");
 
    if (!isDialogExpected)
    {
        // Handler for unexpected dialogs that may appear
        var anyDialog = new GenericDialog(ActiveBrowser, "", true) {HandlerDelegate = HandleUnexpectedDialog};
        Manager.DialogMonitor.AddDialog(anyDialog);
        Manager.DialogMonitor.Start();
 
        Actions.Click(element);
 
        Manager.DialogMonitor.RemoveDialog(anyDialog);
        Manager.DialogMonitor.Stop();
 
        if (_unexpectedDialogAppeared)
        {
            Assert.Fail("An unexpected dialog appeared after clicking: " + element + ". The dialog caption was: " + _unexpectedDialogCaption);
        }
    }
    else
    {
        Actions.Click(element);
    }
}

3. Set up a handler for unexpected dialogs, which attempts to read the dialog text, close the dialog and gracefully fail the test with a message that includes the text of the dialog. It’s great to take a screenshot at this point too, if you can.

Important note: Make sure you’re identifying the dialog as a dialog attached to the browser process. Otherwise tests will fail when they encounter *any system dialog at all*. I can’t tell you how disheartening it is to find that all your tests have failed because your VM’s antivirus software needs updating.

public void HandleUnexpectedDialog(IDialog dialog)
{
    if (dialog.Window.OwnerProcess.ProcessName == "iexplore")
    {
        _unexpectedDialogAppeared = true;
        TestHelper.CaptureScreenForTestLog(ToString());
        try
        {
            _unexpectedDialogCaption = dialog.Window.Caption;
        }
        finally
        {
            var anyDialog = new GenericDialog(ActiveBrowser, "", true) { HandlerDelegate = KillBrowserBecauseOfTooManyDialogs };
            Manager.DialogMonitor.AddDialog(anyDialog);
            Manager.DialogMonitor.Start();
 
            dialog.Window.Close();
 
            Manager.DialogMonitor.RemoveDialog(anyDialog);
            Manager.DialogMonitor.Stop();
        }
    }
}

4. If the dialog fails to close properly, kill all browser processes. Note that it’s passing the failure reason through, so that the test doesn’t just fail because of lack of running browsers.

private static void KillBrowserBecauseOfTooManyDialogs(IDialog dialog)
{
    KillBrowserProcesses("More than one dialog appeared in sequence, so I thought it best to just kill all IE processes and start afresh.");
}
 
public static void KillBrowserProcesses(String reason)
{
    CaptureScreenForTestLog(reason);
    foreach (Process p in Process.GetProcessesByName("iexplore"))
    {
        p.Kill();
    }
 
    Assert.Fail(reason);
}

5. In dialog handling for expected dialogs, also monitor for unexpected dialogs. For example, during a file upload, if the file is not found, an unexpected dialog might appear. Kill it on sight.

/// <summary>
/// Opens the File Upload dialog and attempts to upload a specified file. 
/// (note, in firefox, the windows are called "File Upload")
/// </summary>
/// <param name="reference">The reference to the HtmlInputFile control used for opening the File Upload dialog</param>
/// <param name="fileName">The combined filepath + filename of the file to upload</param>
public void UploadFile(String reference, String fileName)
{
    // A flag to indicate whether a file not found error has occurred
    _unexpectedDialogAppeared = false;
 
    _count = 0;  // number of times a dialog was opened
 
    _fileName = fileName;   // save filename into global variable
    String alertTitle = (ActiveBrowser.BrowserType == BrowserType.InternetExplorer) ? "Choose File to Upload" : "File Upload";
 
    // Handlers for unexpected dialogs that may appear if the file cannot be found
    var logonPrompt = new GenericDialog(ActiveBrowser, "Connect to ", "Cancel", true) { HandlerDelegate = HandleDialog };
    var alert = new GenericDialog(ActiveBrowser, alertTitle, "", false) { HandlerDelegate = HandleDialog };
 
    Manager.DialogMonitor.AddDialog(logonPrompt);
    Manager.DialogMonitor.AddDialog(alert);
    Manager.DialogMonitor.Start();
 
    // Click the field so that the file upload dialog appears
    if (ActiveBrowser.BrowserType != BrowserType.FireFox)
    {
        ClickWhileExpectingDialog(reference);
        alert.WaitUntilHandled(10000);
    }
    else
    {
        ClickWhileExpectingDialog(reference);
        //inputField.MouseClick();
        alert.WaitUntilHandled();
    }
 
    Manager.DialogMonitor.RemoveDialog(logonPrompt);
    Manager.DialogMonitor.RemoveDialog(alert);
    Manager.DialogMonitor.Stop();
 
    // Fail the test if the file was not found
    if (_unexpectedDialogAppeared)
    {
        Assert.Fail("An unexpected dialog appeared while attempting to select file: {0} for upload, the caption was: {1}", fileName, _unexpectedDialogCaption);
    }
}

6. Test it. How else will you know if it works? This test references a simple html file with a button that launches a file upload dialog.

using MbUnit.Framework;a
using System.Windows.Forms;
 
namespace Tests.Common.Test
{
    public class HandleDialogTests
    {
        WebBrowser _webBrowser;
        private GetTestData _testData;
 
        /// <summary>
        /// Initialization for the class. Creates test data required for test cases in this class.
        /// </summary>
        [SetUp]
        public void MyTestInitialize()
        {
            _testData = new GetTestData();
            _webBrowser = new WebBrowser(_testData);
        }
 
        /// <summary>
        /// Clean up for the class.
        /// </summary>
        [TearDown]
        public void MyTestCleanUp()
        {
            TestHelper.CaptureScreenOnFailure(ToString());
            _webBrowser.TestCleanUp();
        }
 
        [Test]
        public void UnexpectedBrowserDialog()
        {
            // Unexpected browser dialogs should cause tests to fail
            _webBrowser.LaunchBrowserAndNavigateTo(StringFormattingHelper.GetCurrentWorkingDirectoryAsUri() + "/resources/fileUpload.html");
            _webBrowser.Click("id=fileupload");
        }
 
        [Test]
        public void UnexpectedWin32Dialog()
        {
            // Unexpected win32 dialogs should not cause tests to fail
            new System.Threading.Thread(() => MessageBox.Show("background thread modal box")).Start();
            _webBrowser.LaunchBrowserAndNavigateTo(StringFormattingHelper.GetCurrentWorkingDirectoryAsUri() + "/resources/fileUpload.html");
            _webBrowser.Click("id=link");
        }
 
        [Test]
        public void FileDialogTimeout()
        {
            _webBrowser.LaunchBrowserAndNavigateTo(StringFormattingHelper.GetCurrentWorkingDirectoryAsUri() + "/resources/fileUpload.html");
            _webBrowser.UploadFile("id=fileupload", StringFormattingHelper.GetCurrentWorkingDirectoryAsUri() + "/resources/file.txt");
        }
 
        [Test]
        public void UnexpectedDialogWhileUploadingFile()
        {
            _webBrowser.LaunchBrowserAndNavigateTo(StringFormattingHelper.GetCurrentWorkingDirectoryAsUri() + "/resources/fileUpload.html");
            _webBrowser.UploadFile("id=fileupload", "badFileName");
        }
    }
}