Problem with initial testing of the library

Feb 21, 2008 at 1:16 PM
Kicking the tires on this as we've been having long debates on UI testing. NUnitForms doesn't really cut it and the API here is closer to what we're thinking about.

Maybe I'm doing something wrong here but whenever an exception is thrown (either by expecting it, like calling a form that doesn't exist) or a test fails, the form under test is left up and running.

Here's a code snippet of a test:

        [Test]
        [ExpectedException(typeof(Core.UIItems.UIActionException))]
        public void ApplicationLaunch_NoArgs_Form2NotDisplayed()
        {
            string path = Path.Combine(Directory.GetCurrentDirectory(), "WhiteLibSpike.WinForm.exe");
            Application application = Application.Launch(path);
            application.GetWindow("Form2", InitializeOption.NoCache);
            application.Kill();
        }

This tries to get Form2 (which doesn't exist) but when the UIActionException is thrown, it leaves the window on the desktop. Not cool for doing any kind of automated testing on our CI server (I know there's mention on the site here about using VMs for testing, but that's a bigger issue politically that can't be solved right now). You get the same behavior when a test fails for whatever reason.

Any thoughts around this? It seems to be the major stumbling block for me right now.



Feb 21, 2008 at 1:42 PM
I was able to correct the problem of left over forms by using a wrapper for the tests. This is similar to what Ben Hall posted, although I just made the wrapper disposable so it would always kill the application off:

class WinFormTestWrapper : IDisposable
    {
        private readonly Application _host = null;
 
        public WinFormTestWrapper(string path)
        {
            _host = Application.Launch(path);
        }
 
        public void Dispose()
        {
            if(_host != null)
                _host.Kill();
        }
 
        public Window GetWindow(string title)
        {
            return _host.GetWindow(title, InitializeOption.NoCache);
        }
    }
 
    [TestFixture]
    public class Form1Test
    {
        private readonly string _path = Path.Combine(Directory.GetCurrentDirectory(), "WhiteLibSpike.WinForm.exe");
 
        [Test]
        public void ShouldDisplayMainForm()
        {
            using(WinFormTestWrapper wrapper = new WinFormTestWrapper(_path))
            {
                Window win = wrapper.GetWindow("Form1");
                Assert.IsNotNull(win);
                Assert.IsTrue(win.DisplayState == DisplayState.Restored);
            }
        }
 
        [Test]
        public void ShouldDisplayCorrectTitleForMainForm()
        {
            using (WinFormTestWrapper wrapper = new WinFormTestWrapper(_path))
            {
                Window win = wrapper.GetWindow("Form1");
                Assert.AreEqual("Form1", win.Title);
            }
        }
 
        [Test]
        [ExpectedException(typeof(Core.UIItems.UIActionException))]
        public void ShouldThrowExceptionIfInvalidFormCalled()
        {
            using (WinFormTestWrapper wrapper = new WinFormTestWrapper(_path))
            {
                wrapper.GetWindow("Form99");
            }
        }
    }

Thoughts?
Feb 21, 2008 at 1:45 PM
Test
ExpectedException(typeof(Core.UIItems.UIActionException))
public void ApplicationLaunchNoArgsForm2NotDisplayed()
{
Application application;
try
{
string path = Path.Combine(Directory.GetCurrentDirectory(), "WhiteLibSpike.WinForm.exe");
application = Application.Launch(path);
application.GetWindow("Form2", InitializeOption.NoCache);
}
finally
{
if (application != null)
{
application.Kill();
}
}
}
Feb 21, 2008 at 1:54 PM


gnash wrote:
Test
ExpectedException(typeof(Core.UIItems.UIActionException))
public void ApplicationLaunchNoArgsForm2NotDisplayed()
{
Application application;
try
{
string path = Path.Combine(Directory.GetCurrentDirectory(), "WhiteLibSpike.WinForm.exe");
application = Application.Launch(path);
application.GetWindow("Form2", InitializeOption.NoCache);
}
finally
{
if (application != null)
{
application.Kill();
}
}
}



Yes, but putting a try/catch/finally around every test you write is going to create a maintainenance problem, not to mention the additional code you have to write for every test. After you create 100 UI tests, this is going to be a problem.

I've done some more with the IDisposable wrapper and like it. It always works and always cleans up after itself. I've also added a generic GetControl<ControlType> method which let's me pull anything I need from the form for testing and comparision.
Feb 21, 2008 at 4:10 PM
Using the using statement with IDisposable still results in extra code in each test. If you want to eliminate the duplication then perhaps you want to make it a member variable of the test and kill it in the teardown. That would involve the least duplication. Teardowns are called even if exceptions are thrown.

[TestFixture]
public class Form1Test
{
   private Application _application;
 
   [TearDown]
   public void TearDown()
   {
      if (_application != null)
      {
         _application.Kill();
      }
   }
}
Feb 21, 2008 at 4:11 PM
Sorry about the duplicates...
Feb 21, 2008 at 10:29 PM


gnash wrote:
Using the using statement with IDisposable still results in extra code in each test. If you want to eliminate the duplication then perhaps you want to make it a member variable of the test and kill it in the teardown. That would involve the least duplication. Teardowns are called even if exceptions are thrown.


I guess it's a question of where you want that "extra" code to be. If it's in the wrapper class, I don't have to worry about it. The newb developer only needs to know he uses the wrapper in a using statement and it's disposed of properly, even with exceptions being thrown (by White or otherwise).

I personally see a break point where a developer will "forget" to create the SetUp and TearDown methods not to mention the fact they have to do this exact same code in every test fixture. You could put it in a base test fixture but again it's not 100% clear that this type of setup/teardown is being done. Personally (just my opinion) I like to have my unit tests very explicit about the environment and state of the world when they run and not assume things are in any kind of condition. That tends to lead to false positives in testing.
Aug 1, 2008 at 8:27 AM
I too would like to see the IDisposable interface/pattern applied to the Application class. This at least gives you the option of implementing the Using statement to kill the app on a failure. It also then follows good practice of cleaning up after yourself.

Lee