Introduction

UIAutomation and White Core provides automation support for most of the actions and properties. There are a lot of situations (check the forums) when this is not sufficient or the implementation is buggy. In some situations one can use mouse and keyboard interface provided in white to get around some of these problems. But none of these guarantee complete automation. In order to get over this one can use white commands for WPF and Silverlight applications. (WinForm support should arrive soon).

How does UIA work for of WPF and Silverlight applications?

Lets looks at inner workings of Silverlight and WPF applications in the context to automation. Custom commands work on these concepts to provide completely flexible automation. Following diagram explains what happens when a command is received by the
ServerSideProviders.png
  1. When the window is displayed the control is created by application under test (AUT). This would happen before a test command is received by the AUT.
  2. When a command is received by AUT, UIA (part on the AUT process) inquires for the peer for the specific control.
  3. The control creates the peer object for the control. The peer object is provided along with control library in the .NET framework or third party controls, depending on the control.
  4. UIA issues the command on the peer using the patterns and properties. Patterns are meant for action and properties for the data of the controls. Essentially these boil down to methods on the peer. From the implementation perspective the peers implement some interfaces to indicate which patterns and properties they support. This means that every control has a different peer which implements different set of interfaces which make sense for that particular control. These interfaces are called providers. e.g. IInvokeProvider (button), IExpandCollapseProvider (Tree, ComboBox), IValueProvider (ComboBox), so on.

Using IValueProvider as magic hook.

IValueProvider has two methods which are implemented by peer who support ValuePattern. These are SetValue(string) and string GetValue(). White uses this provider to pass any random command and extract any random data. So if you using WhiteButton instead of Button (very basic sub-class of Button) then it creates its own peer which understands these strings passed back and forth.
public class WhiteButton : Button
{
    protected override AutomationPeer OnCreateAutomationPeer()
    {
        return new WhiteButtonPeer(this);
    }
}
public class WhiteButtonPeer : ButtonAutomationPeer, IValueProvider
{
    private readonly WhitePeer whitePeer;

    public WhiteButtonPeer(Button button) : base(button)
    {
        whitePeer = WhitePeer.Create(this, button);
    }

    public override object GetPattern(PatternInterface patternInterface)
    {
        return whitePeer.GetPattern(patternInterface);
    }

    public virtual void SetValue(string command)
    {
        whitePeer.SetValue(command);
    }

    public virtual string Value
    {
        get { return whitePeer.Value; }
    }

    public virtual bool IsReadOnly
    {
        get { return whitePeer.IsReadOnly; }
    }
}
Things to note:
  • WhiteButton returns WhiteButtonPeer instead of ButtonAutomationPeer which Button object would have returned.
  • WhiteButtonPeer extends from ButtonAutomationPeer, hence using ButtonAutomationPeer for all existing functionality.
  • WhiteButtonPeer using WhitePeer for implementation of the custom command processing, shared by all other controls as well. (would see detail of that later)
  • WhiteButtonPeer also implements IValueProvider indicating that WhiteButton supports this additional pattern not supported by Button.
WhitePeerUML.png

Custom command to third party control
In case you are using third party controls to which you want to pass custom commands then you can follow the similar approach. Lets call this control FooControl.
  • Extend the control and create your own custom control like WhiteButton may be called CustomFooControl.
  • Find the automation peer of the FooControl. This you can do by going through documentation on the control, name guessing or decompiling the code and looking at the OnCreateAutomationPeer method. Create a peer class extending this peer.
  • Implement the peer as WhiteButtonPeer has been implemented (line by line).
  • If the FooControl's peer already implements IValueProvider interface (responds to ValuePattern) then you can use WhitePeer.CreateForValueProvider() instead of WhitePeer.Create().
//example from WhiteTextBoxPeer
whitePeer = WhitePeer.CreateForValueProvider(this, textBox, () => textBox.Text, value => textBox.Text = value);

Custom Command

It provides one the ability to perform any action on the UI and retrieve any data from it. This is achieved by writing these commands and executing them within the context of the application. Following diagram illustrates the steps via which this is achieved.
CustomCommandSteps.png
Lets look at the steps.
  1. The test creates a custom command proxy object using the custom command factory. The custom command corresponds to a particular UIItem.
  2. The test invokes the method/property on the custom command proxy object. This operation is intercepted by the custom command interceptor.
  3. The command interceptor converts the command into an xml (string) and calls the SetValue pattern method on the automation element behind the UIItem. The command xml contains the name of command implementation class, method name and parameters. It removes "I" from the interface name and looks for the class in same namespace and assembly for the command.
  4. The peer running in the process of AUT, deconstructs the xml to understand the command. Since the command implementation is written by tester and AUT has no knowledge of it, it responds back indicating it cannot execute the command. (essentially asking for a assembly in which the command is implemented).
  5. The test reads the assembly in which the command implementation is defined and sends it across as byte[], again using the SetValue on automation element, value pattern.
  6. The peer loads the assembly from the byte[] sent and returns the acknowledgement of successful load of assembly. The command interceptor (note, not the test) sends the command again. The protocol of handling errors, sending assembly is handled transparently by the command interceptor.
  7. The peer executes the instruction by creating an instance of command implementation written by tester, executes the same method intended and returns the result back to the test in form of xml.
(The steps 4,5 and 6 are needed only once per assembly of commands.)

Serialization mechanism
White custom commands use DataContractSerializer to convert between objects and xml. This means that objects that you exchange between command and the peer should be serializable. If you get serialization exception then you check try following code on your object to see whether it serializes fine or not.
var serializer = new BricksDataContractSerializer();
string s = serializer.ToString(new MyObject("bar"));
var myObject = serializer.ToObject<MyObject>(s, Enumerable.Empty<Type>());

// Because of a bug in white 0.20 you may have to add an empty constructor.
[DataContract] public class MyObject { [DataMember] private string foo; public MyObject(string foo) { this.foo = foo; } public string Foo { get { return foo; } } }

Example

Lets look at via an example. Currently there is no way to get the border thickness of a button. Lets see how we can do this using custom commands. (the source is included in the white source code download. Look for CustomButtonCommandTest).

Custom Command Interface
public interface IButtonCommands
{
    Thickness BorderThickness { get; }
}
Custom Command Implementation
public class ButtonCommands : IButtonCommands
{
    private readonly Button button;

    public ButtonCommands(Button button)
    {
        this.button = button;
    }

    public Thickness BorderThickness
    {
        get { return new Thickness(button.BorderThickness); }
    }
}

[DataContract]
public class Thickness
{
	[DataMember]
	private readonly double bottom;
	[DataMember]
	private readonly double top;

	public Thickness(System.Windows.Thickness thickness)
	{
		bottom = thickness.Bottom;
		top = thickness.Top;
	}

	public double Bottom
	{
		get { return bottom; }
	}

	public double Top
	{
		get { return top; }
	}
}
Test
CustomCommandSerializer.AddKnownTypes(typeof(Thickness));
var button = window.Get<Button>("button");
var wpfWhiteButton = new CustomCommandFactory().Create<IButtonCommands>(button);
Thickness thickness = wpfWhiteButton.BorderThickness;
How does it work?
Before we look at it, you need to make sure that you are using the White custom controls on AUT side. In other works in your application code you need to refer to White.CustomControls/White.CustomControls.Silverlight assembly and use these controls instead on standard controls.

var button = window.Get<Button>("button");
We are interested in getting the thickness of the button, so first we find the button.

var wpfWhiteButton = new CustomCommandFactory().Create<IButtonCommands>(button);
Create an instance of IButtonCommands using the CustomCommandFactory. The factory creates proxy object for the interface. Please note that it doesn't instantiate the actual ButtonCommands class.

Thickness thickness = wpfWhiteButton.BorderThickness;
Invocation of BorderThickness on the proxy object does a lot of things behind the scene, as explained in previous section.

CustomCommandSerializer.AddKnownTypes(typeof(Thickness));
Since custom commands use DataContractSerializer the known type issues needs to be resolved. Custom commands serialization by default support arrays of primitives.

Security

Custom commands approach in a way opens a security hole as the user can potentially program and execute anything they desire in the context of application. Please follow the recommendation below to resolve it.
  • Do not package White.CustomControls.Peers or White.CustomControls.Peers.Silverlight dlls along with production deployment. These dlls implement the mechanism for understanding the command and executing them. This essentially means that if anyone trying to use automation against production deployment it would crash the application. This doesn't mean that one cannot run automation against deployed version. You would need to deploy these additional dlls in test environment separately.
  • Since anyone can download these dls (White.CustomControls.Peers and White.CustomControls.Peers.Silverlight) from codeplex, it still poses a risk. In order to avoid this you need to download white source code and compile White.CustomControls.Peers/White.CustomControls.Peers.Silverlight dlls by providing your own strong name key file, instead of one that comes with white. White by default doesn't sign these assemblies as it would required to be signed by anyone who uses custom commands.

Miscellaneous

What happens to the automation elements/control which already implement ValuePatter/IValueProvider (e.g. ControlType.Edit/TextBox)?
White custom controls peers are recognize the fact that the real SetValue is called by looking at the string passed in. In this case it is not an xml command, so it assumes that it call for real SetValue and responds accordingly.

I see that custom controls for some controls are missing?
Its mainly when the controls are sealed. If this is not true please report a a bug.

How do I enable custom commands for third party controls or my controls?
Please look at the WhiteButton and WhiteButtonPeer example. You need to follow similar steps and reuse the WhitePeer class to delegate everything to.

Last edited Sep 2, 2010 at 8:25 AM by viveksingh, version 32

Comments

No comments yet.