use custom UI item to automate a third party control

Jul 17, 2008 at 2:32 AM
Edited Jul 17, 2008 at 2:39 AM
Hi

I am new to White. It works well with the standard windows control. I am try to use custom UI item to automate a third party control, but don't how to implement in code. If you can provide me sample code to use this control, it will be very helpful for a new white user. I use AutoIt windows info tool (like winspy) get the following info about the control I want to automate because I don't have uispy.exe in my system yet. It is similar to a textbox. I have another control which is like a button. I just want to know which method or property I can use for a third pary textbox or button, is it similar to button.click() or textbox.text? Thanks.

Basic Window info
Title: Application
Class: ClaWin16777216Class1

Basic Control Info
Class:ClaEdit
Instance:1
Advanced (Class) [CLASS:ClaEdit; INSTANCE:1]
ID: 16390
Coordinator
Jul 17, 2008 at 6:44 AM
Just checking, have you seen this?
http://www.codeplex.com/white/Wiki/View.aspx?title=Custom%20UI%20Item&referringTitle=Home

I am not AutoIt literate yet. You can also do window.LogStructure() to get the similar info.
Jul 17, 2008 at 5:43 PM
Thanks for quick reply. I have seen that custom UI code sample. In the sample, Date control consists of three text boxes for day, month and year. They are standard text boxes. But for my case, it is not standard text boxes, so I have difficulty in how to implement them in code.
Jul 18, 2008 at 2:26 PM
Hi,

I got the info about the textbox in UIspy. Since its control type is pane, I don't know how to implement it as custom ui item in code when referring to the custom ui item sample code. I would appreciate a lot if you can help.

AutomationElement
  General Accessibility
    AccessKey:    ""
    AcceleratorKey:    ""
    IsKeyboardFocusable:    "False"
    LabeledBy:    "(null)"
    HelpText:    ""

  State
    IsEnabled:    "True"
    HasKeyboardFocus:    "False"

  Identification
    ClassName:    "ClaEdit_0400000H"
    ControlType:    "ControlType.Pane"
    Culture:    "(null)"
    AutomationId:    "16390"
    LocalizedControlType:    "pane"
    Name:    ""
    ProcessId:    "218736 (Osrk)"
    RuntimeId:    "42 6359338"
    IsPassword:    "False"
    IsControlElement:    "True"
    IsContentElement:    "True"

  Visibility
    BoundingRectangle:    "(966, 221, 197, 20)"
    ClickablePoint:    "(null)"
    IsOffscreen:    "False"

ControlPatterns
Coordinator
Jul 20, 2008 at 12:27 PM
This is my guess implementation for your textbox.

    [ControlTypeMapping(CustomUIItemType.Pane)]
    public class MyTextBox : CustomUIItem
    {
        private TextBox textBox;
        public MyTextBox(AutomationElement automationElement, ActionListener actionListener) : base(automationElement, actionListener) {
          textBox = new TextBox(automationElement, actionListener);
      }

        protected MyTextBox() {}

        public virtual string Text
        {
           get {return Name;}
           set {textBox.Text = value;}
        }
    }

For finding the item.
window.Get<MyTextBox>(....put your search criteria here.....)

Let me know if it works for you?
Jul 21, 2008 at 4:26 PM
Hi,
thanks for the sample code.
everything works fine except textBox1.Text = "hi"; (the program stuck here)
My code is:
            Window window2 = application.GetWindow("Oxford Application", InitializeOption.NoCache);
            SearchCriteria searchCriteria1 = SearchCriteria.ByAutomationId("16390").AndControlType(typeof(MyTextBox));
            MyTextBox textBox1 = (MyTextBox)window2.Get(searchCriteria1);
            textBox1.Text = "hi"; (the program stuck here)

The error is:
Bricks.BricksException was unhandled
  Message="Error invoking MyTextBox.set_Text"
  Source="Bricks.RuntimeFramework"
  StackTrace:
       at Bricks.RuntimeFramework.ReflectedObject.Invoke(MethodInfo methodInfo, Object[] arguments) in D:\bricks\Bricks.RuntimeFramework\ReflectedObject.cs:line 69
       at Bricks.DynamicProxy.DynamicProxyInterceptors.Process(IInvocation invocation, InterceptContext interceptedContext) in D:\bricks\Bricks\DynamicProxy\DynamicProxyInterceptors.cs:line 20
       at Core.Interceptors.CoreInterceptor.Intercept(IInvocation invocation) in d:\white-os\Core\Interceptors\CoreInterceptor.cs:line 29
       at Castle.DynamicProxy.AbstractInvocation.Proceed() in d:\OSS\Castle\Tools\Castle.DynamicProxy2\Castle.DynamicProxy\AbstractInvocation.cs:line 165
       at MyTextBoxProxyd7a5f5c800984e0487ca85a04b4cb82f.set_Text(String value)
       at ConsoleApplication1.Program.Main(String[] args) in C:\Documents and Settings\Bill Hu\My Documents\Visual Studio 2008\Projects\whiteFirstTest\whiteFirstTest\Program.cs:line 81
       at System.AppDomain._nExecuteAssembly(Assembly assembly, String[] args)
       at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
       at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
       at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading.ThreadHelper.ThreadStart()
  InnerException: System.InvalidOperationException
       Message="Unsupported Pattern."
       Source="UIAutomationClient"
       StackTrace:
            at System.Windows.Automation.AutomationElement.GetCurrentPattern(AutomationPattern pattern)
            at Core.UIItems.UIItem.Pattern(AutomationPattern pattern) in d:\white-os\Core\UIItems\UIItem.cs:line 115
            at Core.UIItems.TextBox.set_Text(String value) in d:\white-os\Core\UIItems\TextBox.cs:line 33
            at MyTextBox.set_Text(String value) in C:\Documents and Settings\Bill Hu\My Documents\Visual Studio 2008\Projects\whiteFirstTest\whiteFirstTest\Program.cs:line 31
       InnerException:

Coordinator
Jul 21, 2008 at 8:13 PM
Sorry about that, I should have seen it.
Please change you Text (setter) to:

set
{
                Focus();
                keyboard.PressSpecialKey(KeyboardInput.SpecialKeys.HOME);
                keyboard.HoldKey(KeyboardInput.SpecialKeys.SHIFT);
                keyboard.PressSpecialKey(KeyboardInput.SpecialKeys.END);
                keyboard.LeaveKey(KeyboardInput.SpecialKeys.SHIFT);
                keyboard.PressSpecialKey(KeyboardInput.SpecialKeys.DELETE);
                actionListener.ActionPerformed(Action.WindowMessage);
                keyboard.Send(value, actionListener);
}


I am leaving this issue open, because this is duplicate code from TextBox.Text setter. There should be a better fix for it which I can provide in future release.
Sep 18, 2008 at 10:03 AM
Edited Sep 19, 2008 at 6:47 AM

This doesn't work for me. I have checked and rechecked. Here is my implementation for TimeBar:

//Specify the ControlType which corresponds to the top level of Custom UI Item.
//White needs this when finding this item inside the window.
[ControlTypeMapping(CustomUIItemType.Custom)]
public class TimeBar : CustomUIItem
{
private TimeBar mTimeBar;

public TimeBar(AutomationElement automationElement, ActionListener actionListener) :base(automationElement, actionListener)
{
 mTimeBar = new TimeBar(automationElement, actionListener);
}

protected TimeBar() { }

public virtual string NameProp
{
 get { return mTimeBar.Name;}
}

}

My Code:
TimeBar myTimeBar = mApplicationRunner.AppWindow.Get<TimeBar>("Timebar:20080917120000-20080919120000");

Control Identification from UISpy:

Identification
ClassName: "Timebar"
ControlType: "ControlType.Custom"
Culture: "(null)"
AutomationId: "Timebar:20080917120000-20080919120000"
LocalizedControlType: "custom"
Name: "Timebar"
ProcessId: "612 (i2.Ngdtp.Main)"
RuntimeId: "7 612 14515408"
IsPassword: "False"
IsControlElement: "True"
IsContentElement: "False"

The tree is a little complex (i.e. there are various levels and custom controls), but that shouldn't matter, should it (As I understand it, as far as White is concerned, the UI is a flat structure i.e. in this case it'll look for and find the control within the Window)? What might I be doing wrong here?

Coordinator
Sep 21, 2008 at 8:54 AM
In your implementation:
public TimeBar(AutomationElement automationElement, ActionListener actionListener) :base(automationElement, actionListener)
{
 mTimeBar = new TimeBar(automationElement, actionListener);
}

This is not needed.
mTimeBar = new TimeBar(automationElement, actionListener);
Nor do you need to define timeBar inside timebar.

public virtual string NameProp
{
 get { return mTimeBar.Name;}
}
You need not implement it at all because the UIItem the base class for this already has this property.

Although I dont know what is the problem you are facing? If you are getting an exception can you send me the trace.

You are right about the fact that it doesn't matter to white how complex the tree the structure is.
Sep 24, 2008 at 11:06 AM
Edited Sep 24, 2008 at 11:06 AM
Thank you for your response.

I realsied that I didn't need to do that, but just couldn't get it working and hence was adding that to see if anything is actually found. I couldn't make it work because it genuinely couldn't get to the control. It was a mistake on my part and I've now figured that out and the above works correctly for me.

Sep 29, 2008 at 3:10 PM
Unfortunately I seem to get a MissingMethodException when trying to work with a CustomUIItem. Any suggestions on what I might be doing wrong or is going wrong? I have tried this for a couple of different custom controls in our application and they result in the same issue.

The stack trace is:
--MissingMethodException
at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes)
at System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes)
at System.Activator.CreateInstance(Type type, Object[] args)
at Castle.DynamicProxy.ProxyGenerator.CreateClassProxy(Type targetType, Type[] interfaces, ProxyGenerationOptions options, Object[] constructorArgs, IInterceptor[] interceptors) in d:\OSS\Castle\Tools\Castle.DynamicProxy2\Castle.DynamicProxy\ProxyGenerator.cs:line 354
at Castle.DynamicProxy.ProxyGenerator.CreateClassProxy(Type targetType, Type[] interfaces, ProxyGenerationOptions options, IInterceptor[] interceptors) in d:\OSS\Castle\Tools\Castle.DynamicProxy2\Castle.DynamicProxy\ProxyGenerator.cs:line 307
at Castle.DynamicProxy.ProxyGenerator.CreateClassProxy(Type targetType, ProxyGenerationOptions options, IInterceptor[] interceptors) in d:\OSS\Castle\Tools\Castle.DynamicProxy2\Castle.DynamicProxy\ProxyGenerator.cs:line 301
at Castle.DynamicProxy.ProxyGenerator.CreateClassProxy(Type targetType, IInterceptor[] interceptors) in d:\OSS\Castle\Tools\Castle.DynamicProxy2\Castle.DynamicProxy\ProxyGenerator.cs:line 263
at Bricks.DynamicProxy.DynamicProxyGenerator.CreateProxy(IInterceptor interceptor, Type type) in D:\bricks\Bricks\DynamicProxy\DynamicProxyGenerator.cs:line 23
at Core.UIItems.Container.ContainerItemFactory.Get(SearchCriteria searchCriteria, ActionListener uiItemActionListener) in d:\white-os\Core\UIItems\Container\ContainerItemFactory.cs:line 20
at Core.Sessions.NullWindowSession.Get(ContainerItemFactory containerItemFactory, SearchCriteria searchCriteria, ActionListener actionListener) in d:\white-os\Core\Sessions\NullWindowSession.cs:line 19
at Core.UIItems.Container.CurrentContainerItemFactory.Find(SearchCriteria searchCriteria, WindowSession windowSession) in d:\white-os\Core\UIItems\Container\CurrentContainerItemFactory.cs:line 41
at Core.UIItems.UIItemContainer.Get(SearchCriteria searchCriteria) in d:\white-os\Core\UIItems\UIItemContainer.cs:line 82
Coordinator
Oct 2, 2008 at 9:55 AM
Do you have a empty constructor in TimeBar. This is needed and access modifier can be protected for it. The error message should be clearer here.
Oct 6, 2008 at 12:47 PM
That was the problem, now works fine for me. Many Thanks.
Jul 9, 2009 at 7:35 AM

I need desperate help !!!

Actually there is control we have made named it InfoTextBox that is derived from TextBox and it has been used in our WPF application, when I run the recorder it doesnot show that control.

Secondly how can I get that control and make assertion on it. 

Jan 7, 2010 at 4:47 AM

Hello Friend,

I am having ActiveX SSTree control, which is being identified as 'Pane' control. I am unable to work if the control is identified as 'Pane' using UiAutomation.

Could you please help me in resolving this issue using custom item type?