{"id":393,"date":"2015-03-12T10:48:34","date_gmt":"2015-03-12T10:48:34","guid":{"rendered":"http:\/\/blog.devexperts.com\/?p=393"},"modified":"2022-05-18T10:57:50","modified_gmt":"2022-05-18T10:57:50","slug":"gui-automation-difficulties-when-marketing-collides-with-real-life","status":"publish","type":"post","link":"https:\/\/devexperts.com\/blog\/gui-automation-difficulties-when-marketing-collides-with-real-life\/","title":{"rendered":"Pitfalls of Graphic User Interface Test Automation and How to Overcome Them"},"content":{"rendered":"\r\n
\"mvc_pic.001\"<\/a><\/figure><\/div>\r\n\r\n\r\n\r\n

Usually, it is a good practice to automate any kind of routine. In testing, automation is vital. It helps decrease costs, makes testing tasks faster, and improves the efficiency of your staff. It is a \u201cno-brainer\u201d to think about automating different testing tasks, especially when reading marketing materials or listening to a sales presentation of a \u201cleading-edge-premier\u201d robot software. Those reports are usually based on success stories from the world of the most complicated user interfaces – CAD software, graphics software, or maybe some examples of extremely complex but good web design. Moreover, usually, these reports are true. But you should think carefully before starting a GUI test automation project. In many areas of software development, GUI automation is not so easy and obvious, and your entire automation project can quickly turn into a disaster, despite the visual attractiveness of the tools.<\/p>\r\n\r\n\r\n\r\n

Beware <\/h2>\r\n\r\n\r\n\r\n

First, let’s understand where GUI automation can fail:<\/p>\r\n\r\n\r\n\r\n

  1. Your project is a distributed system, which consists not only of a GUI-side, but also of a server-side, and maybe, some middle tier<\/g><\/li>
  2. Your GUI-side is built on highly customized (skinned, smart, etc<\/g>) controls<\/li>
  3. You not only want to act upon your software,<\/g> but also verify the result of your action<\/li>
  4. You can’t “save” the result of your session in the GUI[1]<\/a><\/li><\/ol>\r\n\r\n\r\n\r\n

    If any of these points are applicable to your GUI automation project, continue reading.<\/p>\r\n\r\n\r\n\r\n

    Custom Controls <\/h2>\r\n\r\n\r\n\r\n

    In a world where look and feel in software are key for successful marketing, customization of standard GUI controls set is a must. Nothing should stop you from making your controls more attractive to the user, and skinning is one of the easiest ways to bring a fresh look. Some of the features that make your controls smarter (such as autofill, or tree-view structures in initially flat-structured controls), may deliver more clarity and usability to the user, but may also become a nightmare for your automation team. Modern robots are usually technology independent or at least compatible with all common programming languages (they will work equally well with Microsoft .net interface, or java Swing\/SWT GUI) – but usually they can only easily recognize standard controls. Even if you just added some background skinning to the standard ListView control (i.e. inherited from a standard class), you still have a risk that this customization will lead to the manual mapping of such a control. In this case, you’ll be forced to explain everything about this control to the robot. This is particularly important when you need to find some element of the control to click on, or invoke the context menu, etc. While this is not a show-stopping problem, it can have a serious impact on the automation effort.<\/p>\r\n\r\n\r\n\r\n

    Another problem, specific to Swing interfaces, is the dynamic controls naming. Very often, programmers do not make an effort to name each and every control in their interface, letting java machine do this work for them in runtime. In this case, GUI automation becomes impossible until you agree on a strict naming convention, and force the developers to implement it.<\/p>\r\n\r\n\r\n\r\n

    Validation <\/h2>\r\n\r\n\r\n\r\n

    To automate an action or even long sequences of actions in the interface is usually a simple task, which doesn\u2019t even require special programming skills. You just use the robot\u2019s built-in recorder and that’s it. Easy \u2013 and it works, too. But in testing, actions are frequently performed to validate the output of the software, and this is where things get complicated. If you are strictly using standard controls with which the robot is familiar, and your result is a highly predictable number or string not containing any variables, the task can still be performed with virtually no programming (robots have a complete set of built-in comparison tools for that). <\/p>\r\n\r\n\r\n\r\n

    If your software has some kind of “save” or export function, you can use the full range of external comparators (including your own utilities) to validate large data sets or complicated file formats. But what if your controls are not readable by the robot? Then the resulting number may be seen on a screen, but may not be easily reached by the robot, because it might have been rendered on a custom-label interface element chosen for its popularity among your programmers and your software fans. <\/p>\r\n\r\n\r\n\r\n

    Most robots have a built-in bitmap comparator, which can be useful in certain cases – such as making screenshots and comparing them to a reference screenshot – but this is not the best way to compare numbers. Imagine if an output value is calculated from several different values displayed in runtime simultaneously on the screen, and cannot be predicted in advance. Since an automation script should be highly reusable, and any bitmap comparison depends on factors such as screen resolution, interface layout, font sizes, etc, this is a strong argument not to use this method.<\/p>\r\n\r\n\r\n\r\n

    Some robots provide special libraries, plugins, etc, that can be injected into your software during the build (compilation) process. While this gives you access to a variety of controls, in 99% of cases you will not want to provide such a build to the end-user, because it’s not the best decision from either security or performance standpoints. It forces you to test one build and deliver another one, which is also not the best idea. Moreover, this kind of injection, will not guarantee you 100% success in reaching the information inside your controls.<\/p>\r\n\r\n\r\n\r\n

    Thick Client <\/h2>\r\n\r\n\r\n\r\n

    All of the problems described above can be annoying, but you can still deal with them one way or another. The next level of complexity is brought by a client-server architecture with a thick client concept. Usually, a thick client needs a connection to the server-side, but performs many actions on its own – such as asynchronously requesting and receiving data from the server. <\/p>\r\n\r\n\r\n\r\n

    The asynchronous nature of those operations can often be addressed by queuing all the actions performed by the testing script – inserting delays, wait functions, and other result-catching activities. But while this process establishes an artificial synchronization between client and server, it also slows down test execution significantly. Now imagine if, in addition to this, the application under test has some triggers that cause overlapping, modal windows, messages, warning panels, tabs, etc., after the data is pushed from the server. <\/p>\r\n\r\n\r\n\r\n

    All this causes uncertainty to your test script, and the script creator is forced to predict and manage all possible actions. Moreover, some of those windows and messages can inform the end-user that some vital conditions of the application execution have changed, for example, that connection to the server has been lost. In this case, you\u2019ll need to add some level of heuristics to determine all the correct conditions, or just accept that all subsequent actions will fail. <\/p>\r\n\r\n\r\n\r\n

    While trying to manage the many possibilities of behavior of such an application, it may become logical to take full control over the server-side during script execution. At this point, you may suddenly face the fact that your robot doesn\u2019t have enough power to do that, or that the tools you already have for this control are not compatible with the robot, and that it will be a significant effort to fix that. And since this effort was not initially planned it will surely extend your project timeframes and costs.<\/p>\r\n\r\n\r\n\r\n

    At this stage your test script will become a complex and long-lasting routine, requiring optimizations, splitting, and parallelism to gain durability and save execution time. Accomplishing these optimizations is closely related to the testing environment.<\/p>\r\n\r\n\r\n\r\n

    Environment <\/h2>\r\n\r\n\r\n\r\n

    Execution facilities for testing are a very important aspect of any testing process. Automation puts some additional requirements on this environment. Virtualization technologies and cloud computing have changed substantially in testing lab deployment, and today it\u2019s possible to set up a flexible testing environment in a corporate or even public cloud within minutes. But this usually implies remote access to the virtual hosts. <\/p>\r\n\r\n\r\n\r\n

    The question is how will the robot survive when the remote connection is down. Of course, it will depend on many factors, including OS, remote connection tool, its settings, etc., but as practice shows, robots are helpless when the whole desktop and mouse context are suddenly gone. The most common mistake is to underestimate these environmental issues at the very beginning of the automation project. <\/p>\r\n\r\n\r\n\r\n

    This leads to very \u201ccreative\u201d solutions on later stages, when the simplicity of the script and parallel execution and other processes aimed at the durability of the script, are totally offset by the overcomplicated and unstable configuration of the test lab.<\/p>\r\n\r\n\r\n\r\n

    In conclusion,<\/strong> a decision to start an automation project, or buy a new super-clicking robot, can\u2019t be made just by looking at the robot\u2019s list of features. Software, which is supposed to be tested, needs proper analysis and skilled resources assigned to its project. The following can be used as a control checklist: <\/p>\r\n\r\n\r\n\r\n