Bugs/Issues Fixed

None found

So far so good.....

Features added

New XML test syntax available

There is now a new test syntaxt, QAXML, which will be the focus for all new QAT development.
Allows much more concise parsing of test files, far less ambiguity etc. However, is still under development, and I recommend using it in conjunction with QASH tests. The harness allows QASH and QAXML to exist in the same workspace, allowing you to migrate only new tests as neccesary.

Known Issues/Bugs

JDK1.3 on linux has a critical bug in Runtime.exec(...)

It is currently not possible to retrieve the exit value of a launched process unless you are in the same Thread as whoever made a call to exec(), which is not the case for our agent. So we cannot accurately determine if a process is running, finished etc - should be fixed in the next release by Sun.

Bugs/Issues Fixed

None found

So far so good.....

Features added

New XML test syntax available

There is now a new test syntaxt, QAXML, which will be the focus for all new QAT development.
Allows much more concise parsing of test files, far less ambiguity etc. However, is still under development, and I recommend using it in conjunction with QASH tests. The harness allows QASH and QAXML to exist in the same workspace, allowing you to migrate only new tests as neccesary.

Bugs/Issues Fixed

None found

So far so good.....

Features added

Replace GNUmakefiles with Ant

The cranky old GNUmakefiles have been replaced by Any build files, allowing more flexibility and speed in compiling the workspace. We no provide a "binary" delivery, containing only the required jars and documentation, and a source distribution containing all the project sources as well.

Bugs/Issues Fixed

BlockView java.lang.ArrayIndexOutOfBoundsException

The folowing exception randomly occurs :
Exception occurred during event dispatching:
java.lang.ArrayIndexOutOfBoundsException
        at javax.swing.text.html.BlockView.layoutMinorAxis(BlockView.java:220)
A bug has been logged against the JDK for this, however it seems to cause no real problems.

Features added

Documentation updated

The documentation has been updated to cover all of the features of the QAT system.

Bugs/Issues Fixed

OutOfMemory while generating the HTML Report

Note that some verbose tests can easily generate output files 10 megs big or more...

Features added

None.

Bugs/Issues Fixed

Saving to a read-only file gives dialog on error

If you try to edit/save a read-only file, and dialog will now pop-up indicating the error. Previously a message was written to stdout.

Features added

Changed Jar file name

The jar file has changed from qa_tester.jar to qat.jar, and each release will be suffixed with the version number.
eg. This release is in qat2.6.6.jar, and qat.jar links to this file.

GUI Project Properties added

You can now specify GUI properties whch will be configurable as GUI components in the Edit Project window.
Currently supported are
qat.project.guiproperty.checkbox..XXX
qat.project.guiproperty.textfield.XXX
qat.project.guiproperty.combobox.XXX  ; qat.project.guiproperty.combobox.XXX.values

The following  are examples of these properties:

CheckBox:
qat.project.guiproperty.checkbox.DEBUG_COMPONENT=false

TextField:
qat.project.guiproperty.textfield.JDMK_RELEASE=4.3

ComboBox:
qat.project.guiproperty.combobox.withDEBUG=YES
qat.project.guiproperty.combobox.withDEBUG.values=YES,NO

Changed all images to .png

The GUI now uses only .png images instead of the mixture of jpg and gif, in solidarity with the opensource boycott of LZW compression patent restrictions imposed by UNISYS. This however limits useage of JDK1.3 and later, since PNG files are not supported in JDK1.2.x.

Improved agent discovery

The agents previously all did a multicast reply all at once, so network flooding may have resulted in some responses being lost. Now each response is randomly sent within a short time-frame, to limit the number of responses which might clash.

Added parser command trace

The command currently being processed by the QASHParser is now displayed in real time in the QAT status bar. Currently the information is fairly basic, but may be improved to include more detail at a later stage.

Bugs/Issues Fixed

Viewing mixed mode output generates NullPointerException

If your some of your trace output is in Html format, and some in plain text, viewing one after the other used to generate a Swing NullPointer exception due to a Swing bug.  Finally a workaround has been found.

Html Report traces being lost

The html report generated files names which were often longer than 256 characters, and lightweight operating systems like Solaris can not handle this, so some traces were lost. There is now a new algorithm for allocating file names, which makes use of the (hopefully unique) hashcode for each test path.

Interrupting runs was a bit buggy

Sometimes interrupting a started test run did not cleanly kill the child processes - hopefully this has now been resolved. It was caused by combination of threading issues, as well as the recently introduced AUTOCLEAN command.

Less threads required per execute command

Each Exec request previously launched a new Thread. Now the ExecProcess object re-uses the current Thread context since we know a new thread is started to handle each agent request. Need to look out for synchro issues arising as a result of this change.

Features added

New layout

The layout has been re-designed to minimise clicks to get to relevant information. Less dependant on the right-click menu now.

Bugs/Issues Fixed

Html Report

Various changes in the Html report generation to simply and improve the imformation presented.

Command-line interface keyword selecting bug

There was a bug in the keyword selection when using the command-line interface and using only th e"-keyword" option..

Less threads required per execute command

Each Exec request previously launched a new Thread. Now the ExecProcess object re-uses the current Thread context since we know a new thread is started to handle each agent request. Need to look out for synchro issues arising as a result of this change.

Features added

Added new QASH command : DAEMONSTART

There is now a new command which is used to launch daemon processes. No trace may be recovered from such a process, nor may it be killed or stopped, since the functionality is undefined for daemon processes. (see the QASH Syntax for details).

Modified QASH command : CMDSTOP

There is a new form of the command which allows you to retrieve the status of the process if it has already exited:
CMDSTOP(process_id, status);
will return the exit value of the process, or a negative value if the process had to be killed.
(see the QASH Syntax for details).

Added new test state :PENDING

There is now a new test state, pending, which occurs if a test failed, but contains non-null bug information.Further revised naming of internal QAT variables

The variables KEYWORDS -> qat.test.keywords, TEST_DESCRIPTION->qat.test.description
New variables introduced :

• qat.test.author
• qat.test.buginfo

Added new test state :PENDING

There is now a new test state, pending, which occurs if a test failed, but contains non-null bug information.

Bugs/Issues Fixed

Html Report

Various changes in the Html report generation to simply and improve the imformation presented.

Command-line interface keyword selecting bug

There was a bug in the keyword selection when using the command-line interface and using only th e"-keyword" option..

Less threads required per execute command

Each Exec request previously launched a new Thread. Now the ExecProcess object re-uses the current Thread context since we know a new thread is started to handle each agent request. Need to look out for synchro issues arising as a result of this change.

Features added

Added new QASH command : DAEMONSTART

There is now a new command which is used to launch daemon processes. No trace may be recovered from such a process, nor may it be killed or stopped, since the functionality is undefined for daemon processes. (see the QASH Syntax for details).

Added new test state :PENDING

There is now a new test state, pending, which occurs if a test failed, but contains non-null bug information.Further revised naming of internal QAT variables

The variables KEYWORDS -> qat.test.keywords, TEST_DESCRIPTION->qat.test.description
New variables introduced :

• qat.test.author
• qat.test.buginfo

Added new test state :PENDING

There is now a new test state, pending, which occurs if a test failed, but contains non-null bug information.

Bugs/Issues Fixed

Html Report

There was a problem in the link generation of the Html report, caused by not using the URLEncoder routines for special characters. All should now be working fine. Many cosmetic changes have been made to improve the report appearance.

Features added

Completely revised naming of QAT variables

The QAT internal and system variables have undergone major renaming to allow new requested functionality to be implemented. The following naming changes will need to be made to remove parsing errors:
TESTPATH -> qat.project.path
AGENT_COUNT - > qat.agent.count
AGENT1_NAME -> agent1.name
AGENT1_PORT -> agent1.port
AGENT1_ARCH - > agent1.os.architecture
AGENT1_OSNAME -> agent1.os.name

Two new variables have been introduced :

qat.qash.filepath - The path to the file currently being parsed.
qat.qash.path     - The path to the root directory of the file currently being parsed.

Your old project settings will automatically be converted to use the new naming schema, however you will need to manually change all your test scripts. As a result of this, versioning has been introduced for the <project>.prj as well as the <project>.ser files.
The QAT remains backwardly compatible, but deprecation warnings will be printed if you continue to use the old syntax. Support for the old syntax will be terminated in version 2.6.5.
 

Added GETTRACEPATHS(...) command

A new QASH command has been added GETTRACEPATHS(agent_id, envtrace, stdouttrace, stderrtrace) which allows you to retrieve the exact paths of the output being generated by a process started on the agent, and relative to the agent. Please note, due to this change, you need to use the same version of agent (2.6.x or higher) to make use of this new command.

Commandline interface improved

The commandline interface has been revised, with some renaming changes (see the QASH Syntax for details).

Parser bug

There was a small problem if you forgot to add an "END" to an "IF" statement, which caused an OutOfMemory exception, now gives a useful error message instead.

New project option

Allow the user to create a new, blank project template with no predefined settings.

Bugs/Issues Fixed

QASHParser improvements

Better error reporting for some syntax errors. Uses less static variables to accomadate new multiple instance mode QAT.

Features added

New QASHParser commands :ENVTRACECONTAINS, STDOUTCONTAINS, STDERRCONTAINS

You can now perform a string check on traces recieved from the agents to verify a string exists in one of the output files. See the QASH Syntax for details.

Bugs/Issues Fixed

QASHParser improvements

Better error reporting for some syntax errors. Uses less static variables to accomadate new multiple instance mode QAT.

Features added

Allow searching of command traces

You can now search through command traces via using the provided search facility of the TraceOutputViewer.

Reworked all the examples

The examples have been completely revised to provide a basis test suite which will be run for each QAT delivery. It will eventually exercise all functionality of the QAT, and could serve as a basis for a workspace structure for new users.

Expand tree/Collapse tree behaviour

There are now two seperate buttons for expanding and collapsing tree nodes in the QAT gui. The tree will be expanded/collapsed one level at a time.

New autoclean functionality

Previously, when calling the REPORTSTATUS command, the parser immediately halts parsing of the QASH files, so it was possible to get into a situation where you didn't correctly clean started commands, and agents. By calling AUTOCLEAN_ON, the parser will automatically clean up any leftover stuff immediately after the call to REPORTSTATUS. This functionality is disabled by default, or can be manually disabled by calling AUTOCLEAN_OFF.

New commandline mode for QAT

It is now possible to launch QAT in commandline mode, allowing keyword test selection, as well as generation of html test reports. For details, please read the relevant section in the Quick Start Guide.

Multiple Qat instances now supported

It is now possible to launch more than one instance of Qat, each session acting independantly of the others, and each session will be saved independantly. Running several Qat gui's in the same Java Virtual Machine allows great savings in memory space when you need to run several projects simultaneously. Single instance useage should be unaffected.

Bugs/Issues Fixed

Agent no longer uses home directory for temporary files

Before Java 2, there was no facility for creating temporary files. The File.createTempFile() function is now used if the agent is launched with a Java 2 JVM, so it no longer uses ~/.qat/agent directory to store temporary trace files. Launching the agent with an earlier version of the JVM will revert to the old behaviour.

Html report generator bug

The Html report generator was not updated to ignore Save/Load project options of the Html interface, and it caused you to load and save a project each time a report was generated, causing interesting side-effects.

Native agent - many bugs fixed

Native agent now uses Standard template library string classes, and far fewer pointers. Code has been cleaned up significantly. Runs large tests suites now with no difficulty, so approaching stability levels of java agent.

QASHParser improvements

Finally cleaned up the parser to correctly cache properties files. Works much faster now as well, and should provide basis for more speed-enhancements in the future.

Features added

Menu icons

More menu icons have been added to increase useability. Added toolbar access to reparse tests.

Added look and feel settings

You can now select between supported look and feels for you particular platform, instead of defaulting to the Metal L&F as was done previously. Use the project properties editor to select. Your selection is saved on a per-project basis.

Agent can run again in JVM 1.1.8

The java agent should be able to run with Java 1.1.8, although it has not been heavily tested.

New test detail option in QAT GUI

New functionality now allows you to view author information and a brief description of each test.

Test sorting changes

Tests are now better sorted alphabetically than previously.

Bugs/Issues Fixed

Native Agent core dump resolved

Most of the native agent segmentation faults have now been resolved. There is still an issue with a core dump everytime any exception is thrown, but this is an unavoidable bug in libC from Sun.

QICTestFinder/QASHTestFinder did not ignore "deleted_files" directories of SCCS workspaces

These directories will now be ignored, in addition to paths containing "SCCS"

Agent Info editor modified

You can now multi-select agents to delete,refresh, kill etc.

QICTestFinder and QICParser renamed

They are now named QASHTestFinder and QASHParser for consistency. Every project using the old names will be automatically changed to use the new names, so the change will be transparent.

Features added

Discovery added to native agent

The native agent can now handle discovery requests from the QAT harness. This will only be available if it is compiled in multi-threaded mode, since the discovery needs to run in a seperate thread. So it probably won't be useable on the Chorus system for a while.

Bugs/Issues Fixed

Native Agent ZIPCLEAN problem

The native agent would, with certain zip files, not fully be able to clean all files which where unzipped, which resulted in not being able to properly clean up the unzipped files, and the agent tmp directory.

Native agent SETAGENT problem

Previously, calling SETAGENT with a non-existing tmp directory paramter, it crashed, and the directories needed to be created by hand. These directories will all be created automatically now if they don't previously exist.

Increased parsing speed

QICParser classes now does much less string searching due to an improved algorthmn in all loop types (FOR, WHILE, REPEAT etc).

Features added

New FUNCTION definition possible for QASH scripts

You can now define functions in your qash scripts to improve code re-use and readability. See the QASH Syntax. guide for details on useage.

Discovery service

You can now discover agents on your network by using the "Discover agents" button in the "Edit agents" dialog. It will automatically fill in any discovered agents, or update existing agents if already defined. Currently not implemented for native agents - will be in next release if I learn how to code UDP sockets in C++.

Bugs/Issues Fixed

Parser output always updated

Previously, when running tests, as each test completed, it was updated/reloaded into the ParserOutput window. Now it will only be updated if it is currently selected. Hopefully this may be the source of the out of memory problems experienced when running a long time with large trace outputs being genereated by the agents.

Features added

GUI layout re-arranged

It now reflects more usefull information, including test run times, detailed progress bar etc.

Bugs/Issues Fixed

Features added

Added Chorus plugin interface for Chorus

There is now a Chorus plugin included in the jar file which allows remote connection to a Chorus machine. It can be enabled by adding the plugin : qat.plugins.chorus.ChorusPlugin to the project properties.

Fixed many C++ agent bugs

The agent was pretty unstable, and lots of memory leaks and threading problems have now been resolved. Please report any re-creatable crashes so I can debug them.

Added console server functionality to the C++ agent

The C++ agent now also allows a Console server connection to be created.

Kill command included in the QAT-Agent protocol

There is a new command which allows the QAT to cause the agent to commit suicide. See the QASh Syntax file for details.

Bugs/Issues Fixed

Features added

Added plugin interface to QAT

Interfaces to allow plugin development have been put in place. This allows the QAT GUI to be fine tuned to specific project requirements, while keeping the QAT itself as a generic test tool. Plugins are stored on a per-project basis, and are configured using the Project Properties edit dialog.

Added console connection functionality to the QAT gui

The QAT nows allows you to connect to the console of any defined agent, and monitor in real time all the traces and requests it is processing. Each connected QAT can define it's own debug level, specified in the Project Properties edit dialog.

Added console server functionality to the agent

It is now possible to connect to running agents and recieve their console output, from the QAT GUI. Each session can specify it's own unique debug level, and new options have been included in the edit project window to fine-tune the console window buffer size and debug level. Porting of this functionality to the C++ version of the agent is underway, and will be very usefull for agents running on machines with no or limited consoles.
Furthermore, the agents' console output may now be duplicated to whoever wants to see it, so multiple users may all see and use the same agent console. Each connection may define it's own debug level.

CHECKAGENT more detailed now

CHECKAGENT now sets all the Java properties prefixed with the name of the agent.
So for example if on our agent we have os.name="Solaris", after a call to CHECKAGENT(AGENT1_NAME,AGENT1_PORT) we can access :zulu.os.name="Solaris"

QICTestFinder modified to work with SCCS

File checked in with SCCS will now no longer appear in the test tree as tests. Any file containing "SCCS" in the directory path will effectively be ignored for testing purposes.

Bugs/Issues Fixed

Repainting problems

Calls to paintImmediately(getVisibleRect()); have been replaced by calls to repaint(), because very buggy painting behaviour was exhibited during test runs. Probably a Swing bug - no time to debug it deeply, so just replaced calls - works much better now.

Small memory leak fixed

QAT ran out of memory after continued useage for several weeks - I was forgetting to delete the TestObject from internal list after call to CMDCLEAN(...), so
after a couple of thousand test runs, it bombs with an Out of Memory error. Thanks goodness I implemented the "Autosave" option!

Small bug in CHECKFILE

The VFile introduced a small bug in the CHECKFILE command, causing it to report false results, but it is now fixed.

Features added

Agent ported to C++

The agent was ported into POSIX compliant C++ to allow generation of agents for OS's not currently supported by Java, as well as for the constraint we have on Chorus, where only one JVM instance may be run at a time. Slight modification was made to the communication between the harness and the agent to allow the C++ agent to recieve Java objects such as Strings, ints etc coming down the line. This means that QAT and it's agents must all be running V2.5 or later. I'd suggest upgrading of all your qa_tester.jar archives lying around, else you'll wonder why your agents have stopped working. The agent source/binary is not included inthe delivery zip - if anybody wants it, just send me a mail webhiker@free.fr and I'll pass it along.

Bugs/Issues Fixed

Keyword selection

Previously it tried to load the trace files for every test selected by the keyword expression - quite a long process. Now the selection listneners are suspended while tests are selected.

Features added

Http interface extended

Can now load/save projects via the http interface as well

Statistics pane added

There is now a third tabbed pane which shows test totals for the current run/project, as well as a running VM memory display to show any possible memory problems.

Bugs/Issues Fixed

Repainting better

When repasrsing selected tests, the underlying windows was not correctly refreshed. This is now fixed. It still doesn't update when reloading all the tests off disk.

Separators inside string constants

The following code :
TOTO=" +Test "
OPTIONS="-Djava.compiler=none"
used to cause problems in the parser, since it currently didn't't check if separators are contained within string quotes or not. This is now fixed if using the QICParser. The bug remains in the QASHParser.

Features added

Added new parser qat.qicparser.QICParser

Uses the new StreamTokenizer class from Java 2, and allows much more strict syntax control, and better parse speed. This will be the basis for a new syntax allowing functions with return values.
The entire parsing process was rewritten to ensure minimal string operations so the entire process is now completely Token based, and where possible, StringBuffer classes were used to minimise memory requirements of String objects. Error reporting should be slightly more accurate than previously.
To use the new parser, simply edit the project properties and change "qat.qashparser.QASHParser" to "qat.qicparser.QICParser". Change also "qat.qashparser.QASHTestFinder" to "qat.qicparser.QICTestFinder".
Select OK, and it will reparse all the tests with the new parser. The parse speed is improved by several orders of magnitude.
Note: all special characters now need to be escaped, eg "c:\tmp" needs to be "c:\\tmp" as it is for java.util.Properties files.

Bugs/Issues Fixed

None

None

Features added

Added Html adaptor for remote manipulation of the QAT

It is now possible to connect to the QAT via a browser, and perform rudimentary functions such as starting/stoppping tests, and browsing the test results.

Added "Generate Html Report" option

It is now possible to generate a browsable HTML report of the project for archiving test runs. This function works in conjuntion with the Html adaptor.

Bugs/Issues Fixed

Multi-client agents

Agents can now manage multiple agents simultaneously. Interrupting test runs will only kill agents started by the harness requesting the kill, so agents can be shared amongst multiple harnesses. It's suggested you make use of the newly added RANDOM command to ensure each SETAGENT command uses a unique offset to prevent interference and overwriting common files.
Previously, interrupting a test run would kill ALL processes running on the agent.

Project properties stripped a bit too much

To present a simple project properties window, all agent variables were deleted before presenting it to the user for editing, but in effect, all variables "AGENT*" were removed, and then replaced after the user finished editing. Instead, now only "AGENT*NAME", "AGENT*PORT", "AGENT*OS" and "AGENT*ARCH" are removed, allowing users to define their own variables starting with "AGENT" in the project properties.

Features added

Added RANDOM function to QASH script

There is now a RANDOM function available in the QASH script to generate random positive integers. See QASH Syntax.

New "Reset Status" project option

If the option is set, all selected tests will be set to status NOTRUN before the first test starts executing.

Bugs/Issues Fixed

 

Fixed elusive memory leak

There was a mysterious memory leak which occurred if you ran a single test over and over again, but not when running many tests in a test run. This has been traced, as now corrected, so we should have no more "Out of memory" errors occurring on the QAT tool.

Agent is now completely asynchronous

The agent can now handle multiple requests simultaneously from several harnesses. Each request from the harness is handled in a new thread context. It allows blocked processes to be killed a lot easier, and improves the buggy behavior of stopping a running test. This was also achieved by changing some synchronous methods to asynchronous on the QAT harness side.

Tests now don't block when being interrupted

Tests could not be nicely interrupted due to socket blocking problems - this seems to have been nicely resolved now.

KILLALL done during eval mode

Under certain circumstances, when re-parsing tests a KILLALL request was sent to the agents.

New packaging for QASHParser

Re-packaged QASHParser and QASHTestfinder to qat.qashparser package.

Implemented the "New project" menu option

You can now create a new blank project by using the system menu option.

Removed the "View trace files in test tree" option

With the improved trace viewing system, it is no longer necessary to view the trace files as nodes in the test tree, so this option has been removed. This has the added benefit of simplifying much of the code, and reducing memory overhead due to fewer needed icons.

Uses default Swing folder icons

The default swing icons are used for all tree icons except the TestSpecification nodes, which are required to indicate differences between the test states of passed, failed etc.

Test nodes not always sorted

If tests were reloaded from within a node on the tree, via the right click menu, they were not sorted. They are now sorted correctly.

QASHParser SLEEP command not interruptable

There was previously no way to interrupt a SLEEP(X) command except by waiting for it to finish, sometimes causing the Harness to block when a test was stopped. It can now be stopped immediately.

Features added

Auto save function

There is now a project option to auto save the test tree after each test result becomes known, which is useful in the event of sudden QAT death due to power failure etc.

Trace files cleared when reloading tests

The *.trace, *.stdout and *.stderr files created for each Test node are now deleted each time the tests are reparsed off the disk. The was no way previously, to clear these files.

Added Import/Export menu options

These new options allow you to save all files associated with a project to a location other than the user.home (Export), and also to load all associated files from an arbitrary location to user.home (Import).

Added more detailed agent information

The environment for each CMDSTART command is recorded with a lot more detail than before, now including os, os version, architecture etc.

Bugs/Issues Fixed

Nothing

Nothing

Features added

Keyword/Parser Output window can be resized now

The parser output and keyword tabbed pane is now a JSplitPane, and can be removed entirely, or resized as needed. The settings are saved between sessions.

Packaged properly now

The classes are all packaged within qat.agent.*, qat.common.*,qat.parser.* and qat.gui.*, with the exception of the utility wrapper classes Qat.class,Agent.class, and the default parser classes QASHParser.class and QASHTestFinder.qash.
Required complete rewrite of the Makefiles.

"Save project as" includes status bar

Previously, when saving a large project, when the tools was copying all the stdout/stderr trace files, it appeared as if the tools had hung, now a lovely status bar makes the wait a little easier.

Bugs/Issues Fixed

Printing caused ClassCastException

Was calling getAllNodes instead of getAllTestNodes to retrieve tree nodes for printing.

Stopping a test run left the node in "running" state

The node is now returned to an UNRESOLVED state if it is interrupted.

Memory leakage problems

A memory leakage problem was caused by a swing bug workaround, and made really bad by the new active viewers attached to all the test tree nodes. This problem has been fixed, and memory useage seems stable at about 60MB - still a lot, but it seems to be a Swing problem, not QAT problem.

"localhost" replaced by actual host name

Adding an agent defaulted the agent name to "localhost", but this only has meaning on the harness machine. Lots of the test scripts use this value to sent to the agent machines etc, so now the actual hostname is supplied by deefault.

CMDSTOP syntax handling differed slightly from CMDSTATUS

Forms like CMDSTATUS(Agent+x+s) were handled by the CMDSTATUS command, but rejected by the CMDSTOP command. They now exhibit similar behaviour.

Features added

Timer run

You can now schedule a test run in the future. This feature is pretty ugly right now, but is functional. It might be expanded a little more if the functionality proves to be usefull.

Test Node updated after a test run

Previously, a test was only visually updated during a test re-parse, or from the initial test parse. Now the node is updated after each test run to reflect any dynamic keyword, testname, include file and command status changes that may have occurred. The disadvantage of this approach is that a serious test failure early on might cause some problems such as undefined test name etc.

Parser output incorporated into main window

The keyword panel and parser output viewer have now been incorporated into the same panel through use of a JTabbedPane to reduce multiple window confusion.

Reload tests from specific directories only

You can now reload tests off disk for specific directories using the right-click menu.

Bugs/Issues Fixed

Trace filename generation centralised

Calculation of the trace file name is now performed exclusively by the TestSpecification object to reduce code duplication.

Features added

Live TraceViewer and ParserViewer objects

These viewers are now automatically updated each time the TestTreenode changes.

NodeMenu uses AbstractAction instead of listeners

Change was made to simplify the event mechanism, and remove code duplication.

Compare results adjusted

Work has started on making this functionality a little better.

Bugs/Issues Fixed

QASH Properties cacheing side-effect

Previously properties files which were included were cached in memory, so they were only loaded once during the lifetime of the QASHParser object. The problem is if you edit a properties file, this changed will never be reflected until you relaunch the JVM to create a new QASHParser object, since it will always use the static one in memory. Now the cache list is refreshed for every QASH file parsed, which should fix the behaviour. Slight performance loss will result from having to read a properties file each time between succesive QASH parses.

Parser output sometimes not found

The method used to calculate a filename to store the parser output has been improved. In some cases it did not work properly.

Graphics object not properly disposed

The StatusWindow object did not explictly call dispose() when forcing a paint. This is now explicitly called to ensure the resources are properly freed.

SaveAs menu option did not save serialized tree

Using the "Save As" menu option did not explicitly save a serialized version of the currently loaded project. This is now saved by default as well.

QASH parser bugs

Several parser bugs were fixed. Several more were created by the fixes, and these have now been fixed.

 

Features added

Test tree is sorted

The children of each node in the test tree are now lexicographically sorted by TEST_NAME.

Bugs/Issues Fixed

QASH syntax error

If a variable was defined with the same characters as a predefined keyword, such as SLEEP_KANGO, it was treated as a keyword followed by illegal characters. This has now been fixed.

ZIPSEND strengthened

Trying to do a ZIPSEND of a non-existent file was not properly handled, and left the agent in a bad state. It now fails a lot more gracefully, and the agent is never contacted.

Modified syntax slightly

The CMDSTART, CMDSTATUS and CMDSTOP can now take a complex ID variable for use in loops. Previously this was not possible because the syntax requires that the variable be non-defined before use. So forms such as STARTCMD(AgentID, "CommandID_"+I, ....) are now allowed.

TraceViewer class sometimes lost trace files

The TraceViewer class sometimes indicated a trace file was not found, even though it exists, due to a missing else which over-wrote the text while the background thread loaded the file.

 

Features added

Compare results functionality

You can now compare the results of two different test runs, and it will list any matching tests by result. This first implementation is quite rudimentary, but it will be fluffed out a bit as the RFE's come in.

 

Bugs/Issues Fixed

Solaris Production release JVM bug in JEditorPane

The TraceViewer object now has a workaround for a bug in the JEditorPane class which prevents autosensing of the content type when loading a non-html page.

QAT System variable name change

The variable NB_OF_MACHINES has been changed to AGENT_COUNT to be more consistent and intuitive.

Trace Viewer clearing problem

If an attempt to view a non-existent trace was made, the trace viewer displayed the previous traces instead. The relevant windows are now cleared properly regardless.

SLEEP command handled incorrectly

The SLEEP command was processed even during evaluation mode (ie when parsing the file without running it) causing long delays in the parse time for the tests.

Features added

Added syntax highlighting

Allows you to select syntax highlighting for when you edit the script files, or the include files of a test.

Parser trace saved

The parser output is now viewable from the GUI, since it is possible to open the harness without a console window using the -jar option. Also limited agent traces are available for each executed command on the agent.

Expand tree tool button

A new button has been added to the tool bar to allow instant expansion of all tree nodes

New right-click tree menu option

Added a menu option to allow parsing of single files via the right-click tree node menu, as well as allowing the parsing of any selected tests. A new system menu item was added for this as well.

Bugs/Issues Fixed

DELFILELOCAL command

Was never in evaluation mode - file was always deleted, else error generated. Now fixed.

Features added

Added CHECKAGENT command

Allows you to check if an agent exists or not. See QASH Syntax  for more details.

Serialization cleaned up

Previously, any changes to the TestSpecification class resulted in incompatible serialized versions of the test tree, and furthermore, the entire JTree object and Nodes etc were saved during the serialization process. Now only the tree root is aved, and the writeObject method of TestSpecification has been over-ridden to allow backward-compatibility for future versions of this class.
Current saved trees will now be much smaller, and loading thus faster.

Added MoveUp and MoveDown buttons to Edit Agents window

This allows agents to be easily moved up or down in the agent list for quick re-ordering of agents.

ParserInterface modified to allow saving and viewing of traces

The parser output is now saved to file, and is viewable via the right-click node menu. If called via the commandline interface, it defaults to standard output. Added ViewOutput option to right-click menu.

Bugs/Issues Fixed

QASHParser messages

Message output for unresolved errors was refined a bit. Also only one error per qash file is reported now, since the rest are usually spurious errors.

Agent methods synchronized

Some strange behaviour was noted for the first test run by the agent after it was started. This seems to have disappeared by making all the agent methods synchronized.

StatusWindow repaint problem

When displaying a Status window during a project save/load, the contents of the window were never painted correctly due to a bug in the Swing classes. This has now been worked around.

Right-click menu disabled

The right-click test menu disabled the "Run selected tests" option when right clicking over a non-test object. This menu item is now never disabled. The other menu items will be enabled/disabled depending on the clicked context.

Repaint problem when loading new project

The test tree was not properly repainting after loading a previously saved project. Now it is properly validated and repainted.

Features added

Clear Keyword History

Added a button to clear the keyword history, which otherwise just builds up each time a new keyword expression is applied. Also trimmed a bit of code fluff found in this section.

Added TestFinderInterface

This allows users to define their own class which decides what files get passed to the parser, and are considered as distinct tests. This now completes the functionality allowing users to freely re-define their own Parser and TestFinder implementations.

TraceViewer class added

Instead of the right-click test menu showing separate stdout and stderr traces for each corresponding CMDTRACE command, there is no only one entry per available trace. It is also viewed through a special TraceViewer now, and also includes environment information for the process trace being viewed, the time it was run on the agent, and the time it was recieved from the agent by the harness.
Furthermore, the content of the trace files is autodetected - if it contains valid HTML 3.2 format, the output will be displayed as HTML, inculding links etc which can be followed.

"Running" state introduced

Tests can now be one of five states Passed, Failed, Unresolved, NotRun and Running. This allows one to visually follow the test run progress through the test tree as the "Running" icon is displayed for each test being executed on the agent.

NotePad line numbers

The Notepad class used to edit files now includes line number information for easy cross-reference to syntax errors reported by the harness.

GUI Layout changed

I've changed the GUI layout to try and make it more user-feely-touchy.

Screen size & position saved

The last screen size & position will be saved and re-used for subsequent sessions. The first time it will default to 640x480 centered in the middle of the screen. These values are stored in the <qa_tester>/harness/session.properties file for anybody wanting to hack them by hand.

Added CHECKFILE command to QASH syntax

You can now check for the existence of a file on the agent. See the QASh syntax document for more details.

Command line interface added

For the real sickos, I have added a commandline interface to the tool, which runs single or groups of tests. For details on the commandline syntax, see the Quick Start Guide section on Command Line running.

Status Bar improved

The status bar will now reflect the totalled state of all tests in the test tree. During a test run it will indicate the current progress with relation to number of selected tests.
It now also includes the number of tests not run.

Bugs/Issues Fixed

No cut & paste in Edit Project window

You can now use the keys Control-X, Control-C and Control-V to Cut, Copy and Paste respectively from
all textfield/text areas.

Null pointer exception when first run

The first time it is run, there was an init problem if no default project and session properties were found. This bug was introduced in version 2.3.2 first release.

No error message if try to run tests with none selected

You will now get a nag window if you try this.

Repaint problem

Nodes were not correctly repainted after each test was completed, due to a bug in the algorithm for traversing the test tree

"Parent QAT not set" problem

In some situations the PopMenu had no handle to the parent harness, and so was not able to call it's test run methods.

No way to actively stop a process

Previous version had no way to stop a command except by timing out (not good for, eg http servers)
Now implemented is a CMDSTOP and SLEEP function, which allows better control over process starting and stopping.

Corrupt/Unsupported tree handling

Trying to load a previously serialized tree not compatible with the QAT tool was not handled correctly. Now the tree is manually re-parsed if it can not be de-serialized.

Save Project As was broken

The method used to calculate whether the selected project name already had a ".prj" suffix or not checked for the last '.' character in the string, but if the user omitted this character, it would find the '.' in the harness base dir ".qat/agent" and use this. Now it checks that the fourth last character is a '.' to determine whether it has a file extension or not.

Internal parser modifications

The QASHParser was modified in some sections to lower the BigO complexity of some of the loops, as well as streamlining the token reading process.

Added test root system variable

Include a new QAT system variable called TESTPATH which will allow the qash tests to know where they are located. It corresponds directly to the value used in the project settings window for "Test directory" path.

Features added

Miscellaneous

• toolbar button to run selected tests
• Notepad class updated to allow cut and paste via system clipboard
• included version information in the About box

Implemented the QASH syntax as an Interface

The QAT harness now uses a generic interface ParserInterface to allow users to write custom shell parsers instead of only using the QASHParser syntax. This means people can implement their own style of syntax as they require, and simply plug it into the harness. It is selectable as a Project setting.

Random file transfer functions

You can now transfer arbitrary files to and from the agent via the GETFILE and SENDFILE commands.
You can also delete these files locally or on the agent by calling the DELFILE_LOCAL and DELFILE_REMOTE commands to ensure correctly written tests don't leave files lying around all over the place.

This was the initial public release of the new QASH Harness. Prior to this it was only in use by members of the inner circle.