Future

Example of Targeted Software Testing using AI

A simple Login app was written (available for download in the Downloads section). This ZIP file contains a .war file that should be added to the webapps directory of a Tomcat9 installation. Once deployed, the app will be accessible via the URL http://localhost:8080/SciFiSoftwareApp

The following user accounts are enabled in the app.

To use this app, first try to log in with credentials which are unknown to the app. You will see an error message "Credentials not recognized". Next, log into the app with the ORDINARY credentials, so the Home page is displayed, then log out. Finally, log into the app with the ADMIN credentials. On the Home screen, click the Report button. The Report page will be displayed, showing the recent login attempts (successful and unsuccessful). Note that the error message displayed earlier is shown in the report.

I saved the app's sourcecode as old.zip

I changed the error message text in the following source code:

src/controller/LoginServlet.java

Next, I saved the updated source code in a file called new.zip, then I uploaded old.zip and new.zip to ChatGPT (don't worry, we won't do this with your source code) along with the Test Cases we had already defined.

This is a list of all the Test Cases used to test the SciFiSoftware App. If you want to see them, the Test Cases are in the SciFiSoftware App ZIP File (see Downloads page).

• SFS TC 01 - Verify the Welcome page is displayed as expected
• SFS TC 02 - Verify the Log In page
• SFS TC 03 - Failed Login
• SFS TC 04 - Successful login (Ordinary role)
• SFS TC 05 - Successful Login (Admin role)
• SFS TC 06 - View Attempted Login Report
• SFS TC 07 - Security Check - Home page access when logged out
• SFS TC 08 - Security Check - Clicking browser Back button does not allow you to see the Home Page or the Report Page
• SFS TC 09 - Security Check - Verify Ordinary user cannot view Reports page

You can probably see which Test Case we need to run: SFS TC 03 - Fail Login right?

OK, so I asked ChatGPT what was the minimal set of Test Cases we needed to run to verify the change.

This is the reply:

Change Detected in LoginServlet.java
Modified Error Message:
diff
- LoginAttemptLogger.getInstance().log(email, false, "Unrecognized credentials");
+ LoginAttemptLogger.getInstance().log(email, false, "Credentials not recognized");

Impact Analysis
The logic is identical — only the wording of the login failure message changed.
This affects:
The login.jsp error display
The LoginAttemptLogger (used in the report.jsp and report test cases).

Minimal Test Cases to Run
To verify this change, only the following test cases need to be re-run:
TC 03 – Failed Login
Confirms the error message is correct in the UI.
Verifies that the failed login path triggers correctly.
TC 06 – View Attempted Login Report
Ensures the new wording (“Credentials not recognized”) appears in the login attempt report.
    

See how it picked out both the SFS TC 03 - Failed Login test (which we guessed) but also the SFS TC 06 - View Attempted Login Report?

It told me the minimal set of TCs to run to verify the change! This is a considerable improvement over relying on the QA Team to interpret Release Notes written by the Developers, which are written from the point-of-view of a Developer, rather than a User of the app you are testing.

It also gave me a list of User Scenarios that we hadn't thought of to test. This will save you time/money and give you a better product!