Skip to main content

Test a change in the browser

Have the agent test a change in the running app before your team reviews it. The agent keeps a Testing Checklist of what needs checking, and /kosuke:automated-qa tests each item against the Preview. Each run records a video per section and publishes into the branch's one QA Report. Its version counts the runs since the last --restart. The pull request links that report.

Check the checklist​

Whenever a turn changes how your app behaves, the agent updates the Testing Checklist of the behavior changed in this chat session, with a severity for each item. It appears as an artifact in the Artifacts tab. Read it, and ask the agent to add or drop items before you run it. If you edited code by hand, ask the agent to bring the checklist up to date.

Run the tests​

Send /kosuke:automated-qa.

Testers open the app in headless browsers and work through the checklist, one section each. Each item ends as passed, failed or blocked. The run publishes the QA Report artifact, at the next version, with the results, the reasons for each failure and one video per section.

To test only part of the list, say so after the command, for example /kosuke:automated-qa section 2.

The Artifacts tab of the same session with the QA Report open, above Testing Checklist and the Adversarial review in the list. A Failed badge sits before the heading "QA Report v1", with "1 failed" and "1 blocked" buttons to its right and the passed count cut off at the edge. Under the branch kosuke/chat-4kd9wp and "7 min", the selected item, "A shipped order's window starts three working days after it shipped", reads "Failed, major", with the tester's reason, what the tester did and the cells they read. Section 2, Shipped orders, comes first with a 0/2 tally, its failed item and a blocked item, "The window moves when an order ships". Section 1, Delivery column, reads 3/3 with three passed items. Every item row reads "No video", and the report shows no video player.

Fix what fails​

  1. Send /kosuke:automated-qa --fix. The agent repairs the failed items, commits each fix and tests those items again.
  2. Push the fixes with Push in the chat footer, or send /kosuke:open-pr, which pushes pending work before it opens or updates the pull request.
  3. Send /kosuke:automated-qa --restart to test every item again on the pushed code.

Each --fix run makes one round of repairs. Send it again for another round.

Put the evidence on the pull request​

/kosuke:open-pr, and the Open PR button that runs it, link the QA Report in the pull request description. To add other links, such as a recording, use -e:

/kosuke:open-pr -e https://example.com/recording.mp4

When the pull request is open, a QA run also labels it qa: passed, qa: blocked or qa: failed. qa: blocked means nothing failed but some item could not be tested. It posts a comment with the counts and links to the report and the videos. A run labels the pull request only when everything is pushed and no item was last tested on an older commit. While the pull request is open, a --fix run that commits a repair labels nothing, because the repair is not pushed. Push it, then send /kosuke:automated-qa again to label it.

The links in the pull request open the chat session, so only members of your workspace can follow them.

Share it outside Kosuke​

To show the change to someone without a Kosuke account, share the Preview.

  1. In the Preview tab, press Share preview, then Create share link.
  2. Copy the link and send it.

Anyone with the link opens the running app. When the session has artifacts, such as the QA Report and its videos, an Artifacts view shows them too. The link works until you press Revoke link. Preview and sandboxes covers sharing.

Next steps​