Software development has changed dramatically—especially over the last few months. In my bubble, the boom in AI-assisted development dates roughly to January 2026, when many developers who had not previously used any such tools began using AI agents for development. Since then, our entire industry has moved far beyond what used to be standard practice. Very few people now doubt that code will no longer be written by hand. However, AI tools still produce better results when they act as an extension of a professional—a software engineer who knows how to communicate with an agent as effectively as possible to achieve a high-quality result.
People working in QA—those who take care of and assume responsibility for the quality of the software produced by a team—must change along with the world. As the way we consume software changes, I believe this field will undergo further massive transformations and the profession will shapeshift even more. For now, however, we cannot merely dream. Software is still used by millions of people who click buttons and fill in fields. So how can we test this kind of software as effectively as possible today?
I would like to summarize my vision and explain what I currently do on my own project. Every project has its own specifics, but I think the type of project I work on is quite common today, both in terms of the product's focus and the technologies we use. I will describe the project shortly to provide some context. First, however, I would like to point out that I currently work in a hybrid role on the project: I produce production code in my AI kitchen for the parts of the app I feel confident working on, while also taking care of the QA side of our project.
What we test
It is an AI chatbot for a company in the legal sector. Put simply, it is a kind of internal ChatGPT into which we incrementally implement ideas from the client's innovation team. Technically, it is a Next.js web application, while the server side runs in NestJS in a separate repository. We also have several specific add-ins, each with its own repository, including add-ins for Word and Outlook. Some time ago, a custom text processor was implemented to help create RAG pipelines in the infrastructure, which users can create in our app as they wish. The client and server sides of the web app are therefore written in TypeScript.
How we used to test
Let us first establish a baseline: how things were done before, or rather until quite recently—literally only a few months ago. Most of the time, manual testing was performed for each ticket or according to a predefined scenario, such as regression tests. Automated tests were created either for a bug fix or an implemented feature, or at various points after implementation, depending on how complex a particular test was supposed to be, how many components in the required versions had to work in it, and ultimately which testing team worked on it. Tests varied in complexity according to the level of the testing pyramid at which they were implemented.
In some cases, this meant only manual testing performed by a tester, which took time, especially when it also involved analysis, monitoring data in the database or cache, and so on. Developing an automated test also took a lot of time—creating one often required three days or more, depending on the type of test.
How we can test now
From my perspective, we currently have three types of testing activities that we can freely combine, reshape into hybrid forms, and incorporate into our workflow:
- manual testing by a person,
- test automation,
- AI agent-driven testing.
Let us examine them one by one, look at the role they play today and how they can be combined, and then do a reality check.
Manual testing
The final product is still usually an application in which a user clicks and enters things using a keyboard and mouse, or often works with the application through a mobile interface. The end user is therefore still a human organism who perceives software through the senses. I once heard in a podcast that game design—making a computer game fun—is still difficult to achieve through LLM programming because the final experience is more than a collection of game mechanics and their visual execution. It is something more, something that feels right to the player—it “feels good.” I believe this can also be applied, in a certain sense, to conventional software, including web applications in our case. The feeling a product creates still plays a major role, as does how easily and effectively the end user can access the value the application provides.
Therefore, as long as people use software, it is good to have a tester—a real person—at the end of the testing strategy. They should focus on the things that matter most: the most frequently used features and interfaces, especially whatever lies at the core of the product. While working on test automation and instructing an AI agent to perform testing, a good strategy is to ask it: “What should I—the human—test?” But that takes us ahead to hybrid methods. Let us move on to the next activity.
Test automation
Test automation is inexpensive, and at the moment it is almost a sin not to create it. The great thing is that its code rarely needs to be written manually anymore. We do, however, need to write—or rather prepare—the rules (instructions for the AI agent) that allow it to generate itself effectively. By this, I mean using an AGENTS.md file (or CLAUDE.md, or whatever else a team uses in its repository or repositories; I will use only the first term from this point on) and defining a foundation for every level of the testing pyramid.
- On the server side, choose a testing tool—Jest in our case, for example—and mention in AGENTS.md that unit tests are written in Jest and that a new unit test should be created alongside a bug fix or feature implementation whenever appropriate, unless there is a compelling reason not to do so.
- Define what integration tests should look like on the server side and instruct AGENTS.md accordingly.
- Prepare the environment for end-to-end tests—in our case, Playwright in the client-side repository—and mention in the relevant AGENTS.md file that this is where they are located.
The person responsible for test automation will then only have to maintain this code, run the tests, and analyze the results. Yes, the tests will create themselves. These tests do not even need to be the focus of PR reviews, because that takes time and developers should concentrate on production code. As of September 2026, developers still review code during code review. If the tests have problems, they should be maintained, for example, by someone on the team dedicated to testing whose time is a little less expensive.
The advantage is that while fixing a bug or implementing a feature, a developer often accumulates a wonderful amount of relevant context about the implementation they are working on. It would be a shame to waste this opportunity and not instruct the AI agent to generate tests while it knows exactly what is going on. Once we add this to the system instructions in AGENTS.md, the developer no longer has to worry about it, and the tests will be generated alongside the implementation.
Testing with an AI agent
We are now reaching a point that will probably blend into the question of how to combine all three activities: manual testing, test automation, and testing with an AI agent. We have already discussed how a developer engineers a solution and the QA setup they should have available. Once a ticket passes PR review—which I still consider a good checkpoint—it can move on to a kind of QA check.
This brings us to the highlight of the process. This is the moment when a tester can take a ticket and perform a comprehensive QA check on what has been implemented. If they connect Jira or Linear—whichever their team uses—to their AI harness, the ticket number is often all they need to get started:
“Let's test TICKET-001. Test the implementation yourself, check what the test coverage is across all levels (unit, integration, e2e). Run all the E2E tests at the end of your testing to make sure it didn't mess anything up elsewhere in the system.”
With Playwright configured correctly and access to the repositories and Jira, an AI agent is often capable enough to work out how to complete a large part of the job. One great thing about Claude Code, which I use, is that it has memory. In addition to the shared system instructions I use, its memory already contains a considerable amount of information about what I have done recently, and it draws from that as well.
AI can pull in the ticket and its description, the exact changes made to the code, and the tests that were implemented... You get the point. In my experience, Claude Code is smart enough to prepare whatever it needs. It can work in the client-side repository using the credentials I keep in the Playwright .env.test file, start temporary Docker containers for the database or cache, or prepare a script and generate data. Testing something today is amazing. We no longer have to think of every little detail and edge case that previously demanded our attention.
Taking a step back
All of this sounds great, but our reality, for example, is that every project—the web client, server, add-ins, and DevOps pipelines—lives in a separate repository. They all reside under one roof in Azure DevOps, but they do not communicate with one another. That is why we prepared not only an AGENTS.md file in every repository, but also another AGENTS.md in the local parent folder where a developer clones all the repositories. It explains what the repositories share and what each one is responsible for. I can then open my AI tool—Claude Code in my case—in that parent folder, and the AI moves into whichever projects it needs.
So far, I have also created another folder containing additional AI instructions just for myself. There I provide more detail about our testing layers, the naming conventions for tasks, branches, and PRs, and the general rules for descriptions. I also note that I do not want many comments in the code, among other things.
The point is that the system consists of several Git repositories. If we want the AI agent to know about all of them, we need to let it operate from a parent folder where it has access to every repository, while also giving it concise information about the system and where to find things.
QA is shifting toward testing the human perception of a product
I discuss an alternative perspective on this issue, including how the suite of automated tests can be expanded with performance or security tests, in Team of testing AI agents.
The conclusion of this entire article is that testing can now genuinely focus on what we subjectively perceive as important. I would say that a tester's subjectivity now plays an even greater role than it did in the past. The job will no longer be about boundary value analysis and similar techniques that we learned while preparing for ISTQB certification. AI will generate these logical tests. Our role will be to empathize with users and use the software as they do. I might even say that it is time for QA to move closer to users than ever before.