Vibe Coding Meets Agentic Coding: Why Playwright Tests Matter

AI coding agents can turn a feature request into working code remarkably quickly. Once the new interface behaves as expected in the browser, the change may appear complete. That observation, however, says little about whether an existing workflow still works. In a product with several user roles, even a seemingly minor permission change can leave someone unable to approve a request that previously went through without a problem.

Regressions are not unique to AI-assisted development. But when teams can implement larger changes more frequently, reliable feedback becomes increasingly important. The useful question is not how much code an agent produces or how many tests it runs, but which requirements those checks actually protect.

Vibe coding is not the same as agentic engineering

Vibe coding is often associated with translating an idea into working code through AI with very little attention paid to the implementation. The term is not used consistently, but the emphasis is usually on speed and experimentation. An experiment can get by with that. Once the application has to run reliably and continue to evolve, a convincing interface is not sufficient evidence of quality.

Agentic coding goes beyond asking an AI model for individual code suggestions. A coding agent can inspect a repository, modify files, run tools, and respond to test results. Whether those capabilities lead to a controlled development process depends on the requirements, constraints, and ways to verify its work.

Martin Fowler makes a related distinction between vibe coding, where reviewing the generated code takes a back seat, and agentic engineering, where developers deliberately oversee the agent’s work and remain accountable for the result. For a team maintaining a product, that distinction matters more than which model happened to produce a particular change.

Why an apparently successful change can still cause regressions

An agent may implement the requested feature correctly while breaking a business rule that was never included in the task description. This is particularly relevant to applications with permissions, approval states, and interdependent workflows. In such systems, the new feature is only part of the acceptance criteria: existing user journeys must continue to behave as intended.

The ability to change many files quickly does not create a fundamentally new category of defect. It does increase the value of feedback that goes beyond a successful build and clean type checking. A web application may start without any technical errors while users remain unable to finish an important task.

Research has begun to examine these limits. A 2026 exploratory study by Mock and Russo compared different models of human-AI collaboration in test-driven development. The findings suggest that agentic workflows can introduce implementation decisions for which suitable tests have not been written. This does not establish a general defect rate for AI software; it reinforces the importance of treating test design as work in its own right.

A separate 2026 research preprint examined 4,882 agent-generated pull requests in Java and Python repositories and identified gaps in coverage of changed code. The study measured code coverage, not the effectiveness of Playwright E2E tests. Its results nevertheless illustrate a broader limitation: the presence of automated tests does not establish that the risks that matter most have been verified.

How a test harness supports coding agents

To verify its changes effectively, a coding agent needs an environment that provides clear instructions and reliable feedback. Such an environment is commonly called a harness. OpenAI’s account of harness engineering describes structured project knowledge, technical rules, automated validation, and feedback loops.

For a browser-based product, four layers can work together:

1. Requirements and boundaries: The agent needs to understand the relevant business rules, user roles, and the parts of the project it may change.
2. Fast technical checks: Linting, type checking, and appropriate unit and integration tests catch many problems early.
3. Targeted browser checks: Playwright verifies agreed critical user journeys in a controlled test environment.
4. Review and release controls: Code changes, results, and outstanding risks remain visible, while independent CI checks inform the merge decision.

No combination of these layers prevents every defect. Together, however, they give an agent better opportunities to identify a problem, investigate its cause, and make corrections based on evidence rather than guesswork.

Why a thousand tests may be worth less than ten good ones

Generating a large number of tests is relatively easy for a coding agent. Deciding which workflows deserve protection is more demanding because the answer cannot be inferred from the user interface alone. It depends on the product’s business rules, roles, failure consequences, and expected outcomes.

Consider a test that checks whether an “Approve” button is visible. It may pass even though clicking the button stores the wrong status. Another test may fail after every small design change because it follows an arbitrary CSS structure. In both cases the suite adds test effort and runtime without reliably checking the business-critical workflow.

Before automating more tests, a team should therefore establish:

• Which workflows are most important to users and the business?
• Who should be allowed to complete a particular action, and who must be prevented from doing so?
• What outcome must be present after an action succeeds?
• Which boundary conditions and failure paths have meaningful consequences?
• Which checks are best implemented at unit, API, or E2E level?

The ISTQB syllabus treats the risk-based selection of test activities as part of professional quality assurance. Playwright likewise recommends verifying user-visible behavior and keeping tests independent. The value of a suite depends less on its size than on the relevance of its assertions and the reproducibility of its results.

Example: A new role changes an approval workflow

Imagine an internal web portal where employees submit requests and team leads review and approve them. The product now needs a deputy role that can approve requests when a team lead is unavailable. A coding agent adds the role and confirms that the deputy can open the request screen. That alone does not establish that the full approval process works under the intended permission rules.

A Playwright test designed around the business requirements follows the process through to its result. It verifies that an authorized deputy can approve a pending request, that the new status is saved correctly, and that the employee who submitted the request can see the update. Another test confirms that an unauthorized user cannot perform the same action. Controlled accounts and repeatable test data make both cases reliable enough to run again after subsequent changes.

These tests protect the approval workflow the portal was built to support, rather than merely checking whether a new screen renders.

Choosing the right checks at each stage of development

Running an entire, growing E2E suite after every small change can slow down development unnecessarily. On the other hand, running only tests associated with a modified file can miss regressions in other parts of the product. A layered execution strategy offers a more practical balance, with the scope of testing determined by the risks involved.

A dedicated test subagent can select and launch suitable checks, inspect failure reports and Playwright traces, and return findings to the coding agent. However, deciding which tests should run and when a failure must block further work is a matter of harness design. Playwright supports targeted execution through features such as test tags; orchestration by a separate subagent is an additional architectural choice.

An agent’s ability to test and correct its own work is valuable, but its self-checks should not be confused with independent approval. Running the agreed checks again in CI before a merge helps the team make that decision on the basis of results it can inspect.

Maintaining a suite people can trust

Good E2E tests depend on stable test accounts, controlled data, a reliable staging/QA/UAT environment, and maintenance when the product’s behavior changes. Tests that pass and fail unpredictably without corresponding product changes become difficult to trust as a release signal.

Teams should therefore review whether each test still protects a meaningful risk or merely adds execution time and maintenance work. A suite may begin with a small set of critical user journeys and grow as the product requires. Performance, security, architecture, and data integrity may also require their own forms of validation. Playwright E2E testing is a valuable layer for browser-based software, not a standalone guarantee of enterprise readiness.

From fast AI output to maintainable software

The answer to AI-assisted development is not to add as many checks as possible to every step. Effective QA should allow teams to keep moving quickly while discovering early enough when a change has damaged functionality people rely on. Achieving that balance requires deliberate test selection and a harness that feeds useful results back into the development process.

Velvionix supports software teams with QA consulting and maintainable Playwright E2E tests for critical user journeys, including agreed CI integration. This work draws on more than five years of hands-on QA engineering experience at a publicly listed company, supported by ISTQB certification. Which workflows to check first, and whether a pilot or an extension of the existing tests is the better fit, depends on the application and its risks.