We Keep Hearing About AI Productivity. The QA Group Chat Told a Different Story.

An anonymous poll, a debate about multimodal testing, and some uncomfortable questions about who benefits from the AI rush.

Recently, I published an article on DEV.to about our company's AI testing practices and the organizational changes that followed.

It was selected for DEV's weekly Top 7 Featured Posts and included in the official DEV newsletter.

The comments were more interesting than I expected. Readers from different backgrounds started sharing their own experiences. They talked about outdated PRDs, agents reporting the same bug repeatedly, isolated test environments, and the mess that automation can expose when people have been quietly filling in the gaps.

One reader from the manufacturing industry even compared our experience with his factory's automated material-tracking project. Different industry, same problem: once you automate a process, all the messy handoffs that people used to handle manually become much harder to hide.

I was still thinking about those discussions when something else caught my attention.

It happened in a QA community group I'm part of.

1. Sixteen People Said They Weren't Employed

The group has around 500 members. Someone started an anonymous poll asking a simple question:

Are you currently employed?

When I checked the results, 76 people had voted.

Sixteen people.

To be clear, this isn't an unemployment survey of the software testing industry. The poll was anonymous, and we have no idea why those 16 people weren't working. They could have been laid off, resigned, or simply be between jobs. Only 76 out of roughly 500 group members participated, so there's no reason to treat this as an industry-wide statistic.

Still, the number got my attention.

I also have a former colleague who works in software testing. They've been out of work for six months and still haven't found another job.

That's no longer just a number in an anonymous poll. It's someone I know and have worked with.

I don't know all the details of their job search, and I wouldn't automatically blame AI for their situation. But six months without finding another role is difficult to ignore.

The group chat quickly filled up with comments.

All we're seeing is layoffs. Nobody's hiring.

Almost 50. Who's going to hire me now?

Is there still a future in this field?

Some people suggested changing careers. Others said it was time to prepare for what's coming.

One comment summed up the mood rather well:

With all this AI-driven productivity improvement, it's getting harder and harder to keep up.

Then, almost as quickly as the conversation started, it turned to AI testing.

2. Can AI Really Compare an App Screen with Its Design?

Someone asked whether AI-powered test automation could compare an actual app screen against its UI design, checking element sizes, colors, positions, and other visual details.

The suggestions came quickly.

Some people proposed OCR to extract information from the design and the actual screen. Others suggested multimodal models. There were also concerns about maintenance: what happens when the UI changes slightly? Do we have to rebuild the whole thing?

Someone joked:

How well will it actually work? Who knows?

I joined the discussion.

My answer was that it's technically possible, but the AI needs enough context and sufficiently detailed instructions. Just throwing a demo page at a model and expecting it to understand everything won't get you very far.

I also pointed out that using a top-tier multimodal model for this kind of work can consume a lot of tokens.

Then came my questions:

Is management paying for the tokens?

And:

If not, they can go to hell.

I made the point one more time:

You can use a top-tier multimodal model, sure. But the token consumption can be huge. If management isn't paying for it, to hell with that.

Yes, I really said that. 😆

The joke aside, I meant it.

There's a difference between proving that something can be done and proving that it's worth doing.

OCR, DOM inspection, image comparison, and multimodal models can all help with UI verification. The right approach depends on what you're trying to check. Not every visual problem needs an expensive model.

But once you start taking screenshots, sending them for analysis, comparing results, and repeating the process across multiple runs, the bill can grow quickly.

Before adopting a solution like this, I'd want to know how many real issues it catches, how many false positives it produces, how much it costs to run, and how much effort is needed to maintain it when the UI changes.

If a company expects testers to use AI but leaves them to pay for their own tokens, something is wrong.

I've seen people in that group joking about collecting free credits from different platforms every morning before starting work.

It's funny, but think about it: employees are being told to embrace AI and improve productivity, while some are trying to keep their experiments running on whatever free credits they can find.

If a company wants AI-driven productivity, AI usage needs to be part of the company's budget.

3. AI Writes the Code, AI Tests It. What Could Possibly Go Wrong?

Another part of the conversation made me laugh.

Someone described an almost completely automated development process:

Describe a requirement, call the product skill, generate a demo and PRD, call the technical skill, generate the technical documentation, architecture, interfaces, and framework. Write the code, generate the tests with AI, run everything, and ship it. Done. A pile of bugs goes live.

That last sentence says a lot.

Someone else complained:

The boss uses AI to generate a demo, decides the idea is viable, and then expects everyone to build it.

Another person added:

Management expects us to deliver exactly that.

And someone joked:

Sounds like all we need is the boss. Everyone else can leave.

I've seen why this attitude develops. A demo can look convincing, and AI can generate a surprising amount of code in a very short time. It's easy to look at that output and assume the hard part is already done.

But a working demo isn't a validated product.

A generated PRD isn't proof that the requirements have been properly discussed. Generated code and generated tests don't automatically share the same understanding of what the system is supposed to do.

I ran into a version of this problem in our own testing work.

In one knowledge-base project, AI found 36 potential defects, while manual testing found 11. After review, 30 defects were confirmed as valid. AI had contributed 21 of them.

Those numbers were encouraging, but something else caught my attention.

Some of the reported issues weren't actually implementation bugs. Product and development had already agreed on changes, but nobody had updated the PRD.

The AI was following the document. The people were following an agreement that existed only in conversations.

In the past, testers often bridged that gap through experience and informal communication. They knew who to ask and which version of the requirements the team was actually following.

The AI didn't have that context.

It did exactly what it had been asked to do, and the result was a test report full of issues that looked legitimate but weren't necessarily bugs.

That's one of the things I find interesting about AI testing. It doesn't just test the software. It can expose how disconnected the people building that software actually are.

4. Why Is Every Company Suddenly Chasing AI?

As the group conversation continued, people started discussing the industry rather than individual jobs.

One member argued that many Chinese companies keep a close eye on what's happening in the US, where much of the current technology leadership comes from. One member said Chinese companies keep a close eye on what's happening in the US, and estimated that the gap between the two markets was at least six months.

That's their estimate, not a figure I've independently verified. But the broader observation is worth discussing.

A lot of companies seem afraid of being left behind.

One company adopts AI for coding, so others feel they need it too. Another introduces AI testing, and suddenly everyone wants an AI testing strategy. When large technology companies start talking about full-stack AI transformation, smaller companies feel pressure to follow.

Someone in the group put it bluntly:

Everyone just copies what's already working elsewhere. AI cuts costs and improves efficiency. What boss wouldn't like that?

Fair point.

But how many companies have actually worked out where AI fits into their business, rather than adopting it simply because competitors are doing so?

Another member brought up education and talent development. For years, the emphasis had been on catching up and getting new technology into production quickly. Now, they argued, the demand is shifting toward people who can innovate, lead, and break new ground.

Whether that shift is happening quickly enough is another question. But the contrast is interesting: following someone else's lead and creating something new require very different skills.

There was also a comment about management thinking:

Most companies are run by professional managers focused on making it through this year. How can you expect them to think long-term?

I understand the frustration behind that.

When a company is under pressure, cost reductions and headcount cuts are easy to measure. Building new capabilities, developing people, and investing in ideas that might pay off several years later are much harder to justify.

AI fits very neatly into the first category. Its potential for the second is real, but much harder to guarantee.

5. Is AI Productivity Becoming a Sales Pitch of Its Own?

One group member raised a more cynical possibility:

Everyone is selling tokens. They need to create the impression that AI improves productivity and that companies will fall behind if they don't use it. That pushes everyone to adopt AI. And with the market already in bad shape, companies can use AI as another reason to cut jobs.

I wouldn't go as far as saying that the entire AI productivity story is manufactured. I use these tools myself, and I've seen what they can do in testing, code analysis, and bug discovery.

But there are commercial incentives here that are worth recognizing.

AI providers want companies to use their models. Companies want to lower costs. Employees are being told that they need to become more productive with AI.

Those interests can reinforce one another, even when nobody has fully worked out the business case.

A company might adopt AI because it doesn't want to fall behind, rather than because it has identified a clear problem that AI can solve.

And once AI is in place, the conversation can quickly shift from what the technology can improve to how many people the company can do without.

One person in the group pointed out another part of the problem: the IT industry went through years of aggressive hiring. Large numbers of people entered the market, and companies expanded their teams rapidly.

Now the market looks very different.

From that perspective, AI isn't necessarily the original cause of the employment pressure. It may be the trigger that makes an existing imbalance more visible.

I think that's an important distinction.

We shouldn't assume every job reduction is caused by AI. Market conditions, years of hiring, changing business models, and company decisions all matter.

But AI gives companies another way to reorganize work, and that can accelerate changes that were already underway.

6. We Keep Pushing Harder. Where Does It End?

One line from the group chat stuck with me:

We keep trying to outdo everyone else, until we end up working ourselves out of a job.

Someone else described the result as a possible vicious cycle.

The IT industry has always moved quickly. One group member joked that if you stop learning for a year or two, you'll fall behind. And people in their thirties and forties are still pushing themselves to learn new things constantly.

I know what that feels like.

New frameworks, new models, new agents, new tools. Before you've finished getting comfortable with one, three more have appeared.

Companies want employees to become more productive. Employees want to stay competitive. AI providers want more adoption. And companies watch their competitors to make sure they're not falling behind.

Everyone has a reason to run faster.

But here's the question: what happens when every company tries to do more with fewer people, while the number of available jobs doesn't grow at the same pace?

We could end up in a strange situation where every company becomes more efficient, but the people working in the industry feel less secure.

We might work so hard trying to beat everyone else that we end up making the whole situation worse for ourselves.

7. Embrace AI, or Get Fired Now. Embrace AI, and Get Fired Later.

There was one more line in the group chat that deserves to be repeated:

If you don't embrace AI, you'll get fired immediately. If you do embrace AI, you'll just get fired later.

It's a joke, but not a particularly comfortable one.

Companies tell employees to learn AI, use AI, and improve productivity. Employees do their best to keep up.

But what happens after they become more productive?

Some worry that they'll end up automating their own jobs. Others are concerned that age and a changing job market will make it harder to find another role if they lose the current one.

People in the group were already suggesting that others should change careers or start preparing for something different.

I don't think learning AI is pointless. And I certainly don't think refusing to use it will protect anyone's job.

But embracing AI doesn't guarantee job security either.

Learning AI can make you more capable. It can't guarantee that a company will continue to need you.

Those are two different things.

Individuals can improve their skills, but they can't control the job market, company budgets, or every organizational decision.

I don't think it's fair to put the entire burden of career survival on whether someone is willing to learn the latest AI tool.

Final Thoughts

I don't want to use the 21.1% from this poll to claim that the testing industry is experiencing mass unemployment.

It was an anonymous poll in a group of around 500 people. Only 76 participated, and 16 said they weren't currently employed. We don't know why.

It isn't an industry-wide survey.

But I think the conversation is worth remembering.

People were worried about losing their jobs, then spent time discussing how AI could help them perform more advanced testing.

They were worried about being replaced by AI, while exploring how multimodal models could improve their automation.

They were concerned about token costs, while looking for free credits from different platforms.

They questioned whether AI was really improving productivity, while worrying that their companies might fall behind if they didn't adopt it.

And I have a former colleague who has been out of work for six months and still hasn't found another job. That makes the conversation feel a lot less theoretical.

I'm embracing AI too. I'm building AI testing workflows, experimenting with agents and models, and changing some of my own assumptions about what automation can do.

I see the possibilities. I'm not interested in pretending otherwise.

But I also don't think productivity gains automatically solve business problems, create new markets, or guarantee better opportunities for employees.

We can build faster, generate more code, run more tests, and produce more output than before.

The harder question is what all that extra productivity is actually going to achieve, and who gets to benefit from it.

There's a Chinese idiom I keep coming back to: 管中窥豹 — looking at a leopard through a narrow tube. You can see a few markings, but never the whole animal.

That's what this poll and the group chat offer: a glimpse, not a complete picture of the QA industry. Seventy-six votes cannot tell us how widespread the problem is, and one group conversation cannot explain what's happening across every company.

But it does reveal a few things worth paying attention to: job anxiety, pressure to adopt AI, arguments over token costs, and uncertainty about who actually benefits when productivity goes up.

We may not be seeing the whole leopard. But what we can see is worth taking seriously.

If we're all becoming more productive, why are so many people becoming less confident about their future?