First @Rhysy, sorry for the laggy reply (an apology I seem to be having to make a lot lately -- I'm not winning awards for timely replies to messages), but maybe there's an upside which is I had some time to think about this.

What I'm thinking is, you could set up a conceptual framework where you think of software usability on a spectrum with software testing.

Level 1: Software is tested such that users don't get stupid error messages thrown in their faces and the program crashes.
Level 2: Software is tested such as to ensure it doesn't make obvious dumb logic errors.
Level 3: Software is tested for subjective aesthetic appeal (colors, fonts, sizing, spacing, etc)
Level 4: Software is tested such that it is actually usable by users

Maybe calling them "levels" isn't quite right because it seems like a lot of people aim for level 3 first -- make my website "pretty" (or desktop app, mobile app, whatever etc).

There are various rigorous software development techniques that are supposed to do a good job at level 1 and level 2. I won't go much into those because the issue you brought up is level 4.

When I lived in California, I worked for a company that did usability testing. The idea was simple. You bring in prospective users of your product (if you have a new version, they could be users of your old version, or they could be users who have never used your product but work in the industry and type of job you're targetting for customers), you sit them down in front of your product, and you give them a big list of tasks to accomplish. And then you sit back and watch, and you're not allowed to help them, you're not allowed to say anything. You just sit there and watch them trying to figure out how to accomplish the tasks from the user interface itself. You watch them struggle. If they get completely stuck, you ask them to "think out loud". They say things like "the task calls for X so I'm looking through all the menus for something that says X..." And of course it's not in the menus, it's somewhere else, or it is in the menus but called something completely different, or whatever. Eventually you may have to tell them how to do it to get them unstuck, but you're really not supposed to help them, you're supposed to just watch them figure it out, or struggle. I've heard it these can feel pretty tortuous for engineers and product managers to watch. In their imaginations, everything is so obvious, then they put their product in front of a real user, and they're confused and struggling. Anyway, after 4 or 5 people, 80% or so of the issues you find overlap, so you go and fix all those issues, then you get another 4 or 5 users and repeat the process. Eventually you get to where most users can accomplish most of the tasks.

That company, even though it made a product for semiconductor manufacturers, got a reputation for making software that is very easy to use.

When I started at my current company, in Colorado, I was shocked to discover the company does no usability testing whatsoever. The guy who runs the company imagines his product is super easy to use. In my mind, it couldn't possibly be, because it's never undergone usability testing. I've told him he should do usability testing, but I gave up trying. He seems to not be able to understand the concept. And he's the boss -- he tells me what to do, not the other way around, so if I say we should do something and he says no, that's it. If I keep bringing it something he doesn't want to hear about, it just gets me in trouble.

I've seen videos online of the usability lab of one of the tech giants. They have these fancy rooms with 1-way mirrors. People can watch from behind the mirrors. To talk to the person undergoing the usability test, they have to push a button to turn a microphone on. They're not supposed to help the person undergoing the test, but sometimes they get completely stuck on a task and have to get external assistance for the usability test to continue.

I've come to believe that even though this company has this fancy usability lab, people higher up the chain of command don't care, because the company's products have all kinds of usability problems. At least in my opinion, they are very bad. I wonder what kind of internal politics leads a company to build these very fancy usability testing facilities only to ignore all of the information produced by them?

I don't know that there's really an "AI" angle on this. I've heard people say that AI just amplifies whatever people are doing. I suspect there is truth to that. Where I work, before AI, there was no testing, and now we have code generated faster with AI, and there's still no testing. There's no testing to verify code doesn't have level 1 error messages, there's no testing to verify the code doesn't have level 2 logic errors, there's not really any testing of level 3 aesthetics -- my boss makes the UI look the way he likes, but we never systematically test that it looks right on all supported devices, or asked any actual users how they feel about the aesthetics, and as I just explained, the company has not ever to my knowledge done a single usability test, so there's no verification of the product's usability.

So in my view, AI has basically taken a low-quality software development process and sped it up. The interesting thing is that so far this hasn't resulted in disaster -- it seems like AI is actually smarter than the average developer so the frequency of Help Desk complaints has trended slightly downward even as code production speed has increased a lot. This might be a general trend throughout the industry -- if AI is mostly smarter than humans at a task, the speed will increase but the quality level will stay about the same or maybe even increase a little, but if the development process didn't already have all the proper verification steps in it, then injecting AI doesn't magically fix that.

The code I've seen that's been generated by AIs just gets more and more convoluted and unreadable as it evolves, and mistakes get ever more hidden.

I've seen something like that. It seems like the AI will generate solutions to each problem it encounters anew, and will not reuse an existing function. This seems especially true if the existing function is approximately what's needed, rather than exactly. Or if the code being duplicated isn't already isolated into a function. A human programmer (me), will extract the code that would be duplicated and modify it so the semantics make sense and it can handle both (or more) use cases.

In order to get quality code, if I'm using an AI to generate code, I have to continually refactor the code it generates. But of course, in order to do that, I have to actually read and understand it all.

At work, where the mentality is to rush code out as fast as possible, having humans read the code isn't possible. For example my boss has said his goal is to get all the developers doing a weeks' worth of work per hour. If you assume a workweek is 40 hours/week, then this represents a 40x speedup. That may at first glance seem crazy, but the fastest developer has done over 80 commits/day -- not every day but he's punched through the ceiling a few times. Before AI he was doing 3-5/day like all the rest of us. Now he's using an "agentic framework" to run many Claude Code instances in parallel. 80 commits/day is ~40x 5/day, so he's already in the ballpark of the 40x speedup my boss is hoping everyone will soon be able to do.

Wayne it continues to sound like the company you work for is the most dysfunctional software company ever! I agree though that others can put up appearances of doing things "right" and still be dysfunctional.

Lol. Well the weird thing is, so far, this hasn't caused a spike in the number of Help Desk tickets. Actually there are somewhat fewer than before using AI. The guy who is the fastest got all the widgets (that people plug into other sites, like WordPress sites) to work with React (large & complex JavaScript framework) in 2 days, and we haven't yet had a single Help Desk ticket about it. (That could be in part because we don't have very many customers using React -- at least not yet, since we never supported it before -- but now that we do that might increase).

I think the mentality here is that we have to use AI to go as fast as possible because if we don't we'll get smoked in the marketplace. If others figure out how to go 10x faster with AI and we don't, we'll get smoked in the marketplace. If others figure out how to go 40x and we only manage 10x, we'll get smoked in the marketplace. Therefore however fast is possible is how fast we have to figure out how to achieve.

I also think we're taking the gamble that whatever technical debt gets created by today's AI models, future AI models that are smarter than today's models will be able to fix. If models continue to improve at their current pace, that looks like it will probably be true. But to me it seems like a gamble. So the decision is to pursue maximum speed and not worry too much about technical debt.

Also, I don't think the company I work at is that unusual. At least not in the market where we operate. The company makes web software for real estate, and it seems like people use clunky WordPress and Wix sites all over the place. Wix especially is buggy junk but it doesn't seem to stop massive numbers of people from using it.

@Wayne Radinsky Yeah, the UX testing you describe is exactly how it was presented in the conference, almost word for word.

In astronomy the whole concept of UX itself is still something novel, even more so the idea that it's something you could actually have a strategy to test for. Here too is something of a similar outcome with commercial software though : the priority is to get more toasters (shiny new results and/or software) rather than fixing the ones we already have (making software easier to use so more people would benefit from it).

Thankfully, it does seem like there is at least now a more vocal sub-community who are sick of having to deal with undocumented code and command line interfaces when a graphical interface would be equally functional and vastly easier. But I still regularly encounter people who build presentations using LaTeX, which just scrambles my brain that anyone would do that. It seems self-evident to me that graphical results should have graphical inputs...

I definitely agree with the AI-as-amplifier notion. Having the ability to write code in a different way is unlikely to mean that most people will automatically adopt different strategies : if they prefer a CLI, then they'll just write more codes with CLI rather than converting to GUIs. But at least in this sector, where open source is increasingly standard, this at least allows the prospect for of revising to one's preferred input method without needing to learn the entire thing.

In fact, one current script I'm working on is exactly this. The original (independent) code provided the core, but the structure was (for me) awful and weird. I've then spent a long time recoding this into something that makes sense to me. ChatGPT was extremely helpful in sanity-checking what this does and making various custom improvements I needed. After getting to a functioning state, I let it slap on a GUI through pure vibe coding (with iterative refinement and careful testing). This was a massive time saver for me : without it, I might have tried to write the GUI myself, but I also might have made do with the original crappy CLI. But this does require a conscious effort and choice to focus on UX rather than devote more resources to the science.

I think of the importance as UI/UX as the Facebook Principle. Sure, you can go around taking photographs and posting physical copies of them to random strangers, but that requires an inordinate amount of effort and marks you out as clearly a weirdo. Give people an interface that makes this effortless and suddenly it's the people who aren't doing this who are the weirdos... yet it still it's something of an uphill struggle to persuade people that this really matters.