Toxic experts

Thursday 2 November 2017This is nearly six years old. Be careful.

I wrote Big-O: how code slows as data grows to explain Big-O notation. My goal was to explain it in broad strokes for people new to the idea. It grew out of thinking I’d been doing about beginners and experts. In the comments, there was an unexpected and unfortunate real-world lesson.

An anonymous coward named “pyon” said I should be ashamed. They pointed out a detail of algorithmic analysis that I had not mentioned. It’s a detail that I had never encountered before. I think it’s an interesting detail, but not one that needed to be included.

Pyon is an example of a toxic expert. People like this know a lot, but they use that knowledge to separate themselves from the unwashed masses of newbs. Rather than teach, they choose to sneer from their lofty perches, lamenting the state of the world around them, filled as it is with People Who Don’t Want To Learn.

The important skill pyon and other toxic experts are missing is how to connect with people. They could use their knowledge to teach, but it’s more important to them to separate themselves from others. Points of correctness are useless without points of connection.

Toxic experts care more about making distinctions between people to elevate themselves than they do about helping people. Beware: they are everywhere you look in the tech world. It’s easy to run into them when you are trying to learn. Ignore them. They don’t know you, and they don’t know what you can do.

Pyon is fixated on a particular detail of algorithmic analysis, and feels that it is essential to understanding Big-O. What I can tell you is that I am doing fine in my 30-year career, and I had never heard that particular detail. My Big-O piece wasn’t meant to be exhaustive. There are entire books written about algorithmic notation. I even wrote at the end, “There’s much more to algorithm analysis if you want to get into the true computer science aspects of it, but this is enough for working developers.”

But pyon can’t see the forest for the trees. Experts have spent a lot of time and energy learning what they know. They love their knowledge. They wouldn’t have been able to get where they are without a passion for the subject. But sometimes they have a hard time seeing how people can be successful without that well-loved knowledge. They’ve lost sight of what it means to be a beginner, and what beginners need to learn.

Toxic experts will latch onto a particular skill and decide that it is essential. For them, that skill is a marker dividing Those-Who-Know from Those-Who-Don’t. These shibboleths vary from expert to expert. In the current case, it’s a detail of algorithmic analysis. I’ve seen other toxic experts insist that it’s essential to know C, or assembly language, or recursion and pointers, and so on.

I’m not saying those aren’t good things to know. The more you know, the better. Every one of these topics will be useful. But they are not essential. You can do good work without them. You certainly don’t deserve to be spat upon.

The ultimate irony is that while pyon and other toxic experts are bemoaning the state of the industry because of missing knowledge, they are highlighting the main skill gap the tech industry needs to fix: empathy.


Hi Ned! I agree. My approach to all such toxic experts has always been: 1) read for the data, 2) evaluate the data, 3) use the data as an opportunity to learn, 4) forget the rest. If you approach those conversations without ego, they are simply another opportunity to learn a new detail that may or may not be beneficial later. It can be hard to remove the emotional content from those interactions because the toxic experts want you to feel bad. Don't. Don't feel bad. Just read for the facts, observe whether they are true, learn what is to be learned, and move on, one step ahead of where you were before. Also, the word "shibboleth" is just awesome. I've always loved that word. Thanks, Ned.
An important part of expertise is knowing enough about the subject to know when the details matter and when they don't. For example, in calculus, we don't need to be ashamed if we differentiate a function without employing the epsilon-delta method. The foundations of calculus are important and interesting, but they are important (partly) because they justify the intuitive idea of the slope of a function and the usual approach to computing it, not because they supersede it.

Similarly, in analysis of algorithms, we rarely need to employ the technical definitions, because the important thing is to get the answer, not to write a proof that would satisfy your undergraduate examiner.

In this case Pyon has mistaken a proof technique for the concept. The usual meaning of "amortized cost" is exactly as Ned originally described it, namely the average cost of an operation in a lengthy sequence of operations. If you don't believe me or Ned, perhaps you might believe Robert Tarjan? "We shall adapt this term [amortize] to computational complexity, meaning by it 'to average over time' or, more precisely, 'to average the running times of operations in a sequence over the sequence.'"

Pyon is right that there are technical definitions in which some of the cost of expensive operations is imputed to the cheap operations, and in which amortized algorithms are arranged so that cheap operations come before expensive operations so that the latter are "paid off" by the extra cost imputed to the former. But the point of these definitions is to make it convenient to write proofs about the average cost of operations in particular algorithms (that is, ones in which expensive operations come after enough cheap ones). If you have an amortized algorithm in which expensive operations sometimes come before the cheap ones, then you might not be able to use these definitions to write your proof, but you'll be able to find another set of definitions that do work.

An expert knows that technical definitions are our servants, not our masters.
Good stuff, Ned. I have encountered this sort of behaviour before inside organizations — not as rude or condescending or bitter thank goodness — but the effect is the same. I guess it's impossible for some so-called experts to let go enough, to have humility enough, to help people that they regard as stupid. This is a problem because it's how they see everyone that needs their help. As far as I'm concerned, if they can't find a way to raise competency or capability in others, preferably with a smile, then whatever knowledge these people have is useless. Their knowledge might as well not exist. What a waste!

On the other hand, Ned, I use your knowledge — in the form of your software — every single day. And I've never even said Thank You. So how awesome are you?

And: Thank You!
Great article Ned. I thought the big-o notation article was really helpful and I'm nowhere close to a beginner.
intentinallyanon 7:22 PM on 3 Nov 2017
Dear Ned,
I can understand your critzism of pyon's comment. I think it is unnecessarily agressive and insulting. Yet, he is correct, and what he mentions is by no means esoteric. The right thing to do with this kind of comment is to let it pass - as hard as it may be, but remember not to feed the trolls - validate the actual critizism, and update your post in case you determine something you said was inaccurate. In the end your goal is to teach someone something, and that better be correct, right? So keep up the good work, and eyes on the target - teaching people, not fighting trolls.
Best, Michael
Eddie and "intentinallyanon": your advice is good. I would take new information, from any source, and update my post to make it the best it can be. In this case, the post needed no updating.
So, you actually think saying that things that have logarithmic complexity have constant complexity is "the best it can be"? I don't believe you.

The problem with mathematical notation is, it speaks about an idealized world. In an idealized world, many technical details disappear, while some others, that we wouldn't care about in the real world, appear. Thinking that we can dismiss the technical details of both kinds at once is just wishful thinking.
@Veky: I don't know what you are talking about. If you have a specific critique of the Big-O piece, make a comment there, with details.
This article is toxic.

It's 2017, and the biggest problem regarding "experts" is not their toxicity, it's that nobody ever takes them seriously because everyone thinks they know better than everyone they meet, until they've got years of first hand experience with those people and piles of credentials.

The fact is most people don't know shit about anything. Most people, if they're good at anything, are good at one thing. Maybe two things. Because it takes decades to get good at anything.

Maybe the perceived "toxicity level" of experts on various subjects is the direct result of years of not being taken seriously by people who have way less knowledge on their expertise than they do. Frankly, after years of being the smartest person on a particular subject, everyone else can forgive some arrogance.

You would probably describe me as toxic. I didn't used to be, but I just don't give a **** anymore what people who don't know anything about my field think of my attitude. If you want me to be nice to you while I'm sharing the wisdom of decades of life experience, then you need to pay me an exceptional salary, which basically nobody is ever willing to do.

Toxic people are a real problem. Your article misses the point.
@simba absolute rubbish! The real experts are the ones who have a modicum of emotional intelligence. Skill doesn't have to be at the expense of empathy.
Ned, thank you for standing up to the behaviour that would not be acceptable in any normal social situation without the benefit of anonymity.

The post on Big-O notation was great because it thought me how to speak about it to people that don't don't share the same technical background as I do.

I learnt a lot from following your blog over the years. But never even said how grateful I have been for these lessons - thank you!
To be clear up front on where I stand: I find Pyon’s argument misguided in terms of how to teach a subject, and their conduct inappropriate. And on the whole, I agree with what you’re saying about people who act this way.

However, I am uneasy about this part of your message:

Every one of these topics will be useful. But they are not essential. You can do good work without them.

Yes, you can. A lot of good work. But you will sometimes do bad work without them, too. That’s the whole reason they’re worth knowing in the first place, after all.

A lot of the time it will not even matter that your work is “bad” by some purist measure, because we write code as a means to an end, and if bad code serves its end well, who cares what it looks like. But not all code is throwaway, and writing code badly in crucial infrastructure projects has huge and often entirely unperceived ripple effects.

So I am very uncomfortable with a purely inclusivity-based view on this issue.

Ironically, a lot of toxic experts are worse at maintaining critical infrastructure, for an inverse version of the exact reason that beginners are bad at it. Neither group understand the impact they’re inadvertently having. The toxic expert is likely to be so focused on the correctness of the code that he discounts the downstream impact of changes that make it “more correct”, and when pointed out, is likely to dismiss the impact with a variation on “well they should’ve known better than to rely on that behaviour”. Never mind that that’s demonstrably not how anyone’s use of upstream code works, including these toxic experts themselves.

But please don’t label someone a toxic expert just because you see them yelling at someone, sometimes even if it seems intended to drive the target away. Of course they should ideally not be yelling. But if your engineer is yelling at you because you are trying to make a change to the plans for a bridge that will make it structurally unsound… then the problem is you making the change, and his yelling is a distant tangential problem. (Also, consider that he may be yelling because he was reduced to it. Sure he should have better ways of conveying himself, but it’s not quite fair to expect immaculate emotional intelligence as the minimum bar before someone is allowed to point out negative consequences of someone else’s technical decisions.)

As Dan Luu writes:

In most company cultures, people feel weird about giving feedback. Everyone has stories about a project that lingered on for months after it should have been terminated because no one was willing to offer explicit feedback. This is a problem even when cultures discourage meanness and encourage feedback: cultures of niceness seem to have as many issues around speaking up as cultures of meanness, if not more. In some places, people are afraid to speak up because they’ll get attacked by someone mean. In others, they’re afraid because they’ll be branded as mean. It’s a hard problem.

I don’t really have a takeaway. I just worry about the rise of people-over-code absolutism, so I guess you get to hear about it, even though the interaction you’re talking about is a very different case, and one where I have no quibbles with your view on it. Most people spend their entire programming life without ever once working on projects that rise to the “engineer yelling about collapsing bridges” metaphor, and it doesn’t make them any less than either, so on the whole, inclusivity takes precedence. Am I arguing for humility from within rather than shame from without, maybe?

A hard problem indeed.
Xavier Combelle 8:18 PM on 12 Nov 2017
after reading the original debate, I was very surprised by the pyon affirmation that I never heard before and as such as jumped into my Cormen (Introduction to algorithms) and was writing here my discover when I go back in your comments where I saw Gareth Reese saying better what I intend to sa

For sure pyon should go back to textbooks instead of giving wrong (in every way possible) comments on the internet.
This is a great article and philosophy, and I'm very sorry to see that some other commenters missed the point.
hi ya'll!

Some people are offened by everything. I won't walk on egg shells.
Some people drain the happpiness from life of those around them.....
Some people do not want knowledge, or improvement.
Good, intelligent people also face burnout.
A sick or hurting person may temporarily look like a toxic person because of the pain they are going through.
Details are necessary. Mastery is a point. Then it is pendactic.
Different personalities may have problems deciding where the line is between mastery and being pendactic.
Some gifted people cannot comprehend why others cannot easily grasp complex ideas like they can.......give them space to work on their emphathy.
Then there are those who need an attitude adjustment......but we all have at one time or another.

Some people only learn the hard way. Think about it.
The thing is, Pyon was correct in spite of being a dick.

Your post wasn't simply missing an irrelevant detail that most people never need to know. Your post wasn't insufficiently exhaustive.

It was just plain wrong. Amortized complexity is a pointwise bound on mean(cost[0:i]) for every i, not mean(cost[0:n]) for one specific large n. If you want to avoid misleading beginners about amortized complexity, this does need to be included.

It's ok to be wrong sometimes - I've certainly made mistakes on my blog. I fix them, thank the person who corrected me and move on. But it's not useful to pretend the mistake was a mostly irrelevant detail.

But this kind of illustrates a very important fact; sometimes the assholes are right. If we ignore them or try to shut them out because we dislike their personality, we lose their knowledge. If you're making CRUD apps that's fine, but if you're working on hard problems that can be a big mistake.
@Chris: I'm not sure why people don't put comments about the big-O piece on the Big-O page.

I remain unconvinced about the need to discuss the ordering of operations. The point I was making was that amortized complexity means that individual operations can take very different amounts of time. This is true. The resulting complexity is the average of the operations over time. This is also true.

There are dozens of pages of details about how to analyze and design algorithms that I had to omit. How the operations have to be ordered was one of those details that I omitted. Full disclosure: it's a detail I didn't even know, and I am a successful software engineer with a long career, which I think is good evidence that it is not a critical detail in an introductory piece.

There was nothing incorrect in my article. There were extra details that were not covered. I stand by it.
@Chris: I'm not sure why people don't put comments about the big-O piece on the Big-O page.

I remain unconvinced about the need to discuss the ordering of operations. The point I was making was that amortized complexity means that individual operations can take very different amounts of time. This is true. The resulting complexity is the average of the operations over time. This is also true.

There are dozens of pages of details about how to analyze and design algorithms that I had to omit. How the operations have to be ordered was one of those details that I omitted. Full disclosure: it's a detail I didn't even know, and I am a successful software engineer with a long career, which I think is good evidence that it is not a critical detail in an introductory piece.

There was nothing incorrect in my article. There were extra details that were not covered. I stand by it.

Your example seems odd: when n=2, all sorts of guarantees are gone. When considering complexity, we discard the lower-order terms, and ignore constants, but we can only do that because we are computing the limit as n grows arbitrarily large. Constraining yourself to n=2 means that you aren't talking about a limit any more, which means all of the analysis is out the window.
Sylvain Galineau 7:41 PM on 15 Nov 2017
Thank you.

Pretty sure I played toxic expert too many times in my earlier days. And it really bugs me because if there is a reason I get tired of the industry now, this is a big one. (And a major reason, incidentally, I do not miss the web standards community). Minimum Necessary Expertise in any given conversation might well be at least as important a skill as
figuring out the MVP. I read somewhere that some folks have renamed “soft skills” to “real skills” - soft being weak/pejorative, especially among men - and as dorky as it may sound I suspect they are quite right.
I've been consistently impressed for a long time by how positive and communicative the assistance is in #python at all levels. Some other channels have sometimes rude and unhelpful "support" -- especially, I think, the more low-level or systems-related channels.

Do you have any idea what might explain this difference? Do you and other senior #python people actively try to counteract rude help? Or is it somehow a consequence of #python being larger? Or something about the Python community or language style in general?
@Wasy: We try hard to keep #python friendly and helpful. It isn't always, but we don't start with the assumption that IRC will be a bunch of rude neckbeards. There's a conscious effort to think about the tone of the room. I think Python in general has been ahead of the curve on valuing people and community in addition to valuing technology.
This is a great post. Luckily I don’t come across this kind of attitude much, but there’s several more examples in the comments here.

It’s great that software is a meritocracy, but it seems that some people believe that knowledge allows them to behave in unpleasant ways, and there is no justification for that.

Pyon’s point may or may not be true, but saying you should be ashamed is utterly bizarre and he definitely should be.
I grew up with a learning disability in mathematics (dyscalcula) and the written language (poor fine motor skills). My spelling ability was average - and reading ability above average - but my hand locks up when writing and my hand writing is very poor.

I now have a degree in electronics and work on projects with budgets that, lets just say, are “staggering” and help keep the world secure. I’m considered a bit of an “expert” by some that I work with in some regards - but it was a hell of a climb to get here, seeing how electronics and math are kind of “closely tied” :)

There are toxic experts abound where I work, neck beards or whatever you want to call them that spit fire and fury if you dare say something that they consider wrong, or try to one up you by throwing in a bunch of fancy jargon to make themselves look smarter than you if you step into their traps.

I notice on a lot of programming forums that these same people exist - born from the womb, walking, talking, calculating and dissecting the world algorithmically - poor things never had mothers or fathers to raise them - taught themselves to read and write and all about differential equations - all while making Einstein look like a simple fool (can you believe he said: “never remember anything you can’t look up in a book?!” Psh!). They don’t have time to be nice, or for relationships! Bah! They are making the world turn, sitting in forums - online - making sure the wheels stay turning and we simpletons down here don’t hurt ourselves...

As much as they make me pity their loathsome existence, having to carry the world on their bloated brains, I just remember that I have an awesome job, an awesome house, a beautiful wife and family that loves me - and most of all, I didn’t have to make it here all alone - I was lucky enough to have some people who were *just* smart enough to help me make it (their brains weren’t as bloated, which I guess helps with the social skills, perhaps?) - and I thank them every day ;)

Add a comment:

Ignore this:
Leave this empty:
Name is required. Either email or web are required. Email won't be displayed and I won't spam you. Your web site won't be indexed by search engines.
Don't put anything here:
Leave this empty:
Comment text is Markdown.