A while back I ranted on Design Meetings are A Waste of Time. As luck would have it, I just had an opportunity to put my money where my mouth is. I had a new design idea that touched many areas of my company. We have a server side group and a client side and subgroups within each. Following my tips, I was able to have very substantive design meetings without bikeshed arguments and derailments.
In analyzing this, I derived the follwing formula:
Pb = Nϕ
where
Pb = Probability of a Bikeshed meeting
N = Number of people at the meeting - meeting organizer
ϕ = Frequency of bikeshed meetings at your company
= # developers who think they’re experts / # of actual experts
In other words, keep the N low. In doing so I found that I was able to isolate the bikeshedders. This way they could only derail a minimal amount of meeting time.
Not to toot my own horn but…it worked. I got very constructive feedback from the actual company experts and marginalized the bikshedders. I am now proceeding with minimal interruption.
Here’s a few reasons why:
The Bikeshed
One common mistake is to not want to leave anyone out of the meeting. Especially if a team is tasked with designing something that affects many groups in the company. They’ll want at least one representative from all those groups if not more. Granted, these people are smart, but like most smart developers they can’t resist helping build that bikeshed. I’ve seen a meeting lost after the first bullet point.
Tip: no more than 5 people at a meeting.
Presumptious
The typical design review meeting goes like this: a team spends weeks engineering a design (or at least a strawman for one), then, they invite other team members to a review meeting giving them at most a few days to prepare. What usually happens is that the other experts will notice things that aren’t quite right with the presented design. When these items are shared, the immediate response is “well, how would you fix it?” It is presumptious to believe these people can fix your design when they have spent but a fraction of the time you’ve spent on it. Now, of course, this implies that the design wasn’t given to a team of junior engineers where the faults and fixes are obvious. I’ve seen some common notes on e.g., scaling issues, security concerns, and performance problems that are brought up by reviewers. What makes the meeting a waste of time is that when an immediate, all encompassing solution is not offered with the critique, it is ignored. Therefore, why have this meeting in the first place.
Tip: Invite only experts whose opinion you value. Get as much detail from their notes as possible (I’m not saying to not ask follow up questions) then take their notes and solve the problem yourself or prove they are incorrect.
Don’t Invite the CTO
This will surely cause the meeting to become a bikeshed argument. After all, what CTO or upper manager could not offer something during a design meeting? What they should do is listen, keep the meeting on track if it starts to stray, and only, only, offer comments where the design is not in sync with company goals. That last bit would almost never happen. So, again, why are they at the meeting?
Tip: don’t invite micromanagement — if you’re CTO is anything like mine you’re meeting will get so side tracked with his attempts at faux-arbitration and “summing up” that you’ll be forced to call a smaller meeting anyway.
Now, I’m not actually discouraging design reviews. I think these massive meetings are pointless. My best advice is to gather small groups together and do multiple meetings. Try to choose the attendees judiciously (for instance don’t invite the guy who hates static polymorphism and the guy who hates runtime polymorphism to the same meeting) so that you get the best bang for your buck.
This is a, unfortunately, a classic mistake. It is illogical, but I’ve seen it happen so many times.
Step 1: Create a project where a new design is needed from scratch.
Step 2: Management puts mid or low-level engineers in charge of the design.
Step 3: The senior engineers are left scratching their heads.
Step 4: The project a) fails or b) gets re-designed by senior engineers making redundant months of work
I have two recent examples. First, our company wanted to design a new proprietary communication protocol api. Essentially replacing an existent client/server API that is big & bulky, difficult (but not impossible) to extend, and slow in some fast path areas. This included the client and server APIs as well as the protocol itself. Sounds big, right? This would touch every application group in the company. So why not get some mid-level engineers to work on it! That’s just what happened.
They came up with a design that neither solved any of the issues with the existing API nor any compelling rationale to the design flaws senior engineers later discovered. After a couple months of design work the team held a design review meeting with the senior engineers company wide. It was not pretty. Their design was confusing, immature, and, most importantly, failed to solve any of the problems with the current API. The seniors emplored management to either have the team cut back their focus — they could have worked on the three subproblems (client, server, and protocol) independently, fix the existing API by evolution rather than revolution, or turn over the reigns to engineers who had more experience.
These suggestions were not heeded. 7 months of wasted work go by and the team finally realizes they can’t fulfill the design implemenation. They decided to kill the project and go the evolution route. After that meeting, the senior engineers were left shaking their collective heads. I heard many a “didn’t we say that months ago?” in the halls.
The second example is a team gathered to design an automated testing framework. For some reason, our company had never gotten automated testing going before. The need was suddenly deemed from on high that this was necessary for every product before any new releases….go!
The team put together was several junior software engineers and (only somewhat technical) quality engineers. Sounds like a disaster and it was. Similarly to our communication API example, designs were done and meetings were had. Flaws were pointed out — too many layers, inconsistent, nonsensical, not extensible, too much abstraction, you name the design mistake it was in there.
8 months went by and the project never really got anywhere. It was started to be redesigned by senior engineers that just got so frustrated they weren’t being listened to. I left before I could find out how the two approaches panned out in the end.
In reviewing those two examples, I’ve have theories as to how they occured and how mid-sized software companies can avoid these mistakes. In the communications API example, the mid-level engineers chosen were held in most-favored engineer status by management. In reality (obvious to all the seniors), these engineers were hard workers but simply lacked good design skills and experience with from-scratch implementation.
The testing framework API example happened because management heard “scripting language” and took that to mean “not as difficult as real code.” In reality it it not the language but the scope and breadth of the effort. This framework required real design skill as it had to work for many use cases across multiple products. Most importantly, though, it touched every single product.
It may very well be boiled down to underestimation in the effort and overestimation of the design teams skills. The lesson here is to always assign far-reaching or complex design roles to your seniors. For less complex designs, don’t let the juniors fly blind. Regardless of the design scope they will most likely need senior guidance.
One more lesson is that engineers love design so if you think a senior wouldn’t be interested because it’s “a scripting language” think again…at least, ask them first.
Ah, management. The perennial scapegoats, right?
Well, sometimes that’s the case — however, there are crucial times when their misleadership can cause the loss of great engineers or bring down a mid-size company altogether.
Case in point. A former company of mine got two good ideas and screwed them up royally. Because of management. Frankly, they didn’t have a clue what they were doing.
The first idea that was hatched was “Process.” That scary beast that can rip apart engineer morale and productivity. Done properly, it can enhance productivity (add other -abilities all you like, but that’s what it comes down to). More on better process in later posts.
So, managers get the idea that “process” needs to happen. It must be done — clearly they’ve been losing money this whole time. An immediate problem, became clear to this engineer, was that none of the managers had the faintest experience in actual process. There were plenty of senior engineers including myself that had gone through the most rigorous CMM process at other companies but our experience was politely decline when advice was offered.
That left either implementing a known process like CMM or Agile poorly — or — to roll your own. And, the latter is just what they did. I think that is slightly worse although both choices are evil. Upper management gathered business analysts and project managers to lead the company process in a big blind collective. Not a thought was given to development management or senior engineers with actual experience in this area, metrics to measure the new process’s effectiveness, what tools to use, what engineering group should implement the process, etc, etc, etc.
As expected (to an experienced engineer) the result was a total disaster. The first project onto which this process was introduced was a soaring failure. The process delayed the project by over a year. Not kidding, no hyperbole. Somehow, since no metrics were taken, this was taken as such a success that it should be rolled out to other projects. As of yet, it has been only a hindrance and annoyance to other projects. The bettter engineers learned to work around it, but projects mis-staffed with all junior engineers have seen very long delays.
The second good idea gone wrong is automated testing. Upper management got the good idea that all this manual testing (by a less than competent SQE staff) was delaying product ship dates by months. Obviously, this was untenable and should be replaced by machine testing where applicable. Sounds good so far, right?
Again, the lack of experience or judgement comes into play and management manages to ruin this initiative. Two things to know — 1) management has no experience implementing an automated testing initiative and 2) management had hired many (~80%) non-technical manual testers. These facts did not deter them. They decided the way to get testing automated was to make the cardinal sin of having developers write their own tests. (See another detailed post on why this is a horrible idea.) And the icing to this cake — they decided to take a group of very junior engineers and have them be the designers and deciders for the automated testing framework. The senior engineers saw this framework, voiced their concern, and were asked to move along.
Surely, they were keeping metrics to see how many fewer bugs made it into the field and how much faster releases were going, right? See above process paragraphs. And, what to do with all those manual testers? In their infinite wisdom, decided to “train” them to program. Better than that, the engineers were still responsible for getting their projects tested and shipped on time. So not only do engineers have to write tests themselves, but they have to put up with poorly trained manual testers mucking with programming. This resulted in a quagmire of epic proportions. I left before seeing how it ever resolved itself - but it didn’t look good.
The moral of this story to upper management is that if you don’t have experience, consult engineers on your team who do before mandating and implementing something nonsensical. Once you’ve decided that something must be done, appoint an experience team to implement it. Don’t blindly lead by appointing other blind managers to head it up.
Bug: ints in C++ as compiled by MSVC on 32 bit platforms can only be 32 bits.
This seems to be the amount of reasonability I’m seeing lately by my SQE. Advice to mid-size software companies with SQE: rationalitly is key. “bugs” like “if i shutdown the computer, the CRT never flushes” are not bugs. Let’s be reasonable. IMHO SQE should understand the paradigm. How do computers work, what are the basics behind<insert your language here> programming, how does your architecture work.
I’ve found this, surprisingly, as a major source of bugs. I’m constantly getting emails about “bugs” that are clearly unreasonable. It seems, simply, to make the SQE role seem important.
Some advice — software is not perfect. It is not built to adapt to every range condition, platform, or permutation. It is expected to reasonably work on the platform it is offered. And nothing more. SQE would be wise to heed this advice. And your company would get products out faster.
I recently brought up the idea of a wiki with the VP of Engineering. I said — you know, it would be nice to have an informal area for, basically, brain dumps, info for SQE, BA’s, other developers, and support staff. A central location for info so lead developers don’t need to be bothered constantly for an answer they gave months ago to someone else.
Now, in my head I’m thinking mediawiki or something of the same ilk. Free and open source. Quick & easy — no overhead. Then I see the VPoE in a meeting. He tells me he’s not going that route but instead with a wiki that support MS office and sharepoint.
Sigh.
I was astonished by this. Not because it’s not open source or free. That would be ideal, but I’m not RMS. I’m astonished that the choice was more about working with our other superfluous, non-free internal tools than anything else.
You should know our company also spends wastes money on Sharepoint. So to this VPoE, naturally, a wiki must support it. Of course, right? And by extension, it must support MSOffice. Sure!
If you’re forced to choose a wiki that supports Sharepoint and that forces you to pay for something you could have gotten for free — that’s a problem. Sharepoint is pointless. You can get a wiki up and going for nothing. If your engineers can’t maneuver the “wiki syntax” then they shouldn’t be working for you — it’s that simple.
My point is there are some things that money can buy. A good source control system springs to mind (I’ll cover that later). But don’t waste money on anything. Even if it slightly misconforms to other company systems. In fact, if it does that you should examine those systems/platforms.
