EN 26: Innovate where needed

Hey there!

This week’s interesting links are the following:

• In Defense of Strategy

• The Ultimate Guide to Developer Counter-Productivity

• Value Engineering and Build vs Rent

The first week in my new role is almost done. It has been chaotic on a personal level, as I've been trying to adjust to it after the 8-month break. This Monday morning, planning the day in my journal, I got a strange feeling after blocking nine hours of the day and seeing how many blocks in the calendar were left.

Innovation is mostly referred as a good thing. We must wow our customers with new, better, delightful, more useful things or invest in new tools, patterns, libraries, etc. for enhanced productivity. Little do we talk about wasteful innovation and its cost and associated risk to the company, and in particular, wasteful innovation coming from engineers going wild.

I’ve already seen this a few times. A potential signal to identify wasteful innovation done by engineers is hearing the following words: “it’s more efficient”. If these magical words are used to justify suspicious architecture or technology choices, as a rough heuristic, that decision is wrong:

• “I built a new renderer because React was too slow, this is more efficient”

• “We should create an event-driven architecture from scratch with sockets, it’s more efficient”

Smaller decisions with similar justifications left without control, while apparently inoffensive, can still come to bite us in the future. It’s bad tech debt, we’re getting loans after loans that we won’t be able to pay, as there are no benefits to reap with our efforts. Regardless of the size of the change, these innovations will only sabotage the business and ourselves.

The curious thing is that some of these decisions happened under the business nose. There were not enough people with technical expertise to challenge or discuss these approaches, and there was no other choice but to trust them. Over time, the trust was eroded, when results slowed down, bugs grew… Was it malicious intent from the devs? I don’t think so, I haven’t met any developer, yet that was actively evil or consciously sabotaging the company.

A few reasons come to mind for the wasteful innovation from engineers to happen:

• Not enough technical leadership and expertise

• Engineering siloed and disconnected from the business and users

• Burned out, defeated people

Not enough technical leadership and expertise

This point has a strong impact on early startups, where none of the founders are technical, the technical founder or CTO doesn’t have enough experience (the typical CTO with 2 years of experience or no breath of experience).

Another typical way of not getting enough technical expertise in the company is to only hire junior developers with no senior engineers to provide guidance, support and expertise, specially on the architecture and engineering culture.

Engineering siloed and disconnected from the business and users

The farther the engineers are from the business impact and the users, the longer the feedback loop is and the less they consider the business and user outcomes in their decisions.

A typical flow has developers involved in the process when there’s already a solution. They don’t participate in the problem space at all, don’t join user interviews or test assumptions with the rest of the team, they act as builders. In essence, in this flow, developers join too late, handoffs are frequent and often there will be high work in progress. Even if they’re following “agile methodologies”, they are only building faster within a larger waterfall.

At the same time, engineers will be organized in engineering teams or worst, front end and back end teams instead of being in teams with all the skills needed (devs, PMs, designers, marketing…) organized for flow.

It can be easy as a developer in a system that only wants you to be in a silo building code without worrying about anything else to forget about the implications of your decisions. When you see a direct connection between the software and users and the business, decisions will be more conscious and aligned. This same argument can be used to advocate for developers to maintain their systems in production (you build it, you run it).

Burned out, defeated people

In the end, we’re humans. If developers are burned out or defeated because things don’t change, they aren’t heard, they’re overworked, etc. they can get into a situation where they don’t care any more and quit quietly to protect themselves. In this situation, some devs can seek solace in going overboard with technical solutions to immerse in the problem-solving.

Innovate in core domains and for competitive advantages

As a general rule, we want to innovate in those areas that are fundamental for the business and provide a competitive advantage. In Domain-Driven Design, these areas would be Core Domains, the domains that the business is all about.

If the business is all about family trees and users can create their trees, see records of their ancestors… why would we spend significant time and money on creating a bespoke subscription system from scratch?

Going a bit low level, unless your business is into building the next React competitor and making money from it, why would you want to build something similar from scratch?

Unless there’s a competitive advantage that justifies an investment in innovation, we should think twice about going wild. In essence, in this section I’m talking about the build vs buy decision, which is covered in one of the articles of today: Value Engineering and Build vs. Rent.

Innovation points

Lastly, there’s the potential waste of innovation of choosing radically new or unfamiliar tech. There was an article I read years ago that mentioned the concept of innovation points. The idea is that we only have a certain number of points to innovate in the tech stack, and that we should be spending them carefully and consciously.

If we were to create a startup or replace our tech stack with all the new shiny technology and patterns available we could have a few problems: unfamiliarity, massive learning curve, lack of expertise, too many changes at once, increased risk, smaller pool of developers for hire…

In a similar vein to the famous “keep it simple” mantra, I would add “keep it (mostly) boring”. Keep it as boring as possible, and use your innovation points to invest in tech that will give a significant improvement to the systems. Remember that, while we go wild with the tech, there are still business and user outcomes that we have to think of. Personally, I would rather spend my time on the latter than the former, without the extra cognitive load. This doesn’t mean that we can’t use new languages or tech, just that we should be mindful of our choices and where to invest.