I have seen this fail (once myself and once through coworker experiences) as well, IMO the best approach is to have as little shared infrastructure between teams as possible even if it means more work in the end

That's the route to shipping features faster for sure, but it costs more, requires duplication of effort, and is really painful when you discover two teams built the same thing and you want to unify on one version rather than paying to run and maintain both. Plus, when the cost cutting comes in the bad times, you find whole teams are axed because you don't need two sets of people doing the same job. The remaining engineers then have to unify the features but without the resources to get the work done and without access to the institutional knowledge required to understand the version they didn't build themselves.

If you're an engineer in a FAANG it's not a bad solution to the shipping issue. If you're anywhere else it's a road to absolute chaos.

In my experience shared infrastructure is less prone to breakage on reorgs because "platform" teams usually get the axe first as they are not directly delivering value (from upper management perspective at least)

IMO each team should maintain their own infra and tools and be free to choose what works best for them within reason, a few exceptions need to be made of course like having unified authorization and the like, infosec is usually better to be a separate team consulting to individual product teams, etc

Based on my experiences, the idea that a genuine bottleneck in tech is being made visible to customer division management and senior management, is generally an authentic way to ensure realism creeps back into the prioritisation and funding processes.

Either you put too many in there, and they are basically idle the time in between. Or you put a limited team, and once even a single team requests something they will have to wait for the developments, let alone 2 requests at the same time.

The OP describes an excellent solution: the team that needs the feature, pays for it in man-hours. In fact, those developers that need it, are pretty familiar with using it, so most of the time capable of doing some developments on it. They can also schedule in the work in their project. The teams that will need it afterwards will get it for free.

It's actually an excellent solution.

But When would this hypothetical occur? If the platform is new-ish, there’s always going to be more work to be done than there are people to do it so there’s no variation in workload - there’s just lots to be done.

When the platform is established and operating in self-service mode, so think Amazon S3, were in a mode where we’re really not expecting features to be added, more about bugfixes and operational capacity at that point.

>> the team that needs the feature, pays for it in man-hours

there’s no free lunch. These developers don’t really care about the platform, they have bigger fish to fry. You need to ensure that infra doesn’t become a tragedy of the commons.

This is such a situation, and it happens all the time in internal products.

Either you over invest, you under invest, or the solution proposed here: jit developments where you can allocate resources as you need them.

> These developers don’t really care about the platform

They care as much as it helps them. The overall vision is steered by the architect responible for that internal product/platform, as said by OP.

Again there’s just no free lunch here, you’ve got a generalist writing a specialist component.

In your example, how confident are you that the full stack product team developer is acutely aware of, i don’t know , let’s say accessibility practices or maybe performance tradeoffs for constructing this component?

The specialist in the platform team member knows this intimately, it’s their specialism.

But hey, if you want some experts to sit idle until some real work comes in, that's all fine by me.

Additionally, this overall means that no one can plan in advance any features that depend on the platform, since global business priorities are constantly shifting.

Not to mention the political angle, where more adept business leaders can always push through their pet platform features to the detriment of what are more obvious business goals. Or at least, it will always seem this way to teams whose requirements keep falling through the cracks.

You don’t NEED to do anything, it’s all choices.

If product A is my breadwinner, i might not even need product B to estimate workload impacts for their feature request because my business strategy might be such that i’ll be allocating the resources based on business priority of product A.

Trying to second guess this by making tech a black hole with no levers to pull by the business is fundamentally unsound.

>> where more adept business leaders can always push through their … Or at least, it will always seem this way

I mean i think you’re calling out the solution with the problem. Just strive to be a better operator, level up to their ability. Exceedingly hard to do but overall win for everyone.

And of course the choice is sometimes simple, but it's often not. It's often the case that both Product A and Product B are breadwinners. One might need a feature that can bring $1M in revenue, and take between 3-4 months, and another might have a feature that should bring in $200k in revenue, but take 0.5-1 month. Which do you prioritize? What if both must be done by some date, but for one there is a chance to push a customer to accept a later date, but it's not sure yet? What happens when it later turns out that the estimation was overly optimistic and it will take twice as long?

> Just strive to be a better operator, level up to their ability. Exceedingly hard to do but overall win for everyone.

Politics in an org is a 0-sum game. If I win, someone else loses, and then their team is going to be the one that feels frustrated, and we're back to square one.

If a platform has shared ownership, then decisions will get implemented by cohorts only thinking for themselves and thus damaging the long-term roadmap. All systems, especially complex ones, need sole owners or they will devolve into what is essentially a pyramid of doom at the product level.

Good communication across teams and good design (i.e. flexible, maintainable, extensible) tends to alleviate the tunnel vision associated with each team building what works for them.

I've had to wait weeks for one-line changes because somebody else had to do them.

That seems like an unnecessary coupling between different products.

I understand this concept of a platform team mainly within one (larger) product. But maybe it's the same thing, just on a different scale.

> What we're striving towards instead is an "inner source" model, where a platform is collaboratively developped and maintained by multiple product teams. Each product team is then empowered to build new features as needed into the platform, and others can reuse them.

I think it breaks down at a certain point of complexity. The platform itself is complex enough that it's very difficult for product teams to understand it holistically. I've seen this multiple times when the product teams were still able to make the changes as needed, but over the long term this approach created a hot mess of platform features which didn't align to each other and no single person understood.

Not sure I'd like to use a platform with a code base that has the whole organisation contributing to it. Sounds like a design-by-committee dumpster fire. Teams need ownership of their product to perform effectively. You can claim that this is collective ownership of the platform, but in reality no one will feel any responsibility for the platform when things don't work or don't align entirely with the particular thing they are currently working on.

And have you ever looked at the source code for most open source software? It's an absolute mess that requires the collective effort of large numbers of people to keep it going. It works, but it's not exactly a good or effective model for internal software.

Related to collective dev/ownership, that is a risk, I absolutely agree. That is going to be the hardest part for sure, maintaining a uniform standard and a high bar for contributions to a common platform.

Finally, I believe the general opinion is that open-source software is higher quality and especially has higher code quality than internal company software. I can't say I studied any directly comparable open-source projects VS our internal tools, but code quality definitely varies significantly between our own internal projects, with some extremely clean and well written, while others are messes of hasty bug fixes and accumulated tech debt.

A platform team is there to consolidate tooling and provide services that enable product teams to move faster.

Think of it as an internal AWS. In small companies AWS is the platform team. If you’re talking about an upstream service that your product team depends on, it’s just an internal product team that’s understaffed. Or poor organization if your product team’s domain has been split up into two teams arbitrarily based on e.g. managerial politics or front end/backend division.

The products using the shared frameworks had slightly different histories and were mostly addressed to different market segments with different conventions, but still neede plenty of common scaffolding that a platform provided. A common platform team seemed to be a natural fit, and to a great extent in worked, but the problems I was discussing earlier were also pretty constant throughout its lifetime.

Sometimes the platform team is staffed by people that have been there forever, as a sort of semi-promotion. But when they know everything, it's easy to have little interest in the difficulties of learning internal concepts: After all, the learning has already been done. This makes the tools be technically capable, and intractable. Other times, the team is easy to get along with, but what they deliver isn't very good at all, and the lack of quality is papered with social skills.

You need people capable of understanding the problem other teams have, and their architectural constraints, and deliver something that will save them time, and they'll prefer to use over some open sourced hodgepodge. They need to think of upgrade paths, or live in a low-repo environment where the platform team can upgrade things for everyone. The customer service attitude should be immaculate, as to make people be happy to ask for help, yet be so good at documentation, or at simple enough architecture, as to make that customer service load be light. Many places can't hire people that meet those kinds of profiles at all, as someone like that will basically excel in most roles in most companies. So yes, you end up with the technically capable, yet gruff guys that nobody wants to talk to: The equivalent of Seinfeld's Soup Nazi... and that's if at least they are very good.

Most team topologies will work if your company is full of empathetic heroes though, so platform teams might not even be needed if you really are that good at hiring.

What I've seen is that platform teams work well in either small scale (because focus is clear to all) and large scale (because of this internal competitive dynamic) but are extremely hard to execute between the two.

Is this any different from a product team where there's competing demands between Customer A and Customer B?

TLDR; platform teams without product team's freedom to deviate (optionality) is corrupt and can destroy a large chunk of engineering velocity.

Sorry in advanced if I state this a little bluntly, but this rhetoric of teams going off and building the same tools all over the place is in my experience not a fault of any team, it's that of the context/organisation they operate in. Framing it as such is IMO destructive for company culture. As if product teams, left to their own, will just degrade into building sub-par platform-esk tools that ruin the org. I don't buy it. Trying to make one thing do two things (the single source of tech complexity) is a trap often catching platform engineers who try to tailor to an audience that is not unified in their needs, still platform teams are incentivised to create just that.

> Platform teams, however, need to build their services as products themselves, with a deep understanding of their customer's needs.

If there is no malfunction like a monopoly or a scam, as a customer I just choose a different product if one is not meeting my needs. The same must be true of a platform, and if it happens a lot that its customers are choosing something else then its time for some hard reflection. This is not just an ideal, but a hard requirement, and something that a lot of orgs just don't have.

So, what do I need as a developer? I can make excellent use of open source libraries within even a couple of minutes, but somehow for my enterprise platform I need to fire up a request and wait for weeks or months to even get to play with it. When I need a tiny little change I can't do anything myself, I need to request it and it is filed in the backlog. The same change, would I have composed it out of open source libraries or had cloud access myself, I could make in minutes. Now it takes days, weeks and I even experienced many months of waiting for a simple task I can do in 5 minutes. Thus, making use of the platform meant fighting for higher cloud privileges so we could do it ourselves and ship at least within a couple of month instead of a year, and dancing around bizarre compliance regulations, mostly in order to either evade or pleasing a chain of risk owners to satisfy audits required for certification. At some point we became almost incapable of shipping.

Not every platform team is as kafkaesque as this though, and it doesn't need to be.

We tried inner sourcing as well and it was quite hard honestly, because each team was just focused on their own goals and contributing to the shared platform or libraries was often an extra investment that defacto penalized their achievements - which _did_ have repercussions on their individual performance reviews. Furthermore, it was done in such an ad-hoc way that it was quite hard to get something sane off the ground. I think you do need a kind of dedicated ownership.

Best experience was in a team where we did everything ourselves and had the required expertise. Second-best was in really close collaboration with a platform team where we also had members going from one team to the other. But even that team failed to build the features we really needed as a product team.

Open source is a good model, and there is one essential thing that platform teams need to provide their users that open source has: autonomy. I have actually never seen this in a platform team in the org I am talking about.

I can use a library, drop it, change it or exchange it for another one. The same _must_ be possible with a platform. It needs to be something that helps its users, not limits them in any way, and it can't ever be rammed down their throats. If your platform doesn't work for me, I should have the autonomy to just use a third party to get the job done. If your magic abstraction over AWS doesn't work, give me an AWS account with admin access and I do it myself. Bonus points if its not an all-or-nothing and I can just use what works and build the rest myself.

If you don't have time to build feature B because feature A is more important, everything I need to build feature B myself should be readily available. For example, I must be able to just fork the platform, build feature B, and when done 'upstream' it to the platform again.