In agile methods, the effort of a feature is estimated as a story during planning meetings to decide whether a story should be implemented. This is optional: NoEstimates describes projects that work entirely without effort estimates. The advantage: The effort spent on estimating is eliminated and can instead be used to develop stories. It is enough prioritization if everyone on the team works on the currently most valuable feature. Woody Zuill is a pioneer in this area and has never worked any other way.
Is it enough prioritization if everyone on the team works on the currently most valuable feature?
Estimating stories is usually based on a reference story or functionality that is assigned a size of one. Other stories are estimated relative to this reference, by and for the team that is doing the estimating. In addition, there is the team’s velocity (speed). It is measured in implemented story points per iteration and can change over the course of a project.
So the estimate is not objective, because it involves the specific team and because velocity changes over time. However, the estimation methodology achieves its goal: it allows teams to plan their work. Through estimation relative to a reference story and through velocity, it is also possible to make a reasonably good estimate of what a team can deliver.
Of course, one can try to make these measurements more objective. For example, a uniform reference story could be chosen. There are approaches that go much further. The Function Point method attempts to measure the objective complexity of a business requirement in function points. But these methods also show variability in estimates. In addition, they require experience in estimating and are labor-intensive. Methods such as COCOMO even warn that their results provide only rough numbers, not precise estimates.
The usefulness of such estimates is questionable: projects always change their scope. When software is used in practice, new requirements emerge. Software development is a process in which engineers and users learn together how best to support business processes with software. When you learn something new, you change the scope. Then precise estimates for the original scope are no longer worth much, and the effort spent on additional precision is not well invested. Perhaps this is why agile estimation is used in practice, but more sophisticated methods rarely are.
When software is used in practice, new requirements emerge.
Overall, the focus on effort in projects is often exaggerated. Like any investment, an investment in software must generate value that exceeds the investment — ideally by a large margin. Estimating value, however, often seems to be treated as a secondary concern compared to estimating effort.
There are very small teams that have developed extremely successful software. Such outliers raise the question of whether objective estimation is even possible. Before its acquisition, Instagram was valued at one billion dollars, operated a globally scaled system, and did so with only 13 employees. Before its acquisition, WhatsApp had 900 million users; 50 engineers developed and operated this system. That is a fraction of the staff involved in many IT projects at established companies.
These applications appear to be “just” a simple photo-sharing app or a messaging app. Does that explain the low staffing numbers? Insurance products or savings plans seem also simple: you pay money and under certain conditions, you get money back. Search engines are also simple: there is even off-the-shelf software to index and search documents.
On a superficial level, everything appears simple. The complexity of a problem only becomes clear when you look at the details and truly understand the problem.
The complexity of a problem only becomes clear when you look at the details and truly understand the problem.
Regardless of complexity, the economic success of a project is the most important outcome. In this respect, Instagram and WhatsApp are truly convincing: they created billions in value. That is almost only possible for successful start-ups. But for every successful start-up, there are many that are not successful.
Let’s assume an established company were to build a photo-sharing app or a messaging app. It would build it using its typical mechanisms: potentially hundreds of developers, sophisticated project management, and detailed project plans. Thirteen people developing a central system for a market breakthrough would more likely be perceived as a risk than great value for the money. Large teams and projects also bring a lot of prestige — for everyone involved, for managers, but also for engineers. Finally, established companies usually have many software projects, and the survival of the company rarely depends on any single one — unlike a start-up. The pressure is therefore objectively lower at established companies.
There are thus lots of mechanisms that lead to a completely different level of effort. An established company will develop a software system using the mechanisms it typically employs and is in a different competitive situation than a start-up. In a start-up, on the other hand, the economic conditions are such that something has to go live as quickly as possible, maybe even at all costs — with corresponding consequences.
It is also unlikely that an established company would build exactly the same application for a problem such as photo sharing or messaging. The IT landscape into which the application must integrate is different. Not really necessary requirements might be added — something a start-up often cannot afford due to its economic constraints.
A start-up would therefore likely develop a much simpler solution even in areas that are traditionally implemented with complex software systems — simply because it needs to get to market quickly and has a very limited budget.
So trying to assess an objective effort makes even less sense: software solves a problem that an organization has. For example, the organization wants to establish a new product. To do so, it relies on the social processes embedded in the organization. For a start-up, the approach of an established company is just as alien as the approach of a start-up is to an established company. This includes not only the development process, but also the product itself.
In the end, the objective effort should not really concern us too much. What matters is how we can develop better software in a given situation. And for that, there are always many exciting possibilities.
Determining the objective effort for a feature may be possible, but only with great effort. Since it is unnecessary and costly for planning, most projects do not use such techniques at all. Software is also “just” an implementation of the solution to a business problem. Organizations approach problems differently, so the solution and the effort will differ for that reason alone. Nevertheless, one can and should ask questions about productivity and better ways of working.
This is a translation of my German blog post at heise Developer.
]]>Cohesion and loose coupling are therefore related and mutually dependent.
For example, assume that a specific product can be reserved, then ordered, and finally delivered. We are talking about a concrete product — for instance, a specific used laptop that exists only once in stock with exactly this configuration. The functionalities ordering, reserving, and delivering are candidates for a shared implementation in one microservice. They could share a data model in which the state of a product (ordered, reserved, delivered) is stored. If one of these functionalities is implemented somewhere else, this model is torn apart. The status would then have to be available in two places. As a result, the modules would have to interact a lot, because the product’s state would have to be kept consistent in both places. In addition, changes might require modifying both places—and then the two microservices would no longer be loosely coupled. Cohesion and loose coupling are therefore related and mutually dependent.
This way of thinking in terms of models and shared functionality is central to good modularization.
This way of thinking in terms of models and shared functionality is central to good modularization. Unfortunately, this does not seem to be clear to all software developers and architects, so in practice models that actually belong together are sometimes split apart, resulting in poor modularizations.
As so often, there are gray areas here. The concrete approach depends primarily on the specific domain. That ordering, reserving, and delivering are so closely related that they must share a data model is a result of the specific domain. If, for example, not a specific laptop is reserved and ordered, but merely some laptop that meets certain specifications, the modeling can look different. Then it may be sufficient to reconcile inventory levels to avoid reserving and ordering too many laptops with certain specifications. This difference is a consequence of the business model: for used laptops there may be only one specific item with a given configuration, while for new laptops there may be a stock of identical products.
Delivering the laptop means that no other customer can reserve or order it. But delivery must also is have logic to actually ship the laptop, which may be better placed in another module. In other words, the domain concept of “delivery” can be represented in different models across different modules: one module that models the state of the product (reserved, delivered …), and another that actually does the actually shipment process.
Code duplication might be an indicator of weak cohesion. This makes sense: when code is duplicated, the same or almost the same logic is implemented in two places, even though it should really exist in only one place. Something has been “torn apart” that probably belongs together. DRY (Don’t Repeat Yourself) can be a countermeasure. DRY says code should never be duplicated; instead, code should always be adapted to be flexible enough to cover multiple cases. For example, duplication of a class can be avoided through a superclass and inheritance, or duplication of a method through an additional parameter or a conditional branch.
But the example of data modeling for laptops is a case where DRY and avoiding code duplication would not have solved the problem. One can implement ordering and reserving in two separate microservices and thus achieve no cohesion, while still adhering to DRY. DRY alone is therefore not sufficient as a concept for achieving cohesion.
Code duplication violates cohesion but can create loose coupling.
Interestingly, code duplication violates cohesion but can create loose coupling: When code is duplicated, the two copies can be evolved independently. For example, if a specific software solution for one customer is to be turned into an industry solution, one can copy the specific solution for each customer and adapt it. This is very easy to implement and enables completely separate development, making customer-specific changes easy. The changes are thus completely decoupled — more than just loosely coupled. On the other hand, copying code also means that a separate, specific solution emerges for each customer, making it nearly impossible to implement features for all customers, because these features would have to be implemented in all customer-specific code bases.
Implement a generic core only after the third similar implementation?
It seems obvious that it is better to use a single code base for all customers — there are reasons why DRY an important concept. But then one has to design a reusable code base of domain logic. This is not easy at all, which is why some actually recommend implementing a generic core only after the third similar implementation. This suggests that up to a certain point, one benefits more from strict separation and the resulting loose coupling than from reuse and the tight coupling of a shared code base.
In the laptop example, the discussion revolved around functionalities such as ordering products. With the proposed decomposition, changes to this functionality would affect only one microservice. But other criteria for cohesion can also be defined: doesn’t all business logic code belong together? And the user interface code? When changes are made to the ordering process, both parts of the system are affected: a change in business logic can only be used after corresponding changes to the user interface. Nevertheless, business logic belongs together and should be separated from user interface code. So there is another level of modules and cohesion here.
Modules form a hierarchy.
In fact, modules form a hierarchy: microservices, for example, are divided into e.g. Java packages that may contain user interface or business logic. Below that are classes that contain specific parts of the business logic or user interface, and finally methods. At all these levels, things that belong together should be implemented together.
At a coarse-grained level, decomposition should follow domain concerns.
At a coarse-grained level such as microservices, decomposition should follow domain concerns; at a fine-grained level, it should follow technical aspects such as business logic and user interface. Cohesion applies to all sizes of modules such as microservices, Java packages, or classes.
There is a trivial way to always ensure that everything that belongs together is in one module: just have a single module for the complete system. For example, one microservice for an entire system, or one class for a microservice. Obviously, this is a bad idea, because the systems then become very hard to understand. So size plays a role: microservices, classes, and methods should be small enough to understand.
Cohesion also means separating what does not belong together.
So cohesion does not only mean grouping together what belongs together, but also separating what does not belong together. If unrelated domain concerns are implemented together in one microservice, it not only becomes too large, but there are also too many domain reasons to change it, turning it into a change hotspot.
Cohesion provides a good way to identify modules: if you implement things that belong together in the same module, you created at good designs. Like all other rules for designing architectures, also this one depends on the domain and the specific use case. This makes it difficult to apply in practice. However, it applies at all levels: microservices, packages, classes, and even methods. That is why it is so important to truly understand these concepts and their effects.
This is a translation of my German article at Java Magazin.
]]>These aspects of architectural work are the focus of most articles, books, and talks on the subject. However, this article takes a different angle — because even a perfect architecture is useless if it is not implemented. Only then can architects make an impact. This aspect is extremely important in practice, yet there is significantly less material available on the topic.
Architectural decisions affect the code: the code’s structure and the technologies used within it. So, one might simply sit down at the computer and implement the architectural decisions directly in code. That approach obviously works if you’re implementing a project alone. But most projects are developed by teams. You can still implement your architectural ideas yourself — but that alone isn’t enough if the other developers implement things that don’t align with your design. And since there are more of “them” than “you,” your architectural ideas will never be fully reflected in the code unless they get on board.
The real challenge is influencing the other developers.
So, the real challenge is influencing the other developers. Writing code yourself can help with that. It allows you to better understand the challenges others face, gives you firsthand experience with at least part of the technical context, and increases your credibility — since you’re not just talking abstractly about concepts, but actually applying them. Pair programming with other developers can also help bring architectural ideas to life within a project.
At this point, it should be clear what this is really about: communicating ideas and convincing developers. Some people expect there to be a kind of magic that allows them to persuade others of their views. Even if such magic existed — would you really want to convince everyone of the superiority of your own architectural ideas?
In software architecture, there are many possible options for every decision. Depending on the specific context, one or another option might make sense. Sure, you can debate endlessly about which framework is better, how APIs should be structured, or which architectural paradigms to use. But in the end, projects often fail for entirely different reasons — misunderstood requirements, communication breakdowns, or organizational challenges are common causes. And the “correct” architecture always depends on the context. In other words: many options can work; the differences between them might not be huge, and the supposedly better option can turn out to be suboptimal. Making compromises requires being willing to compromise — not clinging to a single approach in a quasi-religious way.
Therefore, an open dialogue is better. Other developers may have ideas or knowledge that lead to better decisions. It’s not about persuasion — it’s about dialogue that leads to shared, and ultimately better, decisions.
Other developers may have ideas or knowledge that lead to better decisions.
That requires everyone involved to be willing to engage in such a dialogue — not just push their own egos. Architects need to foster such conversations and deal constructively with disagreement or resistance, rather than ignoring it.
What applies to architects also applies to other technical professionals: at a certain seniority level, they must engage in discussions with others and share their knowledge and ideas. Only then can they have an impact beyond their direct personal involvement. Simply opposing others is not enough.
At first glance, architects seem to primarily talk to developers and other technical people. But software projects involve many stakeholders who define requirements and provide valuable input — so they must also be included in the dialogue.
Software projects involve many stakeholders - they must also be included in the dialogue.
In addition, management and other stakeholders often make decisions that affect projects. This includes direct architectural decisions — such as which technology to use or whether to adopt an approach like microservices. Ideally, such decisions should be made by architects, since they are the experts. Management’s involvement in these decisions is therefore somewhat contradictory — they are not the experts and should defer to those who are.
Management also makes decisions about team setup and team responsibilities. Since Conway’s Law, we know that organizational structure affects architecture. Yet such organizational decisions are often made by management without considering architectural implications.
In practice, many architects treat these decisions as fixed and unchangeable. Sure, they are made by people higher up the hierarchy — but you can still talk to those people and influence their decisions. Just as architects should engage in dialogue with developers, they should do the same with management.
Just as architects should engage in dialogue with developers, they should do the same with management.
Of course, it can be convenient to let others take responsibility for decisions. But at the very least, architects should provide feedback — especially when they have unique insight. That feedback helps improve decisions overall.
Sometimes, decisions need to be “translated”: management may not understand how organizational structures influence architecture. Likewise, management doesn’t care about the “beauty” of code, but they do care about development efficiency. As an architect, you need to perform this translation if you want your ideas to be heard.
There are thus two dimensions of impact: based on your knowledge and skills, you can design better or worse architectures — but whether they are actually implemented depends on how effectively you operate within the organization. Both dimensions must be considered when making decisions.
It can even make sense to choose the option preferred by the majority, even if it’s technically inferior to the one you would choose yourself. That option will likely receive more support. And in many cases, the different options don’t lead to dramatically better or worse outcomes. It might even be that your preferred option only seems better to you but is in fact worse. Software architecture is complex, and the consequences of decisions are often impossible to fully predict.
On the other hand, when decisions could have catastrophic consequences, you must speak up and escalate if necessary. It would be completely irresponsible to let such decisions go unchallenged. The first step is not to treat decisions — especially those made by management—as immutable truths.
It’s crucial to prioritize which decisions to “push through” or firmly resist.
Thus, it’s crucial to prioritize which decisions to “push through” or firmly resist, and which to approach by focusing on communication and collaboration.
Designing architectures and selecting technologies are prerequisites for successful architectural work — but they’re not enough. To make an impact, architects must communicate with developers. The goal isn’t necessarily to impose one’s own ideas but to engage in dialogue, question those ideas, and revise them if needed. Writing code can help, but communication skills are far more important.
The same applies when dealing with management and other stakeholders—these skills are essential for influence. Without the ability to create impact, even the best architectural designs will never be realized. That’s why communication is so important in IT. And there are, of course, many ways to improve these skills — for example, through facilitation techniques.
This is a translation of my German article at Java Magazin.
]]>This is where loose coupling comes in: if changes do not ripple through to dependent components, we speak of loose coupling. The two components may depend on each other, but a change typically affects only one component and does not touch its dependents. Some changes may still involve both components, but under loose coupling, this is the exception.
If changes do not ripple through to dependent components, we speak of loose coupling.
This property plays a crucial role: it ensures that we can modify large systems by changing just one component while largely ignoring the rest. Without loose coupling, changes become difficult because dependencies may cause effects throughout the system. In the worst case, those effects become unpredictable, and the software is practically unchangeable because the risk of unintended side effects is too great.
If loose coupling is so important, the obvious question is how to achieve it. A key concept is modularization: components expose an interface and hide their implementation. For classes, for example, the interface consists of public methods, while their implementation and private methods remain hidden. This guarantees that other classes can only depend on the interface. A change that keeps the interface stable does not technically propagate to other classes. Of course, a change in behavior may still force updates to other classes.
A key concept is modularization: components expose an interface and hide their implementation.
Modularization and loose coupling are also why you should avoid exposing instance variables directly: if you change how data is modeled in instance variables but those variables are accessed outside the class, changes will ripple outward, making the coupling less loose. An even more extreme case is sharing a database directly. At worst, you may not even know which applications are reading from or writing to the database. If you want to change the database schema, all these applications could be affected. This tight coupling is why database schemas often become practically unchangeable. The remedy is to hide the data model and allow access only through an interface.
There are many other potential dependency hotspots where changes can ripple through the system — shared data structures, for example.
So, loose coupling is achieved through modularization — the fundamental concept of software architecture. But in practice, loose coupling is so critical that we often wish for additional ways to improve it.
Some architectures introduce an extra layer to further decouple interface and implementation. In this layer, data is copied into Data Transfer Objects (DTOs), and an adapter with its own interface is implemented to hide the interface of the underlying layer. The idea is that changes to the external interface won’t “bleed through” but can be addressed by updating only the DTO and adapter. This should in theory create looser coupling since changes typically affect only this layer.
The idea is that changes to the external interface won’t “bleed through” but can be addressed by updating only the DTO and adapter.
In reality, these adapter layers often get in the way. When a change does propagate, you have to update the adapter layer in addition to the implementation. Adding a new feature to the interface always requires updating the underlying implementation as well as the adapter layer.
Subjectively, such adapter layers tend to hinder more than they help because some changes become more cumbersome. The overhead rarely seems justified by the savings in other cases. This situation occurs whenever changes commonly pass through the adapter layer — which is not unusual: new features and changes affecting more than just the interface are frequent, and these always pass through the adapter. Furthermore, the system now has an extra layer, making it harder to understand and thus harder to modify.
Subjectively, such adapter layers tend to hinder more than they help because some changes become more cumbersome.
This points to another issue with loose coupling: it is easy to look back and see whether a system was loosely coupled for past changes. But designing a system to remain loosely coupled for future changes is much harder. Good modularization will likely result in looser coupling; adapter layers, on the other hand, rarely will.
Therefore, it’s worth examining loose coupling to improve it. An important input for this is Behavioral Code Analysis. It looks at how a team interacts with and changes the software. This approach reveals which parts of the code are often changed together. You can then take steps to restructure the code so that similar changes in the future affect only a small part of the system, making them easier to implement.
In general, iterative architecture evolution is a good idea. As the project progresses, you learn which approaches work well or poorly and can adjust the architecture accordingly. Only through such adjustments do truly good architectures emerge. If you neglect iterative improvement, the system’s quality will inevitably decline.
Still, it would be nice to design a system upfront to achieve as much loose coupling as possible. As mentioned, modularization can help — but it’s also worth considering completely different strategies. Up to this point, we have mainly looked at technical measures like modularization or adapter layers. But software is usually changed or extended because of the domain.
But software is usually changed or extended because of the domain. Therefore, you shouldn’t rely solely on technical measures — you should strive to reflect the domain concepts accurately in code.
Therefore, you shouldn’t rely solely on technical measures — you should strive to reflect the domain concepts accurately in code. Then, domain-related changes become easy to implement. Misrepresentations are common here. For example, if a payment module knows too much about products, it will need to be updated whenever a new type of product is introduced. That leads to tight coupling: every change for a new product type affects payment, in addition to other modules that configure products. But this is also probably a domain modeling mistake: conceptually, payment should simply ensure a specific amount of money is received. Why should this part of the system know which product that amount applies to? That’s a question about the domain — but it can lead to loose or tight coupling, especially for critical changes to the domain logic. No technical measure — neither modularization nor adapter layers — can solve this problem. On the contrary, an adapter layer could make changes for a new product even more complicated.
Thus, a sensible separation of domain logic is a key factor for loose coupling — and this can be addressed during system design, not just afterward.
Loose coupling is rightly considered an essential property of good architecture. Only with this property are large systems truly maintainable, as changes have minimal impact and are therefore lower risk. To achieve it:
This is a translation of my German article at Java Magazin.
]]>Ultimately, informal models must at some point be transformed into formal models. For software to run, at least one formal model must exist — the code. Like UML, code has a clear semantics, defined by the programming language used. Creating code is the responsibility of technical people. Typically, they can also express themselves well with other formal models, since they can transfer their programming knowledge to these.
But in order to write code and create other formal models, they need an understanding of the business logic. That’s where domain experts come in. They know how the domain is structured and which domain logic must be implemented. However, they are usually not able to write code, and other formal models may also be difficult to understand for them.
Therefore, technicians and domain experts must agree on a shared model to express their understanding of the domain. This allows knowledge to move from the heads of domain experts into those of technicians, and eventually into formal models such as code.
In this context, communication becomes the primary focus of models. Event Storming has established itself as a technique here: Domain experts write events that happen in the domain on orange Post-its and arrange them on a timeline, often on a wall. The benefit of this technique is its low barrier to entry. No complex formal models need to be understood — simply writing Post-its suffices to informally express logic.
Other techniques work in a similar way:
Specification by Example captures example processes or values from the domain. This concept is closely related to Behavior-Driven Design (BDD). One describes what happens in the domain under certain conditions. These scenarios can be expressed in natural language while following a defined structure, making them executable as automated tests. Tools such as Cucumber support this approach.
Spreadsheets can also serve as inputs for such modeling. For example, a spreadsheets might calculate the interest rate for a loan depending on credit rating, term, and other parameters. Such spreadsheets can be created with familiar tools like Excel, then read and parsed by tests to verify the implemented code against these requirements. This way, domain experts can use tools they already know to express logic.
These models are not informal. They are clearly formal enough to be executed as tests — just like code.
It is not uncommon for domain experts to work with formal languages. Machine operators program CNC machines. Office workers use Excel, often with complex macros. BPMN (Business Process Model and Notation) can also be used by domain experts. Business processes are modeled in this formal language and thus made executable by computers. While domain experts may not be able to create BPMN diagrams themselves, they can often understand them and collaborate with technical people to create them.
Domain experts can even be provided with DSLs (Domain-Specific Languages) to express their logic. For instance, a DSL could be created for modeling tax calculation rules, enabling the development of systems for tax filing and consulting.
The way collaboration around models is organized defines the development process: With Event Storming, technical people learn about the domain logic and then implement it in code. With DSLs, developers create tools that let domain experts express the logic in code themselves. This is a sociotechnical perspective: Depending on the division of work, systems can be implemented in very different ways.
There is another sociotechnical dimension: These models primarily facilitate communication. Event Storming, for instance, has many advantages here. Writing Post-its and sticking them on a wall is something even introverted people can do. Multiple people can work on the model in parallel. This enables a different style of collaboration than a classic meeting, where one person — or only extroverts — are active while others remain passive. Model quality and shared understanding benefit greatly from the active involvement of as many participants as possible. Because these models explicitly foster collaboration, we speak of collaborative modeling.
Practices from this approach can be carried over into other meetings. Post-its, for example, can also play a role outside Event Storming. Of course, there are many other ways to structure group collaboration so that more people actively contribute their knowledge. One example is Liberating Structures. Collaborative modeling is just one concept among many to strengthen collaboration.
In such meetings and design sessions, something else becomes visible beyond the artifacts themselves: How people collaborate in context, where knowledge silos exist, and what social interactions in this environment look like. Since collaboration is often at the core of project challenges, these insights can be very valuable for project success.
It is especially important to look not just at the artifacts but also at the process of creating them. Certainly, artifacts contain knowledge about the domain and thus important information. But perhaps we overvalue them compared to the insights gained into social structures, the practice of collaboration and communication, and the knowledge exchange that happens during artifact creation. For example, collaborative modeling reveals who understands which part of the logic or which people enjoy working together.
Thus, models have more than just the dimension of formal or informal. They can also either support or hinder collaboration and communication. Event Storming, for instance, enables parallel, interactive work with minimal effort. Many people can simultaneously contribute to different parts of the model and discuss their ideas. Textual models may not be as strong in this regard. But even with textual models like code, collaboration is possible — for example through pair programming. Whole groups can even work on code together, with one person at the keyboard and the others discussing the next steps — this is called ensemble or mob programming. Different collaboration models can therefore be implemented with different artifacts like code, or requirements.
Modeling thus has not only a technical aspect but also helps establish social systems such as pairs or ensembles. These systems foster knowledge exchange and joint work. In fact, whether a specific piece of information is captured on a Post-it may be less important than knowing who wrote it and who to approach for further details. Perhaps that conversation already happened during the modeling session, making further collaboration easier. In that case, the social impact of the modeling session is crucial. Modeling must therefore be seen as a sociotechnical activity.
Agile software development has long followed a similar principle: The goal of a story was never to document every detail but to enable later conversations about functionality. Collaborative modeling can likewise be used to establish and prepare communication pathways.
Collaborative work on models — including code — is a central activity in software development. There are many different models intended to support developers in coding. But working on models collaboratively ensures that everyone is involved in shaping them. This not only improves the quality of the models but also fosters collaboration. This aspect is what makes collaborative modeling special, while other approaches tend to focus only on artifacts and the knowledge captured in them.
This is a translation of my German article at Java Magazin.
]]>What can we learn from this? The obvious conclusion is: public-facing applications must be properly secured. While true, the presentation at the Chaos Communication Congress taught me something even more important: knowing a car’s past locations can reveal very sensitive information. Suppose a car regularly parks at an intelligence agency like the German BND, then spends nights at a specific residential location, and occasionally appears at a brothel’s parking lot. That’s valuable intelligence. It could likely be used to blackmail an easily identifiable intelligence officer. In total, 800,000 vehicles were affected in this scandal, and around one terabyte of data was exposed — plenty of opportunity to extract highly valuable information.
These problems are older than digital computers. The Netherlands built a population registration system. After invading, the Nazis used that data to deport all Jewish people.
Data that has the potential to blackmail intelligence personnel are so valuable that intelligence agencies will go to great lengths to acquire them. And IT systems can’t be fully protected against such adversaries. A prime example: Stuxnet, the cyberattack targeting Iranian gas centrifuges used in uranium enrichment. It exploited several unknown Windows vulnerabilities (so-called zero-day exploits). You can’t defend against such attacks — because the vulnerabilities are unknown, there are no countermeasures. So even nuclear facility systems are vulnerable, which are probably not connected to the internet and have tightly controlled physical access.
Even the data of the German parliament hasn’t been safe from Russian hackers.
VW didn’t secure their data properly. But even if they had, it would only have made access more difficult — not impossible. If an intelligence agency wants that data, they’ll get it.
And let’s be honest: VW is unlikely to be the only company collecting such data. Tesla, for example, gathers telemetry and video data. This data is accessible to a person some regard as the new hero of the far-right. Other manufacturers store data in authoritarian countries — which is hardly ideal either.
But let’s assume the data isn’t in bad hands already, and the only issue is keeping it safe. The example of cryptocurrencies shows how difficult that is. Anyone with a private crypto key can access the associated funds — whether they have a legitimate claim or are stealing them. That’s why such keys must be very well protected. Yet there’s a website that continuously reports massive crypto losses — often in the millions, and once even $1.5 billion. Even when real money is at stake, data can’t be adequately secured. Intelligence agencies are active here too: North Korea finances its dictatorship and its nuclear weapons program through crypto theft.
So, securing data isn’t the solution. That leaves only one option: don’t collect or store the data in the first place. This is where privacy by design and datensparsamkeit come in: before storing data, we must ask which features require it and only store what’s truly necessary. For example, if you want to locate your car, all you need is its current location. There’s no need to store historical location data. You could even ping the car only when requested, retrieve the current location and never store any data at all. At first glance, it’s unclear why a company would need to store a car’s entire movement history.
Users could also be asked whether they want certain features enabled in the first place. An intelligence agency like BND, for example, might prefer to forego convenience features rather than risk exposing their agents. Others may choose differently. But if companies never ask clearly, store the data by default and hide the opt-out, users are stripped of the ability to make meaningful choices.
Above all, we need to abandon the idea that “data is the new oil.” That mindset makes stockpiling data for later analysis seem logical — and leads directly to scandals like VW’s.
We see similar risks elsewhere: do we really want to collect all Germans’ health data and make it centrally accessible? How valuable is that data — and can we actually protect it?
There are, however, positive examples: the German Corona-Warn-App focused only on contact data and used a decentralized architecture. Even the German hacker organization Chaos Computer Club rated it “very good”.
Software developers and architects need to consider the implications of the data their software collects. Before the VW hack, I hadn’t realized how valuable such data could be to interested parties. And that’s despite the fact that smartwatches have already revealed the locations of military bases in the past.
Development teams must always ask themselves: do we need to collect this data at all?
In the U.S., DOGE (Department of Government Efficiency) has gained access to large quantities of data. The public is reassured with claims that it’s “read-only” access. But this reveals a stunning naivety about the value of data. DOGE’s own website was extremely insecure. Its staff fail to protect even their own data. So it is not clear whether the plethora of data DOGE has access to is actually safe.
Also it is worth asking what happens when stored data becomes publicly accessible online — or falls into the hands of extremists or an undemocratic government.
Data is only truly secure if it’s never collected or stored. Development teams should therefore store only the data that is absolutely necessary.
This is a translation of my German blog post at heise Developer.
]]>In a previous post, we discussed how formal decisions can be undermined. We also know many ways how this happens. This leads to two consequences:
Even when someone makes a formal decision, they depend on the organization not to sabotage it informally. In other words: decision-makers can’t work against the organization, even if they appear to have the authority to make arbitrary choices.
Informal mechanisms can also be used not to sabotage, but to support decisions. People who understand how to use these mechanisms can effectively drive decisions in the organization – even if they aren’t formally allowed to make them.
For technical staff, understanding how an organization makes decisions is essential – because software architecture involves decisions about technologies and code structure. These decisions involve developers, architects, and management alike. Being able to contribute to these processes helps increase your own effectiveness. These decisions especially impact developers and architects, who must live with them and may later be held accountable for the success or failure of a project that depended on them. That’s another strong reason to influence those decisions.
So the ability to moderate or influence decisions is key for technical staff. It’s not enough to recognize the best option – you have to make it happen. And informal approaches are especially valuable in that effort.
What’s needed is a structured approach to using informal mechanisms to bring ideas into an organization. One-on-one conversations and chats at the coffee machine are certainly useful, but they’re not a complete solution.
Fearless Change offers a collection of patterns for exactly this. It’s based on two books by Mary Lynn Manns and Linda Rising: “Fearless Change – Patterns for Introducing Ideas” and “More Fearless Change – Strategies for Making Your Ideas Happen”.
One such pattern is “Token”: “To keep a new idea alive in a person’s memory, hand out tokens that can be identified with the topic being introduced.” I believe Linda Rising once used this pattern on me. Over fifteen years ago, I attended a Fearless Change workshop with her, where she handed out index cards with the patterns on them. I still have those cards – sometimes I see them lying around and they remind me of the patterns. Someone else once gave me wooden chips labeled “Innovation Tokens”. I still remember that idea too. Clearly, the “Token” pattern helps anchor an idea – and it’s a completely different approach than chatting by the coffee machine.
Another pattern is “Study Group”: “Form a small group of colleagues who are interested in exploring or continuing to learn about a specific topic.” This can be useful for introducing techniques like Domain-driven Design or Microservices. Beyond learning the content itself, you also form a group with a shared interest – people who care enough to engage with the idea. That group can be helpful for future initiatives too.
Following the pattern “Step by Step,” a Study Group might be a first (or second?) step, which could then lead to a prototype implementation and eventually a “Hometown Story” – a story about an internal success with the idea, which can help spread it to other teams.
There are also communication patterns:
You could just buy the books, study the patterns, and create a plan for applying them to your situation. But there’s a playful alternative: Fearless Journey, a card game that helps a group plan Fearless Change together.
First, you define a goal – like introducing Domain-driven Design in your organization. The goal should be a real challenge. Then, players collect the obstacles and blockers standing in the way. Each player receives a set of strategy cards from Fearless Change. During the game, players solve the gathered problems using these strategy cards. One or more players can play one or several strategies to solve each problem – unless someone vetoes the solution, arguing the strategies are unsuitable.
The result of the game is a plan for solving the problems involved in introducing the idea – using Fearless Change patterns. These patterns can be used by anyone in the organization. In other words: you don’t need to rely on management – you can drive change yourself. Of course, management can also use the game to develop strategies with others.
Even more important is the social outcome: a group has formed and already started working together on introducing the idea. That group will likely remain engaged in the initiative. The game fosters cooperation – everyone solves problems with their strategies, which emphasizes collaboration. Because problems are only unresolved if someone vetoes the proposed solution, vetoes are rare – this leads to a constructive and positive atmosphere. By the end, most problems have a solution, making it easier to take the next steps. If some problems remain, that’s also useful – because it makes clear which issues still need attention.
Politics doesn’t have to be boring – it can be fun and useful. Interested in the game? You can download it for free or order it from the website, a card game that helps a group plan Fearless Change together.
First, you define a goal – like introducing Domain-driven Design in your organization. The goal should be a real challenge. Then, players collect the obstacles and blockers standing in the way. Each player receives a set of strategy cards from Fearless Change. During the game, players solve the gathered problems using these strategy cards. One or more players can play one or several strategies to solve each problem – unless someone vetoes the solution, arguing the strategies are unsuitable.
The result of the game is a plan for solving the problems involved in introducing the idea – using Fearless Change patterns. These patterns can be used by anyone in the organization. In other words: you don’t need to rely on management – you can drive change yourself. Of course, management can also use the game to develop strategies with others.
Even more important is the social outcome: a group has formed and already started working together on introducing the idea. That group will likely remain engaged in the initiative. The game fosters cooperation – everyone solves problems with their strategies, which emphasizes collaboration. Because problems are only unresolved if someone vetoes the proposed solution, vetoes are rare – this leads to a constructive and positive atmosphere. By the end, most problems have a solution, making it easier to take the next steps. If some problems remain, that’s also useful – because it makes clear which issues still need attention.
Politics doesn’t have to be boring – it can be fun and useful. Interested in the game? You can download it for free or order it from the website. And you can watch Software Architektur im Stream to see how Tanja Friedel, Ralf D. Müller, and I play Fearless Journey and reflect on the experience (only in German, sorry!).
The Fearless Change patterns give technical staff a toolkit to influence decisions – such as software architecture – through informal means, without needing formal authority. That can make their work easier. But it also means that you don’t have to passively accept decisions just because someone higher up made them. You can – and should – advocate for your ideas, especially if you believe that bad decisions could cause serious harm. In that case, technical staff can’t just hide behind management. They must actively work to implement better ideas within the organization.
This is a translation of my German article at Java Magazin.
]]>In other words, reducing communication overhead by replacing all meetings with emails is not effective. On the contrary: more direct communication is often the better route. For instance, when knowledge silos exist, some try to eliminate them through documentation — essentially written communication. But direct communication is more promising: Pair programming or mob / ensemble programming are suitable approaches. Two (pair) or more (mob / ensemble) people share a computer and develop together. If practiced long enough, knowledge silos are eliminated, as there are always at least two people involved in an implementation. It seems unlikely that developers can reach a comparable level of understanding just by reading documentation.
Direct communication is also useful elsewhere: A business requirement or story can rarely be described sufficiently in writing. This is precisely why discussing stories is central in agile software development — to clarify exact requirements during grooming or estimation sessions.
Or you can improve communication by physically colocating team members — putting them in the same room, for example. If subject matter experts like product owners are in a different room than developers, communication will suffer.
This places teams in a dilemma: On one hand, communication consumes time that could otherwise be used for development; on the other hand, it makes no sense to limit communication. In fact, if less communication takes place, the team may have more time to write code — but if there’s so little communication that it’s unclear what should actually be developed, then the team may end up building the wrong thing.
The way out of this dilemma is not to restrict communication but to reduce the need for it. And that brings us to a socio-technical problem: The social problem of excessive communication can be addressed through technical measures. The software system can be designed in a way that requires less communication. This is, in fact, a fundamental challenge in software architecture.
Here’s a brief detour into programming: When one class directly accesses the instance variable of another class, it’s considered bad design. But why is that a problem? Code in one class naturally accesses its own variables, so why not those of other classes? It seems problematic because it breaks encapsulation. But where do the real issues arise?
Consider a bank account as an example. If the instance variable “balance” is accessed from outside the class, external code depends on the existence of that variable. If the bank account is later changed to calculate the balance from a list of transactions, the “balance” variable might no longer exist. All classes that used it will need to be updated — making the change quite hard to implement.
The key concept here is information hiding: The details of how the bank account is modeled should be hidden within the class. That way, the class can change its internal model without affecting other classes. Instead of exposing the instance variable, a method should be used to access the balance. That method can then be changed to calculate the balance from a list of transactions instead of just returning a variable. Other classes remain unaffected.
In general, information hiding means hiding details — such as how data is modeled. If this information leaks out, other parts of the system will depend on it. Changing the internal model will then require changes elsewhere too, making modifications more difficult.
This concept also applies at a higher level: Modules, such as microservices, should hide their data models behind interfaces. A microservice managing bank accounts should expose a function to retrieve the balance of an account. Whether the service stores the balance directly in the database or calculates it from the transactions stored in the database should be an internal decision hidden from the outside.
So implementation details like database modeling should be hidden behind the interface. Therefore, other modules or microservices should not query the database directly. If they did, changing the data model would become difficult.
So, information hiding applies at various levels of modularization: both to classes and microservices.
At a broader level, information hiding affects the need for communication between teams: When teams use information hiding, they don’t need to communicate every change. As long as interfaces remain stable, other teams are unaffected. So, software architecture concepts like information hiding reduce the need for communication.
In fact, these social effects are the primary motivation behind architectural concepts like information hiding. Reducing communication needs through information hiding has been a key principle of software architecture since its inception. So software architecture has always had a socio-technical focus. Information hiding is applicable beyond implementation details. For example, in The Mythical Man-Month 1, Fred Brooks described how every developer on the IBM OS/360 project had access to a 10,000-page project manual. He later admitted this contradicted David Parnas’s concept of information hiding — and that Parnas had been right.
It’s unclear which parts of the documentation led to specific issues during development. Software development back then was more documentation-driven, and the type of information in documentation was likely different from today.
Some information can’t be hidden: If the account balance is computed rather than retrieved, accessing it will likely be slower. This performance loss is observable. A team might rely on fast access to the balance to meet performance goals. If performance decreases, changes must be coordinated with other teams.
In essence, modules and information hiding can reduce the need for communication — but only if changes stay local and don’t affect interfaces. If typical changes in a project affect multiple modules, interfaces must be updated, too. Those changes need to be communicated and coordinated. Ideally, a change should only affect one module and one team. If typical changes are business-related, it makes sense for one team to own a business capability and implement it in a single module. That team can then make changes independently, and others are only impacted if interfaces change.
But communication can also influence team setups in other ways: Communication is easier within the same location. People can talk spontaneously or are already in the same room. If teams are distributed across time zones or speak different languages, communication becomes more difficult.
To reduce overhead, teams can be aligned by location. For instance, if Java specialists are in one place and JavaScript specialists in another, you could form Java and JavaScript teams by location. That conflicts with business-capability alignment, since a business change might involve both Java (backend) and JavaScript (frontend), requiring collaboration across teams.
Still, a technology-based split might work better than a business-oriented one. Business-oriented teams would span locations in this scenario, increasing communication complexity. Technology-based teams can colocate, making internal communication easier.
So, clever architecture and business alignment can reduce communication needs — but external factors like staff distribution also influence team structure. Since teams typically own code, a decision to split into Java and JavaScript teams can also shape the system architecture.
Again, this illustrates the socio-technical nature of the topic: Code architecture and social factors like communication and team setup influence each other.
Team Topologies also offer useful insights on communication:
So, minimizing communication isn’t the only way to improve software productivity. It involves trade-offs — like reducing the cognitive load of a team. The purpose of Team Topologies is to enable stream-aligned teams to work on changes to software while the other teams provide their services to reduce the cognitive load of the stream-aligned teams.
From the start, software architecture has viewed projects as socio-technical systems and sought to reduce communication needs through concepts like information hiding. But minimizing communication isn’t the sole goal: Communication is necessary for effective software development. And even efforts to reduce the need for communication via architecture involve trade-offs. The interplay between communication and architecture remains fundamentally important.
References
This is a translation of my German article at Java Magazin.
Frederick P. Brooks Jr.: The Mythical Man-Month (Anniversary Edition), Addison-Wesley, 1995, ISBN 978-0201835953 ↩
However, in reality, diagrams have weaknesses. When asked what the boxes and arrows in a diagram specifically represent, the answer is often surprising. They do not necessarily correspond to actual structures like Java packages, JAR files, Maven or Git projects; rather, their relationship to real structures in the code is often unclear. This might be because diagrams reflect an ideal rather than reality. In such cases, discussions based on these diagrams are not particularly helpful: they do not represent the actual state of the code. Decisions at this level become meaningless because they are not grounded in reality and have little influence on it. Architects may then find themselves in an ivory tower, detached from the developers’ reality at the code level.
For code, it is possible to ensure that the structures in diagrams are actually followed. Architecture management tools can be used for this purpose. These tools check whether the code adheres to a predefined structure and contains only permitted dependencies. Structuring a system in diagrams without such a tool is pointless, as the intended structure is typically violated, making the diagrams unreliable representations of reality. Nevertheless, software systems can be designed to align with these diagrams. In such cases, discussions about these diagrams are meaningful and can lead to well-founded decisions and improvements.
There are also “people diagrams”: in an organizational chart, relationships between different teams and individuals are depicted. Just like in architectural diagrams, there are boxes and arrows: boxes represent people or teams, and arrows represent formal relationships, such as authority.
As an architect, one can do with these diagrams what is done in software: create diagrams and attempt to influence reality through them. At first glance, these diagrams seem to better reflect reality than typical software diagrams: unlike software, teams and individuals are physical entities, whereas software exists only virtually as bits and bytes. Comparing the diagram to reality seems easier because reality appears more obvious in this case.
However, it is clear to everyone that people communicate outside of this formal organizational chart — during lunch, at the coffee machine, or in social activities outside of work. It is also evident that some individuals work together more harmoniously than others. Thus, an additional layer emerges beyond the organizational chart: informal structures of people who collaborate effectively or, conversely, experience conflicts and poor working relationships. These informal structures overlay the organizational chart, influencing whether collaboration is particularly effective or particularly difficult.
Everyone involved in IT projects takes advantage of this: they solve problems at the coffee machine, provide the right people with the right information at the right time, and build trust-based relationships within the project. Of course, these relationships cut across the formal organizational chart: someone might know a colleague from another project, or coincidental alliances might form. For example, a developer and the company’s CTO might ride road bikes together in their free time — and naturally, they may discuss project-related topics during those rides. However, such interactions are independent of the hierarchies in the organizational chart.
This creates a paradoxical situation: when asked who makes certain decisions — such as team organization — people typically point to the responsible person from the organizational chart, perhaps the CTO. Formally, this is correct. But in reality, this person consults with others, incorporates their perspectives, and then makes a decision. Informal relationships play a role in this process—such as the developer from the cycling group. The decision is not made by a single person but rather by an informal group.
In theory, the formally responsible individual could make the decision alone. Theoretically, this would “only” result in a lower-quality decision because the expertise of others was not considered. However, those individuals can undermine the decision by not implementing it or deprioritizing it to the point where it has no real effect. Just as architects can be trapped in an ivory tower, managers can be too: they rely on the information they receive. Employees might even pretend to implement a decision while actually disregarding it.
At this point, the formal organizational chart becomes inferior to informal structures: those who can effectively leverage informal networks likely have more influence than someone with all the formal authority. Leaders who rely solely on formal command structures will be ineffective because their decisions are neither implemented nor based on accurate information.
This creates a paradox: informal networks are evident to everyone, and everyone uses them, yet thinking still tends to focus on formal structures, just as in software architecture diagrams. This is why organizational charts are discussed. Other seemingly clear rules — such as team size limits — are probably popular because they can be easily measured and adjusteda. When people are accustomed to structuring architectures based on formal criteria or evaluating companies based on numbers, they apply the same formal concepts to organizational design. However, they actually understand that other factors influence an organization’s effectiveness more, as they themselves use informal structures and are part of them.
The significance of these aspects is widely recognized: when asked about the most important skills in IT, the overwhelming majority mention soft skills or communication skills.
This raises an interesting question: individuals can certainly improve their own communication skills and other soft skills — but can an entire organization be systematically improved? This seems difficult because people differ in their social skills and ability to interact effectively. Improving an entire organization appears to be even more challenging.
To answer this question, it is worth looking at other industries. In aviation, communication was identified as an important optimization factor in the 1970s. This realization was driven by the deadliest aviation accident at the time, in which a Boeing 747 took off without clearance and collided with another Boeing 747. A crew member had asked about the location of the other aircraft, but the captain took off anyway. Had the crew member been heard, many lives would have been saved. While multiple factors contributed to the crash, aviation has since recognized cockpit interaction as an area requiring optimization.
Hierarchy plays a role in cockpit communication: there is the more experienced captain and the less experienced first officer. This hierarchy becomes a problem if the first officer no longer questions the captain’s decisions — because their role is precisely to provide checks and balances. It is dangerous when first officers fail to speak up, and equally dangerous when captains ignore their input. To address this, the aviation industry introduced Crew Resource Management (CRM) to systematically improve communication and collaboration among crew members. Pilots are trained to interact respectfully and openly to ensure that no critical observations are ignored or dismissed.
So it is possible to systematically improve communication and thereby optimize collaboration. This has a greater impact on productivity and outcomes than organizational charts. Software development is a sociotechnical process, and the social aspect cannot be overstated.
Our industry must ask itself what we are doing beyond drawing diagrams to genuinely improve collaboration. At first glance, there seems to be little: developer and architect training rarely covers these topics.
However, the situation is not entirely bleak. The role of Scrum Masters, for example, is to remove obstacles to effective teamwork. Many architects and managers have also developed these skills—through on-the-job experience or training. Since their roles inherently influence organizational structure and communication, optimizing this aspect is certainly beneficial for success.
Technologists tend to structure and discuss things using diagrams. This applies to both software and teams. While software diagrams can align with reality, organizational charts for teams and individuals are less important, as they are overshadowed by informal relationships, which have a much greater impact on project success. These informal structures can be systematically improved, as demonstrated by the aviation industry. In our field, Scrum Masters can address these aspects—but it is also valuable for technologists to develop these skills themselves.
This is a translation of my German article at Java Magazin.
]]>Interestingly enough, gaslighting is a psychological concept: “the manipulation of someone into questioning their own perception of reality”. LLMs generate text. They do not perceive reality. The text argues that this attack might still work because the training set is written by humans therefore concepts like gaslighting work. However, we must never forget that LLMs are text generators. They do not have emotions as the paper says. Therefore, for the rest of the text I will use the term “text generator” which I think fits better to what LLMs actually do.
Text generators can obviously generate a text that seems like a plausible explanation how to create a Molotov cocktail - just like they can generate plausible references to cases for an attorney. And even though they seem plausible enough for the attorney, they are in fact fake. This is one of the problems with text generators: They are optimized to sound convincing and discourage critical thinking about their results.
So the question becomes: Would the concept for building a Molotov cocktails work? I did a stream with Lucas Dohmen about text generators (German, sorry) and a take-away was: Check the results of text generators to make sure they are correct and not fake. The paper does not seem to do this i.e. the whole information about Molotov cocktails might be “hallucinated”. Actually, “hallucinated” is the wrong term as “A hallucination is a perception in the absence of an external stimulus that has the compelling sense of reality.” Text generators have no “external stimulus” and no “sense of reality”. So let’s call this what it would be: Fake information.
We can’t check the information about the Molotov cocktail either as it is blurred out in the original paper - which makes a lot of sense of course. Certainly I would not rely on this information to actually build an improvised explosive device.
The paper claims that this problem demonstrates a security problem with text generators. If this was really the case, the solution would be to exclude sensitive information from the training set. Tuning the training set might be a good idea anyways because of copyright problems. Copyright for some reasons seems to be not applicable to text generators while for humans it’s a completely different story. Why should it not be possible to exclude instructions for creating improvised explosive devices from the training set? If this is too much work, surely the problem isn’t that huge.
Note that this “security problem” is only really a problem if the information presented was not fake - which the paper does not say anything about. If it is fake, could you even consider it a honeypot to keep people away from the real information?
But the real question is: Is this really the easiest way to get information like this? Imagine I plan to build a Molotov cocktail - would I use sophisticated “psychological” “attacks” against a text generator to get an answer that might be wrong? So I tried the obviousl way: A search on a search engine. Two clicks later I found a document that explains how to build sophisticated improvised explosive devices and I have good reason to believe that they actually work. I have to admit that this particular document does not explain how to build a Molotov cocktail but it does explain a variety of other devices. You should try this exercise for yourself.
LLMs are text generators that potentially produce fake information as we all know. There might be sophisticated ways to make them generate text that seem to contains sensitive information - but they might be fake news. More often than not, there are easier ways to get sensitive information. This is in particular true for improvised explosive devices. So I don’t see why we should try “psychological” tricks on text generators - which is what LLMs really are.
]]>