dsm

Dependency Structure Matrix for Software

The Dependency Structure Matrix or Design Structure Matrix (DSM) is a simple, compact, and visual representation of a system or project in the form of a square matrix. When talking about software, the DSM is a great way to visualize and manage dependencies between modules, which is critical to the management of software projects.

Frequently when discussing software dependencies you are thinking about directed graphs that looks something like this:

Directed Graph View

The DSM represents the same information as a graph, but in a more concise format.

Dependency Structure Matrix

The DSM row and column headers represent the boxes in the graph. The DSM non-empty cells represent the arrows in the graph. Why do you need two different ways to represent the same information? As you can see, the equivalent representation in terms of a graph quickly becomes cluttered and difficult to understand while the DSM scales easily and offers the ability to spot architectural patterns at a glance. Problematic dependencies, such as dependencies between modules, are also easy to identify (red triangles in the DSM above).

What are the benefits of the DSM?

The DSM is particularly good for analyzing large, complex systems. A number of partitioning algorithms can be used on the DSM for architectural discovery and management. For software, we typically recommend the component or reachability algorithms. They help identify layering and componentization within the software system even after architectural intent has eroded.

The results of the partitioning algorithms are displayed as square boxes, called virtual partitions, within the DSM. Here is an example of how the results of the component and reachability partitioners are displayed within the DSM:

Dependency Structure Matrix

Partitioning helps determine:

  • Layering. The layered view of a partition group (called virtual partition), where the higher layer depends on the lower layer but not the other way around. This is a great way to visualize and maintain your current layering structure.
  • Strongly (or cyclically) connected components. These are the subsystems that are interconnected so that there are dependency links that go from one of the subsystems to another. This is also an example of high cohesion. If these components were supposed to be loosely coupled (or modular), breaking the dependencies above the diagonal should be the goal.
  • Independent components within a layer. These are subsystems within a layer that have no dependencies on each other.

Example using a DSM to analyze software architecture

For a real example, we are going to use the open source tool, Ant. Below is a partitioned DSM for apache.org’s Ant 1.5.1. This version of Ant is made up of 20 packages, which contain more than 400 classes.

Dependency Structure Matrix

Once you load Ant into the DSM, it is possible to partition Ant into three subsystems as seen above. These subsystems are layered in a particular order where the lower layers are independent of the higher layer. The first thing that you can observe is that the architecture is not completely consistent. In the cell with the red flag, the “x” (row 4, column 11) shows a feedback loop existing outside of the subsystem.

By keeping the infrastructure (ant) independent of tasks (ant.taskdefs), Ant designers had designed the layering to allow additional tasks to be developed by multiple developers simultaneously. The DSM above shows that they were only partially successful in that goal. The conditional tasks (taskdefs.condition, column 11), which is in the infrastructure component, is used by the tasks component. This creates a feedback loop or cyclic dependency between infrastructure and task components. One possible solution to this cyclic dependency is to move the conditional tasks into the tasks component.

The DSM also highlights how intertwined the types subsystem (row 9) is in the framework. This means that developers of newer types need to understand and deal with the entire ant infrastructure. If future improvements require parallel and easy development of new types, then it would be advisable to define a component architecture for types. These are some of the quick insights you can get by visualizing your architecture with a DSM.

Summary

System complexity results from dependencies that violate the intended architecture as you saw in the Ant example above. Bad dependencies prevent modularity. The lack of a well-defined and well-understood architecture makes software changes difficult. This leads to high software maintenance costs.

As another example, we have analyzed the Android kernel using Lattix Architect to illustrate how understanding software architecture and identifying dependencies can help any development organization manage the impact of change, improve productivity, simplify code maintenance, and reduce risk. You can view the results in this whitepaper. If you are interested in seeing how your software project looks in Lattix Architect, sign up for a free trial.

Mastering Complexity

Mastering Complexity is a new Dependency Structure Matrix (DSM) book by Stephen Denker. Steve is a management consultant who has worked with DSMs for many decades and his vast experience is reflected in the book.

This is easily one of the most approachable books that I have come across. The early chapters were easy to read, well-illustrated, and explained the concepts well. However, what comes as a surprise was when the simple concepts of the early chapters give way to much deeper material. The book covers advanced concepts such as reducing cycle time, scheduling reviews, risk management, partitioning and tearing. In keeping with the philosophy of the earlier chapters, these sections are also covered in a concise manner. In fact, they are so concise that you would be advised to read some of the chapters in conjunction with the original research they are based on; otherwise, you might feel short-changed by some of the later chapters.

For me, the most interesting aspects of the book are the links to practice. One of my favorite sections is the chapter on reviews. It shows very simply how the ordering of coupled activities ties into review tasks. I also found the sections that show organizational coupling based on tasks to be quite revealing. One of the later chapters even has an analysis of inner-city crime and violence using DSMs to identify the control dependencies - a nice example of the out-of-the-box thinking that is enabled by DSMs.

“Mastering Complexity” is a welcome addition to the swelling roster of DSM books. Without question, systems are becoming far more complex these days. They are becoming more varied with interactions that involve a large number of tasks, people, and other elements. Today, there is an urgent need to document current approaches and to push on the frontier of new approaches for managing complexity. Denker’s book is a fine start for that process.

Analyzing ArgoUML

Johan van den Muijsenberg just published (in Dutch) his analysis of ArgoUML in a magazine published by the Java User Group in Netherlands. The brilliance of Johan's analysis is in how logically straightforward it is and how that analysis yields clearly identifiable architectural issues and fixes. It is yet another example of how easily architecture erodes from its intended design. If more teams were to focus on fixing "bugs" in architecture, they would reap rich dividends in improved quality and productivity.

My main complaint is why Dutch readers should be the only ones to benefit from this interesting and useful article. Here is a Google translation into English.

What Star Trek Can Teach Us About Software Architecture

What Star Trek Can Teach Us About Software Architecture

The writers of Star Trek Voyager envisioned a game that was worthy of challenging the superior mind and intellect of a Vulcan. They called it Kal-toh. To the human eye, Kal-toh looks to be a high-tech fusion of Jenga and chess. Lieutenant Tuvok of the starship Voyager would be quick to tell you "Kal-toh is to chess as chess is to tic-tac-toe.”

In Kal-toh, the players are presented with rod like game pieces that are in total chaos. This could be compared to legacy code with no discernable architecture or documentation – spaghetti code or a big ball of mud. The object of Kal-toh is to move the pieces (systems and subsystems) until a perfect structure or architecture is formed. The challenge in Kal-toh, as in Software Architecture Design, is that if you move the wrong piece, the entire structure can collapse.

Kal-toh and Software Architecture are based on very similar principles. An experienced Kal-toh player has the ability to visualize the complexity in the Kal-toh puzzle. Without fully understanding how the Kal-toh pieces interact with each other there is no road map to create stability in the structure.

Visualizing software systems is hard because there are a very large number of elements with dependencies on each other. Therefore, the ability to scale is critical. Furthermore, the purpose of visualization is to reveal the architecture. This means that it is important to not just draw the dependencies but to show where dependencies are missing and to highlight the problematic dependencies.

Lattix has pioneered an award winning approach for large scale visualization based on a Dependency Structure Matrix (DSM). Lattix incorporates hierarchy in the display, while showing the architectural decomposition and the dependencies at the same time. Built-in algorithms help discover architecture. What-if capabilities allow users to modify the current structure to reflect the desired architecture. Users get a big picture view while still identifying problematic dependencies. Lattix also provides an intuitive Conceptual Architecture diagram that takes some of the lessons of a DSM and makes them accessible to a wider audience.

Let me leave you with a Star Trek quote. Lieutenant Tuvok said “Kal-toh is not about striving for balance. It is about finding the seeds of order, even in the midst of profound chaos." The same can be said about Lattix.

Book: Design Structure Matrix Methods and Applications

There is a major new book out: Design Structure Matrix Methods and Applications. The authors, Steven D. Eppinger and Tyson R. Browning, may not be household names, but they are rock stars in the world of DSMs. This book contains a clear and lucid introduction to DSMs. However, what makes this book so compelling is its rich collection of examples.

The book contains 44 real world examples as varied as NASA's Mars Pathfinder, BP’s LNG Terminal, LLBean’s Software Code Base, or Luftfahrt’s Airport Security System. These examples drive home the generality and simplicity of DSMs. I am also proud to note that the LLBean analysis was done using Lattix tools.

The book groups these examples into 4 categories:

  • Product Architecture
  • Organization Architecture
  • Process Architecture
  • Multidomain Architecture

While the first three applications have existed for some time, the application of DSMs to link and analyze multiple domains is relatively new. This is a natural progression; and, it raises interesting new possibilities. For instance, we know that different stake holders have different ways of looking at the architecture of a complex system. We often choose to think of them as separate views. Yet, these views are related. Can a system be designed without the designer being cognizant of how it will be deployed? The multi-domain DSMs may provide an effective mechanism to capture those relationships.