Git Product home page Git Product logo

moduliths's Introduction

Moduliths

Important
The Moduliths project has been discontinued and been transferred to Spring Modulith. The current 1.3 release branch will see futher bugfix upgrades for as long as Spring Boot 2.7 is under open-source support. For migration instructions, please consult the Spring Modulith migration guide.

A playground to build technology supporting the development of modular monolithic (modulithic) Java applications.

Maven Build

tl;dr

Moduliths is a Spring Boot extension based on ArchUnit to achieve the following goals:

  • Verify modular structure between individual logical modules in monolithic Spring Boot applications.

    Prevents cyclic dependencies as well as explicitly defined allowed dependencies to other modules. Verifies access to public components in API packages (convention based, customizable).

  • Bootstrap a subset of the modules of a monolithic Spring Boot application.

    Limits the bootstrap of Spring Boot (component scanning and application of auto-configuration) to a single module, a module plus its direct dependents or an entire tree.

  • Derive PlantUML documentation about the modules.

    Translates the actual module structure into a PlantUML component diagram via Structurizr for easy inclusion in e.g. Asciidoctor based documentation. Diagrams can be rendered for a single module plus its collaborators or the entire system at once.

Quickstart

  1. Create a simple Spring Boot application (e.g. via Spring Initializer).

  2. Add the Moduliths dependencies to your project:

    <dependencies>
    
      <!-- For the @Modulith annotation -->
      <dependency>
        <groupId>org.moduliths</groupId>
        <artifactId>moduliths-core</artifactId>
        <version>${modulith.version}</version>
      </dependency>
    
      <!-- Test support -->
      <dependency>
        <groupId>org.moduliths</groupId>
        <artifactId>moduliths-test</artifactId>
        <version>${modulith.version}</version>
        <scope>test</scope>
      </dependency>
    </dependencies>
    
    <!-- If you use snapshots -->
    <repositories>
      <repository>
        <id>spring-snapshots</id>
        <url>https://repo.spring.io/libs-snapshot</url>
      </repository>
    </repositories>
  3. Setup your package structure like described here.

  4. Create a module test like described here.

Context

When it comes to designing applications we currently deal with two architectural approaches: monolithic applications and microservices. While often presented as opposed approaches, in their extremes, they actually form the ends of a spectrum into which a particular application architecture can be positioned. The trend towards smaller systems is strongly driven by the fact that monolithic applications tend to architecturally degrade over time, even if – at the beginning of their lives – an architecture is defined. Architecture violations creep into the projects over time unnoticed. Evolvability suffers as systems become harder to change.

Microservices on the other hand promise stronger means of separation but at the same time introduce a lot of complexity as even for small applications teams have to deal with the challenges of distributed systems.

This repo acts as a playground to experiment with different approaches to allow defining modular monoliths, so that it’s easy to maintain modularity over time and detect violations early. This will keep the ability to modify and advance the codebase over time and ease the effort to split up the system in the attempt to extract parts of it into a dedicated project.

The architecture-code-gap

In software projects, architectural design decisions and constraints are usually defined in some way and then have to be implemented in a code base. Traditionally the connection between the architectural decisions and the actual have been naming conventions that easily diverge and cause the architecture actually implemented in the code base to slowly degrade over time. We’d like to explore stronger means of connections between architecture and code and even look into advanced support of frameworks and libraries to e.g. allow testability of individual components within an overall system.

There already exists a variety of technologies that attempts to bridge that gap from the architectural definition side, mostly by trying to capture the architectural definitions in executable form (see jQAssistant and Existing tools) and verify whether the code base adheres to the conventions defined. In this playground, we’re going to explore the opposite way: providing conventions as well as library and framework means, to express architectural definitions directly inside the code base with two major goals:

  1. Getting the validation of the conventions closer to the code / developer — If architectural decisions are driven by the development team, it might feel more natural to define architectural concepts in the code base. The more seamless an architectural rule validation system integrates with the codebase, the more likely it is that the system is used. An architectural rule that can be verified by the compiler is preferred over a rule verified by executing a test, which in turn is preferred over a verification via dedicated build steps.

  2. Integration with existing tools - Even in combination with existing tools, it might just help them to ship generic architectural rules out of the box with the developer just following conventions or explicitly annotating code to trigger the actual validation.

Design goals

  • Enable developers to write architecturally evident code, i.e. provide means to express architectural concepts in code to close the gap between the two.

  • Provide means to verify defined architectural constraints as close as possible to the code (by the compiler, through tests or additional build tools).

  • As little invasive as possible technology. I.e. we prefer documented conventions over annotations over required type dependencies.

Current feature set

A module model for Java packages

The most simple module setup

At its very core, Modulith assumes you have your application centered around a single Java package (let’s assume com.acme.myapp). The application base package is defined by declaring a class that is equipped with the @Modulith annotation. It’s basically equivalent to @SpringBootApplication but indicates you’re opting in into the module programming model and package structures.

Note
Notation conventions
+ – public type
o – package protected type

Every direct sub-package of this package is considered to describe a module:

com.acme.myapp                          (1)
+ @Modulith ….MyApplication

com.acme.myapp.moduleA                  (2)
+ ….MyComponentA(MyComponentB)

com.acme.myapp.moduleB                  (3)
+ ….MyComponentB(MySupportingComponent)
o ….MySupportingComponent

com.acme.myapp.moduleC                  (4)
+ ….MyComponentC(MyComponentA)
  1. The application root package.

  2. moduleA, implicitly depending on moduleB, only public components.

  3. moduleB, not depending on other modules, hiding an internal component.

  4. moduleC, depending on moduleA and thus moduleB in turn.

In this simple scenario, the only additional means of encapsulation is the Java package scope, that allows developers to hide internal components from other modules. This is surprisingly simple and effective. For more complex structural scenarios, see More complex modules.

Running tests for a module

An individual module can be run for tests using the @ModuleTest annotation as follows:

package com.acme.myapp.moduleB;

@RunWith(SpringRunner.class)
@ModuleTest
public class ModuleBTest { … }

Running the test like this will cause the root application class be considered as well as all explicit configuration inside it. The test run will customize the configuration to limit the component scanning, the auto-configuration and entity scan packages to the package of the module test. It will also verify dependencies between the modules. See more on that in More complex modules.

For moduleB this is very simple as it doesn’t depend on any other modules in the application.

Handling module dependencies in tests

Without any further configuration, running an integration test for a module that depends on other modules, will cause the ApplicationContext to start to fail as Spring beans depended on are not available. One option to resolve this is to declare @MockBeans for all dependencies required:

package com.acme.myapp.moduleA;

@RunWith(SpringRunner.class)
@ModuleTest
public class ModuleATest {

  @MockBean MyComponentB myComponentB;
}

An alternative approach to this can be to broaden the scope of the test by defining an alternative bootstrap mode of DIRECT_DEPENDENCIES.

package com.acme.myapp.moduleA;

@RunWith(SpringRunner.class)
@ModuleTest(mode = BootstrapMode.DIRECT_DEPENDENCIES)
public class ModuleATest { … }

This will now inspect the module structure of the system, detect the dependency of Module A to Module B and include the latter into the component scan as well as auto-configuration and entity scan packages. If the direct dependency has dependencies in turn, you now need to mock those using @MockBean in the test setup.

In case you want to run all modules up the dependency chain of the module to be tested use BootstrapMode.ALL_DEPENDENCIES. This will cause all dependent modules to be bootstrapped but unrelated ones to be excluded.

General recommendations

If you find yourself having to mock too many components of upstream modules or include too many modules into the test run, it usually indicates that your modules are too tightly coupled. You might want to look into replacing those direct invocations of beans in other modules by rather publishing an application event from the source module and consume it from the other module. See [sos] for further details.

More complex modules

Sometimes, a single package is not enough to capture all components of a single module and developers would like to organize code into additional packages. Let’s assume Module B is using the following structure:

com.acme.myapp
+ @Modulith ….MyApplication

com.acme.myapp.moduleA
+ ….MyComponentA(MyComponentB)

com.acme.myapp.moduleB
+ ….MyComponentB(MySupportingComponent, MyInternal)
o ….MySupportingComponent
com.acme.myapp.moduleB.internal
+ ….MyInternal(MyOtherInternal, InternalSupporting)
o ….InternalSupporting
com.acme.myapp.moduleB.otherinternal
+ ….MyOtherInternal

In this case we have two supporting packages that contain components that depend on each other (MyInternal depending on InternalSupport in the same package as well as MyOtherInternal in the other supporting package). By convention, on the module level, only dependencies to the top-level module package are allowed. I.e. any type residing in another module that depends on types in either ….moduleB.internal or moduleB.otherInternal will cause an @ModuleTest to fail.

Named interfaces

In case a single public package defining the module root is not enough, modules can define so called named interface packages that will constitute packages that are eligible targets for dependencies from components of other modules.

com.acme.myapp
+ @Modulith ….MyApplication

com.acme.myapp.moduleA
+ ….MyComponentA(MyComponentB)

com.acme.myapp.complex.api
+ @NamedInterface("API") ….package-info.java
com.acme.myapp.complex.spi
+ @NamedInterface("SPI") ….package-info.java
com.acme.myapp.complex.internal
o ….MyInternal

As you can see, we have dedicated packages of the module annotated with @NamedInterface. The annotation will cause each of the packages to be referable from other modules dependencies, whereas non-annotated packages of the module (internal) won’t (including the module root package).

Enforcement of architectural rules

Note
Conventions

[check circle] – already implemented

[question circle] – not yet implemented

Given the module conventions we can already implement a couple of derived rules:

[check circle] Assume top-level module package the API package — If sub-packages are used, we could assume that only the top-level one contains API to be referred to from other modules.

[check circle] Provide an annotation to be used on packages so that multiple different named interfaces to a module can be defined.

[check circle] Prevent invalid dependencies into module internal package. — All module sub-packages by default except explicitly declared as named interface.

[question circle] allowedDependencies would then have to use moduleA.API, moduleB.SPI. If a single named interface exists, referring to the module implicitly refers to the single only named interface.

[question circle] Verify module setup — We can verify the validity of the module setup to prevent configuration errors to go unnoticed:

  • [question circle] Catch invalid module and named interface references in allowedDependencies.

[question circle] Derive default allowed dependencies based on the Spring bean component tree — by default we can inspect the Spring beans in the individual modules, their dependencies and assume the beans structure describes the allowed dependency structure. This can be overridden by explicitly declaring @Module(allowedDependencies = …) on the package level.

[question circle] Correlate actual dependencies with the ones defined (implicit or explicit) — Even with dependencies only defined implicitly by the Spring bean structure, the code can contain ordinary type dependencies that violate the module structure.

[question circle] No cycles on the module level — We should generally disallow cycles on the module level.

Sample applications

  • Spring RESTBucks - an implementation of the RESTBucks API from the ”REST in Practice” book. Primary a showcase for hypermedia APIs but still using Moduliths primarily for documentation purposes.

  • Salespoint - a POS library developed by the TU Dresden to be used in the software engineering lab to teach third semester students how to build web applications with Spring Boot.

Ideas

Unapproached yet

Spring Boot based module tests

Further ideas

  • As Spring Application Events are a recommended means to implement inter-module interaction, we could register an ApplicationListener that exposes API to easily verify events being triggered, event listeners being triggered etc.

Rule verification via APT

Assuming we’re able to get an APT implemented that’s run on top of the current code base, we could run the aforementioned verifications and issue compiler errors for violations.

Existing tools

  • ArchUnit — Tool to define allowed dependencies on a type and package based level, usually executed via JUnit.

  • jQAssistant — Broader tool to analyze projects using a Neo4j-based meta-model and concepts and constraints described via Cypher queries.

  • Structurizr — Software architecture description and visualization tool by Simon Brown. Includes Spring integration via automatic stereotype annotation detection.

Appendix A: Appendix

Further resources

  • [safd] Simon Brown — Software Architecture for Developers (Books, Website)

  • [sos] Oliver Gierke — Refactoring to a System of Systems (Slidedeck, Recording)

  • [whoops] Oliver Gierke — Whoops, where did my architecture go? (Webpage)

Glossary

Named Interface

Given a module, a sub-set of types that constitute the API of the module, i.e. candidates for referral by other modules.

Release instructions

  • mvn versions:set -DnewVersion=$version -DgenerateBackupPoms=false

  • Change /scm/tag im pom.xml to $version

  • Commit against release ticket id

  • Tag commit

  • Push commit and tag

  • mvn clean deploy -Psonatype

  • mvn versions:set -DnewVersion=$snapshotVersion -DgenerateBackupPoms=false

  • Commit against release ticket id with message "Prepare next development iteration."

  • Push commit.

moduliths's People

Contributors

codecholeric avatar dependabot[bot] avatar dirkmahler avatar mschieder avatar odrotbohm avatar olibutzki avatar rbuer avatar thombergs avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

moduliths's Issues

Define dedicated API classes and/or package(s)

I think using the base package of a module as the API package is too much of a constraint.

Let's say we have two modules that share the same database. ModuleB provides some persistent data structures in the form of JPA entities and Spring Data repositories. ModuleA connects its own entities with those of ModuleB via relations (@OneToMany and such).

In this case, we would have to put the JPA entities into the base package of ModuleB which would be awkward and create a lot of dependencies within ModuleB to the base package. Also, it might be that the base package contains a lot of stuff in the end, which I would rather have structured in sub-packages.

Instead, perhaps we could define an annotation that describes which classes or sub-packages are part of the API. Or extend the @module annotation somehow. A feature like that would allow more flexibility in structuring modules.

Add ability to bootstrap upstream modules

Sometimes it might be desirable to not only run an individual module standalone but also it's direct dependencies or even all upstream modules. We should allow a bootstrap mode in @ModuleTest to define which scope of modules is supposed to be run.

Upgrade to ArchUnit 0.9.1

Requires switch to archunit-junit4 artifact (instead of …-junit). Requires to adapt test case in modulith-sample as the exception message has changed.

Add support for JUnit 5

As JUnit 5 now supports meta-annotations, we could equip @ModuleTest with JUnit's annotations so client code doesn't need an @RunWith etc.

@sbrannen's input appreciated. Anything else you see here?

Document Modulith Approach vs. Jigsaw Approach

At some point, people will ask what the difference is between Java's module system and this library. The differences between both approaches should be documented to answer these questions.

Spring Data repository of another module is loaded into @ModuleTest ApplicationContext

Using @ModuleTest, Spring Data JPA repositories are loaded into the ApplicationContext, even if they are located in another module.

I guess this has to do with the Spring Data auto configuration always loading all repositories or none.

It would be nice if the ApplicationContext would not contain Spring Data repositories of other modules than the one under test in order to make the dependencies explicit.

See failing test in https://github.com/thombergs/moduliths/blob/spring-boot-sample/moduliths-sample-spring-boot/src/test/java/com/acme/myproject/booking/BookingModuleTest.java

Improve dependency diagramming options

Currently the PlantUML diagram created contains all dependency types: listens to, depends on and uses. As this could create rather complex diagrams, we should allow multiple diagrams to be created.

Spring Data repository from an internal package is visible

In a module test, Spring Data repositories are added to the classpath even though they are located in an internal package.

See commented out lines in unit test https://github.com/thombergs/moduliths/blob/spring-data-sample/moduliths-sample/src/test/java/com/acme/springdata/moduleB/ModuleBTest.java.

This is because the parent package of the internal package is scanned for Spring Data repositories and thus the repositories in the sub-package are also found.

Perhaps this is something we have to live with since I can't think of a way to exclude certain sub-packages (except perhaps by explicitly marking them as internal with an annotation?).

Provide a code formatter

Should we check in a code formatter to the project? I always hate guessing all the options and IntelliJ can import an Eclipse code formatter quite well... So if you add your formatter @olivergierke , I'll gladly switch to it 😉

Spring Data repository from other module not found although it should be visible

Now that the module-scoped package scanning for Spring Data repositories is in place, it's actually too strict because only the package of the base module is scanned. Packages that are exported from other modules are not scanned.

See test case https://github.com/thombergs/moduliths/blob/spring-data-sample/moduliths-sample/src/test/java/com/acme/springdata/moduleB/ModuleBTest.java which is part of PR #19.

In this test case, RepositoryA lies in the main package of moduleA and thus should be visible in a module test for moduleB.

Implement an annotation processor that issues compiler errors

As suggested by @olivergierke, an AnnotationProcessor checking the module dependencies at compile-time would be awesome.

Implementing an AnnotationProcessor is easy enough. However, the current model classes (Module, Modules etc.) depend on ArchUnit to do the dependency checks. As @codecholeric pointed out, ArchUnit needs compiled classes to work, which are not yet available during annotation processing. At annotation processing time we only have access to Java's Element objects which contain information about the classes about to be compiled (TypeElement, PackageElement etc.).

So, we need to find a way to do the dependency checks based on those Element objects.

Maybe we can map those Elements to ArchUnit's domain objects and then use ArchUnit for the dependency check after all? Just a naive thought, I haven't really thought this through... .

Improve handling of event listeners

Currently Spring application event listeners are not properly handled by the modules abstractions. Take the following structure for example:

com.acme.orders
+ ….Order (publishes OrderCreated events)
com.acme.accounting
o ….OrderEventListener

Further assume that OrderEventListener looks like this:

@Component
class OrderEventListener {

  @EventListener
  void on(OrderCreated event) { … }
}

As we currently only inspect type dependencies and interpret them as indicator for a downstream module dependency this scenario results in accounting to have a module dependency to orders. This is clearly wrong. I suggest to alter the model setup to change to the following:

  • When collecting the module dependencies, explicitly check for modules annotated with @EventListener and make the nature of the dependency explicit (listens-to).
  • Do not consider this type dependency a module dependency or at least don't include the module referred to into the ones that need to be included when bootstrapping upstream dependencies.
  • The additional meta-information can be included in the diagrams created in #16 to distinguish between the different module relationships.

Add ability to define shared modules for a modulith

Shared modules contain code that needs to be included in every test bootstrap, no matter what. Example use cases are modules that contain Spring components automatically picked up by Spring Boot like JPA attribute converter or Converter implementations that are registered with Spring MVC's ConversionService.

Provide module-scoped equivalents to @WebMvcTest and @DataJpaTest

With @WebMvcTest and @DataJpaTest we can easily create an ApplicationContext for the web- or data layer of our Spring Boot Application.

It would be nice to have equivalents to these annotations that only include the web- or data layer of our module under test into the ApplicationContext.

Same goes for the other annotations like @WebfluxTest, @JdbcTest and so on (see https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-testing.html).

Spring component detection should consider all @Bean methods

The Spring component detection currently filters the results we get from @Bean methods to the package under inspection. I.e. all beans that do not reside in the module are not listed as Spring beans (e.g. infrastructure components like DataSources).

We should make sure that return types of @Bean methods are considered no matter what.

Migrate a reasonably-sized application to a Modulith

In order to get some real-world experience, we should migrate a Spring Boot app with more than a couple hundred lines of code to a Modulith. For this to work, the library must have matured a bit and at least a snapshot should be released to Maven Central or JCenter.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.