JSR 277 Early Draft Review
JSR 277: Java Module System
Copyright © 2005 Patrik Beno
Revision 160, saved on 25/10/2006 02:30
Initial draft (published)
Some typos fixed
Table of Contents
This document is an attempt to review and comment work of the expert group presented in the initial early draft of the specification published on October 11, 2005.
I assume readers are familiar with the JSR 277 (and therefore I will not explain or quote it more than it is necessary).
My impressions are three‑fold:
There is some great stuff in this JSR
Some stuff is missing
And finally, I see some design flaws, some of them real hard showstoppers.
Now if you’re ready, read on.
One of the most important statements of this JSR is the fact that it actually obsoletes and deprecates current class loading scheme.
Original class loading model is a hierarchical view on software layers, which is inherently wrong and insufficient when it comes to component‑based architectures. For a long time this model crippled our natural dependency modelling attempts and approaches, ever since the first IDE introduced module/project dependency management features.
The runtime deficiency bothered us with an impedance mismatch as we struggled with various sort of silly problems like class loader isolation, unsatisfied and/or conflicting dependencies and so on. Various different concepts were introduced in order to help us at least partially solve or workaround these problems (some of them both funny as well as dangerous, let me mention at least the infantile parent‑last classloading scheme).
This JSR finally has its chance to fix these problems by introducing the runtime model which is real native directed graph which natively supports true component architecture (at least from dependency management perspective).
Due to redefined class loading model, different versions of the module can be loaded by the module runtime system. Classloader per module is a smart decision that allows this. The problem itself is not that simple at all but the good news is that the model does allow this. (Bad news is it does not specify how exactly this would be supported. Read later on in section “Flaws”.)
You know Maven repositories and how they work, how they are linked together? Now imagine module system can do all and give it to you this at runtime.
This, I think, is effectivelly death of CLASSPATH, WARs/EARs etc.
Actually, this feature makes repositories the successors of the old‑style classloader delegation model. And this is where this model does really make sense.
To highlight this without much talking, I really appreciate dynamic import policies (section 2.7.2) which provides to programatically configure module dependencies when the built‑in declarative approach is not enough for your problem.
While this may introduce some serious implications on manageability of such dynamic imports, I think this is great feature, just because that’s what software should be: dynamic interacting pieces (I’ve never really liked any “final” models or designs).
Just for the record, full review of this JSR (277) is not possible unless all its dependencies are fixed and known (JSR 294). Last time I checked, there was not too much to be seen there.
I have not yet reviewed JSR 291 but I think it (especially its module system) may be a huge overlap with this JSR (277). I don’t understand why there is no sign of any cooperation between these expert groups (not counting single document reference in chapter 10 of JSR 277 EDR).
Module declares dependency on API (e.g. JMS or JDBC) but does not specify any runtime implementation. However, it may not work properly without the expected contract to be fulfilled at runtime.
I would expect JSR such as this one will
define how the runtime expectations like this would be declared (expressed)
define how how these expectations should be resolved at runtime
Maybe the note C.4 (appendix C) is supposed to refer to this problem.
Understanding the Difference
While static dependencies are satisfied via module class loader hierarchy, runtime dependencies are resolved with respect to runtime context (context classloader).
JSR does not specify how the module-aware compiler should compile the compilation unit (module sources)
When some compilation unit (development module) is compiled, the compilation classpath (to put it in the old terms) is composed of
Module direct dependencies
Plus the dependencies exported by these direct dependencies
Compiler uses dependency imports and version constraints to compile development module. The resulting artifact (JAM) should include exact version numbers of the dependencies that were used to build the given module.
This is required for future reference when potential runtime problems (conflicts) need to be resolved.
JSR does not seem to see a module as a dynamic component with its own lifecycle
Init() - Module is initialized (module as a software component, not just module definition)
Destroy() – unload module when it is not used
Again, C.6 is a closest note related to this issue.
(No particular order!)
Sample scenario 188.8.131.52 describes situation when two different module versions are loaded.
This may or may not introduce a class incompatibility to the whole system. It may or may not be regular working deployment. It strongly depends on the actual software architecture and the object/type flow.
The early draft does not offer any solution to this problem. It does not even mention this issue at all. This is not an easy problem at all but unless the JSR provides any reliable solution, it should avoid introducing or proposing any features that could possibly break the runtime compatibility and type safety.
In the mentioned scenario, modules B and C may safely use different version of module D (say, some xml parser library) if they use it internally only and they do not share instances they create.
However, if they share such objects (like returning parsed XML document to module A), there’s nothing that can prevent the module A from trying to pass such instance to module C where the class incompatibility causes an error. In fact, such error may occur virtually anywhere on the object’s way from B via A to C and D.
Understanding the Problem
The problem is that (a) while module dependencies specify mostly direct type interactions between modules, (b) type isolation is determined by data (instance) flow between software layers.
If this problem can be practically solved at all (and I believe it is because software is layered), the module system must impose a set of architectural restrictions on modules and provide some kind of managed environment for them. The limitations of a contrained managed environment may be too restrictive to make this practically usable.
This is a very complex issue that needs much deeper analysis.
Robust system need to choose level of granularity on which it will operate and hold to it. It has to introduce an entity for which contracts and relashionships are defined. Such entity might constitute other higher‑level entities if needed but this entity should not be broken down to smaller pieces.
As for the module system this JSR is trying to define, I believe this entity is obviously a module itself.
Class exports break the granularity rule mentioned above. For module system itself, there is no such thing as class or resource. The only thing that matters is the module.
The module should not be required to explicitly export fragments of its internals (classses or resources). It is not transparent, it is error prone and most important, and it violates the principal idea of module system.
Module may or may not export its dependencies (other modules). However, if module gets imported, it is imported as a whole, no exceptions.
If module needs to hide parts of its internals (like private API implementation), it may export them into separate module and import such a module as a private unexported dependency.
This requirement may seem too restrictive at first sight but it just encourages proper module‑based architecture without the backdoors and hacking or breaking down the elementary building blocks.
In the above example, class B is exported by the module but class A is not which makes the class B actually unusable (I may be able to access class B but I am unable to call method doSomething because the required type is for me unvisible)
Various use cases – various exports applicable
It sounds like a good idea to export only API and hide the implementation classes. At least form the regular client point of view.
But what if you want to extend the default implementation and plug it into the particular framework runtime? The Impl class is not accessible because it was hidden (not exported) by the module developer who thought it was a good idea to expose only API classes.
If JSR 277 expert group is not willing to abandon the idea of class/resource exports, at least the approach should be reversed: use explicit DENY instead of explicit PERMIT.
Attempt for an illustrative example: If I am to secure CIA building or a computer exposed directly to an internet, I choose the strategy of implicit global DENY followed by explicit adressed PERMITs.
If the subject to be secured is a public park, library or a collaborative web forum, more useful (practical) approach is a global PERMIT system with the exeptions expressed via explicit DENYs.
I hope I did enough to make my point.
I thought no one can ever really consider idea of actually encouraging and supporting cyclic dependencies between modules, yet here we are, JSR 277 really enjoys this masochistic nonsense (see 8.1.2, 8.3.4).
Now let me make myself clear:
It is with no doubt useful and necessary to have support for cyclic dependencies between single classes. Without this feature, we would never be able to model parent‑child relationship as properly and effectivelly as we are now.
However, this is where tolerable support for cyclic dependencies ends. The acceptable boundary is clean: compilation unit.
Please realize that you cannot actually compile one class without another. You need to have sources for both Parent and Child on source path and compile them as a single compilation unit. Compiler must do special effort to parse and resolve this cyclic dependency.
Ok, who cares?
Now imagine what this means for module‑aware compiler/build system. It means that you need to compile all modules bound together via cyclic dependency as a single compilation unit => as a single module. Then, compiler/build system must artificially split the resulting output (classes) into separate folders and/or jar/jam files in order to make this monolitic crap look like a modularized architecture (which it is NOT, actually; it is just single source tree packaged in multiple jar files which are effectivelly usable only as a whole).
Still don’t get it?
No suppose you have versioned modules. There is no clean way to introduce backward incompatible change to a single module in a cycle. You cannot update single module without actually rebuilding the others in a cyclic dependency. You would need to rebuild and update version number for all modules in a cycle. This is required because of how the cycle is built (single compilation unit).
Oh, hell, yes, I know, the real hacker will still find the way, but that’s not the point, is it?
Point is that cyclic dependencies between modules actually point out some serious flaws in the model/architecture. And this should never be encouraged, nor tolerated, nor supported by the module system, ever.
Cyclic dependencies between modules are evil because
1) They support and encourage bad practice, bad models, and bad spaghetti architectures.
2) Building such modules becomes more complex and error prone.
3) Dependency management/resolution for module cycle becomes less transparent, more complicated and less robust.
4) They just are (or will become) pain in the ass, sooner or later, and will hunt you down when you expect it the least.
The whole approach to versioning seems like we’re missing something needed to deeply understand the problem we need to solve.
The whole thing seems to be bacwards, it is just wrong.
While I do agree with versioning scheme standardization, I think that with respect to module system, only following two things are important:
What is the developer trying to say by specifying the version constraint?
How can module system understand and satisfy this?
In general, I see only two scenarios:
Static: I am conservative and I choose exact version of the module. I do not want any automatic updates.
Dynamic: I choose module version based on what contract it fulfills. I expect any future updates that are compatible (fulfill the same contract), to be aplied automatically (no matter what version number it is).
For ‘static’ scenario, there is no real job to be done by module system. Provide the required version or fail.
For ‘dynamic’ scenario, module system must
Select list of versions that are compatible with the required version
Negotiate the list with other modules requiring the same dependency
Pick up the most recent version from resulting list
Now how does the module system select the compatible versions?
The answer is in version history. Imported module must specify two things:
Its version history (whole path to the root, up to the version 1.0 or whatever the initial version was).
Oldest version number (from the history) this new version is compatible with.
This simple information is
enough to make any version comparisions with no extra effort
enough to support any versioning schemes, even the plain code names
enough to make the most reliable compatibility decision possible
This scheme is the best because it does not rely on client (importer) assumption about the compatibility of any future versions of the imported module. Instead, it consults the only imperative source: the module version history itself.
Module author specifies
Version history (up to the initial release)
Oldest version this new version is compatible with (must be listed in version history)
Importer (client) specifies
Exact version number of the module. (Missing version means newest, most recent version available)
Classifier that says whether this import statement is static or dynamic. (Default would be dynamic)
I am really very optimistic about this JSR and I hope and believe it will radically change the way we write, build and run software.
However, the long term goal we want to achieve must not be lost from the radar nor can it be distorted by the pseudo‑requirements for quick‑and‑dirty “solutions” wanted by many ‘fast‑food” developers.
The java module system should not be meant to allow people to do quick and dirty hacks. For this purpose, features like “java –classpath lib/*.jar” are provided, and they have their reasons (and right to live).