For quite a while (as in, for over a decade) I've been thinking about the future of my WebObjects-based software. While I love WO, it's now been obsolete and unmaintained for 15 years and as technology evolves, patching it through Project Wonder isn't really fun and can only keep going for so long.
So, over the past couple of years I've been working on a side project to create something to use as my WO replacement. Although it's heavily based on WO, it's a clean room implementation done through 25 years of knowledge of the WO APIs and with some help from the Project Wonder sources. The most egregious examples of theft are probably that the template parser is Project Wonder's WOOgnl parser (copied after asking Mike Schrag nicely) and I shamelessly stole WOLifeBeatThread since, like most people, I suck at socket programming.
Amazingly enough though, this thing is starting to work! It's actually fun to work on this and even if I don't consider it "ready" I've started using it in some of my own projects, dogfooding being an essential part of making something nice. But, it's not finished and below is something of a brain dump; random thoughts and incoherent ramblings about parts of the framework that need some thinking and thus what I'll probably be working on over the next few months.
It's not a full list, if you look for FIXME and CHECKME in the sources, you'll find plenty. Oh yes, plenty. But over the past years, issues have been changing from problem issues to design issues. The framework "works" so I'm at a lovely stage where I'm just deciding how it works, not if.
My plan is to replace WO in all of my software with this framework. Even at this early stage, it already feels really nice to be able to fix and improve one's stack without having to resort to hacks and patches in Project Wonder. I'm aiming for mid to late 2024 to start porting business critical projects and client projects, and really tell other people about this. In the meanwhile, I'd love if interested people try this out and give ideas and feedback.
I sometimes wonder if it's a mistake to make changes from WO. I could just write a plain WO clone, it's quicker, mentally easier and makes it simpler to port existing WO applications. But I think that a better framework is worth the added bother and the few extra hours to port an application. While WO is awesome, the world, web, HTTP and Java have evolved and we've learned a thing or two in the last 20 years.
Besides, creating something is more fun than just copying something.
As should be apparent, my mindset for this project is more "get it right" than "hurry up". I've used WO for a quarter of a century so spending a couple of years on good design now won't hurt too much.
In the end, I want something that feels like WO when I first experienced it: Delightful, fun, well designed, easy to use, intuitive and just plain awesome. I don't feel like these terms apply to a lot of today's web development stacks. I want something so fun and intuitive that my 10, 11 and 13 year old kids can use it and love it. Remember creating a WO website as a kid in the 90s? Yeah. It was awesome. Mostly.
An ng-objects application currently uses the standard maven project layout, with a couple of additional folders in src/main/resources for component templates, application resources and webserver resources. I'm not sure if this will be the final project structure or if we'll want to reinvent NSBundle in some way or form. It's convenient to use the maven structure for now though, since it means we need no custom logic to locate resources or perform a build. It's a plain, standard java application and everything's on the classpath.
An ng-objects application can be built using the vermilingua-maven-plugin which is really just a plain maven (as in, no ant) version of wolifecycle-maven-plugin that can also be used to build WO applications.
vermilingua can build any java project as a runnable application. The only thing you need to add is a build.properties file to your project's root with principalClass=[your app's main class]. An application built like this looks pretty much the same as a WO application and can be deployed to a WO environment and managed by wotaskd/JavaMonitor.
This is familiar and convenient for WO developers. However, at some point I'd like to look into the build process and explore using jpackage which can package applications and bundle a JVM tailored to the application.
ng-objects applications look pretty much like any old WO application to wotaskd/JavaMonitor and can therefore be managed and monitored by them and served through the WO Apache HTTP adaptor. We currently only support starting and stopping apps and seeing if they're alive (Lifebeat) but more advanced functions like refusing new sessions will probably be implemented soon, as we add support for those functionalities.
Since ng-objects templates look like WO templates, the WOLips component editor works pretty well for editing NGComponents, with only a couple of glitches. The choice I'm now facing is:
I'm currently leaning towards the third option. But there's also the option of looking into supporting other IDEs like IDEA or Visual Studio Code. This would be a good option to appeal to people more "hip" and "cool" than me. But starting out with WOLips is probably wise, because it's already there.
Like WO, NGApplication has a global singleton accessed via the static method application() . Having a global application object is not all that nice, most importantly it means you can only have one NGApplication instance available at a time. This can be a problem, not least during testing, where you might want to construct multiple application objects with different properties/configuration.
So. The goal is to get rid of the global NGApplication object. The sole reason I haven't done so already is that I haven't entirely decided how. NGApplication, like WOApplication is very central to the framework and functions in a way a little like a factory/injector with all it's create() methods while also handling a lot of configuration etc. But I'm still not sure we want to use actual dependency injection.
Anyway. The point is that the global NGApplication.application() will probably go at some point, it's just a question of when and how.
NGApplication has an NGProperties object containing it's configuration accessed through application.properties(). There are no static methods to fetch property values like on NSProperties/ERXProperties, that's due to the previously mentioned goal of being able to run multiple NGApplications with different configurations within one JVM.
There are also a few improvements to the property mechanism which I'll detail better later, it's an entire article really. Configuration is an important core feature and just needs to be great.
Having modes like the "Development mode" which ERXApplication provides with isDevelopmentMode() is nice. So we have that method and it's value is currently just based on the -WOMonitorEnabled property.
I'd still like to improve on this. Software usually has multiple modes (Production/Testing/Staging/Local etc) and these will differ between products/environments. Configuring "modes" should therefore be thought about as a whole and though I haven't thought it through, we might end up with something along the lines of application().mode().is( Mode.Development ).
Logging is currently handled very simplistically using slf4j-api. Like WO, ng-objects writes logs and other output to the location specified by -WOOutputPath or System.out if no output path is set.
We currently don't have any other monitoring, but that's something we want to look into as it's simply hugely useful for web applications. We'll probably also want to revisit the use of slf4j and either use Java's own logger and/or create our own NSLog style abstraction, allowing applications to plug into JFR or other monitoring/event recording tools.
WO has the concept of "request handlers" which we've expanded a bit and now call "routing". It's still very primitive and we currently only have:
addRoute( "/some/page", someHandler )addRoute( "/wo/*", someHandler ) (note the wildcard)So routing needs some work to become good. And it will get some work soon, because this will be a pretty central part of the framework.
Frameworks like Spark, Javalin and Helidon have nice routing concepts we can learn from. And this is something that's worth getting right from the start since it would be irritating to silently wreck people's applications with routing syntax changes.
In a way, routing replaces Direct Actions. There's a rudimentary implementation of Direct Actions in ng-objects, mostly because DAs are easy to implement, and for compatibility with older software. But I'm not sure if and in what way these will be in the final product. Exposing class and method names in URLs feels iffy and writing an actual route will always be the nicer way to create static URLs.
The template parser in ng-objects began as a pure copy/paste rip-off of Project Wonder's WOOgnl-parser, although it's seen some changes and cleanup over the years I've been working on this. Like WOOgnl, it handles both single file templates (HTML-files using inline bindings) or the good ol' .wo (html/wod) combination templates. And there's no actual OGNL. If you need logic, write it in Java you bastard.
If there's anything I'd like to improve in this area, it's error reporting. Specifically, I'd like to parse a broken template in it's entirety and report all it's errors at the same time, rather than aborting parsing at the first encountered error. We should also show the template's source and highlight the errors, like we do with Java code in our exception page.
Template/DynamicElement rendering was definitely the most challenging and fun part of the project. And I think we can make it better.
Needs better error reporting. We all know and love WO's kilometer long stack traces from endless nested WOComponentReferences that really should include more context, so we might actually see where our errors originate. Unfortunately, ng-objects' stack traces currently look very similar to WO's and I'm considering ways to improve them. Ideas welcome!
With the advent of virtual threads in JDK 21, I'm also a little excited about exploring multi-threaded rendering. In theory, each component/element could be rendered in it's own thread and the results assembled at the end of the R-R loop. This means automatically multi-threaded web applications, which not only sounds cool but is something I want. It might even be enough to just apply threading to the pushing/pulling of binding values, since that's usually where processor-intensive method invocations will happen. Rendering isn't that hard, obtaining the stuff to render is usually much harder.
However, threaded rendering would require some API changes since NGContext is almost criminally stateful/mutable and thus not very threading-friendly.
(at this point — are you seeing how much fun this is? To no longer be bound by a framework we can't change, so we can do anything we want?)
A final thought on templates: While our bindings/associations currently work like WO's and use KVC/reflection to resolve keypaths to fields and methods, we might want to consider looking into "compiled templates", i.e. compile the templates to java classes before deployment. The reason I'm considering this is that GraalVM has been picking up pace lately and it does not agree with reflection.
This is something I'd consider cool, but it's definitely not a priority.
"Namespace" is our term for what WO usually calls "frameworkName" in dynamic element bindings and APIs. As we know, WO has:
app/null for application resources[frameworkName] for frameworks[global namespace] for dynamic elements, components and DirectAction classesI'm currently using more or less the same pattern, but I think that "global namespace" really needs some thinking. It's very existence is questionable.
There's currently a very simplistic implementation of something called NGPlugin . At application startup NGApplication will use Java's ServiceLoader mechanism to look for provided NGPlugin classes, instantiate them and invoke a load() method on them. Works fine but still very basic, and we'll probably work on this a bit more.
We currently have no localization support and I'm still thinking about if we should. I used WO's localization features two decades ago, but I eventually just stopped using them and implemented my own localization where needed. But, as always, hearing opinions on this matter from people that use WO's localization successfully today would be nice. Localization is indeed a thing so it might be something that's worth taking a second look at.
We haven't implemented any foundation classes. Java has a fine collection of base classes.
OK, so I lied. In WO, KVC is a part of foundation and we need that for keypaths in template bindings. The implementation is rudimentary but still improves on WO's KVC in some ways, for example it works with internal classes like java.util.ImmutableCollections$ListN generated by, for example List.of(). There's a lot of work left in cleanup and in implementing KVCs "type munging" where, for example, if a Double value gets assigned to a BigDecimal field, an automagical conversion is performed. A little weird, but just so darn convenient that I think we'll keep it. As long as the conversion is not lossy, that is.
Our KVC implementation does not support @ operators, like "$products.@sum.price" or "$employees.@sortAsc.name". They're not hard to implement, but I'm hesitant about whether to do it since operators feel like logic leaking into templating, and I'm a firm believer in separating logic and templating. But operators can still be convenient, for example during prototyping (which is an area where WO shines). So still considering this.
Like WO ng-objects applications know very little about HTTP. They only know NGRequest/NGResponse and use "adaptors" to translate these from and to HTTP messages. So, the only real function of an adaptor is accept an HTTP request, create an NGRequest, submit it to the dispatchRequest() method of NGApplication and then hand it's NGResponse object to the client.
Our only HTTP adaptor is currently a small servlet served by Jetty and provided as a separate module. It's built in a standalone/modular way, i.e. ng-objects does not depend on Jetty or the Servlet APIs in any way. And since the adaptor is really just a plain servlet it can be served by any servlet container.
I'd love to write an HTTP adaptor from scratch at some point, but for now the Jetty adaptor works really well, HTTP is messy, and writing an adaptor is not a priority. About the only irritating thing about Jetty is that it's an additional dependency.
Not supported yet but something we definitely will add support for. Mostly an API design issue.
Not supported yet but something we'll add. The first step is to get to know the protocols so I can start thinking about how the APIs should look. And we'll have to ensure that the HTTP APIs are designed in such a way that changing HTTP versions is easy and won't cause any surprises. You want to be able to flip a switch to work with the new protocols transparently, while also getting access to the protocols' improvements.
Our HTTP APIs are a total ripoff of the WO APIs. And while that single WO-style NGResponse class is nice, I'm thinking it might be a good idea to replace it with an NGResponse interface and multiple implementing classes (streaming response, string response etc.). I haven't touched the HTTP APIs a lot yet though, since I think changes to these APIs will have to be thought out alongside the work on WebSockets and newer HTTP versions.
Small thing, really — but a few methods in our API change the order of arguments from WO.
A relic of the WO API's Objective-C origins is the somewhat untraditional ordering of parameters in methods that take a "key" and a "value". NGKeyValueCoding is an example, but in that case I decided to keep the same order as NSKeyValueCoding since method names like takeValueForKey helpfully inform you about the order. So little harm there.
But in some places, that ordering just doesn't work, a notable example being WOMessage.setHeader( value, headerName ). This parameter order (value before key) will absolutely confuse 99% of the world's developers, so NGMessage defines setHeader( headerName, value ) instead. I'm still thinking about the best approach for this though, since both parameters are Strings, meaning that lazily ported WO code will silently fail (and I've already been burnt by that myself). Perhaps we should rename the method to prevent silent failures like that. Still under consideration.
Work on Ajax.framework style functionality is being experimented with in the ajax-experiments branch of the main repo. It seems it won't be hard to implement so I think it's more of a design challenge than a technical one. An important area that I really want to get right.
We barely have any test coverage at the moment. This is primarily because the software's design was (and still is) very fluid and updating unit tests when doing big refactorings gets boring very quickly. This is a little embarrasing though, and we're approaching a stable stage so proper tests will soon be added and become a part of the development process :).
ng-objects is currently not "modular" in the Java/Jigsaw sense, for pretty much the same reason we don't have tests. A fast changing design makes it bothersome to keep module-info.java up to date for every refactoring. However, modularization is something we'll have. I've experimented with modularizing ng-objects allowing builds to generate customized JVMs using jpackage . And It's really nice.
The Java team is hard at work finding a solution to Java's "null problem". Since ng-objects's APIs are in large parts inspired by WO, it blatantly uses null as parameters and return values. But as we know, that's a potential source of problems. And I'm not a big fan of introducing Optional because it just … isn't that great.
So, I'm waiting a little to see how the Java language evolves here. I mention it since it might have an effect on API design, but if nothing happens in the next year or so, I don't think I'll wait for it.
ng-objects does not have anything to do with ORMs or databases. It's a web framework.
That being said, nothing prevents your EOF code from being used with it. I use Cayenne myself which is very EOF-like and I love it. I've been writing and using a Cayenne-based companion framework for ng-objects called ng-hafnium which is a somewhat CRUD-y and a little D2W-y framework based on something I've used in my WO projects for years. I like it a lot so that project will move forward.
But I wanted to mention EOF specifically because EOF code tends to form a huge part of many peoples' WO applications and their domain logic. And yes, you can use it with ng-objects. But I sincerely recommend migrating to Cayenne. A migration is usually easy, it's nice to use and under active development, the guys working on it are absolutely the best and most helpful people and — it's just all around awesome.
ng-objects isn't really a catchy name and I don't really know why I called it that. So... Help!
So, that's it for now. I warned you that this post would sound like the incoherent ramblings of a madman.
As you can see, there's a whole lot happening in my head when it comes to this project. And this is just a tiny part of it. I spend quite some time working on it these days and I'd absolutely love your ideas, thoughts and feedback.