David Ross's Blog Random thoughts of a coder

Although there has been a mind blowing amount of technological change since the beginning of the PC revolution when it comes to building enterprise applications the industry has moved through a small number of distinct architectural patterns.

• Compute power on the client has been continually increasing leading to a migration of processing activities from the mainframe, to the application server, to the desktop or mobile device
• Reducing deployment complexity by shifting from think clients, to web and recently to App Store delivery models
• The quality and depth of functionality available in libraries, frameworks and platforms has meant that smaller teams are able to deliver more complex systems

This post documents the main enterprise architectures that have been utilised over the last twenty years.  By placing the system under review into one or more of these categories will enable the reviewer to quickly hone in on possible architectural flaws and forms the foundation of the review process.

Client Server

Client server based applications such as those developed with Access, Power Builder and classic Visual Basic have client application directly accessing a shared resource such as a database. Unfortunately users would realise they could bypass the GUI and interact directly with underlying database leading to data corruption and/or security issues. While IT departments no longer create Client Server applications Excel spread sheets that source data from a database are effectively equivalent.

• Client software connects directly to database
• Without a layer of stored procedures between the client and the database the design by today’s standards is intrinsically insecure
• Stored procedures need to perform authorisation checks as well as requested business operations to be secure
• Difficult to refactor database as simple changes would affect multiple stored procedures
• Extremely difficult to scale since
• Hard to utilise a caching layer as clients aren’t channelled through a shared service
• Lots of separate connections to the database as connection pooling isn’t utilised

3 Tier

It quickly became apparent that having the client applications directly access the database was problematic. There were performance issues due to the slow client machines available at the time deployments suffered from DLL hell. Simple Line of Business applications starting moving to HTML and more complex GUI’s utilised COM+ to centralise database interactivity to the middleware servers.

• Client software connects to an application server
• Thick clients connect to COM+ or Web Services
• Browser loads HTML pages fully rendered on the server
• Application server is responsible for all connectivity to the database
• Application server is trusted
• User Authentication and Authorisation rules can be pulled out of the database and into the middle tier
• Database tables can be interrogated directly as opposed to utilising stored procedures and still be secure
• Interop issues
• COM+ based solutions were incompatible with Java
• Original Web Services frameworks generated poor quality code and system to system integration proved more difficult than industry expected

N-Tiered

As Web Pages became more complex having rendering code intermingled with database queries became unwieldy. Presentation Patterns were developed that would take a “model” and convert it into XML or HTML. Business Services would interrogate the database and return a “model”. This layering enabled multiple screens to utilise common business components.

• Simplifies development be splitting the software on the application server into logical layers
• Code to render the presentation layer is separate to the code that implements business rules
• Performance increased by adding new layers such as caching
• Often business components will “know” how to retrieve themselves from the database and how to save themselves
• Can’t test application logic without a database
• Can’t change persistence layer easily
• Some systems wrongly placed application layers on separate machines
• Web Server talks to an App Service talks to database
• Latency increases
• Transaction management extremely difficult

Rise of Object Relationship Mappers and Domain Driven Design

A problem with N-Tier architectures was that the Business components had a deep understanding of how they were persisted to the database. For example a request would come into a Web Service that would then call a Business Component, who would then call into a Data Access Layer, which was built using the Table Gateway Pattern. In the Table Gateway Pattern the developer would expose Select, Delete and Update methods for each table within the database and the classes in the domain would map almost exactly to a database table. Thus the business domain components were extremely database centric. Performance issues would arise when data straddled multiple tables as the Table Gateway Pattern struggles with this type of topology leading to either many views being created or the “SELECT N+1” anti-pattern. As Object Relationship Mappers and Domain Driven Design became widespread the shape of the data between the domain layer and the database started to diverge. Domains built with “Plain old Java/C# Objects” POJO/POCO removed the dependency on the database eventually leading to teams utilising NoSQL and Event Sourcing in their designs.

• POJO and POCO classes
• Domain objects are inflated externally by a repository or in the service layer
• Domain objects should be persistence layer agnostic
• Domain objects should not “know” how to connect to database
• Domain objects should only contain data, business rules and validation
• Enables unit tests to be simpler and not have database connectivity
• Once connection to database is broken developers start utilising other persistence stores such as No SQL, files, Web Services etc

Use of Open Source frameworks

Throughout the industry enterprise software development teams found that they were writing plumbing code to fill in gaps in the programming environments that they were using. Eventually Open Source projects have become so advanced and reliable that the entire development stack is now open source for many platforms. Legacy code can often be “modernised” by gradually replacing bespoke plumbing components with the Open Source equivalents.

• Teams no longer wrote cross functional libraries and instead use industry standards for:
• Logging
• Database connectivity
• View Engines
• Aspect Orientated Programming
• Inversion of Control

Continuous Integration and Deployment

While Continuous Integration is not an enterprise pattern the use of automated unit tests and deployments is a distinct stage within the progression of enterprise application architectures. Test Driven Development, Test First Development and Acceptance Test Driven Development all promote the creation of decoupled independent components. Due to the automated regression suite developers are able to make large systemic changes with relatively low risk leading to simpler more maintainable systems that will last longer in production. This has led to shorter development cycles and higher quality solutions.

• Automated builds
• Automated deployments
• Automated tests
• Developer created
• User created

HTML5 and mobile

While HTML5 can be described as the latest HTML standard where it provides richer access to the sensors on the device, higher speed graphics and the ability for applications to function in an occasionally connected environment for many development teams however it is the “line in the sand” where supporting older browsers is no longer necessary. Further Chromes decision to auto update has also contributed to all browser vendors to release far more rapidly and modern frameworks degrading gracefully when running on older browser are also contributing to making web development far easier.

The biggest failure of the Java desktop was the design choice of “Write Once Run Everywhere” which led to the decision to only support controls that were available on all of the platforms that run Java (UNIX, Windows and Mac). Unfortunately this inevitably led to a poor desktop experience on all platforms. Unfortunately teams are now going through the same dilemma today with mobile where HTML5 only provides a subset of the features available on Andriod, IOS and Windows Phone and has slower performance than the native frameworks. Further the look and feel of each of the platforms is significantly different and a web application can look out of place or be confusing for users. This has led to many teams building having to build multiple versions of their client application. This in turn further reduces the role of the Web Server to simply providing data to the client which is solely responsible for the rendering.

• Fast CPU and GPUs on clients means processing can be moved off servers and onto the clients
• Middle tier exposes data and services as opposed to rendering screen layouts
• Model View Controller (MVC) and Model View ViewModel (MVVM) frameworks with two way data binding reduces hand crafted client side code
• Front end code complexity increases to level where unit test of client becomes a necessity
• Since mobile platform’s native presentation framework provides a richer and more integrated user experience than the HTML equivalent teams are forced to create multiple clients for their publicly facing applications.

Big Data

Big data is becoming a significant new stage in enterprise application development. Due to the huge volumes of data streaming into an organisation retrieving and interacting with raw items of data is meaningless. Instead the data needs to be transformed, aggregated and grouped before it can be analysed and many designs include multiple persistence stores.

• Data quantities are so large that classical data processing techniques cannot be used
• Data is continually pre-processed and aggregated
• Clients rarely interact with raw data but utilise aggregates
• Raw data is not thrown away but is available forever as each different analysis tasks will need the data to be projected using a different manner
• Data schema is less relevant as the analysis process overlays the data with a custom schema necessary for the problem that is being solved
• Multiple persistence stores each solving a different problem are utilised on a single project