Over the past year or so there has been a huge increase in the amount of discussion about Service Oriented Architecture (SOA), and the number of blogs and posts on the subject seems to increase daily.
So with over 25 million references to SOA discovered by Google, why bother writing another SOA blog post?
Much of the discussion amongst the SOA community is interesting to other technophiles, but only serves to confuse the majority of readers. Bloggers like Mike Kavis try to bring the focus of SOA back to a business perspective, but the vast majority of articles concentrate on the technology debate.
In recent weeks the rise of a lightweight version of SOA, termed Web Oriented Architecture (WOA), has had the techno-bloggers tapping away at their keyboards. OnStrategies gives us a quick digest of some of the highlights.
Rather than join the technology debate about SOA we’ll take a step back and explain simply how it works, how it can be used and, with the use of a real-world example, describe why a properly planned and implemented Service Oriented Architecture can create a flexible way of aligning business and IT.
Let’s start by looking at what the term Service Oriented Architecture actually means.
The original definition of the word “architecture” can be described as “the art and science of designing and constructing physical structures”. Typically we associate the word “architecture” with the style of buildings, be it the Art Deco style of the Chrysler Building in New York, the modernist formalism of the Bank of China Tower in Hong Kong, or the grand scale of the gothic Milan Cathedral.
Recently the IT industry has used the term Architecture more and more frequently to describe how building blocks of hardware, software and interface protocols can be put together to create systems. The best recent description of its usage that I have seen described Architecture as:
“a subjective mapping from a human perspective (that of the user in the case of abstract or physical artifacts) to the elements or components of some kind of structure or system, which preserves the relationships among the elements or components.”
When it comes to Service Oriented Architecture (SOA), the term means designing a system where each system component provides access to its computational or business resources as a service to other components.
By way of explanation, let’s say that we have a simplified system which receives an order, generates an invoice and faxes it to the purchaser. The workflow of that function can be broken down into four distinct services:
1) Stock availability and pricing service
2) Order creation service
3) Document creation service
4) Faxing service
The theory is that by creating these distinct, separate and self contained services for each of the components we gain greater flexibility. Each of these services could be called from other business functions allowing re-use in subsequent areas of the business.
For example, the business could offer potential clients access to its stock availability and pricing information rather than utilising its own sales staff to process queries. Or it could expand the faxing service to cover email without changes to multiple larger systems - saving money, time and resources.
That, put very simply, forms the basis of all SOA. Those of you familiar with software development will recognise the re-use advantages of this approach which has been the main-stay of functional and object oriented design for many years.
So if this concept of re-use is not new, why all the hype surrounding SOA?
Although the concept is not new, software re-use traditionally required the code to be “tightly coupled”, meaning that the programmers had to understand how both the new code and the re-usable code could be linked together to create an interface which tied them explicitly together. Vendors are now trying to promote the technology, tools and standards that have been developed to allow services to be “loosely coupled” with each other, reducing the complexity of integrating services together. To understand how this works we need to look at how the service industries work in the real world.
Over the past few decades we have seen a growth in the service industry sector, driven partly by economic constraints. Companies have increasingly analysed their strengths and been concentrating on their core expertise, outsourcing those functions that they see as non-core and awarding contracts to companies to service those contracts. Remember in the 1980’s when the telephone sanitiser companies had the contract for wiping telephone handsets, or in the 1990’s when plant maintenance companies came in to water the peace lilies in the foyer?
Usually, each of these contracts would have an associated cost negotiated before the contract was awarded, and was subject to procurement rigours for value for money. They would also of had predefined deliverables and be monitored to ensure the service matched those promised. Finally, each contract commonly had a point of contact for efficient communication with that service organisation.
By building similar capabilities into SOA, the IT industry now has a way of creating re-usable services which can be used by third parties seeking “buy not build” functionality for their applications or for the support of their business functions. By implementing a standard contract approach into the implementation of services, they can be used without a detailed knowledge of bespoke interface considerations. As such, it is relatively easy to decouple from one service provider and move to another, hence the term “loosely coupled” as opposed to “tightly coupled”.
As well as adhering to a contract model and having the ability to be loosely coupled, services generally also allow abstraction, reusability, autonomy, statelessness, discoverability, and composability – depending on the specification and design brief of the service.
ARCHITECTURE
When an architect is commissioned to design a building his first requirement is a design brief. As well as understanding the style of the building, he must also gain knowledge of budget, environmental considerations, location specific requirements, building usage, decision authority, material availability, reasons for embarking on the project etc.
The answers to these questions will determine how the building will look and feel. It will also, however, determine the tools and equipment needed to undertake the task, the best approach to take, what skills are required to deliver the project, the scope of work, bill of materials and project timescales.
The same is true of SOA. Just as there is more than one way to design and construct a building, and more than one set of tools which can be used, so with SOA there are many technologies and methodologies which can be used to deliver an SOA solution and all have different advantages and disadvantages.
In the construction industry there are some people who prefer sustainable hardwood construction rather than metal. Within the proponents of metal construction, there are some which prefer aluminium to steel. Within the advocates of steel, some prefer rivets to welding.
Similarly, in the SOA community some advocates prefer SOAP/web services to CORBA, DCOM, REST, RPC or JINI. Some prefer complex messaging backbones and others simple state protocol transmissions. Some prefer closely coupled, others loosely coupled. Some use a top-down approach to architectural design, whilst others suggest bottom up.
Does it matter? Yes it does.
Which is best? Unfortunately that depends on your design brief.
SOA ARCHITECTURE PRINCIPLES
There are some well defined specific SOA architectural principles governing service design and service definition, namely:
- Service encapsulation - Many web-services are consolidated to be used under the SOA Architecture. Often such services have not been planned to be under SOA.
- Service loose coupling - Services maintain a relationship that minimizes dependencies and only requires that they maintain an awareness of each other
- Service contract - Services adhere to a communications agreement, as defined collectively by one or more service description documents
- Service abstraction - Beyond what is described in the service contract, services hide logic from the outside world
- Service reusability - Logic is divided into services with the intention of promoting reuse
- Service composability - Collections of services can be coordinated and assembled to form composite services
- Service autonomy – Services have control over the logic they encapsulate
- Service optimization – All else equal, high-quality services are generally considered preferable to low-quality ones
- Service discoverability – Services are designed to be outwardly descriptive so that they can be found and accessed via available discovery mechanisms such as service repositories or directories
Further information about these principles can be found here.
There are two approaches to Service Orientated Modelling – “top-down” and “bottom-up”. The top-down approach starts with analysis at a business process level to evaluate re-engineering and workflow, ensuring that service delivery matches business requirements. The “bottom-up” approach starts in the IT stack, analysing systems and system functionality, before existing systems are wrapped using Web services to create a service layer.
In reality, despite much debate over a number of years, success will not come from “top-down” OR “bottom-up”. It is only a mixture of the two which will work effectively, a view shared by Neil Ward-Dutton in his recent post “Which comes first: process or service?”
Let’s take a look at a real life example:
Oncology Hematology Associates (OHA) is a company based in Southwest Indiana, USA, and provides coordinated patient-centred plan for cancer diagnosis and treatment. They have about 100 employees including physicians, nurses and other healthcare professionals.
Before obtaining a little help from Microsoft, OHA had many manual paper based business processes. The nature of their cancer care business meant dealing closely with third party companies, such as testing labs, with much of the information being faxed back and forth.
OHA wanted to find a way to integrate its multiple business processes to improve efficiency, support collaboration, serve patients better, and increase profitability – clear business reasons for adopting SOA.
So with two developers they set about building web services onto their six medical industry preparatory backend systems. They adopted an iterative approach – exposing services through a composition layer to give access to a broad variety of platforms – and integrated all the faxing systems.
They integrated external partners into the systems, such as testing labs, to consume their services. The integration was done with a perspective of maintaining business value rather than redesigning the whole IT architecture.
Creating web services has enabled them to add a new range of medical care products to their business portfolio, obtain results in days rather than weeks, provide better communications amongst physicians and much more. In short, they have increased productivity, increased business opportunity, reduced operating costs and enabled better governance and compliance.
You can read more about their case study here.
TECHNOLOGIES
At this point I considered talking about the tools and technologies that are available to create SOA services. However, this article is really concentrating on the principles of SOA, so I’ll discuss the tools and technologies in more detail in a future blog. Here is a list of some of them with links to their definitions:
Web Services, SOAP, REST, RPC, DCOM, CORBA, BPEL, JINI , WSDL and WCF
What I would say is that it is hardly surprising that there has not been widespread adoption of SOA. Many IT people, never mind the business, must be confused by the proliferation of marketing initiatives, acronyms, conferences etc. relating to the topic. The latest ingredients added to the mix are Web Oriented Architecture (WOA) and Platform Oriented Architecture (POA). Did you know, that SOA + POA + WOA = SOA?
On page 27 of the current issue of Information Age (UK) there is an advert for the SOA & Application Development and Integration Summit 2008 which states that “The question is no longer ‘Should we go SOA?’, its ‘How do we measure the value?’” and that “SOA is changing everything.”
To me these are bold statements to make, considering there has been, in the words of one prominent UK CIO, “no truly successful SOA application”.
The question of how we measure SOA value is key, but without a sound and rational business case those metrics can’t be put in place. Mechanisms to track the contribution SOA makes to the business post-implementation are extremely difficult to establish, but nigh on impossible if it was a technology solution that has been sold to the business. In her post “Looking for SOA success stories”, Anne Thomas Manes identifies this problem when she says:
“I've talked to many companies that have implemented stunningly beautiful SOA infrastructures that support managed communications using virtualized proxies and dynamic bindings. They've deployed the best technology the industry has to offer -- including registries, repositories, SOA management, XML gateways, and even the occasional ESB. Many have set up knowledge bases, best practices, guidance frameworks, and governance processes. And yet these SOA initiatives invariably stall out. The techies just can't sell SOA to the business. They have yet to demonstrate how all this infrastructure yields any business value.”
SERVICE DISCOVERY
When services have been designed to be used together within a single organisation the architect may elect to explicitly link the known services together. Although this is the easiest way to connect services, it is not always possible – especially between geographically (or domain) disparate systems.
Services which have been created for consumption by third parties also need a way to be discovered, so they can be used by third parties.
In order to facilitate this, mechanisms have been introduced to allow services to be discovered across networks, such as the internet. There are numerous protocols which have been created for this and their use depends on the technology used to create the service. Three noteworthy examples are:
Universal Description, Discovery and Integration (UDDI) which provides a way for Web Services to list themselves as discoverable on the internet. Businesses who have created web services can register them within a UDDI, which is split into three central directories:
White Pages — address, contact, and known identifiers
Yellow Pages — industrial categorizations based on standard taxonomies
Green Pages — technical information about services exposed by the business
JINI from Sun Microsystems can be used to advertise and discover Java based Web Services. One limitation with JINI is that it uses multicasts to advertise and discover and as such can only really be used within a local network.
SLP has been used for many years to advertise network resources such as printers and file shares. SLP can also be used to advertise web services, but does not have a facility to maintain a repository of services with descriptions, unlike UDDI.
IDENTITY MANAGEMENT
One of the challenges to be overcome for successful SOA deployment between third parties is that of identity management. When services are made available for use online, it is fair to assume that a payment mechanism will be built into the service contract description, or that sensitive information could be passed between the services.
In order for this charging mechanism to work effectively there is a requirement for an assurance that whoever consumes the service is the same entity that contracted the service. If a third party can assume the identity of the service requestor or the service provider (or both) there is the potential for serious security breaches. Once a transaction over the service begins, information about that transaction needs to be maintained at both ends of the transaction. When a connection between the service requestor and the service provider is terminated and needs to be re-established, protocols must be adopted to ensure the state of the transaction is not lost.
The problem identity management introduces is a need for tightly bound security on loosely bound services, and such tight security can defeat the purpose entirely of having loosely bound services. There are a number of security standards being driven forward to try to address these issues and an article by Eric Roch on IT Toolbox, which was written to specifically cover the security risks of SOA, is an excellent place to find out more background information on SOA security measures.
CONCLUSION
In its simplest form SOA can be used to put service wrappers around existing functionality to create a re-usable software infrastructure, which can help businesses move quickly in changing market conditions. It can simplify integration of existing legacy systems and help align IT services with business operations, as can be seen in the real-world example given above. SOA, when implemented on a small scale, can give benefits that can be easily demonstrated to the business.
But SOA can also be rolled out as an architectural vision across the enterprise, and many large software vendors and integrators are positioning themselves with products to try and achieve this vision.
As vendors try to differentiate themselves in the market place we see, yet again, the emergence of new abbreviations and technology-based terminology. Although this stimulates and progresses a technological debate within the IT industry, not for the first time a perception has grown in the business community that IT is overhyping a new technology.
The nature of IT is that it is continually evolving, and trying to sell a long term IT vision to the business has always been a stumbling block to adoption of new technologies. For SOA to succeed, the business has to believe that IT, having committed to an SOA strategy, will not be re-seduced by the finer points of some new technology.
Joe McKendrick at ZDNet recently wrote that:
“The business side needs to be educated that SOA is a goal worthy of shooting for. And key performance indicators need to be tied into SOA efforts. But SOA is a journey, not a destination, and organizations that expect overnight benefits to the business are bound to be disappointed.”
Even though SOA could generate huge flexibility and long term savings, there is an associated increased cost to large SOA implementations over and above non-SOA solutions.
At a time of short-term financial planning, entering a long-term technology relationship can be a step too far for the business, and I fear one of the first budgets to be cut will be SOA.
Nice article covering lots of ground!
"it is relatively easy to decouple from one service provider and move to another"
That's long been a goal, and a oft-stated benefit of integration. But I've never, not even once, seen it actually happen in any meaningful way. Sure, low-level, well-defined and fairly standardized items can potentially be swapped with little fanfare (e.g. change EDI VAN providers) but that isn't the big bang for the buck.
Having the ability to swap out services sounds nice, but the reality is that for services created in house, there won't be anything to swap out with (who will create overlapping services?). For those that might be licensed from commercial providers, do we think Oracle, SAP, MS, Google, et al will provide the same operations on the same services? The same sematics? The same formats? They haven't done so thus far, what makes us think the services they provide will be any different?
I bring this up because the "swappable" benefit that is often promoted as a benefit of SOA and used as a selling point, but rarely works out. Don't like how service X is working? Swap it out for another! Oh, wait. An equivalent doesn't exist--anywhere.
"...I fear one of the first budgets to be cut will be SOA."
Why would SOA even have a budget? If SOA is a style of architecture, not a distinct and new level of architecture, and a way that we approach arranging our components, then why should it have its own budget? Wouldn't we just apply the principles to how things are done? Presumably we already have architecture processes and principles in place. We still have projects to do. We still have capabilities to create and deploy. We're following *some* sort of architecture doing those, so why not just follow SO principles instead? SO doesn't require any particular technologies (though some are helpful) so why not make do with what you have? Do things in an SO way, no need to spend a bunch of money.
Posted by: Rob Eamon | May 01, 2008 at 05:11 PM
Hi Rob,
Thanks for the comment ... it is much appreciated.
While I agree that SOA hasn’t taken off in the way many vendors hoped with regard to repositories of interchangeable services, I think we must place SOA in that historical context and move forward from there. I remember being at a seminar on SOA and listening to a futurologist's fantasy about how services will analyse a landscape of service portfolios and request a contract with the cheapest provider of a sub-service from hundreds of possible providers. Make of that what you will.
My personal view is that loose coupling helps the business in many ways…
Not least that changes in business strategy can be quickly mirrored though SOA to maintain alignment. You may not have overlapping services, but just want to swap out old for new functionality.
On the budget front, SOA may be a style of architecture, but there are plenty of projects that need a little extra time and money spent over and above what the project would normally take to complete because the scope has been SOA’ed. Not to mention that there are many vendors quite happy to promote and sell their latest ESBs etc. to their corporate clients.
This is where I think SOA could feel the axe being wielded by the business. The “no need to spend a bunch of money” message may be getting drowned out in the very vocal SOA tools market place.
Posted by: Paul Wallis | May 06, 2008 at 12:17 AM
"The 'no need to spend a bunch of money' message may be getting drowned out in the very vocal SOA tools market place."
That's exactly right. Analyst groups can share some of the blame for the noise too. People seem quick to blame the vendors. Certainly there is some cause there but analysts contribute far more FUD IMO.
Posted by: Rob Eamon | May 06, 2008 at 07:23 PM
Hi
I'm reading your article and I'm wondering what you think about my ideas that I posted about 1 month ago in yahoo groups: "to write the code as close as possible to the way of thinking of the users. With different words to align the code to the way of thinking of the business"
Posted by: Kamen | May 25, 2008 at 03:23 PM
Hi Kamen,
Having read your post, I think a major challenge to overcome would be persuading the business to co-operate in the way you describe.
Posted by: Paul Wallis | May 26, 2008 at 05:58 PM
You define SOA as "designing a system where each system component provides access to its computational or business resources as a service to other components".
I just wish you'd said "designing systems" rather than "designing a system". An important shift happens when you start to look at systems of systems rather than each system in isolation.
Posted by: Richard Veryard | July 30, 2008 at 03:49 AM
Hi Richard,
Thanks for the comment, and you make a good point. I guess it is a question of how you define a system. Most "systems" can be broken into sub-systems which can be called systems in their own right, or can themselves be classed as a sub-system of a larger system.
This is one of the points I've addressed in my post "Auditing IT Systems" which looks at the difficulties in auditing when the boundaries round business processes can be different to the demarcation between IT Systems.
Identifying how technologies such as Cloud are used within sub-systems can be critical to assessing risk as part of a successful Audit.
Posted by: Paul Wallis | September 16, 2008 at 10:43 AM