System Connections
May 2004 Summary: This chapter builds on Chapter 3 by describing how to connect with individual systems. Each system allows certain types of access and restricts others. This chapter presents a series of related patterns that will help you analyze the alternative methods and the tradeoffs to consider when you choose your system connections. ContentsConnecting to Layered Applications The previous chapter described different strategies for the design of an integration layer and the tradeoffs involved in choosing an alternative. After you decide to use an integrating layer, you must then decide exactly how you are going to connect each system that you intend to integrate. As you consider how to connect to different information systems, you will encounter a mix of technical architectures. Each technical architecture is designed to allow certain types of access and to restrict others. These access restrictions constrain the set of integration options that are available to you. This chapter discusses a series of related patterns that will help you analyze the alternative methods and the tradeoffs to consider when you choose your system connections. Connecting to Layered ApplicationsWhen trying to find connection points to an existing application, you need to study the way these applications are structured. The prevailing architectural style for business applications is Layered Application [Trowbridge03]. Many applications fall into the more specific form of the Three-Layered Services Application pattern. The components of this pattern are shown in Figure 1.
Figure 1. A three-layered services application This architectural style defines three distinct layers:
Each application layer presents an opportunity for an application to interact with the integration layer, as shown in Figure 2.
Figure 2. Multiple choices when connecting an application Because each layer has different requirements and purposes, you face different challenges when connecting to each layer. Similarly, there are three primary patterns you can use to connect an application to an integration layer:
To make an informed choice about which pattern to use, you must know the options that are viable, and you must evaluate their impact on the overall integration architecture. This chapter describes each pattern and elaborates on the benefits and liabilities of each one. As in other chapters, this chapter uses visual models to show the associations between the patterns. Figure 3 shows the patterns as circles and simple associations between them as lines. As the discussion progresses, more complex pattern relationships build upon this basic diagram.
Figure 3. System connection patterns The first kind of integration to consider is Data Integration. Data IntegrationMany applications keep large volumes of data in data stores such as flat files and relational, hierarchical, and object-oriented databases. Other applications that need this information can access it directly from these data stores. Connecting applications through the data store is relatively simple. Usually, you use FTP, scheduled batch files, database management system (DBMS) utilities, extract transform and load (ETL) tools, and integration servers. If you decide to integrate at the data layer, there are three patterns that represent the three types of data integration to consider: Shared Database, Maintain Data Copies, and File Transfer. These patterns are shown in Figure 4.
Figure 4. Three kinds of data integration Figure 4 introduces a new pattern relationship to the visual model from Figure 3. The triangle indicates refinement between the base pattern (Data Integration) and the derivative patterns (Shared Database, Maintain Data Copies, and File Transfer). More precisely, this relationship is defined as follows: "A specific pattern refines a more abstract pattern if the specific pattern's full description is a direct extension of the more general pattern. That is, the specific pattern must deal with a specialization of the problem the general pattern addresses, must have a similar (but more specialized) solution structure, and must address the same forces as the more general pattern, but may also address additional forces. To make an analogy with object-oriented programming  the refines relationship is similar to inheritance." [Noble98] Applying this relationship to the problem of data integration means that if you decide to integrate systems at the data layer, you must further refine your decision by choosing one of three alternatives. You could share a single instance of a database between multiple applications by using the Shared Database pattern, as shown in Figure 5.
Figure 5. Multiple applications sharing the same data store Another approach is to create multiple copies of a database and distribute them throughout your enterprise according to the Maintain Data Copies pattern. If you do this, you must then maintain these separate copies, which introduces synchronization and replication. Figure 6 shows the Data Replication pattern, which refines Maintain Data Copies.
Figure 6. Data Replication, a refinement of Maintain Data Copies Yet another approach is to use File Transfer. In this pattern, one application produces a file and transfers it so that other applications can consume it. Files can then be produced at regular intervals to synchronize two or more systems. The Data Integration pattern, later in this chapter, describes these alternatives in detail and provides the benefits and liabilities of each approach. This discussion should help you choose the kind of data integration that is appropriate for your requirements. Using Data Integration is made easier by the abundance of tool support provided by many companies. However, in spite of the low cost and the maturity of the tools, accessing the data store is not viable in some cases. A less invasive way of connecting applications is to connect through the presentation layer. Presentation IntegrationWhen applications that have user interfaces join an integration architecture, other applications can connect to them through the presentation byte stream as shown in Figure 7.
Figure 7. Connecting to an application through the presentation layer Presentation connectivity represents the least invasive way of connecting applications because the other applications appear to the host application as humans who are interacting through the user interface. Therefore, this form of integration does not require any changes to the host. The disadvantage is that simulating user interaction is cumbersome and inefficient. It requires parsing the data or functionality out of the byte stream, which effectively reverses the transformations performed by the presentation logic. In addition, applications that connect through the presentation layer can only access what is also available to a human user. The granularity of the exposed data and functionality is very coarse; burying potentially-rich APIs. One advantage of Presentation Integration is that it can be an inexpensive way to integrate applications. Although the connection is inexpensive, it is also easily disrupted. By its very nature, presentation integration is tightly coupled to the host application's presentation. If the application's presentation changes, the presentation integration solution must also change. Instead of sharing data or parsing through a byte stream to access a system's functionality, it is often preferable to access a system's business logic directly through Functional Integration. Functional IntegrationBy connecting directly to the business logic layer, Functional Integration enables other applications and services to take advantage of the business logic that is encapsulated in other systems, as shown in Figure 8.
Figure 8. Integrating applications and services at the business logic layer Functional Integration connects applications through interfaces and specifications. Unfortunately, not all applications in an integration architecture have interfaces and specifications. Quite often, the applications that have APIs do not expose their data and functions at a granularity level that is suitable for integration. Functional Integration can be very effective in the right circumstances. A good example of when to use Functional Integration is for a credit scoring application or service. Credit Scoring ExampleMany financial applications require a credit score to qualify an applicant for a specific loan. Salespeople often want a quick and accurate response from a credit scoring system so they can quickly prequalify customers and steer them to an affordable alternative. In addition, credit scores depend on a number of dynamic factors, which means that credit scores can quickly become out of date. Because credit scores are calculated values, sharing data means relying on credit scores that might be out of date or duplicating raw input data and forcing each application to implement the credit scoring algorithm independently. A better solution is to share the business logic encapsulated in the credit scoring system and to then expose it through some type of functional interface that other systems can consume. For this level of integration, you need Functional Integration. Kinds of Functional IntegrationAfter you choose to use Functional Integration, you must further refine your decision by choosing one of the three alternatives:
Figure 9 shows these three alternatives as distinct derivatives, or refinement patterns, of Functional Integration.
Figure 9. Three kinds of Functional Integration Later in this chapter, the Functional Integration pattern describes these alternatives in detail and discusses the benefits and liabilities of each approach. This discussion should help you choose the kind of functional integration that is appropriate for your requirements. System Connection PatternsData, presentation, and functional integration represent ways in which seasoned integration architects connect applications. But given your specific requirements, how do you connect applications within your integration architecture? Sometimes the applications you are integrating limit your choices. If there is more than one way you can connect them, you must evaluate the tradeoffs associated with each potential choice. The Data Integration, Presentation Integration, and Functional Integration patterns later in this chapter distill the knowledge required to make an informed decision. Two general points are worth noting. First, an application that uses one style of connection can communicate with another application that uses a different style. For example, Presentation Integration may be the only way to extract data from a preexisting application, but you can still insert this data into the target application using Data Integration. Second, despite sometimes similar names, a specific integration layer pattern can be used with any system connection pattern. For example, a Portal Integration solution can extract data from existing applications by using Presentation Integration, Functional Integration, or Data Integration. Figure 10 shows three system connection patterns, with derivative patterns and associations.
Figure 10. System connection patterns and their relationships Table 1 summarizes these patterns and provides the problem/solution pairs they represent. Table 1: System Connections Patterns
|
