Once upon a time, many organizations achieved their online revenue goals with a simple e-commerce website. Modern digital commerce solutions are, in software engineering terms, distributed systems. To be successful, they must continuously communicate with many different systems, across the internet. Some are hosted externally by other vendors and interact with front-end user experience. Others are back-office systems maintained by internal resources or other 3rd party vendors. Digital commerce platform integration is an absolute requirement for successfully delivering the best possible customer experience.
In our experience, 3rd party system integration is the most challenging and high-risk aspect of any digital transformation, but this is particularly true when including digital commerce. To put it bluntly, there is no room for failure when the customer has their credit card in hand. The risk of a misfire increases when legacy, back-office systems that have been highly customized over the years are added to the mix.
Integration process to the rescue
Over the years, we have continuously improved a process explicitly designed for smooth and seamless platform integration. Like any thoughtful approach, it begins with discovery and includes analysis, documentation, and testing. Although it may be scaled up or down, depending on the scope of a specific initiative, we’d like to share a brief summary of our 6-step process.
Step 1 – Identify all 3rd party systems
First, we identify and list every system for integration in the specific implementation. Then, for each system, we briefly describe its purpose. A typical project will include a payment gateway, an analytics system, and an ERP, just to name a few examples. Most of our clients discover there are at least ten 3rd party or internal systems impacted by customer interactions with their digital commerce platform.
Step 2 – Identify all integration points for each system
Once we have a complete list of all the included systems, it’s necessary to understand precisely how they interact. There are almost always situations when a digital commerce platform will integrate with one 3rd party system in multiple ways. Each communication sends and receives data for different reasons, so it is critically important to list and define each integration point separately.
Step 3 – Document requirements and specifications for each integration point
Before we design and implement each integration point, we gather and document requirements and specifications.
Here are some of the details we consider for each integration point:
- The purpose or need of the integration point (e.g., orders need to be imported into the WMS for fulfillment processing)
- Any functional requirements (e.g., ability to configure the timing of a scheduled job)
- Process flow, i.e., all steps that need to be taken
- The interface details, i.e., the protocol (e.g., SFTP, HTTP request, etc.), the format (e.g., XML), the endpoint (e.g., URL of the FTP server, URL of the web service, etc.)
- Execution mode (e.g., scheduled job, trigger on the website, etc.)
- The frequency (e.g., once a day, real-time)
- Expected data volume
- Any assumptions, risks, and dependencies
- Any security requirements or specifications (e.g., VPN tunnel, credentials needed to access the other system, etc.)
- Data mapping, i.e., list all fields with their description, type, nullable indicator, corresponding field in both the source and in the destination system
- Exception handling, i.e., behavior and steps that need to be taken for each of the possible error codes
Step 4 – Design the System Architecture
Now that all systems have been identified, together with all the integration points for each system, and we know the requirements and specifications for each integration point, we’re ready to make the right decisions when it comes to designing the Systems Architecture. Keep in mind both the need to fulfill the current requirements but also leave the “doors open” for any changes that are likely to occur in the future, such as the addition of new systems or the update or replacement of existing ones. It might make sense to implement an Enterprise Service Bus (ESB) or at least design a de-coupled integration layer in the application.
The system architecture should include at least one diagram that shows all systems and how they communicate with each other, over the network. The following image is a past example of a system-level topology diagram created for a client by Luminos Labs. Such documentation should constantly be maintained by the party responsible for the maintenance and security of the entire solution.
Step 5 – Design the future state of each of the integration points
Of course, it’s important to remember that this is a documentation of change. Now that we know everything we can about the current state of the entire distributed system, the objective of step 5 is to develop a detailed technical design for each integration point in the future state. At this level, the plan should conform to the decisions made in the Systems Architecture Document we created during step 4.
At Luminos Labs we create a UML class diagram that captures most classes, their properties, and how they interact with one another. In cases including more complex process flows, UML sequence diagrams can be very helpful to make sure we’ve designed the right approach.
Step 6 – Implement and unit test the new integration point
We’re finally ready to start coding! Before introducing a new system to the solution, we create POCs and test each integration point outside of the main application, as it allows you to have full control over testing the different scenarios.
Digital commerce platform integration may not be easy, but it can be done
Integrating 3rd party systems with an evolving digital commerce solution is typically the most challenging and unpredictable phase of a platform implementation. In our experience, following the 6-step process we’ve defined significantly reduces risk and increases the likelihood of success.
Extra Tip: especially when you’re integrating with legacy back-office systems that have been highly customized over the years, try not to get into the weeds of those systems. Instead, work with your client’s IT team and have them provide or develop clean endpoints (either web services or flat files).