API integration tutorial: Latest trends and strategies
A comprehensive collection of articles, videos and more, hand-picked by our editors
While large enterprises have demonstrated the ability to modernize supply chain and data networking architectures in order to reduce the costs of business, what about mid-tier companies? What steps can they take to realize similar costs savings.
By submitting your personal information, you agree that TechTarget and its partners may contact you regarding relevant content, products and special offers.
In this piece, we take a look at how Redwood Supply Chain Solutions, a subsidiary of Redwood Logistics, modernized supply chain and data architectures by critically addressing -- including overcoming the challenges surrounding electronic data interchange (EDI) -- standards.
Wal-Mart has demonstrated the value of building a modern supply chain and data networking architecture for reducing costs and meeting customer demand. Now, smaller and mid-tier companies are looking to take advantage of the same innovations. This requires logistics and data solutions architects to work in parallel, said Eric Rempel, CIO of Redwood Logistics.
This is a much more difficult challenge for mid-tier companies that don't have the luxury of dictating standards to logistics suppliers around data exchange formats. Most consumer products companies, shippers and retailers have moved to EDI standards for exchanging information about shipping needs, delivery status and inventory stocking levels. But Rempel joked, "EDI is sort of a standard that is not a standard."
Easing the integration with EDI
Eric RempelCIO, Redwood Supply Chain
A key component of Redwood's strategy lies in leveraging a new generation of EDI to API gateways. They recently adopted MuleSoft's Anypoint platform, which makes it easier to wrap B2B gateways with RESTful APIs and abstract away the complexity of EDI standards. This strategy has helped Redwood focus more attention on the logistics side rather than writing more integrations and mappings, said Rempel.
"Anypoint B2B brings API-led connectivity to traditional B2B integration, and allows organizations to drive innovation in their business by applying a modern connectivity framework to legacy technologies," said Ken Yagen, vice president of products at MuleSoft. "This means that if an organization wanted to create a mobile app, it can expose and share the data from the app through a modern API layer and over traditional EDI protocols out to the partner."
Redwood works with hundreds of companies that each use slight variants in their EDI messaging and communication protocols. These must be transformed when communicating with partners. They have built an EDI transformation layer that converts messages into XML or JSON formats for high-capacity processing capabilities in house. The MuleSoft platform is used to handle inbound and outbound communication to guarantee no message loss and to transform data into the format most appropriate for the customer.
"EDI messages are unwieldy for business analysts to work with because of the way data is represented in the message," said David Chao, director of industry solutions at MuleSoft. "This makes it more difficult (i.e., slower, more expensive) for changes to be made. In addition, the mobile and Web channels that need to display much of the information that is contained within EDI messages speak JSON."
Why EDI is complex
Variants in EDI messaging formats might be as simple as placing similar information in different segments of the file header. The variations become more complicated when the messages are used to describe items, charges, physical characteristics and the spoilage properties of goods. "X.12 messages are static enough that people can see what you are trying to accomplish," Rempel said, "but an EDI integration team needs to define the mapping into a normalized format."
EDI segments are like element text values in XML, but that's where the similarities end. Files based on EDI standards are not human readable. "They are not nested like XML/JSON, but rather EDI docs are flat text files with fixed length segment codes and values," MuleSoft's Chao said. "These fields vary in length and are separated by varying delimiters. Testing is imperative, as simple parsing errors can easily go overlooked without good test cases and proper documentation."
Despite published ANSI X.12 standards and specs, many companies had no choice but to modify usage for valid business reasons. Over time in the logistics industry, transportation providers and shippers alike were forced to abide by, and even publish their own, modified specs. This is one reason why EDI files go in specified mailboxes upon reaching each trading partner.
Break the dependency on EDI standards
Rempel has found that there are several categories of enterprises. Many are starting to look at moving away from traditional documents based on EDI standards to JSON structures. JSON and REST messages are more human-readable and better able to leverage APIs for communication. Many mid-tier companies are still dependent on sending EDI messages via FTP and AS2 protocols. FTP suffers reliability problems, while AS2 suffers from problems with complexity and rarity.
One good practice for bridging this gap in the short run is to simply use Web services for sending EDI messages in the existing format. This can lower the cost of modernizing the infrastructure while building out a more efficient supply chain architecture. This approach still needs to send some legacy messages back to the existing applications such as EDI 997 notifications indicating message receipts.
Logistics integration starts with data
Third-party logistics (3PL) providers are transitioning to a greater role as data management providers. This includes orchestrating B2B services, customizing rules and looking at the logistics network holistically. The traditional approach is to look at individual loads and shipments with a focus on price and services. But the real value comes from taking a network perspective to logistics. This involves thinking through pieces like the location of distribution centers and consolidating freight into and out of regions.
A key element of building this kind of modern infrastructure lies in figuring out how to improve the orchestrations of data and business rules across shippers and retailers involved in the supply chain. "There are a lot of synergies about how you manage data and how you manage transportation," Rempel said. It is important to work with data engineers, logistics engineers and process engineers to design the scope and process. This makes it possible to simultaneously engineer how the data flows in and out, which affects how freight flows in and out.
Start with business process
Doing this kind of transformation needs to start with a discovery session that looks at what a consumer goods supplier does from a business perspective. "They need to understand their business objectives, how they view transport, how they interact with partners and what they do with the data around this," Rempel said.
It is also important to document this process and reiterate it back to the business executives and each business location to ensure agreement. "It is critical to understand this before talking about technology or EDI," he said.
The next step is to document how the current state looks and what a future state could look like. At this point, the solutions architect can begin to look at where the data resides. In some cases, this might just look like finding an easy way to transform data to make it more accessible to partners. In others, solutions architects might discover the enterprise is only generating some of the data elements required by partners or for implementing business rules. In these cases, the solutions architect needs to look at where assumptions can be made or practices put in place to supplement the data like the dimensions of sized cases sent out of a warehouse.
Once this infrastructure is in place, it is easier for businesspeople to change business rules and processes without the need for redeployment and changing code. "This is one of the competitive advantages of what we are building on MuleSoft," Redwood's Rempel said. "We can get connectivity going and do message transformation without a major IT endeavor. This makes it easier for business users to do business process definition and analysis."
Decouple the logistics and data architectures
Another good practice is to decouple the elements in the data and logistics architectures. This makes it easier to implement improvements without breaking existing processes. "Our whole goal is to create immutable functions so if any one service is not functioning it does not impact others. This is why we like to separate business logic from transaction management," said Rempel.
On the data network side, it is important to decouple the systems so that customers can send in data as they want. On the logistics side, decoupling makes it easier to consider different transport modalities (rail, ship or trucks) or distribution architectures like hub and spoke or point-to-point. This approach makes it easier to implement different transformations, business logic, and workflows for various use cases.
Going forward, Rempel is hoping MuleSoft will make it easier to move away from managing legacy connectors like AS2 and FTP. This will make it easier to focus their efforts on adding more value to their customers' logistics processes.
An introduction to modern supply chain management using EDI.
Learn more about the relationship between EDI standards and SCM.
Learn about the key benefits of strong supply chain management.