API integration, explained for the person who signs it off

At a wholesale plant nursery, orders arrive through an online ordering site that garden centres use. Each morning a member of staff opens the list, and types every order again into the stock system, so that the plants are reserved and the delivery vans are loaded. On a busy spring Monday there are ninety orders. By Wednesday a customer has been sent three hundred of the wrong lavender.

The owner asks the site's developer whether the two systems can be connected. The answer is "yes, with an API integration, about two weeks". It may well be true. The owner has no way to judge it, because the two weeks cover only the easy half of the job.

What API integration is

An API, an application programming interface, is a way for one piece of software to ask another for information or tell it to do something, without a person in between. An API integration is two systems exchanging data through such an interface, by agreed rules, automatically. The order goes from the ordering site to the stock system by itself, and the member of staff does something else.

The need is as old as business software. A long-standing reference on the subject, Enterprise Integration Patterns, opens its introduction with the sentence "Interesting applications rarely live in isolation." It goes on to say that every integration, whatever the technology, has to deal with the same few facts, among them: "Networks are unreliable", "Any two applications are different", and "Change is inevitable". It lists four established ways for systems to share data: one writes a file that the other reads, both use the same database, one calls the other directly, or they pass messages. An API is the usual present-day form of the third. Which of the four fits depends on the systems, and sometimes the right answer for an older system is a file.

Those facts are the reason the developer's two weeks deserve a second look. Sending an order from one system to another on a day when everything works is the easy half. The work is in the days when something does not.

What to settle before anything is built

You can judge an integration proposal without technical knowledge by asking whether five things have been written down. If they have not, the estimate is for the easy half.

• The path. Which two systems, which data, in which direction. "Connect the site to stock" is three or four paths: orders one way, stock levels the other, perhaps customers and prices too. Each is its own piece of work.
• Who holds the truth. For every piece of data that crosses, one system is right when the two disagree. If a price is changed in both places on the same day, which one wins?
• What happens on a bad day. The stock system is down for an hour. The same order is sent twice. Two messages arrive in the wrong order. A product exists on one side and not on the other. Each needs a decided answer, not a shrug.
• How a fault is seen. An integration that stops quietly is worse than no integration, because people have stopped checking. Someone must be told when an order has not gone through, on the day.
• Who owns each side. The other system will change. Who hears about it, and who pays for the adjustment?

A sixth question comes before all of these when one of the systems is old: does it offer a way in at all? Many older systems have no API. They can often still be connected, through their database or their files, and which route is safe has to be checked, not assumed. The same inquiry shows how much else already depends on that system, which is one of the questions in legacy applications: what makes one, and what to ask.

Connector, configuration or custom

Once the path is described, there are usually three ways to build it, and a proposal should say which it has chosen and why.

• An existing connector. Someone has already built the link between these two products. Quick, and limited to what its maker decided it should do.
• Configuration of something you own. One of the systems, or a platform you already pay for, can be set up to do the exchange.
• A custom integration. Written for this path. It does exactly what is needed, and it is yours to maintain.

None is better in general. A connector that covers the path and handles the bad days is the right answer. A custom build is right when no connector does, or when the rules are particular to the business.

The same questions apply when the system on the other side is an AI service, with a few more added, set out in AI integration: connecting a model to systems you run. And when the real trouble is a whole process run by hand across several systems, the integration is one part of a larger piece of work, described in process automation: what to settle before building.

The nursery's one path turns out to be two

Back to the nursery, which is invented. Before accepting the two weeks, the owner asks for the five things in writing.

The path turns out to be two paths. Orders go from the site to stock. Stock levels have to come back, because the site is currently selling plants that sold out on Friday, which nobody had connected to the retyping. Truth: the stock system is right about quantities, the site is right about what the customer asked for. Bad days: the stock system is switched off for a backup every night for forty minutes, exactly when garden centres place evening orders. Those orders must wait and be sent afterwards, once each. A plant that exists on the site and not in stock stops that one order and tells a named person, and the rest go through.

On the way in: the stock system is twelve years old and has no API, and its supplier confirms it can import a file every five minutes. That is good enough for orders and for stock levels.

The estimate changes from two weeks to five. It is a better estimate, and the owner now knows what the five weeks buy: the morning retyping ends, the site stops selling what is gone, and when an order fails, someone knows before the customer does.

Before you sign off an integration

• Write down each path separately: from where, to where, what data.
• For each piece of data, name the system that is right.
• List the bad days you have already had by hand. They will happen to the integration too.
• Ask any supplier how a failure will be noticed, and by whom.

Those answers, for one integration path, are what our API and systems integration work begins with: a fixed-scope Integration Assessment that sets out what has to move between the two systems, where it can go wrong, and the smallest reliable way to connect them, before anything is built.

Sources

• Gregor Hohpe and Bobby Woolf, Introduction, in Enterprise Integration Patterns, 2003.