Combining Monads: Revisiting the API sequencing problem

So far we have written functions that use a single monad. We will now see how we can combine the functionality of multiple monads. To do this, let us revisit the API sequencing problem.

The API Sequencing Problem

As before, we want to determine the country of a person by using web APIs to determine their IP address and then calculate the country from that.

From the diagram, we see that we can do this by calling api.ipify.org and passing the ip address to ipwho.is. In case the call to ipwho fails then we have a backup flow that calls ipinfo.io followed by countrycode.dev. If the backup flow also fails, then computation is an error, otherwise the computation succeeds and we have the country as the result.

The Result monad introduced in that article will keep track of the success or error states and allows us to just compose the various functions together to create the overall computation.

Refer to the previous article to understand how we solved the problem with monads, as we will be building upon that solution here.

Monad Example: Sequencing API Calls

A common requirement is to call various APIs in a sequence, any of which may fail. We see how to implement this requirement in a clean way using monads.

Playful PythonSiddharta Govindaraj

Now we add in a new requirement: Apart from computing the result, we also want to know the trace of which services were used in calculating the answer. Something like this:

Connecting to https://api.ipify.org/?format=json
Success
Connecting to http://ipwho.is/122.13.22.253
Failure!
Connecting to https://ipinfo.io/122.13.22.253/geo
Success
Connecting to https://countrycode.dev/api/countries/iso2/IN
Success

This kind of trace can be easily handled by the Writer monad.

Writer Monad and the Tracing Robot

After that small discussion on Y Combinator, we are back to monads! In the previous article on monads, we saw how we can use the the Result monad to chain a sequence of API calls together while automatically handling any failures along the way. In this article we will learn

Playful PythonSiddharta Govindaraj

Taken individually, the problem is simple to solve. The complication here is that we need the functionality of both the Result monad as well as the Writer monad.

Combining Monads

At first glance, it might seem as though we can just double wrap the value with both the monads like this

return Trace(Ok(value), logs)

Unfortunately, this doesn't work because when you do a map or a flatmap the function will only unwrap the outermost monad before applying the function.

What we need to do here is to write a new monad that has a combination of both the functionality. Fortunately, that is easy to do.

So let us create our monad from first principles. A monad wraps

• a value
• along with some additional context. In this example, the context contains two pieces of information: success/failure, and the logs

That leads us to these constructors

class TraceOk:
    def __init__(self, value, logs):
        self.value = value
        self.logs = logs

class TraceError:
    def __init__(self, error, logs):
        self.error = error
        self.logs = logs

Notice how the constructors are just a combination of the constructors of Result and Writer

Implementing map

Next, we need to implement the map function.

In the TraceOk state, the monad takes the value and applies the function to the value, creating a new monad with the new value but retaining the same context

class TraceOk:
    def __init__(self, value, logs):
        self.value = value
        self.logs = logs

    def map(self, fn):
        return TraceOk(fn(self.value), self.logs)

In the TraceError state, it does nothing, just returning itself

class TraceError:
    def __init__(self, error, logs):
        self.error = error
        self.logs = logs

    def map(self, fn):
        return self

Implementing flapmap

Finally we need to implement flatmap

flatmap is supposed to take the value and apply the function to the value. However, the output value will itself be a monad value, either a  TraceOk or a  TraceError. We need to merge the context returned by the function with our own context and flatmap should return that merged output. In our case, that means merging the logs together before returning the output.

That leads to an implementation like this

class TraceOk:
    def __init__(self, value, logs):
        self.value = value
        self.logs = logs

    def map(self, fn):
        return TraceOk(fn(self.value), self.logs)

    def flatmap(self, fn):
        match out:= fn(self.value):
            case TraceOk():
                return TraceOk(out.value, self.logs + out.logs)
            case TraceError():
                return TraceError(out.error, self.logs + out.logs)

The TraceError state is simple. flatmap would not do any computation and will return itself just like map

class TraceError:
    def __init__(self, error, logs):
        self.error = error
        self.logs = logs

    def map(self, fn):
        return self

    def flatmap(self, fn):
        return self

With that our monad implementation is complete.

Using the monad

Now that we have the monad, we can put it to use. Most of the code remains the same, we just need to modify the make_get_request function to return the new monad

def make_get_request(url):
    logs = [f'Connecting to {url}']
    try:
        response = requests.get(url)
        code = response.status_code
        if code == 200:
            logs.append('Success')
            return TraceOk(response.json(), logs)
        logs.append('Failure!')
        return TraceError(f"status {code} from {url}", logs)
    except Exception as e:
        logs.append('Failure!')
        return TraceError(f"Error connecting to {url}", logs)

The function now logs which url it is connecting to and whether it was a success or failure and stores that context in the monad.

The rest of the code is mostly unchanged. The functions compose the relevant computations using map and flatmap and are blissfully unaware of the context being managed automatically by the monad

def get_ip():
    return (make_get_request("https://api.ipify.org/?format=json")
            .map(lambda data: data["ip"]))

def get_country_from_ipwhois(ip):
    return (make_get_request(f"http://ipwho.is/{ip}")
            .map(lambda data: data["country"]))

def get_country_code_from_ipinfo(ip):
    return (make_get_request(f"https://ipinfo.io/{ip}/geo")
            .map(lambda data: data["country"]))

def get_country_from_country_code(country_code):
    return (make_get_request(f"https://countrycode.dev/api/countries/iso2/{country_code}")
            .map(lambda data: data[0]["country_name"]))

Finally, we compose the functions as per the required behaviour

def get_country_from_ip(ip):
    match output := get_country_from_ipwhois(ip):
        case TraceOk():
            return output
        case TraceError():
            return (TraceOk(ip, output.logs)
                    .flatmap(get_country_code_from_ipinfo)
                    .flatmap(get_country_from_country_code))

def get_country():
    match output := get_ip().flatmap(get_country_from_ip):
        case TraceOk():
            print(f"You are from {output.value}")
            print(*output.logs, sep='\n')
        case TraceError():
            print(output.error)
            print(*output.logs, sep='\n')

get_country()

The only change here is in get_country_from_ip where if the main flow fails, then the code extracts the logs up to that point and passes it along to the backup flow code. This way the logs for the entire sequence of both the main flow as well as the backup flow will be preserved.

If you were to block ipwho.is in the firewall and then run the code you will get an output exactly like what we wanted

You are from India
Connecting to https://api.ipify.org/?format=json
Success
Connecting to http://ipwho.is/122.13.22.253
Failure!
Connecting to https://ipinfo.io/122.13.22.253/geo
Success
Connecting to https://countrycode.dev/api/countries/iso2/IN
Success

We can clearly see how the code execution proceeded. We see which call failed and how the code went to the backup flow. All by just changing the monad being used. How cool is that?