Algorithms Aren’t Always the Answer

Lenders increasingly use AI when making loan decisions. This technology lets finance providers judge many applications faster than would otherwise be possible.

At Reparo, we understand the benefits of these technologies in lending. But we also believe that there are situations where they are insufficient.

The reliance on predefined criteria means that businesses in unique situations are often denied capital. This is the case even if the company has a good business plan and is a strong prospect to repay the loan.

In these situations, we believe algorithms are flawed.

A-Levels Results Show the Downside of Relying on Algorithms

While discussions over the limitations of algorithms are nothing new, these problems were brought into sharp focus by the recent A-Levels results scandal. The inaccurate grades and subsequent government U-turn away from algorithmically generated results towards those awarded by teachers was a high-profile example of the failure of AI.

It is clear that even if the algorithm generated fair results for many—or even the majority—of students, the repercussions in the cases when students’ grades were downgraded unfairly were so significant that they outweighed any positive benefits.

For example, the BBC highlighted a student who almost lost her university place and a £4,000 a year scholarship after the algorithm downgraded her A in English to a B.

Had the decision been upheld, this could have had a significant effect on the student’s future opportunities.

A Lack of Explanation Is a Major Problem

One of the significant issues of AI-generated decisions is the lack of an explanation they provide. Where a human will explain the reasoning behind a particular outcome, AI will not.

Even those behind the technology will often simply point to the various factors the algorithm considers. This leaves those subject to unusual decisions with little understanding of what went wrong.

Take the example of the Tammy Dobbs, a US resident who had her cerebral palsy assistance cut from 56 hours a week to just 32 hours a week by a newly introduced healthcare algorithm.

The Verge explains how neither the patient nor the assessor understood the reasoning behind the decision. Nonetheless, the consequences of the choice were lifechanging—much like those faced by A-Level students in the UK this year, or businesses denied loan funding.

Algorithms Can Produce Unintended Results

Another significant issue is how signals used by the algorithm can produce unexpected outcomes.

In the case of the A-Level results, this manifested itself in some students receiving way above or below the grade they were expected to receive.

One well-known example of this is a U.S. Army system that researchers were training to differentiate between two sets of photos: Set A contained images of camouflaged tanks in trees, and Set B contained images of just trees. The system was tested and could successfully differentiate between the two types of photo.

However, when the system was then trialled in the Pentagon, it did no better than chance at differentiating the photos. It turned out that all the original photos in Set A had been taken on a cloudy day, while all those in Set B were taken on a sunny day. The system had learned to distinguish between sunny and cloudy days rather than photos containing tanks or trees.

While this error was spotted quickly and was ultimately harmless, these types of problem can have a real-world impact.

For example, an Amazon hiring algorithm was unintentionally biased against female applicants after it was trained using applications previously submitted to the company—most of which were from men.

Sometimes, a Personal Touch Can Help

These cases show that while algorithms can be helpful, they aren’t always the answer. That’s why at Reparo, a member of our team always looks at each loan application we receive.

The companies we work with often have complicated situations that require detailed review. We use our experience to analyse each application and talk to the directors of the business to understand their plans.

This allows us to help a wide variety of businesses, including those that have previously been turned down by traditional lenders.

To discuss how we can help your business, get in touch with one of the team at contact@reparofinance.co.uk or on 0161 451 5710.