The power of predicting spot market freight rates to secure capacity

Understanding the freight market is a difficult and oftentimes elusive practice for shippers. Despite best efforts to control freight spend through contracted rates, shippers remain at the mercy of carriers when the market shifts and tender rejections lead to an increased need to source from the spot market. This was particularly true in the last two years, as shifting demand patterns created capacity constraints and sent freight rates soaring.

Predicting spot market freight rates is a powerful tool for navigating the changing landscape of truckload procurement, and is fraught with challenges, but also with opportunities. This article examines the dynamics of the spot market and the challenges of spot rate prediction and explores how AI and machine learning are the keys to prediction accuracy and actionability in the spot freight market.

The components of a spot market rate

Spot market freight rates are inherently volatile and change daily based on numerous variables. In their simplest form, spot rates reflect supply versus demand – the number of trucks and drivers available versus the number of available loads, but in reality, this is only one component to be considered relative to other factors.

Market conditions

Macroeconomic conditions, like supply and demand, are the largest driver of spot market freight rates. Some market conditions are foreseeable - think rate increases during produce and peak holiday seasons. Others are unforeseeable, as witnessed in the wake of COVID-19 with sudden shifts in consumer habits and demand for imported goods.

Load characteristics

Spot market freight rates are also dictated by specific load details. Shippers must consider these factors when planning to make spot freight more “attractive” to carriers, particularly in a constrained market.

Lane: Shipping into a head haul market (major hubs like Los Angeles or Chicago) typically costs less than shipping out of those markets. Shipping to remote destinations can be more costly since carriers likely have to deadhead for a reload.

Commodity: Lower-value goods like paper, packaging, and grains tend to be less costly than high-value goods like electronics and machinery due to stricter service requirements. Gross weight also impacts rates, as heavy cargo puts more strain on equipment and requires more fuel.

Equipment: The type of equipment required to move a load impacts availability and thus spot rates. Undoubtedly, a reefer unit costs more on average than a standard dry van or flatbed.

Timing

Since the spot freight market is constantly fluctuating, timing plays a crucial role, but can be difficult to navigate. On one hand, securing rates with significant lead time may result in lower spot rates since carriers, and specifically brokers, have more time to negotiate and secure capacity. On the other hand, carriers may inflate rates for loads shipping in a week or more if they are not confident in potential market shifts.

Challenges and shortcomings of predicting spot freight rates

Anticipating shifts in the market and predicting spot freight rates is an urgent priority for shippers and freight brokers attempting to regain control of freight costs. Most large U.S. brokerages and freight technology companies have developed systems to contextualize current market trends and predict future spot rates, with varying levels of success. Several challenges contribute to the shortcomings in these systems and limit their ability to consistently predict accurate spot freight rates.

Complexity

The first issue in spot freight rate prediction is the sheer complexity of modern supply chains. It is challenging to account for variables that may cause rate deviation - for example, loads that require a drop trailer, multi-stop, HVHR, or other accessories. Each unique carrier, lane, driver, and mode offers an opportunity for deviation from the ‘average’ rate.

Instability

Spot pricing is based primarily on whether there is an excess or shortage of capacity at any given moment, and that capacity changes daily. Instability can be attributed to things like major economic shifts as witnessed with the 2020 boom of e-commerce and import demand, or by natural disasters and weather events, which create unexpected and dramatic rate fluctuations that ripple throughout the carrier network. Accounting for this level of instability has proven difficult for spot rate prediction systems.

Data limitations

Prediction technology is limited by the breadth and accuracy of data available to it. Freight brokerages may have a clear dataset of loads moved within their internal network but lack data on the freight market as a whole. Outside platforms attempt to create a view of the freight market based on nationwide tender acceptance, lane rates, and historical data but often do not have significant data on smaller fleets and owner-operators, who move a large amount of the non-contracted freight that enters the spot market. These technologies can show historical data but provide little insight into future trends and often present spot rate ranges with too much variability to be useful.

Considering the challenges in predicting spot market freight rates, it’s natural to wonder: is good prediction possible?

The power of predicting spot freight rates

Let’s start by acknowledging that if it is possible to predict something, certainly humans have tried. Baseball scores, aeroplane crashes, election outcomes, stock performance - you name it, someone has attempted to predict it. So it is not the ability to predict, but the ability to predict with accuracy that is in question. The power of prediction lies in limiting error and improving accuracy - the portion of predictions that fall within certain acceptable bounds.

Defining prediction accuracy

Defining the boundaries of accuracy is critical when determining a system’s success in predicting spot market freight rates. As a general rule, any technology with an error rate over 15% is essentially useless and ineffective. In this scenario, a load in the U.S. that costs $1,000 would be off by $150 on average, far too risky for companies to consider as a viable option. On the opposite end is defining how accurate a prediction system needs to be to provide value – at what point does the reward of prediction technology outweigh the risks?

The answer to this question depends largely on who is asking. For a shipper buying capacity in the spot market, a higher level of error may be acceptable. Sometimes they buy on the high end, sometimes on the low end, but it tends to average out. On the other hand, a broker using predictive technology to bid on freight is more prone to the “selection effect” and may not benefit from systems with a higher error rate.  In the broker pricing scenario, a broker is bidding against other brokers – if they bid too high, they are not awarded the load, but if they bid too low, they win the business but may be unable to buy capacity at that price, ultimately resulting in a loss. Over time, losses accumulate, but the broker is not making up for those, since they are not awarded business that is priced on the high end.

Accuracy is critical to spot freight predictions, as errors in the negative direction can be devastating to budgets and profitability. With so many variables at play, how can we get to a position of predicting spot market freight rates with confidence?

It’s important to note, most major freight brokerages have implemented spot rate prediction software in some form, with varying levels of success. Most of these systems were incubated before 2020 when the freight market was comparatively stable and predictable. Few - if any - could hold up to the sudden volatility of the freight market through the pandemic, causing companies to switch systems on and off while they attempted to adjust algorithms to account for instability. Most brokerages have not publicly released accuracy data of their spot rate prediction systems, but, strictly anecdotally, it is likely there were significant losses incurred in the early stages of implementation.

Levels of automation and AI

The risk associated with early spot rate prediction systems, combined with the high level of technological sophistication required to build them, nearly ensures that not all shippers and brokers will adopt fully automated spot rate technology. These companies will, however, likely use AI to augment their spot market knowledge and improve decision-making. AI technology in spot rate prediction can provide several levels of automation:

Human in the loop: This is the lowest level of automation. AI gives guidance to a human on spot market rates, but the human ultimately determines pricing, carrier selection, and tendering.

Human on the loop: Mid-level automation. AI makes suggestions on spot market rates and a human approves the suggestions or makes counter-offers. Upon approval, AI takes over quoting, carrier negotiations, and tendering.

Human out of the loop: Closest to full automation. AI auto-prices spot loads and tenders to carriers without any human input or approval.

At first glance, human out-of-the-loop technology may seem incredibly risky, particularly in an industry that has historically relied on professionals with years of market knowledge and experience to price spot freight. To understand the benefits of full AI automation we must first understand the power of prediction.

Moving beyond human vs artificial intelligence

The power of prediction and full automation lies in its ability to scale indefinitely. Consider a human tasked with pricing spot freight - a sales representative or pricing agent - how many spot loads could that single person quote in a day? If we assume a qualified and experienced professional can process one quote every five minutes, that would result in 12 quotes an hour, and approximately 100 quotes a day. On the other hand, fully automated prediction AI could process between 2,000 and 5,000 spot quotes in the same amount of time.

From a productivity standpoint, AI clearly outperforms a human in spot rate prediction - but what about quality?  Is AI more accurate in pricing than the human?  The answer to this question is not so simple, as it requires that we think about quality differently. Errors in the 5,000 quotes produced by AI can be analysed from a systemic level. What information did the system have or not have, or what action did it take or not take, that resulted in a given error rate? Once that is identified, humans can adjust AI parameters to systematically perform differently to improve accuracy. Conversely, for errors in the 100 quotes produced by a human, we can only analyse them at an anecdotal level because we cannot know what information the human considered or why they chose a particular action. There is no formula for human error, so it is difficult to adjust and linearly improve accuracy.

The question about automation and artificial intelligence inevitably turns to one of replacing humans and putting people out of work. But to reframe this question – just because humans have been doing the work, is it necessary? Is it productive? What could those same people be doing if they were not doing manual calculations and spot pricing?

To answer these questions, we must remember there are innumerable tasks that are simple for an average human but would be nearly impossible for AI and robotics. Consider tasks that a human toddler could do, like walking across a room full of people, picking up a coin, or identifying a hand-drawn image of a cat. Each requires a high level of specificity in programming to accomplish even one of those tasks - AI and robots simply do not have the capacity to adapt to unknown, changing environments the way humans do. Similarly, we must acknowledge that humans are not innately designed for complex mathematics and statistics. We are capable of doing them, but we lack the working memory and procedural processes to do them as efficiently and accurately as AI systems can.

Harnessing the power of AI to predict spot market freight rates

AI is a powerful tool for spot freight rate prediction, whether through the augmentation of human skills and expertise or full automation of the price-to-tender process. AI is an ever-evolving system that uses machine learning to improve accuracy over time, far more efficiently than an average human could. With the right AI technology, procurement teams can secure the best possible spot market rate for each shipment, even under capacity and time constraints, without the additional stress of manually managing bids.  AI helps procurement teams and shippers achieve greater operational efficiency and improve spot freight costs while providing actionable insights into carrier and market dynamics.

Transporeon leverages the power of AI to drive dynamic spot freight pricing and capacity procurement. Their AI technology navigates complexities in the spot market with real-time data on carrier positioning, load density, and market conditions and automates carrier negotiation on the shipper’s behalf to ensure the best outcome for each spot shipment.

Contact Transporeon to schedule a demo and learn more about how an approach to spot freight procurement that is driven by AI and machine learning can be your strongest strategy.