Ted CrossMachine learning provides an opportunity for E&P companies to improve their data-driven decision-making and tackle problems that traditional analytical approaches have struggled to solve. Nevertheless, machine learning is not a shortcut. It requires more than just running data through an algorithm to generate insights. Like any other analytics method, it faces similar challenges
In this article, we will look at some of the most common mistakes in developing machine learning capabilities that we have seen while working with oil and gas operators in every major shale play across the western hemisphere. Our journey has exposed us to a plethora of operator errors, and yes, we’ve made a few ourselves too!
1. Not aligning the asset team with corporate
Achieving synergy between the asset team and the corporate division is crucial for advanced analytics projects. E&P companies can struggle with hold-ups due to the lack of communication between their asset team and the corporate wing, finding themselves constantly on the back foot, navigating roadblocks that could have been avoided with proper alignment.
Smaller, private-equity-backed operators may have the upper hand in maintaining alignment due to the compact nature of their teams, but larger corporations often struggle with the challenge of effectively communicating across teams. The key is to pivot the mindset from viewing data-driven workflows as threats to embracing them as opportunities. This shift can foster buy-in across all levels of the organization.
2. Failing to set firm business goals
In the enthusiasm to harness the power of machine learning, some operators jump in without having a concrete plan for how it will add value to their business. This approach is similar to setting sail without a compass—you may move forward, but without a definite direction, it can lead to wasted time, resources, and effort.
Take for instance, a small operator in the Permian Basin that starts leveraging machine learning without having a clear goal. They probably will end up using it like a shiny new toy, dabbling in multiple areas without achieving substantial results.
To avoid this pitfall, start by setting tangible goals. Examples could range from “forecast our 2024 PDP production”, “decide on proppant intensity for Wolfcamp B wells in new development area”, to “screen next two opportunities in the DJ using machine learning”.
3. Being a perfectionist with subsurface data
The quest for the ‘perfect’ subsurface data can prove to be a barrier. While updating subsurface models is vital, it’s important to realize that perfection is often a moving target. Waiting for the ‘perfect’ data before kick-starting machine learning initiatives can result in missing out on opportunities for growth and optimization.
Imagine an operator spending six months refining their subsurface model before they are ready to implement machine learning… There is a big chance their competition has already started making strides using a ‘good enough’ subsurface data model and gradually iterating it over time. Don’t let it be you.
We have basin-wide subsurface grids available to our customers in every shale play for exactly that purpose – to give our customers a great starting place.
4. Not keeping track of units and formats
Seemingly trivial mistakes related to units of measurement and formats can lead to disproportionately large errors.
Quick embarrassing story – a couple of years ago, we were working on a forecasting pilot with a customer. The forecasts kept coming back way lower than expected. We went back to the drawing board, updated our subsurface model, and changed our filtering, but eventually, we discovered the issue was a barrels vs. gallons completions fluid discrepancy.
You’d be amazed how commonly we have seen this type of issue. It’s imperative to have a robust tracking mechanism, which is why we have full units & format support within Novi Data Engine.
5. Building “single-target” models
Machine learning models come in different forms, with varying levels of complexity and detail. Often, operators fall into the trap of creating “single-target” models, which predict a singular data point like one-year cumulative oil or Estimated Ultimate Recovery (EUR). While a single-target model is insightful, it is restrictive and limited in its utility. We also generally discourage focusing your forecasts on EURs, since they are an estimate and incorporate significant interpretation.
Take for instance a mid-sized operator that has developed a model that is excellent at forecasting one-year cumulative oil but lacks the flexibility to predict other essential data points like quarterly production or accurate economics.
Novi Model Engine takes care of “multi-target” forecasting for you, predicting a full-time series of
production. Typically, it will use machine learning to forecast for the first three years, followed by a modified Arps extension to get volumes to a EUR, but that is configurable!
6. Ignoring Secondary Streams
Often, operators overlook secondary production streams—until they are suddenly faced with a major issue. One operator we partnered with experienced this first hand. They had a pad curtailed due to an unexpected lack of Salt Water Disposal (SWD) capacity, which resulted in significant production delays and financial losses.
This example underscores the importance of forecasting secondary streams. Machine learning models can accurately predict these streams, and simplifies the process, letting you select each stream you want to forecast.
7. Not Quantifying Uncertainty
In an industry where accuracy can directly impact profitability, understanding the uncertainty associated with any forecast whether machine learning, physics-based, or analog-type curve-based, is critical. Many machine learning models output a single value when generating a forecast, but at Novi, we believe in providing a fuller picture.
Novi has incorporated uncertainty estimates into its standard forecasting outputs. We have found that operators, and especially management, become much more comfortable with forecasts once they understand the uncertainty. There are different ways to tackle this, but you must make sure to have a plan to get organizational buy-in. Quantified uncertainty can also be used for advanced workflows, like ranking risk-adjusted inventory, or designing pilots
8. Not designing a proper evaluation
“How do we know the model is good?” Of course, everyone using machine learning tools must ask this question. Ensuring your machine learning model is performing as expected requires a successful evaluation that has two components: a well-defined framework and a carefully selected benchmark. But not all operators invest the necessary time and resources into this crucial step.
At Novi, we have built “time machine” functionality into Model Engine, allowing the user to set a cutoff date beyond which data will be withheld from the model–this approach simulates the real-world scenario of using a forecasting model. In addition to the evaluation framework, machine learning practitioners should choose a benchmark, such as pre-drill AFE forecasts.
Why is this important? ML practitioners often have no idea how accurate a forecast should be. They might say “within 10%”, but is that in aggregate? At the well-level? At 90 day, 1 year, or something else? How many of their pre-drill forecasts were within 10%? The benchmark grounds the analysis, setting up the company for success.
9. Not building an integration plan
It’s not enough to have a successful evaluation and have built an accurate machine learning model; integrating it into existing workflows is equally important. As with any new workflow, it is best to integrate with existing processes, such as AFE reviews, quarterly planning, or teaser screening.
To start, you can present the machine learning forecasts in parallel with the existing method, and over time, you can work them more substantially. For instance, you might have one or two quarters where machine learning forecasts are presented alongside standard reservoir engineer PDP forecasts. After building trust, you may shift to having the reservoir engineer only hand-forecast a subset of wells where the machine learning forecast has changed dramatically since last quarter or was significantly off.
10. Underestimating resources required to write, QC, and maintain machine learning code
A widespread misconception about machine learning is the belief that the creation of a model is a one-time effort. Writing, quality checking, and maintaining machine learning code requires dedicated resources.
Often, a curious reservoir engineer will build & run machine learning models on their desktop, maybe with some intriguing results. This is a far cry from “production” code, which is built, tested, and used continuously.
We have seen operators successfully do this, but you are looking at a minimum of several FTEs dedicated exclusively to the machine learning project, with project costs potentially breaking $1MM/year.
Machine learning code isn’t static, but rather, it’s dynamic and constantly evolving. This dynamism is due to a variety of factors such as new data becoming available, alterations in industry best practices, and advancements in machine learning techniques. Consequently, maintaining machine learning code requires continual attention and effort from a dedicated petrotechnical team.
11. Viewing machine learning as an all-or-nothing proposition
Many operators fall into the trap of viewing machine learning as a full replacement for their current workflows. In reality, machine learning should be viewed as a tool that enhances and complements traditional methods, not a solution that overtakes established systems overnight.
Machine learning offers the best chance at learning from the hundreds of thousands of unconventional wells drilled over the last decade, but of course rate-transient analysis or fracture modeling can still be useful at understanding your heavily-instrumented science pads. It’s about harnessing the best of both worlds.
Machine learning can improve your forecast accuracy, provide data-driven insights that complement traditional methods, and save your reservoir engineers’ time.
Conclusion
As a reservoir engineering team, you have most likely faced one of the challenges listed above, but trust me when I say that this is only a fraction of the lessons we learned while developing forecasting models for leading oil and gas companies. It’s easy to be swayed by the promise of advanced analytics and overlook the complexities that come with it. However, by avoiding common pitfalls and thoughtfully and strategically integrating machine learning into workflows, operators can unlock tremendous value.
Without a doubt, machine learning has enormous potential. It is important to remember, however, that while technology is powerful, it does not represent the entire picture. The true value is provided by those who effectively use this tool to drive decision-making: engineers, geologists, data scientists, and others.
Why build a machine learning model when you can plug & play a prebuilt one?
Machine learning model building can be a tedious task. It’s challenging and incredibly time-consuming, and there’s always a risk of error. But what if there was a better way?
At Novi Labs, we offer you the chance to skip the laborious process of building models from scratch. We have the best oil well drilling optimization tools and processes—all in one place.
Instead of navigating the machine learning maze alone, why not let us be your guide?
You can book a call with one of our experts or learn more about our platform.
News, trends and data for the US upstream industry
Novi Energy Newsletter
