As consolidators move to acquire acreage in core shale basins, smaller operators are looking for opportunities in underappreciated basins and fringe areas in core basins. This brings up the question of whether machine learning based approaches are valid in areas that do not have thousands of historical wells to learn from in similar rock and with a variety of spacing and completion examples. An operator seeking to expand into such an area must deal with higher uncertainty, whether through machine learning or any other forecasting method.
Forecasting the Northwest Shelf of the Delaware Basin
One of these new areas of interest is the Northwest Shelf of the Delaware Basin. These wells are primarily targeted at the First Bone Spring Lime, Paddock, Blinebry, San Andres and Abo formations. Our first attempt to create a forecasting model here was to only use these zones on the crest of the shelf to train and test a model. Data was cutoff at 2022-02-10 to test the model accuracy on wells that came online after this date. Figure 1 shows the wells in these zones along the Northwest Shelf that were used to train this model. Restricting to this area, and to horizontal wells with modern style designs (lateral length > 3000 feet, fluid > 400 gals/ft, proppant > 500 lbs/ft) leaves us with only 363 wells.
Figure 1: Northwest Shelf horizontal wells with modern completion designs.In the post 2022-02-10 test set (Figure 2) , we have 21 wells that we can use to judge the accuracy of the model. Figure 3 shows the median absolute percent error (MAPE) and median percent error (MPE) for these 20 wells. We use MAPE as the preferred method of judging error because it is not easily moved by outliers, and can be easily understood as it relates to missing produced volumes. Half of the wells in the test set have an absolute percentage error less than the MAPE. The median percent error is instructive for determining if the model is systematically over or under-forecasting the test set wells. In Figure 3, we see MAPE scores in the low 20% range, similar to core Delaware Basin models. However, this model is systematically under-forecasting production of our post 2022 wells, with the median percent error at -20% at around 9 months of production.
Figure 2: 21 horizontal wells on the NW Shelf comprising the test set. These wells comprise the post 2022 wells. The model was trained only on data before 2022.
Improving Forecast Accuracy with Expanded Data
Given the data constraints we have to deal with in this area, how can we improve this model?
Figure 3: Model test set errors for NW Shelf only model. This model used the 363 modern wells on the NW Shelf to forecast the 20 post 2022 wells in these zones. Mean Absolute Percent Error (MAPE) was inline with larger Delaware Basin models, but the model systematically under-forecasted the post-2022 test set.One idea may be to include more data from nearby areas in the northern Delaware Basin. While different geologically, these wells provide the model with more examples of varied spacing and completions. This increases the well count from 363 to 885. Figure 3 shows the new dataset with northern Delaware wells included.
Figure 4: Well set in the expanded northern Delaware + NW Shelf model. After observing the systematic under-forecasting of the NW Shelf only model, additional nearby information from other zones may help forecast accuracy for the NW Shelf.In Figure 5, we can see the yearly trend of wells that are included in the model training and testing. In the plot below, blue represents wells that were excluded by our filters for modern-style wells. Yellow represents modern-style horizontal wells included in the training set, and red represents the overall test set wells, including northern Delaware Basin wells away from the Northwest Shelf. We observe that only in 2015+ do a majority of wells fall into the “modern” criteria.
Figure 5: Well count of northern Delaware Basin and NW Shelf wells used in the model. Yellow represents the model training set, blue shows wells that were filtered out due to a non-modern design, and red shows the test set, composed of wells which came online after Feb 2022.Results: Comparing Model Accuracy and Bias
With this model, we can now forecast our post-2022 test set wells and compare the results to the NW Shelf only model. Figure 6 shows model results for the same set of 20 post-2022 NW Shelf wells. The model utilizing both NW Shelf and northern Delaware Basin wells (in black) shows lower median absolute errors at almost all time points, and shows a percent error closer to 0. The generalized model has performed better in both accuracy and bias. Furthermore, the magnitude of these errors is similar to models for the full Midland and Delaware Basins, validating to the utility of machine learning in new development areas.
Figure 6: Comparison of errors in the NW Shelf only model (red) and the generalized northern Delaware + NW Shelf model. This comparison isolated the 20 NW Shelf test set wells. The upper chart shows the Median Absolute Percent Error (MAPE) for each model. At all time points except 60 and 210 days, the generalized northern Delaware model shows a lower error. After 102 days, the generalized model median error is almost exactly zero, meaning 10 wells were over-forecasted and 10 were under-forecasted. The generalized model is not systematically biased, unlike the NW Shelf only model.Conclusion
This study shows that incorporating nearby data sources can significantly improve the accuracy of production forecasts in data-limited areas. By integrating northern Delaware Basin data, we reduced systematic error and increased overall accuracy, achieving results comparable to models developed for core basins. For operators expanding into emerging fields, these findings highlight the value of machine learning when thoughtfully calibrated with additional data, providing actionable insights even in areas with limited historical well information.
Key Takeaways
- Filtering to NW Shelf modern horizontal wells (lateral length > 3,000 feet, fluid > 400 gals/ft, proppant > 500 lbs/ft) yielded 363 training wells.
- Using a 2022-02-10 data cutoff produced a post-2022 NW Shelf test set of 21 wells to evaluate forecast accuracy.
- The NW Shelf-only model delivered MAPE in the low 20% range but showed systematic under-forecasting with ~-20% median percent error at ~9 months.
- Adding nearby northern Delaware Basin wells increased the modern-well dataset from 363 to 885 and expanded spacing and completion examples.
- In the expanded model, most modern-design wells occurred in 2015+ based on the yearly distribution of included versus filtered wells.
- The generalized northern Delaware + NW Shelf model reduced MAPE at most time points and brought median percent error close to 0 for the 20 NW Shelf test wells.
