JMP® shines a light on energy use

Central Hudson Gas & Electric delves into 50 years of data to help project tomorrow’s energy requirements

ChallengeTo assess patterns of energy use for each customer class to determine how rates might be adjusted by the service provider.
SolutionUsing JMP to analyze and model 50 years of hourly data to understand the electric usage patterns for different customer groups and how the data interrelates on daily, monthly and yearly cycles with temperature.
ResultsCentral Hudson Gas & Electric has developed descriptive models that help them understand ways to serve the various customer cohorts economically and predictive models that help forecast hourly electric demand.

In his capacity as a cost-of-service analyst for the Central Hudson Gas & Electric Corporation (CHG&E), Larry Arvidson builds models. These models allow him to assess patterns of energy use for each customer class to determine how rates should be adjusted. Recently, he has been using JMP statistical discovery software from SAS to do so – and, in the process, has become a true believer in JMP software’s considerable prowess.

His models, though, remained a work in progress. He decided to take his research to the annual JMP users conference in Research Triangle Park, North Carolina to learn what he could from other JMP users about how to improve those models.

And improve them he did. It turned out he was over-fitting, which he corrected, and today he continues to fine-tune his models.

Arvidson knows his work very well, but he is not a statistician by training. That’s where JMP comes in – assisting Arvidson as he delves into a wealth of data to better understand energy-use patterns.

A ‘marvelous data set’ unfurled

Central Hudson Gas & Electric Corporation serves approximately 295,000 electric customers in the Hudson River Valley of New York, a roughly 2,600-square-mile area. For cost-of-service studies, Arvidson runs a load research program that gives hourly demand by customer class (residential, commercial and industrial, with subsets within each), which is necessary for allocating costs per class.

One day Arvidson discovered, as he describes it, “this marvelous data set of almost 50 years of hourly data, going back to 1960. And, given that I had this powerful tool called JMP, I realized I could put the data together and build a better model than had previously existed.”

So he began working with the data.

Arvidson’s intent was to identify patterns in electric use by visually exploring a data set of hourly electrical load and temperature from January 1960 to February 2008, break down the load by customer class and then build and test models to predict demand by class.

The objective of Arvidson’s research is to provide a better understanding of this data and how it interrelates on daily, monthly and yearly cycles with temperature, and then ultimately to use the information to deliver energy as cost-efficiently as possible.

JMP is a relatively new tool for Arvidson, but, he says, “being a SAS® lover, JMP came naturally,” where other tools had failed.

These are tremendous data sets – up to 440,000 rows of data. “Fifty years of hourly data,” says Arvidson, “was too much for spreadsheet programs to absorb.

“And even if you got the data loaded, spreadsheets are too clumsy to manipulate it. One of the great things about JMP is that it can take in data in many different forms – a text file or even comma-separated or proprietary spreadsheet formats – and then you can stack it, split it or rearrange it in order to simplify your analysis.

“I first built a lot of graphs and rearranged the data set to get a good feel for the descriptive part of it – where there might be missing values, for example, or how temperature could be correlated, or not, with hourly load.”

He began studying the growth in the CHG&E system over that 50-year time frame to determine how to model it, examining annual, weekly and daily cycles.

“I ordered a book on time series analysis from JMP’s great support staff,” Arvidson says, “and they sent it to me along with an exercise. I started working through the book chapter by chapter, and I came to chapter three, which dealt with using sine and cosine formulas to model cycles.

“So I experimented with them, and the more I experimented, the better it looked. I showed my analyses to a JMP account executive in New York City, and he recommended I attend the JMP conference to share my work with others.”

He attended the conference as both presenter and pupil.

Arvidson had accomplished a disaggregation of the system load for a three-year period and then shown the contribution of each customer class to the hourly system load over that span. However, he was still looking for ways to improve his models.

“Basically, there were two purposes to what I was doing,” says Arvidson. “First, to try to build a better mousetrap, and then to clearly show the contribution of each customer class to the system.”

The gamut

Arvidson uses JMP to examine current data as well. CHG&E regularly collects hourly data from 500 or so meters. Arvidson loads that data into JMP, then breaks it down by customer class and computes demand per customer per hour for each class of customer.

“In essence,” he says, “we’re trying to get to kilowatts per customer per hour by rate class, and there are 10 sub-classes of customers. So there’s a lot of data.”

Arvidson makes comprehensive use of JMP software’s many tools. He employs table summaries, subsetting, sorting, stacking and splitting on a regular basis.

He describes the effectiveness of JMP at identifying outliers and visualizing patterns using coloring by columns.

“If I color by hour, I get a pattern of hourly usage on my graphs; if I color by year, I see the difference among years.

“I can analyze, explore and visualize the data in any column. I’m often adding a formula to help check the data, comparing one row against the previous two rows to make sure the change from one hour to the next is not too great. Then there’s the ability to hide and exclude things by row or column.”

He uses all the Analyze platforms in JMP – Distribution, Fit Y by X, Matched Pairs and Fit Model with profiling and simulations as well as Time Series with spectral density – the gamut.

“In sum,” says Arvidson, “JMP is just so quick and powerful and offers so many options for attacking the data.”

What’s next? “Now it’s over to a wider team that includes the business and marketing experts,” said Arvidson. “That team may choose to determine ways to serve the various customer cohorts economically and efficiently and possibly develop programs to encourage and incent conservation by customers in a particular class.”

Arvidson concludes, “As for the data, I will keep examining it. As for the models, I’ll keep refining them to keep pace with changing conditions.”

Power lines
One of the great things about JMP is that it can take in data in many different forms – a text file or even comma-separated or proprietary spreadsheet formats – and then you can stack it, split it or rearrange it in order to simplify your analysis.
Larry Arvidson

Cost-of-Service Analyst
Central Hudson Gas & Electric Corporation

Contact JMP Sales

Back to Top