Do Agricultural Market Advisory Services Beat the Market? Evidence from the Wheat Market Over 1995-1998

March 2001                                                                                                                    

Mark A. Jirik, Scott H. Irwin, Darrel L. Good, Joao Martines-Filho and Thomas E. Jackson [*]

Copyright 2001 by Mark A. Jirik, Scott H. Irwin, Darrel L. Good, Joao Martines-Filho and Thomas E. Jackson. All rights reserved. Readers may make verbatim copies of this document for non-commercial purposes by any means, provided that this copyright notice appears on all such copies.

DISCLAIMER

The advisory service marketing recommendations used in this research represent the best efforts of the AgMAS Project staff to accurately and fairly interpret the information made available by each advisory service. In cases where a recommendation is vague or unclear, some judgment is exercised as to whether or not to include that particular recommendation or how to implement the recommendation. Given that some recommendations are subject to interpretation, the possibility is acknowledged that the AgMAS track record of recommendations for a given program may differ from that stated by the advisory service, or from that recorded by another subscriber. In addition, the net advisory prices presented in this report may differ substantially from those computed by an advisory service or another subscriber due to differences in simulation assumptions, particularly with respect to the geographic location of production, cash and forward contract prices, expected and actual yields, carrying charges and government programs.

The purpose of this report is to address two basic performance questions for market advisory services in wheat: 1) Do market advisory services, on average, outperform an appropriate market benchmark? and 2) Do market advisory services exhibit persistence in their performance from year-to-year?  Data on wheat net price received for advisory services, as reported by the AgMAS Project, are available for the 1995, 1996, 1997 and 1998 crop years.  Not only do market advisory programs in wheat consistently fail to “beat the market,” their performance is significantly worse than the market.  On average, market advisory service performance is about $14 per acre below benchmark revenue, an economically non-trivial amount by any reasonable standard. The predictability results provide little evidence that future advisory service pricing performance can be predicted from past performance. 

Do Agricultural Market Advisory Services Beat the Market?

  Evidence from the Wheat Market Over 1995-1998

Farmers in the US continue to identify price and income risk as one of their greatest management challenges.  Using a survey of midwestern grain farmers, Patrick and Ullerich (1996) report that price variability is the highest rated source of risk by crop farmers. Coble, Patrick, Knight and Baquet (1999) survey farmers in Indiana, Mississippi, Nebraska and Texas and find that crop price variability, by a wide margin, is rated as having the most potential to affect farm income.  Norvell and Lattz (1999) survey a random sample of Illinois farmers and show that price and income risk management rank second (following computer education and training) among ten business categories in which farmers identify needs for additional consulting services.  The desire for greater assistance with price and income risk management is not limited to large farms, as the proportion of farmers expressing this preference actually is highest for those operating medium-sized Illinois farms (500-999 acres). 

Farmers view market advisory services as a significant source of market information and advice in their quest to manage price risks associated with grain marketing. In a rating of seventeen risk management information sources, Patrick and Ullerich (1996) report that the rank of market advisors and computerized information services is surpassed only by farm records.  Schroeder, Parcell, Kastens and Dhuyvetter (1998) find that a sample of Kansas farmers rank market advisory services as the number one source of information for developing price expectations. Norvell and Lattz (1999) find that twenty-one percent of Illinois respondents currently use marketing consultants, and that such consultants tie for first (with accountants), in a list of seven, as likely to be most important to their business in the future. 

Given the high value that farmers place upon market advisory services, it is somewhat surprising that only two academic studies investigate the pricing performance of advisory services. [1]   The dearth of studies seems even more anomalous in light of the large number of studies on grain marketing strategies. [2]  The lack of studies on market advisory services is most likely due to the difficulty in obtaining data on the stream of recommendations provided by services.

Gehrt and Good (1993) analyze the performance of five advisory services for corn and soybeans over 1985-1989. Martines-Filho (1996) examines the pre-harvest corn and soybean marketing recommendations of six market advisory services over 1991-1994.  Most recently, Irwin, Good, Martines-Filho and Jackson (2000) investigate the performance of 25 advisory services in marketing corn and soybeans over 1995-1998. The evidence in these three studies suggests a modest ability to "beat the market."

This discussion points to a need for further research on the performance of market advisory services.  Previous studies only examine advisory service performance in marketing corn and soybeans.  It is not known whether the results generalize to other commodities with different production and consumption characteristics.  Wheat represents an interesting additional market to examine advisory service performance. It differs significantly from corn and soybeans with respect to the timing and location of production, yield growth trends, seasonality and consumption uses.  Hence, we would expect different marketing patterns, and potentially, different results than have been reported for corn and soybeans.

The purpose of this report is to investigate the performance of agricultural market advisory services in marketing wheat.  Following Irwin, Good, Martines-Filho and Jackson (2000) two key performance questions will be addressed: 1) Do market advisory services, on average, outperform an appropriate wheat market benchmark? and 2) Do market advisory services exhibit persistence in their wheat performance from year-to-year?  The data for the study is provided by the Agricultural Market Advisory Service (AgMAS) Project, which has been collecting wheat track records for at least 20 advisory services since September 1994.  At the present time, track records are available for the 1995, 1996, 1997 and 1998 crop years. Since the AgMAS Project subscribes to all of the services and collects "real-time" recommendations, the data are not subject to survivorship bias.  While the sample of advisory services is non-random, it is constructed to be generally representative of the majority of advisory services offered to farmers.  The availability of only four crop years is a limitation of the analysis, but the time period considered does include years of rapidly increasing and decreasing wheat prices.

The procedure used to compute net wheat prices for each advisory service is outlined in the earlier AgMAS report by Jirik, Good, Irwin, Jackson and Martines-Filho (2000).  In particular, after the stream of recommendations is collected for a given commodity in a particular crop year, the net price that would have been received by a wheat farmer that precisely follows the set of marketing recommendations is computed.  This net price is the weighted average of the cash sale price plus or minus gains/losses associated with futures and options transactions.  Brokerage costs are accounted for, as are storage costs and marketing loan payments.

The tests used to determine average performance of market advisory services and predictability of performance through time have been widely applied in the financial literature (e.g., Elton, Gruber, and Rentzler, 1987; Lakonishok, Shleifer and Vishny, 1992; Irwin, Zulauf, and Ward, 1994; Jaffe and Mahoney, 1999; Metrick, 1999; Carpenter and Lynch, 1999).  Two tests of performance relative to a benchmark are used: i) the proportion of services exceeding the benchmark price and ii) the average percentage difference between the net price of services and the benchmark price.  Three tests of predictability are used: i) the correlation of advisory service pricing performance measures from year-to-year, ii) the predictability of “winner” and “loser” categories from year-to-year and iii) the differences between pricing performance measures for “top” and “bottom” performing advisory services.

The market advisory services included in this evaluation do not comprise the population of market advisory services available to farmers.  The included services also are not a random sample of the population of market advisory services.  Neither approach is feasible because no public agency or trade group assembles a list of advisory services that could be considered the "population."  Furthermore, there is not a generally agreed upon definition of an agricultural market advisory service.  To assemble a sample of services for the AgMAS Project, criteria were developed to define an agricultural market advisory service and a list of services assembled.

The first criterion used to identify services is that a service has to provide marketing advice to farmers. Some of the services tracked by the AgMAS Project do provide speculative trading advice, but that advice must be clearly differentiated from marketing advice to farmers for the service to be included.  The terms "speculative" trading of futures and options versus the use of futures and options for "hedging" purposes are used for identification purposes only.  A discussion of what types of futures and options trading activities constitute hedging, as opposed to speculating, is not considered.

The second criterion is that specific advice must be given for making cash sales of the commodity, in addition to any futures or options hedging activities.  In fact, some marketing programs evaluated by the AgMAS Project do not make any futures and options recommendations.  However, marketing programs that make futures and options hedging recommendations, but fail to clearly state when cash sales should be made, or the amount to be sold, are not considered.

The original sample of market advisory services that met the two criteria were drawn from the list of  "Premium Services" available from the two major agricultural satellite networks, Data Transmission Network (DTN) and FarmDayta in the summer of 1994. [3] ^{, [4]}   While the list of advisory services available from these networks was by no means exhaustive, it did have the considerable merit of meeting a market test.  Presumably, the services offered by the networks were those most in demand by farm subscribers to the networks.  In addition, the list of available services was cross-checked with other farm publications to confirm that widely-followed advisory firms were included in the sample.  It seems reasonable to argue that the resulting sample of services was (and remains) generally representative of the majority of advisory services available to farmers.

The sample for 1995 includes 24 market advisory services for wheat.  For a variety of reasons, deletions and additions to the 1995 sample occur over time. [5]   In 1996, the total number of advisory services is 23, while in 1997 the total is 20.  In 1998, the total number of advisory services is 21. A directory of the advisory services included in the study can be found at the AgMAS Project website (http://web.aces.uiuc.edu/farmdoc/agmas/).

As mentioned earlier, sample selection biases may plague advisory service databases.  The first form is survival bias, which occurs if only advisory services that remain in business at the end of a given period are included in the sample.  Survival bias significantly biases measures of performance upwards since "survivors" typically have higher performance than "non-survivors" (Brown, Goetzmann, Ibbotson, and Ross, 1992).  This form of bias should not be present in the AgMAS database of advisory services because all services ever tracked are included in the sample.  The second and more subtle form of bias is hindsight bias, which occurs if data from prior periods are "back-filled" at the point in time when an advisory service is added to the database.  Statistically, this has the same effect as survivorship bias because data from surviving advisory services are back-filled.  This form of bias should not be present in the AgMAS database because recommendations are not back-filled when an advisory service is added.  Instead, recommendations are collected only for the crop year after a decision has been made to add an advisory service to the database.

The actual daily process of collecting recommendations for the sample of advisory services begins with the purchase of subscriptions to each of the services.  Staff members of the AgMAS Project read the information provided by each advisory service on a daily basis.  The information is received electronically, via DTN, websites or e-mail.  For the services that provide two daily updates, typically in the morning and at noon, information is read in the morning and afternoon.  In this way, the actions of a farmer-subscriber are simulated in “real-time.”

The recommendations of each advisory service are recorded separately.  Some advisory services offer two or more distinct marketing programs.  This typically takes the form of one set of advice for marketers who are willing to use futures and options (although futures and options are not always used), and a separate set of advice for farmers who only wish to make cash sales. [6]   In this situation, both strategies are recorded and treated as distinct strategies to be evaluated. [7]

Several procedures are used to check the recorded recommendations for accuracy and completeness.  Whenever possible, recorded recommendations are cross-checked against later status reports provided by the relevant advisory service.  Also, at the completion of the crop year, it is confirmed whether cash sales total exactly 100%, all futures positions are offset, and all options positions are offset or expire.

The final set of recommendations attributed to each advisory service represents the best efforts of the AgMAS Project staff to accurately and fairly interpret the information made available by each advisory service.  In cases where a recommendation is considered vague or unclear, some judgment is exercised as to whether or not to include that particular recommendation or how to implement the recommendation.  Given that some recommendations are subject to interpretation, the possibility is acknowledged that the AgMAS track record of recommendations for a given service may differ from that stated by the advisory service, or from that recorded by another subscriber.

At the end of a crop year, all of the (filled) recommendations are aligned in chronological order.  The advice for a given crop year is considered to be complete for each advisory service when cumulative cash sales of the commodity reach 100%, all open futures positions covering the crop are offset, all open option positions covering the crop are either offset or expired, and the advisory service discontinues giving advice for that crop year.  The returns to each recommendation are then calculated in order to arrive at a weighted-average net price that would be received by a farmer who precisely follows the marketing advice (as recorded by the AgMAS Project).

In order to simulate a consistent and comparable set of results across the different advisory services, certain explicit assumptions are made.  These assumptions are intended to accurately depict marketing conditions for a representative, southwest Illinois farm.  An overview of the simulation assumptions is presented below.  Complete details of the simulation assumptions can be found in Jirik, Good, Irwin, Jackson and Martines-Filho (2000).

Wheat Class and Geographic Location

An issue of first importance is the appropriate class of wheat and location of production to use in the simulation.  In the US, six classes of wheat are grown and there are five wheat futures contracts traded on three different exchanges.  The simulation is designed to reflect conditions facing a representative soft red winter wheat farmer in southwest Illinois.  Whenever possible, data are collected for the West Southwest Crop Reporting District in Illinois as defined by the National Agricultural Statistics Service (NASS) of the US Department of Agriculture (USDA).  Thirteen counties (Cass, Pike, Scott, Morgan, Sangamon, Christian, Calhoun, Greene, Macoupin, Montgomery, Jersey, Madison, and Bond) make up this District.  For ease of reading, this area will be referred to in the remainder of this report as southwest Illinois, unless it is necessary to reference the actual crop or price reporting district.

There are two principal reasons that soft red winter wheat in southwest Illinois is used as the basis for the simulation.  The first reason is that soft red winter wheat recommendations are the most common class of wheat recommendations made by advisory programs.  The programs included in this study either specifically make recommendations for this class of wheat or the recommendations most closely align with this class of wheat.  There are three programs included in the former category; that is, they specifically identify recommendations by class of wheat.  The remaining programs do not specifically identify the class of wheat, but several pieces of evidence point in the direction of soft red winter wheat as the target class: i) most futures hedging advice refers to the Chicago Board of Trade (CBOT) wheat contract, ii) the programs generally make harvest recommendations for June and early July, the harvest period for winter wheat and iii) the programs that give basis advice generally recommend basis levels in soft red winter wheat production areas.

The second reason that soft red winter wheat in southwest Illinois is used in the simulation is data availability.  An exhaustive search was conducted for a public series of daily cash and forward contract prices for interior elevators in major hard red winter, hard red spring, and soft red winter wheat production areas of the US.  Several public sources of cash spot prices were located for each of the different classes.  However, the only public source of forward contract prices is Illinois Ag Market News, and this agency only reports bids for soft red winter wheat.  This is an important limiting factor, as many advisory programs make substantial use of pre-harvest forward contracts.  It may be possible to obtain forward contract prices from private sources in other regions, but this is costly and may result in forward price data of uncertain accuracy.

An important question is the degree to which performance results based on soft red winter wheat production in southwest Illinois can be generalized to other classes and locations of wheat production in the US.  To provide some perspective on this issue, yields and prices for two other areas of wheat production in the US are compared to southwest Illinois.  Figure 1 presents the relationship between deviations from trend for the West Southwest Illinois Crop Reporting District (soft red winter), Southwest Kansas Crop Reporting District (hard red winter), and Northeast South Dakota Crop Reporting District (hard red spring) over 1972-1998.  The correlation of the deviations from trend shows a weak positive relationship between the yield deviations for southwest Illinois and the other two regions.  There is only a slight tendency for southwest Illinois wheat yields to be above trend at the same time that Kansas or South Dakota yields are above trend, and vice versa.

The history of daily cash prices for wheat in Illinois, Kansas and South Dakota is presented in Figures 2 for the period June 1995 through May 1999.  Soft red winter wheat prices are presented for the West Southwest Illinois Price Reporting District, hard red winter wheat prices are shown for the Western Kansas Price Reporting District and hard red spring wheat prices are shown for the East River South Dakota Price Reporting District.  These price districts most closely match the crop districts used above to compare yields.  Price changes are analyzed because the time series properties of commodity prices strongly suggest that unbiased estimates of price correlations should be based on price changes rather than price levels (e.g., Brown, 1985).  The correlations are highly positive between Illinois and the other two areas.  Not surprisingly, a high correlation is observed between Illinois and Kansas, as these two areas produce winter wheat.  It is interesting to note that the correlation estimate of 0.83 is quite close to similar estimates reported in studies of optimal wheat cross-hedging (e.g., Brorsen, Buck and Koontz, 1998). The correlation is also high between Illinois and South Dakota, even though Illinois produces winter wheat and South Dakota produces spring wheat. Finally, while these correlations are based on cash prices, it is expected that similar correlations exist across futures prices for the different wheat classes, due to inter-market spread trading and arbitrage.

The previous results present a mixed picture regarding the degree to which performance results based on soft red winter wheat production in southwest Illinois can be generalized to other classes and locations of wheat production in the US.  On one hand, there appears to be little relationship in wheat yields across classes and locations.  On the other hand, there is a highly positive relationship between wheat prices across classes and locations. It is an empirical question whether the lack of a relationship between yields or the positive relationship between prices has the dominant impact on performance evaluations.  One plausible outcome is that the low correlation in yields is more than offset by the high correlation in prices, and hence, it is reasonable to generalize performance evaluations for soft red winter wheat production in southwest Illinois to other wheat classes and locations.  An equally plausible outcome is that the low correlation in yields more than offsets the high correlation in prices, and hence, it is unreasonable to generalize performance evaluations for soft red winter wheat production in southwest Illinois to other wheat classes and locations.  Until empirical evidence is available on this question, caution is suggested before attempting to generalize the performance results to other wheat classes and locations.

Marketing Window

In general, a two-year marketing window, spanning June 1^st of the year prior to harvest through May 31^st of the year following harvest, is used in the analysis.  The beginning date is selected because it reflects a “realistic” time when new crop sales begin.  The ending date is selected to be consistent with the ending date for wheat marketing years as defined by the USDA.  There are some exceptions to the marketing window definition.  The most frequent exceptions are when programs have relatively small amounts (20 percent or less) of cash wheat  unsold at the end of a window.  In such cases, the actual sales recommendations on the indicated dates are recorded. Finally, note that throughout the remainder of this report, the term "crop year" is used to represent the two-year marketing window.

There are three exceptions to the marketing window that should be highlighted.  One service held 1997 wheat far beyond the end of the 1997 marketing window and two services did the same for 1998 wheat.  More specifically, as of May 31, 2000, the Allendale (futures only) service had not recommended any cash sales for either the 1997 or 1998 wheat crops.  However, both crops were fully hedged using wheat futures.  As of May 31, 2000, Ag Profit by Hjort Associates had not sold any of the 1998 wheat crop.  In order to complete the analysis for these two services, the futures positions and all remaining cash quantities are marked-to-the-market as of May 31, 2000. 

Prices

The cash price assigned to each cash sale recommendation is the West Southwest Illinois Price Reporting District closing, or overnight, bid.  Similarly, the forward contract price assigned to all pre-harvest forward sales is the forward bid for the West Southwest Price Reporting District.  The cash and forward contract data are collected and reported by the Illinois Department of Ag Market News.    Cash and forward contract prices in this area best reflect prices for the assumed geographic location of the representative southwest Illinois farmer (West Southwest Illinois Crop Reporting District). Futures prices and options premia are Chicago Board of Trade quotes.

Quantity Sold

Since most of the advisory program recommendations are given in terms of the proportion of total production (e.g., “sell ten percent of 1998 crop today”), some assumption must be made about the amount of production to be marketed.  For the purposes of this study, if the per-acre yield is assumed to be 50 bushels, then a recommendation to sell ten percent of the wheat crop translates into selling five bushels.  When all of the advice for the marketing period has been carried out, the final per-bushel selling price is the average price for each transaction weighted by the amount marketed in each transaction.

When making hedging or forward contracting decisions prior to harvest, the actual yield is unknown.  Hence, an assumption regarding the amount of expected production per acre is necessary to accurately reflect the returns to marketing advice.  When yield is near or above trend, there is normally not a problem in meeting forward pricing obligations.  Hence, in a “normal” crop year, expected yield is assumed to equal trend yield for the entire pre-harvest period.  The adjustment from expected to actual yield in this case is assumed to occur on the first day of wheat harvest.  The expected yield for the West Southwest Illinois Crop Reporting District is computed from a linear regression trend model of actual yields from 1972 through the year previous to harvest.  For example, the trend yield forecast for 1998 is based on a regression using 1972 to 1997 yield data. 

When actual yield is substantially below trend, and forward pricing obligations are based on trend yields, a farmer may have difficulty meeting such obligations.  This raises the issue of updating yield expectations in “short” crop years to minimize the chance of defaulting on forward pricing obligations. A relatively simple procedure is used to update yield expectations in short crop years.  First, trend yield is used as the expected yield until the May USDA Crop Production Report is released, typically around May 10^th.  Second, if the USDA wheat yield estimate for southwest Illinois is 20 percent (or more) lower than trend yield, a “reasonable” farmer is assumed to change yield expectations to the lower USDA estimate.  Third, as with normal crop years, the adjustment to actual yield is assumed to occur on the first day of harvest.

Brokerage Costs

Brokerage costs are incurred when farmers open or close positions in futures and options markets.  For the purposes of this study, it is assumed that brokerage costs are $50 per contract for round-turn futures transactions, and $30 per contract to enter or exit an options position.  Further, it is assumed that CBOT wheat futures or options contracts are used, and the contract size for each commodity is 5,000 bushels.  Therefore, per-bushel brokerage costs are one cent per bushel for a round-turn futures transaction and 0.6 cents per bushel for each options transaction.

Carrying Costs

An important element in assessing returns to an advisory program is the economic cost associated with storing grain instead of selling grain immediately at harvest.  The cost of storing grain after harvest (carrying costs) consists of two components: physical storage charges and the opportunity cost incurred by foregoing sales when the crop is harvested.  Physical storage charges can apply to off-farm (commercial) storage, on-farm storage, or some combination of the two.  Opportunity cost is the same regardless of the type of physical storage.

For the purposes of this study, it is assumed that all storage occurs off-farm at commercial sites.  Storage charges are assigned beginning with the first day after the end of a harvest window.  Physical storage charges have a fixed component (in-charge) of four cents per bushel that is assigned the day storage begins.  The variable component is 2.5 cents per bushel per month, with this charge pro-rated to the day when the cash sale is made.  The storage costs represent the typical storage charges for the 1995-1998 wheat crops quoted in a telephone survey of southwest Illinois elevators.

The interest charge for storing grain is the interest rate compounded daily from the end of wheat harvest to the date of sale.  The interest rate used is the average rate for all commercial agricultural loans for the third quarter of the harvest year as reported in the Agricultural Finance Databook published by the Board of Governors of the Federal Reserve Board.  This interest rate has been around nine percent per year for the four years of this study.

The price of wheat is below the loan rate during significant periods of time in the 1998-1999 marketing year, so that use of the marketing loan program is an important part of marketing strategies during this period.  Most of the advisory programs tracked by the AgMAS Project for the 1998 crop make specific recommendations regarding the timing and method of implementing the loan program for the entire wheat crop.  These recommendations are implemented as given wherever feasible.  Several decision rules have to be developed even in this case, in particular, for pre-harvest forward contracts.  For a few programs, loan recommendations are incomplete or not made at all.  For these cases, it is necessary to develop a more complete set of decision rules for implementing the loan program in the marketing of wheat.  All loan-related decision rules are based on the assumption of a “prudent” or “rational” farmer, within the context of the intent of the loan program.  More specifically, it is assumed that a farmer will take advantage of the price protection offered by the loan program, even in the absence of specific advice from an advisory program. Further information on the decision rules used to implement marketing loan recommendations can be found in Jirik, Good, Irwin, Jackson and Martines-Filho (2000).

Market Benchmark

Simply comparing the net price received across advisory services will not answer the question of whether advisory services as a group enhance the income of farm subscribers.  Instead, a comparison to a benchmark price (or prices) is needed to evaluate the performance of advisory services relative to pricing opportunities offered by the market.  In the stock market, mutual funds are evaluated with respect to market benchmark performance criteria (e.g., Bodie, Kane, and Marcus, 1989).  These benchmarks typically are indexes of stock market returns over the period of evaluation, e.g., the Dow Jones Industrial Average and Standard and Poor’s 500.

The selection of a benchmark for advisory service performance evaluations is examined in a study by Good, Irwin and Jackson (1998).  They argue that the most appropriate market benchmark is the average price over the entire, relevant marketing horizon.  Applied to wheat, the marketing window for a given crop spans two calendar years, beginning on the first business day of June in the year prior to harvest, and extends through the last business day of May in the year after harvest.  Hence, the market benchmark is calculated as the average of the daily southwest Illinois cash wheat bids available for the two-year marketing window.  Pre-harvest cash prices represent cash-forward bids for harvest delivery in southwest Illinois, while daily spot prices for southwest Illinois are used for the post-harvest period.

Three adjustments are made to the daily cash prices to make the average cash price benchmark consistent with the calculated net advisory prices for each marketing program.  The first is to take a weighted average price, to account for changing yield expectations, instead of taking the simple average of the daily prices.  The daily weighting factors for pre-harvest prices in normal years are based on the calculated trend yield, while the weighting of the post-harvest prices is based on the actual reported yield for southwest Illinois.  In short-crop years, yield expectations are updated with the release of the USDA May Crop Production Report, using the same procedure applied to advisory program recommendations.  The second adjustment is to compute post-harvest cash prices on a harvest equivalent basis, which is done by subtracting carrying charges (storage and interest) from post-harvest spot cash prices.  The daily carrying charges are calculated in the same manner as those for net advisory prices.

A third adjustment to the average cash price benchmark is made only for 1998.  This adjustment is based on the logic that a  “prudent” or “rational” farmer will take advantage of the price protection offered by the marketing loan program when following the benchmark average price strategy.  Based on this argument, the average cash price benchmark is adjusted by the addition of marketing loan benefits.  Bushels marketed in the pre-harvest period according to the benchmark strategy (approximately 53 percent) are treated as forward contracts with the benefits assigned at harvest.  Bushels marketed each day in the post-harvest period (approximately 47 percent) are awarded marketing loan benefits in existence for that particular day. 

In order to test the sensitivity of performance results to the choice of market benchmark,  two alternative versions of the previous average cash price benchmark also are considered in the analysis.  The first alternative benchmark averages prices for the 20-month period starting in October of the year previous to harvest and ending in May of the year after harvest. The only difference between this alternative and the 24-month benchmark is the exclusion of the pre-harvest period previous to October.  Hence, this alternative benchmark places more weight on post-harvest prices than pre-harvest prices.  The second alternative benchmark averages prices only for a 16-month crop year, which excludes prices previous to February.

Net price received for the sample of market advisory services for the 1995, 1996, 1997 and 1998 crop years is reported in Tables 1. [8]   Note that some of the market advisory services included in the table are not evaluated for all four years.  The four-year averages and standard deviations are calculated only for the 18 services that are evaluated for all four years.

As shown in Table 1, the annual average net advisory price for wheat ranges from $2.36 per bushel in 1998 to $3.81 per bushel in 1996.  The four-year average for the 18 services is $3.15 per bushel.  The range of four-year average net advisory prices is large, with a low of $2.76 per bushel and a high of $3.48 per bushel.  Not surprisingly, the range within the individual years is even more substantial.  The most dramatic example is 1997, where the minimum is $1.34 per bushel and the maximum is $3.90 per bushel.  Even in years with less market price volatility, such as 1998, the range in performance typically is around two dollars per bushel. 

The three alternative market benchmark prices for wheat are shown at the bottom of Table 1.  Four-year averages of the market benchmarks differ by one cent per bushel or less.  However, this masks large differences within some of the years, particularly 1998.  These data suggest advisory service performance results for wheat may be sensitive to the selected benchmark.  

Wheat revenue results for the advisory services are presented in Table 2.  For a given year, revenue is computed as the net advisory price times the actual yield. [9] Revenue results are reported to provide perspective on the economic magnitude of differences in pricing performance.  In addition, annual yield variation may cause average revenue and average price results to differ across services.  In particular, the impact of the relatively good and poor pricing performance may be reduced or exaggerated depending on whether it is associated with large or small wheat crops.  The four-year average advisory revenue for all 18 services is $151 per acre, and ranges from a low of $134 per acre to a high of $173 per acre.  The range of revenue for individual years can be quite large, twice exceeding $100 per acre (1997 and 1998).

Two statistical tests are used to test the null hypothesis that average market advisory service pricing performance does not differ from that of the market benchmark.  The first test is based on the proportion of services exceeding the benchmark price.  This test is considered because it is not influenced by extremely high or low advisory prices.  The second test is based on the average percentage difference between the net price of services and the benchmark price.  This test is useful because it takes into account the average magnitude of differences from the benchmark.

Independence of Observations

Before considering the statistical tests and results, an important issue needs to be explored that may have a substantial impact on the results.  The issue is whether the sample observations on net advisory price are independent, both within and across years.  The most likely form of dependence is positive correlation, which, if ignored, would cause sample standard deviation estimates across advisory services to be understated.  This in turn would cause the statistical significance of hypothesis test results to be overstated. 

There are two potential ways that independence could be violated in the sample of market advisory service prices.  The first potential source of dependence is correlation of net advisory prices through time for a given service.  This form of correlation may exist due to persistence in the performance of advisory services through time (winners continue to win, losers continue to lose).  It may also exist due to the overlapping nature of the crop years; each crop year is two calendar years long, and each set of contiguous crop years overlaps by one year.  If this correlation through time exists, it would be inappropriate to pool samples of net advisory prices across crop years for the same reason as discussed above.  As will be shown in a following section, this form of correlation generally is minimal, and therefore, it is reasonable to pool net advisory prices across crop years. 

A second potential source of dependence perhaps is less obvious.  It is possible that net advisory prices for a given commodity and crop year are correlated because of the existence of similar programs offered by the same market advisory service.  For example, Agri-Visor offers four marketing programs, which may not differ substantially in outcomes due to similar methods of analysis and similar underlying strategies.  The potential impact of this form of correlation is examined by creating one net advisory price for each of the market advisory firms that offer multiple programs. [10]  A single price is computed by averaging net advisory prices across programs for a given year and commodity.  Pricing performance results are qualitatively similar to those using the full set of disaggregated advisory prices, suggesting that net prices of advisory programs for the same firm are uncorrelated or no more correlated than net prices from different firms.  Hence, use of net advisory prices by program in tests of market performance does not appear to be a substantive problem.

A formal test of the null hypothesis that the proportion of advisory services "beating" the market benchmark is insignificant requires the specification of an appropriate test statistic. First, define the sample estimate of the proportion for a given year as,

where k is the number of advisory services that have net prices exceeding the market benchmark price and n is the total number of advisory services in the sample. Anderson, Sweeney and Williams  (1996) show that the sample estimator of the proportion, , is distributed binomially with an expected value of  p and a standard error of , where p is the true value of the proportion in the population. They also note that the sampling distribution of  is approximately normal so long as  and .  Since both conditions are met for all of the samples considered here, the normality approximation is invoked.  The form of the test statistic based on the above assumptions is,

where p₀ is the assumed value of p under the null hypothesis.  The remaining issue is the expected proportion (p₀) under the null hypothesis.  The efficient market hypothesis (Fama, 1970) implies that the expected probability of “beating the market” is the same as the result of flipping a coin and showing heads, or 0.5.  Setting , the test statistic is,

A formal test of the null hypothesis that the average percentage difference between the net price of services and the benchmark price is zero also requires the specification of an appropriate test statistic.  First, define the percentage difference for the i^th advisory service for a given crop year as,

where NAP_i is the net advisory price for the i^th advisory service and BP is the market benchmark price for the same crop year. The sampling distribution of  is well-known and does not need to be described in detail here.  The test statistic for a null hypothesis of zero average percentage difference is,

where  is the estimated standard deviation of the percentage differences across the n advisory services in the sample.  The t-statistic follows a t-distribution with n-1 degrees of freedom. 

It is possible to think of r_i as the “return” to following the recommendations of a particular market advisory service.  This raises the question of whether the calculated “returns” are risk-adjusted.  One method of adjusting returns for risk that has been used in a number of studies stock investment strategies (e.g., Friend, Blume and Crocket, 1970; Ritter, 1991) is to match the average risk of the investments to the risk of the benchmark.  Hence, if the average risk of advisory services is equal to risk of the market benchmark, then market advisory returns can be considered risk-adjusted returns.  Evidence on the appropriateness of this “risk-matching” assumption for advisory services can be found in Tables 1 and 2, where the standard deviations for the advisory services and market benchmarks can be found in the last column of each table.  As shown in Table 1, the average standard deviation for net advisory prices in wheat is $0.86 per bushel, substantially greater than the standard deviations for the three benchmarks.  Turning to Table 2, the average standard deviation for advisory service revenue is $33 per acre, again larger than the standard deviations for the three benchmarks, but closer than in the case of net prices. Overall, the comparisons suggest the risk of the market benchmarks does not match the average risk of the advisory services, and hence, it is likely inappropriate to consider computed “returns” as being risk-adjusted.

It is important to emphasize that the tests discussed in this section address the pricing performance of market advisory services as a group.  In other words, average pricing performance across all services is considered.  This is a different issue than the pricing performance of a particular advisory service.  It is possible that advisory services as a group fail to beat the market, yet at the same time there exist a small number of services that are exceptions to this outcome. In the stock market, this argument is often made with respect to the performance of the Fidelity Magellan Fund.  Testing whether an “exceptional” advisory service beats the market requires more data than is available for this study and different statistical methods (Marcus, 1990). 

Table 3 reports results of the proportional test of wheat pricing performance for each year and all four years pooled. [11]   Statistical significance is based on a null hypothesis proportion of 0.5, the same as the proportion of heads observed in the flips of a fair coin.  Individual year results are somewhat sensitive to the benchmark considered.  For example, the proportion of programs above the 24-month benchmark price in 1998 is 0.05 and statistically smaller than 0.5, while the proportion of programs above the 16-month benchmark is 0.29 and insignificantly different from 0.5.  However, the proportion pooled across the four years does not vary substantially across the benchmarks, ranging from 0.32 to 0.34.  Pooled four-year proportions based on all three benchmarks are significantly different from 0.5 at the one-percent level. Individual year results generally show proportions significantly less than 0.5 in three of the four years:  1996, 1997 and 1998.  The smallest proportions are found in 1997 and 1998. Finally, there is only one case where a proportion is significantly greater than 0.5 (1995, 24-month benchmark).

Results for the average return test of pricing performance are reported in Table 4.  Pooled four-year and individual year test results are qualitatively similar to the proportional test results. Point estimates of the four-year average return range from –9.75 to –10.48 percent.  All of the four-year average returns are significantly different from zero at the one-percent level.  In some individual years the magnitude of underperformance is surprisingly large.  For example, average return estimates for 1997 range from –18.89 to –23.01 percent.

In statistical terms, the pricing performance test results presented in this section are clear.  Not only do market advisory programs in wheat consistently fail to “beat the market,” their performance is significantly worse than the market.  The level of under-performance is striking and consistent.  Point estimates of proportions for individual years are less than 0.5 in ten of twelve test cases.  Likewise, point estimates of average return for individual years are negative in ten of twelve test cases.  Finally, the average return of the services over the four crop years is about –10 percent, regardless of which of the three benchmarks is considered.

Given the statistical results summarized above, a relevant question to ask is whether the pricing under-performance of advisory programs also is economically significant.  While "economic significance" is a vague concept, it is important nonetheless.  A useful perspective on this question is gained by examining wheat revenue per acre (see Appendix Table A2).  The best point estimate of advisory revenue return probably is the simple average across the three benchmarks.  This “grand average” revenue return across all four crop years and three benchmarks is –10.20 percent, which translates into advisory revenue averaging $14 per acre below benchmark revenue. [12]   By any reasonable standard, this is an economically non-trivial level of under-performance. [13]  

The pricing performance results for wheat stand in sharp contrast to those reported for corn and soybeans.   Irwin, Good, Jackson and Martines-Filho (2000) analyze the pricing performance of corn and soybean market advisory programs tracked by the AgMAS Project over 1995-1998.  They find that market advisory services in corn and soybeans have a “small” ability to “beat the market,” with combined corn and soybean revenue for the advisory programs averaging about $4 per acre more than benchmark revenue.  Two explanations seem plausible for the divergence in results across corn and soybeans and wheat.  First, the divergence may simply be an artifact of a relatively small sample of years, where wheat advisory performance is by chance unusually poor and/or corn and soybean advisory performance is unusually good.  Second, advisory programs may be more skillfull in analyzing and forecasting corn and soybean prices than wheat prices.

The results of the analysis also have implications for the ongoing debate about market efficiency and risk management strategies in agriculture.  One view is that grain markets (cash, futures and options) are not efficient and, therefore, provide opportunities for farmers to systematically earn additional profits through marketing (e.g., Wisner, Blue and Baldwin, 1998).  The other view is that grain markets are at least efficient with respect to the type of strategies available to farmers (e.g., Zulauf and Irwin, 1998).  Since the returns of wheat advisory programs over 1995-1998 are significantly less than transactions cost, including the cost of the programs, the results are consistent with market efficiency in the sense of Grossman and Stiglitz (1980). Furthermore, the performance results suggest market advisory services have less access to information than that available to other wheat market participants and/or inferior analytical skills.

Finally, it is interesting to compare the pricing performance results for market advisory services to that of other investment professionals.  Malkiel (1999) reports that only 33 percent of active mutual fund managers beat the returns to the S&P 500 stock index over 1974-1998.  Clements (1999) notes that only nine percent of active managers beat the S&P 500 in the decade ending in 1998. The performance of agricultural market advisory services in wheat is roughly comparable.

Predictability of Advisory Service Performance

Even if advisory programs as a group generate negative returns, there is a wide range in performance for any given year.  For example, wheat net advisory prices for 1997 vary from $1.34 per bushel to $3.90 per bushel (Table 1).  While this example is the most dramatic, the variation across advisors in other years also is substantial.  This raises the important question of the predictability of advisory service performance from year-to-year.  In other words, is past performance indicative of future results?  Three tests of predictability are used: i) the correlation of advisory service prices, ranks and percentage differences from the benchmark across overlapping and non-overlapping pairs of adjacent crop years, ii) predictability of “winner” and “loser” categories across overlapping and non-overlapping pairs of adjacent crop years and iii) differences between prices, ranks and percentage differences from the benchmark for “top” and “bottom” performing advisory services across overlapping and non-overlapping pairs of adjacent crop years. The testing procedures have been widely applied in studies of financial investment performance (e.g., Elton, Gruber, and Rentzler, 1987; Irwin, Zulauf and Ward, 1994; Lakonishok, Shleifer and Vishny, 1992; Malkiel, 1995). 

The distinction between overlapping and non-overlapping crop years is due to the fact that each marketing window is two calendar years in length, and hence, two adjacent marketing windows overlap by one calendar year.  This overlap may influence predictability results, in that persistence between overlapping years may be due to “true” persistence in performance or the overlapping nature of the periods of comparison.  Persistence for non-overlapping years presumably reflects only “true” persistence in pricing performance. 

Predictability Tests

The first test of predictability is based on the correlation between performance measures of individual market advisory services across overlapping and non-overlapping pairs of crop years.  Brorsen and Townsend (1998) show that this type of test is reasonably powerful in detecting performance persistence in managed futures funds if it exists. For a given commodity, the first step in this testing procedure is to form the sample of all advisory services that are active in both adjacent years (overlapping or non-overlapping).  The second step is to rank each advisory service in the first year of the pair (e.g., t = 1997) based on net price received. Then the services are sorted in descending order.  For example, the service with the highest net advisory price is ranked number one, and the service with the lowest net advisory price is assigned a rank equal to the total number of services for that commodity in the given year.  The third step is to sort and rank the sample of services in the second year of the pair (e.g., t + 1  = 1998). The fourth step is to estimate the correlation coefficient between performance measures for the two adjacent crop years t and t+1 as follows,

where  is the sample average of net advisory prices for year t,  is the sample average of net advisory ranks for year t and  is the sample average of net advisory percentage differences from the market benchmark for year t. Finally, using Bartlett’s approximation for the standard error ( ) of the correlation coefficient, the following test statistic is used to test the null hypothesis of no predictability across the adjacent pair of years,

where j = NAP_t,t+1, RK_t,t+1 and r_t,t+1. The sampling distribution of the test statistic Z_j approximately follows a standard, normal distribution.

The second test of predictability is based on placing advisory services into “winner” and “loser” categories across overlapping and non-overlapping pairs of adjacent crop years. The resulting 2 x 2 contingency table of winner and loser counts allows the use of non-parametric statistical testing procedures.  Carpenter and Lynch (1999) indicate this test is well-specified and powerful in detecting persistence in mutual fund returns. The first step in this testing procedure is to form the sample of all advisory services that are active in both adjacent years (overlapping or non-overlapping).  The second step is to rank each advisory service in the first year of the pair (e.g., t = 1997) based on net price received. Then the services are sorted in descending order. The third step is to form two groups of services in the first year of the pair: winners are those services in the top half of the rankings and losers are services in the bottom half. The third step is to rank each advisory service in the second year of the pair (e.g., t +1 = 1998) based on net price received and once again form winner and loser groups of services. The fourth step is to compute the following counts for the advisory services in the pair of years: WW = winner t-winner t+1, WL = winner t-loser t+1, LW = loser t-winner t+1, LL = loser t-loser t+1. The fifth step is to compute the following odds ratio,

which estimates the ratio of the odds of a winning service in t being a winning service in t+1 to the odds of a losing service in t being a winning service in t+1. The null hypothesis of no predictability is true when the odds ratio equals one.  Christenson (1997) notes that it is more convenient mathematically to test the equivalent null hypothesis that the natural logarithm of the odds ratio equals zero.  In this case, the test statistic is,

where

The sampling distribution of the test statistic Z_t,t+1 asymptotically follows a standard, normal distribution

The third test of predictability is based on the differences between prices, ranks and percentage differences from the benchmark for “top” and “bottom” performing advisory services across overlapping and non-overlapping pairs of adjacent crop years. This test is based on the observation that predictability in advisory service performance may not exist across all advisory services, but it is possible that sub-groups of advisory services may exhibit predictability.  In particular, predictability may only be found at the extremes of performance.  That is, only top-performing services in one year may tend to perform well in the next year, or only poor-performing services may perform poorly in the next year. Carpenter and Lynch (1999) indicate this type of test also is well-specified and powerful in detecting persistence in mutual fund returns.

The first step in this testing procedure is to sort services by pricing performance in the first year of the pair and group services by quantiles (thirds and fourths).  The second step is to compute the average pricing performance for the quantiles formed in the first year of the pair in the second year of the pair. For example, the pricing performance of the top fourth quantile formed in 1995 is computed for 1996. The third step is to compute the following differences in pricing performance for the top- and bottom-performing quantiles,

where  and  are the average net advisory prices for the top and bottom quantiles (thirds or fourths) formed in year t and tracked in year t+1, respectively,   and  are the average net advisory ranks for the top and bottom quantiles (thirds or fourths) formed in year t and tracked in year t+1, respectively,  and  and  are the average net advisory returns for the top and bottom quantiles (thirds or fourths) formed in year t and tracked in year t+1, respectively. The fourth step is to estimate the mean and standard deviation of the above differences across all possible pairs of years.  Finally, the following test statistic can be used to test the null hypothesis of no predictability,

where  is the mean estimate across the possible pairs of years,  is the standard deviation estimate across the possible pairs of years and .  In the case of overlapping crop years,  since there are three pairs of years (1995 vs.1996, 1996 vs.1997, 1997 vs.1998).  In the case of non-overlapping crop years,  since there are two pairs of years (1995 vs.1997, 1996 vs.1998). 

Predictability Test Results

Results of the test of predictability based on the correlation between performance measures of individual market advisory services across overlapping pairs of crop years are presented in Table 5. [14] ^, [15]   Figure 3 presents a graphical illustration of the rank correlation across crop years for wheat.  Estimated correlation coefficients for 1995 vs. 1996 and 1996 vs. 1997 are near zero in absolute magnitude and insignificantly different from zero for all three performance measures.  In contrast, each of the three correlations estimated for 1997 vs. 1998 are relatively large, at about 0.80.  All three are significantly different from zero in this case. The net result is a small average correlation coefficient across the three pairs of years, ranging from about 0.20 to 0.30.  These comparisons suggest some positive consistency of pricing performance in wheat through time.

Estimated correlation coefficients and tests of significance for non-overlapping pairs of adjacent crop years are presented in Table 6.  The results differ sharply from those for overlapping years.  All six of the estimated correlations are negative.  Most striking is the large absolute magnitude and significance of the correlations for 1995 vs. 1997.  These correlations are statistically significant and about –0.45 in magnitude. The average correlation for the two pairs of non-overlapping years ranges from –0.31 to –0.36, indicating a tendency for performance reversal. Since the non-overlapping results tend to be in the opposite direction of the correlations observed for overlapping years, the correlation of performance through time appears to be quite fragile, in the sense of being sensitive to the nature of the comparisons.

Results of the “winner” and “loser” predictability test for overlapping crop years are shown in Table 7.  It is worth noting that this test of predictability is not as sensitive to outliers in pricing performance, either positive or negative, as the previous correlation tests.  Hence, it is possible for the results to differ across the two sets of tests.  The winner and loser counts, however, follow the pattern found in the previous correlation tests.  The first two pairs of years (1995 vs. 1996 and 1996 vs. 1997) show there is little difference in the odds of a winner or loser in one period being a winner or loser in the subsequent period.  As an example, consider the results for 1996 and 1997.  Of the nine winners in 1996, four are winners (top half) in 1997 and five are losers (bottom half).  The corresponding odds ratio is 0.80, which indicates that the odds (4/5) of a winning service in 1996 being a winning service in 1997 are 0.80 times the odds (5/5) of a losing service in 1996 being a winning service in 1997.  The winner and loser counts for 1997 vs. 1998 contrast markedly with those of the previous two pairs of year. In this case, the odds ratio is 81 and significantly different from one at the one-percent level.   The odds ratio after pooling all pairs of years over 1995-1998 is 2.38 and significantly different from one at the ten percent level.  The overall significance is obviously driven be the results for the 1997 vs. 1998 comparison. Nonetheless, these comparisons suggest some positive consistency of pricing performance in wheat through time.

Results of the winner and loser predictability test for non-overlapping crop years are shown in Table 8.  Significant odds ratios are not found for either of the two pairs of years or for the pooled total.  Mirroring the correlation results, a tendency for reversals is evident, in that winners are more likely to lose in the next year and vice versa.  Once again, the contrast in the results between overlapping and non-overlapping data highlights the sensitivity of the results to the nature of the comparisons.

Results for the test of predictability based on the difference between pricing performance for “top” and “bottom” performing advisory services across overlapping pairs of adjacent crop years are shown in Table 9.  Nominally there is some evidence that top services outperform bottom services.  In both cases, the average net advisory price for services in the top quantile (thirds or fourths) exceeds the average net advisory price for services in the bottom quantile. This is most evident when comparing average prices for the top fourth and bottom fourth, with net prices for the top group exceeding those of the bottom group by $0.41 per bushel. However, t-statistics indicate that neither of the positive price premiums for top performers is significantly different from zero, although some of the lack of significance certainly can be attributed to the fact that only three observations are used to compute the test statistics.  It is interesting to observe that the average return for all quantiles is substantially negative, indicating that regardless of which quantile is selected net advisory prices are less than the market benchmark.  Top performers simply underperform the market less than bottom performers.

Results for the test of predictability based on the difference between pricing performance for “top” and “bottom” performing advisory services across non-overlapping pairs of adjacent crop years are shown in Table 10.  These results tend to be just the opposite of those observed for overlapping years. In all cases, the average net advisory price for services in the top quantile (thirds or fourths) is below the average net advisory price for services in the bottom quantile. For example, net prices for the top fourth of services in wheat, on average, are $0.39 per bushel less than the comparable average prices for bottom fourth services.  Once again, t-statistics indicate that none of the negative premiums for top performers is significantly different from zero. Finally, it is worth noting that both the top third and top fourth quantiles generate average returns that are substantially negative, so these “top” services not only trail bottom performers, but also the market benchmark.

The practical implications of the contrary top- and bottom-performer results (at least nominally) for overlapping versus non-overlapping years are striking.  Consider the case of a farmer who uses 1995 performance results to select a top-fourth advisory service.  As shown in Table A9 in the Appendix, the 1995 and 1996 comparisons suggest that services in the top fourth outperform services in the bottom fourth by $1.18 per bushel.  However, since the 1995 marketing window ends on May 31, 1996, halfway through the 1996 marketing window and one day before the beginning of the 1997 marketing window, the farmer could fully implement their choice of advisory service only for the 1997 crop. The comparisons in Table A15 show that top-performing advisory services in 1995 tend to be the bottom-performing services in 1997, just the opposite of what the farmer expected. In fact, top-fourth performing services underperform bottom-fourth performing services in 1997 by $0.74 per bushel. Similar results tend to be found for other years.

Overall, the test results presented in this section provide little evidence that future advisory service performance in wheat can be usefully predicted from past performance.  Most test results show no statistically significant predictability.  When predictability is found, it is sensitive to the nature of the comparisons (overlapping versus non-overlapping crop years) and statistical test considered. The previous conclusion does not mean it is impossible to predict advisory service performance. There may be other variables associated with performance that can be used for prediction.  For example, Chevalier and Ellison (1999) study whether mutual fund performance is related to characteristics of fund managers that indicate ability, knowledge or effort, and find that managers who attended higher-SAT undergraduate institutions generate systematically higher returns.  Barber and Odean (2000) examine the trading records of individual stock investors and report that frequent trading substantially depresses investment returns.  Similar factors, such as education of advisors, cash only services versus futures and options services, frequency of futures and options trading, or storage costs, may be useful in predicting the performance of agricultural market advisory services in wheat.

Summary

Farmers view market advisory services as a significant source of market information and advice in their quest to manage price risks associated with commodity marketing.  Previous studies only examine advisory service performance in marketing corn and soybeans.  It is not known whether the results generalize to other commodities with different production and consumption characteristics.  Wheat represents an interesting additional market to examine advisory service performance. It differs significantly from corn and soybeans with respect to the timing and location of production, yield growth trends, seasonality and consumption uses.  Hence, we would expect different marketing patterns, and potentially, different results than have been reported for corn and soybeans.

The purpose of this report is to investigate the performance of agricultural market advisory services in marketing wheat.  Two key performance questions are addressed: 1) Do market advisory services, on average, outperform an appropriate wheat market benchmark? and 2) Do market advisory services exhibit persistence in their wheat performance from year-to-year?  Market advisory service recommendations for wheat are available from the AgMAS Project for the 1995, 1996, 1997 and 1998 crop years.  At least 20 advisory programs are included for each year. While the sample of advisory services is non-random, it is constructed to be generally representative of the majority of advisory services available to farmers. The tests used to determine average performance of market advisory services and predictability of performance through time have been widely applied in the financial literature.

Tests of pricing performance relative to a market benchmark are based on the proportion of programs exceeding the benchmark price and the average percentage difference between the net price of advisory programs and the benchmark price.  In statistical terms, the pricing performance test results are clear.  Not only do market advisory programs in wheat consistently fail to “beat the market,” their performance is significantly worse than the market.  The level of under-performance is striking and consistent, with the proportion of programs above market benchmarks for the four-year period ranging from 0.32 to 0.34.  Point estimates of the four-year average return relative to market benchmarks range from –9.75 to –10.48 percent.

Given the statistical results summarized above, a relevant question to ask is whether the pricing under-performance of advisory programs also is economically significant.  A useful perspective on this question is gained by examining wheat revenue per acre.  The best point estimate of advisory revenue return probably is the simple average across the benchmarks.  This “grand average” revenue return across all four crop years and three benchmarks is –10.20 percent, which translates into advisory revenue averaging $14 per acre below benchmark revenue.  By any reasonable standard, this is an economically non-trivial level of under-performance.

The pricing performance results for wheat stand in sharp contrast to those reported for corn and soybeans.   Irwin, Good, Jackson and Martines-Filho (2000) analyze the pricing performance of corn and soybean market advisory programs tracked by the AgMAS Project over 1995-1998.  They find that market advisory services in corn and soybeans have a “small” ability to “beat the market,” with combined corn and soybean revenue for the advisory programs averaging about $4 per acre more than benchmark revenue.  Two explanations seem plausible for the divergence in results across corn and soybeans and wheat.  First, the divergence may simply be an artifact of a relatively small sample of years, where wheat advisory performance is by chance unusually poor and/or corn and soybean advisory performance is unusually good.  Second, advisory programs may be more skillfull in analyzing and forecasting corn and soybean prices than wheat prices.

Three tests of predictability are used and, in general, they provide little evidence future advisory service performance in wheat can be usefully predicted from past performance.  The average correlation coefficient relating performance from one year to the next generally is insignificantly different from zero.  Winner and loser counts generally indicate little difference in the odds of a winner or loser in one period being a winner or loser in the subsequent period.  Average pricing performance of top-performing services is insignificantly different from that of bottom-performing services. Finally, when predictability is found, it is sensitive to the nature of the comparisons (overlapping versus non-overlapping crop years) and statistical test considered.

Anderson, D.R., D.J. Sweeney and T.A. Williams.  Statistics for Business and Economics, Sixth Edition.  West Publishing Company:  Minneapolis/St. Paul, 1996.

Barber, B.M. and T. Odean."  "Trading is Hazardous to Your Wealth: The Common Stock Investment Performance of Individual Investors."  Journal of Finance. 55(2000):773-806.

Bodie, Z., A. Kane, and A.J. Marcus.  Investments.  Irwin: Homewood, IL, 1989.

Brorsen, B.W., D.W. Buck and S.R. Koontz.  “Hedging Hard Red Winter Wheat: Kansas City versus Chicago.”  Journal of Futures Markets. 18(1998):449-466. 

Brorsen, B.W. and J. Townsend.  "Performance Persistence for Managed Futures." Proceedings of the 1998 NCR‑134 Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management. Department of Agricultural Economics, Kansas State University, pp. 337-354.

Brown, S. J., W. Goetzmann, R.G. Ibbotson, and S.A. Ross. "Survivorship Bias in Performance Studies." Review of Financial Studies. 5(1992):553-580.

Brown, S.L. “A Reformulation of the Portfolio Model of Hedging.” American Journal of Agricultural Economics..  67(1985):508-512.

Carpenter, J.N. and A.W. Lynch. “ Survivorship Bias and Attrition Effects in Measures of Performance Persistence.”  Journal of Financial Economics. 54(1999):337-374.

Chevalier, J. and G. Ellison.  "Are Some Mutual Fund Managers Better Than Others? Cross-Sectional Patterns in Behavior and Performance."  Journal of Finance. 54(1999):875-899.

Christensen, R.  Log-Linear Models and Logistic Regression, Second Edition.  New York:  Springer-Verlag, 1997.

Clements, J.  "Debunking Some Mutual-Fund Myths."  The Wall Street Journal, March 16, 1999.

Coble, K.H., G.F. Patrick, T.O. Knight, and A.E. Baquet. “Crop Producer Risk Management Survey:  A Preliminary Summary of Selected Data.”  Information Report 99-001, Department of Agricultural Economics, Mississippi State University, September 1999.

Elton, E. J., M. J. Gruber, and J. C. Rentzler. "Professionally Managed, Publicly Traded Commodity Funds." Journal of Business. 60(1987):175-199.

Fama, E. "Efficient Capital Markets: A Review of Theory and Empirical Work." Journal of Finance. 30(1970):1043-1053.

Friend, I., M. Blume and J. Crockett. Mutual Funds and Other Institutional Investors. New York: McGraw-Hill, 1970.

Gehrt, D. and Good.  "Evaluation of Market Advisory Services for Corn and Soybeans."  Journal of the American Society of Farm Managers and Rural Appraisers. 57(1993):1-7.

Good, D.L., S.H. Irwin and T.E. Jackson. "Development of a Market Benchmark for AgMAS Performance Evaluations." AgMAS Project Research Report 1998-02, December 1998.

Grossman, S. and J. E. Stiglitz. "On the Impossibility of Informationally Efficient Markets." The American Economic Review. 70(1980):393-408,.

Irwin, S.H., C.R. Zulauf, and B.L. Ward.  "The Predictability of Managed Futures Returns." Journal of Derivatives.  2(1994): 20-27.

Irwin, S.H., D.L. Good, J. Martines-Filho and T.E. Jackson. "Do Agricultural Market Advisory Services Beat the Market? Evidence from the Corn and Soybean Markets Over 1995-1998." AgMAS Project Research Report 2000-03, November 2000.

Jaffe, J.F.  and J.M. Mahoney.  "The Performance of Investment Newsletters." Journal of Financial Economics. 53(1999):289-307.

Jirik, M.A., D.L. Good, S.H. Irwin, T.E. Jackson, and J. Martines-Filho. “The 1995 Through 1998 Pricing Performance of Market Advisory Services for Wheat.”  AgMAS Project Research Report 2000-02, June 2000.

Kastens, T.L. and T.C. Schroeder.  "Efficiency Tests of July Kansas City Wheat Futures." Journal of Agricultural and Resource Economics. 21(1996):187-198.

King, R.P., L.S. Lev and W.E. Nefstad.  "A Position Report for Farm-Level Marketing Management."  Review of Agricultural Economics. 17(1995):205-212.

Lakonishok, J., A. Shleifer, and R.W. Vishny.  "The Structure and Performance of the Money Management Industry."  Brookings Papers:  Microeconomics  (1992):339-391.

Malkiel, B.G. “Returns from Investing in Equity Mutual Funds 1971 to 1991.” Journal of Finance. 50(1995):549-572.

Malkiel, B.G. A Random Walk Down Wall Street. New York: W.W. Norton and Company, 1999.

Marcus, A.J.  “The Magellan Fund and Market Efficiency.”  Journal of Portfolio Management.  16(1990):85-88.

Marten, J. "Farmers Want Market News, Not Advice."  Farm Journal Extra, June 1994.

Martines-Filho, J.G.  Pre-Harvest Marketing Strategies for Corn and Soybeans:  A Comparison of Optimal Hedging Models and Market Advisory Service Recommendations.  Unpublished Ph.D. dissertation, The Ohio State University, 1996.

Metrick, A.  "Performance Evaluation with Transactions Data:  The Stock Selection of Investment Newsletters."  Journal of Finance. 54(1999):1743-1776.

Norvell, J. M.  and D. H.  Lattz.  “Value-Added Crops, GPS Technology and Consultant Survey: Summary of a 1998 Survey to Illinois Farmers.”  Working Paper, College of Agricultural, Consumer, and Environmental Sciences, University of Illinois, July 1999.

Otte, J. "Marketing Matters -- How Well Do Market Advisors Deliver?"  Prairie Farmer, July 19, 1986.

Patrick, G.F. and S. Ullerich.  "Information Sources and Risk Attitudes of Large-Scale Farmers, Farm Managers, and Agricultural Bankers."  Agribusiness. 12(1996):461-471.

Patrick, G.F., W.N. Musser, and D.T. Eckman.  "Forward Marketing Practices and Attitudes of Large-Scale Midwestern Grain Farmers."  Review of Agricultural Economics. 20(1998):38-53.

Powers, L.  "How to Measure Your Pro's Performance."  Top Farmer, April 1993, p. 17.

Ritter, J.R.  "The Long-Run Performance of Initial Public Offerings."  Journal of Finance. 46(1991):3-27.

Schroeder, T.C., J.L. Parcell, T.L. Kastens, and K.C. Dhuyvetter.  "Perceptions of Marketing Strategies; Farmers vs. Extension Economists.”  Journal of Agricultural and Resource Economics. 23(1998):279-293.

Wisner, R.N., E.N. Blue and E.D. Baldwin.  “Preharvest Marketing Strategies Increase Net Returns for Corn and Soybean Growers.” Review of Agricultural Economics. 20(1998):288-307.

Zulauf, C.R. and S.H. Irwin.  “Market Efficiency and Marketing to Enhance Income of Crop Farmers.” Review of Agricultural Economics. 20(1998):308-331.

Click here for Order Form