Do Agricultural Market Advisory Services Beat the Market? Evidence from the Corn and Soybean Markets Over 1995-1998

November 2000                                                                                                             

Scott H. Irwin, Darrel L. Good, Joao Martines-Filho and Thomas E. Jackson [*]

Copyright 2000 by Scott H. Irwin, Darrel L. Good, Joao Martines-Filho and Thomas E. Jackson. All rights reserved. Readers may make verbatim copies of this document for non-commercial purposes by any means, provided that this copyright notice appears on all such copies.

DISCLAIMER

The advisory service marketing recommendations used in this research represent the best efforts of the AgMAS Project staff to accurately and fairly interpret the information made available by each advisory program.  In cases where a recommendation is vague or unclear, some judgment is exercised as to whether or not to include that particular recommendation or how to implement the recommendation.  Given that some recommendations are subject to interpretation, the possibility is acknowledged that the AgMAS track record of recommendations for a given program may differ from that stated by the advisory service, or from that recorded by another subscriber.  In addition, the net advisory prices presented in this report may differ substantially from those computed by an advisory service or another subscriber due to differences in simulation assumptions, particularly with respect to the geographic location of production, cash and forward contract prices, expected and actual yields, carrying charges and government programs

The purpose of this paper is to address two basic performance questions for market advisory services: 1) Do market advisory services, on average, outperform an appropriate market benchmark? and 2) Do market advisory services exhibit persistence in their performance from year-to-year?  Data on corn and soybean net price received for advisory services, as reported by the AgMAS Project, are available for the 1995, 1996, 1997 and 1998 crop years.  Performance test results suggest that, on average, market advisory services exhibit a small ability to "beat the market" for the 1995 through 1998 corn and soybean crops.  It is debatable whether the performance of advisory services also is economically significant. The predictability results provide little evidence that future advisory service pricing performance can be predicted from past performance. 

            Farmers in the US continue to identify price and income risk as one of their greatest management challenges.  Using a survey of midwestern grain farmers, Patrick and Ullerich (1996) report that price variability is the highest rated source of risk by crop farmers. Coble, Patrick, Knight and Baquet (1999) survey farmers in Indiana, Mississippi, Nebraska and Texas and find that crop price variability, by a wide margin, is rated as having the most potential to affect farm income.  Norvell and Lattz (1999) survey a random sample of Illinois farmers and show that price and income risk management rank second (following computer education and training) among ten business categories in which farmers identify needs for additional consulting services.  The desire for greater assistance with price and income risk management is not limited to large farms, as the proportion of farmers expressing this preference actually is highest for those operating medium-sized Illinois farms (500-999 acres). 

            Farmers view market advisory services as a significant source of market information and advice in their quest to manage price risks associated with grain marketing. In a rating of seventeen risk management information sources, Patrick and Ullerich (1996) report that the rank of market advisors and computerized information services is surpassed only by farm records.  Schroeder, Parcell, Kastens and Dhuyvetter (1998) find that a sample of Kansas farmers rank market advisory services as the number one source of information for developing price expectations. Norvell and Lattz (1999) find that twenty-one percent of Illinois respondents currently use marketing consultants, and that such consultants tie for first (with accountants), in a list of seven, as likely to be most important to their business in the future. 

              Given the high value that farmers place upon market advisory services, it is somewhat surprising that only two academic studies investigate the pricing performance of advisory services.[1]  The dearth of studies seems even more anomalous in light of the large number of studies on grain marketing strategies.[2]  The lack of studies on market advisory services is most likely due to the difficulty in obtaining data on the stream of recommendations provided by services.

            Gehrt and Good (1993) analyze the performance of five advisory services for corn and soybeans over the 1985 through 1989 crop years.[3]  Assuming a representative farmer follows the hedging and cash market recommendations for each advisory service, a net price received for each year is computed and compared to a benchmark price.  They generally find that corn and soybean farmers obtained a higher price by following the marketing recommendations of advisory services.  Martines-Filho (1996) examines the pre-harvest corn and soybean marketing recommendations of six market advisory services over 1991 through 1994.  He computes the harvest time revenue that results from a representative farmer following the pre-harvest futures and options hedging recommendations and selling 100 percent of production at harvest.  Average advisory service revenue over the four years is larger than benchmark revenue for both corn and soybeans.

            While a useful starting point, the two previous studies have important limitations.  First, the sample of advisory services is quite small, with the largest sample including only six advisory services.  Second, the results may be biased due to the nature of the sample selection process.  The literature on the performance of mutual funds and investment newsletters highlights the sample selection biases that plague many performance results (e.g., Brown, Goetzmann, Ibbotson, and Ross, 1992; Jaffe and Mahoney, 1999; Metrick, 1999).  The most relevant bias for previous studies of market advisory services is survivorship bias, which results from tracking only advisory services that remain in business at the end of a sample period.

            The previous discussion suggests the academic literature provides farmers with little basis for evaluating and selecting advisory services.  In 1994, the Agricultural Market Advisory Service (AgMAS) Project was initiated, with the goal of providing unbiased and rigorous evaluation of market advisory services for farmers.  The AgMAS Project has collected marketing recommendations for about 25 market advisory services each crop year.  The AgMAS Project subscribes to all of the services that are followed, and as a result, "real-time" recommendations are obtained.  This prevents the data from being subject to survivorship bias.

            After the stream of recommendations is collected by AgMAS staff for a given commodity in a particular crop year, the net price that would have been received by a farmer that precisely follows the set of marketing recommendations is computed.  This net price is the weighted average of the cash sale price plus or minus gains/losses associated with futures and options transactions.  Brokerage costs are accounted for, as are the costs of storing any portion of the crop beyond harvest.  So far, the AgMAS Project has reported corn and soybean results for the 1995, 1996, 1997 and 1998 crop years. (Good, Irwin, Jackson, and Price, 1997; Jackson, Irwin, and Good, 1998; Jackson, Irwin, and Good, 1999; Good, Irwin, Jackson, Jirik and Martines-Filho, 2000).

            The annual AgMAS comparison of net price received for advisory services provides important information that farmers can use in selecting a service.  However, the comparisons to date are descriptive only and do not rigorously address the central questions regarding pricing performance.  Following the literature on mutual fund and investment newsletter performance (e.g., Jaffe and Mahoney, 1999), two basic questions need to be answered: 1) Do market advisory services, on average, outperform an appropriate market benchmark? and 2) Do market advisory services exhibit persistence in their performance from year-to-year? 

            The purpose of this report is to address the previous two questions for corn and soybeans using the net advisory prices reported by the AgMAS Project for the 1995, 1996, 1997 and 1998 crop years.  The results update those found in Irwin, Jackson and Good (1999) by adding data for the 1998 crop year.  At least 21 advisory services are included in the evaluations for each commodity and crop year.  While the sample of advisory services is non-random, it is constructed to be generally representative of the majority of advisory services offered to farmers.  The availability of only four crop years is a limitation of the analysis, but the time period considered does include years of rapidly increasing and decreasing corn and soybean prices.

            The tests used to determine average performance of market advisory services and predictability of performance through time have been widely applied in the financial literature (e.g., Elton, Gruber, and Rentzler, 1987; Lakonishok, Shleifer and Vishny, 1992; Irwin, Zulauf, and Ward, 1994; Jaffe and Mahoney, 1999; Metrick, 1999; Carpenter and Lynch, 1999).  Two tests of performance relative to a benchmark are used: i) the proportion of services exceeding the benchmark price and ii) the average percentage difference between the net price of services and the benchmark price.  Three tests of predictability are used: i) the correlation of advisory service pricing performance measures from year-to-year, ii) the predictability of “winner” and “loser” categories from year-to-year and iii) the differences between pricing performance measures for “top” and “bottom” performing advisory services.

            The market advisory services included in this evaluation do not comprise the population of market advisory services available to farmers.  The included services also are not a random sample of the population of market advisory services.  Neither approach is feasible because no public agency or trade group assembles a list of advisory services that could be considered the "population."  Furthermore, there is not a generally agreed upon definition of an agricultural market advisory service.  To assemble a sample of services for the AgMAS Project, criteria were developed to define an agricultural market advisory service and a list of services assembled.

            The first criterion used to identify services is that a service has to provide marketing advice to farmers. Some of the services tracked by the AgMAS Project do provide speculative trading advice, but that advice must be clearly differentiated from marketing advice to farmers for the service to be included.  The terms "speculative" trading of futures and options versus the use of futures and options for "hedging" purposes are used for identification purposes only.  A discussion of what types of futures and options trading activities constitute hedging, as opposed to speculating, is not considered.

            The second criterion is that specific advice must be given for making cash sales of the commodity, in addition to any futures or options hedging activities.  In fact, some marketing programs evaluated by the AgMAS Project do not make any futures and options recommendations.  However, marketing programs that make futures and options hedging recommendations, but fail to clearly state when cash sales should be made, or the amount to be sold, are not considered.

            The original sample of market advisory services that met the two criteria were drawn from the list of  "Premium Services" available from the two major agricultural satellite networks, Data Transmission Network (DTN) and FarmDayta in the summer of 1994.[4]^{, [5]}  While the list of advisory services available from these networks was by no means exhaustive, it did have the considerable merit of meeting a market test.  Presumably, the services offered by the networks were those most in demand by farm subscribers to the networks.  In addition, the list of available services was cross-checked with other farm publications to confirm that widely-followed advisory firms were included in the sample.  It seems reasonable to argue that the resulting sample of services was (and remains) generally representative of the majority of advisory services available to farmers.

            The original sample for 1995 includes 25 market advisory services for both corn and soybeans.  For a variety of reasons, deletions and additions to the original sample occur over time.[6]  In 1996, the total number of advisory services is 26 for corn and 24 for soybeans, while in 1997 the total is 23 for corn and 21 for soybeans.  In 1998, the total is again 23 for corn, but the total number of services for soybeans increases to 22.[7]  A directory of the advisory services included in the study can be found at the AgMAS Project website (http://web.aces.uiuc.edu/farmdoc/agmas/).

            As mentioned earlier, sample selection biases may plague advisory service databases.  The first form is survival bias, which occurs if only advisory services that remain in business at the end of a given period are included in the sample.  Survival bias significantly biases measures of performance upwards since "survivors" typically have higher performance than "non-survivors" (Brown, Goetzmann, Ibbotson, and Ross, 1992).  This form of bias should not be present in the AgMAS database of advisory services because all services ever tracked are included in the sample.  The second and more subtle form of bias is hindsight bias, which occurs if data from prior periods are "back-filled" at the point in time when an advisory service is added to the database.  Statistically, this has the same effect as survivorship bias because data from surviving advisory services are back-filled.  This form of bias should not be present in the AgMAS database because recommendations are not back-filled when an advisory service is added.  Instead, recommendations are collected only for the crop year after a decision has been made to add an advisory service to the database.

            The actual daily process of collecting recommendations for the sample of advisory services begins with the purchase of subscriptions to each of the services.  Staff members of the AgMAS Project read the information provided by each advisory service on a daily basis.  The information is received electronically, via DTN, websites or e-mail.  For the services that provide two daily updates, typically in the morning and at noon, information is read in the morning and afternoon.  In this way, the actions of a farmer-subscriber are simulated in “real-time.”

            The recommendations of each advisory service are recorded separately.  Some advisory services offer two or more distinct marketing programs.  This typically takes the form of one set of advice for marketers who are willing to use futures and options (although futures and options are not always used), and a separate set of advice for farmers who only wish to make cash sales.[8]  In this situation, both strategies are recorded and treated as distinct strategies to be evaluated.[9]

            Several procedures are used to check the recorded recommendations for accuracy and completeness.  Whenever possible, recorded recommendations are cross-checked against later status reports provided by the relevant advisory service.  Also, at the completion of the crop year, it is confirmed whether cash sales total exactly 100%, all futures positions are offset, and all options positions are offset or expire worthless.

            The final set of recommendations attributed to each advisory service represents the best efforts of the AgMAS Project staff to accurately and fairly interpret the information made available by each advisory service.  In cases where a recommendation is considered vague or unclear, some judgment is exercised as to whether or not to include that particular recommendation.  This occurs most often when a service suggests “a farmer might consider” a position, or when minimal guidance is given as to the quantity to be bought or sold.  Given that some recommendations are subject to interpretation, the possibility is acknowledged that the AgMAS track record of recommendations for a given service may differ from that stated by the advisory service, or from that recorded by another subscriber.

            At the end of a crop year, all of the (filled) recommendations are aligned in chronological order.  The advice for a given crop year is considered to be complete for each advisory service when cumulative cash sales of the commodity reach 100%, all open futures positions covering the crop are offset, all open option positions covering the crop are either offset or expired, and the advisory service discontinues giving advice for that crop year.  The returns to each recommendation are then calculated in order to arrive at a weighted-average net price that would be received by a farmer who precisely follows the marketing advice (as recorded by the AgMAS Project).

            In order to simulate a consistent and comparable set of results across the different advisory services, certain explicit assumptions are made.  These assumptions are intended to accurately depict marketing conditions for a representative, central Illinois farm.  An overview of the simulation assumptions is presented below.  Complete details of the simulation assumptions can be found in Good, Irwin, Jackson, Jirik and Martines-Filho (2000).

Geographic Location

            The simulation is designed to reflect conditions facing a representative central Illinois corn and soybean farmer.  Whenever possible, data are collected for the Central Crop Reporting District in Illinois as defined by the National Agricultural Statistics Service (NASS) of the US Department of Agriculture (USDA).  Eleven counties (DeWitt, Logan, McLean, Marshall, Macon, Mason, Menard, Peoria, Stark, Tazewell, and Woodford) make up this District.

Marketing Window

            A two-year marketing window, spanning September of the year before harvest through August of the year after harvest, is used in the analysis.  For example, the 1997 marketing window is September 1, 1996 through August 31, 1998.  The beginning date is selected because  services in the sample generally begin to make recommendations around this date.  The ending date is selected to be consistent with the ending date for corn and soybean crop years as defined by the US Department of Agriculture (USDA).  There are a few exceptions to the marketing window definition.  Some advisory services have relatively small amounts (15% or less) of cash corn or soybeans unsold as of the end of a window.  Several advisory services also begin pre-harvest hedges prior to the September 1 start of the window.  In these cases, the actual sales recommendations on the indicated dates are recorded. Finally, note that throughout the remainder of this report, the term "crop year" is used to represent the two-year marketing window.

Prices

            The price assigned to each cash sale recommendation is the central Illinois closing, or overnight, bid.  The data are collected and reported by the Illinois Department of Ag Market News. The central Illinois price is the mid-point of the range of bids by elevators in the North Central and South Central Price Reporting Districts, as defined by the Illinois Department of Ag Market News.  Prices in this 25-county area best reflect prices for the assumed geographic location of the representative central Illinois farmer (Central Illinois Crop Reporting District). The central Illinois market also is used for cash-forward contract transactions.  Futures prices and options premia are Chicago Board of Trade (CBOT) quotes.

Quantity Sold

            Since most of the advisory service recommendations are given in terms of the proportion of total production (e.g.,, “sell 5% of 1997 crop today”), some assumption must be made about the amount of production to be marketed.  For the purposes of this study, if the per-acre yield is assumed to be 100 bushels, then a recommendation to sell 5% of the corn crop translates into selling 5 bushels.  When all of the advice for the crop year has been carried out, the final per-bushel selling price is the average price for each transaction weighted by the amount marketed in each transaction.

            When making hedging or forward contracting decisions prior to harvest, the actual yield is unknown.  Hence, an assumption regarding the amount of expected production per acre is necessary to accurately reflect the returns to marketing advice.  Prior to harvest, the best estimate of the current year’s expected yield is assumed to be a function of yield in previous years.  In this study, the assumed yield prior to harvest is based on a linear regression trend yield, while the actual reported yield is used from the harvest period forward.

Brokerage Costs

            Brokerage costs are incurred when farmers open or lift positions in futures and options markets.  For the purposes of this study, it is assumed that brokerage costs are $50 per contract for a round-turn for futures transactions, and $30 per contract to enter or exit an options position.  Further, it is assumed that CBOT corn and soybean futures are used, and the contract size for each commodity is 5,000 bushels.  Therefore, per-bushel brokerage costs are 1 cent per bushel for a round-turn futures transaction and 0.6 cents per bushel for each options transaction.

Carrying Costs

            An important element in assessing returns to an advisory service is the economic cost associated with storing grain instead of selling grain immediately at harvest.  The cost of storing grain after harvest (carrying costs) consists of two components: physical storage charges and the opportunity cost incurred by foregoing sales when the crop is harvested.  Physical storage charges can apply to off-farm (commercial) storage, on-farm storage, or some combination of the two.  Opportunity cost is the same regardless of the type of physical storage.

            For the purposes of this study, it is assumed that all storage occurs off-farm at commercial sites.  Carrying costs are assumed to begin after the last day of harvest.  Physical storage charges are assumed to be a flat 13 cents per bushel from the end of harvest through December 31.  After January 1, physical storage charges are assumed to be 2 cents per month (per bushel), with this charge pro-rated to the day when the cash sale is made.  The storage costs represent the typical storage charges quoted in a non-random telephone survey of central Illinois elevators.

            The interest charge for storing grain is the interest rate compounded daily from the end of harvest to the date of sale.  The interest rate used is the average rate for all commercial agricultural loans for the fourth quarter of the harvest year and the first three quarters of the next calendar year as reported in the Agricultural Finance Databook published by the Board of Governors of the Federal Reserve Board.  This interest rate has been around 9% per year for the four years of this study.

            In addition to the storage and interest costs, another charge is assigned to corn (but not soybeans).  This charge, referred to as a “shrink charge”, is commonly deducted by commercial elevators on “dry” corn that is delivered to the elevator to be stored, and reflects a charge for drying and volume reduction (shrinkage) which occurs in drying the corn from (typically) 15% to 14% moisture.  The charge for drying is a flat 2 cents per bushel, while the charge for volume reduction is 1.3% per bushel.  The charge for this volume reduction is calculated as 1.3% times the average harvest-time cash price for each crop year.  For example, for the 1998 corn crop the harvest-time cash price was $1.91 per bushel, so the charge for volume reduction was 2.5 cents per bushel ($1.91*0.013).

            The price of both corn and soybeans is below the loan rate during significant periods of time in the 1998-1999 crop year, so that use of the marketing loan program is an important part of marketing strategies during this period.  Most of the advisory services tracked by the AgMAS Project for the 1998 crop make specific recommendations regarding the timing and method of implementing the loan program for the entire corn and soybean crops.  These recommendations are implemented as given wherever feasible.  Several decision rules have to be developed even in this case, in particular, for pre-harvest forward contracts.  For a few services, loan recommendations are incomplete or not made at all.  For these cases, it is necessary to develop a more complete set of decision rules for implementing the loan program in the marketing of corn and soybeans.  All loan-related decision rules are based on the assumption of a “prudent” or “rational” farmer, within the context of the intent of the loan program.  More specifically, it is assumed that a farmer will take advantage of the price protection offered by the loan program, even in the absence of specific advice from an advisory service. Further information on the decision rules used to implement marketing loan recommendations can be found in Good, Irwin, Jackson, Jirik and Martines-Filho (2000).

Market Benchmark

            Simply comparing the net price received across advisory services will not answer the question of whether advisory services as a group enhance the income of farm subscribers.  Instead, a comparison to a benchmark price (or prices) is needed to evaluate the performance of advisory services relative to pricing opportunities offered by the market.  In the stock market, mutual funds are evaluated with respect to market benchmark performance criteria (e.g., Bodie, Kane, and Marcus, 1989).  These benchmarks typically are indexes of stock market returns over the period of evaluation, such as the Dow Jones Industrial Average and Standard and Poor’s 500.

            The selection of appropriate benchmarks for advisory service performance evaluations is treated thoroughly in a report by Good, Irwin and Jackson (1998).  They argue that, conceptually, a useful benchmark should: 1) be simple to understand and to calculate; 2) represent the returns to a marketing strategy that could be implemented by farmers; 3) be directly comparable to the net advisory price received from following the recommendations of a market advisory service; 4) not be a function of the actual recommendations of the advisory services or of the actual marketing behavior of farmers, but rather should be external to their marketing activities; and 5) be stable, so that it represents the range of prices made available by the market throughout the crop year instead of representing the price during a small segment of the crop year.  The market benchmark price that Good, Irwin and Jackson argue is the most consistent with the above criteria is the average cash price for corn and soybeans over the entire marketing horizon.  The marketing window used in the AgMAS project for a given crop spans two calendar years, beginning on the first business day of September in the year prior to harvest, and extends through the last business day of August in the year after harvest.  As its name suggests, the benchmark is calculated as the average of the daily central Illinois cash grain bids available for the two-year marketing window.  Pre-harvest cash prices represent cash-forward bids for harvest delivery in central Illinois, while daily spot prices for central Illinois are used for the post-harvest period.

            Three adjustments are made to the daily cash prices to make the average cash price benchmark consistent with the calculated net advisory prices for each marketing program.  First, instead of taking the simple average of the daily prices, a weighted average price is calculated to account for changing yield expectations.  The daily weighting factors for pre-harvest prices are based on the calculated trend yield, while the weighting of the post-harvest prices is based on the actual reported yield for central Illinois.  The second adjustment to the daily cash prices is to adjust the post-harvest cash prices to a harvest equivalent by subtracting carrying charges.  The daily carrying charges are calculated in the same manner as those for the net advisory price. Complete details of the construction of this benchmark price can be found in Good, Irwin and Jackson (1998).

            A third adjustment to the average cash price benchmark is made only for 1998.  This adjustment is based on the logic that a  “prudent” or “rational” farmer will take advantage of the price protection offered by the marketing loan program when following the benchmark average price strategy.  Based on this argument, the average cash price benchmark is adjusted by the addition of marketing loan benefits.  Bushels marketed in the pre-harvest period according to the benchmark strategy (approximately 53 percent) are treated as forward contracts with the benefits assigned at harvest.  Bushels marketed each day in the post-harvest period (approximately 47 percent) are awarded marketing loan benefits in existence for that particular day. 

            In order to test the sensitivity of performance results to the choice of market benchmark,  two alternative versions of the previous average cash price benchmark also are considered in the analysis.  The first alternative benchmark averages prices for the 20-month period starting in January of the year of harvest and ending in August of the year after harvest. The only difference between this alternative and the 24-month benchmark is the exclusion of the pre-harvest period previous to January.  Hence, this alternative benchmark places more weight on post-harvest prices than pre-harvest prices.  The second alternative benchmark averages prices only for the 16-month period starting in May of the year of harvest and ending in August of the year after harvest.

            Net price received for the sample of market advisory services for the 1995, 1996, 1997 and 1998 crop years is reported in Tables 1 and 2.[10] Note that some of the market advisory services included in the table are not evaluated for all four years.  The four-year averages and standard deviations are calculated only for the 19 services that are evaluated for all four years.

            As shown in Table 1, the average net advisory price for corn ranges from $2.17 per bushel in 1998 to $3.04 per bushel in 1995.  The four-year average for the 19 services is $2.53 per bushel.  The range of four-year average net advisory prices is large, with a low of $2.36 and a high of $2.83.  Not surprisingly, the range within the individual years is even more substantial.  The most dramatic example is 1995, where the minimum is $2.29 per bushel and the maximum is $3.90 per bushel.  Even in years with less market price volatility, such as 1998, the range in performance is just under $0.60 per bushel. 

            The three alternative market benchmark prices for corn are shown at the bottom of Table 1.  Four-year averages of the market benchmarks differ by two cents per bushel or less.  However, this masks large differences within some of the years, particularly 1995.  These data suggest advisory service performance results for corn may be sensitive to the selected benchmark.  

            As reported in Table 2, the average net advisory price for corn ranges from $5.82 per bushel in 1998 to $7.27 per bushel in 1996.  The four-year average for the 19 services is $6.50 per bushel.  Again, the range of four-year average net advisory prices is large, with a low of $6.32 and a high of $6.88.  As with corn, the range within the individual years is even more substantial.  The most dramatic example is 1995, where the range in advisory prices exceeds two dollars per bushel.

            Since many subscribers to market advisory services produce both corn and soybeans, it is of interest to examine a combined measure of corn and soybean pricing performance for each market advisory service.  One way to aggregate the results is to calculate the per-acre revenues implied by the pricing performance results.[11]  The per-acre revenue for each commodity is found by multiplying the net advisory price for each market advisory service by the actual central Illinois corn or soybean yield for each year.  A simple average of the two per acre revenues is then taken to reflect a farm that uses a 50/50 rotation of corn and soybeans.

            Table 3 contains the combined corn and soybeans revenue results.  As with Tables 1 and 2, a four-year average is calculated only for services that were included in the study for all four years.  In addition, market advisory services that provide recommendations for corn but not soybeans are excluded.  The four-year average revenue for all 19 market advisory services is $325 per acre.  The four-year average for individual services ranges from a low of $312 per acre to a high of $349 per acre. 

            Two statistical tests are used to test the null hypothesis that average market advisory service pricing performance does not differ from that of the market benchmark.  The first test is based on the proportion of services exceeding the benchmark price.  This test is considered because it is not influenced by extremely high or low advisory prices.  The second test is based on the average percentage difference between the net price of services and the benchmark price.  This test is useful because it takes into account the average magnitude of differences from the benchmark.

            Before considering the statistical tests and results, an important issue needs to be explored that may have a substantial impact on the results.  The issue is whether the sample observations on net advisory price are independent, both within and across years.  The most likely form of dependence is positive correlation, which, if ignored, would cause sample standard deviation estimates across advisory services to be understated.  This in turn would cause the statistical significance of hypothesis test results to be overstated. 

            There are several potential ways that independence could be violated in the sample of market advisory service prices.  One potential violation is positive correlation of corn pricing performance for a market advisory service in a given year with its soybean pricing performance in the same year.  In other words, do services that do well in corn also tend to do well in soybeans in the same year?  If so, statistical tests that pool pricing performance of services for corn and soybeans may overstate the significance of positive or negative performance because the standard deviation across the corn and soybean observations would be understated.

            Correlation across corn and soybeans in a given year is computed three ways.  First, the correlation of rank across corn and soybeans for a given year is computed. To do this, the rank of each advisory service with respect to the other services is calculated separately for corn and soybeans.  The services are ranked in descending order.  For example, the service with the highest net advisory price is ranked number one, and the service with the lowest net advisory price is assigned a number equal to the total number of observations for that commodity in the given year.  The final step is to compute the correlation of the corn and soybean ranks.  Second, the simple correlation between the net advisory corn and soybean price levels is computed for a given year.  Third, the correlation of advisory service performance with respect to the 24-month market benchmark price is calculated.[12]  The “return” to market advice is calculated as the percentage difference between the net advisory price and the 24-month market benchmark price for the commodity.  A graphical view of the rank correlations is presented in Figure 1.

            The correlation results for market advisory corn and soybean pricing performance within the same crop year are presented in Table 4.  The results are similar across the different measures of correlation.  Significant positive correlation between corn and soybean pricing results is found in 1995 and 1997, but not for 1996 or 1998.  This may be due to the fact that the price patterns for corn and soybeans were somewhat different for the 1996 and 1998 crop years, while corn and soybean prices moved (generally) in the same direction during the 1995 and 1997 crop years.  While market advisory services do not make exactly the same recommendations for corn and soybeans in any given year, there often is a significantly positive correlation in their corn and soybean pricing performance.  This suggests it is inappropriate to pool separate corn and soybean pricing results when conducting statistical tests.

            A second potential source of dependence is correlation of net advisory prices through time for a given service and commodity.  This form of correlation may exist due to persistence in the performance of advisory services through time (winners continue to win, losers continue to lose).  It may also exist due to the overlapping nature of the crop years; each crop year is two calendar years long, and each set of contiguous crop years overlaps by one year.  If this correlation through time exists, it would be inappropriate to pool samples of net advisory prices across crop years for the same reason as discussed above.  As will be shown in the following section, this form of correlation generally is minimal, and therefore, it is reasonable to pool net advisory prices across crop years. 

            A third potential source of dependence perhaps is less obvious.  It is possible that net advisory prices for a given commodity and crop year are correlated because of the existence of similar programs offered by the same market advisory service.  For example, AgriVisor offers four marketing programs, which may not differ substantially in outcomes due to similar methods of analysis and similar underlying strategies.  The potential impact of this form of correlation is examined by creating one net advisory price for each of the market advisory firms that offer multiple programs.[13]  A single price is computed by averaging net advisory prices across programs for a given year and commodity.  Pricing performance results are qualitatively similar to those using the full set of disaggregated advisory prices, suggesting that net prices of advisory programs for the same firm are uncorrelated or no more correlated than net prices from different firms.  Hence, use of net advisory prices by program in tests of market performance does not appear to be a substantive problem.

            A formal test of the null hypothesis that the proportion of advisory services "beating" the market benchmark is insignificant requires the specification of an appropriate test statistic. First, define the sample estimate of the proportion for a given year and commodity as,

where k is the number of advisory services that have net prices exceeding the market benchmark price and n is the total number of advisory services in the sample. Anderson, Sweeney and Williams  (1996) show that the sample estimator of the proportion, , is distributed binomially with an expected value of  p and a standard error of, where p is the true value of the proportion in the population. They also note that the sampling distribution of  is approximately normal so long as  and .  Since both conditions are met for all of the samples considered here, the normality approximation is invoked.  The form of the test statistic based on the above assumptions is,

where p₀ is the assumed value of p under the null hypothesis.  The remaining issue is the expected proportion (p₀) under the null hypothesis.  The efficient market hypothesis (Fama, 1970) implies that the expected probability of “beating the market” is the same as the result of flipping a coin and showing heads, or 0.5.  Setting , the test statistic is,

             A formal test of the null hypothesis that the average percentage difference between the net price of services and the benchmark price is zero also requires the specification of an appropriate test statistic.  First, define the percentage difference for the i^th advisory service for a given crop year and commodity as,

where NAP_i is the net advisory price for the i^th advisory service and BP is the market benchmark price for the same commodity and crop year. The sampling distribution of  is well-known and does not need to be described in detail here.  The test statistic for a null hypothesis of zero average percentage difference is,

where  is the estimated standard deviation of the differences across the n advisory services in the sample.  The t-statistic follows a t-distribution with n-1 degrees of freedom. 

            As noted earlier, r_i can be thought of as the “return” to following the recommendations of a particular market advisory service.  This raises the question of whether the calculated “returns” are risk-adjusted.  One method of adjusting returns for risk that has been used in a number of studies stock investment strategies (e.g., Friend, Blume and Crocket, 1970; Ritter, 1991) is to match the average risk of the investments to the risk of the benchmark.  Hence, if the average risk of advisory services is equal to risk of the market benchmark, then market advisory returns can be considered risk-adjusted returns.  Evidence on the appropriateness of this “risk-matching” assumption for advisory services can be found in Tables 1, 2 and 3, where the standard deviations for the advisory services and market benchmarks can be found in the last column of each table.  As shown in Table 1, the average standard deviation for advisory services in corn is $0.42 per bushel, near the middle of the range of standard deviations for the three benchmarks.  Examining Table 2, the average standard deviation for advisory services in soybeans is $0.67 per bushel, again near the middle of the range of standard deviations for the three benchmarks.  Turning to Table 3, the average standard deviation for advisory service revenue is $36 per acre, near the top of the range of standard deviations for the three benchmarks.  Overall, the comparisons suggest the risk of the market benchmarks roughly matches the average risk of the advisory services, and hence, computed “returns” may be considered risk-adjusted. However, given the short time-period considered in these comparisons, a risk-adjusted interpretation of advisory returns should be treated with a good bit of caution.

            It is important to emphasize that the tests discussed in this section address the pricing performance of market advisory services as a group.  In other words, average pricing performance across all services is considered.  This is a different issue than the pricing performance of a particular advisory service.  It is possible that advisory services as a group fail to beat the market, yet at the same time there exist a small number of services that are exceptions to this outcome. In the stock market, this argument is often made with respect to the performance of the Fidelity Magellan Fund.  Testing whether an “exceptional” advisory service beats the market requires more data than is available for this study and different statistical methods (Marcus, 1990). 

            Table 5 reports results of the proportional test of corn pricing performance for each year and all four years pooled.  Statistical significance is based on a null hypothesis proportion of 0.5, the same as the proportion of heads observed in the flips of a fair coin.  Individual year results are quite sensitive to the benchmark considered.  For example, the proportion of services above the 24-month benchmark price in 1995 is 0.72 and statistically different from 0.5, while the proportion of services above the 16-month benchmark is only 0.12 and also significantly different from 0.5.  A similar contrast in test results is found in 1998.  The overall proportions for the four years are not as variable across the benchmarks, ranging from 0.46 to 0.59.  Pooled four-year proportions based on the 24-month and 20-month proportions are insignificantly different from 0.5, while the 16-month benchmark proportion is significant at the ten-percent level.

            Table 6 shows the results of the proportional test of soybean pricing performance for each year and all four years pooled. Like corn, individual year results are sensitive to the benchmark considered.  The most dramatic contrast again can be found in 1995, where the proportion of services above the 24-month benchmark price is 0.84 and significantly different from 0.5, while the proportion of services above the 16-month benchmark is 0.56 and not significantly different from 0.5.  Despite the variation across benchmarks in the individual years, the pooled proportions for the four years are similar across benchmarks, ranging only from 0.65 to 0.73.  All of the four-year proportions are significantly greater than 0.5 at the one-percent level.

            Table 7 reports proportional test results for combined corn and soybean revenue.  Given the evidence of positive correlation between the pricing performance of advisory services for corn and soybeans in the same year, it is inappropriate to simply pool the separate net price observations for corn and soybeans to test combined performance.  Instead, corn and soybean net prices are aggregated to form a single observation on per-acre revenue for each advisor and year, and then proportions are computed.  The per-acre combined revenues are those first presented in Table 3, with the per-acre revenue for each commodity found by multiplying the net advisory price for each market advisory service by the actual central Illinois corn or soybean yield for each year.  A simple average of the two per acre revenues is then taken to reflect a farm that uses a 50/50 rotation of corn and soybeans. As would be expected, the proportions for revenue per acre tend to fall between the proportions for corn and soybean net advisory prices and show a similar pattern of variation across the alternative benchmarks in a given year.  Combined corn and soybean performance for the entire four-year period varies little across the benchmarks, with the proportion of services above the benchmark ranging from 0.55 to 0.62.  Four-year proportions are significantly different from 0.5 for the 20-month and 16-month benchmark, but not the 24-month benchmark.

            Results for the average return test of pricing performance are reported in Tables 8, 9 and 10.  Individual year and four-year average test results for corn, shown in Table 8, are qualitatively similar to the proportional test results. Point estimates of the four-year average return range from –0.26 to 1.54 percent.  However, none of the four-year average returns for the three benchmarks are significantly different from zero.  Individual year and four-year average results for soybeans, reported in Table 9, also are qualitatively similar to the proportional test results.  Point estimates of the four-year average soybean return range from 2.17 to 3.00 percent, substantially higher than for corn. All three of the four-year average soybean returns are significantly different from zero.  Results of the average return test for combined corn and soybean revenue, found in Table 10, also differ little from the proportional test results.  Point estimates of the four-year average revenue return range from 0.90 to 2.08 percent, which, as expected, is between the ranges for corn and soybeans.  Four-year average revenue returns are significantly different from zero for the 20-month and 16-month benchmarks, but not the 24-month benchmark.

            In statistical terms, the pricing performance test results presented in this section are fairly clear.  Minimal evidence is found that market advisory services consistently and significantly “beat the market” in corn.  There is substantial evidence that market advisory services consistently and significantly “beat the market” in soybeans.  When corn and soybean net advisory prices are combined into revenue per acre, evidence also is found that market advisory services significantly outperform the market.  Overall, the statistical results suggest that market advisory services have some ability to outperform broad market benchmarks.

            Given the statistical results summarized above, a relevant question to ask is whether the pricing performance of advisory services also is economically significant.  While "economic significance" is a vague concept, it is important nonetheless.  Perhaps the best perspective on this question is gained by re-examining returns for corn and soybean revenue per acre.  Given the sensitivity of measured returns to the benchmark considered, the best point estimate of revenue returns probably is the simple average across the three benchmarks.  This “grand average” revenue return across all four crop years and three benchmarks is 1.4 percent, which translates into about $4 per acre above benchmark revenue.[14]  While this level of return is probably best characterized as “small,” it also appears to be non-trivial, particularly in comparison to the cost of the services.  Good, Irwin, Jackson, Jirik and Martines-Filho (2000) report that the average cost of the services is $295 for the 1998 crop year. For a 1,000 acre corn and soybean farm, this translates into an average cost of about 30 cents per acre.  Put in different terms, this is roughly equal to the average 50/50 revenue from one acre of corn and soybeans over 1995-1998. There are two important reasons to be cautious about concluding that advisory returns generate even a "small" level of economic significance: i) the results are based on a limited sample of years, and ii) returns tend to be concentrated in one market, soybeans.

            The results of the analysis have implications for the ongoing debate about market efficiency and marketing strategies in agriculture.  One view is that grain markets (cash, futures and options) are not efficient and, therefore, provide opportunities for farmers to systematically earn additional profits through marketing (e.g Wisner, Blue and Baldwin, 1998).  The other view is that grain markets are at least efficient with respect to the type of strategies available to farmers (e.g.,, Zulauf and Irwin, 1998).  Since the return of advisory services over 1995-1998 significantly exceeds transactions costs in several cases, including the cost of the services, the results potentially imply a rejection of market efficiency in the sense of Grossman and Stiglitz (1980).[15]  A firm conclusion cannot be reached due to the uncertainties pointed out with respect to economic significance.  In addition, there is uncertainty about the appropriate adjustment for risk or a complete accounting for the costs of implementing advisory service recommendations.  It may be that important costs are ignored, such as search costs, monitoring costs and related management costs.  Nevertheless, the performance results suggest market advisory services, at least to a modest extent, have some access to information not available to other market participants and/or superior analytical skills.

            Finally, it is interesting to compare the pricing performance results for market advisory services to that of other investment professionals.  Malkiel (1999) reports that only 33 percent of active mutual fund managers beat the returns to the S&P 500 stock index over 1974-1998.  Clements (1999) notes that only 9 percent of active managers beat the S&P 500 in the decade ending in 1998. By comparison, the performance of agricultural market advisory services is quite strong, with a little more than half of the services beating the market in corn and about two-thirds beating the market in soybeans.  This divergence may simply reflect a unique time period in corn and soybean markets, relatively less efficient commodity markets, the skillfulness of advisory services, or an inappropriate adjustment for advisory service risk. 

            Even if, as a group, advisory services generate positive returns, there is a wide range in performance for any given year.  For example, soybean net advisory prices for 1995 vary from $5.71 per bushel to $7.94 per bushel (see Table 2).  While this example probably is the most dramatic, the variation across advisors in other cases is substantial.  This raises the important question of the predictability of advisory service performance from year-to-year.  In other words, is past performance indicative of future results?  Three tests of predictability are used: i) the correlation of advisory service prices, ranks and percentage differences from the benchmark across overlapping and non-overlapping pairs of adjacent crop years, ii) predictability of “winner” and “loser” categories across overlapping and non-overlapping pairs of adjacent crop years and iii) differences between prices, ranks and percentage differences from the benchmark for “top” and “bottom” performing advisory services across overlapping and non-overlapping pairs of adjacent crop years. The testing procedures have been widely applied in studies of financial investment performance (e.g., Elton, Gruber, and Rentzler, 1987; Irwin, Zulauf and Ward, 1994; Lakonishok, Shleifer and Vishny, 1992; Malkiel, 1995). 

            The distinction between overlapping and non-overlapping market years is due to the fact that each marketing window is two calendar years in length, and hence, two adjacent marketing windows overlap by one calendar year.  This overlap may influence predictability results, in that persistence between overlapping years may be due to “true” persistence in performance or the overlapping nature of the periods of comparison.  Persistence for non-overlapping years presumably reflects only “true” persistence in pricing performance. 

            The first test of predictability is based on the correlation between performance measures of individual market advisory services across overlapping and non-overlapping pairs of crop years.  Brorsen and Townsend (1998) show that this type of test is reasonably powerful in detecting performance persistence in managed futures funds if it exists. For a given commodity, the first step in this testing procedure is to form the sample of all advisory services that are active in both adjacent years (overlapping or non-overlapping).  The second step is to rank each advisory service in the first year of the pair (e.g., t = 1997) based on net price received. Then the services are sorted in descending order.  For example, the service with the highest net advisory price is ranked number one, and the service with the lowest net advisory price is assigned a rank equal to the total number of services for that commodity in the given year.  The third step is to sort and rank the sample of services in the second year of the pair (e.g., t + 1  = 1998). The fourth step is to estimate the correlation coefficient between performance measures for the two adjacent crop years t and t+1 as follows,

where  is the sample average of net advisory prices for year t,  is the sample average of net advisory ranks for years t and  is the sample averages of net advisory percentage differences from the market benchmark for years t. Finally, using Bartlett’s approximation for the standard error () of the correlation coefficient, the following test statistic is used to test the null hypothesis of no predictability across the adjacent pair of years,

            The second test of predictability is based on placing advisory services into “winner” and “loser” categories across overlapping and non-overlapping pairs of adjacent crop years. The resulting 2 x 2 contingency table of winner and loser counts allows the use of non-parametric statistical testing procedures.  Carpenter and Lynch (1999) indicate this test is well-specified and powerful in detecting persistence in mutual fund returns. For a given commodity, the first step in this testing procedure is to form the sample of all advisory services that are active in both adjacent years (overlapping or non-overlapping).  The second step is to rank each advisory service in the first year of the pair (e.g., t = 1997) based on net price received. Then the services are sorted in descending order. The third step is to form two groups of services in the first year of the pair: winners are those services in the top half of the rankings and losers are services in the bottom half. The third step is to rank each advisory service in the second year of the pair (e.g.,t +1 = 1998) based on net price received and once again form winner” and loser groups of services. The fourth step is to compute the following counts for the advisory services in the pair of years: WW = winner t-winner t+1, WL = winner t-loser t+1, LW = loser t-winner t+1, LL = loser t-loser t+1. The fifth step is to compute the following odds ratio,

which estimates the ratio of the odds of a winning service in t being a winning service in t+1 to the odds of a losing service in t being a winning service in t+1. The null hypothesis of no predictability is true when the odds ratio equals one.  Christenson (1997) notes that it is more convenient mathematically to test the equivalent null hypothesis that the natural logarithm of the odds ratio equals zero.  In this case, the test statistic is,

where

            The sampling distribution of the test statistic Z_t,t+1 asymptotically follows a standard, normal distribution

            The third test of predictability is based on the differences between prices, ranks and percentage differences from the benchmark for “top” and “bottom” performing advisory services across overlapping and non-overlapping pairs of adjacent crop years. This test is based on the observation that predictability in advisory service performance may not exist across all advisory services, but it is possible that sub-groups of advisory services may exhibit predictability.  In particular, predictability may only be found at the extremes of performance.  That is, only top-performing services in one year may tend to perform well in the next year, or only poor-performing services may perform poorly in the next year. Carpenter and Lynch (1999) indicate this type of test also is well-specified and powerful in detecting persistence in mutual fund returns. It is also robust to the presence of survivorship bias in returns.

            For a given commodity, the first step in this testing procedure is to sort services by pricing performance in the first year of the pair and group services by quantiles (thirds and fourths).  The second step is to compute the average pricing performance for the quantiles formed in the first year of the pair in the second year of the pair. For example, the pricing performance of the top fourth quantile formed in 1995 is computed for 1996. The third step is to compute the following differences in pricing performance for the top- and bottom-performing quantiles,

where  and  are the average net advisory prices for the top and bottom quantiles (thirds or fourths) formed in year t and tracked in year t+1, respectively,   and  are the average net advisory ranks for the top and bottom quantiles (thirds or fourths) formed in year t and tracked in year t+1, respectively,  and  and  are the average net advisory returns for the top and bottom quantiles (thirds or fourths) formed in year t and tracked in year t+1, respectively. The fourth step is to estimate the mean and standard deviation of the above differences across all possible pairs of years.  Finally, the following test statistic can be used to test the null hypothesis of no predictability,

where  is the mean estimate across the possible pairs of years,  is the standard deviation estimate across the possible pairs of years and .  In the case of overlapping crop years,  since there are three pairs of years (1995/1996, 1996/1997, 1997/1998).  In the case of non-overlapping crop years,  since there are two pairs of years (1995/1997, 1996/1998). 

            Results of the test of predictability based on the correlation between performance measures of individual market advisory services across overlapping pairs of crop years are presented in Table 11.[16]  Figure 2 presents a graphical illustration of the rank correlation across crop years for corn.  Figure 3 shows the same relationships for soybeans, and Figure 4 for revenue.  Turning to corn, correlation coefficients for 1995 vs. 1996 and 1997 vs. 1998 generally are in the range of 0.50 to 0.65 for all three performance measures.  Five of the six correlation coefficients for these pairs of years are significantly different from zero. In contrast, each of the three correlations estimated for 1996 vs. 1997 is moderately negative and insignificant. The net result is a small average correlation coefficient across the three pairs of years, about 0.25 to 0.30.  These comparisons suggest some predictability of pricing performance in corn through time.

            All of the estimated correlation coefficients for soybeans are positive, but only one is significantly different from zero (net price correlation, 1997 vs. 1998).  When averaged across the three pairs of crop years, the correlations are only about 0.20 to 0.25.  This evidence suggests, at best, limited predictability of pricing performance for soybeans. 

            Revenue correlation coefficients for 1995 vs. 1996 and 1997 vs.1998 range widely, from a low of 0.15 to a high of 0.58.  Four of the six correlation coefficients for these pairs of years are significantly different from zero. Each of the three correlations estimated for 1996 vs. 1997 is moderately negative and insignificant. Once again, the net result is a small average correlation coefficient across the three pairs of years.  However, these comparisons do suggest some predictability of revenue performance over time.

            Results of the test of predictability based on the correlation between performance measures of individual market advisory services across non-overlapping pairs of crop years are presented in Table 12. The results for corn differ sharply from those for overlapping years.  Five of the six estimated correlations for corn are negative.  Most striking is the large absolute magnitude and significance of the correlations for 1995 vs. 1997.  These correlations are statistically significant and range between –0.52 and –0.68. The average correlation for the two pairs of non-overlapping years ranges from –0.19 to –0.38. Soybean correlation coefficients for non-overlapping years also tend to be negative, but none are significantly different from zero.  Revenue correlation coefficients for non-overlapping years show a similar pattern to those for corn.  Overall, the non-overlapping results tend to be in the opposite direction of the correlations observed for overlapping years, and suggests correlation of performance through time is quite fragile, in the sense of being sensitive to the nature of the comparisons.

            Results of the “winner” and “loser” predictability test for overlapping crop years are shown in Table 13.  It is worth noting that this test of predictability is not as sensitive to outliers in pricing performance, either positive or negative, as the previous correlation tests.  Hence, it is possible for the results to differ across the two sets of tests.  The winner and loser counts for corn, soybeans and revenue indicate little difference in the odds of a winner or loser in one period being a winner or loser in the subsequent period.  As an example, consider the results for corn in 1996 and 1997.  Of the eleven winners in 1996, six are winners (top half) in 1997 and five are losers (bottom half).  The corresponding odds ratio is 1.44, which indicates that the odds (6/5) of a winning service in 1996 being a winning service in 1997 are only 1.44 times the odds (5/6) of a losing service in 1996 being a winning service in 1997.  The odds ratio for all the cases in Table 13 ranges from 0.44 to 3.06.  None of the odds ratios are significantly different from one.  This evidence provides no indication of predictability of advisory service pricing performance.

            Results of the winner and loser predictability test for non-overlapping crop years are shown in Table 14.  The winner and loser counts for corn are slightly more favorable, given that the odds ratio for 1996 and 1998 is significantly different from one.  However, the pooled results for corn are insignificant.  Soybean results are quite similar to the overlapping case, with no significant odds ratios.  Likewise, revenue results for overlapping comparisons indicate no significant odds ratios.

            Results for the test of predictability based on the difference between pricing performance for “top” and “bottom” performing advisory services across overlapping pairs of adjacent crop years are shown in Table 15.  Nominally there is some evidence that top services outperform bottom services.  In all cases, the average net advisory price for services in the top quantile (thirds or fourths) exceeds the average net advisory price for services in the bottom quantile. This is most evident when comparing average prices for the top fourth and bottom fourth, with net prices for the top group exceeding those of the bottom group by $0.17 and $0.18 per bushel for corn and soybeans, respectively.  Revenue for the top fourth exceeds the revenue of the bottom fourth by an average of $8 per acre. However, t-statistics indicate that none of the positive price premiums for top performers is significantly different from zero, although some of the lack of significance certainly can be attributed to the fact that only three observations are used to compute the test statistics. 

            Results for the test of predictability based on the difference between pricing performance for “top” and “bottom” performing advisory services across non-overlapping pairs of adjacent crop years are shown in Table 16.  These results tend to be just the opposite of those observed for overlapping years. In all cases, the average net advisory price for services in the top quantile (thirds or fourths) is below the average net advisory price for services in the bottom quantile. For example, net prices for the top fourth of services in corn and soybeans, on average, are $0.12 and $0.13 per bushel, respectively, less than the comparable average prices for bottom fourth services.  Revenue for the top fourth is below the revenue of the bottom fourth by an average of $12 per acre. Once again, t-statistics indicate that none of the negative premiums for top performers is significantly different from zero. It is worth noting that all of the top third and top fourth quantiles generate average returns that are substantially negative, so these “top” services not only trail bottom performers, but also the market benchmark.

            The practical implications of the contrary top- and bottom-performer results (at least nominally) for overlapping versus non-overlapping years are striking.  Consider the case of a farmer who uses 1995 performance results to select a top-fourth advisory service.  As shown in Table A1 in the Appendix, the 1995 and 1996 comparisons suggest that services in the top fourth outperform services in the bottom fourth by $0.17 per bushel.  However, since the 1995 marketing window ends on August 31, 1996, halfway through the 1996 marketing window and one day before the beginning of the 1997 marketing window, the farmer could fully implement their choice of advisory service only for the 1997 crop. The comparisons in Table A10 show that top-performing advisory services in 1995 tend to be the bottom-performing services in 1997, just the opposite of what the farmer expected. In fact, bottom-performing services outperform top-performing services in 1997 by $0.29 per bushel. Similar results tend to be found for other years and for soybeans and revenue.

            Overall, the test results presented in this section provide little evidence that future advisory service performance can be usefully predicted from past performance.  Most test results show no statistically significant predictability.  When predictability is found, it is sensitive the nature of the comparisons (overlapping versus non-overlapping crop years) and statistical test considered. The previous conclusion does not mean it is impossible to predict advisory service performance. There may be other variables associated with performance that can be used for prediction.  For example, Chevalier and Ellison (1999) study whether mutual fund performance is related to characteristics of fund managers that indicate ability, knowledge or effort, and find that managers who attended higher-SAT undergraduate institutions generate systematically higher returns.  Barber and Odean (2000) examine the trading records of individual stock investors and report that frequent trading substantially depresses investment returns.  Similar factors, such as education of advisors, cash only services versus futures and options services, frequency of futures and options trading, or storage costs, may be useful in predicting the performance of agricultural market advisory services.

            Farmers view market advisory services as a significant source of market information and advice in their quest to manage price risks associated with grain marketing.  Given the high value that farmers place upon market advisory services, it is somewhat surprising that only two academic studies investigate the pricing performance of advisory services.  The lack of studies is most likely due to the difficulty in obtaining data on the stream of recommendations provided by services.

            In 1994, the Agricultural Market Advisory Service (AgMAS) Project was initiated, with the goal of providing unbiased and rigorous evaluation of market advisory services for crop farmers.  The AgMAS Project has collected marketing recommendations for about 25 market advisory services each crop year.  The Project subscribes to all of the services that are followed, and as a result, "real-time" recommendations are obtained.  This prevents the data from being subject to survivorship and hindsight biases.

            The purpose of this paper is to address two basic performance questions for corn and soybeans using the net price received reported by the AgMAS Project for the 1995, 1996, 1997 and 1998 crop years.  The two basic questions are: 1) Do market advisory services, on average, outperform an appropriate market benchmark? and 2) Do market advisory services exhibit persistence in their performance from year-to-year?  At least 21 advisory services are included in the evaluations for each commodity and crop year.  While the sample of advisory services is non-random, it is constructed to be generally representative of the majority of advisory services available to farmers.  The tests used to determine average performance of market advisory services and predictability of performance through time have been widely applied in the financial literature.

            Tests of pricing performance relative to a market benchmark are based on the proportion of services exceeding the benchmark price and the average percentage difference between the net price of services and the benchmark price.  In statistical terms, the pricing performance test results provide little evidence that market advisory services consistently and significantly “beat the market” in corn.  There is substantial evidence that market advisory services consistently and significantly “beat the market” in soybeans.  When corn and soybean net advisory prices are combined into revenue per acre, some evidence also is found that market advisory services significantly outperform the market.  Overall, the statistical results suggest that market advisory services have some ability to outperform broad market benchmarks.

            It is debatable whether the performance of advisory services also is economically significant. Perhaps the best perspective on this question is gained by examining returns for corn and soybean revenue per acre.  For all three crop years, returns averaged 1.4 percent above benchmark revenue, which translates into about $4 per acre.  While this level of return is probably best characterized as “small,” it also appears to be non-trivial, particularly in comparison to the cost of the services.  However, there are two important reasons to be cautious about concluding that advisory returns generate even a "small" level of economic significance: i) the results are based on a small sample of years, and ii) returns are concentrated in only one market, soybeans.

            Three tests of predictability are used and, in general, the they provide little evidence that advisory service pricing performance can be predicted from year-to-year.  The average correlation coefficient relating performance from one year to the next generally is insignificantly different from zero.  Winner and loser counts for corn, soybeans and revenue indicate little difference in the odds of a winner or loser in one period being a winner or loser in the subsequent period.  Finally, average performance of top-performing services is insignificantly different from that of bottom-performing services.

            In conclusion, the results of this study suggest that market advisory services exhibited some ability to "beat the market" for the 1995 through 1998 corn and soybean crops.  Possible explanations for this result include: i) a unique time period in corn and soybean markets, ii) inefficient commodity markets, iii) the skillfulness of advisory services or iv) a return to risk.  Determining which explanation is correct will be an important subject for future research as more data on market advisory service performance becomes available. 

Anderson, D.R., D.J. Sweeney and T.A. Williams.  Statistics for Business and Economics, Sixth Edition.  West Publishing Company:  Minneapolis/St. Paul, 1996.

Barber, B.M. and T. Odean."  "Trading is Hazardous to Your Wealth: The Common Stock Investment Performance of Individual Investors."  Journal of Finance 55(2000):773-806.

Bodie, Z., A. Kane, and A.J. Marcus.  Investments.  Irwin: Homewood, IL, 1989.

Brorsen, B.W. and J. Townsend.  "Performance Persistence for Managed Futures." Proceedings of the 1998 NCR‑134 Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management. Department of Agricultural Economics, Kansas State University, pp. 337-354.

Brown, S. J., W. Goetzmann, R.G.Ibbotson, and S.A.Ross. "Survivorship Bias in Performance Studies." Review of Financial Studies 5(1992):553-580.

Carpenter, J.N. and A.W. Lynch. “ Survivorship Bias and Attrition Effects in Measures of Performance Persistence.”  Journal of Financial Economics 54(1999):337-374.

Chevalier, J. and G. Ellison.  "Are Some Mutual Fund Managers Better Than Others? Cross-Sectional Patterns in Behavior and Performance."  Journal of Finance 54(1999):875-899.

Christensen, R.  Log-Linear Models and Logistic Regression, Second Edition.  New York:  Springer-Verlag, 1997.

Clements, J.  "Debunking Some Mutual-Fund Myths."  The Wall Street Journal, March 16, 1999.

Coble, K.H., G.F. Patrick, T.O. Knight, and A.E. Baquet. “Crop Producer Risk Management Survey:  A Preliminary Summary of Selected Data.”  Information Report 99-001, Department of Agricultural Economics, Mississippi State University, September 1999.

E. J. Elton, M. J. Gruber, and J. C. Rentzler. "Professionally Managed, Publicly Traded Commodity Funds." Journal of Business 60(1987):175-199.

Fama, E. "Efficient Capital Markets: A Review of Theory and Empirical Work." Journal of Finance 30(1970):1043-1053.

Friend, I., M. Blume and J. Crockett. Mutual Funds and Other Institutional Investors. New York: McGraw-Hill, 1970.

Gehrt, D. and Good.  "Evaluation of Market Advisory Services for Corn and Soybeans."  Journal of the American Society of Farm Managers and Rural Appraisers 57(1993):1-7.

Good, D.L., S.H. Irwin, T.E. Jackson, M.A. Jirik and J. Martines-Filho “1998 Pricing Performance of Market Advisory Services for Corn and Soybeans.”  AgMAS Project Research Report 2000-01, February 2000.

Good, D.L., S.H. Irwin, T.E. Jackson, and G.K Price. "1995 Pricing Performance of Market Advisory Services for Corn and Soybeans."  AgMAS Project Research Report 1997-01, March 1997.

Good, D.L., T.E. Jackson, and S.H. Irwin. "Development of a Market Benchmark for AgMAS Performance Evaluations." AgMAS Project Research Report 1998-02, December 1998.

Grossman, S. and J. E. Stiglitz. "On the Impossibility of Informationally Efficient Markets." The American Economic Review 70(1980):393-408,.

Irwin, S.H., C.R. Zulauf, and B.L. Ward.  "The Predictability of Managed Futures Returns." Journal of Derivatives  2(1994): 20-27.

Irwin, S.H., T.E. Jackson and D.L. Good. "Do Agricultural Market Advisory Services Beat the Market? Evidence from the Corn and Soybean Markets Over 1995-1997." AgMAS Project Research Report 1999-03, October 1999.

Jackson, T.E., S.H. Irwin, and D.L. Good.  "1996 Pricing Performance of Market Advisory Services for Corn and Soybeans."  AgMAS Project Research Report 1998-01, January 1998.

Jackson, T.E., S.H. Irwin, and D.L. Good.  "1997 Pricing Performance of Market Advisory Services for Corn and Soybeans." AgMAS Project Research Report 1999-01, February 1999.

Jaffe, J.F.  and J.M. Mahoney.  "The Performance of Investment Newsletters." Journal of Financial Economics 53(1999):289-307.

Kastens, T.L. and T.C. Schroeder.  "Efficiency Tests of July Kansas City Wheat Futures." Journal of Agricultural and Resource Economics  21(1996):187-198.

King, R.P., L.S. Lev and W.E. Nefstad.  "A Position Report for Farm-Level Marketing Management."  Review of Agricultural Economics 17(1995):205-212.

Lakonishok, J., A. Shleifer, and R.W. Vishny.  "The Structure and Performance of the Money Management Industry."  Brookings Papers:  Microeconomics  (1992):339-391.

Malkiel, B.G. “Returns from Investing in Equity Mutual Funds 1971 to 1991.” Journal of Finance 50(1995):549-572.

Malkiel, B.G. A Random Walk Down Wall Street. New York: W.W. Norton and Company, 1999.

Marcus, A.J.  “The Magellan Fund and Market Efficiency.”  Journal of Portfolio Management  16(1990):85-88.

Marten, J. "Farmers Want Market News, Not Advice."  Farm Journal Extra, June 1994.

Martines-Filho, J.G.  Pre-Harvest Marketing Strategies for Corn and Soybeans:  A Comparison of Optimal Hedging Models and Market Advisory Service Recommendations.  Unpublished Ph.D. dissertation, The Ohio State University, 1996.

Metrick, A.  "Performance Evaluation with Transactions Data:  The Stock Selection of Investment Newsletters."  Journal of Finance 54(1999):1743-1776.

Norvell, J. M.  and D. H.  Lattz.  “Value-Added Crops, GPS Technology and Consultant Survey: Summary of a 1998 Survey to Illinois Farmers.”  Working Paper, College of Agricultural, Consumer, and Environmental Sciences, University of Illinois, July 1999.

Otte, J. "Marketing Matters -- How Well Do Market Advisors Deliver?"  Prairie Farmer, July 19, 1986.

Patrick, G.F. and S. Ullerich.  "Information Sources and Risk Attitudes of Large-Scale Farmers, Farm Managers, and Agricultural Bankers."  Agribusiness 12(1996):461-471.

Patrick, G.F., W.N. Musser, and D.T. Eckman.  "Forward Marketing Practices and Attitudes of Large-Scale Midwestern Grain Farmers."  Review of Agricultural Economics 20(1998):38-53.

Powers, L.  "How to Measure Your Pro's Performance."  Top Farmer, April 1993, p. 17.

Ritter, J.R.  "The Long-Run Performance of Initial Public Offerings."  Journal of Finance 46(1991):3-27.

Schroeder, T.C., J.L. Parcell, T.L. Kastens, and K.C. Dhuyvetter.  "Perceptions of Marketing Strategies; Farmers vs. Extension Economists.  Journal of Agricultural and Resource Economics  23(1998):279-293.

Wisner, R.N., E.N. Blue and E.D. Baldwin.  “Preharvest Marketing Strategies Increase Net Returns for Corn and Soybean Growers.” Review of Agricultural Economics. 20(1998):288-307.

Zulauf, C.R. and S.H. Irwin.  “Market Efficiency and Marketing to Enhance Income of Crop Farmers.” Review of Agricultural Economics 20(1998):308-331.

Endnotes

Click here for Order Form