farmdoc : AgMAS: Report 2002-01:The Pricing Performance of Market Advisory Services In Corn and Soybeans Over 1995-2000

Report 2002-01:The Pricing Performance of Market Advisory Services In Corn and Soybeans Over 1995-2000

Copyright 2002 by Scott H. Irwin, Joao Martines-Filho and Darrel L. Good. All rights reserved. Readers may make verbatim copies of this document for non-commercial purposes by any means, provided that this copyright notice appears on all such copies.

The advisory service marketing recommendations used in this research represent the best efforts of the AgMAS Project staff to accurately and fairly interpret the information made available by each advisory service. In cases where a recommendation is vague or unclear, some judgment is exercised as to whether or not to include that particular recommendation or how to implement the recommendation. Given that some recommendations are subject to interpretation, the possibility is acknowledged that the AgMAS track record of recommendations for a given program may differ from that stated by the advisory service, or from that recorded by another subscriber. In addition, the net advisory prices presented in this report may differ substantially from those computed by an advisory service or another subscriber due to differences in simulation assumptions, particularly with respect to the geographic location of production, cash and forward contract prices, expected and actual yields, storage charges and government programs.

This material is based upon work supported by the Cooperative State Research, Education and Extension Service, U.S. Department of Agriculture, under Project Nos. 98-EXCA-3-0606 and 00-52101-9626. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the view of the U.S. Department of Agriculture

Farmers in the US consistently identify price and income risk as one of the greatest management challenges they face. The roller coaster movement of corn and soybean prices over the last decade is ample evidence of the uncertainty and risk facing grain farmers. Surveys suggest that numerous farmers view market advisory services as an important tool in managing price and income risk (e.g., Patrick and Ullerich, 1996; Patrick, Musser, and Eckman; 1998; Schroeder, Parcell, Kastens, and Dhuyvetter, 1998; Norvell and Lattz, 1999; Pennings, Irwin, Good and Gomez, 2001). As a result, farmers need information on the performance “track record” of market advisory services to help them identify successful alternatives for marketing and price risk management.

Given the high value that farmers place upon market advisory services, it is somewhat surprising that only two published studies investigate the pricing performance of advisory services. [2] Gehrt and Good (1993) analyze the performance of five advisory services for corn and soybeans over the 1985 through 1989 crop years. [3] Assuming a representative farmer follows the hedging and cash market recommendations for each advisory service; a net price received for each year is computed and compared to a benchmark price. They generally find that corn and soybean farmers obtained a higher price by following the marketing recommendations of advisory services. Martines-Filho (1996) examines the pre-harvest corn and soybean marketing recommendations of six market advisory services over 1991 through 1994. He computes the harvest time revenue that results from a representative farmer following the pre-harvest futures and options hedging recommendations and selling 100% of production at harvest. Average advisory service revenue over the four years is larger than benchmark revenue for both corn and soybeans.

While a useful starting point, the two published studies have important limitations. First, the cross-section of advisory services tracked for each crop year is quite small, with the largest sample including only six advisory services. Second, the results may be subject to survivorship bias, a consequence of tracking only advisory services that remain in business at the end of a sample period. The literature on the performance of mutual funds provides ample evidence of the upward bias in performance results that can result from survivorship bias (e.g., Brown, Goetzmann, Ibbotson and Ross, 1992; Carpenter and Lynch, 1999). Third, the results may be subject to hindsight bias because advisory service recommendations were not collected on a “real-time” basis (Jaffe and Mahoney, 1999). Hindsight bias is the tendency to collect or record profitable recommendations and ignore or minimize unprofitable recommendations after the fact.

This discussion suggests the academic literature provides farmers with little basis for evaluating the performance of market advisory services. The Agricultural Market Advisory Service (AgMAS) Project was initiated in 1994 with the goal of providing unbiased and rigorous evaluation of market advisory services. [4] ^{, [5]} The AgMAS Project has collected marketing recommendations for no fewer than 23 market advisory programs each crop year since the project was initiated. While the sample of advisory services is non-random, it is constructed to be generally representative of the majority of advisory services offered to farmers. Further, the sample of advisory services includes all programs tracked by the AgMAS Project over the study period, so pricing performance results should not be plagued by survivorship bias. Finally, the AgMAS Project subscribes to all of the services that are followed and records recommendations on a real-time basis. This should prevent the pricing performance results from being subject to hindsight bias.

The purpose of this research report is to evaluate the pricing performance of market advisory services for the 1995-2000 corn and soybean crops. The results for 1995-1999 were released in earlier AgMAS research reports (e.g., Martines-Filho, Irwin and Good, 2000), while results for the 2000 crop year are new. Certain explicit assumptions are made to produce a consistent and comparable set of results across the different advisory programs. These assumptions are intended to accurately depict “real-world” marketing conditions. Several key assumptions are: i) with a few exceptions, the marketing window for a crop year runs from September before harvest through August after harvest, ii) cash prices and yields refer to a central Illinois farm, iii) storage is assumed to occur at on-farm or commercial sites, and iv) marketing loan recommendations made by advisory programs are followed wherever feasible. Based on these assumptions, the net price received by a subscriber to a market advisory program is calculated for the 1995-2000 corn and soybean crops.

Four basic indicators of performance are applied to advisory program prices and revenues over 1995-2000. The first indicator is the proportion of advisory programs that beat benchmark prices. The second indicator is the difference between the average price of advisory programs and benchmarks. The third indicator is the average price and risk of advisory programs relative to the average price and risk of benchmarks. The fourth indicator is the predictability of advisory program performance from year-to-year. Both market and farmer benchmarks are developed for the evaluations. All benchmarks are computed using the same assumptions applied to advisory service track records.

At the outset, it is important to point out that only six crop years are available to analyze market advisory service pricing performance. From a purely statistical standpoint, samples with ten or fewer observations typically are considered “sparse.” On the surface, this suggests the sample may not contain enough information to draw conclusions about advisory service pricing performance. There are several reasons why this may not be the case. First, Anderson (1974) explored the reliability of agricultural return-risk estimates based on sparse data sets and found the surprising result that even as few as three or four observations can be very useful. Second, even though the number of crop years is limited, at least 23 advisory programs are tracked for each crop year. This has the potential to substantially increase the information provided by the sample. Third, from a practical, decision-making standpoint, samples with six observations often are considered adequate to reach conclusions. The results of university crop yield trials represent a well-known example. A typical presentation of the results includes only current year yields and two-year or three-year averages. In many cases, even the two-year and three-year averages cannot be presented because of turnover in the varieties tested from year-to-year. [6] Despite the limitations, this type of yield trial data is widely used by farmers in making variety selections. On balance, then, it seems reasonable to argue that the six years of data currently available on advisory service pricing performance may be used to make some modest conclusions. Caution obviously is in order given the possibility of results being due to random chance in a relatively small sample of crop years.

This report has been reviewed by the AgMAS Review Panel, which provides independent, peer-review of AgMAS Project research. The members of this panel are: Frank Beurskens, Director of Product Strategy for e-markets; Jeffrey A. Brunoehler, Market President of the AMCORE Bank in Mendota, Illinois; Renny Ehler, farmer in Champaign County, Illinois; Chris Hurt, Professor in the Department of Agricultural Economics at Purdue University; Terry Kastens, Associate Professor in the Department of Agricultural Economics at Kansas State University and farmer in Rawlins County, Kansas; and Robert Wisner, University Professor in the Department of Economics at Iowa State University.

The next section of the report describes the procedures used to collect the data on market advisory service recommendations. The second section describes the methods and assumptions used to calculate the returns to advisory service marketing advice. The third section presents the methods and assumptions used to compute benchmark prices, which are used to evaluate advisory service pricing performance. The fourth section of the report presents 2000 pricing results for corn and soybeans. The fifth section presents a summary of the combined results for the 1995-2000 crop years. The sixth section discusses the performance evaluation results for 1995-2000. The final section presents a summary and conclusions.

The market advisory services included in this evaluation do not comprise the population of market advisory services available to farmers. The included services also are not a random sample of the population of market advisory services. Neither approach is feasible because no public agency or trade group assembles a list of advisory services that could be considered the "population." Furthermore, there is not a generally agreed upon definition of an agricultural market advisory service. To assemble the sample of services for the AgMAS Project, criteria were developed to define an agricultural market advisory service and a list of services was assembled.

To date, four criteria have been used to determine which advisory services are included in the AgMAS study. First, marketing recommendations from an advisory service must be received electronically in real time. The recommendations may come in the form of satellite-delivered pages, Internet web pages or e-mail messages. Services delivered electronically generally ensure that recommendations are made available to the AgMAS Project at the same time as farm subscribers. This form of delivery also ensures that recommendations are received in “real-time.” This avoids the problem of recommendations being delivered after the date of implementation intended by an advisory service. Such a problem could occur frequently with recommendations delivered via the postal service.

The second criterion used to identify services is that a service has to provide marketing recommendations to farmers rather than (or in addition to) speculators or “traders.” Some of the services tracked by the AgMAS Project do provide speculative trading advice, but that advice must be clearly differentiated from marketing advice to farmers for the service to be included. The terms "speculative" trading of futures and options versus the use of futures and options for "hedging" purposes are used for identification purposes only. A discussion of what types of futures and options trading activities constitute hedging, as opposed to speculating, is not considered in the study.

The third criterion is that marketing recommendations from an advisory service must be in a form suitable for application to a representative farmer. That is, the recommendations have to specify the percentage of the crop involved in each transaction --cash, futures or options-- and the price or date at which each transaction is to be implemented. It is also helpful if advisory services make specific recommendations about implementation of the marketing loan program, but that is not required. Note that some advisory services evaluated by the AgMAS Project do not make any futures and options recommendations, so it is not necessary to make such recommendation to be included in the study. Services that make futures and options hedging recommendations, but fail to clearly state when cash sales should be made, or the amount to be sold, are not considered for inclusion.

The fourth criterion is that advisory services must provide “blanket” or “one-size fits all” marketing recommendations so there is no uncertainty about implementation. While different programs for basic types of subscribers may be tracked for an advisory service (e.g., a cash only program versus a futures and options hedging and cash program), it is not feasible to track services that provide “customized” recommendations for individual clients.

A fifth criterion will be added in the future to address the issue of whether a candidate service is a viable, commercial business. This issue has arisen due to the extremely low cost and ease of distributing information over the Internet, either via e-mail or a website. It is possible for an individual with little actual experience and no paying subscribers to start a “market advisory service” by using the Internet. Hence, there is a need to exclude firms that are not viable commercial concerns. At the same time, any filter in this regard should not be so restrictive that newer and smaller advisory services are excluded from the AgMAS study for an unreasonably long period of time. This same issue is prevalent when evaluating the performance of other types of professional investment advisors, such as commodity trading advisors. In these cases, it is not unusual to screen firms by the length of track record and amount of funds under management. [7] An analogous screen for market advisory services would be based on the length of time the service has provided recommendations and the number of paying subscribers. The specific criterion that will be used is that a candidate advisory service must have provided recommendations to paying subscribers for a minimum of two marketing years before the service can be included in the AgMAS study. This criterion should exclude non-viable services, while at the same time providing a relatively low hurdle for new and legitimate market advisory services.

The original sample of market advisory services was drawn from the list of Premium Services available from the two major agricultural satellite networks, Data Transmission Network (DTN) and FarmDayta, in the summer of 1994. [8] While the list of advisory services available from these networks was by no means exhaustive, it did have the considerable merit of meeting a market test. Presumably, the services offered by the networks were those most in demand by farm subscribers to the networks. In addition, the list of available services was cross-checked with other farm publications to confirm that widely followed advisory firms were included in the sample. It seems reasonable to argue that the resulting sample of services was generally representative of the majority of advisory services available to farmers.

Additions and deletions to the sample of advisory services have occurred over time. Additions largely have been due to the increasing availability of market advisory services via alternative means of electronic delivery, in particular, websites and e-mail. Deletions have occurred for a variety of reasons. A total of 36 and 35 advisory service programs for corn and soybeans, respectively, have been included in the sample at some point in time. Table 1 contains the complete list of advisory programs and includes a brief explanation why each program that was not included for all crop years was added or deleted from the sample. The term “advisory program” is used because several advisory services have more than one distinct marketing program. For example, AgLine by Doane, Brock, Pro Farmer, and Stewart-Peterson Advisory Services each have two distinct marketing programs, Risk Management Group has three distinct marketing programs and AgriVisor has four distinct marketing programs. Allendale provides two distinct programs for corn, but only one for soybeans.

The total number of advisory programs evaluated for the 2000 crop year is 27 for corn and 26 for soybeans. Two new programs were added for the 2000 crop year: Co-Mark and Grain Marketing Plus. One program, Ag Profit by Hjort, was deleted from the sample for the 2000 crop year. This service went out of business at the end of August 2000 without giving any specific recommendations for 2000 corn and soybean crops. Two other programs, Cash Grain and Stewart-Peterson Strictly Cash, were discontinued during the 2000 crop year. However, these programs made specific recommendations for the 2000 corn and soybean crops until being discontinued during Fall 2000 (last recommendations: September 18, 2000 for Cash Grain and October 26, 2000 for Stewart-Peterson Strictly Cash). As will be discussed below, excluding these two programs from the 2000 sample could result in a form of selection bias, particularly if discontinuation is related to poor performance. Including these programs for the 2000 crop year does require an assumption about marketing the cash positions remaining after the discontinuation date. A similar issue has been treated extensively in the literature on the performance of commodity funds and commodity trading advisors (e.g., Elton, Gruber and Rentzler, 1987). In this literature, if a commodity fund or trading advisor is discontinued before the end of a calendar year, some form of benchmark returns are substituted for the missing returns after the discontinuation date. Following this logic, the cash positions for Cash Grain and Stewart-Peterson Strictly Cash that remained after the date of discontinuation were sold using the same strategy as the market benchmarks utilized for this study (the details of the construction of these benchmarks are given in the “Benchmark Prices” section). In effect, this simply means that cash bushels after the date of discontinuation are sold in equal amounts over the remaining days of the 2000 marketing window. Finally, note that any futures or options positions that remain open on the date of discontinuation are closed on that date using settlement futures prices or options premiums. [9]

Three forms of survivorship bias may be potential problems when assembling an advisory program database. Survival bias significantly biases measures of performance upwards since "survivors" typically have higher performance than "non-survivors" (e.g., Brown, Goetzmann, Ibbotson, and Ross, 1992; Carpenter and Lynch, 1999). The first and most direct form of survivorship bias occurs if only advisory programs that remain in business at the end of a given sample period are included in the sample. This form of bias should not be present in the AgMAS database of advisory programs because all programs that have been tracked over the entire time period of the study are included in the sample. The second form of survivorship bias occurs if discontinued advisory programs are deleted from the sample for the year when they are discontinued. This is a form of survivorship bias because only survivors for the full crop year are tracked. The AgMAS database of advisory programs should not be subject to this form of bias because programs discontinued during a crop year remain in the sample for that crop year. As noted above, cash positions remaining after the date of discontinuation are sold using the same strategy as the market benchmarks utilized for this study. The third and most subtle form of survivorship bias occurs if data from prior periods are "back-filled" at the point in time when an advisory program is added to the database. This is a form of survivorship bias because data from surviving advisory programs are back-filled. The AgMAS database should not be subject to this form of bias because recommendations are not back-filled when an advisory program is added. Instead, recommendations are collected only for the crop year after a decision has been made to add an advisory program to the database.

Another important consideration when assembling a database on advisory program recommendations is hindsight bias (Jaffe and Mahoney, 1999). This is the tendency to collect or record profitable recommendations and ignore or minimize unprofitable recommendations after the fact. Since the AgMAS Project subscribes to all of the services that are followed and records recommendations on a real-time basis, the database of recommendations should not be subject to hindsight bias. The information is received electronically, via DTN, website or e-mail. For the programs that provide multiple daily updates, typically in the morning and at noon, information is recorded for all updates. In this way, the actions of a farmer-subscriber are simulated in real-time.

When recording recommendations of each advisory program, specific attention is paid to which year’s crop is being sold, (e.g., 2000 crop year), the amount of the commodity to be sold, which futures or options contract is to be used (where applicable), and any price targets that are mentioned (e.g., sell cash corn when March 2001 futures reaches $2.40). If a price target is given and not immediately filled, such as a stop order in the futures market, the recommendation is noted until the order is either filled or canceled.

Some advisory programs offer two or more distinct marketing programs. This typically takes the form of one set of advice for marketers who are willing to use futures and options (although futures and options are not always used), and a separate set of advice for farmers who only wish to make cash sales. [10] In this situation, both strategies are recorded and treated as distinct strategies to be evaluated. In the past, when a service clearly differentiated strategies based on the availability of on-farm versus off-farm (commercial) storage, only the off-farm storage strategy was tracked. Starting with the 2000 corn and soybean crops, if a service clearly differentiates on-farm and off-farm storage strategies at harvest, both strategies are recorded. [11]

Several procedures are used to check the recorded recommendations for accuracy and completeness. Whenever possible, recorded recommendations are crosschecked against later status reports provided by the relevant advisory program. Also, at the completion of the crop year, it is confirmed whether cash sales total exactly 100%, all futures positions are offset, and all options positions are offset or expire.

The final set of recommendations attributed to each advisory program represents the best efforts of the AgMAS Project staff to accurately and fairly interpret the information made available by each advisory program. In cases where a recommendation is considered vague or unclear, some judgment is exercised as to whether or not to include that particular recommendation. This occurs most often when a program suggests that “a farmer might consider” a position, or when minimal guidance is given as to the quantity to be bought or sold. Given that some recommendations are subject to interpretation, the possibility is acknowledged that the AgMAS track record of recommendations for a given program may differ from that stated by the advisory program, or from that recorded by another subscriber.

At the end of the marketing period, all of the (filled) recommendations are aligned in chronological order. The advice for a given crop year is considered to be complete for each advisory program when cumulative cash sales of the commodity reach 100%, all futures positions covering the crop are offset, all option positions covering the crop are either offset or expire, and the advisory program discontinues giving advice for that crop year. The returns to each recommendation are then calculated in order to arrive at a weighted average net price that would be received by a farmer who precisely follows the marketing advice (as recorded by the AgMAS Project).

In order to produce a consistent and comparable set of results across the different advisory programs, certain explicit assumptions are made. These assumptions are intended to accurately depict “real-world” marketing conditions. Note that discussion in the following sections center on the 2000 crop year. Similar discussion and examples for the 1995-1999 crop years can be found in earlier AgMAS pricing reports (e.g., Martines-Filho, Irwin and Good, 2000).

Geographic Location

The simulation is designed to reflect conditions facing a representative central Illinois corn and soybean farmer. Whenever possible, data are collected for the Central Crop Reporting District in Illinois as defined by the National Agricultural Statistics Service (NASS) of the US Department of Agriculture (USDA). The eleven counties (DeWitt, Logan, McLean, Marshall, Macon, Mason, Menard, Peoria, Stark, Tazewell, and Woodford) that make up this District are highlighted in Figure 1.

Caution should be used when applying the results to other areas of the US, because yields and basis patterns may be quite different from those of central Illinois. The differences in yields and basis patterns could have a substantial impact on prices computed for farmers or advisory services in another area. The resulting change could be either up or down relative to AgMAS advisory prices and benchmarks, depending on local conditions. Appendix A to this report, entitled “A Cautionary Note on the Use of AgMAS Net Advisory Prices and Benchmarks,” contains further discussion on this point.

The time period over which a farmer normally makes pricing decisions for a particular crop is termed the “marketing window.” It also can be referred to as the pricing “decision-horizon” or “timeline” of a farmer. A marketing window does not necessarily equal the time period of observed market activity. The reason is that not taking action (e.g., not hedging pre-harvest) is one type of decision that can be made during a marketing window.

In the present context, the objective is to define the normal marketing window of a representative farmer who subscribes to the advisory programs tracked by the AgMAS Project. Good, Hieronymus and Hinton (1980) provide a useful starting point. They define the marketing window for an Illinois grain farmer as the period extending from the initial production planning time until the end of the storage season. First production decisions in Illinois normally occur in October through November of the year preceding planting (e.g., fall tillage and application of fertilizer), while the storage season typically extends through July or August of the year following harvest. This results in a marketing window between 21 and 23 months in length.

The actual pricing patterns of advisory programs included in the AgMAS study provide helpful empirical evidence for defining the relevant marketing window. As noted earlier, observed market positions cannot directly reveal the intended pricing window of a representative farmer following advisory program recommendations. However, averages over time and advisors should be suggestive as to the typical starting and ending points used to make recommendations for a crop. Figure 2 presents the average “marketing profile” of advisory programs in corn and soybeans over the 1995-1999 crop years. [12] The marketing profiles show the average amount of corn and soybean crops priced (sold) by advisory programs, on a cumulative basis, each day over the two-year period beginning in September of the year before harvest and ending August of the year after harvest. The profiles suggest that a farmer following the recommendations of market advisory programs included in the AgMAS study, on average, will begin making significant marketing decisions (pricing more than one percent) in September of the year before harvest and will not complete marketing until August of the year after harvest. [13]

Overall, this discussion indicates it is reasonable to assume a 24-month marketing window for a representative farmer subscribing to advisory programs. In the case of the 2000 crop, the marketing window is then defined as the two-year period beginning September 1, 1999 and ending on August 31, 2001. Such a specific definition raises the issue of exceptions. For example, one program in corn and two programs in soybeans started their first hedge program for the 2000 crop year in the middle of July 1999. Two other advisory programs had a relatively small amount (20-25%) of cash corn unsold as of August 31, 2001. These bushels were sold in the spot cash market on September 19, 2001. One program maintained relatively large (50% for corn) long call “re-ownership” positions using November soybean options contracts. This position expired worthless on October 19, 2001. Given that the marketing window is defined as the “normal” window, it is argued that a representative farmer would approach the marketing window with some flexibility, particularly for recommendations that do not extend too far outside the limits of the marketing window. Since the transactions in question for the 2000 crop do not extend much outside the limits of the marketing window, they are included in the relevant advisory program’s track record. [14] Finally, note that throughout the remainder of this report, the term "crop year" is used to represent the two-year marketing window.

Prices

The price assigned to each cash sale recommendation is the central Illinois closing, or overnight, bid. The North and South Central Illinois Price Reporting Districts are highlighted in Figure 3. The data are collected and reported by the Illinois Department of Ag Market News. [15] The central Illinois price is the mid-point of the range of bids by elevators in the North Central and South Central Price Reporting Districts, as defined by the Illinois Department of Ag Market News. Prices in this 35-county area best reflect prices for the assumed geographic location of the representative central Illinois farmer (Central Illinois Crop Reporting District).

Pre-harvest cash forward contract prices for fall delivery are also needed. Pre-harvest bids collected by the Illinois Department of Ag Market News are used when available. The central Illinois pre-harvest price is the mid-point of the daily range of pre-harvest bids by elevators in the North Central and South Central Price Reporting Districts, again, as defined by the Illinois Department of Ag Market News. Pre-harvest forward prices from this source are available for corn and soybeans from February 1, 2000 to September 1, 2000.

Since the marketing window for the 2000 corn and soybean crops begins in September 1999, and the Illinois Department of Ag Market News did not begin to report actual cash forward bids until February 1, 2000, pre-harvest prices need to be estimated for the first few months of the marketing window. For a date between September 1, 1999 and January 31, 2000, a two-step estimation procedure is adopted. First, the forward basis for the period in question is estimated by the average forward basis for the first five days the Illinois Department of Ag Market News reports actual forward contract bids (February 1-7, 2000) . [16] Second, the estimated forward basis is added to the settlement price of the Chicago Board of Trade (CBOT) 2000 December corn futures contract or 2000 November soybean futures contract between September 1, 1999 and January 31, 2000. This estimation procedure is expected to be a reasonably accurate reflection of actual forward prices for the early period of the marketing window, as the actual price of the harvest futures contract is used and only the forward basis is estimated. In addition, the estimation procedure is typically applied to a relatively small number of transactions. The average net amount sold before February 1^st over 1995-1999 is only 12% for corn and 10% for soybeans, and many of these transactions are in futures or options contracts rather than forward contracts.

Some market advisory programs recommended the use of post-harvest forward contracts to sell part of the 2000 corn and soybean crops. The Illinois Department of Ag Market News did report post-harvest bids for January 2001 delivery from September 5, 2000 to December 1, 2000. They also report post-harvest bids for March 2001 delivery from December 4, 2000 to January 31, 2001. These bids for central Illinois are used wherever applicable. For the 2000 crop year, forward bids are available to match all advisory program recommendations.

In the future, if the positions recommended by advisory programs either do not match the delivery periods or are made after the Illinois Department of Ag Market News stops reporting post-harvest forward contract prices, the following procedure will be used to estimate the post-harvest forward contract prices needed in the analysis. First, three elevators in central Illinois agreed to supply data on spot and forward contract prices on the dates when advisors made such recommendations. Each of these elevators is in a different county in the Central Illinois Crop Reporting District (Logan, McClean, DeWitt). Second, the spread between each elevator’s forward price and spot price will be calculated for the relevant date. Third, the forward spread will be averaged across the three elevators for the same date. Fourth, the average forward spread from the three elevators will be added to the central Illinois cash price (discussed at the beginning of the section) to arrive at an estimated post-harvest forward contract price for central Illinois. This procedure was used in a few cases for the 1998 and 1999 crop years.

The fill prices for futures and options transactions generally are the prices reported by the programs. In cases where a program did not report a specific fill price, the settlement price for the day is used. This method does not account for liquidity costs in executing futures and options transactions. [17]

Since most of the advisory program recommendations are given in terms of the proportion of total production (e.g., “sell 5% of 2000 crop today”), some assumption must be made about the amount of production to be marketed. For the purposes of this study, if the per-acre yield is assumed to be 100 bushels, then a recommendation to sell 5% of the corn crop translates into selling 5 bushels. When all of the advice for the marketing period has been carried out, the final per-bushel selling price is the average price for each transaction weighted by the amount marketed in each transaction.

The above procedure implicitly assumes that the “lumpiness” of futures and/or options contracts is not an issue. Lumpiness is caused by the fact that futures contracts are for specific amounts, such as 5,000 bushels per CBOT corn futures contract. For large-scale farmers, it is unlikely that this assumption adversely affects the accuracy of the results. This may not be the case for small- to intermediate-scale farmers who are less able to sell in 5,000-bushel increments. [18]

When making hedging or forward contracting decisions prior to harvest, the actual yield is unknown. Hence, an assumption regarding the amount of expected production per acre is necessary to accurately reflect the returns to marketing advice. Prior to harvest, the best estimate of the current year’s expected yield is likely to be a function of yield in previous years. In this study, the assumed yield prior to harvest is the calculated trend yield, while the actual reported yield is used from the harvest period forward. The expected yield for 2000 is based upon a log-linear regression trend model of actual yields from 1972 through 1999 for the Central Illinois Crop Reporting District. Previous research suggests this type of trend model provides a reasonable fit to corn and soybean yield data (Fackler, Young and Carlson, 1993; Zanini, 2001).

In central Illinois, the expected 2000 yield for corn is calculated to be 149 bushels per acre. Therefore, recommendations regarding the marketing quantity made prior to harvest, are based on yields of 149 bushels per acre. For example, a recommendation to forward contract 20% of expected 2000 production translates into a recommendation to contract 29.8 bushels per acre (20% of 149). The actual reported corn yield in central Illinois in 2000 is 159 bushels per acre. The same approach is used for soybean evaluations. The calculated 2000 trend yield for soybeans in central Illinois is 48.5 bushels per acre, and the actual yield in 2000 is 47 bushels per acre.

It is assumed that after harvest begins, farmers have reasonable ideas of what their actual realized yield will be. Since harvest occurs at different dates each year, estimates of harvest progress as reported by NASS in central Illinois are used. Harvest progress estimates typically are not made available soon enough to identify precisely the beginning of harvest, so an estimate is made based upon available data. Specifically, the date on which 50% of the crop is harvested is defined as the mid-point of harvest. The entire harvest period then is defined as a five-week window, beginning two and one-half weeks before the harvest mid-point, and ending two and one-half weeks after the harvest mid-point. In most years, a five-week window will include at least 80% of the harvest.

For 2000, the harvest period for corn is defined as September 8, 2000 through October 12, 2000. For soybeans, the harvest period is September 20, 2000 through October 24, 2000. Therefore, for corn, recommendations made after September 8 are applied on the basis of the actual yield of 159 bushels per acre. For soybeans, recommendations made after September 20 are applied on the basis of the actual yield of 47 bushels per acre.

The issue of changing yield expectations typically is not dealt with in the recommendations of the advisory programs. For the purpose of this study, the actual harvest yield must exactly equal total cash sales of the crop at the end of the marketing time frame. Hence, an adjustment in yield assumptions from expected to actual levels must be applied to cash transactions at some point in time. In this analysis, an adjustment is made in the amount of the first cash sale made after the beginning of the harvest period. For example, if a program advises forward contracting 50% of the corn crop prior to harvest, this translates into sales of 74.5 bushels per acre (50% of 149). However, when the actual yield is applied to the analysis, sales-to-date of 74.5 bushels per acre imply that only 46.85% of the actual crop has been contracted. In order to compensate, the amount of the next cash sale is adjusted to align the amount sold. In this example, if the next cash sale recommendation is for a 10% increment of the 2000 crop, making the total recommended sales 60% of the crop, the recommendation is adjusted to 13.15% of the actual yield (20.91 bushels), so that the total crop sold to date is 60% of 159 bushels per acre (74.5 + 20.9 = 95.4 = 0.6*159). After this initial adjustment, subsequent recommendations are taken as percentages of the 159 bushels per acre actual yield, so that sales of 100% of the crop equal sales of 159 bushels per acre.

While the amount of cash sales is adjusted to reflect the change in yield information, a similar adjustment is not made for futures or options positions that are already in place. For example, assume that a short futures hedge is placed in the December 2000 corn futures contract for 25% of the 2000 crop prior to harvest. Since the amount hedged is based on the trend yield assumption of 149 bushels per acre, the futures position is 37.25 bushels per acre (25% of 149). After the yield assumption is changed, this amount represents a short hedge of 23.4% (37.25/159). The amount of the futures position is not adjusted to move the position to 25% of the new yield figure. However, any futures (or options) positions recommended after the beginning of harvest are implemented as a percentage of the actual yield.

If actual yield is substantially below trend, and forward pricing obligations are based on trend yields, a farmer may have difficulty meeting such obligations. This raises the issue of updating yield expectations in “short” crop years to minimize the chance of defaulting on forward pricing obligations. While not yet encountered in the AgMAS evaluations of corn and soybeans, this situation has arisen in the evaluation of wheat (Jirik, Irwin, Good, Jackson and Martines-Filho, 2000).

As in wheat, a relatively simple procedure will be used to update yield expectations in any future corn or soybean short crop years. First, trend yield will be used as the expected yield until the August USDA Crop Production Report is released, typically around August 10^th. Second, if the USDA corn or soybean yield estimate for the Central Illinois Crop Reporting District is 20% (or more) lower than trend yield, a “reasonable” farmer is assumed to change yield expectations to the lower USDA estimate. Third, as with normal crop years, the adjustment to actual yield is assumed to occur on the first day of harvest.

The 20% threshold is intentionally relatively large for at least three reasons. First, it is desirable to make adjustments to the trend yield expectation on a limited number of occasions. Given the large variability in annual yields, a small threshold could result in frequent adjustments. Second, it is not uncommon for early yield estimates to deviate significantly from the final estimate. A small threshold could result in unnecessary adjustments prior to harvest. Third, yield shortfalls of less than 20% are unlikely to create delivery problems for a farmer.

Brokerage costs are incurred when farmers open or close positions in futures and options markets. For the purposes of this study, it is assumed that brokerage costs are $50 per contract for round-turn futures transactions, and $30 per contract to enter or exit an options position. Further, it is assumed that CBOT corn and soybean futures and options contracts are used, which have a contract size of 5,000 bushels. Therefore, per-bushel brokerage costs are one cent per bushel for a round-turn futures transaction and 0.6¢ per bushel for each options transaction.

LDP and Marketing Assistance Loan Payments

While the 1996 “Freedom-to-Farm” Act did away with government set-aside and target price programs, price protection for farmers in program crops such as corn and soybeans was not eliminated entirely. Minimum prices are established through a “loan” program. Specifically, if market prices are below the Commodity Credit Corporation (CCC) loan rate for corn or soybeans, farmers can receive payments from the US government that make up the difference between the loan rate and the lower market price. [19] There is considerable flexibility in the way the loan program can be implemented by farmers. This flexibility presents the opportunity for advisory programs to make specific recommendations for the implementation of the loan program. Additionally, the prices of both corn and soybeans were below the loan rate during significant periods of time in the 2000-2001 marketing year, so that use of the loan program was an important part of marketing strategies. As a result, net advisory program prices may be substantially impacted by the way the provisions of the loan program are implemented. Finally, all of the advisory programs tracked by the AgMAS project for the 2000 crop year make specific recommendations regarding the timing and method of implementing the loan program for the entire corn and soybean crops.

Before describing the decision rules, it is useful to provide a brief overview of the loan program mechanics. Then, the rules developed to implement the loan program in the absence of specific recommendations can be described more effectively.

Program Mechanics

There are two mechanisms for implementing the price protection benefits of the loan program. The first mechanism is the loan deficiency payment (LDP) program. LDPs are computed as the difference between the loan rate for a given county and the posted county price (PCP) for a particular day. PCPs are computed by the USDA and change each day in order to reflect the average market price that exists in the county. For example, if the county loan rate for corn is $2.00 per bushel and the PCP for a given day is $1.50 per bushel, then the LDP is $0.50 per bushel. If the PCP increases to $1.60 per bushel, the LDP will decrease to $0.40 per bushel. Conversely, if the PCP decreases to $1.40 per bushel, the LDP will increase to $0.60 per bushel. [20]

LDPs are made available to farmers over the period beginning with corn or soybean harvest and ending May 31^st of the calendar year following harvest. Farmers have flexibility with regard to taking the LDP, because they may simply elect to take the payment when the crop is sold in a spot market transaction (before the end of May in the particular marketing year), or choose to take the LDP before the crop is delivered and sold. Note that LDPs cannot be taken after a crop has been delivered and title has changed hands.

The second mechanism is the non-recourse marketing assistance loan program. A loan cannot be taken on any portion of the crop for which an LDP has been received. Under this program, farmers may store the crop (on the farm or commercially), maintain beneficial interest, and receive a loan from the CCC using the stored crop as collateral. The loan rate is the established rate in the county where the crop is stored and the interest rate is established at the time of loan entry. Corn and soybean crops can be placed under loan anytime after the crop is stored through May 31^stof the following calendar year. The loan matures on the last day of the ninth month following the month in which the loan was made.

Farmers may settle outstanding loans in two ways: i) repaying the loan during the 9-month loan period, or ii) forfeiting the crop to the CCC at maturity of the loan. Under the first alternative, the loan repayment rate is the lower of the county loan rate plus accrued interest or the marketing loan repayment rate, which is the PCP. If the PCP is below the county loan rate, the economic incentive is to repay the loan at the posted county price. The difference between the loan rate and the repayment rate is a marketing loan gain (MLG). If the PCP is higher than the loan rate, but lower than the loan rate plus accrued interest, the incentive is also to repay the loan at the PCP. In this case only, interest is charged on the difference between the PCP and the loan rate. If the PCP is higher than the loan rate plus accrued interest, the incentive is to repay the loan at the loan rate plus interest. In this latter case, interest is based on the loan rate.

Under the second alternative, the farmer stores the crop to loan maturity and then transfers title to the CCC. The farmer retains the proceeds from the initial loan. This was generally not an attractive alternative in the 2000 marketing year since the PCP was often below the cash price of corn and soybeans. Repaying the loan at the PCP and selling the crop at the higher cash price was economically superior to forfeiture.

The non-recourse loan program establishes the county loan rate as a minimum price for the farmer, as does the LDP program. For the 2000 crop, the sum of LDPs plus marketing loan gains was subject to a payment limitation of $150,000 per person. Forfeiture on the loans provided the mechanism for receiving a minimum of the loan rate on bushels in excess of the payment limitation.

The average loan rates for the 2000 corn and soybean crops across the eleven counties in the Central Illinois Crop Reporting District are $1.95 and $5.41 per bushel, respectively. Spot cash prices fell below these loan rates for almost all of the 2000 post-harvest period for corn and the entire 2000 post-harvest period for soybeans. This is reflected in Figure 4, which shows corn and soybean LDP or MLG rates for central Illinois during the 2000 post-harvest period. [21] ^{, [22]}For corn, LDPs or MLGs are relatively high during harvest, varying from $0.40 to $0.45 per bushel, and then fall to zero or near zero by the end of calendar year 2000. As cash corn prices fall during the winter and spring of 2001, corn LDP/MLGs increase. Soybean LDPs or MLGs are high during the 2000 harvest time, varying from $0.80 to $1.00 per bushel and decrease to $0.50 at the end of December. During the winter and spring, they increase to almost $1.40 per bushel. As cash soybean prices increase during the summer of 2001, soybean MLGs decrease to $0.20 per bushel at the beginning of July 2001.

Decision Rules for Programs with a Complete Set of Loan Recommendations

If an advisory program makes a complete set of loan recommendations, the specific advice is implemented wherever feasible. However, specific decision rules are still needed regarding pre-harvest forward contracts because it is possible for an advisory program to recommend taking the LDP on those sales before it is actually harvested and available for delivery in central Illinois. To begin, it is assumed that amounts sold for harvest delivery with pre-harvest forward contracts are delivered first during harvest. Since LDPs must be taken when title to the grain changes hands, LDPs are assigned as these “forward contract” quantities are harvested and delivered. This necessitates assumptions regarding the timing and speed of harvest. Earlier it was noted that a five-week harvest window is used to define harvest. This window is centered on the day nearest to the mid-point of harvest progress as reported by NASS. Various assumptions could be implemented regarding harvest progress during this window. Lacking more precise data, a reasonable assumption is that harvest progress for an individual representative farm is a linear function of time.

Tables 2 and 3 summarize the information used to assign LDPs to pre-harvest forward contracts. The second column shows the amount harvested assuming a linear model. The third column shows the LDP available on each date of the harvest window and the fourth column presents the average LDP through each harvest date. An example will help illustrate use of the tables. Assume that an advisory program recommends, at some point before harvest, that a farmer forward contract 50% of expected soybean production. This translates into 24.25 bushels per acre when the percentage is applied to expected production (0.50*48.5 = 24.25). Next, convert the bushels per acre to a percentage of actual production, which is 51.6% (24.25/47 = 0.516). To determine the LDP payment on the 51.6% of actual production forward contracted, simply read down Table 3 to October 6, 2000, which is the date when 51.6% of harvest is assumed to be complete. The average LDP up to that date (September 20, 2000- October 6, 2000) is $0.83 per bushel; the last column of Table 3. This is the LDP amount assigned to the forward contract bushels.

Note that LDPs for any sales (spot, forward contracts, futures or options) recommended during harvest are taken only after all forward contract obligations are fulfilled. In addition, crops placed under loan by an advisory program do not accumulate interest opportunity costs because proceeds from the loan can be used to offset interest costs that otherwise would accumulate.

Decision Rules for Programs with a Partial Set of Loan Recommendations Or No Loan Recommendations

If an advisory program makes a partial set of loan recommendations, the available advice is implemented wherever feasible. In the absence of specific recommendations, it is assumed that crops priced before May 31, 2001 are not placed under loan. Those crops receive program benefits through LDPs. After May 31, 2001, eligible crops (unpriced crops for which program benefits have not yet been collected) are assumed to be under loan until priced.

In the absence of specific recommendations, rules for assigning LDPs and MLGs are developed under the assumption that loan benefits are established when the crop is priced or as soon after pricing that is allowed under the rules of the program. This principle is consistent with the intent of the loan program to fix a minimum price when pricing decisions are made. Two rules are most important in the implementation of this principle. First, LDPs on pre-harvest sales (forward contracts, futures or options) are established as the crop is harvested. Second, if the LDP or MLG is zero on the pricing date, or the first date of eligibility to receive a loan benefit, those values are assigned on the first date when a positive value is observed, assuming a beneficial interest in that portion of the crop has been maintained. Specific rules for particular marketing tools and situations follow:

1) Pre-harvest forward contracts. The same decision rules are applied as discussed in the previous section. Specifically, it is assumed that amounts sold for harvest delivery with pre-harvest forward contracts are delivered first during harvest. LDPs, if positive, are assigned as these “forward contract” quantities are harvested and delivered. This necessitates assumptions regarding the timing and speed of harvest. A linear model of harvest progress is assumed in the five-week harvest window. The specific information used to assign LDPs to pre-harvest forward contracts is again found in Tables 2 and 3. As a final point, note that LDPs for any other sales (spot, futures or options) recommended during harvest are taken only after all pre-harvest forward pricing obligations are fulfilled.

2) Pre-harvest short futures. Pre-harvest pricing using futures contracts is treated in the same manner as pre-harvest forward contracts. LDPs are assigned on open futures positions as the crop is harvested, or as soon as a positive LDP is available, if the futures position is still in place and cash sales have not yet been made. These are assigned after forward contracts have been satisfied. If the underlying crop is sold before there is a positive LDP, then that portion of the crop receives a zero LDP. During the harvest window, if the futures position is offset before a positive LDP is available and the crop has not yet been sold in the cash market, that portion of the crop is eligible for loan benefits on the next pricing recommendation.

3) Pre-harvest put option purchases. Long put option positions, which establish a minimum futures price, are treated in the same manner as pre-harvest short futures.

4) Post-harvest forward contracts. The main issue with respect to post-harvest forward contracts is when to assign the LDPs or MLGs. Those can be established on the date the contract is initiated, on the delivery date of the contract, or anytime in between. Following the general principle outlined earlier, LDPs and MLGs for post-harvest contracts are assigned on the date the contract is initiated or the first day with positive benefits prior to delivery on the contract.

5) Post-harvest short futures. As with post-harvest forward contracts, the main issue with post-harvest short futures positions is when to assign loan benefits. These are assigned when the short futures position is initiated or as soon as a positive benefit is available if the futures position is still in place and cash sales have not been made. If the underlying crop is sold before a positive LDP is available, that portion of the crop receives a zero LDP. If the short futures position is offset before a positive LDP is available and the cash crop has not yet been sold, that portion of the crop is eligible for loan benefits on the next pricing recommendation.

6) Post-harvest long put positions. Long put option positions established after the crop is harvested are treated in the same manner as post-harvest short futures

7) Spot sales before May 31, 2001. If a spot cash sale of corn or soybeans is recommended before May 31, 2001, it is assumed that the LDP, if positive, is established that same day.

8) Loan program after May 31, 2001. Since LDPs are not available after May 31, 2001, it is assumed that any corn or soybeans in storage and not priced as of this date, for which loan benefits have not been established, are entered in the loan program on that date. This is a reasonable assumption since spot prices are below the loan rate for both soybeans and near the loan rate for corn in central Illinois on May 31, 2001 and a prudent farmer would take advantage of the price protection offered by the loan program. When the crops are subsequently priced (cash sale, forward contract, short futures, or long put option), the marketing loan gain, if positive, is assigned on that day. Forfeiture is not an issue for these bushels because all cash sales were made before the end of the nine-month loan period. Note also that the $150,000 payment limitation is not considered in the analysis, as production is based on one acre of corn and/or soybeans.

An important element in assessing returns to an advisory program is the economic cost associated with storing grain instead of selling grain immediately at harvest. The cost of storing grain after harvest consists of two components: physical storage costs and the opportunity cost incurred by foregoing sales when the crop is harvested. Physical storage costs depend on the type of storage available and the horizon used by a farmer to make storage decisions. From a representative farmer’s perspective, there are four relevant physical storage scenarios: i) on-farm storage using a short-run decision-horizon, ii) off-farm (commercial) storage using a short-run decision-horizon, iii) on-farm storage using a long-run decision-horizon and iv) off-farm (commercial) storage using a long-run decision-horizon. Short-run in this context is defined to be one storage season, usually the ten-month period after the harvest of a particular crop. Long-run is defined to be any decision-horizon longer than one storage season. In each of the previous scenarios, the physical storage charge should be the relevant marginal cost of physical storage (Williams and Wright, 1991). In contrast, opportunity cost should be the same regardless of the type of physical storage used or whether a short- or long-run decision-horizon is considered.

Previous AgMAS pricing reports have considered only one scenario: commercial storage using a short-run decision-horizon. Starting with the 2000 crop year, net advisory prices and benchmarks are computed using physical storage costs applicable to each of the four storage scenarios. In all cases, storage and interest charges are assigned beginning October 13, 2000 for corn and October 25, 2000 for soybeans, the first dates after the end of the respective 2000 harvest windows. It should be noted that the cost of drying corn to 15% moisture and the cost of drying soybeans to storable moisture are not included in the calculations. This cost is incurred whether the grain is stored or sold at harvest, or whether the grain is stored on-farm or off-farm. Therefore, this cost is irrelevant to the analysis and excluded.

The first scenario considered is on-farm storage and a short-run decision-horizon. Because pre-existing storage facilities are assumed to be available on-farm, the marginal cost of physical storage equals the on-farm variable cost of physical storage. Estimates of the on-farm variable cost of physical storage are drawn from a recent study conducted at Kansas State University (Dhuyvetter, Hamman and Harner, 2000). The estimates assume storage occurs in a 25,000 bushel round metal bin, the “medium-sized” storage capacity examined in the Kansas State study. The first component of on-farm physical storage is a flat charge of 6.7¢ per bushel for conveyance, aeration, insecticide and repairs. The flat charge is applied to both corn and soybeans and reflects the fact that most physical costs of on-farm storage are “one-time” in nature. That is, once the decision is made to store, most costs are pre-determined and do not vary with the length of storage.

The second component of on-farm physical storage is shrinkage. Corn shrinkage is assumed in the Kansas State study to start at one-percent per bushel for the first month of storage and increase at a rate of one-tenth of one percent for each month stored thereafter. For example, if corn is stored six months, the total shrinkage is assumed to be 1.5% per bushel. Agricultural engineering specialists at the University of Illinois and Purdue University indicated that the on-farm shrink schedule for corn used in the Kansas State study is reasonable. In addition, the schedule is consistent with published research about shrinkage of corn stored on-farm (Hurburgh, Bern, Wilcke and Anderson, 1983). Given that the harvest-time cash price of corn in central Illinois for 2000 is $1.64 per bushel, the shrink charge assigned to corn stored on-farm for one-month is 1.64¢ per bushel ($1.64*0.01*100). The shrink charge is increased 0.16¢ per bushel ($1.64*0.001*100) for each additional month of storage. [23]

Since the Kansas State study did not estimate shrinkage costs for soybeans, the same agricultural engineering specialists noted above were consulted for a reasonable estimate. This turned out to be a constant 0.25% per bushel shrink factor. Given that the harvest-time cash price of soybeans in central Illinois for 2000 is $4.56 per bushel, the flat shrink charge assigned to soybeans is 1.14¢ per bushel ($4.56*0.0025*100). [24]

As noted earlier, storage costs include the physical cost of storage and interest opportunity costs. Interest cost is computed using the 2000 harvest cash price and an annual interest rate of ten percent. Specifically, the interest charge for storing grain on-farm is computed as the harvest price times the interest rate compounded daily from the end of harvest to the date of sale. [25] The interest rate is the average rate for all new commercial agricultural loans in the fourth quarter of 2000 as reported in the Agricultural Finance Databook, which is published by the Board of Governors of the Federal Reserve Board. Interest rates for the fourth quarter are assumed to most accurately reflect actual opportunity costs on agricultural loans related to storage.

The second scenario considered is storage off-farm at commercial facilities and a short-run decision-horizon. The marginal cost of physical storage in this case is the sum of commercial storage, drying and shrinkage charges. As in the past, storage costs at commercial elevators in 2000 are drawn from an informal telephone survey of nine central Illinois elevators. [26] Based on this information, physical commercial storage charges are assumed to be a flat 13¢ per bushel from the end of harvest through December 31. After January 1, physical storage charges are assumed to be 2¢ per month (per bushel), with this charge pro-rated to the day when the cash sale is made. The drying charge to reduce corn moisture from 15% to 14% is a flat 2¢ per bushel, while the charge for shrinkage is 1.3% per bushel. [27] The cost of commercial shrinkage is based on the harvest price (no shrinkage is assumed for soybeans in commercial storage). Given that the harvest-time cash price of corn in central Illinois for 2000 is $1.64 per bushel, the charge for volume reduction is 2.13¢ per bushel ($1.64*0.013*100). Therefore, the flat shrink charge assigned to all stored corn is 4.13¢ per bushel. [28] Interest opportunity cost is computed using the same procedures and assumptions as outlined above for on-farm storage.

The third and fourth scenarios shift to a long-run decision-horizon, where the on-farm scenario is applicable to a farmer considering the construction of new on-farm storage facilities and the commercial scenario is applicable to a farmer that plans on using commercial storage facilities over the long-run. Since all costs are variable in the long-run, the relevant marginal physical storage cost in both of these scenarios is the total cost. Dhuyvetter, Hamman and Harner (2000) estimate the on-farm fixed cost of physical storage for a 25,000 bushel round, metal bin to be 14.6¢ per year. This fixed cost can be added to the on-farm variable cost estimate discussed earlier to compute the total physical cost of on-farm storage. Presumably, commercial physical storage charges paid by farmers reflect total variable and fixed costs of storage at commercial facilities. Consequently, the commercial storage costs discussed earlier in the context of short-run decisions also represent long-run commercial physical costs.

A comparison of the estimated costs of storage for corn and soybeans in the 2000 crop year is found in Tables 4 and 5, respectively. The first item of note is that the on-farm variable cost of physical storage changes little for corn as the storage length increases and is constant for soybeans as the storage length increases. The reason is the previously mentioned “one-time” nature of most physical costs of on-farm storage. As shown in Figure 5, this results in a “non-linear” relationship between on-farm variable costs of storage per month and the length of storage. For example, the on-farm variable cost for corn stored two months after harvest is about 4.3¢ per month. This can be compared to the on-farm variable cost of corn stored six months after harvest of about 1.5¢ per month. The second item of note is the much lower level of on-farm variable costs versus commercial storage costs. Of course, this is not surprising given that variable on-farm storage costs do not include fixed costs, while commercial storage costs presumably reflect total variable and fixed storage costs at commercial facilities. The third item of note is the similar level of total on-farm costs (variable plus fixed) and total commercial costs for all but the shortest and longest storage lengths. Figure 4 illustrates this finding on a per month basis. This result is not surprising assuming reasonably competitive conditions in the market for storage. If total on-farm storage costs were substantially less than total commercial costs, this would encourage a rapid expansion of on-farm storage and vice versa. In fact, the proportion of on-farm versus off-farm storage capacity in Illinois has been roughly equal for a number of years. [29] This is consistent with a basic equilibrium in the storage market where total on-farm costs and commercial costs are about the same.

Given the information presented in Tables 4 and 5, it is possible to compute net advisory prices and benchmarks under each of the four storage scenarios described at the beginning of this section. It turns out that only two sets of storage costs are necessary to represent all four scenarios. Most obviously, on-farm storage costs in the short-run are estimated by on-farm variable storage costs (fourth column in Tables 4 and 5). Commercial storage costs in the short-run and long-run can be estimated by commercial storage costs (last column in Tables 4 and 5). Based on the equilibrium argument made above, on-farm storage costs in the long-run can also be estimated based on commercial storage costs. Therefore, in the remainder of this report, reference will be made only to on-farm variable storage costs and commercial storage costs. It is left to the reader to interpret commercial storage cost results in terms of one of the three applicable scenarios.

The calculation of storage charges may be impacted by an advisory program’s loan recommendations and/or the decision rules discussed in the previous section. Specifically, during the period corn or soybeans are placed under loan, interest costs are not accumulated, as the proceeds from the loan can be used to offset interest opportunity costs that otherwise would accumulate. This most commonly occurs after May 31, 2001, when it is assumed that all unpriced grain, for which loan benefits have not been collected, is placed under loan until priced. If a crop is priced (forward contracts, futures or options) while under loan but stored beyond the time of pricing, interest opportunity costs are accumulated from the day of pricing until the time storage ceases (since it is assumed the loan is repaid on the date of pricing).

It could be argued that interest opportunity costs should be charged based on the LDP available at harvest but not taken by an advisory program. This adjustment is not made because it would not substantially impact the results due to the small interest opportunity costs involved.

A final issue related to storage costs is the use of different strategies based on the availability of on-farm storage. Specifically, advisory programs may issue one set of recommendations assuming on-farm storage is available and another set of recommendations assuming only commercial storage is available. From a practical standpoint, the alternative strategies must be differentiated during harvest before grain is placed in on-farm or commercial facilities. After harvest, when grain has already been placed in on-farm or commercial storage facilities, such advice is of little practical value to most farmers. Hence, if a program clearly differentiates on-farm and commercial storage strategies during harvest of the 2000 crop, the on-farm recommendations are used in computing the net advisory price under on-farm variable costs and the commercial recommendations are used in computing the net advisory price under commercial costs. [30] In this case, the net advisory price for a program under the two alternative storage cost assumptions will vary due to the difference in costs and underlying strategies. If a service does not clearly differentiate on-farm and commercial storage strategies during harvest of the 2000 crop, the same recommendations are used in computing net advisory prices under on-farm variable and commercial storage costs. In this case, the net advisory price for a program under the two alternative storage cost assumptions will vary only due to the difference in costs, as the underlying strategies are the same.

The essential concept underlying performance evaluation of market advisory programs is fairly simple: the comparison of the net prices generated by advisory programs with prices that could have been obtained by a farmer through one or more appropriate alternative strategies (Sharpe, Alexander and Bailey, 1999, p. 829). The comparison strategies are commonly referred to as benchmarks because they serve as objective standards of performance, much like a yardstick provides an objective measurement of distance. Within this broad framework, two basic types of performance evaluation can be applied to market advisory programs. The first type is based on comparison to “peer-group” benchmarks, whereby net advisory prices are compared to each other or the average price across all advisory programs. The second type is based on comparison to “external” benchmarks, whereby net advisory prices are compared to prices from strategies that do not depend upon market advisory program behavior. In financial markets, it is commonplace to compare investment performance to external benchmarks, such as the Dow-Jones Industrials Index, S&P 500 Index and Wilshire 5000 Index.

The AgMAS study focuses on performance evaluation using external benchmarks. While peer-group evaluation provides useful information about the rank of advisory programs, it cannot answer the question of whether performance of advisory programs as a group or an individual advisory program is “superior” or “inferior” in an absolute economic sense. To answer this question, external benchmarks must be specified based on theories of market pricing.

The first class of external benchmarks is based on the theory of efficient markets. This theory assumes that market participants are rational and that competition instantaneously eliminates all profitable arbitrage opportunities. In its strongest form, efficient market theory predicts that market prices always fully reflect available public and private information (Fama, 1970). The practical implication is that no trading strategy can consistently beat the return offered by the market. Hence, the return offered by the market becomes the relevant benchmark. In the context of the AgMAS study, a market benchmark should measure the average price offered by the market over the marketing window of a representative farmer who follows advisory program recommendations. The average price is computed in order to reflect the returns to a naïve, “no-information” strategy of marketing equal amounts of grain each day during the marketing window. The difference between advisory prices and the market benchmark measures the value of advisory service information. The theory of efficient markets predicts this difference, on average, will equal zero. [31]

If all market participants are rational in the way efficient market theory assumes, then the only interesting external benchmarks are market benchmarks. However, there is growing evidence that many market participants may not be fully rational in the efficient market sense. Hirshleifer (2001) provides a comprehensive review of the judgment and decision biases that appear to affect securities market investors, such as framing effects, mental accounting, anchoring and overconfidence. He also provides an exhaustive review of empirical studies that attempt to measure the potential impact of such biases on securities prices and investment returns. As an example, Barber and Odean (2000) finds that individual stock investors under-perform the market by an average of one-and-a-half percentage points per year, an economically significant amount, particularly when viewed over long investment horizons. He argues that a combination of overconfidence and excessive trading explains this finding. New “behavioral” theories of market pricing have been developed based on the assumption that market participants are subject to judgment and decision biases (e.g., Daniel, Hirshleifer and Subrahmanyam, 1998).

Behavioral market theory suggests that the average return actually achieved by many market participants may be less than that predicted by efficient market theory, due to the judgment and decision biases that plague most participants. As a result, the average return actually received by market participants becomes an appropriate external benchmark. In the context of the AgMAS study, a behavioral benchmark should measure the average price actually received by farmers for a crop. The difference between net advisory prices and a farmer benchmark measures the value of market advisory service information relative to the information used by farmers. Behavioral market theory does not predict a specific value for this difference. It may be positive, negative or zero, depending on the impact of judgment and decision biases on advisory programs versus farmers.

It is important to point out that the value of advisory program information provided by comparison to a farmer benchmark is not as precise as that offered by a market benchmark. The reason is that many farmers are known to follow market advisory program recommendations, at least to some extent. Hence, a farmer benchmark already reflects the impact of market advisory program information to some degree. While it is not possible to know the true dimensions of this problem, it should be kept it in mind when making the comparison. [32] This helps to explain the importance usually placed on market benchmarks as a standard in performance comparisons. Simply put, market benchmarks allow a clear and straightforward interpretation of performance results, which may not be the case for a farmer benchmark.

Finally, it is important to re-iterate that market and farmer benchmarks convey quite different information about the performance of market advisory programs, even though both are forms of a relative benchmark. This should be carefully considered when making performance comparisons based on the two types of benchmarks. In addition, there are some desirable properties from a practical perspective that both types of benchmarks should possess: i) they should be relatively simple to understand and to calculate; ii) they should represent the returns to a marketing strategy that can be implemented by farmers; and iii) they should be directly comparable to net advisory prices (Jackson, Irwin and Good, 1998).

For most of the life of the AgMAS Project, net advisory prices were compared only to market benchmarks. Starting with performance evaluation for the 2000 crop year, net advisory prices are compared to both market and farmer benchmarks.

Market Benchmarks

As noted above, a market benchmark is designed to measure the average price offered by the market to farmers. The appropriate time period for computing the average price is the marketing window of a farmer who follows the recommendations of the advisory programs included in the AgMAS study. This window was defined earlier (see the “Market Window” section) as the 24-month period that begins on September 1^st of the year before harvest and ends on August 31^st of the year after harvest. A 24-month market benchmark is simply computed as the average price over the two-year marketing window.

Figure 6 presents average marketing profiles for market benchmarks and advisory programs in corn and soybeans over the 1995-1999 crop years. For comparison purposes, average marketing profiles for 24- and 20-month market benchmarks are included. The 20-month benchmark simply deletes the first four months of the 24-month marketing window from the computations of the average market price. As a result, this benchmark is based on the average price over the period that begins on January 1 of the year of harvest and ends on August 31 of the year after harvest. For both corn and soybeans, the market benchmarks appear to provide a surprisingly good “fit” to the average profile of the advisory programs. More specifically, if a simple linear trend regression is fit to the average profile of the advisory programs (not shown), the estimated trend line is remarkably close to the 24-month benchmark for corn and the 20-month benchmark for soybeans. The profiles of the two market benchmarks also bracket the average starting date of significant pricing by advisory programs (one percent or more).

The results discussed in the previous paragraph suggest there is some uncertainty about specification of the most appropriate market benchmark for corn and soybean performance evaluations. Leamer (1983) argues persuasively (and famously) that in this type of situation it is crucial to understand the “fragility” of results when key assumptions are changed. Consequently, both a 24-month and a 20-month market benchmark will be used in comparisons to net advisory prices. Only the 24-month market benchmark has been used for the last several years in AgMAS pricing reports. Cash forward prices for central Illinois are used during the pre-harvest period, while daily spot prices for central Illinois are used for the post-harvest period. The same forward and spot price series applied to advisory program recommendations are used to construct both market benchmarks. Details on the forward and cash price series can be found in the earlier “Prices” section of this report.

Three adjustments are made to the daily cash prices to make the 24-month and 20-month average cash price benchmarks consistent with the calculated net advisory prices for each marketing program. The first is to take a weighted-average price, to account for changing yield expectations, instead of taking the simple average of daily prices. This adjustment is consistent with the procedure described previously in the "Yields and Harvest Definition" section. The daily weighting factors for pre-harvest prices are based on the calculated trend yield, while the weighting of the post-harvest prices is based on the actual reported yield for central Illinois. The second adjustment is to compute post-harvest cash prices on a harvest equivalent basis, which is done by subtracting on-farm variable or commercial storage costs (physical storage, shrinkage and interest) from post-harvest spot cash prices. The daily storage charges are calculated in the same manner as those for net advisory prices. The third adjustment is made with respect to the loan program. In the context of evaluating advisory program recommendations, it was argued earlier that a “prudent” or “rational” farmer would take advantage of the price protection offered by the loan program, even in the absence of specific advice from an advisory program. This same logic suggests that a “prudent” or “rational” farmer will take advantage of the price protection offered by the loan program when following the benchmark average price strategy. Based on this argument, the 24-month and 20-month average cash price benchmarks are adjusted by the addition of LDPs and MLGs. Bushels marketed in the pre-harvest period according to the benchmark strategy are treated as forward contracts, with the LDPs assigned at harvest. Bushels marketed each day in the post-harvest period are awarded the LDP or MLG in existence for that particular day. Finally, just as in the case with comparable advisory program recommendations, interest opportunity costs are not charged to the benchmark after May 31, 2001 to reflect the assumption that stored grain is placed under loan.

While the 24- and 20-month market benchmark prices can obviously differ for a given crop year, averages of the two benchmark prices across crop years are not expected to differ substantially. First, the difference in the marketing windows for the two benchmarks is relatively small, as the 20-month benchmark reduces the 24-month marketing window by only about 17%. Second, given a sufficiently large sample of crop years and efficient corn and soybean markets (cash, futures and options), annual averages of different average price benchmarks should be similar when stated on a harvest equivalent basis. Of course, if corn and soybean markets were inefficient the equivalence would not hold. In particular, if pre-harvest prices contain a “drought premium” as some argue (e.g., Wisner, Baldwin and Blue, 1998), then the 24-month benchmark price may be consistently higher than the 20-month benchmark price.

In contrast to averages, the variation of 24- and 20-month market benchmark prices across crop years is expected to differ. The first reason for the difference is the well-known result in statistics that the sampling variation of the mean (average) is inversely related to the sample size used to compute the average (e.g., Griffiths, Hill and Judge, 1993, p.82). Since the sample of daily prices used in computing the 24-month benchmark is larger than the sample for the 20-month benchmark, the variation of the 24-month benchmark should be smaller than variation of the 20-month benchmark. The second reason for the difference is a “time diversification” effect. Specifically, the 20-month benchmark is not generated by randomly sampling prices over the entire 24-month marketing window, but instead all prices are sampled after deleting the first four months from the 24-month window. This non-random effect shortens the time interval over which prices are sampled. All else constant, the shorter time interval for the 20-month market benchmark will lead to larger variation across crop years than the 24-month benchmark. Since both the sample size and time diversification effects work in the same direction, the 20-month market benchmark is expected to vary more than the 24-month benchmark. [33]

A practical concern with the market benchmarks is that a farmer may not be able to implement the benchmark strategies since they involve marketing a small portion of the crop every day. There are two reasons to believe this concern is not overly serious. First, a number of companies have developed and offer grain “index” contracts that allow farmers to receive the average market price over a pre-specified time interval. Representative examples can be found at, http://www.cargillaghorizons.com/aghorizons/performancemarketing/index.htm, http://www.cgb.com and http://www.e-markets.com. Second, a strategy of routinely selling at less frequent intervals closely approximates the market benchmark prices. For example, a farmer might consider alternative “tracking” strategies of marketing only once a month or once every other month over the 24-month window. Using mid-month prices, a tracking strategy of marketing only once a month (24 times) generates average prices over 1995-2000 that are quite close to 24-month market benchmark prices. The average difference is only two cents per bushel for corn and one cent per bushel for soybeans, and the maximum difference for any particular crop year is eight cents per bushel in corn and five cents per bushel in soybeans. A tracking strategy of marketing once every other month (12 times) also generates average prices over 1995-2000 that are quite close to 24-month market benchmark prices. The average difference is only three cents per bushel for corn and five cents per bushel for soybeans.

The average difference results for the benchmark tracking strategies should not be a surprise given the previous argument about averages of different benchmark prices in efficient markets. More surprising is the result that the tracking strategies vary only two to four cents per bushel more than the 24-month benchmark over 1995-2000. This is evidence that the time diversification effect can swamp the sample size effect, as the tracking strategies are based on dramatically smaller samples (12 or 24 observations compared to about 500 observations) but have only a marginally higher variation. Time diversification is approximately the same for the tracking strategies and the 24-month benchmark because sample observations for the tracking strategies, like the benchmark, are spaced equally across the 24-month marketing window.

Farmer Benchmarks

As noted earlier, a farmer benchmark is designed to measure the average price received by farmers for a crop. This type of benchmark should reflect the actual behavior of farmers in marketing grain, and include all of the transactions (e.g., cash, forward, futures and options) that farmers employ in this regard. In theory, a farmer benchmark should not be difficult to calculate. First, a representative sample of grain farmers in the relevant geographic area would be drawn (randomly). Next, the average price received by each farmer would be computed (using the same assumptions as in the computation of net advisory prices and market benchmarks). Last, the farmer benchmark would be computed as the weighted-average price received by all farmers in the sample, with the weights equal to the sample proportion of the crop produced by each farmer.

In practice, the detailed type of data needed to construct a valid farmer benchmark is not available, so an approximation must be used. The only known approximation is the USDA average price received series. This series has some good features and bad features with respect to measuring the average price received by farmers. On the plus side, the USDA series reflects the actual pattern of cash grain marketing transactions by farmers, and thus, incorporates the marketing windows and timing strategies actually used by farmers; includes forward contract transactions for both the pre-harvest and post-harvest periods, with the transactions recorded at the forward price, not the spot price at time of delivery; and grain sales are adjusted to industry standards for moisture. On the negative side, the USDA series is only available on a statewide basis; includes cash transactions for different grades and quality of grain sold by farmers; does not include futures and options trading profits/losses of farmers; and reflects a mix of old and new crop sales by farmers.

Fortunately, none of the problems mentioned above are prohibitive with respect to the use of the USDA series as a measure of the average price received by farmers. Since spatial basis patterns are relatively stable, it is straightforward to adjust the USDA series to an alternative geographic location. It turns out that this type of adjustment is not necessary for AgMAS performance evaluations because central Illinois prices closely mirror the average price for the entire state of Illinois. For example, the average cash price of corn and soybeans for central Illinois over January 1995-December 2001 differs from the state average price by about one cent (state average higher for corn, lower for soybeans). The correlation of daily prices for central Illinois and the state is 0.96 for corn and 0.99 for soybeans. Hence, from a statistical standpoint, central Illinois and state average prices are equivalent.

It is not possible to adjust the USDA series to a constant grade and quality, to reflect futures and options trading profits/losses of farmers or to only reflect new crop sales because the data simply are not available. However, the resulting biases probably are small and some may work in opposite directions. Consider the grade and quality issue first. It is well known that some fraction of the corn crop is discounted relative to the standard number two yellow corn grade. This is also true for the soybean crop relative to the standard number one yellow soybean grade, but likely to a smaller extent than corn. As a result, the USDA average price received reflects a weighted-average of both undiscounted and discounted grain sales. The weights are unknown, but the direction of the bias relative to average prices for the standard grade is clearly downward. In other words, when compared to the average price at the standard grade, the USDA average price received should be adjusted upwards to reflect the impact of discounts.

A key question, of course, is the magnitude of the grade and quality bias discussed above. An extensive search of the literature was conducted and no previous study was uncovered that directly measured the proportion of corn and soybeans sold at a discount or the average magnitude of price discounts in central Illinois (or other Midwestern US areas). The Federal Grain Inspection Service of the US Department of Agriculture (FGIS) was contacted and staff indicated that FGIS does not have an historical series of this type. One older study was located that contained some information on the issue. Hill, Kunda and Rehtmeyer (1983) reported the results of a 1982 survey of grain elevator operators in Illinois. One question in this survey asked elevator operators to estimate the percentage of corn and soybean receipts at country elevators that typically exceed grade factors. Unfortunately, the results were not netted across grade factors, so it is not possible to estimate the typical proportion of the crop sold at a discount (if a lot is over one grade limit it will have a higher than average chance of being over the grade limit for other factors). In addition, the average magnitude that grade factors were exceeded is not reported, so it is impossible to estimate the dollar value of the average discount. Nonetheless, the results provide some perspective on the quality issue. For corn delivered in the fall, the percentage typically above a grade factor ranged from 0.2 to 7.5% of deliveries. For soybeans delivered in the fall, the percentages were about the same, except for foreign material, where over 30% of the bushels delivered typically exceeded the grade factor. When winter and summer delivery was considered, the percentages increased somewhat for corn and decreased for soybeans. Other than foreign material for soybeans, this evidence suggests that less than 10% of the corn and soybean crops in the early 1980s were sold at a discount to the standard grade.

To provide more recent evidence on quality, the nine central Illinois elevators surveyed annually for commercial storage costs were queried in December 2001 about the average quality of corn and soybean crops. The most frequent response from the elevator managers in this informal survey was that less than one percent of corn and soybeans is sold at a discount (for non-moisture factors) relative to the standard grade. The range was from zero to less than five percent. The largest estimate of the average dollar value of discounts was two to three cents per bushel. These figures provide enough information to make a very rough estimate of maximum quality bias in the USDA average price received series. Using the maximum proportion of five percent and the maximum average discount value of three cents from the informal survey, the downward bias relative to the standard grade would be only 0.15¢ per bushel (0.05*3). Furthermore, if the average discount is three cents, then one-third of the crop would have to be sold at a discount to induce a downward bias even as large as one cent (0.33*3 = 1). In sum, while the evidence is limited and sketchy, it does suggest that any downward quality bias in the USDA average price received series, at least for corn and soybeans in central Illinois, is quite small.

Now, consider the potential bias from omission of futures and options profits/losses. If a farmer uses futures and options exclusively for “pure” hedging purposes, they will consistently take short positions at about the same points in the marketing window each year. Unless futures prices are biased upwards or downwards, this type of hedging will not result in large profits or losses, as the price changes from upward and downward price trends should roughly offset over time. [34] If a farmer uses futures and options to engage in “selective” hedging, they may have large profits or losses related to the timing of trading. While no direct evidence on the profits or losses of farmers is available in this context, there is convincing evidence that small traders in general consistently lose money in futures and options markets. [35] It seems reasonable to assume that farmers engaged in selective hedging are similar to other small traders, and hence, selective farmer hedging in futures and options markets results in aggregate trading losses. [36] Given that pure hedging is expected to yield zero profits on average and selective hedging is expected to yield losses on average, the net effect of the two types of futures and options trading by farmers should be negative. In this case, when compared to average prices at the standard grade, the USDA average price received should be adjusted downward to reflect the impact of net trading losses.

As before, the key question is the potential magnitude of the bias from omission of futures and options losses. The key piece of evidence in this regard is the limited scale of farmer trading in futures and options markets. Surveys have consistently reported that relatively few farmers directly use futures and options contracts on a regular basis (e.g., Patrick, Musser, and Eckman, 1998). Given this information, it is reasonable to argue that the magnitude of farmers’ net losses from futures and options trading, in aggregate, should be small. As a result, the upward bias in the USDA average price received from the omission of futures and options net losses should be small.

Next, consider the potential bias from mixing old crop and new crop sales during the 12-month marketing year used to compute the USDA average price received. The first step is to determine the potential magnitude of the problem. Fortunately, bounds for the “shifting” of old crop sales into the next marketing year can be computed by dividing ending stocks for a marketing year by crop production for the same marketing year (e.g., September 1, 2000 soybean stocks divided by 1999 soybean production). Over the 1995/1996 through 2000/2001 marketing years, on-farm ending stocks in Illinois averaged four percent of statewide corn production and three percent of statewide soybean production. These percentages are the lower bounds on shifting because farmers presumably own on-farm stocks and sales of these stocks will be shifted to the next marketing year. Over the 1995/1996 through 2000/2001 marketing years, total ending stocks (on-farm and off-farm) in Illinois averaged 12% of statewide corn production and 8% of statewide soybean production. These percentages are the upper bounds on shifting assuming farmers own all of the stocks in off-farm storage facilities. Clearly, this assumption is unrealistic, as commercials own some, if not most, of the stocks in off-farm facilities at the end of a marketing year. The bottom-line is that shifting of old crop sales into the next marketing year, on average, is somewhere between 4 and 12% of corn production and 3 and 8% of soybean production. This suggests the magnitude of shifting from one crop year to the next probably is not large.

The second step is to determine the impact shifting old crop sales will have on the USDA average price received. Consider the simplest case where old crop sales in the next marketing year are made at spot prices for the new crop and the same proportion is shifted every year. The same price received would result as in the no shifting case. Only to the degree that the proportion shifted varies from year-to-year will the average price received differ from the no-shifting case. The proportion does vary from year-to-year, but not by a substantial amount. For example, on-farm ending stocks in Illinois varied from only two to six percent of corn production over the 1995/1996 through 2000/2001 marketing years. The impact of this variability on average price received will depend on farmers’ ability to time shifts to take advantage of favorable spreads between old crop and new crop prices. If farmers as a group have timing ability in this context, then the USDA average price received will be biased upwards relative to the average price at the standard grade. However, given the difficulty of predicting old crop-new crop price spreads (Lence and Hayenga, 2001) and the small absolute magnitude of actual shifting of sales, it seems reasonable to argue that the bias in average price received from shifting old crop sales across marketing years is quite small.

In total, the evidence and arguments discussed above suggests that the net systematic bias in the USDA average price received relative to average prices for the standard grade is small, at least for corn and soybeans in central Illinois. It is difficult to construct a scenario where the level of bias would materially effect performance evaluation of market advisory programs. However, direct evidence on the bias issue is limited and sketchy, so it is difficult to reach a firm conclusion. The USDA average price received is best viewed as an approximation of the “true” average price received by farmers.

Several adjustments are made to the USDA average price received data for the state of Illinois in order to make the computed farmer benchmark consistent with net advisory prices. To begin, mid-month on-farm or commercial storage charges are applied to the monthly average price received in the 12-month marketing year (September through August). Next, the annual weighted-average price received is computed using the percentage of the crop marketed in each calendar month as the weights. Finally, actual state average LDPs and MLG’s are added for the 1998, 1999 and 2000 crops. [37]

Given the uncertainties involved in measuring the average price received by farmers, it would be useful to specify alternative farmer benchmarks. Unfortunately, as the discussion in this section has detailed, there simply is no alternative measure that reflects the actual marketing behavior of farmers. The inability to provide information on the sensitivity of performance comparisons to alternative farmer benchmarks is a limitation of the analysis and should be kept in mind when viewing the results.

Finally, it is interesting the consider arguments about the expected difference in averages and variation between the farmer benchmark and the market benchmarks. If corn and soybean markets are efficient and farmers are rational, then the average price across crop years for the farmer and market benchmarks should be similar. Under these assumptions, the variation in farmer benchmark prices across crop years could be smaller or larger than the variation in market benchmark prices, depending on the length of the marketing window used by farmers and the exact nature of the marketing strategies implemented by farmers.

Unfortunately, it is not possible to determine the average marketing window or the pricing pattern of farmers using USDA monthly marketing weights. For perspective, average monthly USDA marketing weights for corn and soybeans in Illinois over 1995-2000 are presented in Figure 7. These weights reflect the pattern of grain deliveries by farmers to commercial facilities over the 12-month marketing year. Grain deliveries do not necessarily reflect the pricing pattern of farmers due to the use of forward pricing instruments. There is ample survey evidence that significant numbers of farmers use pre-harvest forward contracts to price a portion of their crops, and that post-harvest forward contracts are widely used, particularly for January delivery (e.g., Patrick, Musser and Eckman, 1998; Pennings, Good, Irwin and Gomez, 2001). The difficulty is that almost no concrete evidence exists on the exact length of the typical marketing window of farmers or the precise pattern of forward pricing. Anecdotal evidence suggests that farmers may use a marketing window not unlike the 24-month and 20-month windows assumed for the market benchmarks, but the amount of pre-harvest forward pricing is far less than assumed for the market benchmarks. All else equal, this would lead to the expectation that the variation of farmer benchmark prices would exceed that for the market benchmarks.

Under rationality, it is still possible for the variation of farmer benchmark prices to be smaller than for market benchmarks if farmers employ market-timing strategies that successfully reduce price variation. Alternatively, if farmers are subject to the same judgment and decision biases as appears to be the case for participants in other markets, then it would be reasonable to expect the farmer benchmark to have a lower average price and higher variation than the market benchmarks. Which of the above scenarios is correct can only be determined empirically.

Net Advisory Prices and Benchmarks for 2000

Net advisory prices and benchmarks for the 2000 corn and soybean crops are presented in Tables 6 through 11. These results are new and add to the sample of net advisory prices and benchmarks previously available for analysis. For a specific example of how marketing recommendations are translated into a final net advisory price that incorporates the simulation assumptions, see Jackson, Irwin and Good (1996). It is important to emphasize that all of the net advisory prices and benchmarks presented in Tables 6 through 11 are stated on a harvest equivalent basis using either on-farm variable or commercial storage costs.

Net advisory prices and benchmarks for corn in 2000 assuming on-farm variable storage costs are presented in Table 6. In addition, this table shows the components of the advisory prices and benchmarks. The 2000 average net advisory price for all 27 corn programs is $2.21 per bushel under the assumption of on-farm variable costs. It is computed as the unadjusted cash sales price ($1.97 per bushel) minus storage charges ($0.11 per bushel) plus futures and options gain ($0.04 per bushel) minus brokerage costs ($0.02 per bushel) plus LDP/MLG gain ($0.32 per bushel). The range of net advisory prices for corn in 2000 assuming on-farm variable storage costs is $1.90 to $2.90 per bushel. Corresponding benchmark prices range from $2.06 per bushel (USDA farmer benchmark) to $2.15 per bushel (24-month average market benchmark).

Net advisory prices and benchmarks for soybeans in 2000 assuming on-farm variable storage costs are presented in Table 7. The 2000 average net advisory price for all 26 soybean programs is $5.51 per bushel under the assumption of on-farm variable costs. It is computed as the unadjusted cash sales price ($4.70 per bushel) minus storage charges ($0.15 per bushel) plus futures and options gain ($0.04 per bushel) minus brokerage costs ($0.02 per bushel) plus LDP/MLG gain ($0.94 per bushel). The range of net advisory prices for soybeans in 2000 assuming on-farm variable storage costs is $5.05 to $6.88 per bushel. Corresponding benchmark prices range from $5.38 per bushel (USDA farmer benchmark) to $5.47 per bushel (24-month average market benchmark).

Since many Corn Belt farmers grow both corn and soybeans, it also is useful to examine a combination of the results for the corn and soybean marketing programs. In order to do this, gross revenue is calculated for a central Illinois farmer who follows both the corn and soybean marketing advice of a given program. It is assumed that the representative farmer splits acreage equally (50/50) between corn and soybeans and achieves corn and soybean yields equal to the actual yield for the area in 2000. The 50/50 advisory revenues are computed on a per acre basis and compared with the revenue a central Illinois farmer could have received based on benchmark prices for both corn and soybeans. Advisory revenue per acre is calculated only for those programs that offer both corn and soybean marketing advice.

Advisory program revenues and benchmarks in 2000 assuming on-farm variable storage costs are presented in Table 8. The average revenue achieved by following both the corn and soybean programs offered by an advisory program is $306 per acre. The range of 50/50 advisory revenue in 2000 assuming on-farm variable storage costs is $276 to $393 per acre. Corresponding benchmark revenues range from $291 per acre (USDA farmer benchmark) to $300 per acre (24-month average market benchmark).

For comparison purposes, the annual subscription cost of each advisory program also is listed in the last column of Table 8. Subscription costs average $333 per program, a level that does not appear to be large relative to total farm revenue, whether a large or small farm is considered. For a 1,000 acre farm, subscription costs average about one-tenth of one percent of total advisory revenue. For a 250 acre farm, subscription costs average about four-tenths of one percent of total advisory revenue. While subscription costs do not appear to be large relative to revenue, it is important to point out that the cost of implementing, monitoring and evaluating the strategies recommended by advisory programs is not incorporated in the comparisons. Such costs are difficulty to quantify, but informal feedback from farmers suggests they are not trivial, especially in terms of the opportunity cost of management time.

Net advisory prices and benchmarks for corn in 2000 assuming commercial storage costs are presented in Table 9. The 2000 average net advisory price for all 27 corn programs is $2.13 per bushel under the assumption of commercial storage costs. It is computed as the unadjusted cash sales price ($1.97 per bushel) minus storage charges ($0.20 per bushel) plus futures and options gain ($0.04 per bushel) minus brokerage costs ($0.02 per bushel) plus LDP/MLG gain ($0.32 per bushel). The range of net advisory prices for corn in 2000 assuming commercial storage costs is $1.79 to $2.78 per bushel. Corresponding benchmark prices range from $1.95 per bushel (USDA farmer benchmark) to $2.09 per bushel (24-month average market benchmark).

Net advisory prices and benchmarks for soybeans in 2000 assuming commercial storage costs are presented in Table 10. The 2000 average net advisory price for all 26 soybean programs is $5.45 per bushel under the assumption of commercial storage costs. It is computed as the unadjusted cash sales price ($4.70 per bushel) minus storage charges ($0.21 per bushel) plus futures and options gain ($0.04 per bushel) minus brokerage costs ($0.02 per bushel) plus LDP/MLG gain ($0.94 per bushel). The range of net advisory prices for soybeans in 2000 assuming commercial storage costs is $5.00 per to $6.83 per bushel. Corresponding benchmark prices range from $5.29 per bushel (USDA farmer benchmark) to $5.42 per bushel (24-month average market benchmark).

Advisory program revenues and benchmarks in 2000 assuming commercial storage costs are presented in Table 11. The average revenue achieved by following both the corn and soybean programs offered by an advisory program is $298 per acre when commercial storage costs are assumed. The range of 50/50 advisory revenue in 2000 assuming commercial storage costs is $265 per to $381 per acre. Corresponding benchmark revenues range from $279 per acre (USDA farmer benchmark) to $293 per acre (24-month average market benchmark).

Figures 8 and 9 show the pattern of corn prices for the 2000 crop year based on on-farm variable and commercial storage costs, respectively. The top chart shows daily cash prices from September 1, 1999 through August 31, 2001. The pre-harvest prices are the cash forward contract prices for harvest delivery. The middle chart is a repeat of the top chart with daily LDP or MLG added to the daily price. For the pre-harvest period, the LDP is the average LDP available at harvest time. The third chart offers a different perspective, in that post-harvest daily cash prices are adjusted for cumulative storage costs (interest, physical storage, and shrinkage charges). The chart illustrates the pattern of harvest equivalent prices plus LDP or MLG.

Corn prices for the 2000 crop year are highest in the pre-harvest period, with the cash forward contract price remaining well above $2.00 and moving near $2.40 in late April. Prices decline into harvest as average yields and total production exceed expectations, but make a significant post-harvest recovery in November and December as basis levels strengthen. Prices decline in the spring of 2001 in the face of good planting weather, but recover somewhat in July and August due to some areas of dry weather in the US. The price pattern for the 2000 crop year is typical of a large crop year.

Figures 10 and 11 show the pattern of soybean prices for the 2000 crop year based on on-farm variable and commercial storage costs, respectively. The three charts are the same as for corn, depicting daily cash prices, cash prices plus LDP/MLG, and cash prices plus LDP/MLG minus storage charges. Soybean prices for the 2000 crop follow a similar pattern to that for corn. Cash prices are more volatile in the pre-harvest period, peaking just above the CCC loan rate in May. That is the only time the cash price is above the loan rate in the entire crop year. Prices decline sharply during the growing season, but make a small rally into harvest. Prices fall in the early spring of 2001 under the weight of another large South American crop, but rally modestly in the summer months due to some concern about dry weather in parts of the US. The price pattern for the 2000 crop year reflects dual harvest periods, the US in the fall months and South America in the spring months. The largest LDPs/MLGs occur in the spring of 2001.

Net advisory prices, revenue and benchmarks for the 1995-2000 crop years are reported in Tables 12 through 14. In order to obtain a consistent set of net advisory prices and benchmarks for the entire sample period, commercial storage costs are assumed. It is not possible to present parallel results assuming on-farm variable costs of storage, because the AgMAS Project first computes net advisory prices and benchmarks under this alternative storage cost assumption for the 2000 crop year. Also note that some of the market advisory programs included in the tables are not evaluated for all six years.

As shown in Table 12, the average advisory price for corn ranges between $2.02 per bushel in 1999 and $3.03 per bushel in 1995. Range statistics reveal that net advisory prices for corn vary substantially within individual crop years. The most dramatic example is 1995, where the minimum is $2.29 per bushel and the maximum is $3.90 per bushel. Even in years with less market price volatility, it is not unusual for the range of prices across advisory programs to be nearly a dollar per bushel. The three alternative benchmark prices for corn are shown at the bottom of Table 12. The variation in benchmark prices from year-to-year is similar to that of average net advisory prices. However, there can be substantial differences in benchmark prices for a particular crop year. For example, the 24-month market benchmark in 1998 is $2.24 per bushel, while the USDA farmer benchmark is only $1.97 per bushel. These data suggest performance results for corn may be sensitive to the selected benchmark.

As reported in Table 13, the average advisory price for soybeans ranged from $5.45 per bushel in 2000 to $7.27 per bushel in 1996. Similar to corn, the range of individual net advisory prices within a crop year is substantial. The most dramatic example is 1999, where the range in advisory prices approaches $2.50 per bushel. The three alternative benchmark prices for soybeans are shown at the bottom of Table 13. The variation in soybean benchmark prices from year-to-year is similar to that of average net advisory prices. Once again, there can be substantial differences in benchmark prices for a particular crop year.

Table 14 contains the combined corn and soybeans revenue results. The lowest average advisory revenue, $298 per acre, occurred in 2000, while the highest average advisory revenue, $369 per acre, occurred in 1996. Given the results for corn and soybeans, the large range of individual advisory revenues within a crop year is not surprising. Nonetheless, it is startling to see the possible economic impact of following the best versus the worst performer in a given crop year. For example, in three of the six crop years (1995, 1999 and 2000), the range in advisory revenue exceeds $100 per acre.

For the reader’s convenience, Tables 15 through 17 report the most recent two-year averages (1999-2000), three-year averages (1998-2000), four-year averages (1997-2000), five-year averages (1996-2000) and six-year averages (1995-2000) of net advisory prices, revenues and benchmarks. [38] The averages are computed in these tables only for the advisory programs active in each of the indicated crop years. The reported averages may reflect survivorship bias as a result of this assumption, which should be considering when viewing the averages. [39] Finally, note that the average, minimum and maximum reported for each column in the Tables 15 through 17 are computed across the advisory program averages in each column.

Information on the sources of the differences between net advisory prices and benchmarks in corn and soybeans is found in Table 18. Panel A shows average net advisory prices and benchmarks broken out by component. Panel B presents the average difference in the components between advisory programs and the benchmarks. All of the averages in the table assume commercial storage costs. In cases where the average net advisory price is above the average benchmark price (e.g., net advisory price in corn vs. the USDA farmer benchmark) the difference is largely explained by the higher net cash sales price of advisory programs. The average net futures and options gain of advisory programs is relatively small, as is the difference in LDP/MLGs between advisors and the benchmarks.

Four basic indicators of performance are applied to advisory program prices and revenues over 1995-2000. The first indicator is the proportion of advisory programs that beat benchmark prices. A valuable feature of this directional indicator is that it is not influenced by extremely high or low advisory prices. The second indicator is the difference between the average price of advisory programs and benchmarks. This indicator is useful because it takes into account both the direction and magnitude of differences from benchmark prices. The third indicator is the average price and risk of advisory programs relative to the average price and risk of the benchmarks. Evaluations based on this indicator are important because risk is incorporated into the performance comparisons. The fourth indicator is the predictability of advisory program performance from year-to-year. This indicator provides information on the value of past pricing performance in predicting future performance.

Before considering the performance evaluation results, a couple of important issues need to be discussed. First, the results presented in this section of the report address the performance of market advisory programs as a group. In other words, average pricing performance across all programs is considered. This is a different issue than the pricing performance of a particular advisory program. [40] Simply put, it is inappropriate to make performance inferences for an individual advisory program based on aggregate results. Second, farmers subscribe to market advisory programs for a variety of reasons. For example, Pennings, Good, Irwin and Gomez (2001) survey farmer-subscribers and find that the two highest rated uses of market advisory programs are marketing information and market analysis. While the quality of marketing information and market analysis is likely to be positively correlated with the returns to marketing recommendations, this does not necessarily have to be the case. It is possible that advisory programs provide valuable information and analysis to farmer-subscribers, yet fail to exhibit superior pricing performance.

The first, and simplest, indicator of pricing performance is the proportion of advisory programs that beat the market or farmer benchmarks. Positive performance is indicated if the proportion of advisory programs beating a benchmark exceeds 50%, the proportion one would observe if advisory performance is random, like flipping a fair coin. A noteworthy feature of this “directional” indicator is that it is not influenced by extremely high or low advisory prices or revenue.

The proportion of advisory programs in corn, soybeans and 50/50 advisory revenue above the benchmarks over 1995-2000 is presented in Table 19. Note that average proportions for 1995-2000 are computed over the full set of advisory programs, and therefore, do not necessarily equal the average of the individual crop year proportions. This “grand” average equally weights each of the net advisory prices or revenues in the sample, whereas an average of the individual crop year averages would equally weight the crop years. The first average is preferred for the present purpose as it implies an equal probability of selecting an individual advisory program across the entire sample. [41]

Considering corn first (Panel A: Table 19), there is some variation in the proportion of net advisory prices above the two market benchmarks for individual crop years, particularly 1998, but the patterns are similar overall. There also does not appear to be any discernable trend in the proportions for either benchmark over the six crop years. The average proportion for 1995-2000 is 51% versus the 24-month benchmark and 59% versus the 20-month benchmark, indicating a slight to marginal chance of advisory prices in corn beating market benchmark prices. In contrast, the proportion of net advisory prices above the USDA farmer benchmark exceeds 50% each crop year and appears to increase somewhat over time. The average proportion above the USDA farmer benchmark over 1995-2000 is 74%. This is substantially higher than the average proportions versus the market benchmarks and indicates a sizeable chance of market advisory programs generating net prices higher than the USDA farmer benchmark.

Moving to soybeans (Panel B: Table 19), there is more variation in the proportion of net advisory prices above the two market benchmarks for individual crop years. Particularly sharp differences are observed in 1998 and 1999, where the spread between the proportions is between 26 and 45 percentage points. There also appears to be a noticeable downward trend in the proportions versus the 24-month benchmark. No clear trend is apparent for the proportions versus the 20-month benchmark. Despite these differences for individual crop years, the average proportions for 1995-2000, 61% versus the 24-month benchmark and 70% versus the 20-month benchmark, both indicate a better than average chance of advisory prices beating market benchmark prices in soybeans. Once again, the proportions above the USDA farmer benchmark are all above 50% and appear to increase somewhat over time. The average proportion above the USDA farmer benchmark over 1995-2000 is 74%, the same as found for corn. This indicates a sizeable chance of market advisory programs generating net prices in soybeans higher than the USDA farmer benchmark.

Given the combined nature of 50/50 advisory revenue, it is not surprising that revenue proportions (Panel C: Table 19) typically are between those of corn and soybeans. The average proportion for 1995-2000 is 57% versus the 24-month benchmark and 66% versus the 20-month benchmark, indicating a marginal to better than average chance of advisory revenue beating market benchmark revenue. The proportion of advisory revenues above the USDA farmer benchmark exceeds 50% each crop year and averages 77% over 1995-2000. This indicates a sizable chance of advisory revenue beating USDA farmer benchmark revenue. It is interesting to note that 100% of the advisory programs in 1998 generated revenue that exceeded the USDA farmer benchmark, despite the fact that less than 100% did so in corn and soybeans. This simply reflects a situation where some programs had gains above the farm benchmark in one commodity that more than offset the losses below the benchmark in the other commodity.

Overall, the directional performance results over 1995-2000 suggest several key findings. First, advisory programs in corn do not consistently beat market benchmarks, but they do consistently beat the farmer benchmark. Second, advisory programs in soybeans tend to beat both market and farmer benchmarks. Third, in terms of 50/50 revenue, advisory programs only marginally beat market benchmarks, but consistently beat the farmer benchmark. So, the results provide mixed performance evidence with respect to market benchmarks and consistently positive evidence with respect to the USDA farmer benchmark.

Finally, it is interesting to compare the directional pricing performance results for market advisory programs to that of other investment professionals. Malkiel (1999) reports a typical estimate of the proportion of active mutual funds managers that beat the stock market. Specifically, he shows that only 33% of active mutual fund managers generate returns higher than the S&P 500 stock index over 1974-1998. By comparison, market advisory programs perform better, with more than half of the programs beating the market in corn and about two-thirds beating the market in soybeans. This divergence may simply reflect a unique time period in corn and soybean markets, relatively less efficient commodity markets, the skillfulness of advisory programs, or a return to risk.

The second indicator of pricing performance is the difference between the average price of advisory programs and the market or farmer benchmarks. This indicator takes into account both the direction and magnitude of differences from the benchmarks. The results found in Tables 20 and 21 basically tell the same story as those based on the proportion beating the benchmarks. Average differences from market benchmarks for corn over 1995-2000 (panel A: Table 20) are small, ranging from zero to three cents per bushel. [42] At 11¢ cents per bushel, the average difference from the farmer benchmark for corn is larger. Average differences for soybeans over 1995-2000 (panel B: Table 20) are even larger for both types of benchmarks, ranging from 13 to 17¢ per bushel versus market benchmarks and equaling 22¢ per bushel versus the farmer benchmark. Average differences for 50/50 advisory revenue range from three to seven dollars per acre for market benchmarks over 1995-2000 (Table 21). The average revenue difference versus the USDA farmer benchmark is $14 per acre. Note that the average differences can mask considerable variability across the benchmarks within a crop year and across crop years. A dramatic example of this occurred in 1998 for soybeans (Panel B: Table 20), where the average difference from the 24-month market benchmark is –4¢ per bushel, while the average difference from the USDA farmer benchmark is +64¢ per bushel.

It should be pointed out that average differences versus the farmer benchmark appear to be non-trivial from an economic decision-making perspective. For example, the average advisory return relative to the farmer benchmark ($14 per acre) is over four percent of average farmer benchmark revenue. This represents a substantial increase in the average returns to farm operator management, labor and capital in Illinois (e.g., Lattz, Cagley and Raab, 2001).

At this juncture, the findings should be considered only suggestive. The reason is that the statistical significance of the results has not been investigated. In other words, are the returns to marketing advice simply the result of random chance or do they reflect truly positive pricing performance? Standard statistical tests are available to help answer this question. In the present case, the appropriate test is the matched sample t-test of zero difference in the mean of net advisory prices and a particular benchmark. The samples are matched because the same crop year receives different “treatments” from advisory programs and benchmarks. The treatments correspond to the differing marketing strategies applied by advisory programs and the benchmarks.

Application of the t-test to average pricing performance is complicated by the fact that net prices across programs are positively related. This type of statistical test assumes that sample differences are generated independently (e.g., Griffiths, Hill and Judge, 1993, p. 152). [43] It should come as no surprise that this assumption is violated for market advisory programs. Many of the programs appear to use similar methods of analysis, and all make heavy use of similar supply and demand information (primarily from the USDA). Furthermore, alternative programs offered by the same advisory service are likely to generate similar pricing results. Statisticians call this an “implicit factor” problem.

Correlation coefficients estimated across net advisory prices provide evidence on the magnitude of the dependence problem. The sample is limited to the 17 programs active in all six crop years of the AgMAS study in order to maximize the number of time-series observations available for each pair of programs. The possible range in correlation coefficients is –1 and +1, with –1 indicating perfect negative correlation in advisory prices and +1 indicating a perfect positive correlation in advisory prices. A correlation of zero indicates no (linear) relationship. A few of the estimated correlations are negative, but, as expected, the vast majority of the correlations are positive. [44] The average correlation coefficient across all possible pairs of advisory programs (136) is 0.73 for corn, 0.78 for soybeans and 0.65 for revenue. These estimates are quite high and confirm that dependence across advisory programs is a serious problem in testing the statistical significance of average price performance. [45]

The high level of correlation across net advisory prices and revenue basically creates an information problem in the sample. Take the case of corn. There are 152 computed net advisory prices across all programs and crop years. However, the 152 net advisory prices are not independent, due to the strong positive correlation across programs. The key question is the amount of independent information contained in the sample of 152 net advisory prices. It is not possible to come up with a precise estimate, but it is certainly far less than 152 observations. Similar logic holds for soybeans and 50/50 advisory revenue.

The bottom-line from this discussion is that an assumption of independence for advisory prices and revenue will overstate the reliability of sample estimates. This in turn will bias statistical tests towards a conclusion that pricing performance is significantly positive. The approach taken here to deal with the problem is “conservative.” Specifically, statistical tests assume the minimum possible number of independent observations in the sample. This minimum is six observations, one for each crop year. The tests are conservative since conclusions are based on the minimal possible assumption about the amount of information in the sample. If test results based on this conservative assumption indicate statistical significance, then a high degree of confidence can be placed on conclusions. The cost of this approach is an increased probability that positive pricing performance is mistakenly attributed to chance.

Implementing the conservative testing approach is straightforward. First, the average net advisory price or revenue is computed across all programs active in a crop year, and it is considered the return for an “average” advisory program. Second, the averaging process is repeated for each of the crop years to form a sample of six observations for the average advisory program. These averages can be found in Tables 12 through 14 under the “Descriptive Statistics” heading. Third, benchmark prices are subtracted from each of the average advisory prices or revenues. Fourth, a matched sample t-test is applied to the six difference observations to determine if average price performance is statistically significant.

Differences from the benchmarks each crop year and statistical test results for an average advisory program are presented in Table 22. Note that the average differences reported in Table 22 are nearly identical to those reported in Tables 20 and 21. This outcome is not surprising. The average differences in Table 22 assume an equal weighting of the six crop years, while the average differences in Tables 20 and 21 assume an equal weighting of each net advisory price or revenue in the sample. The two types of averages differ only because the number of advisory programs changes across crop years. Since this change is quite small across crop years, the difference in the two types of averages is negligible.

The impact of the conservative approach to testing the significance of average differences is reflected in the standard error estimates. This statistic measures the “typical” error, without regard to sign, in estimating the average difference between advisory programs and a particular benchmark (Mirer, 1995, p. 238). [46] For example, the standard error estimate for the average difference in soybeans versus the 24-month market benchmark indicates that the typical error in estimating the true difference, without regard to sign, is five cents per bushel. A measure of reliability is needed because a sample is being used to make an inference about the “true” population difference, and the sample will not perfectly reflect the characteristics of the population. This is the essence of the role of random chance in estimation. The key point in this regard is that standard error estimates vary inversely with sample size. [47] As a result, standard error estimates (typical estimation errors) will be much larger if it is assumed that six independent observations are available as opposed to, say, 152 independent observations.

With this background, the statistical test results in Table 22 can be considered. The relevant information in the sample for testing statistical significance is summarized by the t-statistic, which is just the ratio of the average difference estimate to the standard error estimate. The two-tail p-value indicates the probability of observing a value of the t-statistic (or higher in absolute value) across many random samples. It is usually argued that p-values must be equal to or smaller than 0.05 to confidently conclude that average differences do not equal zero (Griffiths, Hill and Judge, 1993, p. 134). Stated differently, there should be less than a 1 out of 20 chance that the wrong conclusion is reached. In corn, the p-values for average differences versus both market benchmarks are substantially larger than 0.05, so it can be concluded that average differences are insignificantly different from zero. Just the opposite conclusion is reached versus the USDA farmer benchmark. The p-value of 0.02 indicates the average difference of 11¢ per bushel in corn is highly significant. Soybean results versus the market benchmarks are mixed, with statistical significance indicated for the average difference from the 20-month benchmark, but not the 24-month benchmark. With a p-value of 0.07, the 24-month average difference just misses the cutoff for significance. Like corn, the average difference of 23¢ per bushel in soybeans versus the USDA farmer benchmark is significantly different from zero. Test results for 50/50 advisory revenue follow a similar pattern as in soybeans. Overall, the test results indicate no evidence of statistically significant average price performance in corn versus market benchmarks, mixed evidence of significant performance in soybeans and 50/50 advisory revenue versus market benchmarks and consistent evidence of significant performance versus the farmer benchmark in corn, soybeans and 50/50 advisory revenue.

When viewing statistical test results, it is always important to assess whether the nature of the sample information or the comparisons biases the results in one direction or the other. There is in fact a systematic trend in corn and soybean price movements during the sample period that has an important impact on the tests results. Figure 12 shows the average pattern of corn and soybean prices over the 24-month marketing window for the 1995-2000 crop years. These charts are based on the same harvest equivalent forward and spot cash prices (including LDP/MLGs) used to compute net advisory prices and the market benchmarks. The downward trend in corn and soybean prices over the 24-month window is substantial, with pre-harvest highs in corn and soybean prices about 70¢ and 90¢ per bushel, respectively, higher than post-harvest lows. A marketing strategy that systematically priced more heavily in the pre-harvest period relative to the post-harvest period would have generated much higher returns than a strategy that did not.

Now consider the average marketing profiles for corn and soybeans shown in Figure 13. The market benchmark and advisory program profiles were presented earlier in Figure 6 and the USDA marketing weights were presented in Figure 7. Since the USDA marketing weights represent grain deliveries rather than pricing, a hypothetical marketing profile for farmers also is presented (labeled “Farmers ?”). It is based on a similar marketing window as the market benchmarks and advisory programs, but reflects substantially less pricing in the pre-harvest period. [48] In light of the downward price trends, the marketing profiles make it is easy to understand why market benchmarks and advisory programs generated higher prices than the farmer benchmark over the last six crop years.

The key question is whether the price trends and marketing patterns of the last six years provide a reliable picture of the future. Scenario analysis is helpful in illustrating the range of possible outcomes. Consider first a scenario where future upward price trends offset the downward price movements of the last six crop years and advisors and farmers do not significantly change their marketing behavior. Future performance results under this scenario will be just the opposite of those for the last six crop years because farmers will benefit relatively more than advisors from the upward price trends. Of course, it is possible for advisory programs to outperform farmers in an environment of rising prices if they time strategy changes better than farmers. Consider an alternative scenario where downward price trends continue to be the norm and advisors and farmers do not significantly change their marketing behavior. Future performance results basically will be the same as those observed over the 1995-2000 sample period. Farmers could equal the performance of advisors under a downward price trend scenario if they systematically increase pre-harvest pricing. These scenarios show that future performance differences could range from complete reversal to no change, depending on future price trends.

In sum, it is difficult to know whether a high degree of confidence should be placed on the average price results for 1995-2000. Pricing performance depends on a complex set of variables that include corn and soybean price behavior, advisory program strategies and the marketing behavior of farmers. It is on open question whether the behavior of these variables in the last six crop years provides a reliable guide for the future. The persistence of downward price trends generally observed over 1995-2000 is an especially hotly debated issue. While the results clearly provide some evidence on the pricing performance of advisory programs, there is simply no replacement for a larger sample of crop years when attempting to reach firm conclusions. In particular, more observations are needed on crop years with rising prices. Longer-term evidence on the performance of farmers versus the market would also be especially helpful.

Even if average price results for 1995-2000 persist into the future, the results will be open to differing interpretations. The reason is that the definition of “skill” and “luck” in pricing performance depends on the market theory considered. Based on efficient market theory, marketing skill is defined only as the component of average advisory price that exceeds a market benchmark. The component of average advisory price represented by the difference between the market benchmark price and the farmer benchmark price is considered luck. If this difference is positive, it should not be attributed to the marketing skill of advisory programs under efficient market theory because a simple no-information strategy of marketing equal amounts each time period could have achieved the same results. Based on behavioral market theory, marketing skill is defined as the entire difference between the average advisory price and the farmer benchmark, assuming the difference is positive. A luck component is not defined in this framework. Regardless of the source of performance improvement over the farmer benchmark, it is regarded as marketing skill.

Figure 14 shows the division of average price performance over 1995-2000 into skill and luck components based on efficient market theory and behavioral market theory. The number at the top of each bar is the average difference between advisory price or revenue and the USDA farmer benchmark over 1995-2000 (see Tables 20 and 21). The skill and luck components are computed as a proportion of this average difference to facilitate comparison across prices and revenue. Based on efficient market theory and the 24-month market benchmark (Panel A), only 5% of the 11¢ per bushel average difference between advisory prices and the farmer benchmark is attributed to skill. The comparison is more favorable for soybeans, with about 50% of the 22¢ per bushel average difference between advisory prices and the farmer benchmark in soybeans attributed to skill. About 25% of the $14 per acre average difference between advisory revenue and farmer benchmark revenue is attributed to skill. The components attributed to skill versus luck are higher for the 20-month market benchmark (Panel B), but do not change conclusions markedly. In contrast, behavioral market theory (Panel C) attributes all of the average differences between advisory programs and the farmer benchmark to skill. The differing interpretations cannot be reconciled, as they reflect profoundly different views about market behavior

Comparison of average advisory prices or revenues to benchmarks is an important indicator of performance. However, average price or revenue comparisons may not provide a complete picture of performance. For example, two advisory programs can generate the same average advisory price, but the risk of the programs may differ substantially. The difference in risk may be the result of using different pricing tools (cash, forward, futures or options), different timing of sales and variation in the implementation of marketing strategies.

A number of theoretical frameworks have been developed to analyze decision-making under risk. One of the simplest and most popular is the mean-variance (EV) model, which uses variance as a measure of risk. The basic idea in this case is to look at risk as the chance farmers will fail to achieve the net price they expect based on following an advisory program. This approach to quantifying risk does not measure the possibility of loss alone. Risk is seen as uncertainty: the likelihood that what is expected will fail to happen, whether the outcome is better or worse than expected. So an unexpected return on the upside or the downside – a net price of $2.50 or $1.50 per bushel when a net price of $2.00 per bushel is expected – counts in determining the risk of an advisory program. Thus, an advisory program whose net price does not depart much from its expected (mean) price is said to carry little risk. In contrast, an advisory program whose net price is quite volatile from year-to-year, often departing from expected net price, is said to be quite risky.

To apply the EV model to a particular decision, either distributions of outcomes must be normal or decision-makers must have quadratic utility functions (Hardaker, Huirne and Anderson, 1997, p.141). If either or both of these conditions hold, then risky choices can be divided into efficient and inefficient sets based on the famous EV efficiency rule: if the mean of choice A is greater than or equal to the mean of choice B, and the variance of A is less than or equal to the variance of B, with at least one strict inequality holding, then A is preferred to B by all risk-averse decision makers. Since quadratic utility has the unlikely characteristic that absolute risk aversion increases with the level of the outcome, application of the EV model usually is based upon an assumption of normally distributed outcomes. This presents a potential problem in the case of market advisory programs that employ options strategies. Such strategies are designed to create non-normal price distributions by truncating undesirable prices, either on the downside or the upside, or both. Fortunately, simulation analysis suggests that the EV model produces reasonably accurate results even in cases where options strategies are employed (Hanson and Ladd, 1991; Ladd and Hanson, 1991; Garcia, Adam and Hauser, 1994).

The basic data needed for assessing market advisory pricing performance in an EV framework are presented in Table 23. For each advisory program tracked in all six crop years of the AgMAS study, the six-year average net advisory price or revenue and standard deviation of net advisory price or revenue is reported. The average price and standard deviation of the three benchmarks also are reported. Standard deviation is substituted for variance as the measure of risk because it easier to understand. [49] Performance results are the same whether standard deviation or variance is used to measure risk (Hardaker, Huirne and Anderson, 1997, p.143), hence the use of the simpler measure. Standard deviation estimates can be thought of as the “typical” variation in net advisory prices from year-to-year. The larger the standard deviation for an advisory program, the less likely a farmer is to get exactly the net price expected, though it is possible by chance to get a higher price instead of a lower one for any particular time period.

The sample of advisory programs for the EV analysis is limited to those tracked all six crop years in order to maximize the number of observations available to estimate risk (standard deviation). [50] Even with this restriction, six observations would appear to be a relatively small sample for estimating the risks of market advisory programs. However, as noted in the introduction, Anderson (1974) explored the reliability of agricultural return-risk estimates based on limited data and found the surprising result that even as few as three or four observations can be very useful. Nonetheless, the standard deviations reported in Table 23 may be somewhat inaccurate estimators of the true risks of advisory programs. With that in mind, the standard deviations suggest that the risk of advisory programs varies substantially. In corn, the standard deviations range from a low of $0.18 per bushel to a high of $0.67 per bushel. In soybeans, the standard deviations range from a low of $0.35 per bushel to a high of $1.03 per bushel. Finally, revenue standard deviations for the 17 programs range from a low of $18 per acre to a high of $44 per acre. Standard deviations of the benchmark prices tend to be near the average standard deviation of the 17 advisory programs for corn, soybeans and 50/50 advisory revenue.

The average price and risk (standard deviation) for individual advisory programs and the benchmarks are plotted in Figures 15 through 17. Panel A in each of the figures is divided into four quadrants based on the average price (or revenue) and standard deviation of the 24-month market benchmark, while panel B is divided into four quadrants based on the average price (or revenue) and standard deviation of the USDA farmer benchmark. Advisory programs in the upper left quadrant of each chart have a higher average price (or revenue) and less risk than the benchmark, which is the most desirable outcome from a farmer’s perspective. According to the EV efficiency rule introduced earlier, advisory programs in this quadrant are said to “dominate” the 24-month market benchmark or the USDA farmer benchmark. Advisory programs in the lower right quadrant have a lower price and more risk than the benchmark, which is the least desirable outcome from a farmer’s perspective. The 24-month market benchmark or the USDA farmer benchmark dominates an advisory program located in this quadrant. The two remaining quadrants reflect a higher price and more risk than the benchmarks or a lower price and less risk than the benchmarks. A farmer may prefer an advisory program to the benchmark in either of these two quadrants, but this depends on personal preference for risk relative to average price.

The data plotted in panel A of Figure 15 show that only 1 of the 17 advisory programs in corn dominates the 24-month market benchmark (upper left quadrant). Six advisory programs are dominated by the 24-month market benchmark (lower right quadrant). In contrast, panel B in Figure 15 indicates stronger performance, with 10 of the 17 advisory programs in corn dominating the USDA farmer benchmark (upper left quadrant). No program in corn is dominated by the USDA farmer benchmark (lower right quadrant).

The data plotted in panel A of Figure 16 indicate that 6 of the 17 advisory programs in soybeans dominate the 24-month market benchmark (upper left quadrant), while only four advisory programs are dominated by this market benchmark (lower right quadrant). Panel B in Figure 16 again suggests stronger performance, with 13 of the 17 advisory programs dominating the USDA farmer benchmark (upper left quadrant). Only one program in soybeans is dominated by the USDA farmer benchmark (lower right quadrant).

Similar patterns are evident for 50/50 advisory revenue. Panel A of Figure 17 shows that in terms of revenue only 1 of the 17 advisory programs dominates the 24-month market benchmark (upper left quadrant), while 9 of the 17 are dominated by this market benchmark (lower right quadrant). Panel B in Figure 17 shows 8 of the 17 programs dominates the USDA farmer benchmark (upper left quadrant), and no program is dominated by this farmer benchmark (lower right quadrant).

A key motivation for this analysis is to determine whether consideration of risk alters performance conclusions based only upon average price. This is most easily assessed by comparing the proportion of advisory programs that beat the benchmarks in terms of price in Table 19 with the proportion of programs that dominate the benchmarks in terms of average price and risk (upper left quadrant proportions in Figures 15-17). For corn, 51% of the advisory programs beat the 24-month market benchmark based on price alone over 1995-2000. This drops to 6% when risk is considered. The same proportions for the USDA benchmark in corn drop from 73 to 59%. For soybeans, 60% of the advisory programs beat the 24-month market benchmark based on price alone over 1995-2000, while only 35% do so when risk is considered. The proportions for the USDA benchmark in soybeans actually increase from 74 to 76%. For 50/50 advisory revenue the declines are the steepest, with 56% of the advisory programs beating the 24-month market benchmark based on price alone over 1995-2000 and only 6% doing so when risk is considered. The proportions for the USDA benchmark in terms of advisory revenue decrease sharply from 77 to 47%. Overall, the results indicate that consideration of risk tends to weaken conclusions about the performance of advisory programs.

The comparisons also imply that at least part of the skill components discussed in the previous section (Figure 14) can be attributed to risk. From an efficient market theory perspective, the limited size of the skill premium in corn and 50/50 advisory revenue appears to be largely explained by risk. The more substantial size of the skill premium in soybeans is only partly explained by risk. From a behavioral market theory perspective, the substantial skill premium in corn is only partly explained by risk. The skill premium based on this theoretical perspective in soybeans appears to be unaffected by the consideration of risk, while the skill premium in 50/50 advisory revenue appears to be substantially explained by risk. Overall, the comparisons reveal the importance of considering risk in performance evaluations of market advisory programs.

Two other issues with respect to risk need to be considered. The first is the sensitivity of EV comparisons to the alternative market benchmarks. Comparing the results for the 24-month and 20-month market benchmarks, the number of programs in the upper-left quadrant increases from one to six for corn (panel A: Figure 15), from six to ten for soybeans (panel A: Figure 16), and from one to six for 50/50 advisory revenue (panel A: Figure 17). These comparisons suggest EV performance results are somewhat sensitive to changing the market benchmark specification. Nonetheless, the qualitative implications of the EV comparisons are not substantially different for the two market benchmarks. The second issue is the statistical significance of EV performance differences. Paralleling the argument in the previous section, it is possible that positive performance of advisory programs in an EV context is due to random chance. Collender (1989) developed a statistical test that can be applied to EV comparisons. The joint confidence regions for this test are exceptionally large and indicate advisory program performance is not significantly different from any of the benchmarks. [51] With only six observations to estimate both the mean and standard deviation, the power of this particular test to detect positive performance is low.

Finally, the mean-variance evaluation presented in this section can be extended to portfolios of advisory programs. For example, a soybean portfolio might consist of 50% marketed by advisory program #1, 25% marketed by advisory program #2 and 25% marketed by advisory program #3. Within a mean-variance framework, modern portfolio theory (MPT) can be used to form portfolios that have the highest return for a given level of risk. MPT produces optimal portfolios by taking advantage of the diversification opportunities available through combinations of advisory programs. In fact, it is possible for a portfolio of advisory programs to generate higher prices and less risk than a benchmark, even if individual advisory programs that make up the portfolio do not. The potential improvement in performance depends on the degree that net advisory prices or revenues are uncorrelated. Application of MPT to market advisory programs represents an interesting area of future research.

Even if, as a group, advisory programs generate positive marketing returns, there is a wide range in performance for any given year. For example, soybean net advisory prices for 1995 vary from $5.71 per bushel to $7.94 per bushel (see Table 13). While this example is one of the most dramatic, the variation across advisors in other cases is substantial. This raises the important question of the predictability of advisory program performance from year-to-year. In other words, is past performance indicative of future performance? Three types of predictability tests are used to answer this question: i) the predictability of “winner” and “loser” categories across crop years, ii) the correlation of advisory program ranks across crop years and iii) the differences between prices for “top” and “bottom” performing advisory programs across crop years. The testing procedures have been widely applied in studies of financial investment performance (e.g., Elton, Gruber, and Rentzler, 1987; Irwin, Zulauf and Ward, 1994; Lakonishok, Shleifer and Vishny, 1992; Malkiel, 1995).

The first test of predictability is based on placing advisory programs into “winner” and “loser” categories across adjacent crop years. This non-parametric test is robust to outliers, which is important when analyzing predictability across all advisory programs. For a given commodity, the first step in this testing procedure is to form the sample of all advisory programs that are active in adjacent crop years. The second step is to rank each advisory program in the first year of the pair (e.g., t = 1997) based on net advisory price. For example, the program with the highest net advisory price is ranked number one, and the program with the lowest net advisory price is assigned a rank equal to the total number of programs for that commodity in the given crop year. Then the programs are sorted in descending rank order. The third step is to form two groups of programs in the first year of the pair: winners are those programs in the top half of the rankings and losers are programs in the bottom half. The third step is to rank each advisory program in the second year of the pair (e.g., t +1 = 1998) based on net advisory price and once again form winner and loser groups of programs. The fourth step is to compute the following counts for the advisory programs in the pair of crop years: winner t-winner t+1, winner t-loser t+1, loser t-winner t+1, loser t-loser t+1. If advisory program performance is unpredictable, approximately the same counts will be found in each of the four combinations. The appropriate statistical test in this case is known as Fisher’s Exact Test (Conover, 1999, pp.188-189). [52]

Results of the winner and loser predictability test are shown in Table 24. Winner and loser counts for individual crop years indicate a modest difference, at best, in the chance of a winner or loser in one period being a winner or loser in the subsequent period. As an example, consider the results for corn in 1997 and 1998. Of the eleven winners (top half) in 1997, six are winners in 1998 and five are losers (bottom half). Of the twelve losers in 1997, five are winners in 1998 and seven are losers. In other words, the conditional probability of a winner from 1997 repeating in 1998 is 55% (6/11) and the conditional probability of a loser from 1997 repeating in 1998 is 56% (7/12). These probabilities are only slighter higher than what would result from flipping a coin (randomness). There is only one case (50/50 revenue, 1999 vs. 2000) where individual year counts are significantly different from the equal distribution expected under an assumption of no predictability. These results imply that the performance of winning and losing advisory programs is not predictable through time.

Pooled counts for 1995-2000 also are shown in Table 24. Pooling provides an overall test of predictability and should improve the power of the tests by increasing sample size. Results after pooling are strongest for soybeans, where the conditional probability of a repeat winner is 61% and the conditional probability of a repeat loser is 62%. Conditional probabilities are less than or equal to 60% for corn and 50/50 advisory revenue. The p-value for pooled test results in corn (0.09) suggests marginally significant predictability, while the p-value for soybeans (0.02) indicates significant predictability. No predictability is indicated for 50/50 advisory revenue. The pooled corn and soybean test results need to be viewed cautiously given that Fisher’s Exact Test assumes sample observations are independent. As discussed in the section on average price performance, this clearly is not the case, and therefore, the p-values reported in Table 24 overstate the true significance of the results. In this light, the pooled results provide only limited evidence of predictability in the performance of winning and losing advisory programs through time.

While predictability may be limited or non-existent across all advisory programs, it is possible for sub-groups of advisory programs to exhibit predictability. In particular, predictability may only be found at the extremes of performance. That is, only top-performing programs in one year may tend to perform well in the next year, or only poor-performing programs may perform poorly in the next year, or both. This is the motivation for the second test of predictability, which is based on the correlation between ranks of all advisory programs active in adjacent pairs of crop years. For a given commodity, the first step in this testing procedure is to once again form the sample of all advisory programs that are active in both adjacent crop years. The second step is to rank each advisory program in the first year of the pair (e.g., t = 1997) based on net advisory price. Then the programs are sorted in descending rank order. The third step is to sort and rank the sample of programs in the second year of the pair (e.g., t + 1 = 1998). The fourth step is to compute the correlation coefficient between ranks for the two adjacent crop years. If advisory program performance is unpredictable, the estimated correlation will be near zero. Assuming the standard error of the correlation coefficient is approximately equal to , the appropriate statistical test is a Z-test.

Results of the rank correlation predictability test are presented in Table 25. Rank correlation coefficients for corn range from of 0.01 to 0.53. Statistically significant correlations are found for three of the five comparisons in corn. The range of rank correlation coefficients for soybeans, 0.10 to 0.65, is similar to the range for corn. However, statistically significant correlations are found for only one of the five comparisons in soybeans. Rank correlation coefficients for 50/50 revenue have the widest range, from 0.00 to 0.72. Statistically significant correlations are found for two of the five revenue comparisons. Once again, caution should be used when considering the reported p-values, as they overstate the significance of the rank correlation estimates due to the dependence across advisory programs. Average rank correlation coefficients across the five comparisons are nearly identical for corn, soybeans and 50/50 advisory revenue. With values of either 0.34 or 0.35, the average rank correlations suggest some predictability in the pricing performance of top- and bottom-performing market advisory programs. It is interesting to observe that the strongest evidence of predictability is concentrated in the last three crop years of the sample.

Given the rank correlation tests results, it is important to determine the magnitude of predictability in top- and bottom-performing advisory programs. Hence, the third test of predictability is based on the difference between net advisory prices for top- and bottom-performing advisory programs across adjacent crop years. For a given commodity, the first step in this testing procedure is to sort programs by net advisory price in the first year of the pair and group programs by quantiles (thirds and fourths). The second step is to compute the average net advisory price for the quantiles in the second year of the pair. Note that the same programs make up the quantiles in the first and second year of the pair. For example, the average price of the top fourth quantile formed in 1995 is computed for 1996. The third step is to compute the difference in average price for the top- and bottom-performing quantiles. If performance for the top- and bottom-performing quantiles is the same, the difference will equal zero. The appropriate statistical test in this case is a matched sample t-test of the difference in the means of the top- and bottom-performing quantiles. There are a total of five comparisons (1995 vs. 1996, 1996 vs. 1997, 1997 vs. 1998, 1998 vs. 1999 and 1999 vs. 2000), so there are four degrees of freedom for the t-test. Since differences are computed for an “average” advisory program in top- and bottom-performing quantiles, dependence across individual advisory programs is not an issue, and p-values for the t-test are unbiased. Carpenter and Lynch (1999) recommend this test because it is well-specified and among the most powerful in their comparison of several predictability tests for mutual funds.

Results for the t-test of predictability are shown in Table 26. The first column under each commodity heading shows the average price of the different quantiles in the first year of the comparisons (five in total). The average price for the first year is “in-sample” because this is the formation year for the quantiles. The second column under each heading reports the average price of the same quantiles in the second year of the comparisons. The average price for the second year is “out-of-sample” because this is the year after formation of the quantiles. In all cases, the average price or revenue of the top quantile relative to the bottom quantile declines substantially from the first to the second year of the comparisons. Nonetheless, the average difference between top- and bottom-performing quantiles for the second year of the pair is consistently positive. For example, programs in the top third beat the bottom third in the second year by an average of 14¢ per bushel in corn, 29¢ per bushel in soybeans and $14 per acre for revenue. Average differences are significantly different from zero for both cases in corn and 50/50 revenue and marginally significant in soybeans. Average prices for the top quantile out-of-sample also exceed benchmark prices for the same period (1996-2000). Top third returns beat the 24-month market benchmark by an average of 5¢ per bushel in corn, 26¢ per bushel in soybeans and $9 per acre for 50/50 revenue. Top fourth returns beat the 24-month market benchmark by an average of 9¢ per bushel in corn, 31¢ per bushel in soybeans and $12 per acre for 50/50 revenue.

The quantile results provide evidence that the performance of top- and bottom-performing market advisory programs can be predicted across adjacent crop years. However, the evidence is not sufficient to conclude that performance predictability is useful from an economic standpoint, due to the overlapping nature of the marketing windows for each crop year. To see the point, consider the case of a farmer who uses 1995 performance results to select a top-performing advisory program. Since the 1995 marketing window ends on August 31, 1996, halfway through the 1996 marketing window and one day before the beginning of the 1997 marketing window, the farmer could not implement their selection of an advisory program until the 1997 crop year. Performance would have to persist across three crop years, 1995, 1996 and 1997, for a farmer to benefit from the predictability.

Quantile results for non-overlapping crop years are shown in Table 27. The testing procedure is the same as before, except there are only four comparisons (1995 vs. 1997, 1996 vs. 1998, 1997 vs. 1999, and 1998 vs. 2000) and three degrees of freedom for the t-test. The results for non-overlapping crop years continue to show a positive difference between top- and bottom-performing quantiles in the second year of the pair. However, the magnitude of the differences is substantially smaller than in the case of adjacent crop years. For example, programs in the top fourth beat the bottom fourth in the second year only by an average of 1¢ per bushel in corn, 14¢ per bushel in soybeans and $1 per acre for revenue. None of the average differences are significantly different from zero. These results indicate predictability of pricing performance for top and bottom advisory programs is short-lived, in the sense that performance does not persist long enough to be taken advantage of by farmers.

The predictability results presented so far are all based on individual crop year comparisons. It is possible for performance to be predictable over long time horizons, but unpredictable over short horizons due to a large amount of “noise” in performance from year-to-year (e.g., Summers, 1986). This is consistent with the argument that over the long-term “cream rises to the top” in terms of performance. To assess long-term predictability, the sample is limited to the 17 programs active in all six crop years of the study. Next, net advisory prices are averaged for each of the 17 programs for the first three crop years of the sample (1995-1997) and the second three years (1998-2000). The three tests of predictability are then applied to the two sets of averages. The results are striking, in that virtually no evidence of predictability is found for any of the tests. Winner-loser counts are quite close to what is expected under randomness, rank correlations are all insignificantly different from zero (corn and soybean correlations are actually negative), and the average difference between top- and bottom-performing programs is very small (zero difference for 50/50 advisory revenue). [53] These results occur despite the fact that the same program is ranked first in both sub-periods for corn and 50/50 advisory revenue.

The test results presented in this section provide little evidence that the pricing performance of advisory programs can be usefully predicted from past performance. This conclusion does not mean it is impossible to predict advisory program performance. There may be other variables that are useful for predicting performance. Chevalier and Ellison (1999) study whether mutual fund performance is related to characteristics of fund managers that indicate ability, knowledge or effort, and find that managers who attended higher-SAT undergraduate institutions generate systematically higher returns. Barber and Odean (2000) examine the trading records of individual stock investors and report that frequent trading substantially depresses investment returns. Similar factors, such as education of advisors, cash only programs versus futures and options programs, frequency of futures and options trading, or storage costs, may be useful in predicting the performance of market advisory programs.

Surveys suggest that numerous farmers view market advisory services as an important tool in managing price and income risk. As a result, farmers need information on the performance “track record” of market advisory services to help them identify successful alternatives for marketing and price risk management. The Agricultural Market Advisory Service (AgMAS) Project was initiated in 1994 with the goal of providing unbiased and rigorous evaluation of market advisory services.

The purpose of this research report is to evaluate the pricing performance of market advisory services for the 1995-2000 corn and soybean crops. No fewer than 23 market advisory programs are available for each crop year. While the sample of advisory services is non-random, it is constructed to be generally representative of the majority of advisory services offered to farmers. Further, the sample of advisory services includes all programs tracked by the AgMAS Project over the study period, so pricing performance results should not be plagued by survivorship bias. The AgMAS Project subscribes to all of the services that are followed and records recommendations on a real-time basis, which should prevent pricing performance results from being subject to hindsight bias.

Certain explicit assumptions are made to produce a consistent and comparable set of results across the different advisory programs. These assumptions are intended to accurately depict “real-world” marketing conditions. Several key assumptions are: i) with a few exceptions, the marketing window for a crop year runs from September before harvest through August after harvest, ii) cash prices and yields refer to a central Illinois farm, iii) storage is assumed to occur at on-farm or commercial sites, and iv) marketing loan recommendations made by advisory programs are followed wherever feasible. Based on these assumptions, the net price received by a subscriber to market advisory programs is calculated for the 1995-2000 corn and soybean crops.

Two different types of benchmarks are developed for the performance evaluations. Efficient market theory implies that the return offered by the market is the relevant benchmark. In the context of this study, a market benchmark should measure the average price offered by the market over the marketing window of a representative farmer who follows advisory program recommendations. Both a 24-month and a 20-month market benchmark are specified in order to test the fragility of performance results to different market benchmark assumptions. Behavioral market theory suggests that the average return actually achieved by market participants as an appropriate benchmark. In the context of the present study, a behavioral benchmark should measure the average price actually received by farmers for a crop. A farmer benchmark is specified based upon the USDA average price received series for corn and soybeans in Illinois. All benchmarks are computed using the same assumptions applied to advisory program track records.

Four basic indicators of performance are applied to advisory program prices and revenues over 1995-2000. The first indicator is the proportion of advisory programs that beat benchmark prices. Between 51 and 59% of the programs in corn have net advisory prices above market benchmarks over 1995-2000, while 74% of the programs have prices above farmer benchmarks. Performance is stronger in soybeans. Between 61 and 70% of advisory programs in soybeans have advisory prices above the market benchmarks over 1995-2000 and 74% are above the farmer benchmarks. Between 57 and 66% of advisory programs have revenue above the market benchmarks over 1995-2000, while 77% have revenue above the farmer benchmark. The results provide mixed performance evidence with respect to market benchmarks and consistently positive evidence with respect to the USDA farmer benchmark.

The second indicator is the difference between the average price of advisory programs and the market or farmer benchmarks. The results basically tell the same story as those based on the proportion beating the benchmarks. Average differences from market benchmarks for corn over 1995-2000 are small, ranging from zero to three cents per bushel. At 11¢ per bushel, the average difference from the farmer benchmark for corn is larger. Average differences for soybeans over 1995-2000 are even larger for both types of benchmarks, ranging from 13 to 17¢ per bushel versus market benchmarks and equaling 22¢ per bushel versus the farmer benchmark. Average differences for advisory revenue range from three to seven dollars per acre for market benchmarks over 1995-2000. The average revenue difference versus the USDA farmer benchmark is $14 per acre.

Statistical tests indicate no evidence of significant average pricing performance in corn versus market benchmarks, mixed evidence of significant performance in soybeans and 50/50 advisory revenue versus market benchmarks and consistent evidence of significant performance versus the farmer benchmark in corn, soybeans and 50/50 advisory revenue. Caution should be used when considering the results, due to the relatively small sample of crop years available for analysis. In particular, the presence of sharp downward price trends in most crop years makes it difficult to determine whether the 1995-2000 sample provides a reliable guide to future differences in pricing performance.

The third indicator is the average price and risk of advisory programs relative to benchmarks. A small number of advisory programs in corn generate a combination of average price and risk superior to market benchmarks over 1995-2000. In sharp contrast, a majority of programs in corn generate a combination of average price and risk superior to the USDA farmer benchmark. A moderate number of programs in soybeans generate a combination of average price and risk superior to market benchmarks, while most programs generate a combination superior to the USDA farmer benchmark. Relatively few advisory programs generate a combination of revenue and risk superior to market benchmarks. A moderate number of programs produce a revenue combination superior to the USDA farmer benchmark. The results indicate that consideration of risk tends to weaken performance results based only upon average price.

The fourth indicator is the predictability of advisory program performance from year-to-year. “Winner” and “loser” predictability results are similar for corn, soybeans and advisory revenue. The conditional probability of a winner (top half of programs) repeating averages 57% and the conditional probability of a loser (bottom half of programs) repeating averages 60%. These probabilities are only slighter higher than what would result from flipping a coin (randomness), and provide scant evidence that pricing performance for all advisory programs can be predicted from past performance. The performance of top- and bottom-performing programs does not appear to be predictable in a useful sense either. For example, comparisons of non-overlapping crop years show that programs in the top fourth beat the bottom fourth only by an average of 1¢ per bushel in corn, 14¢ per bushel in soybeans and $1 per acre for 50/50 advisory revenue.

Overall, the results provide an interesting picture of the performance of market advisory programs in corn and soybeans. There is limited evidence that advisory programs as a group outperform market benchmarks, particularly after considering risk. In contrast, substantial evidence exists that advisory programs as a group outperform the farmer benchmark, even after taking risk into account. Whether the superior performance of advisory programs versus the farmer benchmark is attributed to skill or luck depends on the theoretical perspective. Efficient market theory favors a luck interpretation, while behavioral market theory favors a skill interpretation. Regardless of the theoretical perspective, there is little evidence that advisory programs with superior performance can be usefully selected based on past performance.

Anderson, J.R. “Sparse Data, Estimational Reliability, and Risk-Efficient Decisions.” American Journal of Agricultural Economics, 55(1974): 564-572.

Barber, B.M. and T. Odean. “Trading is Hazardous to Your Wealth: The Common Stock Investment Performance of Individual Investors.” Journal of Finance, 55(2000): 773-806.

Brown, S.J., W. Goetzmann, R.G. Ibbotson, and S.A. Ross. "Survivorship Bias in Performance Studies." Review of Financial Studies, 5(1992): 553-580.

Carpenter, J.N. and A.W. Lynch. “ Survivorship Bias and Attrition Effects in Measures of Performance Persistence.” Journal of Financial Economics, 54(1999): 337-374.

Chevalier, J. and G. Ellison. "Are Some Mutual Fund Managers Better Than Others? Cross-Sectional Patterns in Behavior and Performance." Journal of Finance, 54(1999): 875-899.

Collender, R.N. “Estimation Risk in Farm Planning Under Risk.” American Journal of Agricultural Economics, 71(1989): 996-1002.

Conover, W.J. Practical Nonparametric Statistics, Third Edition. John Wiley and Sons: New York, 1999.

Daniel, K.D., D. Hirshleifer and A. Subrahmanyam. “Investor Psychology and Security Market Over- and Underreactions.” Journal of Finance, 53(1998): 1839-1886.

Dhuyvetter, K.C., G.L. Hamman, and J.P. Harner, III. The Economics of On-Farm Farm Storage, MF-2474, Kansas State University Agricultural Experiment Station and Cooperative Extension Service, October 2000. (http://www.agecon.ksu.edu/kdhuyvetter/pdf%20files/mf2474.pdf)

Elton, E. J., M. J. Gruber, and J. C. Rentzler. "Professionally Managed, Publicly Traded Commodity Funds." Journal of Business, 60(1987): 175-199.

Fackler, P.L., D.L. Young, and G.A. Carlson. "Estimates of Trend and Variability Patterns in U.S. Crop Yields," in Quantifying Long Run Agricultural Risks and Evaluating Farmers' Responses to Risk. Proceedings of a seminar sponsored by the Southern Regional Project S-252, Jekyll Island, Georgia, March 1993

Fama, E. "Efficient Capital Markets: A Review of Theory and Empirical Work." Journal of Finance, 30(1970): 1043-1053.

Garcia, P., B.D. Adam and R.J. Hauser. “The Use of Mean-Variance for Commodity Futures and Options Hedging Decisions.” Journal of Agricultural and Resource Economics, 19(1994): 32-45.

Gehrt, D. and Good. "Evaluation of Market Advisory Services for Corn and Soybeans." Journal of the American Society of Farm Managers and Rural Appraisers, 57(1993): 1-7.

Good, D.L., T.A. Hieronymus, and R.A. Hinton. Price Forecasting and Sales Management: Corn, Soybeans, Cattle and Hogs. Cooperative Extension Service, College of Agriculture, University of Illinois at Urbana-Champaign, 1980.

Griffiths, W.E., R.C. Hill and G.C. Judge. Learning and Practicing Econometrics. John Wiley and Sons, Inc.: New York, 1993.

Hanson, S.D. and G.W. Ladd. “Robustness of the Mean-Variance Model with Truncated Probability Distributions.” American Journal of Agricultural Economics, 73(May 1991): 436-45

Hardaker, J.B., R.B.M. Huirne and J.R. Anderson. Coping with Risk in Agriculture. CAB International: New York, 1997.

Hartzmark, M.L. "Returns to Individual Traders of Futures: Aggregate Results." Journal of Political Economy, 95(1987): 1292-306.

Hill, L., E. Kunda, and C. Rehtmeyer. “Price Related Characteristics of Illinois Grain Elevators, 1982,” AE-4561, Department of Agricultural Economics, University of Illinois at Urbana-Champaign, September 1983.

Hirshleifer, D. “Investor Psychology and Asset Pricing.” Journal of Finance, 56(2001): 1533-1597.

Hurburgh, Jr., C.R., C.J. Bern, W.F. Wilcke, M.E. Anderson. “Shrinkage and Corn Quality Changes in On-Farm Handling Operations.” Transactions of the ASAE, 26(1983): 854-1857

Irwin, S.H., C.R. Zulauf, and B.L. Ward. "The Predictability of Managed Futures Returns." Journal of Derivatives, 2(1994): 20-27.

Jackson, T.E., S.H. Irwin and D.L. Good. "1996 Pricing Performance of Market Advisory Services for Corn and Soybeans," AgMAS Project Research Report 1998-01, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, January 1998, (/agmas/reports/index.html)

Jaffe, J.F. and J.M. Mahoney. "The Performance of Investment Newsletters." Journal of Financial Economics, 53(1999): 289-307.

Jirik, M.A., S.H. Irwin, D.L. Good, T.E. Jackson, and J. Martines-Filho. “The 1995 Through 1998 Pricing Performance of Market Advisory Services for Wheat,” AgMAS Project Research Report 2000-02, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, June 2000. (/agmas/reports/index.html)

Kastens, T.L. and T.C. Schroeder. "Efficiency Tests of July Kansas City Wheat Futures." Journal of Agricultural and Resource Economics, 21(1996): 187-198.

King, R.P., L.S. Lev and W.E. Nefstad. "A Position Report for Farm-Level Marketing Management." Review of Agricultural Economics, 17(1995): 205-212.

Ladd, G.W. and S.D. Hanson. “Price Risk Management with Options: Optimal Market Positions and Institutional Value.” Journal of Futures Markets, 11(1991): 737-750.

Lakonishok, J., A. Shleifer, and R.W. Vishny. "The Structure and Performance of the Money Management Industry." Brookings Papers: Microeconomics, (1992): 339-391.

Lattz, D.H., C.E. Cagley, and D.D. Raab. Summary of Illinois Farm Records for 2000, Circular 1375B, University of Illinois Extension, 2001.

Leamer, E. E. “Let’s Take the Con Out of Econometrics.” American Economic Review, 73(1983):31-43.

Lence, S.H. and M.L. Hayenga. “On the Pitfalls of Multi-year Rollover Hedges: The Case of Hedge-to-Arrive Contracts.” American Journal of Agricultural Economics, 83(2001): 107-119.

Malkiel, B.G. “Returns from Investing in Equity Mutual Funds 1971 to 1991.” Journal of Finance, 50(1995): 549-572.

Malkiel, B.G. A Random Walk Down Wall Street. W.W. Norton and Company: New York, 1999.

Marcus, A.J. “The Magellan Fund and Market Efficiency.” Journal of Portfolio Management, 16(1990): 85-88.

Marten, J. "Farmers Want Market News, Not Advice." Farm Journal Extra, June 1994.

Martines-Filho, J. Pre-Harvest Marketing Strategies for Corn and Soybeans: A Comparison of Optimal Hedging Models and Market Advisory Service Recommendations. Unpublished Ph.D. dissertation, The Ohio State University, 1996.

Martines-Filho, J., S.H. Irwin, D.L. Good, B.G. Stark, W. Shi, R.L. Webber, L.A. Hagedorn, and S.L. Williams. “Advisory Service Marketing Profiles for Corn Over 1995-1999,” AgMAS Project Research Report 2002-02, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, forthcoming, May 2002a.

Martines-Filho, J., S.H. Irwin, D.L. Good, B.G. Stark, W. Shi, R.L. Webber, L.A. Hagedorn, and S.L. Williams. “Advisory Service Marketing Profiles for Soybeans Over 1995-1999,” AgMAS Project Research Report 2002-03, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, forthcoming, May 2002b.

Martines-Filho, J., D.L. Good and S.H. Irwin. “1999 Pricing Performance of Market Advisory Services for Corn and Soybeans,” AgMAS Project Research Report 2000-04, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, December 2000. (/agmas/reports/index.html)

Metrick, A. "Performance Evaluation with Transactions Data: The Stock Selection of Investment Newsletters." Journal of Finance, 54(1999): 1743-1776.

Mirer, T. W. Economics and Statistics, Third Edition. Prentice Hall, Englewood Cliffs, NJ, 1995.

Norvell, J.M. and D.H. Lattz. “Value-Added Crops, GPS Technology and Consultant Survey: Summary of a 1998 Survey to Illinois Farmers,” Working Paper, College of Agricultural, Consumer, and Environmental Sciences, University of Illinois, July 1999.

Otte, J. "Marketing Matters -- How Well Do Market Advisors Deliver?" Prairie Farmer, July 19, 1986.

Patrick, G.F. and S. Ullerich. “Information Sources and Risk Attitudes of Large Scale Farmers, Farm Managers, and Agricultural Bankers.” Agribusiness, 12(1996): 461-471.

Patrick, G.F., W.N. Musser, and D.T. Eckman. "Forward Marketing Practices and Attitudes of Large-Scale Midwestern Grain Farmers." Review of Agricultural Economics, 20(1998): 38-53.

Pennings, J.M.E., D.L. Good, S.H. Irwin and J.K. Gomez. “The Role of Market Advisory Services in Crop Marketing and Risk Management: A Preliminary Report of Survey Results,” AgMAS Project Research Report 2001-02, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, March 2001. (/agmas/reports/index.html)

Powers, L. "How to Measure Your Pro's Performance." Top Farmer, April 1993, p. 17.

Schroeder, T.C., J.L. Parcell, T.L. Kastens, and K.C. Dhuyvetter. "Perceptions of Marketing Strategies; Farmers vs. Extension Economists." Journal of Agricultural and Resource Economics, 23(1998): 279-293.

Sharpe, W.F., G.J. Alexander, and J.V. Bailey. Investments, Sixth Edition. Prentice Hall: Upper Saddle River, New Jersey, 1999.

Summers, L.H. “Does the Stock Market Rationally Reflect Fundamental Values?” Journal of Finance, 41(1986):591-601.

Zanini, F.C. Estimation of Farm-Level Corn and Soybean Yield Distributions in Illinois. Ph.D. Dissertation, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, 2001.

Zulauf, C.R. and S.H. Irwin. “Market Efficiency and Marketing to Enhance Income of Crop Producers.” Review of Agricultural Economics, 20(1998): 308-331.

Wisner, R.N., E.N. Blue and E.D. Baldwin. “Preharvest Marketing Strategies Increase Net Returns for Corn and Soybean Growers.” Review of Agricultural Economics, 20(1998): 288-307.

Williams, J.C. and B.D. Wright. Storage and Commodity Markets. Cambridge University Press: Cambridge, 1991.

The net advisory prices and benchmarks computed by the AgMAS Project are designed to reflect “real-world” marketing conditions and assure that net advisory service prices and benchmarks are computed on a rigorously comparable basis. This latter point is especially important, as performance evaluations must compare “apples to apples” and not “apples to oranges.” Comparison problems may arise if prices computed by an individual farmer or another market advisory service are compared to AgMAS net advisory prices and benchmarks.

First, and foremost, AgMAS net advisory prices and benchmarks are stated on a harvest equivalent basis. This means that spot cash prices for post-harvest sales are adjusted for storage costs, which include physical storage charges, shrinkage charges and interest opportunity costs. The impact of this assumption is illustrated in the top panel of Figure 18 for corn and the bottom panel for soybeans. The top line in each chart shows the 2000 harvest cash price for each crop (corn: $1.64 per bushel; soybeans: $4.56 per bushel). The bottom line reflects a cash sale at the same harvest price one to eleven months after harvest, with the cash price adjusted for commercial costs of storage. As a specific example, consider a six-month storage horizon for corn. In this case, the cash price of the sale six-months after harvest is assumed to be $1.64 per bushel, the same as the harvest cash price (equivalent to saying cash prices do not change over the six-month storage period). However, the harvest equivalent price for the sale six months after harvest is only $1.34 per bushel after adjusting for commercial storage costs. Thus, the difference between unadjusted and adjusted post-harvest prices in this example is 30¢ per bushel, a substantial difference by any standard. The magnitude of the difference is larger for longer storage horizons and for soybeans relative to corn. Note also that the difference will not be as large if on-farm variable costs of storage are assumed instead of commercial costs.

This discussion should make clear the potential pitfalls in comparing the unadjusted average cash price for an individual farmer or another market advisory service to the harvest equivalent advisory prices and benchmarks computed by the AgMAS Project. If such a comparison is made, it is not difficult to imagine a scenario where it is mistakenly concluded that the performance of the farmer or market advisory service is superior to the advisory services, market benchmarks and farmer benchmarks included in the AgMAS Project.

Second, AgMAS evaluations assume a particular geographic location. Specifically, the evaluation is designed to reflect conditions facing a representative central Illinois corn and soybean farmer. This means comparisons made by farmers or advisory services in other areas of the US may not be valid, because yields and basis patterns may be quite different. The differences in yields and basis patterns could have a substantial impact on prices computed for farmers or advisory services in another area. The resulting bias could be either up or down relative to AgMAS advisory prices and benchmarks, depending on local conditions.

Third, wherever feasible, marketing loan recommendations from advisory programs are followed by the AgMAS Project. Consequently, marketing loan payments or benefits are incorporated into net advisory prices. Market and farmer benchmark prices also include marketing loan payments or benefits. Hence, it would not be appropriate to compare prices for individual farmers or another market advisory service if marketing loan payments or benefits are not included in the prices or included in some other way.

In sum, it is inappropriate to directly compare prices for individual farmers or another market advisory service to AgMAS net advisory prices or benchmarks unless the same assumptions are used. To make valid comparisons, AgMAS assumptions regarding storage costs, yield, basis, and marketing loans have to be applied.

Appendix B: Statistical Model

For a given commodity and benchmark the statistical model underlying the average price performance tests can be stated as,

where is the net price for the i^thadvisory program in the t^th crop year, is the benchmark price in the t^thcrop year, is the expected value (mean) of the difference between the net price for the i^thadvisory program and the benchmark price and is the error term for the i^thadvisory program in the t^th crop year. Note that the model assumes the expected value of the difference between net advisory prices and the benchmark is the same for all programs and crop years. The statistical assumptions about the error term are,

The first assumption, , implies that errors are normally distributed with an expected value of zero and constant variance equal to . The next assumption, , implies that errors for the same advisory program are not correlated through time. The last assumption, , implies that errors for the same crop year are not correlated across advisory programs. The discussion in the section on average price performance focuses on correlation across advisory programs because this is considered the most serious problem. As shown in the section on predictability of performance, there is some evidence that net prices for advisory programs are positively correlated through time. However, this correlation is substantially smaller in magnitude and does not appear to be as serious of a problem.

[1] Scott H. Irwin is a Professor in the Department of Agricultural and Consumer Economics at the University of Illinois at Urbana-Champaign. Joao Martines-Filho is Manager of the AgMAS and farmdoc Projects in the Department of Agricultural and Consumer Economics at the University of Illinois at Urbana-Champaign. Darrel L. Good is Interim Head and Professor in the Department of Agricultural and Consumer Economics at the University of Illinois at Urbana-Champaign. The authors gratefully acknowledge the research assistance of Wei Shi, Brian Stark and Rick Webber, graduate students in the Department of Agricultural and Consumer Economics at the University of Illinois at Urbana-Champaign. Invaluable assistance with estimating on-farm storage costs was provided by Kevin Dhuyvetter, Department of Agricultural Economics, Kansas State University, Lowell Hill, Department of Agricultural and Consumer Economics at the University of Illinois at Urbana-Champaign, Marvin Paulsen, Department of Agricultural Engineering at the University of Illinois at Urbana-Champaign, and Dirk Maier, Department of Agricultural and Biological Engineering, Purdue University. Helpful comments on this research report were received from members of the AgMAS Project Review Panel.

[2] King, Lev and Nefstad (1995) examine the corn and soybean recommendations of two market advisory services for a single year. The focus of their study is not pricing performance, but a demonstration of the market accounting program Market Tools. Several analyses have appeared in the popular farm press. Marten (1984) examines the performance of six advisory services for corn and soybeans over 1981 through 1983. Otte (1986) investigates the performance of three services for corn over the period 1980 through 1984. Each of these studies indicates the average price generated by the services exceeds a benchmark price. Top Producer magazine has provided evaluations of some advisory services in corn, soybeans and wheat for a number of years (e.g., Powers, 1993). Kastens and Schroeder (1996) examine futures trading profits based on the information reported in Top Producer for the 1998-1996 crop years. They find negative trading profits for wheat and positive trading profits for corn and soybeans.

[3] Throughout this report, the term "crop year" refers to the marketing window for a particular crop. This is done to simplify the presentation and discussion of market advisory service performance results. A “crop year” is more than twelve calendar months in length and includes pre-harvest and post-harvest marketing periods.

[4] Dr. Darrel L. Good and Dr. Scott H. Irwin of the University of Illinois at Urbana-Champaign jointly direct the project. Correspondence with the AgMAS Project should be directed to: Dr. Joao Martines-Filho, AgMAS Project Manager, 434a Mumford Hall, 1301 West Gregory Drive, University of Illinois at Urbana-Champaign, Urbana, IL 61801; voice: (217)333-2792; fax: (217)333-5538; e-mail: agmas@uiuc.edu. The AgMAS Project also has a website that can be found at the following address: /agmas/.

[5] Funding for the AgMAS project is provided by the following organizations: Illinois Council on Food and Agricultural Research; Cooperative State Research, Education, and Extension Service, U.S. Department of Agriculture; Economic Research Service, U.S. Department of Agriculture; the Risk Management Agency, U.S. Department of Agriculture, and the Initiative for Future Agriculture and Food Systems, U.S. Department of Agriculture.

[6] The University of Illinois Variety Testing program is a well-known example of this type of yield trial. The results of this research program can be found at http://www.cropsci.uiuc.edu/vt/.

[7] For example, Managed Accounts Reports (MAR), a well-known provider of performance information for hedge funds and commodity trading advisors, requires that commodity trading advisors have a 12-month record of trading actual client accounts and a minimum of $500,000 under management to be tracked in their database. More specific details can be found at MAR’s website (http://www.marhedge.com).

[8] When the AgMAS study began in 1994, DTN and FarmDayta were separate companies. The two companies merged in 1996.

[9] Only two other programs have gone out of business during the 1995-2000 crop years. The Agri-Edge (cash only) and Agri-Edge (hedge) programs were discontinued on January 27, 1998 when Agri-Edge went out of business. The same procedure described in the text was used to complete 1997 cash sales of corn and soybeans for these two programs.

[10] Some of the programs that are depicted as “cash only” have some futures-related activity, due to the use of hedge-to-arrive contracts, basis contracts and options.

[11] Only one service, Ag Review, clearly differentiated on-farm versus off-farm strategies during harvest of the 2000 corn and soybean crops. Further details on this issue can be found at the end of the “Storage Costs” section.

[12] A detailed explanation of the construction of the marketing profiles and results for individual advisory programs and crop years can be found in Martines-Filho et al. (2002a, 2002b). Note that these reports do not contain marketing profiles for the 2000 crop year. The AgMAS Project will compute the 2000 profiles at a later date.

[13] It is important to emphasize that the marketing profiles in Figure 2 represent the average of all advisory programs across five crop years (1995-1999), and thus, do no imply that market advisory programs use the same marketing strategies through time. The average masks substantial variation in marketing profiles across advisory programs for a given crop year and, in some cases, across crop years for the same advisory program. For example, during some portions of the two-year marketing window the average amount priced each crop year varies as much as 30 percentage points from the highest year to the lowest year.

[14] It is acknowledged that recommendations outside of the two-year marketing window could exceed the flexibility of a representative farmer. For example, it seems unreasonable to assume a representative farmer would hold stocks more than a year after the end of the marketing window. Because there are no hard-and-fast rules for making such decisions, future exceptions will be considered on a case-by-case basis.

[15] The daily prices can be found in The Wall Street Journal and at the following website: http://www.ams.usda.gov/mnreports/GX_GR113.txt.

[16] The average forward basis (cash forward prices for fall delivery minus December 2000 corn or November 2000 soybeans futures prices) over February 1-7, 2000 was -$0.2775 per bushel for corn and -$0.2770 per bushel for soybeans. A weekly version of the basis data is published at the following website: /marketing/basis/index.asp.

[17] Liquidity costs reflect the fact that non-floor traders must buy at the ask price and sell at the bid price. The difference between the bid and ask prices, termed the bid-ask spread, is the return earned by floor traders for “making the market.”

[18] The practical importance of “lumpiness” problems even for small farms may be limited, due to the availability of “mini-contracts” at the Mid-America Exchange (now a subsidiary of the Chicago Board of Trade). These futures and options contracts are specified in 1,000-bushel increments.

[19] For a complete description of the programs discussed in this section, see the following Farm Service Agency fact sheets: Nonrecourse Marketing Assistance Loans and Loan Deficiency Payments, March 1998; Feed Grains, March 1998; and Soybeans and Minor Oilseeds, July 1998. These can be found at http://www.fsa.usda.gov/pas/publications/facts/pubfacts.htm.

[20] Technically, the USDA computes LDPs for the current date using PCPs for the previous day.

[21] LDP and MLG data were obtained from the interactive LDP database at the Center for Agricultural and Rural Development (CARD) at the Iowa State University ( http://www.card.iastate.edu/).

[22] The time period for each chart begins on the first day of harvest, as determined for this study, and ends on August 31, 2001. The first day of corn harvest is assumed to be September 8, 2000. The first day of soybean harvest is assumed to be September 20, 2000.

[23] On-farm shrink charges are not applied to corn sold via a pre-harvest forward contract or harvest spot sale.

[24] On-farm shrink charges are not applied to soybeans sold via a pre-harvest forward contract or harvest spot sale.

[25] The daily interest rate, r, is computed as follows:

or 0.02612 percent per day.

[26] Commercial storage costs, as measured by the telephone survey, have not changed over the six years of the AgMAS study (1995-2000). It appears that commercial elevator storage charges have been stable for a substantial period of time. A 1982 survey of Illinois elevators by Hill, Kunda and Rehtmeyer (1983) revealed an average flat charge for storage of corn and soybeans from harvest through January of 12.9¢ per bushel and 14.2¢ per bushel, respectively. The average monthly storage charge after January was 2.1¢ per bushel for corn and 2.4¢ per bushel for soybeans. The average drying charge for corn was 2.3¢ per bushel. The majority of the surveyed elevators were located in central Illinois. These costs are similar to those used by the AgMAS study for the 1995 through 2000 crop years.

[27] The commercial drying charge is not applied to corn that is sold via a pre-harvest forward contract or harvest spot sale. Also, note that on-farm variable costs of storage do not include the cost of drying corn from 15% down to 14% moisture. This charge is assumed to only apply to post-harvest storage at commercial facilities.

[28] The commercial shrink charge is not applied to corn that is sold via a pre-harvest forward contract or harvest spot sale.

[29] Based on estimates reported in USDA December stocks reports, on-farm and off-farm storage averaged 53 and 47 percent of total storage capacity in Illinois over 1995-2000. There is no discernable trend in the proportions and they vary little from year-to-year.

[30] As was noted earlier, only one service, Ag Review, clearly differentiated on-farm versus off-farm strategies during harvest of the 2000 corn and soybean crops.

[31] Weaker versions of the theory of efficient markets predicts advisory services may profit to the degree they have superior access to information and/or superior analytical ability (e.g., Zulauf and Irwin, 1998). While logically appealing, it is quite difficult, if not impossible, to specify market benchmarks based on weaker versions of the theory because it requires knowledge of the average access to information and analytical ability of market participants.

[32] A recent national survey of advisory service subscribers by the AgMAS Project provides some perspective on the dimensions of this problem. While only 11 percent of the survey respondents said they followed market advisory service recommendations closely, two-thirds indicated they followed the recommendations loosely. Further, when asked to rate the impact of advisory service recommendations on their marketing, subscribers gave an average rating of six on a nine-point scale, with a one indicating no impact at all and a nine indicating great impact. To the extent that farmers subscribe to market advisory services, these results suggest that the average price received by farmers for a crop is influenced by the marketing advice of advisory services. For more detail on the survey results see Pennings, Good, Irwin and Gomez (2001).

[33] The sample size effect can be estimated in advance, given that the standard error of the sample mean (average) price is , where is the standard deviation of daily prices and T is the sample size. For the 24-month market benchmark, the sample size is about 500 business days, whereas the sample size for the 20-month market benchmark is about 420 business days. Hence, for a given standard deviation of daily prices, , the standard errors will differ by a factor equal to , which works out to be about nine percent. This difference is what should be observed over a large number of repeated random samples. The actual difference in standard errors will be larger due to the time diversification effect, which is much more complicated to derive analytically.

[34] The question of bias in futures prices has a long and contentious history in the economics literature. If a bias exists in corn and soybean futures prices, the available evidence suggests the magnitude is small from an economic perspective. This evidence generally is based on long samples of futures prices. Over short sample periods, futures prices can have sharp upward or downward trends. Probably the most dramatic example is the upward trend in grain futures prices between 1972 and 1975. See Zulauf and Irwin (1998) for a thorough discussion and additional references.

[35] A classic paper in the literature on who wins and loses in futures and options markets is Hartzmark (1987).

[36] The argument here is that selective hedging by farmers, in aggregate, results in trading losses. This does not preclude the possibility that some individual farmers consistently earn trading profits through selective hedging.

[37] State average LDPs and MLG’s for Illinois were collected from on-line Farm Service Agency reports at: http://www.fsa.usda.gov/dafp/psd/reports.htm.

[38] Terms like “two-year average” are used to refer to averages of net advisory prices over multiple crop years.

[39] A measure of survivorship bias can be computed by subtracting multiple-year averages based only on the programs active in the first crop year of each sample from the overall averages presented in Tables 15 through 17. The differences vary between +1 and -4¢ per bushel for corn, 0 and -6¢ per bushel for soybeans and +$1 and -$2 per acre for advisory revenue. Since positive numbers indicate survivorship bias, the comparisons suggest survivorship bias is negligible or non-existent in the overall averages in Tables 15 through 17.

[40] For example, one possibility is that advisory programs as a group fail to beat market benchmarks, yet at the same time some programs have “exceptional” performance. Testing whether performance is exceptional for a particular advisory program requires different statistical tests than the ones used here (Marcus, 1990).

[41] The different forms of averaging will produce equal estimates only if a time-series cross-section data set is “balanced.” That is, the number of programs is the same for each crop year and there are no missing observations. This clearly is not the case here. It turns out that, after rounding, the two different methods of averaging produce the same estimates of the average proportion.

[42] Differences are calculated as advisory price minus benchmark price. So, a positive difference indicates an advisory price above the benchmark price, and vice versa.

[43] See Appendix B for presentation of the statistical model underlying this discussion.

[44] The full set of correlation results is available from the authors upon request.

[45] Technically speaking, correlation coefficients should be estimated for the difference between net advisory prices and a particular benchmark since the matched sample t-test is based on price differences not prices levels. Difference correlations are also estimated and show similar evidence of dependence across advisory programs. Differencing does create a more complex pattern of dependence, as roughly half the estimated correlations are positive and the other half negative. The average absolute value of the difference correlations is about 0.50. These results are available from the authors upon request.

[46] In more formal terms, “typical” means one can be 95% confident the true value of the difference will be contained in an interval about two standard errors above and below the average difference estimate.

[47] The standard error of the average difference is estimated as , where is the standard deviation of differences across crop years and T is the sample size (six in this case).

[48] Readers should be aware that the marketing profile for farmers is nothing more than (hopefully) an informed guess. In the section on farmer benchmark prices, it was noted that almost no concrete evidence exists on the exact length of the typical marketing window of farmers or the precise pattern of forward pricing.

[49] For a given advisory program, the formula for estimating standard deviation is,

where T is the number of crop years in the sample, y_t is the advisory program’s net price for the t^th crop year and is the average net advisory price over the T crop years.

[50] The restriction means that only advisory programs active all six crop years are included in the average price and risk evaluation. As a result, there is the potential for survivorship bias in the average price and risk comparisons to the benchmarks. Survivorship bias in the average estimates appears to be non-existent, as the average prices and average revenue for the 17 programs are actually less than the average prices and revenue computed across all advisory programs active in the 1995-2000 sample period. It is quite difficult to assess the degree of survivorship bias in advisory program standard deviation estimates with the limited number of crop years available.

[51] These results are available from the authors upon request.

[52] Fisher’s Exact Test is the appropriate statistical test because both row and column totals are pre-determined in the 2 x 2 contingency table formed on the basis of winner and loser counts.

[53] These results are available from the authors upon request.