farmdoc : AgMAS: Report 2005-01: The Pricing Performance of Market Advisory Services in Corn and Soybeans Over 1995-2003

Report 2005-01: The Pricing Performance of Market Advisory Services in Corn and Soybeans Over 1995-2003

Copyright 2005 by Scott H. Irwin, Darrel L. Good, Joao Martines-Filho and Lewis A. Hagedorn. All rights reserved. Readers may make verbatim copies of this document for non-commercial purposes by any means, provided that this copyright notice appears on all such copies.

Farmers in the U.S. consistently identify price and income risk as one of the greatest management challenges they face. The roller coaster movement of corn and soybean prices over the last decade is ample evidence of the uncertainty and risk facing grain farmers. Surveys suggest that numerous farmers view market advisory services as an important tool in managing price and income risk (e.g., Sogn and Kraner, 1977; Smith, 1989; Patrick and Ullerich, 1996; Patrick, Musser and Eckman; 1998; Schroeder et al., 1998; Norvell and Lattz, 1999; Pennings et al., 2001). Furthermore, Davis and Patrick (2000) and Katchova and Miranda (2004) find that the use of market advisory services has a significant influence on the use of forward pricing by farmers.

A limited number of academic studies investigate the pricing performance of market advisory services.[2] In the earliest study, Marquardt and McGann (1975) evaluate the accuracy of cash price predictions for 10 private and public outlook newsletters in corn, soybeans, wheat, cattle and hogs over 1970-1973. They find that futures prices generally are a more accurate source of forecasts than either the private or public newsletters. Gehrt and Good (1993) analyze the performance of five advisory services for corn and soybeans over the 1985 through 1989 crop years.[3] Assuming a representative farmer follows the hedging and cash market recommendations for each advisory service; a net price received for each year is computed and compared to a benchmark price. They generally find that corn and soybean farmers obtained a higher price by following the marketing recommendations of advisory services. Martines-Filho (1996) examines the pre-harvest corn and soybean marketing recommendations of six market advisory services over 1991 through 1994. He computes the harvest time revenue that results from a representative farmer following the pre-harvest futures and options hedging recommendations and selling 100% of production at harvest. Average advisory service revenue over the four years is larger than benchmark revenue for both corn and soybeans. Kastens and Schroeder (1996) examine the futures trading profits of seven to ten market advisory services for the 1988-1996 crop years. They report negative gross trading profits for wheat and positive gross trading profits for corn and soybeans. The authors indicate that incorporating brokerage commissions and subscription costs would have substantially diminished trading returns.

While a useful starting point, previous studies have important limitations. First, the cross-section of advisory services tracked for each crop year is quite small, with the largest sample including only ten advisory services. Second, the results may be subject to survivorship bias, a consequence of tracking only advisory services that remain in business at the end of a sample period. The literature on the performance of mutual funds, hedge funds and commodity trading advisors provides ample evidence of the upward bias in performance results that can result from survivorship bias (e.g., Brown et al., 1992; Schneeweis, McCarthy and Spurgin, 1996; Brown, Goetzmann and Ibbotson, 1999). Third, the results may be subject to hindsight bias if advisory service recommendations were not collected on a "real-time" basis (Jaffe and Mahoney, 1999). Hindsight bias is the tendency to collect or record profitable recommendations and ignore or minimize unprofitable recommendations after the fact.

This discussion suggests the academic literature provides farmers with a limited basis for evaluating the performance of market advisory services. The Agricultural Market Advisory Service (AgMAS) Project was initiated in 1994 with the goal of providing unbiased and rigorous evaluation of market advisory services.[4] [5] The AgMAS Project has collected marketing recommendations for no fewer than 23 market advisory programs each crop year since the project was initiated. While the sample of advisory services is non-random, it is constructed to be generally representative of the majority of advisory services offered to farmers. Further, the sample of advisory services includes all programs tracked by the AgMAS Project over the study period, so pricing performance results should not be plagued by survivorship bias. Finally, the AgMAS Project subscribes to all of the services that are followed and records recommendations on a real-time basis. This should prevent the pricing performance results from being subject to hindsight bias.

The purpose of this research report is to evaluate the pricing performance of market advisory services for the 1995-2003 corn and soybean crops. The results for 1995-2001 were released in earlier AgMAS research reports (e.g., Irwin, Martines-Filho and Good, 2002), while results for the 2002 and 2003 crop years are new. Following the literature on mutual fund and investment newsletter performance (e.g., Metrick, 1999; Jaffe and Mahoney, 1999), two basic questions will be addressed in the report: 1) Do market advisory services, on average, outperform appropriate benchmarks? and 2) Do market advisory services exhibit persistence in their performance from year-to-year? Certain explicit assumptions are made to produce a consistent and comparable set of results across the different advisory programs. These assumptions are intended to accurately depict "real-world" marketing conditions facing a representative central Illinois corn and soybean farmer. Several key assumptions are: i) with a few exceptions, the marketing window for a crop year runs from September before harvest through August after harvest, ii) on-farm or commercial physical storage costs, as well as interest opportunity costs, are charged to post-harvest sales, iii) brokerage costs are subtracted for all futures and options transactions and iv) Commodity Credit Corporation (CCC) marketing loan recommendations made by advisory programs are followed wherever feasible. Based on these and other assumptions, the net price received by a subscriber to a market advisory program is calculated for the 1995-2003 corn and soybean crops.

Four basic indicators of performance are applied to advisory program prices and revenues over 1995-2003. The first indicator is the proportion of advisory programs that beat benchmark prices. The second indicator is the difference between the average price of advisory programs and benchmarks. The third indicator is the average price and risk of advisory programs relative to the average price and risk of benchmarks. The fourth indicator is the predictability of advisory program performance from year-to-year. Both market and farmer benchmarks are developed for the evaluations. All benchmarks are computed using the same assumptions applied to advisory service track records.

At the outset, it is important to point out that only nine crop years are available to analyze market advisory service pricing performance. From a purely statistical standpoint, samples with ten or fewer observations typically are considered "sparse." On the surface, this suggests the sample may not contain enough information to draw conclusions about advisory service pricing performance. There are several reasons why this may not be the case. First, Anderson (1974) explored the reliability of agricultural return-risk estimates based on sparse data sets and found the surprising result that even as few as three or four observations can be very useful. Second, even though the number of crop years is limited, at least 23 advisory programs are tracked for each crop year. This has the potential to substantially increase the information provided by the sample. Third, from a practical, decision-making standpoint, samples with nine or fewer observations often are considered adequate to reach conclusions. The results of university crop yield trials represent a well-known example. A typical presentation of the results includes only current year yields and two-year or three-year averages. In many cases, even the two-year and three-year averages cannot be presented because of turnover in the varieties tested from year-to-year.[6] Despite the limitations, this type of yield trial data is widely used by farmers in making variety selections. On balance, then, it seems reasonable to argue that the nine years of data currently available on advisory service pricing performance may be used to make some careful conclusions. Caution obviously is in order given the possibility of results being due to random chance in a relatively small sample of crop years.

This report has been reviewed by a member of the AgMAS Review Panel, which provides independent, peer-review of AgMAS Project research. The member who reviewed this report is Diana Klemme, Vice President, Director - Grain Division, Grain Service Corporation, Atlanta, Georgia.

The next section of the report describes the procedures used to collect the data on market advisory service recommendations. The second section describes the methods and assumptions used to calculate the returns to advisory service marketing advice. The third section presents the methods and assumptions used to compute benchmark prices. The fourth section of the report presents 2002 and 2003 pricing results for corn and soybeans. The fifth section presents a summary of the combined results for the 1995-2003 crop years. The sixth section discusses the performance evaluation results for 1995-2003. The final section presents a summary and conclusions.

The market advisory services included in this evaluation do not comprise the population of market advisory services available to farmers. The included services also are not a random sample of the population of market advisory services. Neither approach is feasible because no public agency or trade group assembles a list of advisory services that could be considered the "population." Furthermore, there is not a generally agreed upon definition of an agricultural market advisory service. To assemble the sample of services for the AgMAS Project, criteria were developed to define an agricultural market advisory service and a list of services was assembled.

Five criteria are used to determine which advisory services are included in the AgMAS study. First, marketing recommendations from an advisory service must be received electronically in real time. The recommendations may come in the form of satellite-delivered pages, Internet web pages or e-mail messages. Services delivered electronically generally ensure that recommendations are made available to the AgMAS Project at the same time as farm subscribers. This form of delivery also ensures that recommendations are received in "real-time." This avoids the problem of recommendations being delivered after the date of implementation intended by an advisory service. Such a problem could occur frequently with recommendations delivered via the postal service.

The second criterion is that a service has to provide marketing recommendations to farmers rather than (or in addition to) speculators or "traders." Some of the services tracked by the AgMAS Project do provide speculative trading advice, but that advice must be clearly differentiated from marketing advice to farmers for the service to be included. The terms "speculative" trading of futures and options and "hedging" use of futures and options are only used to identify whether a service is focused on speculators or farmers. Within a clearly defined farm marketing program, a distinction between speculative and hedging use of futures and options is not necessary.

The third criterion is that marketing recommendations from an advisory service must be in a form suitable for application to a representative farmer. That is, the recommendations have to specify the percentage of the crop involved in each transaction --cash, futures or options-- and the price or date at which each transaction is to be implemented. It is also helpful if advisory services make specific recommendations about implementation of the marketing loan program, but that is not required. Note that some advisory services evaluated by the AgMAS Project do not make any futures and options recommendations, so it is not necessary to make such recommendation to be included in the study. Services that make futures and options hedging recommendations, but fail to clearly state when cash sales should be made, or the amount to be sold, are not considered for inclusion.

The fourth criterion is that advisory services must provide "blanket" or "one-size fits all" marketing recommendations so there is no uncertainty about implementation. While different programs may be tracked for an advisory service (e.g., a cash only program versus a futures and options hedging and cash program), it is not feasible to track services that provide "customized" recommendations for individual clients.

A fifth criterion addresses the issue of whether a candidate service is a viable, commercial business. This issue has arisen due to the extremely low cost and ease of distributing information over the Internet, either via e-mail or a website. It is possible for an individual with little actual experience and no paying subscribers to start a "market advisory service" by using the Internet. Hence, there is a need to exclude firms that are not viable commercial concerns. At the same time, any filter in this regard should not be so restrictive that newer and smaller advisory services are excluded from the AgMAS study for an unreasonably long period of time. This same issue is prevalent when evaluating the performance of other types of professional investment advisors, such as commodity trading advisors. In these cases, it is not unusual to screen firms by the length of track record and amount of funds under management.[7] An analogous screen for market advisory services can be based on the length of time the service has provided recommendations and the number of paying subscribers. The specific criterion used is that a candidate advisory service must have provided recommendations to paying subscribers for a minimum of two marketing years before the service can be included in the AgMAS study. This criterion should exclude non-viable services, while at the same time providing a relatively low hurdle for new and legitimate market advisory services.

The original sample of market advisory services was drawn from the list of Premium Services available from the two major agricultural satellite networks, Data Transmission Network (DTN) and FarmDayta, in the summer of 1994.[8] While the list of advisory services available from these networks was by no means exhaustive, it did have the considerable merit of meeting a market test. Presumably, the services offered by the networks were those most in demand by farm subscribers to the networks. In addition, the list of available services was cross-checked with other farm publications to confirm that widely followed advisory firms were included in the sample. It seems reasonable to argue that the resulting sample of services was generally representative of the majority of advisory services available to farmers.

Additions and deletions to the sample of advisory services have occurred over time. Additions largely have been due to the increasing availability of market advisory services via alternative means of electronic delivery, in particular, websites and e-mail. Deletions have occurred for a variety of reasons. A total of 39 and 38 advisory service programs for corn and soybeans, respectively, have been included in the sample at some point in time. Table 1 contains the complete list of advisory programs and includes a brief explanation why each program not included for all crop years was added or deleted from the sample. The term "advisory program" is used because several advisory services have more than one distinct marketing program. For example, AgLine by Doane, Brock, Pro Farmer and Stewart-Peterson Advisory Services each have two distinct marketing programs, Risk Management Group has three distinct marketing programs and AgriVisor has four distinct marketing programs. Allendale provides two distinct programs for corn, but only one for soybeans.

The total number of advisory programs evaluated for the 2002 crop year is 27 for corn and 26 for soybeans. The number of advisory programs evaluated for the 2003 crop year is 26 for corn and 25 for soybeans. One program, Grain Marketing Plus, was deleted from the sample for the 2003 crop year. This service went out of business at the end of March 2003. As of this date, no recommendations were given by Grain Marketing Plus for the 2003 crop year. An additional program, Co-Mark, went out of business in July 2003. Since this program had completed recommendations for 2002 crops and issued some recommendations for the 2003 crops by July 2003, it is included for both the 2002 and 2003 crop years.

Three forms of survivorship bias may be potential problems when assembling an advisory program database. Survival bias significantly biases measures of performance upwards since "survivors" typically have higher performance than "non-survivors" (e.g., Brown et al., 1992; Schneeweis, McCarthy and Spurgin, 1996; Brown, Goetzmann and Ibbotson, 1999). The first and most direct form of survivorship bias occurs if only advisory programs that remain in business at the end of a given sample period are included in the sample. This form of bias should not be present in the AgMAS database of advisory programs because all programs that have been tracked over the entire time period of the study are included in the sample. The second form of survivorship bias occurs if discontinued advisory programs are deleted from the sample for the year when they are discontinued. This is a form of survivorship bias because only survivors for the full crop year are tracked. The AgMAS database of advisory programs should not be subject to this form of bias because programs discontinued during a crop year remain in the sample for that crop year. [9] The third and most subtle form of survivorship bias occurs if data from prior periods are "back-filled" at the point in time when an advisory program is added to the database. This is a form of survivorship bias because data from surviving advisory programs are back-filled. The AgMAS database should not be subject to this form of bias because recommendations are not back-filled when an advisory program is added. Instead, recommendations are collected only for the crop year after a decision has been made to add an advisory program to the database.

Another important consideration when assembling a database on advisory program recommendations is hindsight bias (Jaffe and Mahoney, 1999). This is the tendency to collect or record profitable recommendations and ignore or minimize unprofitable recommendations after the fact. Since the AgMAS Project subscribes to all of the services that are followed and records recommendations on a real-time basis, the database of recommendations should not be subject to hindsight bias. The information is received electronically, via DTN, website or e-mail. For the programs that provide multiple daily updates, information is recorded for all updates. In this way, the actions of a farmer-subscriber are simulated in real-time.

When recording recommendations of each advisory program, specific attention is paid to which year's crop is being sold (e.g., 2003 crop year), the amount of the commodity to be sold, which futures or options contract is to be used (where applicable) and any price targets that are mentioned (e.g., sell cash corn when March 2004 futures reaches $2.40). If a price target is given and not immediately filled, such as a stop order in the futures market, the recommendation is noted until the order is either filled or canceled. Recommendations for farm marketing programs are not screened for "speculative" versus "hedging" uses of futures and options. Consequently, all futures and options trades presented to farmers as a part of marketing recommendations are included.

As noted above, some advisory services offer two or more distinct marketing programs. This typically takes the form of one set of advice for marketers who are willing to use futures and options (although futures and options are not always used) and a separate set of advice for farmers who only wish to make cash sales.[10] In this situation, both strategies are recorded and treated as distinct strategies to be evaluated. Some programs also differentiate advice based on the availability of on-farm storage. In the past, when a service clearly differentiated strategies based on the availability of on-farm versus off-farm (commercial) storage, only the off-farm storage strategy was tracked. Starting with the 2000 corn and soybean crops, if a service clearly differentiates on-farm and off-farm storage strategies at or before harvest, both strategies are recorded. [11]

Several procedures are used to check the recorded recommendations for accuracy and completeness. Whenever possible, recorded recommendations are crosschecked against later status reports provided by the relevant advisory program. Also, at the completion of the crop year, it is confirmed whether cash sales total exactly 100%, all futures positions are offset and all options positions are offset or expire.

The final set of recommendations attributed to each advisory program represents the best efforts of the AgMAS Project staff to accurately and fairly interpret the information made available by each advisory program. In cases where a recommendation is considered vague or unclear, some judgment is exercised as to whether or not to include that particular recommendation or how to implement the recommendation. Given that some recommendations are subject to interpretation, the possibility is acknowledged that the AgMAS track record of recommendations for a given program may differ from that stated by the advisory program, or from that recorded by another subscriber.

At the end of the marketing period, all of the (filled) recommendations are aligned in chronological order. The advice for a given crop year is considered to be complete for each advisory program when cumulative cash sales of the commodity reach 100%, all futures positions covering the crop are offset, all option positions covering the crop are either offset or expire and the advisory program discontinues giving advice for that crop year. In order to produce a consistent and comparable set of results across the different advisory programs, certain explicit assumptions are made. The assumptions are intended to accurately depict "real-world" marketing conditions facing a representative central Illinois corn and soybean farmer. Based on these assumptions, the returns to each recommendation are then calculated in order to arrive at a weighted average net price that would be received by a farmer who precisely follows the marketing advice (as recorded by the AgMAS Project). It should be interpreted as the harvest-equivalent net price received by a farmer because post-harvest sales are adjusted for physical storage and interest opportunity costs.

The discussion about marketing assumptions in the following sections centers on the 2002 and 2003 crop years. It is important to note that some assumptions have changed over time. Specific information on assumptions for the 1995-2001 crop years can be found in earlier AgMAS pricing reports (e.g., Irwin, Martines-Filho and Good, 2003). Assumed values for key variables used in the simulation of advisory service performance over the 1995-2003 crop years can be found in Appendix A.

The simulation is designed to reflect conditions facing a representative central Illinois corn and soybean farmer. Whenever possible, data are collected for the Central Crop Reporting District in Illinois as defined by the National Agricultural Statistics Service (NASS) of the U.S. Department of Agriculture (USDA). The eleven counties (DeWitt, Logan, McLean, Marshall, Macon, Mason, Menard, Peoria, Stark, Tazewell and Woodford) that make up this District are highlighted in Figure 1.

Caution should be used when applying the results to other areas of the US, because yields and basis patterns may be quite different from those of central Illinois. Differences in yields and basis patterns could have a substantial impact on prices computed for farmers or advisory services in another area. The resulting change could be either up or down relative to AgMAS advisory prices and benchmarks, depending on local conditions. Appendix B to this report, entitled "A Cautionary Note on the Use of AgMAS Net Advisory Prices and Benchmarks," contains further discussion on this point.

The time period over which a farmer normally makes pricing decisions for a particular crop is termed the "marketing window." It also can be referred to as the pricing "decision-horizon" or "timeline" of a farmer. A marketing window does not necessarily equal the time period of observed market activity. The reason is that not taking action (e.g., not hedging pre-harvest) is one type of decision that can be made during a marketing window.

In the present context, the objective is to define the normal marketing window of a representative farmer who subscribes to the advisory programs tracked by the AgMAS Project. Good, Hieronymus and Hinton (1980) provide a useful starting point. They define the marketing window for an Illinois grain farmer as the period extending from the initial production planning time until the end of the storage season. First production decisions in Illinois normally occur in October through November of the year preceding planting (e.g., fall tillage and application of fertilizer), while the storage season typically extends through July or August of the year following harvest. This results in a marketing window between 21 and 23 months in length. Chafin and Hoepner (2002) reach a similar conclusion in their text on commodity marketing:

In building an integrated marketing plan, crop producers must keep in mind the fact that pricing decisions on a single crop span a two-year period: the growing year and the storage year. The first stage of a crop "marketing year" begins in November as production plans are being made for the new crop and continues throughout the growing season until the end of harvest. During the second stage of the "marketing year," pricing of the harvested (old) crop begins at the end of the 12-month "growing" year and continues for the next 12-month storage year. Thus, the pricing of a single crop spans 730 days-the "growing year" plus the "storage year." (p. 326)

The actual pricing pattern of advisory programs included in the AgMAS study provides further information for defining the relevant marketing window. As noted earlier, observed market positions cannot directly reveal the intended pricing window of a representative farmer following advisory program recommendations. However, averages over time and advisors should be suggestive as to the typical starting and ending points used to make recommendations for a crop. Figure 2 presents the average "marketing profile" of advisory programs in corn and soybeans over the 1995-2001 crop years.[12] The marketing profiles show the average amount of corn and soybean crops priced (sold) by advisory programs, on a cumulative basis, each day over the two-year period beginning in September of the year before harvest and ending in August of the year after harvest. The profiles suggest that a farmer following the recommendations of market advisory programs included in the AgMAS study, on average, will begin making significant marketing decisions (pricing more than one percent) in September of the year before harvest and will not complete marketing until August of the year after harvest.[13]

Overall, this discussion indicates it is reasonable to assume a 24-month marketing window for a representative farmer subscribing to advisory programs. In the case of the 2002 crop, the marketing window is defined as the two-year period beginning September 1, 2001 and ending on August 31, 2003. In the case of the 2003 crop, the marketing window is defined as the two-year period beginning September 1, 2002 and ending on August 31, 2004. Two further issues need to be discussed with respect to the market window. The first issue is exceptions to the specific definition. For example, one program in corn started its first hedging position for the 2002 crop year at the end of March 2001. One other advisory service had a relatively small amount (10%) of cash soybeans unsold in its programs as of August 31, 2003. These bushels were sold in the spot cash market by September 3, 2003. On the other hand, for the 2003 crop year, seven programs in corn and three programs in soybeans started hedging recommendations before September 1, 2002. The earliest case occurred in corn with a first recommendation given in the middle of October 2001. Because the marketing window is defined as the "normal" window, it is argued that a representative farmer would approach the marketing window with some flexibility, particularly for recommendations that do not extend too far outside the limits of the marketing window. While a few of the 2002 and 2003 recommendations extend considerably beyond the limits of the marketing window, most do not. All of the transactions in question are nonetheless included in the relevant advisory program's track record in the interest of completeness and accuracy.[14] The second issue is the definition of business days within the marketing window. This issue arises because different entities in the agricultural sector have different policies with respect to holidays. For the purposes of this study, an "official" business day within the marketing window is defined as a business day where the Chicago Board of Trade is open and cash prices are reported by the Illinois Department of Ag Market News. Finally, note that throughout the remainder of this report the term "crop year" is used to represent the two-year marketing window.

The price assigned to each cash sale recommendation is the central Illinois closing, or overnight, bid. The data are collected and reported by the Illinois Department of Ag Market News.[15] The central Illinois price is the mid-point of the range of bids by elevators in the North Central and South Central Price Reporting Districts, as defined by the Illinois Department of Ag Market News. The North and South Central Illinois Price Reporting Districts are highlighted in Figure 3. Prices in this 35-county area best reflect prices for the assumed geographic location of the representative central Illinois farmer (Central Illinois Crop Reporting District).

Pre-harvest cash forward contract prices for fall delivery are also needed. Pre-harvest bids collected by the Illinois Department of Ag Market News are used when available. The central Illinois pre-harvest price is the mid-point of the daily range of pre-harvest bids by elevators in the North Central and South Central Price Reporting Districts, again, as defined by the Illinois Department of Ag Market News. Pre-harvest forward prices are available from this source for the 2002 corn and soybean crops during the February 4, 2002 through August 30, 2002 period. Pre-harvest forward prices are available for the 2003 corn and soybeans crops from the February 3, 2003 through September 3, 2003 period.

The marketing window for the 2002 and 2003 corn and soybean crops begins in September 2001 and September 2002, respectively. Since the Illinois Department of Ag Market News did not begin to report actual cash forward bids until February 4, 2002 for 2002 crops and February 3, 2003 for 2003 crops, pre-harvest prices need to be estimated for the first five months of each marketing window. For dates between September 1, 2001 - February 1, 2002 and September 1, 2002 - January 31, 2003, a three-step estimation procedure is adopted. First, the average forward basis for the first five days the Illinois Department of Ag Market News reports actual forward contract bids is computed (February 4-8, 2002 for 2002 crops and February 3-7, 2003 for 2003 crops) .[16] Second, the forward basis is widened in a linear fashion moving back in time from February to September. This is based on the findings in several studies that the forward basis for corn, soybeans and wheat widens systematically the more distant the time before harvest (Harris and Miller, 1981; Elam and Woodworth, 1989; Brorsen, Coombs and Anderson, 1995; Townsend and Brorsen, 2000; Shi et al., 2004). The widening "factor" for 2002 and 2003 crops is estimated based on the average change in weekly forward basis bids for central Illinois over the 1975-2001 pre-harvest periods (0.06¢ per bushel per week for corn and 0.05¢ per bushel per week for soybeans).[17] The weekly change is converted to a daily change by dividing the estimated averages by five (0.01¢ per bushel per day for corn and soybeans). The resulting adjustment to the estimated forward basis (and estimated forward contract bids) is rather modest. For example, the widening adjustment on the first day of the marketing window (September 1, 2002 and September 1, 2003 for the 2002 and 2003 crops, respectively) is about one cent per bushel for both corn and soybeans.[18] Third, the estimated forward basis computed in the previous two steps is added to the settlement price of the Chicago Board of Trade (CBOT) new crop futures prices for the 2002 crop year (2002 December corn futures contract or 2002 November soybean futures contract) between September 1, 2001 and February 1, 2002 and the CBOT new crop futures price for the 2003 crop year (2003 December corn futures contract or 2003 November soybean futures contract) between September 1, 2002 and January 31, 2003.

The estimation procedure outlined above is expected to be a reasonably accurate reflection of actual forward prices for the early period of the marketing window, as the actual price of the harvest futures contract is used and only the forward basis is estimated. In addition, the estimation procedure typically is applied to a relatively small number of transactions. For example, the average net amount sold before February 1st over 1995-2001 is only 13% for corn and 8% for soybeans, and many of these transactions are in futures or options contracts rather than forward contracts.

Some market advisory programs recommended the use of post-harvest forward contracts to sell part of the 2002 and 2003 corn and soybean crops. The Illinois Department of Ag Market News reported post-harvest bids for January 2002 and 2003 deliveries. Post-harvest bids also were reported for March 2002 and 2003 deliveries. These central Illinois bids are used wherever applicable. However, four positions recommended by advisory programs for the 2002 corn and soybean crops either did not match the January or March delivery period or were made before the Illinois Department of Ag Market News began reporting post-harvest forward contract prices. The following procedure was adopted to estimate the additional post-harvest forward contract prices needed in three of the cases. First, three elevators in central Illinois who agreed to supply data on spot and forward contract prices on the dates when advisors made such recommendations were contacted. Each of these elevators is in a different county in the Central Illinois Crop Reporting District (Logan, McClean, DeWitt). Second, the spread between each elevator's forward price and spot price is calculated for the relevant date. Third, the forward spread is averaged across the three elevators for the same date. Fourth, the average forward spread from the three elevators is added to the central Illinois cash price (discussed at the beginning of the section) to arrive at an estimated post-harvest forward contract price for central Illinois. This same procedure was used in a few cases for the 1998 and 1999 crop years. In one case for corn, none of the three elevators had forward contract prices available on the date of the advisory program's recommendation. The estimate used in this case is the average actual spot basis for the delivery period of the forward contract for the previous three years.

The fill prices for futures and options transactions generally are the prices reported by the programs. In cases where a program did not report a specific fill price, the settlement price for the day is used.

Since most of the advisory program recommendations are given in terms of the proportion of total production (e.g., "sell 5% of 2003 crop today"), some assumption must be made about the amount of production to be marketed. For the purposes of this study, if the per-acre yield is assumed to be 100 bushels, then a recommendation to sell 5% of the corn crop translates into selling 5 bushels. When all of the advice for the marketing period has been carried out, the final per-bushel selling price is the average price for each transaction weighted by the amount marketed in each transaction.

The above procedure implicitly assumes that the "lumpiness" of futures and/or options contracts is not an issue. Lumpiness is caused by the fact that futures contracts are for specific amounts, such as 5,000 bushels per CBOT corn futures contract. For large-scale farmers, it is unlikely that this assumption adversely affects the accuracy of the results. This may not be the case for small- to intermediate-scale farmers who are less able to sell in 5,000-bushel increments. [19]

When making hedging or forward contracting decisions prior to harvest, the actual yield is unknown. Hence, an assumption regarding the amount of expected production per acre is necessary to accurately reflect the returns to marketing advice. Prior to harvest, the best estimate of current year expected yield is likely to be a function of yield in previous years. In this study, the assumed yield prior to harvest is the calculated trend yield, while the actual reported yield is used from the harvest period forward. The expected yield for 2002 is based upon a log-linear regression trend model of actual yields from 1972 through 2001 for the Central Illinois Crop Reporting District. The expected yield for 2003 is based upon a log-linear regression trend model of actual yields from 1972 through 2002 for the Central Illinois Crop Reporting District. Previous research suggests this type of trend model provides a reasonable fit to corn and soybean yield data (Fackler, Young and Carlson, 1993; Zanini, 2001).

In central Illinois, the expected yield for corn is calculated to be 154.9 bushels per acre in 2002 and 156.1 bushels per acre in 2003. Therefore, recommendations regarding the marketing quantity made prior to harvest for the 2002 and 2003 crop years are based on yields of 154.9 and 156.1 bushels per acre, respectively. For example, a recommendation to forward contract 20% of expected 2003 production translates into a recommendation to contract 31.2 bushels per acre (20% of 156.1). The actual reported corn yield in central Illinois is 149 bushels per acre in 2002 and 183 bushels per acre in 2003. The same approach is used for soybean evaluations. The calculated 2002 trend yield for soybeans in central Illinois is 49.3 bushels per acre and the actual yield is 51 bushels per acre. The calculated 2003 trend yield for soybeans in central Illinois is 50 bushels per acre and the actual yield is 38 bushels per acre.

It is assumed that after harvest begins, farmers can make reasonably accurate projections of realized yields. Therefore, recommendations made after the start of harvest are assumed to be based on actual yields instead of expected yields. Since harvest does not occur during the same exact period each year, data on harvest progress are needed to establish the relevant harvest window, and in particular, the date that harvest begins. Harvest progress data are reported by NASS for the central Illinois Crop Reporting District; however, the reports typically are not made available soon enough to identify precisely the beginning of harvest. Consequently, the exact "location" of the harvest window cannot be identified based upon available data. The following alternative procedure is used to estimate the harvest window each year. First, the business day nearest to 50% completion of harvest is defined as the mid-point of harvest. Second, the entire harvest period is defined as a five-week window, beginning twelve business days before the mid-point of harvest, and ending twelve business days after the mid-point of harvest (a total of 25 business days, or five weeks). In most years, the five-week window includes at least 80% of the harvest.

Since NASS harvest progress reports are made weekly, the exact date of the harvest mid-point is not known. However, it is possible to estimate the date of the mid-point using the weekly progress numbers of the two reports that encompass 50% harvest progress. As an example, the NASS estimate of corn harvest progress in central Illinois is 40% on September 30, 2001. Harvest progress is estimated to be 67% in the next report on October 7, 2001. A daily progress estimate for this week can be constructed by taking the difference of these estimates and dividing the result by seven; in this example, harvest progressed at rate of approximately 3.86% per day. Counting forward from 40% at a rate of 3.86% per day, the business day closest to 50% progress is October 3, 2001. This mid-point is used to construct the harvest window for corn by counting backwards and forwards twelve business days. The same procedure is used to determine the harvest window for soybeans.

The harvest period for corn in 2002 is defined as September 19, 2002 through October 23, 2002. For soybeans, the harvest period is September 20, 2002 through October 24, 2002. Therefore, recommendations for corn made after September 18, 2002 are applied on the basis of the actual yield of 149 bushels per acre. For soybeans, recommendations made after September 19, 2002 are applied on the basis of the actual yield of 51 bushels per acre.

The harvest period for corn in 2003 is defined as September 18, 2003 through October 22, 2003. For soybeans, the harvest period is September 17, 2002 through October 21, 2002. Therefore, recommendations for corn made after September 16, 2003 are applied on the basis of the actual yield of 183 bushels per acre. For soybeans, recommendations made after September 16, 2003 are applied on the basis of the actual yield of 38 bushels per acre.

The issue of changing yield expectations typically is not dealt with in the recommendations of the advisory programs. For the purpose of this study, the actual harvest yield must exactly equal total cash sales of the crop at the end of the marketing time frame. Hence, an adjustment in yield assumptions from expected to actual levels must be applied to cash transactions at some point in time. In this analysis, an adjustment is made in the amount of the first cash sale made after the beginning of the harvest period. For example during the 2003 crop year, if a program advises forward contracting 50% of the corn crop prior to harvest, this translates into sales of 78.05 bushels per acre (50% of 156.1). However, when the actual yield is applied to the analysis, sales-to-date of 78.05 bushels per acre imply that only 42.65% of the actual crop has been contracted. In order to compensate, the amount of the next cash sale is adjusted to align the amount sold. In this example, if the next cash sale recommendation is for a 10% increment of the 2003 crop, making the total recommended sales 60% of the crop, the recommendation is adjusted to 17.35% of the actual yield (31.75 bushels), so that the total crop sold to date is 60% of 183 bushels per acre (78.05 + 31.75 = 109.8 = 0.6*183). After this initial adjustment, subsequent recommendations are taken as percentages of the 183 bushels per acre actual yield, so that sales of 100% of the crop equal sales of 183 bushels per acre.

While the amount of cash sales is adjusted to reflect the change in yield information, a similar adjustment is not made for futures or options positions that are already in place. For example, assume that a short futures hedge is placed in the December 2003 corn futures contract for 25% of the 2003 crop prior to harvest. Since the amount hedged is based on the trend yield assumption of 156.1 bushels per acre, the futures position is 39.03 bushels per acre (25% of 156.1). After the yield assumption is changed, this amount represents a short hedge of 21.3% (39.03/183). The amount of the futures position is not adjusted to move the position to 25% of the new yield figure. However, any futures (or options) positions recommended after the beginning of harvest are implemented as a percentage of the actual yield.

If actual yield is substantially below trend, and forward pricing obligations are based on trend yields, a farmer may have difficulty meeting such obligations. This raises the issue of updating yield expectations in "short" crop years to minimize the chance of defaulting on forward pricing obligations. While not yet encountered in the AgMAS evaluations of corn and soybeans, this situation has arisen in the evaluation of wheat (Jirik, Irwin, Good, Jackson and Martines-Filho, 2000).

As in wheat, a relatively simple procedure will be used to update yield expectations in any future corn or soybean short crop years. First, trend yield will be used as the expected yield until the August USDA Crop Production Report is released, typically around August 10th. Second, if the USDA corn or soybean yield estimate for the Central Illinois Crop Reporting District is 20% (or more) lower than trend yield, a "reasonable" farmer is assumed to change yield expectations to the lower USDA estimate. Third, as with normal crop years, the adjustment to actual yield is assumed to occur on the first day of harvest.

The 20% threshold is intentionally relatively large for at least three reasons. First, it is desirable to make adjustments to the trend yield expectation on a limited number of occasions. Given the large variability in annual yields, a small threshold could result in frequent adjustments. Second, it is not uncommon for early yield estimates to deviate significantly from the final estimate. A small threshold could result in unnecessary adjustments prior to harvest. Third, yield shortfalls of less than 20% are unlikely to create delivery problems for a farmer.

Several costs are associated with hedging positions in futures and options markets. Brokerage commissions are the first type of hedging cost incurred when farmers open or close positions on an exchange. For the purposes of this study, it is assumed that brokerage costs are $50 per contract for round-turn futures transactions and $30 per contract to enter or exit an options position. Further, it is assumed that CBOT corn and soybean futures and options contracts are used, which have a contract size of 5,000 bushels. Therefore, per-bushel brokerage costs are one cent per bushel for a round-turn futures transaction and 0.6¢ per bushel for each options transaction.

Liquidity costs are the second type of hedging cost incurred when farmers open or close positions on an exchange. These costs reflects the fact that non-floor traders generally must buy at the ask price and sell at the bid price (e.g., Working, 1967; Roll, 1984). The difference between the bid and ask prices, termed the bid-ask spread, is the return earned by floor traders for "making the market." In other words, the bid-ask spread represents the cost paid to execute a trade quickly at prevailing market prices. Liquidity costs are not explicitly accounted for in this study because "fill" prices for futures and options transactions are reported by advisory programs for most transactions. Fill prices presumably already reflect liquidity costs. In cases where a program did not report a specific fill price, the settlement price for that day is used. Liquidity costs are not incorporated for settlement transactions, but this should not represent a significant omission since such transactions are a relatively small component of all futures and options transactions. In addition, liquidity costs should be minimized during the settlement period of the daily trading session due to the relatively high trading volume that typically occurs at that time (e.g., Thompson, Eales and Seibold, 1993).

Mark-to-market costs are a third type of hedging cost that may be incurred by farmers in the course of holding futures and options positions on an exchange. These costs can be incurred as a result of the margining system used for futures and some options positions. Specifically, when a farmer opens a futures position a "good faith" margin deposit is required, typically around 5% of contract value. The initial margin can be deposited in the form of available cash, borrowed funds or an interest bearing instrument such as U.S. treasury bills. So, depending on the form of the deposit, the farmer may experience interest opportunity costs, actual interest costs or interest earnings on the initial margin. If the futures position subsequently accrues losses beyond a certain point (e.g., the futures price increases while holding a short position) a further margin deposit is required. In this way, it is possible for interest borrowing costs to accumulate as losses are experienced. If the futures position subsequently accrues gains, no further margin deposit is required but interest may be earned on the accrued profits. The process of marking-to-the market for futures positions occurs daily and is based on settlement futures prices. The question in the present context is the magnitude of mark-to-the market costs for futures positions in agricultural markets. Previous studies suggest that mark-to-market costs are quite small for hedging positions in agricultural futures markets (Nelson, 1985; Alexander, Musser and Mason, 1986; Matthews and Holthausen, 1991). This is a sensible result, as hedging profits, which generate interest earnings, should approximately offset hedging losses, which generate interest charges, in efficient markets over time. Mark-to-market costs are therefore not incorporated in the simulation of advisory program performance for this study.

It is important to emphasize that the above discussion is not meant to imply that cash flow risk is not an important component of the risk of following advisory program recommendations. While interest costs and earnings for a margin account more than likely cancel each other out over time, hedge positions can still generate large negative cash flows during particular time periods. Zulauf et al. (2001) examine routine pre-harvest marketing strategies for representative Ohio corn and soybean producers over 1986-1999 and find that cash outflow during short crop years can be substantial. For example, cash outflow for a standard short hedging strategy (50 percent of expected production at planting) during the drought of 1988 exceeds $100 per acre. This highlights the potential for large cash outflows that may result from following advisory program recommendations.

While the 1996 "Freedom-to-Farm" Act did away with government set-aside and target price programs, price protection for farmers in program crops such as corn and soybeans was not eliminated entirely. Minimum prices are established through a "loan" program. Specifically, if market prices are below the Commodity Credit Corporation (CCC) loan rate for corn or soybeans, farmers can receive payments from the U.S. government that make up the difference between the loan rate and the lower market price.[20] There is considerable flexibility in the way the loan program can be implemented by farmers. This flexibility presents the opportunity for advisory programs to make specific recommendations for the implementation of the loan program. The price of both corn and soybeans was below the loan rate during significant periods of time in the 1998-2001 marketing years, so that use of the loan program was an important part of marketing strategies. As a result, net advisory program prices were substantially impacted by the way the provisions of the loan program were implemented. Since the price of corn and soybeans in 2002 and 2003 was below the loan rate only briefly, the loan program was at most a minor factor in determining net advisory prices. Nonetheless, any specific advisory program recommendations about the timing and method of implementing the loan program for the 2002 and 2003 crop years are implemented.

Before describing the decision rules, it is useful to provide a brief overview of the loan program mechanics. Then, the rules developed to implement the loan program in the absence of specific recommendations can be described more effectively.

There are two mechanisms for implementing the price protection benefits of the loan program. The first mechanism is the loan deficiency payment (LDP) program. LDPs are computed as the difference between the loan rate for a given county and the posted county price (PCP) for a particular day. PCPs are computed by the USDA and change each day in order to reflect the average market price that exists in the county. For example, if the county loan rate for corn is $2.00 per bushel and the PCP for a given day is $1.50 per bushel, then the LDP is $0.50 per bushel. If the PCP increases to $1.60 per bushel, the LDP will decrease to $0.40 per bushel. Conversely, if the PCP decreases to $1.40 per bushel, the LDP will increase to $0.60 per bushel.[21]

LDPs are made available to farmers over the period beginning with corn or soybean harvest and ending May 31st of the calendar year following harvest. Farmers have flexibility with regard to taking the LDP, because they may simply elect to take the payment when the crop is sold in a spot market transaction (before the end of May in the particular marketing year), or choose to take the LDP before the crop is delivered and sold. Note that LDPs cannot be taken after a crop has been delivered and title has changed hands.

The second mechanism is the non-recourse marketing assistance loan program. A loan cannot be taken on any portion of the crop for which an LDP has been received. Under this program, farmers may store the crop (on the farm or commercially), maintain beneficial interest, and receive a loan from the CCC using the stored crop as collateral. The loan rate is the established rate in the county where the crop is stored and the interest rate is established at the time of loan entry. Corn and soybean crops can be placed under loan anytime after the crop is stored through May 31st of the following calendar year. The loan matures on the last day of the ninth month following the month in which the loan was made.

Farmers may settle outstanding loans in two ways: i) repaying the loan during the 9-month loan period, or ii) forfeiting the crop to the CCC at maturity of the loan. Under the first alternative, the loan repayment rate is the lower of the county loan rate plus accrued interest or the marketing loan repayment rate, which is the PCP. If the PCP is below the county loan rate, the economic incentive is to repay the loan at the posted county price. The difference between the loan rate and the repayment rate is a marketing loan gain (MLG). If the PCP is higher than the loan rate, but lower than the loan rate plus accrued interest, the incentive is also to repay the loan at the PCP. In this case only, interest is charged on the difference between the PCP and the loan rate. If the PCP is higher than the loan rate plus accrued interest, the incentive is to repay the loan at the loan rate plus interest. In this latter case, interest is based on the loan rate. Under the second alternative, the farmer stores the crop to loan maturity and then transfers title to the CCC. The farmer retains the proceeds from the initial loan.

The non-recourse loan program establishes the county loan rate as a minimum price for the farmer, as does the LDP program. For the 2002 and 2003 crops, the sum of LDPs plus marketing loan gains was subject to a payment limitation of $150,000 per person. Forfeiture on the loans or use of commodity certificates provide a mechanism for receiving a minimum of the loan rate on bushels in excess of the payment limitation.

The average loan rates for the 2002 and 2003 corn crops across the eleven counties in the Central Illinois Crop Reporting District are $2.06 and $2.04 per bushel, respectively. The average loan rates for the 2002 and 2003 soybean crops across the eleven counties in the Central Illinois Crop Reporting District are $5.16 and $5.14 per bushel, respectively. Spot cash prices for corn and soybeans fall below these loan rates briefly or not at all during the 2002 and 2003 post-harvest periods. This is reflected in Figures 4 and 5, which show corn and soybean LDP or MLG rates for central Illinois during the 2002 and 2003 post-harvest periods. [22], [23] For 2002 crops, positive LDPs or MLGs are observed in corn for about a month during the summer of 2003 and in soybeans for a brief period during October 2002. For 2003 crops, positive LDPs or MLGs are limited to about half of the harvest period in corn and never occur in soybeans. As mentioned earlier, the limited availability of LDP or MLG payments reflects the relatively strong pattern of spot cash prices for corn and soybeans in 2002 and 2003.

If an advisory program makes a complete set of loan recommendations, the specific advice is implemented wherever feasible. However, specific decision rules are still needed regarding pre-harvest forward contracts because it is possible for an advisory program to recommend taking the LDP on those sales before it is actually harvested and available for delivery in central Illinois. To begin, it is assumed that amounts sold for harvest delivery with pre-harvest forward contracts are delivered first during harvest. Since LDPs must be taken when title to the grain changes hands, LDPs are assigned as these "forward contract" quantities are harvested and delivered. This necessitates assumptions regarding the timing and speed of harvest. Earlier it was noted that a five-week harvest window is used to define harvest. This window is centered on the day nearest to the mid-point of harvest progress as reported by NASS. Various assumptions could be implemented regarding harvest progress during this window. Lacking more precise data, a reasonable assumption is that harvest progress for an individual representative farm is a linear function of time.

Tables 2 through 5 summarize the information used to assign LDPs to pre-harvest forward contracts. The second column shows the amount harvested assuming a linear model. The third column shows the LDP available on each date of the harvest window and the fourth column presents the average LDP through each harvest date. An example for 2003 will help illustrate use of the tables. Assume that an advisory program recommends, at some point before harvest, that a farmer forward contract 50% of expected corn production. This translates into 78.1 bushels per acre when the percentage is applied to expected production (0.50*156.1 = 78.1). Next, convert the bushels per acre to a percentage of actual production, which is 42.7% (78.1/183 = 0.427). To determine the LDP payment on the 42.7% of actual production forward contracted, simply read down Table 3 to October 2, 2003, which is the date when 42.7% of harvest is assumed to be complete. The average LDP up to that date (September 18, 2003- October 2, 2003) is $0.01 per bushel; the last column of Table 3. This is the LDP amount assigned to the forward contract bushels.

Note that LDPs for any sales (spot, forward contracts, futures or options) recommended during harvest are taken only after all forward contract obligations are fulfilled. Grain industry practices may actually offer more flexibility in establishing LDPs than is assumed here. In addition, so long as prices remain below the loan rate, crops placed under loan by an advisory program do not accumulate interest opportunity costs because proceeds from the loan can be used to offset interest costs that otherwise would accumulate.

Decision Rules for Programs with a Partial Set of Loan Recommendations
Or No Loan Recommendations

If an advisory program makes a partial set of loan recommendations, the available advice is implemented wherever feasible. In the absence of specific recommendations, it is assumed that crops priced before May 31st but after harvest are not placed under loan. Those crops receive program benefits, if any, through LDPs. After May 31st, eligible crops (unpriced crops for which any program benefits have not yet been collected) are assumed to be under loan until priced only if cash prices prevailing on May 31st are near or below the loan rate.

In the absence of specific recommendations, rules for assigning LDPs and MLGs are developed under the assumption that loan benefits are established when the crop is priced or as soon after pricing that is allowed under the rules of the program. This principle is consistent with the intent of the loan program to fix a minimum price when pricing decisions are made. Two rules are most important in the implementation of this principle. First, LDPs on pre-harvest sales (forward contracts, futures or options) are established as the crop is harvested. Second, if the LDP or MLG is zero on the pricing date, or the first date of eligibility to receive a loan benefit, those values are assigned on the first date when a positive value is observed, assuming a beneficial interest in that portion of the crop has been maintained. Specific rules for particular marketing tools and situations follow:

1) Pre-harvest forward contracts. The same decision rules are applied as discussed in the previous section. Specifically, it is assumed that amounts sold for harvest delivery with pre-harvest forward contracts are delivered first during harvest, although not all buyers require that forward contract bushels be delivered first. LDPs, if positive, are assigned as these "forward contract" quantities are harvested and delivered. This necessitates assumptions regarding the timing and speed of harvest. A linear model of harvest progress is assumed in the five-week harvest window. The specific information used to assign LDPs to pre-harvest forward contracts is again found in Tables 2 through 5. As a final point, note that LDPs for any other sales (spot, futures or options) recommended during harvest are taken only after all pre-harvest forward pricing obligations are fulfilled.

2) Pre-harvest short futures. The use of futures contracts to price during the pre-harvest seasons is treated in the same manner as pre-harvest forward contracts. LDPs are assigned on open futures positions as the crop is harvested, or as soon as a positive LDP is available, if the futures position is still in place and cash sales have not yet been made. These are assigned after forward contracts have been satisfied. If the underlying crop is sold before there is a positive LDP, then that portion of the crop receives a zero LDP. During the harvest window, if the futures position is offset before a positive LDP is available and the crop has not yet been sold in the cash market, that portion of the crop is eligible for loan benefits on the next pricing recommendation.

3) Pre-harvest put option purchases. Long put option positions, which establish a minimum
futures price, are treated in the same manner as pre-harvest short futures.

4) Post-harvest forward contracts. The main issue with respect to post-harvest forward contracts is when to assign the LDPs or MLGs. Those can be established on the date the contract is initiated, on the delivery date of the contract, or anytime in between. Following the general principle outlined earlier, LDPs and MLGs for post-harvest contracts are assigned on the date the contract is initiated or the first day with positive benefits prior to delivery on the contract.

5) Post-harvest short futures. As with post-harvest forward contracts, the main issue with post-harvest short futures positions is when to assign loan benefits. These are assigned when the short futures position is initiated or as soon as a positive benefit is available if the futures position is still in place and cash sales have not been made. If the underlying crop is sold before a positive LDP is available, that portion of the crop receives a zero LDP. If the short futures position is offset before a positive LDP is available and the cash crop has not yet been sold, that portion of the crop is eligible for loan benefits on the next pricing recommendation.

6) Post-harvest long put positions. Long put option positions established after the crop is harvested are treated in the same manner as post-harvest short futures.

7) Spot sales before May 31st. If a spot cash sale of corn or soybeans is recommended before May 31st but after harvest, it is assumed that the LDP, if positive, is established that same day.

8) Loan program after May 31st. LDPs are not available after May 31st for 2002 and 2003 crops. In previous years it was assumed that any corn or soybeans in storage and not priced as of this date, for which loan benefits had not been established, were entered in the loan program on that date. However, this is not a reasonable assumption for 2002 and 2003 crops since spot prices were well above the loan rate for corn and soybeans in central Illinois on May 31, 2002 and May 31, 2003. A prudent farmer would not necessarily enter the loan program under these circumstances, and hence, when crops are subsequently priced (cash sale, forward contract, short futures, or long put option), no marketing loan gain is assigned on that day.

An important element in assessing returns to an advisory program is the economic cost associated with storing grain instead of selling grain immediately at harvest. The cost of storing grain after harvest consists of two components: physical storage costs and the opportunity cost incurred by foregoing sales when the crop is harvested. Physical storage costs depend on the type of storage available and the horizon used by a farmer to make storage decisions. From a representative farmer's perspective, there are four relevant physical storage scenarios: i) on-farm storage using a short-run decision-horizon, ii) off-farm (commercial) storage using a short-run decision-horizon, iii) on-farm storage using a long-run decision-horizon and iv) off-farm (commercial) storage using a long-run decision-horizon. Short-run in this context is defined to be one storage season, usually the ten-month period after the harvest of a particular crop. Long-run is defined to be any decision-horizon longer than one storage season. In each of the previous scenarios, the physical storage charge should be the relevant marginal cost of physical storage (Williams and Wright, 1991). In contrast, opportunity cost should be the same regardless of the type of physical storage used or whether a short- or long-run decision-horizon is considered.

Early AgMAS pricing reports consider only one scenario: commercial storage using a short-run decision-horizon. Starting with the 2000 crop year, net advisory prices and benchmarks are computed using physical storage costs applicable to each of the four storage scenarios. In all cases for 2002, storage and interest charges are assigned beginning October 24, 2002 for corn and October 25, 2002 for soybeans, the first dates after the end of the respective 2002 harvest windows. In all cases for 2003, storage and interest charges are assigned beginning October 23, 2003 for corn and October 22, 2003 for soybeans, the first dates after the end of the respective 2003 harvest windows. It should be noted that the cost of drying corn to 15% moisture and the cost of drying soybeans to storable moisture are not included in the calculations. This cost is incurred whether the grain is stored or sold at harvest, or whether the grain is stored on-farm or off-farm. Therefore, this cost is irrelevant to the analysis and excluded.

The first scenario considered is on-farm storage and a short-run decision-horizon. Because pre-existing storage facilities are assumed to be available on-farm, the marginal cost of physical storage equals the on-farm variable cost of physical storage. Estimates of the on-farm variable cost of physical storage are drawn from a recent study conducted at Kansas State University (Dhuyvetter, Hamman and Harner, 2000). The estimates assume storage occurs in a 25,000 bushel round metal bin, the "medium-sized" storage capacity examined in the Kansas State study. The first component of on-farm physical storage is a flat charge of 6.7¢ per bushel for conveyance, aeration, insecticide and repairs. The flat charge is applied to both corn and soybeans and reflects the fact that most physical costs of on-farm storage are "one-time" in nature. That is, once the decision is made to store, most costs are pre-determined and do not vary with the length of storage.

The second component of on-farm physical storage is shrinkage. Corn shrinkage is assumed in the Kansas State study to start at one-percent per bushel for the first month of storage and increase at a rate of one-tenth of one percent for each month stored thereafter. For example, if corn is stored six months, the total shrinkage is assumed to be 1.5% per bushel. Agricultural engineering specialists at the University of Illinois and Purdue University indicated that the on-farm shrink schedule for corn used in the Kansas State study is reasonable. In addition, the schedule is consistent with published research about shrinkage of corn stored on-farm (Hurburgh, Bern, Wilcke and Anderson, 1983). Given that the harvest-time cash price of corn in central Illinois for 2002 is $2.43 per bushel, the shrink charge assigned to corn stored on-farm for one-month in 2002 is 2.43¢ per bushel ($2.43*0.01*100). The shrink charge in 2002 is increased 0.24¢ per bushel ($2.43*0.001*100) for each additional month of storage. Given that the harvest-time cash price of corn in central Illinois for 2003 is $2.04 per bushel, the shrink charge assigned to corn stored on-farm in 2003 for one-month is 2.04¢ per bushel ($2.04*0.01*100). The shrink charge in 2003 is increased 0.20¢ per bushel ($2.04*0.001*100) for each additional month of storage.[24]

Since the Kansas State study did not estimate shrinkage costs for soybeans, the same agricultural engineering specialists noted above were consulted for a reasonable estimate. This turned out to be a constant 0.25% per bushel shrink factor. Given that the harvest-time cash price of soybeans in central Illinois for 2002 is $5.28 per bushel, the flat shrink charge assigned to soybeans in 2002 is 1.32¢ per bushel ($5.28*0.0025*100). Given that the harvest-time cash price of soybeans in central Illinois for 2003 is $6.66 per bushel, the flat shrink charge assigned to soybeans in 2003 is 1.67¢ per bushel ($6.66*0.0025*100).[25]

As noted earlier, storage costs include the physical cost of storage and interest opportunity costs. Interest cost in 2002 is computed using the 2002 harvest cash price and an annual interest rate of 6.7%. Interest cost in 2003 is computed using the 2003 harvest cash price and an annual interest rate of 6.3%. Specifically, the interest charge for storing grain on-farm is computed as the harvest price times the interest rate compounded daily from the end of harvest to the date of sale. The interest rate is the average rate for all other farm operating loans for Seventh Federal Reserve District agricultural banks in the fourth quarter of 2002 and 2003 as reported in the Agricultural Finance Databook, which is published by the Board of Governors of the Federal Reserve Board. Interest rates for the fourth quarter are assumed to most accurately reflect actual opportunity costs on agricultural loans related to storage.[26]

The second scenario considered is storage off-farm at commercial facilities and a short-run decision-horizon. The marginal cost of physical storage in this case is the sum of commercial storage, drying and shrinkage charges. As in the past, storage costs at commercial elevators in 2002 and 2003 are drawn from an informal telephone survey of nine central Illinois elevators.[27] Based on this information, physical commercial storage charges are assumed to be a flat 13¢ per bushel from the end of harvest through December 31. After January 1, physical storage charges are assumed to be 2¢ per month (per bushel), with this charge pro-rated to the day when the cash sale is made. The drying charge to reduce corn moisture from 15% to 14% is a flat 2¢ per bushel, while the charge for shrinkage is 1.3% per bushel.[28] The cost of commercial shrinkage is based on the harvest price (no shrinkage is assumed for soybeans in commercial storage). Given that the harvest-time cash price of corn in central Illinois for 2002 is $2.43 per bushel, the charge for volume reduction is 3.16¢ per bushel ($2.43*0.013*100). Therefore, the flat shrink and drying charge assigned to all stored corn in 2002 is 5.16¢ per bushel. Given that the harvest-time cash price of corn in central Illinois for 2003 is $2.04 per bushel, the charge for volume reduction is 2.65¢ per bushel ($2.04*0.013*100). Therefore, the flat shrink and drying charge assigned to all stored corn in 2003 is 4.65¢ per bushel.[29] Interest opportunity cost is computed using the same procedures and assumptions as outlined above for on-farm storage.

The third and fourth scenarios shift to a long-run decision-horizon, where the on-farm scenario is applicable to a farmer considering the construction of new on-farm storage facilities and the commercial scenario is applicable to a farmer that plans on using commercial storage facilities over the long-run. Since all costs are variable in the long-run, the relevant marginal physical storage cost in both of these scenarios is the total cost. Dhuyvetter, Hamman and Harner (2000) estimate the on-farm fixed cost of physical storage for a 25,000 bushel round, metal bin to be 14.6¢ per year. This fixed cost can be added to the on-farm variable cost estimate discussed earlier to compute the total physical cost of on-farm storage. Presumably, commercial physical storage charges paid by farmers reflect total variable and fixed costs of storage at commercial facilities. Consequently, the commercial storage costs discussed earlier in the context of short-run decisions also represent long-run commercial physical costs.

A comparison of the estimated costs of storage for corn and soybeans in the 2002 and 2003 crop years is found in Tables 6 through 9, respectively. The first item of note is that the on-farm variable cost of physical storage changes little for corn as the storage length increases and is constant for soybeans as the storage length increases. The reason is the previously mentioned "one-time" nature of most physical costs of on-farm storage. As shown in panel A of Figures 6 and 7, this results in a "non-linear" relationship between on-farm variable costs of storage per month and the length of storage. For example, the on-farm variable cost for corn stored two months after harvest in either crop year is about 5¢ per month. This can be compared to the on-farm variable cost of corn stored six months after harvest of about 2¢ per month. The second item of note is the much lower level of on-farm variable costs versus commercial storage costs. Of course, this is not surprising given that variable on-farm storage costs do not include fixed costs, while commercial storage costs presumably reflect total variable and fixed storage costs at commercial facilities. The third item of note is the similar level of total on-farm costs (variable plus fixed) and total commercial costs for all but the shortest and longest storage lengths. Figures 6 and 7 illustrate these findings on a per month basis. This result is not surprising assuming reasonably competitive conditions in the market for storage. If total on-farm storage costs were substantially less than total commercial costs, this would encourage a rapid expansion of on-farm storage and vice versa. In fact, the proportion of on-farm versus off-farm storage capacity in Illinois has been roughly equal for a number of years.[30] This is consistent with a basic equilibrium in the storage market where total on-farm costs and commercial costs are about the same.

Given the information presented in Tables 6 through 9, it is possible to compute net advisory prices and benchmarks under each of the four storage scenarios described at the beginning of this section. It turns out that only two sets of storage costs are necessary to represent all four scenarios. Most obviously, on-farm storage costs in the short-run are estimated by on-farm variable storage costs (fourth column in Tables 6 through 9). Commercial storage costs in the short-run and long-run can be estimated by commercial storage costs (last column in Tables 6 through 9). Based on the equilibrium argument made above, on-farm storage costs in the long-run can also be estimated based on commercial storage costs. Therefore, in the remainder of this report, reference will be made only to on-farm variable storage costs and commercial storage costs.

The calculation of storage charges may be impacted by an advisory program's loan recommendations and/or the decision rules discussed in the previous section. Specifically, during the period corn or soybeans are placed under loan, interest costs are not accumulated, as the proceeds from the loan can be used to offset interest opportunity costs that otherwise would accumulate. This most commonly occurs after May 31st, when un-priced grain for which loan benefits have not been collected can be placed under loan until priced.[31] If a crop is priced (forward contracts, futures or options) while under loan but stored beyond the time of pricing, interest opportunity costs are accumulated from the day of pricing until the time storage ceases (since it is assumed the loan is repaid on the date of pricing).

It could be argued that interest opportunity costs should be charged based on the LDP available at harvest but not taken by an advisory program. This adjustment is not made because it would not substantially impact the results due to the small interest opportunity costs involved.

A final issue related to storage costs is the use of different strategies based on the availability of on-farm storage. Specifically, as noted earlier in the "Data Collection" section, advisory programs may issue one set of recommendations assuming on-farm storage is available and another set of recommendations assuming only commercial storage is available. From a practical standpoint, the alternative strategies must be differentiated before grain is placed in on-farm or commercial facilities. After harvest, when grain has already been placed in on-farm or commercial storage facilities, such advice is of little practical value to most farmers. Hence, if a program clearly differentiates on-farm and commercial storage strategies at or before harvest of the 2002 and 2003 crops, the on-farm recommendations are used in computing the net advisory price under on-farm variable costs and the commercial recommendations are used in computing the net advisory price under commercial costs. In this case, the net advisory price for a program under the two alternative storage cost assumptions will vary due to the difference in costs and underlying strategies. If a service does not clearly differentiate on-farm and commercial storage strategies during harvest of the 2002 and 2003 crops, the same recommendations are used in computing net advisory prices under on-farm variable and commercial storage costs. In this case, the net advisory price for a program under the two alternative storage cost assumptions will vary only due to the difference in costs, as the underlying strategies are the same.[32]

The essential concept underlying performance evaluation of market advisory programs is fairly simple: the comparison of the net prices generated by advisory programs with prices that could have been obtained by a farmer through one or more appropriate alternative strategies (Sharpe, Alexander and Bailey, 1999, p. 829). The comparison strategies are commonly referred to as benchmarks because they serve as objective standards of performance, much like a yardstick provides an objective measurement of distance. Within this broad framework, two basic types of performance evaluation can be applied to market advisory programs. The first type is based on comparison to "peer-group" benchmarks, whereby net advisory prices are compared to each other or the average price across all advisory programs. The second type is based on comparison to "external" benchmarks, whereby net advisory prices are compared to prices from strategies that do not depend upon market advisory program behavior. In financial markets, it is commonplace to compare investment performance to external benchmarks, such as the Dow-Jones Industrials Index, S&P 500 Index and Wilshire 5000 Index.

The AgMAS study focuses on performance evaluation using external benchmarks. While peer-group evaluation provides useful information about the rank of advisory programs, it cannot answer the question of whether performance of advisory programs as a group or an individual advisory program is "superior" or "inferior" in an absolute economic sense. To answer this question, external benchmarks must be specified based on theories of market pricing.

The first class of external benchmarks is based on the theory of efficient markets. This theory assumes that market participants are rational and that competition instantaneously eliminates all profitable arbitrage opportunities. In its strongest form, efficient market theory predicts that market prices always fully reflect available public and private information (Fama, 1970). The practical implication is that no trading strategy can consistently beat the return offered by the market (e.g., Brorsen and Anderson, 1994; Brorsen and Irwin, 1996; Zulauf and Irwin, 1998). Hence, the return offered by the market becomes the relevant benchmark. In the context of the AgMAS study, a market benchmark should measure the average price offered by the market over the marketing window of a representative farmer who follows advisory program recommendations. The average price is computed in order to reflect the returns to a naïve, "no-information" strategy of marketing equal amounts of grain each day during the marketing window. The difference between advisory prices and the market benchmark measures the value of advisory service information. The theory of efficient markets predicts this difference, on average, will equal zero.[33]

If all market participants are rational in the way efficient market theory assumes, then the only interesting external benchmarks are market benchmarks. However, there is growing evidence that many market participants may not be fully rational in the efficient market sense. Hirshleifer (2001) provides a comprehensive review of the judgment and decision biases that appear to affect securities market investors, such as framing effects, mental accounting, anchoring and overconfidence. He also provides an exhaustive review of empirical studies that attempt to measure the potential impact of such biases on securities prices and investment returns. As an example, Barber and Odean (2000) find that individual stock investors under-perform the market by an average of one-and-a-half percentage points per year, an economically significant amount, particularly when viewed over long investment horizons. They argue that a combination of overconfidence and excessive trading explains this finding. Brorsen and Anderson (2001) provide an illuminating discussion of how judgment and decision biases may impact farm marketing. Finally, new "behavioral" theories of market pricing have been developed based on the assumption that market participants are subject to judgment and decision biases (e.g., Daniel, Hirshleifer and Subrahmanyam, 1998).

Behavioral market theory suggests that the average return actually achieved by many market participants may be less than that predicted by efficient market theory, due to the judgment and decision biases that plague most participants. As a result, the average return actually received by market participants becomes an appropriate external benchmark. In the context of the AgMAS study, a behavioral benchmark should measure the average price actually received by farmers for a crop. The difference between net advisory prices and a farmer benchmark should then measure the value of market advisory service information relative to the information used by farmers. Behavioral market theory does not predict a specific value for this difference. It may be positive, negative or zero, depending on the impact of judgment and decision biases on advisory programs versus farmers. Finally, it is important to emphasize that the farmer benchmark should be based on the pricing performance of farmers who do not follow the advisory programs tracked by the AgMAS Project, otherwise, the value of market advisory service information relative to the information used by farmers cannot be "cleanly" disentangled.

It is important to re-iterate that market and farmer benchmarks convey quite different information about the performance of market advisory programs, even though both are forms of an external benchmark. This should be carefully considered when making performance comparisons based on the two types of benchmarks. In addition, there are some desirable properties from a practical perspective that both types of benchmarks should possess: i) they should be relatively simple to understand and to calculate; ii) they should represent the returns to a marketing strategy that can be implemented by farmers; and iii) they should be directly comparable to net advisory prices (Good, Irwin and Jackson, 1998).

As pointed out in the previous section, a market benchmark is designed to measure the average price offered by the market to farmers. The appropriate time period for computing the average price is the marketing window of a farmer who follows the recommendations of the advisory programs included in the AgMAS study. This window was defined earlier (see the "Marketing Window" section) as the 24-month period that begins on September 1st of the year before harvest and ends on August 31st of the year after harvest. A 24-month market benchmark is simply computed as the average price over the two-year marketing window. It should be noted that this specification of a market benchmark is substantially different than common practice of using the average harvest price as a market benchmark. The analysis found later in this section implies that using the average price during a relatively short time period, such as harvest, may introduce excessive year-to-year variation in the benchmark.

Figure 8 presents average marketing profiles for market benchmarks and advisory programs in corn and soybeans over the 1995-2001 crop years. For comparison purposes, average marketing profiles for 24- and 20-month market benchmarks are included. The 20-month benchmark simply deletes the first four months of the 24-month marketing window from the computations of the average market price. As a result, this benchmark is based on the average price over the period that begins on January 1 of the year of harvest and ends on August 31 of the year after harvest. For both corn and soybeans, the market benchmarks appear to provide a surprisingly good "fit" to the average profile of the advisory programs. More specifically, if a simple linear trend regression is fit to the average profile of the advisory programs (not shown), the estimated trend line is remarkably close to the 24-month benchmark for corn and the 20-month benchmark for soybeans.

The results discussed in the previous paragraph suggest there is some uncertainty about specification of the most appropriate market benchmark for corn and soybean performance evaluations. Leamer (1983) argues persuasively (and famously) that in this type of situation it is crucial to understand the "fragility" of results when key assumptions are changed. Consequently, both a 24-month and a 20-month market benchmark will be used in comparisons to net advisory prices. Cash forward prices for central Illinois are used during the pre-harvest period, while daily spot prices for central Illinois are used for the post-harvest period. The same forward and spot price series applied to advisory program recommendations are used to construct both market benchmarks. Details on the forward and cash price series can be found in the earlier "Prices" section of this report.

Three adjustments are made to the daily cash prices to make the 24-month and 20-month average cash price benchmarks consistent with the calculated net advisory prices for each marketing program. The first is to take a weighted-average price, to account for changing yield expectations, instead of taking the simple average of daily prices. This adjustment is consistent with the procedure described previously in the "Yields and Harvest Definition" section. The daily weighting factors for pre-harvest prices are based on the calculated trend yield, while the weighting of the post-harvest prices is based on the actual reported yield for central Illinois. The second adjustment is to compute post-harvest cash prices on a harvest equivalent basis, which is done by subtracting on-farm variable or commercial storage costs (physical storage, shrinkage and interest) from post-harvest spot cash prices. The daily storage charges are calculated in the same manner as those for net advisory prices. The third adjustment is made with respect to the loan program. In the context of evaluating advisory program recommendations, it was argued earlier that a "prudent" or "rational" farmer would take advantage of the price protection offered by the loan program, even in the absence of specific advice from an advisory program. This same logic suggests that a "prudent" or "rational" farmer will take advantage of the price protection offered by the loan program when following the benchmark average price strategy. Based on this argument, the 24-month and 20-month average cash price benchmarks are adjusted by the addition of LDPs and MLGs. Bushels marketed in the pre-harvest period according to the benchmark strategy are treated as forward contracts, with the LDPs assigned at harvest. Bushels marketed each day in the post-harvest period are awarded the LDP or MLG in existence for that particular day. Finally, just as in the case with comparable advisory program recommendations, un-priced grain on May 31st is placed under loan if the market price is near or below the loan rate. Interest opportunity costs are not charged to the benchmark after this date if cash prices on the date of loan redemption are below the CCC loan rate.[34] Since market prices were substantially above the loan rate on May 31, 2003 and May 31, 2004 for both corn and soybeans, it is assumed that un-priced grain is not placed under loan on these dates for the 2002 and 2003 crops.

While the 24- and 20-month market benchmark prices can obviously differ for a given crop year, averages of the two benchmark prices across crop years are not expected to differ substantially. First, the difference in the marketing windows for the two benchmarks is relatively small, as the 20-month benchmark reduces the 24-month marketing window by only about 17%. Second, given a sufficiently large sample of crop years and efficient corn and soybean markets (cash, futures and options), the law of one price implies that annual averages of different average price benchmarks should be equal when stated on a harvest equivalent basis (Brorsen and Anderson, 1994). Of course, if corn and soybean markets are inefficient, the equivalence would not hold. In particular, if pre-harvest prices contain a "drought premium" as some argue (e.g., Wisner, Baldwin and Blue, 1998), then the 24-month benchmark price may be consistently higher or lower than the 20-month benchmark price, depending on the evolution of the drought premium.[35]

In contrast to averages, the variation of 24- and 20-month market benchmark prices across crop years is expected to differ. One reason for the difference is the well-known result in statistics that the sampling variation of the mean (average) is inversely related to the sample size used to compute the average (e.g., Griffiths, Hill and Judge, 1993, p.82). Since the sample of daily prices used in computing the 24-month benchmark is larger than the sample for the 20-month benchmark, the variation of the 24-month benchmark should be smaller than variation of the 20-month benchmark. Another reason is that the volatility of spot prices for storable commodities such as corn and soybeans increases as one moves from early in the 12-month marketing year (e.g., harvest) to later in the marketing year (Williams and Wright, 1991; Peterson and Tomek, 2003). The increase in volatility is driven by the decline in stocks that normally occurs during the marketing year. Specifically, available stocks are largest at harvest and then decline through the remainder of the marketing year, and consequently, a given demand shock will have the largest impact on price later in the marketing year. In terms of market benchmarks, this implies that the 20-month benchmark, which gives more weight to prices later in the marketing year, will be more volatile than the 24-month benchmark.[36]

A practical concern with the market benchmarks is that a farmer may not be able to implement the benchmark strategies since they involve marketing a small portion of the crop every day. There are two reasons to believe this concern is not overly serious. First, a number of companies have developed and offer grain "index" contracts that allow farmers to receive the average market price over a pre-specified time interval. An extensive discussion of these new contracts can be found in the AgMAS Research Report by Hagedorn et al. (2003). Second, a strategy of routinely selling at less frequent intervals closely approximates the market benchmark prices. For example, a farmer might consider alternative "tracking" strategies of marketing only once a month or once every other month over the 24-month window.[37] Using mid-month prices, a tracking strategy of marketing only once a month (24 times) generates average prices over 1995-2003 that are quite close to 24-month market benchmark prices. The average difference is only two cents per bushel for corn and soybeans, with a maximum difference for any particular crop year is eight cents per bushel in corn and five cents per bushel in soybeans. A tracking strategy of marketing once every other month (12 times) also generates average prices over 1995-2003 that are quite close to 24-month market benchmark prices. The average difference is only two cents per bushel for corn and five cents per bushel for soybeans.

The average difference results for the benchmark tracking strategies should not be a surprise given the previous argument about averages of different benchmark prices in efficient markets. More surprising is the result that the variation of the tracking strategies across crop years is only two to four cents per bushel (three to nine percent) more than the 24-month benchmark over 1995-2003. This is surprising because the tracking strategies are based on dramatically smaller samples, 12 or 24 observations compared to about 500 observations for the 24-month benchmark, but have only a marginally higher variation across crop years. The most likely explanation is that observations for the tracking strategies are not selected at random, but are instead equally spaced across the entire marketing window. Further research is needed to fully understand the behavior of tracking strategies in corn and soybean markets.

As noted earlier, a farmer benchmark is designed to measure the average price received by farmers for a crop. This type of benchmark should reflect the actual behavior of farmers in marketing grain, and include all of the transactions (e.g., cash, forward, futures and options) that farmers employ in this regard. In addition, the farmer benchmark should be based on the pricing performance of farmers who do not follow the advisory programs tracked by the AgMAS Project. In theory, such a farmer benchmark should not be difficult to calculate. First, a representative sample of grain farmers in the relevant geographic area who do not follow the programs in the AgMAS Project would be drawn (randomly). Next, the average price received by each farmer would be computed (using the same assumptions as in the computation of net advisory prices and market benchmarks). Last, the farmer benchmark would be computed as the weighted-average price received by all farmers in the sample, with the weights equal to the sample proportion of the crop produced by each farmer.

In practice, the detailed type of data needed to construct a valid farmer benchmark is not available, so an approximation must be used. The only known approximation is the USDA average price received series. In Illinois, this series is based on information collected in monthly mail and telephone surveys of about 200 grain dealers, processors and elevators that actively purchase grain from farmers (Harden, 2003). The survey is conducted by the Illinois Agricultural Statistics Service, the state office for the National Agricultural Statistics Service of the USDA.[38] Surveyed firms report total quantities and gross value for grain purchased directly from farmers (USDA, NASS, 2002). Total quantities are reported on a dry, or shrunk, basis at the standard moisture content for the commodity. Total gross value is the value of bushels purchased from farmers after deducting price discounts and adding premiums for quality factors and moisture content and adding premiums for direct delivery to mill, processor, river terminal or rail terminal. Check-off fees and charges for drying, cleaning, storing or grading are not deducted. The general principle used to determine the timing of transactions is the month when grain is purchased, that is, when cash changes hand between the firm and farmers. Hence, cash sales and forward contracts are reported for the month of delivery. Basis, minimum price, option and hedge-to-arrive contracts also are reported for the month of delivery. Alternatively, deferred payment and delayed pricing contracts are reported in the month when payment is received. The average price received estimate for a month is the total gross value across all surveyed firms divided by total quantities summed across all surveyed firms. This estimate may incorporate statistical adjustments that reflect size differences across reporting firms and other factors.

The USDA price received series has both strengths and weaknesses with respect to measuring the average price received by (unadvised) farmers. On the positive side, the USDA series reflects the actual pattern of cash grain marketing transactions by farmers, and thus, incorporates the marketing windows and timing strategies actually used by farmers; includes forward contract transactions for both the pre-harvest and post-harvest periods, with the transactions recorded at the forward price, not the spot price at the time of delivery; and grain sales are adjusted to industry standards for moisture. On the negative side, the USDA series is only available in the form of a state average; includes cash transactions for different grades and quality of grain sold by farmers; does not include futures and options trading profits/losses of farmers; reflects a mix of old and new crop sales by farmers; and is based on the pricing behavior of both unadvised and advised farmers.

Fortunately, none of the problems mentioned above appear to be prohibitive with respect to the use of the USDA series as a measure of the average price received by farmers. Consider first the state average nature of the series. It is straightforward to adjust the USDA series to an alternative geographic location, since spatial basis patterns are relatively stable. This type of adjustment turns out not to be necessary for AgMAS performance evaluations because central Illinois prices closely mirror the average price for the entire state of Illinois. Based on an analysis of weekly prices, the average cash price for central Illinois over January 1995 - December 2003 differs from the state average price by only about one-half cent and one cent, respectively, for corn and soybeans (state average lower for both corn and soybeans). The correlation of changes in weekly prices for central Illinois and the state is 0.97 for corn and 0.99 for soybeans. Hence, from a statistical standpoint, central Illinois and state average prices are nearly equivalent.

While it is not possible to adjust the USDA series to a constant grade and quality, to reflect futures and options trading profits/losses of farmers or to only reflect new crop sales, because the data simply are not available, the resulting biases probably are small and some may work in opposite directions. Examining the grade and quality issue first, it is well known that some fraction of the corn crop is discounted relative to the standard number two yellow corn grade. This is also true for the soybean crop relative to the standard number one yellow soybean grade, but likely to a smaller extent than corn. As a result, the USDA average price received reflects a weighted-average of both undiscounted and discounted grain sales. The weights are unknown, but the direction of the bias relative to average prices for the standard grade is clearly downward. In other words, when compared to the average price at the standard grade, the USDA average price received should be adjusted upwards to reflect the impact of discounts.

A key question, of course, is the magnitude of the grade and quality bias discussed above. An extensive search of the literature was conducted and no previous study was uncovered that directly measured the proportion of corn and soybeans sold at a discount or the average magnitude of price discounts in central Illinois (or other Midwestern U.S. areas). The Federal Grain Inspection Service of the U.S. Department of Agriculture (FGIS) was contacted and staff indicated that FGIS does not have an historical series of this type. One older study was located that contained some information on the issue. Hill, Kunda and Rehtmeyer (1983) reported the results of a 1982 survey of grain elevator operators in Illinois. One question in this survey asked elevator operators to estimate the percentage of corn and soybean receipts at country elevators that typically exceed grade factors. Unfortunately, the results were not netted across grade factors, so it is not possible to estimate the typical proportion of the crop sold at a discount (if a lot is over one grade limit it will have a higher than average chance of being over the grade limit for other factors). In addition, the average magnitude that grade factors were exceeded is not reported, so it is impossible to estimate the dollar value of the average discount. Nonetheless, the results provide some perspective on the quality issue. For corn delivered in the fall, the percentage typically above a grade factor ranged from 0.2 to 7.5% of deliveries. For soybeans delivered in the fall, the percentages were about the same, except for foreign material, where over 30% of the bushels delivered typically exceeded the grade factor. When winter and summer delivery was considered, the percentages increased somewhat for corn and decreased for soybeans. Other than foreign material for soybeans, this evidence suggests that less than 10% of the corn and soybean crops in the early 1980s were sold at a discount to the standard grade.

To provide more recent evidence on quality, the nine central Illinois elevators surveyed annually for commercial storage costs were queried in December 2001 about the average quality of corn and soybean crops. The most frequent response from the elevator managers in this informal survey was that less than one percent of corn and soybeans is sold at a discount relative to the standard grade. The range was from zero to less than five percent. The largest estimate of the average dollar value of discounts was two to three cents per bushel. These figures provide enough information to make a very rough estimate of maximum quality bias in the USDA average price received series. Using the maximum proportion of five percent and the maximum average discount value of three cents from the informal survey, the downward bias relative to the standard grade would be only 0.15¢ per bushel (0.05*3). Furthermore, if the average discount is three cents, then one-third of the crop would have to be sold at a discount to induce a downward bias even as large as one cent (0.33*3 = 1). In sum, while the evidence is limited and sketchy, it does suggest that any downward quality bias in the USDA average price received series, at least for corn and soybeans in central Illinois, is quite small.

Now, consider the potential bias from omission of futures and options profits/losses. If a farmer uses futures and options exclusively for "pure" hedging purposes, they will consistently take short positions at about the same points in the marketing window each year.[39] Unless futures prices are biased upwards or downwards, this type of hedging will not result in large profits or losses, as the hedge profits and losses from upward and downward price trends should roughly offset over time.[40] If a farmer uses futures and options to engage in "selective" hedging, they may have large profits or losses related to the timing of trading. Unfortunately, no direct evidence on the profits or losses of farmers is available in this context. Indirect evidence is provided in a study by McNew and Musser (2002), who examine data from a real-time forward pricing game employed by farmer marketing clubs in Maryland over 1994-1998. They find that forward pricing profits for all clubs, although statistically insignificant, averaged about 10¢ per bushel per year. The difficulty with this evidence is that it is difficult to know whether the experience in a game setting can be generalized to actual farmer behavior. The literature on who wins and loses in futures markets provides further indirect evidence on the question. Studies in this literature have long shown that small traders consistently lose money in futures markets (e.g., Stewart, 1949; Ross, 1975; Hartzmark, 1987). It seems reasonable to argue that farmers engaged in selective hedging are similar to other small traders, and hence, selective farmer hedging in futures and options markets likely results in aggregate trading losses.[41] Given that, in aggregate, pure hedging is expected to yield zero profits on average and selective hedging is expected to yield losses on average, the net effect of the two types of futures and options trading by farmers should be negative. In this case, when compared to average prices at the standard grade, the USDA average price received should be adjusted downward to reflect the impact of net trading losses.

As before, the key question is the potential magnitude of the bias from omission of futures and options losses. The key piece of evidence in this regard is the limited scale of farmer trading in futures and options markets. Surveys have consistently reported that relatively few farmers directly use futures and options contracts on a regular basis (e.g., Patrick, Musser and Eckman, 1998). Given this information, it is reasonable to argue that the magnitude of farmers' net losses from futures and options trading, in aggregate, should be small. As a result, the upward bias in the USDA average price received from the omission of futures and options net losses should be small.

Next, consider the potential bias from mixing old crop and new crop sales during the 12-month marketing year used to compute the USDA average price received. The first step is to determine the potential magnitude of the problem. Fortunately, bounds for the "shifting" of old crop sales into the next marketing year can be computed by dividing ending stocks for a marketing year by crop production for the same marketing year (e.g., September 1, 2000 soybean stocks divided by 1999 soybean production). Over the 1995/1996 through 2003/2004 marketing years, on-farm ending stocks in Illinois averaged three percent of statewide corn production and two percent of statewide soybean production. These percentages are the lower bounds on shifting because farmers presumably own on-farm stocks and sales of these stocks will be shifted to the next marketing year. Over the 1995/1996 through 2003/2004 marketing years, total ending stocks (on-farm and off-farm) in Illinois averaged 11% of statewide corn production and 7% of statewide soybean production. These percentages are the upper bounds on shifting; assuming farmers own all of the stocks in off-farm storage facilities. Clearly, this assumption is unrealistic, as commercials own some, if not most, of the stocks in off-farm facilities at the end of a marketing year. The bottom-line is that shifting of old crop sales into the next marketing year, on average, is somewhere between 3 and 11% of corn production and 2 and 7% of soybean production. This suggests the magnitude of shifting from one crop year to the next probably is not large.

The second step is to determine the impact shifting old crop sales will have on the USDA average price received. Consider the simplest case where old crop sales in the next marketing year are made at spot prices for the new crop and the same proportion is shifted every year. The same price received would result as in the no shifting case. Only to the degree that the proportion shifted varies from year-to-year will the average price received differ from the no-shifting case. The proportion does vary from year-to-year, but not by a substantial amount. For example, on-farm ending stocks in Illinois varied from only two to six percent of corn production over the 1995/1996 through 2003/2004 marketing years. The impact of this variability on average price received will depend on farmers' ability to time shifts to take advantage of favorable spreads between old crop and new crop prices. If farmers as a group have timing ability in this context, then the USDA average price received will be biased upwards relative to the average price at the standard grade. However, given the difficulty of predicting old crop-new crop price spreads (Lence and Hayenga, 2001) and the small absolute magnitude of actual shifting of sales, it seems reasonable to argue that the bias in average price received from shifting old crop sales across marketing years is quite small.

The last issue to consider is that the USDA average price received series reflects the pricing behavior of unadvised and advised farmers, where advised refers to the programs tracked by the AgMAS Project. As pointed out earlier, this means it may not be possible to "cleanly" disentangle the value of market advisory service information relative to the information used by farmers, as the USDA series already reflects the impact of market advisory program information to some degree. A national survey of advisory service subscribers by the AgMAS Project provides some perspective on the dimensions of this problem (Pennings et al., 2001). While only 11% of the survey respondents said they followed market advisory service recommendations closely, two-thirds indicated they followed the recommendations loosely. Further, when asked to rate the impact of advisory service recommendations on their marketing, subscribers gave an average rating of six on a nine-point scale, with a one indicating no impact at all and a nine indicating great impact. To the extent that farmers subscribe to market advisory services, these results suggest that the average price received by farmers for a crop is influenced by the marketing advice of advisory services.

This discussion suggests that a key unknown is the proportion of farmers that subscribe to advisory services. Unfortunately, this information is proprietary, so it is not possible to provide exact figures for the programs tracked by the AgMAS Project. Several studies have reported survey evidence on the use of advisory services, marketing newsletters and marketing consultants (defined generically), with estimates ranging widely from 21.1 percent of Illinois farmers (Norvell and Lattz, 1998) to 66 percent of farmers nationwide (Smith, 1989). It is uncertain what these estimates imply for the proportion of farmers that subscribe to the programs tracked by the AgMAS Project. On one hand, the programs tracked by the AgMAS Project are among the most popular and widely-followed. On the other hand, the same programs clearly are a subset of all advisory services, marketing newsletters and marketing consultants offered to farmers. While the available evidence is sketchy and uncertain, it nonetheless does suggest that a non-trivial proportion of central Illinois farmers likely subscribe to the advisory programs tracked by the AgMAS Project. It therefore can be reasonably concluded that the average price received by central Illinois farmers for corn and soybeans is impacted to some degree by the information provided by these same programs.

Another key unknown is the pricing performance of unadvised versus advised farmers. Patrick, Musser and Eckman (1998) survey large-scale Midwestern grain farmers and find that farmers using marketing consultants typically received higher prices than those that did not. While this evidence cannot be generalized to all farmers because of the skewed size distribution of farm operations in the sample, it does nevertheless seem to be a plausible outcome. Additional evidence is provided in a recent study by McBride and Johnson (2004). Data from the 2001 Agricultural Resource Management Survey (ARMS) survey, which is conducted by the USDA, was analyzed in this particular study. The focus was a sample of 1,149 cash grain farms throughout the U.S. Regression analysis revealed that the use of "farm management services" increased a modified measure of net farm income by $4,000 per operation. Furthermore, using a farm management service for market advice was one of only four management actions that had a statistically significant impact on farm financial performance after controlling for other economic factors, farm structure and operator characteristics. The survey did not explicitly define "farm management service," so it cannot be known with certainty whether respondents included agricultural market advisory services in their definition of the term.[42] However, it seems reasonable to assume that most respondents would have considered advisory services to be included in the definition based on the context of the question.[43] If this assumption is correct, the results of the study provide evidence, albeit indirect, that the financial performance of advisory service subscribers is enhanced compared to non-subscribers. However, in addition to the previous definitional issue, the results of McBride and Johnson's study do not disentangle whether the income enhancement is the result of improved information and analysis, improved input pricing performance, improved output pricing performance or some combination of the three.

Overall, the available evidence supports the view that advised farmers outperform unadvised farmers in terms of pricing crops. Combined with the evidence that a substantial proportion of central Illinois corn and soybean producers subscribe to advisory programs, a reasonable conclusion is that the USDA average price received series is biased upward relative to the price received by unadvised farmers. Regrettably, there is nothing that can be done about this problem without other sources of data on farmer pricing performance. The USDA average price received is probably best viewed as an estimate of the upper bound for the average price received by unadvised farmers.

To summarize, the evidence and arguments discussed above suggest that the net systematic bias in the USDA average price received due to spatial, quality, futures/options and old/new crop factors is small, at least for corn and soybeans in central Illinois. It is difficult to construct a scenario where the overall level of bias from these factors would materially effect performance evaluation of market advisory programs. A more difficult problem is presented by the mixture of unadvised and advised farmers that the USDA average price received reflects. This "mixing" likely biases the USDA price received series upward relative to the price received by unadvised farmers. Given the limited evidence on the extent that central Illinois farmers use the programs tracked by the AgMAS Project and the precise impact of their recommendations, it is difficult to assess the magnitude of the bias. Overall, the USDA average price received should be viewed as only an approximation of the "true" average price received by unadvised farmers. For this reason, comparison of advisory program pricing performance to a USDA average price received benchmark is not likely to be as precise as comparisons provided by the market benchmarks.

Several adjustments are made to the USDA average price received data for the state of Illinois in order to make the computed farmer benchmark consistent with net advisory prices. To begin, mid-month on-farm or commercial storage charges are applied to the monthly average price received in the 12-month marketing year (September through August). Next, the annual weighted-average price received is computed using the percentage of the crop marketed in each calendar month as the weights. Finally, actual state average LDPs and MLG's are added for the 1998-2003 crops.[44]

Given the uncertainties involved in measuring the average price received by farmers, it would be useful to specify alternative farmer benchmarks. One alternative approach would be to use USDA marketing weights and central Illinois cash market prices for the 12-month marketing year to compute farmer benchmarks. This would have the advantage of eliminating any bias due to spatial, quality, old/new crop factors and the mixing of advised and unadvised farmers. However, a significant disadvantage of the alternative approach is that farmers' use of forward contracts would not be reflected. There is ample survey evidence that many farmers use pre-harvest forward contracts to price a portion of their crops, and that post-harvest forward contracts are commonly used, particularly for January delivery (e.g., Patrick, Musser and Eckman, 1998; Coble et al., 1999; Isengildina et al., 2004). In addition, the alternative would still not reflect futures and options profits/losses of farmers. The impact of this alternative specification is nonetheless an interesting question and future research will be devoted to it.

Finally, it is interesting to consider arguments about the expected difference in averages and variation between the farmer benchmark and the market benchmarks. If corn and soybean markets are efficient and farmers are rational, then the average price across crop years for the farmer and market benchmarks should be similar. Under these assumptions, the variation in farmer benchmark prices across crop years could be smaller or larger than the variation in market benchmark prices, depending on the length of the marketing window used by farmers and the exact nature of the marketing strategies implemented by farmers.

Unfortunately, it is not possible to determine the average marketing window or the pricing pattern of farmers using USDA monthly marketing weights. For perspective, average monthly USDA marketing weights for corn and soybeans in Illinois over 1995-2001 are presented in Figure 9. These weights reflect the pattern of grain purchases by commercial facilities from farmers over the 12-month marketing year. Grain purchases, as defined by the USDA, do not necessarily reflect the pricing pattern of farmers due to the use of forward pricing instruments. As noted above, there is considerable evidence that many farmers use pre- and post-harvest forward contracts to price a portion of their crops. However, the evidence on the magnitude of forward contracting by farmers is more limited.

Three studies provide the best evidence that is available on the magnitude of forward contracting, as a large number of farmers are randomly sampled in each study. The first, by Coble et al. (1999), surveyed farmers in four states a number of questions regarding risk management, including the percent of crop production in 1998 priced before harvest. Based on the responses reported in the study, it can be estimated that farmers in Indiana and Nebraska (the closest states to Illinois) priced 15.7% of corn and 14.0% of soybean production pre-harvest. The second study, by Katchova and Miranda (2004), used data reported in the 1999 ARMS survey by the USDA. Farmers in this survey were asked about their use of marketing contracts for the 1999 crop. The definition of marketing contracts included forward contracts, futures and options contracts, formula pricing contracts, delayed price contracts, minimum price contracts, fixed basis contracts, futures fixed contracts, and other contracts. Based on the information reported in Katchova and Miranda's study, it can be estimated that farmers in the U.S. priced 5.0% of corn and 5.2% of soybean production in 1999 using marketing contracts. The third study also used data from the USDA ARMS survey (USDA/NASS, 2003). In this case, respondents to the annual ARMS survey were about their use of marketing contracts for the 2001 crop. It was reported that farmers in the Corn Belt region (Illinois, Indiana, Iowa, Missouri and Ohio) marketed 10.1% of corn and 9.0% of soybeans through marketing contracts. The estimates from the three studies suggest that the magnitude of forward pricing is modest, but nonetheless, large enough to make the USDA monthly marketing weights potentially misleading indicators of the true pricing pattern of farmers. It is also important to emphasize that the estimates discussed here pertain to only three crop years and there may be considerable variation in the magnitude of forward pricing across other crop years. For example, Coble et al. (1999) also asked farmers how much of their 1999 production they expected to price before harvest. The responses indicate that farmers in Indiana and Nebraska expected to price an average of 26.9% of corn and 23.1% of soybeans pre-harvest in 1999.[45]

A further difficulty is that almost no concrete evidence exists on the exact length of the typical marketing window of farmers. The two studies discussed above only investigated the magnitude of forward pricing, not the timing of such decisions. Without evidence to the contrary, it seems reasonable to argue that many farmers use a marketing window not unlike the 24-month and 20-month windows assumed for the market benchmarks, but the amount of pre-harvest forward pricing is far less than is assumed for the market benchmarks. The two surveys suggest that pre-harvest forward pricing by farmers typically is in the range of 10 to 20%, compared to an average of 51 and 42% for 24-month and 20-month benchmarks, respectively, over 1995-2003. All else equal, this would lead to the expectation that the variation of farmer benchmark prices would exceed that for the market benchmarks.

Under rationality, it is still possible for the variation of farmer benchmark prices to be smaller than for market benchmarks if farmers employ market-timing strategies that successfully reduce price variation. Alternatively, if farmers are subject to the same judgment and decision biases as appears to be the case for participants in other markets, then it would be reasonable to expect the farmer benchmark to have a lower average price and higher variation than the market benchmarks. Which of the above scenarios is correct can only be determined empirically.

Net advisory prices and benchmarks for the 2002 and 2003 corn and soybean crops are presented in Tables 10 through 21. These results are new and add to the sample of net advisory prices and benchmarks previously available for analysis. For a specific example of how marketing recommendations are translated into a final net advisory price that incorporates the simulation assumptions, see Jackson, Irwin and Good (1996). It is important to emphasize that all of the net advisory prices and benchmarks presented in Tables 10 through 21 are stated on a harvest equivalent basis using either on-farm variable or commercial storage costs.

Net advisory prices and benchmarks for corn in 2002 assuming on-farm variable storage costs are presented in Table 10. In addition, this table shows the components of the advisory prices and benchmarks. The 2002 average net advisory price for all 27 corn programs is $2.21 per bushel under the assumption of on-farm variable costs. It is computed as the unadjusted cash sales price ($2.33 per bushel) minus storage charges ($0.07 per bushel) plus futures and options gain (-$0.03 per bushel) minus brokerage costs ($0.02 per bushel) plus LDP/MLG gain ($0.00 per bushel).[46] The range of net advisory prices for corn in 2002 assuming on-farm variable storage costs is $1.85 to $2.46 per bushel. Corresponding benchmark prices range from $2.16 per bushel (24-month and 20-month average market benchmarks) to $2.22 per bushel (farmer benchmark).

Net advisory prices and benchmarks for corn in 2003 assuming on-farm variable storage costs are presented in Table 11. The 2003 average net advisory price for all 26 corn programs is $2.31 per bushel under the assumption of on-farm variable costs. It is computed as the unadjusted cash sales price ($2.38 per bushel) minus storage charges ($0.07 per bushel) plus futures and options gain ($0.01 per bushel) minus brokerage costs ($0.02 per bushel) plus LDP/MLG gain ($0.01 per bushel). The range of net advisory prices for corn in 2003 assuming on-farm variable storage costs is $2.07 to $2.70 per bushel. Corresponding benchmark prices range from $2.30 per bushel (24-month average market benchmark) to $2.31 per bushel (20-month average market benchmark and farmer benchmark).

Net advisory prices and benchmarks for soybeans in 2002 assuming on-farm variable storage costs are presented in Table 12. The 2002 average net advisory price for all 26 soybean programs is $5.28 per bushel under the assumption of on-farm variable costs. It is computed as the unadjusted cash sales price ($5.38 per bushel) minus storage charges ($0.08 per bushel) plus futures and options gain (-$0.02 per bushel) minus brokerage costs ($0.02 per bushel) plus LDP/MLG gain ($0.02 per bushel). The range of net advisory prices for soybeans in 2002 assuming on-farm variable storage costs is $4.64 to $6.19 per bushel. Corresponding benchmark prices range from $5.03 per bushel (24-month average market benchmark) to $5.49 per bushel (farmer benchmark).

Net advisory prices and benchmarks for soybeans in 2003 assuming on-farm variable storage costs are presented in Table 13. The 2003 average net advisory price for all 25 soybean programs is $6.25 per bushel under the assumption of on-farm variable costs. It is computed as the unadjusted cash sales price ($6.44 per bushel) minus storage charges ($0.06 per bushel) plus futures and options gain (-$0.10 per bushel) minus brokerage costs ($0.03 per bushel) plus LDP/MLG gain ($0.00 per bushel). The range of net advisory prices for soybeans in 2003 assuming on-farm variable storage costs is $3.70 to $7.67 per bushel. Corresponding benchmark prices range from $5.99 per bushel (24-month average market benchmark) to $7.33 per bushel (farmer benchmark).

Since many Corn Belt farmers grow both corn and soybeans, it also is useful to examine a combination of the results for the corn and soybean marketing programs. In order to do this, gross revenue is calculated for a central Illinois farmer who follows both the corn and soybean marketing advice of a given program. It is assumed that the representative farmer splits acreage equally (50/50) between corn and soybeans and achieves corn and soybean yields equal to the actual yield for the area in 2002 and 2003. The 50/50 advisory revenues are computed on a per acre basis and compared with the revenue a central Illinois farmer could have received based on benchmark prices for both corn and soybeans. Advisory revenue per acre is calculated only for those programs that offer both corn and soybean marketing advice.

Advisory program revenues and benchmarks in 2002 assuming on-farm variable storage costs are presented in Table 14. The average revenue achieved by following both the corn and soybean programs offered by an advisory program is $299 per acre. The range of 50/50 advisory revenue in 2002 assuming on-farm variable storage costs is $260 to $328 per acre. Corresponding benchmark revenues range from $289 per acre (24-month average market benchmark) to $305 per acre (farmer benchmark).

Advisory program revenues and benchmarks in 2003 assuming on-farm variable storage costs are presented in Table 15. The average revenue achieved by following both the corn and soybean programs offered by an advisory program is $330 per acre. The range of 50/50 advisory revenue in 2003 assuming on-farm variable storage costs is $290 to $380 per acre. Corresponding benchmark revenues range from $324 per acre (24-month average market benchmark) to $351 per acre (farmer benchmark).

For comparison purposes, the annual subscription cost of each advisory program also is listed in the last column of Tables 14 and 15. Subscription costs average $353 per program in 2002 and $359 per program in 2003, levels that do not appear to be large relative to total farm revenue, whether a large or small farm is considered. Subscription costs average only 18¢ per acre for a 2,000 acre farm and 72¢ per acre for a 500 acre farm.

Net advisory prices and benchmarks for corn in 2002 assuming commercial storage costs are presented in Table 16. The 2002 average net advisory price for all 27 corn programs is $2.15 per bushel under the assumption of commercial storage costs. It is computed as the unadjusted cash sales price ($2.33 per bushel) minus storage charges ($0.14 per bushel) plus futures and options gain (-$0.03 per bushel) minus brokerage costs ($0.02 per bushel) plus LDP/MLG gain ($0.00 per bushel). The range of net advisory prices for corn in 2002 assuming commercial storage costs is $1.80 to $2.43 per bushel. Corresponding benchmark prices range from $2.09 per bushel (20-month average market benchmark) to $2.11 per bushel (farmer benchmark).

Net advisory prices and benchmarks for corn in 2003 assuming commercial storage costs are presented in Table 17. The 2003 average net advisory price for all 26 corn programs is $2.24 per bushel under the assumption of commercial storage costs. It is computed as the unadjusted cash sales price ($2.38 per bushel) minus storage charges ($0.14 per bushel) plus futures and options gain ($0.01 per bushel) minus brokerage costs ($0.02 per bushel) plus LDP/MLG gain ($0.01 per bushel). The range of net advisory prices for corn in 2003 assuming commercial storage costs is $1.95 to $2.67 per bushel. Corresponding benchmark prices range from $2.22 per bushel (20-month average market benchmark and farmer benchmark) to $2.23 per bushel (24-month average market benchmark).

Net advisory prices and benchmarks for soybeans in 2002 assuming commercial storage costs are presented in Table 18. The 2002 average net advisory price for all 26 soybean programs is $5.24 per bushel under the assumption of commercial storage costs. It is computed as the unadjusted cash sales price ($5.38 per bushel) minus storage charges ($0.12 per bushel) plus futures and options gain (-$0.02 per bushel) minus brokerage costs ($0.02 per bushel) plus LDP/MLG gain ($0.02 per bushel). The range of net advisory prices for soybeans in 2002 assuming commercial storage costs is $4.59 to $6.15 per bushel. Corresponding benchmark prices range from $4.98 per bushel (24-month average market benchmark) to $5.41 per bushel (farmer benchmark).

Net advisory prices and benchmarks for soybeans in 2003 assuming commercial storage costs are presented in Table 19. The 2003 average net advisory price for all 25 soybean programs is $6.22 per bushel under the assumption of commercial storage costs. It is computed as the unadjusted cash sales price ($6.44 per bushel) minus storage charges ($0.08 per bushel) plus futures and options gain (-$0.10 per bushel) minus brokerage costs ($0.03 per bushel) plus LDP/MLG gain ($0.00 per bushel). The range of net advisory prices for soybeans in 2002 assuming commercial storage costs is $3.69 to $7.67 per bushel. Corresponding benchmark prices range from $5.95 per bushel (24-month average market benchmark) to $7.27 per bushel (farmer benchmark).

Advisory program revenues and benchmarks in 2002 assuming commercial storage costs are presented in Table 20. The average revenue achieved by following both the corn and soybean programs offered by an advisory program is $294 per acre when commercial storage costs are assumed. The range of 50/50 advisory revenue in 2002 assuming commercial storage costs is $256 to $320 per acre. Corresponding benchmark revenues range from $284 per acre (24-month average market benchmark) to $295 per acre (farmer benchmark).

Advisory program revenues and benchmarks in 2003 assuming commercial storage costs are presented in Table 21. The average revenue achieved by following both the corn and soybean programs offered by an advisory program is $324 per acre when commercial storage costs are assumed. The range of 50/50 advisory revenue in 2003 assuming commercial storage costs is $288 to $369 per acre. Corresponding benchmark revenues range from $317 per acre (24-month average market benchmark) to $341 per acre (farmer benchmark).

Figures 10 through 13 present the 24-month price pattern for corn and soybeans in central Illinois for the 2002 and 2003 marketing years. The top panel (bottom panel) in each figure shows daily corn (soybean) cash prices through the marketing window (from September 1st prior to the harvest calendar year through August 31st after the harvest calendar year). Pre-harvest prices are cash forward contract prices for harvest delivery, while post-harvest prices are spot cash prices. In 2002, pre-harvest forward contract bids for corn generally declined from the fall of 2001 into the spring of 2003. Bids were below the loan rate from early March through early May. Poor growing conditions, particularly in the eastern growing areas, pushed prices higher into harvest. Prices declined immediately after harvest, in a typical pattern for smaller than expected crops; remained generally flat during the winter and spring of 2003; and declined sharply in July 2003 on the basis of prospects for a large 2003 crop. In 2003, pre-harvest forward contract bids were remarkably stable from the fall of 2002 into June 2003. Bids dropped below the loan rate briefly in July and early August. Prices moved steadily higher from November 2003 into the spring of 2004 on the basis of a very high rate of consumption and prospects for small carryover stocks. Prices once again declined sharply in the late spring and summer of 2004 as prospects for a huge 2004 crop unfolded.

In 2002, pre-harvest forward bids for soybeans were well under the loan rate through most of the growing season. A smaller than expected crop pushed prices higher into harvest and strong demand resulted in prices remaining above the loan rate during the post-harvest period. Weather and crop concerns resulted in a spring rally. Prices declined into July and August as production potential seemed to be quite large. The August rally was generated by dry weather and widespread incidence of soybean aphids. In 2003, pre-harvest forward bids were near the loan rate for much of the time from the fall of 2002 through early August 2003. Prices advanced sharply from August to the spring of 2004, driven by a much smaller than expected U.S. crop, very strong demand, and a decline in South American production. Increased acreage in the U.S. and an ideal growing season pushed prices sharply lower from the spring into the summer of 2004.

Net advisory prices, revenue and benchmarks for the 2000-2003 crop years, assuming on-farm variable storage costs, are reported in Tables 22 through 24. Results are not presented for earlier crop years because the AgMAS Project first computed net advisory prices and benchmarks under on-farm variable storage costs for the 2000 crop year. Net advisory prices, revenue and benchmarks for the 1995-2003 crop years, assuming commercial storage costs, are reported in Tables 25 through 27. In both sets of results, please note that some of the market advisory programs included in the tables are not evaluated for all crop years. Finally, in order to obtain a consistent set of net advisory prices and benchmarks for the entire sample period, the following discussion focuses on the net advisory prices, revenue and benchmarks where commercial storage costs are assumed.

Table 25 shows the average advisory price for corn ranges between $1.99 per bushel in 2001 and $3.03 per bushel in 1995 (based on commercial storage costs). Range statistics reveal that net advisory prices for corn vary substantially within individual crop years. The most dramatic example is 1995, where the minimum is $2.29 per bushel and the maximum is $3.90 per bushel. Even in years with less market price volatility, it is not unusual for the range of prices across advisory programs to be near a dollar per bushel. The three alternative benchmark prices for corn are shown at the bottom of Table 25. The variation in benchmark prices from year-to-year is similar to that of average net advisory prices. However, there can be substantial differences in benchmark prices for a particular crop year. For example, the 24-month market benchmark in 1998 is $2.24 per bushel, while the farmer benchmark is only $1.97 per bushel. These data suggest performance results for corn may be sensitive to the selected benchmark.

As reported in Table 26, the average advisory price for soybeans ranged from $5.24 per bushel in 2002 to $7.27 per bushel in 1996 (based on commercial storage costs). Similar to corn, the range of individual net advisory prices within a crop year is substantial. The most dramatic example is 2003, where the range in advisory prices is just under $4 per bushel. The three alternative benchmark prices for soybeans are shown at the bottom of Table 26. The variation in soybean benchmark prices from year-to-year is similar to that of average net advisory prices. Once again, there can be substantial differences in benchmark prices for a particular crop year.

Table 27 contains the combined corn and soybeans revenue results (based on commercial storage costs). The lowest average advisory revenue, $287 per acre, occurred in 2001, while the highest average advisory revenue, $369 per acre, occurred in 1996. Given the results for corn and soybeans, the large range of individual advisory revenues within a crop year is not surprising. Nonetheless, it is startling to see the possible economic impact of following the best versus the worst performer in a given crop year. For example, in three of the nine crop years (1995, 1999 and 2000) the range in advisory revenue exceeds $100 per acre.

For the reader's convenience, Tables 28 through 30 report the most recent two-year averages (2002-2003), three-year averages (2001-2003), four-year averages (2000-2003), five-year averages (1999-2003), six-year averages (1998-2003), seven-year averages (1997-2003), eight-year averages (1996-2003) and nine-year averages (1995-2003) of net advisory prices, revenues and benchmarks (based on commercial storage costs).[47] The averages are computed in these tables only for the advisory programs active in each of the indicated crop years. The reported averages may reflect survivorship bias as a result of this assumption, which should be considered when viewing the averages.[48] Finally, note that the average, minimum and maximum reported for each column in the Tables 28 through 30 are computed across the advisory program averages in each column.

Information on the sources of the differences between net advisory prices and benchmarks in corn and soybeans is found in Table 31. Panel A shows average net advisory prices and benchmarks broken out by component. Panel B presents the average difference in the components between advisory programs and the benchmarks. All of the averages in the table assume commercial storage costs. In corn, when the average net advisory price is above the average benchmark price (20-month market benchmark and farmer benchmark) the difference is primarily explained by either a higher net cash sales price or larger marketing loan benefits for advisory programs. The net result of futures and options positions for advisory programs in corn is small (-1¢ per bushel after brokerage costs). In soybeans, when the average net advisory price is above the average benchmark price (24- and 20-month market benchmarks) the difference is explained by a combination of higher net cash sales price and larger marketing loan benefits for advisory programs. The net result of futures and options positions for advisory programs in soybeans is zero cents per bushel after brokerage costs.

Four basic indicators of performance are applied to advisory program prices and revenues over 1995-2003. The first indicator is the proportion of advisory programs that beat benchmark prices. A valuable feature of this directional indicator is that it is not influenced by extremely high or low advisory prices. The second indicator is the difference between the average price of advisory programs and benchmarks. This indicator is useful because it takes into account both the direction and magnitude of differences from benchmark prices. The third indicator is the average price and risk of advisory programs relative to the average price and risk of the benchmarks. Evaluations based on this indicator are important because risk is incorporated into the performance comparisons. The fourth indicator is the predictability of advisory program performance from year-to-year. This indicator provides information on the value of past pricing performance in predicting future performance.

Before considering the performance evaluation results, two important issues need to be discussed. First, the results presented in this section of the report address the performance of market advisory programs as a group. In other words, average pricing performance across all programs is considered. This is a different issue than the pricing performance of a particular advisory program.[49] Simply put, it is inappropriate to make performance inferences for an individual advisory program based on aggregate results. Second, farmers subscribe to market advisory programs for a variety of reasons. For example, marketing information and market analysis are the two highest rated uses of market advisory programs by farmer-subscribers (Pennings et al., 2004). While the quality of marketing information and market analysis is likely to be positively correlated with the returns to marketing recommendations, this does not necessarily have to be the case. It is possible that advisory programs provide valuable information and analysis to farmer-subscribers, yet fail to exhibit superior pricing performance.

The first, and simplest, indicator of pricing performance is the proportion of advisory programs that beat the market or farmer benchmarks. Positive performance is indicated if the proportion of advisory programs beating a benchmark exceeds 50%, the proportion one would observe if advisory performance is random, like flipping a fair coin. A noteworthy feature of this "directional" indicator is that it is not influenced by extremely high or low advisory prices or revenue.

The proportion of advisory programs in corn, soybeans and 50/50 advisory revenue above the benchmarks over 1995-2003 is presented in Table 32. Note that average proportions for 1995-2003 are computed over the full set of advisory programs, and therefore, do not necessarily equal the average of the individual crop year proportions. This "grand" average equally weights each of the net advisory prices or revenues in the sample, whereas an average of the individual crop year averages would equally weight the crop years. The first average is preferred for the present purpose as it implies an equal probability of selecting an individual advisory program across the entire sample.[50]

Considering corn first (Panel A: Table 32), there is some variation in the proportion of net advisory prices above the two market benchmarks for individual crop years, particularly 1998, but the patterns are similar overall. There also does not appear to be any discernable trend in the proportions for either benchmark over the nine crop years. The average proportion for 1995-2003 is 50% versus the 24-month benchmark and 59% versus the 20-month benchmark, indicating a zero to marginal chance of advisory prices in corn beating market benchmark prices. In contrast, the proportion of net advisory prices above the farmer benchmark equals or exceeds 50% each crop year. The average proportion above the farmer benchmark over 1995-2003 is 68%. This is larger than the average proportions versus the market benchmarks and indicates a better chance of market advisory programs generating net prices higher than the farmer benchmark. However, there has been a noticeable downtrend in proportions versus the farmer benchmark since 1998.

Moving to soybeans (Panel B: Table 32), there is more variation in the proportion of net advisory prices above the two market benchmarks for individual crop years. Particularly sharp differences are observed in 1998 and 1999, where the spread between proportions for the two market benchmarks is 45 and 36 percentage points, respectively. No clear trend is apparent for the proportions versus either market benchmark. Despite these differences for individual crop years, the average proportions for 1995-2003, 65% versus the 24-month benchmark and 72% versus the 20-month benchmark, both indicate a better than average chance of advisory prices beating market benchmark prices in soybeans. The average proportion above the farmer benchmark over 1995-2003 is 54%. This indicates a small chance of market advisory programs generating net prices in soybeans higher than the farmer benchmark. In addition, there has been a sharp downtrend in proportions versus the farmer benchmark since 1998.

Given the combined nature of 50/50 advisory revenue, it is not surprising that revenue proportions (Panel C: Table 32) typically are between those of corn and soybeans. The average proportion for 1995-2003 is 59% versus the 24-month benchmark and 68% versus the 20-month benchmark, indicating a marginal to better than average chance of advisory revenue beating market benchmark revenue. The average proportion above the farmer benchmark over 1995-2003 is 62%. This indicates a moderate chance of advisory revenue beating farmer benchmark revenue. Mirroring the results for corn and soybeans, a sharp downtrend is observed in proportions versus the farmer benchmark since 1998. It is interesting to note that 100% of the advisory programs in 1998 generated revenue that exceeded the farmer benchmark, despite the fact that less than 100% did so in corn and soybeans. This simply reflects a situation where some programs had gains above the farmer benchmark in one commodity that more than offset the losses below the benchmark in the other commodity.

Overall, the directional performance results over 1995-2003 suggest several key findings. First, advisory programs in corn do not consistently beat market benchmarks, but tend to consistently beat the farmer benchmark. Second, advisory programs in soybeans exhibit just the opposite pattern, consistently beating the market benchmarks but not the farmer benchmark. Third, in terms of 50/50 revenue, advisory programs show marginal consistency in beating both the market benchmarks and the farmer benchmark. So, the results provide mixed performance evidence with respect to both the market benchmarks and the farmer benchmark.

Finally, it is interesting to compare the directional pricing performance results for market advisory programs to that of other investment professionals. Malkiel (1999) reports a typical estimate of the proportion of active mutual funds managers that beat the stock market. Specifically, he shows that only 33% of active mutual fund managers generate returns higher than the S&P 500 stock index over 1974-1998. By comparison, market advisory programs perform better, with about half of the programs beating the market in corn and about two-thirds beating the market in soybeans. This divergence may simply reflect a unique time period in corn and soybean markets, relatively less efficient commodity markets, the skill of advisory programs, a return to risk, or some combination of these factors.

The second indicator of pricing performance is the difference between the average price of advisory programs and the market or farmer benchmarks. This indicator takes into account both the direction and magnitude of differences from the benchmarks.[51] The results found in Tables 33 and 34 basically tell the same story as those based on the proportion beating the benchmarks. Average differences from market benchmarks for corn over 1995-2003 (panel A: Table 33) are small, ranging from 1 to 3¢ cents per bushel.[52] At 8¢ cents per bushel, the average difference from the farmer benchmark for corn is larger. Average differences from market benchmarks for soybeans over 1995-2003 (panel B: Table 33) are substantial, ranging from 14 to 16¢ per bushel. In contrast, the average difference from the farmer benchmark for soybeans is -1¢ per bushel. Average differences for 50/50 advisory revenue range from $4 to 7 per acre for market benchmarks over 1995-2003 (Table 34). The average revenue difference versus the farmer benchmark is similar at $7 per acre. Note that the average differences can mask considerable variability across the benchmarks within a crop year and across crop years. A dramatic example of this occurred in 2003 for soybeans (Panel B: Table 33), where the average difference from the 24-month market benchmark is +27¢ per bushel, while the average difference from the farmer benchmark is -105¢ per bushel.

An important consideration is the size of the average differences versus the farmer benchmark from an economic decision-making perspective. The average advisory return relative to the farmer benchmark is $7 per acre, or about two percent of average farmer benchmark revenue. Even though this return is small and entirely from corn, it nonetheless represents a non-trivial increase in net farm income (defined as returns to farm operator management, labor and capital), typically about $50 per acre for grain farms in Illinois (Lattz, Cagley and Raab, 2004). The comparison does not account for yearly subscription costs, which is not a major problem because subscription costs are quite small relative to revenue. As noted earlier, subscription costs are only 18¢ per acre for a 2,000 acre farm and 72¢ per acre for a 500 acre farm. A more serious issue is fully accounting for the cost of implementing, monitoring and managing the marketing strategies recommended by advisory programs. Such costs are difficult to measure, but may well be substantial (Tomek and Peterson, 2001).

At this juncture, the findings should be considered only suggestive. The reason is that the statistical significance of the results has not been investigated. In other words, are the returns to marketing advice simply the result of random chance or do they reflect truly positive pricing performance? A number of different statistical tests can be used to determine the significance of observed differences in sample means. In the present context, it is critical to recognize that there is a "natural" pairing in the sample data that can be used to increase the power of statistical tests (Snedecor and Cochran, 1989). More specifically, net advisory prices and benchmark prices for the same crop year are paired, in the sense that the same crop year receives different "treatments" from advisory programs and benchmarks. The treatments correspond to the differing marketing strategies used by advisory programs and benchmarks. Given that the sample data are paired, the appropriate test of the null hypothesis of zero difference between the mean of net advisory and benchmark prices is the paired t-test.

Application of the paired t-test to average pricing performance is complicated by the fact that net prices across programs are positively related. This type of statistical test assumes that sample differences are generated independently (Snedecor and Cochran, 1989, pp. 101).[53] It should come as no surprise that this assumption is violated for market advisory programs. Many of the programs appear to use similar methods of analysis and all make heavy use of similar supply and demand information (primarily from the USDA). Furthermore, alternative programs offered by the same advisory service are likely to generate similar pricing results. Statisticians call this an "implicit factor" problem.

Correlation coefficients estimated across net advisory prices most directly provide evidence on the magnitude of the dependence problem. However, the sample is not large enough to independently estimate all possible pair-wise correlations.[54] Useful evidence can be generated by estimating "market model" regressions for each commodity. This entails simply regressing net advisory prices (or revenue) for a given program on a market benchmark. If net advisory prices share a common "market factor" the explanatory power of the regressions will be high. In order to maximize the number of time-series observations available for each program, the sample for this analysis is limited to the 15 programs active in all nine crop years. The explanatory power of the market model regressions turns out to be quite substantial, with an average of 0.76 in corn, 0.78 in soybeans and 0.68 for revenue, and the regressions all have positive slope estimates. [55],[56]

The high level of dependence across net advisory prices and revenue basically creates an information problem in the sample. Take the case of corn. There are 232 computed net advisory prices across all programs and crop years. However, the 232 net advisory prices are not independent, due to the strong positive correlation across programs. The key question is the amount of independent information contained in the sample of 232 net advisory prices. It is not possible to precisely estimate the true number of independent observations, but it is certainly far less than 232. Similar logic holds for soybeans and 50/50 advisory revenue.

The bottom-line from this discussion is that an assumption of independence for advisory prices and revenue will overstate the reliability of sample estimates. This in turn will bias statistical tests towards a conclusion that pricing performance is significantly positive. The approach taken here to deal with the problem is "conservative." Specifically, statistical tests assume the minimum possible number of independent observations in the sample. This minimum is nine observations, one for each crop year. The tests are conservative since conclusions are based on the minimal possible assumption about the amount of information in the sample. If test results based on this conservative assumption indicate statistical significance, then a high degree of confidence can be placed on conclusions. The cost of this approach is an increased probability that positive pricing performance is mistakenly attributed to chance.

Implementing the conservative testing approach is straightforward.[57] First, the average net advisory price or revenue is computed across all programs active in a crop year, and it is considered the return for an "average" advisory program. Second, the averaging process is repeated for each of the crop years to form a sample of nine observations for the average advisory program. These averages can be found in Tables 25 through 27 under the "Descriptive Statistics" heading. Third, benchmark prices are subtracted from each of the average advisory prices or revenues. Fourth, a paired t-test is applied to the nine difference observations to determine if average price performance is statistically significant.

Differences from the benchmarks for each crop year and statistical test results for an average advisory program are presented in Table 35. Note that the average differences reported in Table 35 are nearly identical to those reported in Tables 33 and 34. This outcome is not surprising. The average differences in Table 35 assume an equal weighting of the nine crop years, while the average differences in Tables 33 and 34 assume an equal weighting of each net advisory price or revenue in the sample. The two types of averages differ only because the number of advisory programs changes across crop years. Since this change is quite small across crop years, the difference in the two types of averages is negligible.

The impact of the conservative approach to testing the significance of average differences is reflected in the standard error estimates. This statistic measures the "typical" error, without regard to sign, in estimating the average difference between advisory programs and a particular benchmark (Mirer, 1995, p. 238).[58] For example, the standard error estimate for the average difference in soybeans versus the 24-month market benchmark indicates that the typical error in estimating the true difference, without regard to sign, is four cents per bushel. A measure of reliability is needed because a sample is being used to make an inference about the "true" population difference, and the sample will not perfectly reflect the characteristics of the population. This is the essence of the role of random chance in estimation. The key point in this regard is that standard error estimates vary inversely with sample size.[59] As a result, standard error estimates (typical estimation errors) will be much larger if it is assumed that nine independent observations are available as opposed to, say, 232 independent observations.

With this background, the statistical test results in Table 35 can be considered. The relevant information in the sample for testing statistical significance is summarized by the t-statistic, which is just the ratio of the average difference estimate to the standard error estimate. The two-tail p-value indicates the probability of observing a value of the t-statistic (or higher in absolute value) across many random samples. It is usually argued that p-values must be equal to or smaller than 0.05 to confidently conclude that average differences do not equal zero (Griffiths, Hill and Judge, 1993, p. 134). Stated differently, there should be less than a 1 out of 20 chance that the wrong conclusion is reached. In corn, the p-values for average differences versus both market benchmarks are larger than 0.05, so it can be concluded that average differences are insignificantly different from zero. Note, however, that the p-value for the 20-month benchmark just misses the cutoff for significance. The p-value of 0.01 indicates the average difference of 8¢ per bushel versus the farmer benchmark in corn is highly significant. In soybeans, the p-values for average differences versus both market benchmarks are smaller than 0.05, so it can be concluded that average differences are significantly different from zero. In contrast to the results for corn, the average difference of 1¢ per bushel in soybeans versus the farmer benchmark is insignificantly different from zero. Test results for 50/50 advisory revenue show mixed results. With the market benchmarks, results show statistical significance for the average difference from the 20-month benchmark, but not from the 24-month benchmark. The average difference of $7 per acre versus the farmer benchmark also is not significantly different from zero.

Overall, the test results with respect to market benchmarks indicate mixed evidence of statistically significant average price performance in corn, consistent evidence of significant performance in soybeans and mixed evidence for 50/50 advisory revenue. The test results with respect to the farmer benchmark indicate statistically significant performance only in corn.

When viewing statistical test results, it is always important to assess whether the nature of comparisons or the sample information influence the results in one direction or the other. One possibility is that the results may differ when examined in percentage terms instead of unit terms ($/bushel or $/acre). In other words, a 10¢ average difference will be much higher in percentage terms for corn compared to soybeans because the level of corn prices is much lower. To facilitate direct comparisons across corn, soybeans and advisory revenue, average percentage differences for 1995-2003 are computed and presented in Table 36. Average differences between the advisory programs and benchmarks for corn are 0.2%, 1.8% and 4.1% for the 24-month market, 20-month market and farmer benchmarks, respectively. The same average differences for soybeans are 2.7%, 2.6% and 0.6% and for revenue 1.3%, 2.2% and 2.6%, respectively. With one exception, the same hypothesis test conclusions are reached based on the percentage differences and the unit differences. The exception is the difference between advisory prices and the 20-month benchmark in corn, which is significant using percentage differences but is not when using unit differences.

While not obvious, the LDP/MLG strategy assumed for the market benchmarks may have an influence on the results. As discussed in the "Market Benchmarks" section, it is assumed that LDP/MLGs are taken when grain is delivered. The result is that bushels forward contracted before harvest receive the LDP/MLG available during the early part of harvest, and in effect, remaining bushels receive the average LDP/MLG available for the rest of the marketing year. This approach is consistent with the original intent of the loan program to assure that farmers do not have to sell crops below the loan rate, regardless of the timing of their sales. However, there is a second and equally plausible strategy based on the theory of storable commodity markets. This theory predicts that spot prices will increase linearly after harvest at the rate of storage costs (e.g., Tomek and Robinson, 2003, Ch. 9). Hence, the difference between a fixed loan rate and the market price will be the largest at harvest. If this theory is correct, the optimal strategy for a prudent farmer following the market benchmark strategy would be to take the LDP/MLG available at harvest. Furthermore, if market advisors and farmers are in reality aware of this pattern and take advantage of it, existing market benchmarks may be biased downwards due to the requirement of taking the LDP/MLG available on the date of post-harvest sales instead of the presumably larger harvest LDP/MLG.

Market benchmarks over 1998-2003 are recomputed using the average harvest LDP/MLG to test whether performance results are sensitive to the assumed LDP/MLG strategy. Differences from the revised benchmarks for each crop year and statistical test results for an average advisory program are presented in Table 37. The impact of changing the LDP/MLG strategy is most easily seen by comparing the average differences in Table 37 to those found in Table 35. Average differences for advisory programs versus the 24-month benchmark based on the average harvest LDP/MLG decline by 1.5¢ per bushel for corn, 0.4¢ cents per bushel for soybeans and $2.30 per acre for 50/50 revenue. Average differences for advisory programs versus the 20-month benchmark based on the average harvest LDP/MLG decline by 2.1¢ per bushel for corn, 0.4¢ cents per bushel for soybeans and $3.00 per acre for 50/50 revenue.[60] Note that the decline in the average differences exactly equals the increase in the market benchmarks due to the change in LDP/MLG strategy. The only change in statistical significance occurs in the case of advisory program revenue versus 20-month market benchmark revenue. The average difference is no longer significant for the revised benchmark. Overall, these results indicate that performance results are not highly sensitive to the assumed LDP/MLG strategy for market benchmarks. While performance declines marginally with the change in LDP/MLG strategy, qualitative conclusions about advisory program performance are unaffected.

Price patterns represent another potential source of bias in the performance results. It turns out there are systematic patterns in corn and soybean price movements during the sample period that may have an important impact on the tests results. Figure 14 shows the average pattern of corn and soybean prices over the 24-month marketing window for the 1995-2003 crop years. These charts are based on the same harvest equivalent forward and spot cash prices (including LDP/MLGs) used to compute net advisory prices and the market benchmarks. The downward trend in corn prices over the 24-month window is substantial, with the high in pre-harvest prices about 55¢ per bushel higher than the post-harvest low (net of storage costs). A marketing strategy in corn that systematically priced more heavily in the pre-harvest period relative to the post-harvest period would have generated much higher returns than a strategy that did not. The price pattern in soybeans is noticeably different, with prices roughly flat for the pre-harvest period and then rising sharply through the post-harvest period before dropping off sharply. In this case, a marketing strategy that systematically priced more heavily in the first two-thirds of the post-harvest period would perform the best.

Now consider the average marketing profiles for corn and soybeans shown in Figure 15. The market benchmark and advisory program profiles were presented earlier in Figure 8 and the USDA marketing weights were presented in Figure 9. As noted earlier in the "Farmer Benchmark" section, USDA marketing weights represent grain purchases, which are not necessarily the same as pricing weights due to farmers' use of forward contracts. Only a hypothetical marketing profile for farmers is presented (labeled "Farmers ?") as a result. It is based on a similar marketing window as the market benchmarks and advisory programs, but reflects substantially less pricing in the pre-harvest period.[61] In light of the downward price trend in corn, the marketing profiles reveal why market benchmarks and advisory programs in corn generated higher average prices than the farmer benchmark over the last nine crop years. More than likely, farmers priced much less of the corn crop in the pre-harvest period than the market benchmarks or advisory programs. In contrast, the price trends in soybeans favored the marketing pattern of farmers, allowing them to perform about the same as advisory programs and actually outperform the market benchmarks.

In sum, pricing performance depends on a complex set of variables that include corn and soybean price behavior, advisory program strategies and the marketing behavior of farmers. It is on open question whether the behavior of these variables in the last nine crop years provides a reliable guide for the future. The persistence of downward price trends frequently observed over 1995-2003 in corn is an especially hotly debated issue. Further study is needed to determine whether the price patterns observed over 1995-2003 are representative of patterns in the long-run. This information would help to clarify whether market conditions during 1995-2003 bias performance comparisons in any particular direction.

Comparison of average advisory prices or revenues to benchmarks is an important indicator of performance. However, average price or revenue comparisons may not provide a complete picture of performance. For example, two advisory programs can generate the same average advisory price, but the risk of the programs may differ substantially. The difference in risk may be the result of using different pricing tools (cash, forward, futures or options), different timing of sales and variation in the implementation of marketing strategies.

A number of theoretical frameworks have been developed to analyze decision-making under risk. One of the simplest and most popular is the mean-variance (EV) model, which uses variance as a measure of risk. The basic idea in this case is to look at risk as the chance farmers will fail to achieve the net price they expect based on following an advisory program. This approach to quantifying risk does not measure the possibility of loss alone. Risk is seen as uncertainty: the likelihood that what is expected will fail to happen, whether the outcome is better or worse than expected. So an unexpected return on the upside or the downside - a net price of $2.50 or $1.50 per bushel when a net price of $2.00 per bushel is expected - counts in determining the risk of an advisory program. Thus, an advisory program whose net price does not depart much from its expected (mean) price is said to carry little risk. In contrast, an advisory program whose net price is quite volatile from year-to-year, often departing from expected net price, is said to be quite risky.

To apply the EV model to a particular decision, either distributions of outcomes must be normal or decision-makers must have quadratic utility functions (Hardaker, Huirne and Anderson, 1997, p.141). If either or both of these conditions hold, then risky choices can be divided into efficient and inefficient sets based on the famous EV efficiency rule: if the mean of choice A is greater than or equal to the mean of choice B and the variance of A is less than or equal to the variance of B, with at least one strict inequality holding, then A is preferred to B by all risk-averse decision makers. Since quadratic utility has the unlikely characteristic that absolute risk aversion increases with the level of the outcome, application of the EV model usually is based upon an assumption of normally distributed outcomes. This presents a potential problem in the case of market advisory programs that employ options strategies. Such strategies are designed to create non-normal price distributions by truncating undesirable prices, either on the downside or the upside, or both. Fortunately, simulation analysis suggests that the EV model produces reasonably accurate results even in cases where options strategies are employed (Hanson and Ladd, 1991; Ladd and Hanson, 1991; Garcia, Adam and Hauser, 1994).

The basic data needed for assessing market advisory pricing performance in an EV framework are presented in Table 38. For each of the 15 advisory programs tracked in all nine crop years of the AgMAS study, the nine-year average net advisory price or revenue and standard deviation of net advisory price or revenue is reported. The average price and standard deviation of the three benchmarks also are reported. Standard deviation is substituted for variance as the measure of risk because it easier to understand. Performance results are the same whether standard deviation or variance is used to measure risk (Hardaker, Huirne and Anderson, 1997, p.143), hence the use of the simpler measure. Standard deviation estimates can be thought of as the "typical" variation in net advisory prices from year-to-year. The larger the standard deviation for an advisory program, the less likely a farmer is to get exactly the net price expected, though it is possible by chance to get a higher price instead of a lower one for any particular time period.[62]

The sample of advisory programs for the EV analysis is limited to those which are tracked all nine crop years in order to maximize the number of observations available to estimate risk (standard deviation).[63] Even with this restriction, nine observations would appear to be a relatively small sample for estimating the risks of market advisory programs. However, as noted in the introduction, Anderson (1974) explored the reliability of agricultural return-risk estimates based on limited data and found the surprising result that even as few as three or four observations can be useful. Nonetheless, the standard deviations reported in Table 38 may be somewhat inaccurate estimators of the true risks of advisory programs. With that in mind, the standard deviations suggest that the risk of advisory programs varies substantially. In corn, the standard deviations range from a low of $0.19 per bushel to a high of $0.66 per bushel. In soybeans, the standard deviations range from a low of $0.56 per bushel to a high of $1.09 per bushel. Finally, revenue standard deviations for the 15 programs range from a low of $20 per acre to a high of $49 per acre. Standard deviations of the benchmark prices tend to be near the average standard deviation of the 15 advisory programs for corn, soybeans and 50/50 advisory revenue.

Just as in the previous section, it is important to consider the level of aggregation for the EV analysis. One possibility is to examine the mean and standard deviation of the average advisory program constructed for the average price tests. Unfortunately, this is not useful in the present context because the risk of the average program will be smaller than that typically experienced by subscribers to individual advisory programs (due to diversification effects). The better alternative is to consider a single randomly selected advisory program (e.g., Elton, Gruber and Rentzler). Estimates of the average price and risk of a randomly selected advisory program are found by taking the average across the average price and standard deviation estimates, respectively, for the 15 advisory programs presented in Table 38. The resulting estimates, presented in the row labeled "Randomly Selected Program," reflect the average price and risk for a strategy of selecting at random one of the 15 programs over 1995-2003.

The average price and risk (standard deviation) for the randomly selected advisory program, individual programs and the benchmarks are plotted in Figures 16 through 18. Each figure is divided into four quadrants based on the average price (or revenue) and standard deviation of the randomly selected advisory program ("average program"). Any observation in the upper left quadrant of each chart has a higher average price (or revenue) and less risk than the randomly selected program. According to the EV efficiency rule introduced earlier, individual programs or benchmarks in this quadrant are said to "dominate" the randomly selected program. A risk-averse farmer will prefer an individual program or benchmark in this case. Contrarily, observations in the lower right quadrant have a lower price and more risk than the randomly selected program. According to the EV efficiency rule, the randomly selected program dominates individual programs or benchmarks in this quadrant. A risk-averse farmer will prefer the randomly selected advisory program in this case. The two remaining quadrants reflect a higher price and more risk than the randomly selected program or a lower price and less risk than the randomly selected program. The randomly selected program neither dominates nor is dominated in these two quadrants. A risk-averse farmer's choice in these cases depends on personal preference for risk relative to average price.[64]

The data plotted in Figure 16 indicate that a randomly selected program in corn has a higher average price and lower standard deviation than two of the three benchmarks (20-month market benchmark and farmer benchmark), and hence, advisory programs dominate these two benchmarks. The exception is the 24-month market benchmark, where the randomly selected program has both a higher average price and standard deviation. Figure 17 indicates that, a randomly selected program in soybeans does not dominate any of the three benchmarks, as the average program has a higher average price and a higher standard deviation compared to the market benchmarks and a lower average price and a lower standard deviation compared to the farmer benchmark. Figure 18 indicates that a randomly selected program also does not dominate any of the three benchmarks in terms of 50/50 revenue. This is clearly the case for the 24-month market benchmark. However, the average program just misses dominating the 20-month market benchmark and the farmer benchmark, as the increase in risk of the average program is only slightly larger than the risk of either of these two benchmarks. It is also interesting to note that a randomly selected advisory program is not dominated by any of the benchmarks across corn, soybeans and 50/50 revenue.[65]

The EV comparisons indicate that consideration of risk weakens evidence about the pricing performance of advisory programs in some cases. The most salient example is the performance of advisory programs versus the market benchmarks in soybeans. Based on average price alone, advisory programs in soybeans significantly outperform both market benchmarks, but when both average price and risk are considered, advisory programs no longer dominate due to substantially higher risk. However, from an economic decision-making perspective, consideration of risk does not change qualitative conclusions about the economic significance of advisory program revenue versus the farmer benchmarks. The average advisory return relative to the farmer benchmark is seven dollars per acre with only a negligible increase in risk. As noted in the previous section, this return is small but nonetheless represents a non-trivial increase in net farm income per acre for grain farms in Illinois.

Finally, the mean-variance evaluation presented in this section can be extended to portfolios of advisory programs. For example, a soybean portfolio might consist of 50% marketed by advisory program #1 and 50% marketed by advisory program #2. The potential improvement in performance by following a combination of programs depends on the degree that net advisory prices or revenues are uncorrelated. Stark et al. (2003) analyze the potential risk reduction among market advisory programs for corn and soybeans. Under the assumption that programs are equally-weighted and randomly-selected (naïve diversification), results from this study show that increasing the number of programs reduces portfolio expected risk, but the marginal decrease in risk from adding a new program decreases rapidly with portfolio size. The risk reduction benefit from this type of diversification among advisory programs is relatively small because advisory prices, on average, are highly correlated. For example, a one service portfolio has only a 20%, 16% and 32% higher standard deviation than the minimum risk portfolio (all programs equally-weighted) for corn, soybeans and 50/50 revenue, respectively. Most risk reduction benefits are achieved with small portfolios. For instance, a four service portfolio has only 5%, 4% and 9% higher risk than the minimum risk portfolio for corn, soybeans and 50/50 revenue, respectively. Based on these results, there does not appear to be strong justification for farmers adopting portfolios with a large number of advisory programs.

For a more complete analysis of the possible benefits from diversification among advisory programs, it is necessary to evaluate portfolios constructed using modern portfolio theory (MPT). Under this approach, an efficient set of optimal portfolios of market advisory programs is constructed by minimizing portfolio variance for each level of expected price or revenue. The resulting optimal portfolios generally will not be equally-weighted across programs. It is possible for an optimal portfolio of advisory programs to generate higher prices and less risk than a benchmark, even if individual advisory programs that make up the portfolio do not. Cabrini et al. (2004) estimate mean-variance efficient portfolios of market advisory programs and find that the number of programs included in optimal portfolios usually is small, in the range of two to four programs in most cases. However, in some cases up to six advisory programs are included. The optimization results provide some evidence that an efficient portfolio provides greater risk/return benefits compared to market and farmer benchmarks. In a holdout period analysis, efficient portfolios have superior performance in terms of average price, but fail to dominate the benchmarks in terms of both average price and risk. The main difficulty in generating optimal portfolios is obtaining accurate estimates of the means, variance and correlations for individual programs from the available data.

Even if, as a group, advisory programs generate positive marketing returns, there is a wide range in performance for any given year. For example, soybean net advisory prices in 2003 vary from $3.69 per bushel to $7.67 per bushel (see Table 26). While this example is one of the most dramatic, the variation across advisors in other cases is substantial. This raises the important question of the predictability of advisory program performance from year-to-year. In other words, is past performance indicative of future performance? Three types of predictability tests are used to answer this question: i) the predictability of "winner" and "loser" categories across crop years, ii) the correlation of advisory program ranks across crop years and iii) the differences between prices for "top" and "bottom" performing advisory programs across crop years. The testing procedures have been widely applied in studies of financial investment performance (e.g., Elton, Gruber and Rentzler, 1987; Irwin, Zulauf and Ward, 1994; Lakonishok, Shleifer and Vishny, 1992; Malkiel, 1995).[66]

The first test of predictability is based on placing advisory programs into "winner" and "loser" categories across adjacent crop years. This non-parametric test is robust to outliers, which is important when analyzing predictability across all advisory programs. For a given commodity, the first step in this testing procedure is to form the sample of all advisory programs that are active in adjacent crop years. The second step is to rank each advisory program in the first year of the pair (e.g., t = 1997) based on net advisory price. For example, the program with the highest net advisory price is ranked number one and the program with the lowest net advisory price is assigned a rank equal to the total number of programs for that commodity in the given crop year. Then the programs are sorted in descending rank order. The third step is to form two groups of programs in the first year of the pair: winners are those programs in the top half of the rankings and losers are programs in the bottom half. The fourth step is to rank each advisory program in the second year of the pair (e.g., t +1 = 1998) based on net advisory price and once again form winner and loser groups of programs. The fifth step is to compute the following counts for the advisory programs in the pair of crop years: winner t-winner t+1, winner t-loser t+1, loser t-winner t+1, loser t-loser t+1. If advisory program performance is unpredictable, approximately the same counts will be found in each of the four combinations. The appropriate statistical test in this case is known as Fisher's Exact Test (Conover, 1999, pp.188-189).[67]

Results of the winner and loser predictability tests are shown in Table 39. Winner and loser counts for individual crop years indicate a modest difference, at best, in the chance of a winner or loser in one period being a winner or loser in the subsequent period. As an example, consider the results for corn in 1997 and 1998. Of the twelve winners in 1997, seven are winners in 1998 and five are losers. Of the eleven losers in 1997, five are winners in 1998 and six are losers. In other words, the conditional probability of a winner from 1997 repeating in 1998 is 56% (7/12) and the conditional probability of a loser from 1997 repeating in 1998 is 55% (6/11). Averaged across all comparisons, the conditional probability of a winner (loser) repeating is 56% (55%) for corn, 55% (53%) for soybeans and 56% (54%) for 50/50 revenue. These probabilities are only slighter higher than what would result from flipping a coin (randomness). There are only two cases (corn: 1999 vs. 2000; 50/50 revenue: 1999 vs. 2000) where individual year counts are significantly different from the equal distribution expected under an assumption of no predictability. Even in these cases, caution should be used when considering the reported p-values, because it is likely overstated due to the observed dependence across advisory programs.[68] Overall, these results imply that the performance of winning and losing advisory programs is not predictable through time.

While predictability may be limited or non-existent across all advisory programs, it is possible for sub-groups of advisory programs to exhibit predictability. Specifically, predictability may be found only at the extremes of performance. That is, only top-performing programs in one year may tend to perform well in the next year, or only poor-performing programs may perform poorly in the next year, or both. This is the motivation for the second test of predictability, which is based on the correlation between ranks of all advisory programs active in adjacent pairs of crop years. For a given commodity, the first step in this testing procedure is to once again form the sample of all advisory programs that are active in both adjacent crop years. The second step is to rank each advisory program in the first year of the pair (e.g., t = 1997) based on net advisory price. Then the programs are sorted in descending rank order. The third step is to sort and rank the sample of programs in the second year of the pair (e.g., t + 1 = 1998). The fourth step is to compute the correlation coefficient between ranks for the two adjacent crop years. If advisory program performance is unpredictable, the estimated correlation will be near zero. Assuming the standard error of the correlation coefficient is approximately equal to , the appropriate statistical test is a Z-test.

Results of the rank correlation predictability test are presented in Table 40. Rank correlation coefficients for corn range from of -0.12 to +0.53. Statistically significant correlations are found for four of the nine comparisons in corn. The range of rank correlation coefficients for soybeans, +0.03 to +0.65, is similar to the range for corn. However, statistically significant correlations are found for only one of the nine comparisons in soybeans. Rank correlation coefficients for 50/50 revenue have the widest range, from -0.17 to +0.72. Statistically significant correlations are found for two of the nine revenue comparisons. Once again, caution should be used when considering the reported p-values, as they likely overstate the significance of the rank correlation estimates due to the dependence across advisory programs. Average rank correlation coefficients across the eight comparisons are nearly identical for corn, soybeans and 50/50 advisory revenue. With average values from 0.24 to 0.26, the rank correlations suggest marginal predictability in the pricing performance of top- and bottom-performing market advisory programs.[69]

The rank correlation tests results suggest it is useful to determine the magnitude of predictability in top- and bottom-performing advisory programs. Hence, the third test of predictability is based on the difference between net advisory prices for top- and bottom-performing advisory programs across adjacent crop years. For a given commodity, the first step in this testing procedure is to sort programs by net advisory price in the first year of the pair and form groups of programs. The first grouping consists of the top third of programs, middle third of programs and bottom third of programs. The second grouping consists of the top fourth of programs, second fourth of programs, third fourth of programs and bottom fourth of programs. The last grouping is simply to form the top two and bottom two programs. Notice that the groupings proceed from a relatively large number of programs in the top- and bottom-performing segments (e.g., thirds) to only a few programs (two). Hence, if advisory program performance from year-to-year is persistence at the extremes, these groupings should reveal it.

The second step of the grouping procedure is to compute the average net advisory price for the groups in the second year of the pair. Note that the same programs make up the groups in the first and second year of the pair. For example, the average price of the top fourth group formed in 1995 is computed for 1996. The third step is to compute the difference in average price for the top- and bottom-performing groups. If performance for the top- and bottom-performing groups is the same, the difference will equal zero. The appropriate statistical test in this case is a paired t-test of the difference in the means of the top- and bottom-performing groups. There are a total of eight comparisons (1995 vs. 1996, 1996 vs. 1997, 1997 vs. 1998, 1998 vs. 1999, 1999 vs. 2000, 2000 vs. 2001, 2001 vs. 2002 and 2002 vs. 2003), so there are seven degrees of freedom for the t-test. Since differences are computed for an "average" advisory program in top- and bottom-performing groups, dependence across individual advisory programs is not an issue, and p-values for the t-test are unbiased. Carpenter and Lynch (1999) recommend this test because it is well-specified and among the most powerful in their comparison of several predictability tests for mutual funds.

Results for the t-test of predictability for the different groupings are shown in Table 41. The first column under each commodity heading shows the average price of the different groups in the first year of the comparisons (eight in total). The average price for the first year is "in-sample" because this is the formation year for the groups. The second column under each heading reports the average price of the same groups in the second year of the comparisons. The average price for the second year is "out-of-sample" because this is the year after formation of the groups. In all cases, the average price or revenue of the top group relative to the bottom group declines substantially from the first to the second year of the comparisons. Nonetheless, the average difference between top- and bottom-performing groups for the second year of the pair is consistently positive. Furthermore, the average differences increase substantially as the groupings become successively narrower. Programs in the top third beat the bottom third in the second year by an average of 10¢ per bushel in corn, 25¢ per bushel in soybeans and $9 per acre for revenue, while the top two programs beat the bottom two programs in the second year by an average of 18¢ per bushel in corn, 43¢ per bushel in soybeans and $27 per acre for revenue.[70] This pattern suggests predictability is most pronounced near the top and bottom of advisory program performance rankings. Statistical significance is observed in two of three cases for thirds, all three cases for fourths, but only one of three cases for the top- and bottom-two groups. It is not surprising that less statistical significance is found for top- and bottom-two comparisons since these groups only average net advisory prices for two programs. Consequently, average differences will be more volatile from year-to-year compared to average differences based on thirds and fourths. Finally, note that average prices for the top group out-of-sample also exceed benchmark prices for the same period (1996-2003). Top third returns beat the 24-month market benchmark by an average of 4¢ per bushel in corn, 28¢ per bushel in soybeans and $8 per acre for 50/50 revenue. Top two returns beat the 24-month market benchmark by an average of 8¢ per bushel in corn, 54¢ per bushel in soybeans and $20 per acre for 50/50 revenue.

The grouped results appear to provide strong evidence that the performance of top- and bottom-performing market advisory programs can be predicted across adjacent crop years. However, the evidence is not sufficient to conclude that performance predictability is useful from an economic standpoint, due to the overlapping nature of the marketing windows for each crop year. First, to the degree that old and new crop prices are correlated and advisory programs follow similar strategies across crop years, the overlapping may induce "artificial" predictability in performance across adjacent pairs of overlapping crop years. Second, the overlapping creates a practical problem if a farmer attempts to take advantage of the observed predictability. To see the point, consider the case of a farmer who uses 1995 performance results to select a top-performing advisory program. Since the 1995 marketing window ends on August 31, 1996, halfway through the 1996 marketing window and one day before the beginning of the 1997 marketing window, the farmer could not implement the selection of an advisory program until the 1997 crop year. Performance would have to persist across three crop years, 1995, 1996 and 1997, for a farmer to benefit from the predictability.

Grouped results for non-overlapping crop years are shown in Table 42. The testing procedure is the same as before, except there are only seven comparisons (1995 vs. 1997, 1996 vs. 1998, 1997 vs. 1999, 1998 vs. 2000, 1999 vs. 2001, 2000 vs. 2002 and 2001 vs. 2003) and six degrees of freedom for the paired t-test. The results are strikingly different than the previous results for overlapping crop years. Average differences between top- and bottom-performing groups in the second year of the pair are near zero or negative in all but two cases (soybeans for thirds and fourths comparisons). All of the average differences for the second year are statistically insignificant. These results indicate predictability of pricing performance for top and bottom advisory programs is short-lived, in the sense that performance does not persist long enough to be taken advantage of by farmers.

The predictability results presented so far are all based on individual crop year comparisons. It is possible for performance to be predictable over longer time horizons, but unpredictable over shorter horizons due to the large amount of "noise" in performance over shorter horizons (e.g., Summers, 1986). This is consistent with the argument that over the long-term "cream rises to the top" in terms of performance. To assess long-term predictability, the sample is again limited to the 15 programs active in all nine crop years of the study. Next, net advisory prices are averaged for each of the 15 programs during the first four crop years (1995-1998) and the last four crop years (2000-2003). The 1999 crop year is excluded in order to make the averages for the two periods non-overlapping. The three tests of predictability are then applied to the two sets of averages. Results show that winner-loser counts are quite close to what is expected under randomness. Rank correlations between the two periods are +0.19 for corn, +0.22 for soybeans and +0.36 for revenue. None of the rank correlations are significantly different from zero. However, average differences between top- and bottom-performing programs, presented in Table 43, provide some evidence of predictability. Consistent evidence of predictability is found for top- and bottom-performing programs in soybeans across all three groupings. Of particular interest are the results for the top two programs, which outperform the bottom two programs by 19¢ per bushel in corn, 47¢ per bushel in soybeans and $25 per acre for 50/50 revenue.[71] These are clearly substantial differences in economic terms. Nonetheless, the differences should be viewed cautiously because only one "long-term" comparison is available, and hence, it is not possible to test for statistical significance. The results also are heavily influenced by a single advisory program that is ranked first among the 15 programs in both four-year periods for corn, soybeans and revenue. If this advisory program is excluded from the analysis, the difference in performance during the second period between the top two and bottom two programs declines to 10¢ per bushel in corn, 30¢ per bushel in soybeans and only $2 per acre for advisory revenue.

The test results presented in this section suggest that it is difficult to usefully predict the year-to-year pricing performance of advisory programs based on past pricing performance. There is some evidence that performance is more predictable over longer time horizons, particularly at the extremes of performance rankings. It should be emphasized that the analysis in this section does not necessarily rule out other variables that may be useful for predicting performance. Chevalier and Ellison (1999) study whether mutual fund performance is related to characteristics of fund managers that indicate ability, knowledge or effort and find that managers who attended higher-SAT undergraduate institutions generate systematically higher returns. Barber and Odean (2000) examine the trading records of individual stock investors and report that frequent trading substantially depresses investment returns. Similar factors, such as education of advisors, cash only programs versus futures and options programs, frequency of futures and options trading, or storage costs, may be useful in predicting the performance of market advisory programs.

Surveys suggest that farmers view market advisory services as an important tool in managing price and income risk. As a result, farmers need information on the performance "track record" of market advisory services to help them identify successful alternatives for marketing and price risk management. The Agricultural Market Advisory Service (AgMAS) Project was initiated in 1994 with the goal of providing unbiased and rigorous evaluation of market advisory services.

The purpose of this research report is to evaluate the pricing performance of market advisory services for the 1995-2003 corn and soybean crops. No fewer than 23 market advisory programs are available for each crop year. While the sample of advisory services is non-random, it is constructed to be generally representative of the majority of advisory services offered to farmers. Further, the sample of advisory services includes all programs tracked by the AgMAS Project over the study period, so pricing performance results should not be plagued by survivorship bias. The AgMAS Project subscribes to all of the services that are followed and records recommendations on a real-time basis, which should prevent pricing performance results from being subject to hindsight bias.

Certain explicit assumptions are made to produce a consistent and comparable set of results across the different advisory programs. These assumptions are intended to accurately depict "real-world" marketing conditions facing a representative central Illinois corn and soybean farmer. Several key assumptions are: i) with a few exceptions, the marketing window for a crop year runs from September before harvest through August after harvest, ii) on-farm or commercial physical storage costs, as well as interest opportunity costs, are charged to post-harvest sales, iii) brokerage costs are subtracted for all futures and options transactions and iv) Commodity Credit Corporation (CCC) marketing loan recommendations made by advisory programs are followed wherever feasible. Based on these and other assumptions, the net price received by a subscriber to a market advisory program is calculated for the 1995-2003 corn and soybean crops.

Two different types of benchmarks are developed for the performance evaluations. Efficient market theory implies that the return offered by the market is the relevant benchmark. In the context of this study, a market benchmark should measure the average price offered by the market over the marketing window of a representative farmer who follows advisory program recommendations. Both a 24-month and a 20-month market benchmark are specified in order to test the sensitivity of performance results to different market benchmark assumptions. Behavioral market theory suggests that the average return actually achieved by market participants as an appropriate benchmark. In the context of the present study, a behavioral benchmark should measure the average price actually received by farmers for a crop. A farmer benchmark is specified based upon the USDA average price received series for corn and soybeans in Illinois. All benchmarks are computed using the same assumptions applied to advisory program track records.

Four basic indicators of performance are applied to advisory program prices and revenues over 1995-2003. The first indicator is the proportion of advisory programs that beat benchmark prices. Between 50 and 59% of the programs in corn have net advisory prices above market benchmarks over 1995-2003, indicating a zero to marginal chance of advisory prices in corn beating market benchmark prices. In contrast, 68% of the programs have prices above the farmer benchmark in corn. Between 65 and 72% of advisory programs in soybeans have advisory prices above the market benchmarks over 1995-2003, suggesting a better than average chance of advisory prices beating market benchmark prices in soybeans. The proportion of advisory programs above the farmer benchmark in soybeans is only 54%, indicating a small chance of programs generating net prices in soybeans higher than the farmer benchmark. Between 59 and 68% of advisory programs have revenue above the market benchmarks over 1995-2003, while 62% have revenue above the farmer benchmark. This indicates a moderate chance of advisory revenue beating farmer benchmark revenue. Overall, the directional test results provide mixed performance evidence with respect to the market benchmarks and the farmer benchmark.

The second indicator is the difference between the average price of advisory programs and the market or farmer benchmarks. The results basically tell the same story as those based on the proportion beating the benchmarks. Average differences from market benchmarks for corn over 1995-2003 are small, ranging from 1 to 3¢ cents per bushel. At 8¢ cents per bushel, the average difference from the farmer benchmark for corn is larger. Average differences from market benchmarks for soybeans over 1995-2003 are substantial, ranging from 14 to 16¢ per bushel. In contrast, the average difference from the farmer benchmark for soybeans is -1¢ per bushel. Average differences for advisory revenue range from $4 to 7 per acre for market benchmarks over 1995-2003. The average revenue difference versus the farmer benchmark is $7 per acre. An important consideration is the size of the average differences versus the farmer benchmark from an economic decision-making perspective. The average advisory return relative to the farmer benchmark is about two percent of average farmer benchmark revenue. Even though this return is small and entirely from corn, it nonetheless represents a non-trivial increase in net farm income, typically about $50 per acre for grain farms in Illinois.

Statistical test results with respect to market benchmarks indicate no evidence of significant average price performance in corn, consistent evidence of significant performance in soybeans and mixed evidence for 50/50 advisory revenue. The test results with respect to the farmer benchmark indicate consistent evidence of significant performance in corn, no evidence of significant performance in soybeans and mixed evidence of significant performance for 50/50 advisory revenue.

The third indicator is the average price and risk of advisory programs relative to benchmarks. The results indicate that consideration of risk weakens evidence about the pricing performance of advisory programs in some cases. The most salient example is the performance of advisory programs versus the market benchmarks in soybeans. Based on average price alone, advisory programs in soybeans significantly outperform both market benchmarks, but when both average price and risk are considered, advisory programs no longer dominate due to substantially higher risk. However, from an economic decision-making perspective, consideration of risk does not change qualitative conclusions about the economic significance of advisory program revenue versus the farmer benchmark. The average advisory return relative to the farmer benchmark is seven dollars per acre with only a negligible increase in risk.

The fourth indicator is the predictability of advisory program performance. "Winner" and "loser" predictability results are similar for corn, soybeans and advisory revenue. The conditional probability of winner and loser programs (top half and bottom half) repeating from year-to-year are only slighter higher than what would result from flipping a coin (randomness) and provide little evidence that pricing performance for all advisory programs can be predicted from past performance. The performance of top- and bottom-performing programs from year-to-year does not appear to be predictable in a useful sense either. For example, comparisons of non-overlapping crop years show that average differences between top- and bottom-performing groups are near zero or negative in all but two cases and none of the average differences are significantly different from zero. The test results suggest that it is difficult to usefully predict the year-to-year pricing performance of advisory programs based on past pricing performance. However, there is some evidence that performance is more predictable over longer time horizons, particularly at the extremes of performance rankings.

In conclusion, the results of this study provide an interesting picture of the performance of market advisory programs in corn and soybeans. There is limited evidence that advisory programs as a group outperform market benchmarks, particularly after considering risk. This supports the view that grain markets (cash, futures and options) are efficient with respect to the types of marketing strategies available to farmers (e.g., Zulauf and Irwin, 1998) over the view that grain markets are inefficient and provide substantial opportunities for farmers to gain additional profits through marketing (e.g., Wisner, Blue and Baldwin, 1998). The evidence is more positive with respect to the farmer benchmark, even after taking risk into account. This raises the possibility that even though advisory services do not "beat the market," they nonetheless provide the opportunity for some farmers to improve performance relative to the market. Mirroring debates about stock investing (e.g., Damato, 2001), the relevant issue is then whether farmers can most effectively improve marketing performance by pursuing "active" strategies, like those recommended by advisory services, or "passive" strategies, which involve routinely spreading sales across the marketing window. Recently, a number of grain companies began offering averaging or "indexing" contracts that allow farmers to easily implement a passive approach to marketing (Smith, 2001). The rising interest in these new marketing contracts suggests the potential for historic changes in the approach farmers' use to market crops. Future research that provides a better understanding of the costs and benefits of active versus passive approaches to marketing will be especially valuable.

Alexander, V.J., W.N. Musser and G. Mason. "Futures Markets and Firm Decisions Under Price, Production and Financial Uncertainty." Southern Journal of Agricultural Economics, 18(1986):39-49.

Anderson, J.R. "Sparse Data, Estimational Reliability and Risk-Efficient Decisions." American Journal of Agricultural Economics, 55(1974):564-572.

Anderson, K. B., and H. P. Mapp. "Risk Management Programs in Extension." Journal of Agricultural and Resource Economics, 21(1996):31-38.

Barber, B.M. and T. Odean. "Trading is Hazardous to Your Wealth: The Common Stock Investment Performance of Individual Investors." Journal of Finance, 55(2000):773-806.

Bradley, E.L. and L.G. Blackwood. "Comparing Paired Data: A Simultaneous Test for Means and Variances." American Statistician, 43(1989):234-235.

Brorsen, B.W. and K. Anderson. "Cash Wheat Marketing: Strategies for Real People." Journal of Agribusiness, 12(1994):85-94.

Brorsen, B.W. and K. Anderson. "Implications of Behavioral Finance for Farmer Marketing Strategy Recommendation." Proceedings of the 2001 NCR 134 Conference on Applied Commodity Price Analysis, Forecasting and Market Risk Management, Department of Agricultural Economics, Colorado State University. (http://agecon.lib.umn.edu)

Brorsen, B.W., J. Coombs and K. Anderson. "The Cost of Forward Contracting Wheat." Agribusiness, 11(1995):349-354.

Brorsen, B.W. and S.H. Irwin. "Improving the Relevance of Research on Price Forecasting and Marketing Strategies." Agricultural and Resource Economics Review, 25(1996):68-75.

Brown, S.J., W. Goetzmann, R.G. Ibbotson and S.A. Ross. "Survivorship Bias in Performance Studies." Review of Financial Studies, 5(1992):553-580.

Brown, S.J., W.N. Goetzmann and R.G. Ibbotson. "Offshore Hedge Funds: Survival and Performance, 1989-95." Journal of Business, 72(1999):91-117.

Cabrini, S.M., B.G. Stark, H. _nal, S.H. Irwin, D.L. Good and J. Martines-Filho. "Efficiency Analysis of Agricultural Market Advisory Services: A Non-Linear Mixed-Integer Programming Approach," Manufacturing and Service Operations Management, 6(2004):237-252.

Carpenter, J.N. and A.W. Lynch. "Survivorship Bias and Attrition Effects in Measures of Performance Persistence." Journal of Financial Economics, 54(1999):337-374.

Chafin, D.G. and P.H. Hoepner. Commodity Marketing from a Producer's Perspective, Second Edition. Interstate Publishers: Danville, Illinois, 2002.

Chevalier, J. and G. Ellison. "Are Some Mutual Fund Managers Better Than Others? Cross-Sectional Patterns in Behavior and Performance." Journal of Finance, 54(1999):875-899.

Coble, K.H., G.F. Patrick, T.O. Knight and A.E. Baquet. "Crop Producer Risk Management Survey: A Preliminary Summary of Selected Data." Information Report 99-001, Department of Agricultural Economics, Mississippi State University, September 1999.

Collender, R.N. "Estimation Risk in Farm Planning Under Risk." American Journal of Agricultural Economics, 71(1989):996-1002.

Colino, E.V., S.M. Cabrini, S.H. Irwin, D.L. Good and J.Martines-Filho. "Advisory Service Marketing Profiles for Soybeans in 2001." AgMAS Project Research Report 2004-02, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, April 2004.

Colino, E.V., S.M. Cabrini, S.H. Irwin, D.L. Good and J.Martines-Filho. "Advisory Service Marketing Profiles for Corn in 2001." AgMAS Project Research Report 2004-01, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, April 2004.

Conover, W.J. Practical Nonparametric Statistics, Third Edition. John Wiley and Sons: New York, 1999.

Damato, K. "Index Funds: 25 Years in Pursuit of the Average." The Wall Street Journal, April 9, 2001, pp. R1, R6.

Daniel, K.D., D. Hirshleifer and A. Subrahmanyam. "Investor Psychology and Security Market Over- and Under-reactions." Journal of Finance, 53(1998):1839-1886.

Davis, T.D. and G.F. Patrick. "Forward Pricing Behavior of Soybean Producers." Selected Paper, Annual Meeting of the American Agricultural Economics Association Meetings, 2000.

Dhuyvetter, K.C., G.L. Hamman and J.P. Harner, III. The Economics of On-Farm Farm Storage, MF-2474, Kansas State University Agricultural Experiment Station and Cooperative Extension Service, October 2000. (http://www.agecon.ksu.edu/kdhuyvetter/pdf%20files/mf2474.pdf)

Elam, E. and J. Woodworth. "Forward Selling Soybeans with Cash Forward Contract, Futures Contracts, and Options." Arkansas Business and Economic Review, 22(1989):10-20.

Elton, E. J., M. J. Gruber and J. C. Rentzler. "Professionally Managed, Publicly Traded Commodity Funds." Journal of Business, 60(1987):175-199.

Fackler, P.L., D.L. Young and G.A. Carlson. "Estimates of Trend and Variability Patterns in U.S. Crop Yields," in Quantifying Long Run Agricultural Risks and Evaluating Farmers' Responses to Risk. Proceedings of a seminar sponsored by the Southern Regional Project S-252, Jekyll Island, Georgia, March 1993.

Fama, E. "Efficient Capital Markets: A Review of Theory and Empirical Work." Journal of Finance, 25(1970):383-417.

Fama, E.F. and J.D. MacBeth. "Risk, Return and Equilibrium: Empirical Tests." Journal of Political Economy, 81(1973):607-636.

Frino, A. and D.R. Gallagher. "Tracking S&P 500 Index Funds." Journal of Portfolio Management, 28(2001):44-55.

Garcia, P., B.D. Adam and R.J. Hauser. "The Use of Mean-Variance for Commodity Futures and Options Hedging Decisions." Journal of Agricultural and Resource Economics, 19(1994):32-45.

Gehrt, D.W. and D.L. Good. "Evaluation of Market Advisory Services for Corn and Soybeans." Journal of the American Society of Farm Managers and Rural Appraisers, 57(1993):1-7.

Good, D.L., T.A. Hieronymus and R.A. Hinton. Price Forecasting and Sales Management: Corn, Soybeans, Cattle and Hogs. Cooperative Extension Service, College of Agriculture, University of Illinois at Urbana-Champaign, 1980.

Good, D.L., S.H. Irwin and T.E. Jackson. "Development of a Market Benchmark Price for AgMAS Performance Evaluations." AgMAS Project Research Report 1998-02, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, December 1998.

Griffiths, W.E., R.C. Hill and G.C. Judge. Learning and Practicing Econometrics. John Wiley and Sons, Inc.: New York, 1993.

Hagedorn, L., S.H. Irwin, D.L. Good, J. Martines-Filho, B.J. Sherrick and G.D. Schnitkey. "New Generation Grain Marketing Contracts." AgMAS Project Research Report 2003-01, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, January 2003. (http://www.farmdoc.uiuc.edu/agmas/reports/index.html)

Hanson, S.D. and G.W. Ladd. "Robustness of the Mean-Variance Model with Truncated Probability Distributions." American Journal of Agricultural Economics, 73(1991):436-445.

Hardaker, J.B., R.B.M. Huirne and J.R. Anderson. Coping with Risk in Agriculture. CAB International: New York, 1997.

Harden, A. "Personal Communication." Illinois Agricultural Statistics Service, June 4, 2003.

Harris, H.M. and S.E. Miller. "An Analysis of Cash Contracting Corn and Soybeans in South Carolina" in National Conference on Grain Marketing Patterns, Southern Cooperative Series Bulletin No. 307, March 1981, pp. 290-300.

Hartzmark, M.L. "Returns to Individual Traders of Futures: Aggregate Results." Journal of Political Economy, 95(1987):1292-306.

Hill, L., E. Kunda and C. Rehtmeyer. "Price Related Characteristics of Illinois Grain Elevators, 1982," AE-4561, Department of Agricultural Economics, University of Illinois at Urbana-Champaign, September 1983.

Hirshleifer, D. "Investor Psychology and Asset Pricing." Journal of Finance, 56(2001):1533-1597.

Hurburgh, Jr., C.R., C.J. Bern, W.F. Wilcke and M.E. Anderson. "Shrinkage and Corn Quality Changes in On-Farm Handling Operations." Transactions of the ASAE, 26(1983):854-1857.

Irwin S.H., J. Martines-Filho and D.L. Good. "The Pricing Performance of Market Advisory services in Corn and Soybeans over 1995-2001." AgMAS Project Research Report 2003-05, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, July 2003. (http://www.farmdoc.uiuc.edu/agmas/reports/index.html)

Irwin, S.H., C.R. Zulauf and B.L. Ward. "The Predictability of Managed Futures Returns." Journal of Derivatives, 2(1994):20-27.

Isengildina, O., J.M.E Pennings, S.H. Irwin and D.L. Good. "Crop Farmers' Use of Market Advisory Services." AgMAS Project Research Report 2004-03, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, May 2004. (http://www.farmdoc.uiuc.edu/agmas/reports/index.html)

Jackson, T.E., S.H. Irwin and D.L. Good. "1996 Pricing Performance of Market Advisory Services for Corn and Soybeans," AgMAS Project Research Report 1998-01, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, January 1998. (http://www.farmdoc.uiuc.edu/agmas/reports/index.html)

Jaffe, J.F. and J.M. Mahoney. "The Performance of Investment Newsletters." Journal of Financial Economics, 53(1999):289-307.

Jirik, M.A., S.H. Irwin, D.L. Good, T.E. Jackson and J. Martines-Filho. "The 1995 Through 1998 Pricing Performance of Market Advisory Services for Wheat," AgMAS Project Research Report 2000-02, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, June 2000. (http://www.farmdoc.uiuc.edu/agmas/reports/index.html)

Katchova, A.L. and M.J. Miranda. "Two-Step Econometric Estimation of Farm Characteristics Affecting Marketing Contract Decisions." American Journal of Agricultural Economics, 86(2004):88-102.

Kastens, T.L. and T.C. Schroeder. "Efficiency Tests of July Kansas City Wheat Futures." Journal of Agricultural and Resource Economics, 21(1996):187-198.

King, R.P., L.S. Lev and W.E. Nefstad. "A Position Report for Farm-Level Marketing Management." Review of Agricultural Economics, 17(1995):205-212.

Ladd, G.W. and S.D. Hanson. "Price Risk Management with Options: Optimal Market Positions and Institutional Value." Journal of Futures Markets, 11(1991):737-750.

Lakonishok, J., A. Shleifer and R.W. Vishny. "The Structure and Performance of the Money Management Industry." Brookings Papers: Microeconomics, (1992):339-391.

Lattz, D.H., C.E. Cagley and D.D. Raab. Summary of Illinois Farm Business Records for 2003, Circular 1388-04, University of Illinois Extension, 2004.

Leamer, E. E. "Let's Take the Con Out of Econometrics." American Economic Review, 73(1983):31-43.

Lence, S.H. and M.L. Hayenga. "On the Pitfalls of Multi-year Rollover Hedges: The Case of Hedge-to-Arrive Contracts." American Journal of Agricultural Economics, 83(2001):107-119.

Malkiel, B.G. "Returns from Investing in Equity Mutual Funds 1971 to 1991." Journal of Finance, 50(1995):549-572.

Malkiel, B.G. A Random Walk Down Wall Street. W.W. Norton and Company: New York, 1999.

Marcus, A.J. "The Magellan Fund and Market Efficiency." Journal of Portfolio Management, 16(1990):85-88.

Marten, J. "Farmers Want Market News, Not Advice." Farm Journal Extra, June 1994.

Martines-Filho, J. Pre-Harvest Marketing Strategies for Corn and Soybeans: A Comparison of Optimal Hedging Models and Market Advisory Service Recommendations. Unpublished Ph.D. Dissertation, The Ohio State University, 1996.

Martines-Filho, J., S.H. Irwin, D.L. Good, S.M. Cabrini, B.G. Stark, W. Shi, R.L. Webber, L.A. Hagedorn and S.L. Williams. "Advisory Service Marketing Profiles for Corn Over 1995-2000," AgMAS Project Research Report 2003-03, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, June 2003a. (http://www.farmdoc.uiuc.edu/agmas/reports/index.html)

Martines-Filho, J., S.H. Irwin, D.L. Good, S.M. Cabrini, B.G. Stark, W. Shi, R.L. Webber, L.A. Hagedorn and S.L. Williams. "Advisory Service Marketing Profiles for Soybeans Over 1995-1999," AgMAS Project Research Report 2003-04, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, June 2003b. (http://www.farmdoc.uiuc.edu/agmas/reports/index.html)

Martines-Filho, J., D.L. Good and S.H. Irwin. "1999 Pricing Performance of Market Advisory Services for Corn and Soybeans," AgMAS Project Research Report 2000-04, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, December 2000. (http://www.farmdoc.uiuc.edu/agmas/reports/index.html)

Marquardt, R. and A.F. McGann. "Forecasting Commodity Prices." Commodity Journal, 10(1975):29-33.

Mathews, K. H. and D. M. Holthausen. "A Simple Multiperiod Minimum Risk Hedge Model." American Journal of Agricultural Economics, (1991):1020-1026.

McBride, W.D. "Personal E-mail Communication." Economic Research Service, U.S. Department of Agriculture, March 2005.

McBride, W.D. and J.D. Johnson. "Approaches to Management and Farm Business Success," Paper presented at the 2004 American Agricultural Economics Association meetings, Denver, Colorado, August 1-4, 2004.

McNew, K. and W.N. Musser. "Farmer Forward Pricing Behavior: Evidence from Marketing Clubs." Agricultural and Resource Economics Review, 31(2002):200-210.

Metrick, A. "Performance Evaluation with Transactions Data: The Stock Selection of Investment Newsletters." Journal of Finance, 54(1999):1743-1776.

Mirer, T. W. Economics and Statistics, Third Edition. Prentice Hall, Englewood Cliffs, NJ, 1995.

Nelson, R. D. "Forward and Futures Contracts as Preharvest Commodity Marketing Instruments." American Journal of Agricultural Economics, 67(1985):15-23.

Norvell, J.M. and D.H. Lattz. "Value-Added Crops, GPS Technology and Consultant Survey: Summary of a 1998 Survey to Illinois Farmers," Working Paper, College of Agricultural, Consumer and Environmental Sciences, University of Illinois, July 1999.

Otte, J. "Marketing Matters -- How Well Do Market Advisors Deliver?" Prairie Farmer, July 19, 1986.

Patrick, G.F. and S. Ullerich. "Information Sources and Risk Attitudes of Large Scale Farmers, Farm Managers and Agricultural Bankers." Agribusiness, 12(1996):461-471.

Patrick, G.F., W.N. Musser and D.T. Eckman. "Forward Marketing Practices and Attitudes of Large-Scale Midwestern Grain Farmers." Review of Agricultural Economics, 20(1998):38-53.

Paul, A.B., R.G. Heifner and J.W. Helmuth. "Farmers' Use of Forward Contracts and Futures Markets." National Economic Analysis Division, Economic Research Service, U.S. Department of Agriculture, Agricultural Economic Report No. 320, 1976.

Pennings, J.M.E., D.L. Good, S.H. Irwin and J.K. Gomez. "The Role of Market Advisory Services in Crop Marketing and Risk Management: A Preliminary Report of Survey Results," AgMAS Project Research Report 2001-02, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, March 2001. (http://www.farmdoc.uiuc.edu/agmas/reports/index.html)

Pennings, J.M.E., O. Isengildina, S.H. Irwin and D.L. Good. "Modeling the Impact of Market Advisory Services on Producers' Decisions." Journal of Agricultural and Resource Economics, 29(2004):308-327.

Peterson, H.H. and W.G. Tomek. "How Much of Commodity Price Behavior Can a Rational Expectations Storage Model Explain?" Staff Paper No. 04-04, Department of Agricultural Economics, Kansas State University, November 2003.

Powers, L. "How to Measure Your Pro's Performance." Top Farmer, April 1993, p. 17.

Roll, R. "A Simple Implicit Measure of the Effective Bid-Ask Spread in an Efficient Market." Journal of Finance, 39(1984):1127-1139.

Roll, R. "A Mean/Variance Analysis of Tracking Error." Journal of Portfolio Management, 19(1992):13-22.

Ross, R. L. "Financial Consequences of Trading Commodity Futures Contracts." Illinois Agricultural Economics, (1975):27-31.

Schneeweis, T., D. McCarthy and R. Spurgin. "Survivor Bias in Commodity Trading Advisor Performance." Journal of Futures Markets, 16(1996):757-772

Schroeder, T.C., J.L. Parcell, T.L. Kastens and K.C. Dhuyvetter. "Perceptions of Marketing Strategies; Farmers vs. Extension Economists." Journal of Agricultural and Resource Economics, 23(1998):279-293.

Sharpe, W.F., G.J. Alexander and J.V. Bailey. Investments, Sixth Edition. Prentice Hall: Upper Saddle River, New Jersey, 1999.

Shi, Wei, S.H. Irwin, D.L. Good and L.A. Hagedorn. "The Cost of Forward Contracting Corn and Soybeans," Working Paper, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, February 2005.

Smith, L.H. "Can Robots Replace a Marketing Mastermind?" Top Producer, November 2001, pp. 12-13.

Smith, L.H. "The Agony and the Ecstacy: The 2003 Crop Year Delivered Victory and Defeat." Top Producer, November 2004, pp. 14-15, 19-21.

Smith, R.D. "National Assessment of Producer Marketing Alternatives: Practices and Attitudes," Texas Agricultural Extension Service Bulletin, Texas A&M University, 1989.

Snedecor, G.W. and W.G. Cochran. Statistical Methods, Eighth Edition. Iowa State Press: Ames, Iowa, 1989.

Sogn, A.B. and K.J.N. Kraner. "Factors that Determine Where a Farmer Buys and Sells," Bulletin B662, Agricultural Experiment Station, South Dakota State University, 1977.

Stark, B.G., S.M. Cabrini, S.H. Irwin, D.L. Good and J. Martines-Filho. "Portfolios of Agricultural Market Advisory Services: How Much Diversification is Enough?" AgMAS Project Research Report 2003-02, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, April 2003. (http://www.farmdoc.uiuc.edu/agmas/reports/index.html)

Stewart, B. "An Analysis of Speculative Trading in Grain Futures." Technical Bulletin No. 1001, U.S. Department of Agriculture, 1949.

Summers, L.H. "Does the Stock Market Rationally Reflect Fundamental Values?" Journal of Finance, 41(1986):591-601.

Thompson, S., J. S. Eales and D. Seibold. "Comparison of Liquidity Costs Between the Kansas City and Chicago Wheat Futures Contracts." Journal of Agricultural and Resource Economics, 18(1993):185-197.

Tomek, W.G. and H.H. Peterson. "Risk Management in Agricultural Markets: A Review." Journal of Futures Markets, 21(2001):953-985.

Tomek, W.G. and K.L. Robinson. Agricultural Product Prices, Fourth Edition. Cornell University Press: Ithaca, NY, 2003.

Townsend, J.P. and B.W. Brorsen. "Cost of Forward Contracting Hard Red Winter Wheat." Journal of Agricultural and Applied Economics. 32(1995):89-94.

US Department of Agriculture, National Agricultural Statistics Service, Agricultural Statistics Board. "2002 Prices Received Survey: Interviewer's Manual." July 2002.

Williams, J.C. and B.D. Wright. Storage and Commodity Markets. Cambridge University Press: Cambridge, 1991.

Wisner, R.N., E.N. Blue and E.D. Baldwin. "Preharvest Marketing Strategies Increase Net Returns for Corn and Soybean Growers." Review of Agricultural Economics, 20(1998):288-307.

Working, H. "Tests of a Theory Concerning Floor Trading on Commodity Exchanges." Food Research Institute Studies: Supplement, 7(1967):195-239.

Zanini, F.C. Estimation of Farm-Level Corn and Soybean Yield Distributions in Illinois. Unpublished Ph.D. Dissertation, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, 2001.

Zulauf, C.R. and S.H. Irwin. "Market Efficiency and Marketing to Enhance Income of Crop Producers." Review of Agricultural Economics, 20(1998):308-331.

Zulauf, C.R., D.W. Larson, C.K. Alexander and S.H. Irwin. "Performance of Pre-Harvest Pricing Strategies in Ohio Corn Markets: The Role of Cash Flow Risk." Journal of Agricultural and Applied Economics, 33(2001):103-116.

[1] Scott H. Irwin is the Laurence J. Norton Professor of Agricultural Marketing in the Department of Agricultural and Consumer Economics at the University of Illinois at Urbana-Champaign. Darrel L. Good is a Professor in the Department of Agricultural and Consumer Economics at the University of Illinois at Urbana-Champaign. Joao Martines-Filho is a Professor in the Escola Superior de Agricultura Luiz de Queiroz (ESALQ) at the University of São Paulo, Brazil and former Manager of the AgMAS and farmdoc Projects in the Department of Agricultural and Consumer Economics at the University of Illinois at Urbana-Champaign. Lewis A. Hagedorn is a former graduate research assistant with the AgMAS Project in the Department of Agricultural and Consumer Economics at the University of Illinois at Urbana-Champaign. The authors gratefully acknowledge the research assistance of Wei Shi, Silvina Cabrini, and Evelyn Colino; AgMAS graduate research assistants in the Department of Agricultural and Consumer Economics at the University of Illinois at Urbana-Champaign. Invaluable assistance with estimating on-farm storage costs was provided by Kevin Dhuyvetter, Department of Agricultural Economics, Kansas State University, Lowell Hill, Department of Agricultural and Consumer Economics at the University of Illinois at Urbana-Champaign, Marvin Paulsen, Department of Agricultural Engineering at the University of Illinois at Urbana-Champaign, and Dirk Maier, Department of Agricultural and Biological Engineering, Purdue University. Helpful comments on this research report were received from members of the AgMAS Project Review Panel.

[2]King, Lev and Nefstad (1995) examine the corn and soybean recommendations of two market advisory services for a single year. The focus of their study is not pricing performance, but a demonstration of the market accounting program Market Tools. Some analyses also have appeared in the popular farm press. Marten (1984) examines the performance of six advisory services for corn and soybeans over 1981 through 1983. Otte (1986) investigates the performance of three services for corn over the period 1980 through 1984. Both studies indicate the average price generated by services exceeds a benchmark price. Top Producer magazine has provided evaluations of advisory services in corn, soybeans and wheat for a number of years (e.g., Powers, 1993; Smith, 2004).

[3] Throughout this report, the term "crop year" refers to the marketing window for a particular crop. This is done to simplify the presentation and discussion of market advisory service performance results. A "crop year" is more than twelve calendar months in length and includes pre-harvest and post-harvest marketing periods.

[4] Dr. Darrel L. Good and Dr. Scott H. Irwin of the University of Illinois at Urbana-Champaign jointly direct the Project. Correspondence with the AgMAS Project should be directed to: AgMAS Project Manager, 406 Mumford Hall, 1301 West Gregory Drive, University of Illinois at Urbana-Champaign, Urbana, IL 61801; voice: (217)333-2792; fax: (217)333-5538; e-mail: agmas@uiuc.edu. The AgMAS Project also has a website that can be found at the following address: http://www.farmdoc.uiuc.edu/agmas/.

[5] Funding for the AgMAS project is provided by the following organizations: Illinois Council on Food and Agricultural Research; Cooperative State Research, Education, and Extension Service, U.S. Department of Agriculture; Economic Research Service, U.S. Department of Agriculture; Risk Management Agency, U.S. Department of Agriculture; and Initiative for Future Agriculture and Food Systems, U.S. Department of Agriculture.

[6] The University of Illinois Variety Testing program is a well-known example of this type of yield trial. The results of this research program can be found at http://www.cropsci.uiuc.edu/vt/.

[7] For example, Managed Accounts Reports (MAR), a well-known provider of performance information for hedge funds and commodity trading advisors, requires that commodity trading advisors have a 12-month record of trading actual client accounts and a minimum of $500,000 under management to be tracked in their database. More specific details can be found at MAR's website (http://www.marhedge.com).

[8] When the AgMAS study began in 1994, DTN and FarmDayta were separate companies. The two companies merged in 1996.

[9] As shown in Table 1, the AgMAS Project stopped tracking 14 programs at some point over the 1995 - 2003 crop years. Eight programs went out of business or merged with other programs: Ag Profit by Hjort, Agri-Edge (cash only), Agri-Edge (hedge), Cash Grain, Co-Mark, Grain Marketing Plus, Stewart-Peterson Strictly Cash and Zwicker Cycle Letter. Data collection for six additional programs was discontinued because the programs stopped providing specific cash market recommendations or recommendations were no longer deemed applicable to U.S. producers: Ag Alert for Ontario, Agri-Mark, Grain Field Report, Harris Weather/Elliot Advisory, North American Ag and Prosperous Farmer. Excluding these 14 programs from the sample could result in a form of selection bias, particularly if discontinuation is related to poor performance. Including a discontinued program for a crop year does require an assumption about marketing the cash positions remaining after the discontinuation date. A similar issue has been treated extensively in the literature on the performance of commodity funds and commodity trading advisors (e.g., Elton, Gruber and Rentzler, 1987). In this literature, if a commodity fund or trading advisor is discontinued before the end of a calendar year, some form of benchmark returns are substituted for the missing returns after the discontinuation date. Following this logic, the cash positions that remained after the date of discontinuation were sold using the same strategy as the market benchmarks utilized for this study (the details of the construction of these benchmarks are given in the "Benchmark Prices" section). In effect, this simply means that cash bushels after the date of discontinuation are sold in equal amounts over the remaining days of the crop year. Finally, note that any futures or options positions that remain open on the date of discontinuation are closed on that date using settlement futures prices or options premiums.

[10] Some of the programs that are depicted as "cash only" have some futures-related activity, due to the use of hedge-to-arrive contracts, basis contracts and/or options.

[11] It turns out that no program in 2002 or 2003 met this requirement for differentiating on-farm and off-farm strategies. Consequently performance results for on-farm and off-farm storage costs are based on the same set of recommendations.

[12] A detailed explanation of the construction of the marketing profiles and results for individual advisory programs and crop years can be found in Martines-Filho et al. (2003a, 2003b) and Colino et al. (2004a, 2004b). Note that these reports do not contain marketing profiles for the 2002 and 2003 crop years. The AgMAS Project will compute the 2002 and 2003 profiles at a later date.

[13] It is important to emphasize that the marketing profiles in Figure 2 represent the average of all advisory programs across seven crop years (1995-2001). The averages mask substantial variation in marketing profiles across advisory programs for a given crop year and, in some cases, across crop years for the same advisory program. For example, the range (maximum minus minimum) in the net amount priced on an individual day, across all programs and crop years, is 373% for corn and 355% for soybeans.

[14] It is acknowledged that recommendations outside of the two-year marketing window could exceed the flexibility of a representative farmer. For example, it seems unreasonable to assume a representative farmer would hold stocks more than a year after the end of the marketing window. Because there are no hard-and-fast rules for making such decisions, future exceptions will be considered on a case-by-case basis.

[15] The daily spot prices can be found in The Wall Street Journal and at the following website: http://www.ams.usda.gov/mnreports/GX_GR113.txt.

[16] The average forward basis (cash forward prices for fall delivery minus December 2002 corn or November 2002 soybeans futures prices) over February 4-8, 2002 was -$0.2615 per bushel for corn and -$0.2595 per bushel for soybeans. The average forward basis (cash forward prices for fall delivery minus December 2003 corn or November 2003 soybeans futures prices) over February 3-7, 2003 was -$0.2025 per bushel for corn and -$0.2335 per bushel for soybeans. A weekly version of the basis data is published at the following website: http://www.farmdoc.uiuc.edu/marketing/basis/index.asp.

[17] See Shi et al. (2004) for an in-depth discussion of issues related to estimating the trend component in pre-harvest forward basis bids.

[18] Note that estimated pre-harvest forward basis bids for similar periods over 1995-2001 are not widened by the same factor. This is a new procedure introduced for the 2002 and 2003 crops only.

[19] The practical importance of "lumpiness" problems even for small farms may be limited, due to the availability of "mini-contracts" at the Chicago Board of Trade. These futures and options contracts are specified in 1,000-bushel increments.

[20] For a complete description of the programs discussed in this section, see the following Farm Service Agency fact sheets: Nonrecourse Marketing Assistance Loans and Loan Deficiency Payments, March 1998; Feed Grains, March 1998; and Soybeans and Minor Oilseeds, July 1998. These can be found at http://www.fsa.usda.gov/pas/publications/facts/pubfacts.htm.

[21] Technically, the USDA computes LDPs for the current date using PCPs for the previous day.

[22]LDP and MLG data were obtained from the interactive LDP database at the Center for Agricultural and Rural Development (CARD) at the Iowa State University ( http://www.card.iastate.edu/).

[23] The time period for each chart begins on the first day of harvest, as determined for this study, and ends on August 31, 2002 or August 31, 2003. The first day of corn harvest is assumed to be September 19, 2002 for the 2002 crop and September 18, 2003 for the 2003 crop. The first day of soybean harvest is assumed to be September 20, 2002 for the 2002 crop and September 17, 2003 for the 2003 crop.

[24] On-farm shrink charges are not applied to corn sold via a pre-harvest forward contract or harvest spot sale.

[25] On-farm shrink charges are not applied to soybeans sold via a pre-harvest forward contract or harvest spot sale.

[27] Commercial storage costs, as measured by the telephone survey, have not changed over the nine years of the AgMAS study (1995-2003). It appears that commercial elevator storage charges have been stable for a substantial period of time. A 1982 survey of Illinois elevators by Hill, Kunda and Rehtmeyer (1983) revealed an average flat charge for storage of corn and soybeans from harvest through January of 12.9¢ per bushel and 14.2¢ per bushel, respectively. The average monthly storage charge after January was 2.1¢ per bushel for corn and 2.4¢ per bushel for soybeans. The average drying charge for corn was 2.3¢ per bushel. The majority of the surveyed elevators were located in central Illinois. These costs are similar to the costs used by the AgMAS study for the 1995 through 2003 crop years.

[28] The commercial drying charge is not applied to corn that is sold via a pre-harvest forward contract or harvest spot sale. Also, note that on-farm variable costs of storage do not include the cost of drying corn from 15% down to 14% moisture. This charge is assumed to only apply to post-harvest storage at commercial facilities.

[29] The commercial shrink charge is not applied to corn that is sold via a pre-harvest forward contract or harvest spot sale.

[30] Based on estimates reported in USDA December stocks reports, on-farm and off-farm storage averaged 53 and 47% of total storage capacity in Illinois over 1995-2003. There is no discernable trend in the proportions and they vary little from year-to-year.

[31] When cash prices during the June 1 through August 31 period are both below and above CCC loan rates, different procedures are used for computing interest opportunity costs on redemption dates where the cash price is below the loan rate and vice versa. For redemption dates when the cash price is below the relevant CCC loan rate, no interest opportunity cost is charged. This reflects the fact that interest is not charged on CCC loans for redemption days where the cash price is below the loan rate. For redemption dates when the cash price is above the relevant CCC loan rate, the CCC loan must be re-paid with interest. Interest opportunity cost in this case is computing using annual CCC interest rates.

[32] No program in 2002 and 2003 met the requirement for differentiating on-farm and off-farm strategies. Consequently, performance results for all programs under on-farm and off-farm storage costs are based on the same set of recommendations.

[33]Weaker versions of the theory of efficient markets predicts advisory services may profit to the degree they have superior access to information and/or superior analytical ability (e.g., Zulauf and Irwin, 1998). While logically appealing, it is quite difficult, if not impossible, to specify market benchmarks based on weaker versions of the theory because it requires knowledge of the average access to information and analytical ability of market participants.

[34] As with advisory programs, different procedures are used for computing interest opportunity costs on days when the cash price is below the loan rate and vice versa. Refer to footnote 31 for specific details on the computations.

[35] It is typically argued that the drought premium is most pronounced during the spring months before harvest. If this is the case, then the 20-month benchmark price should, on average, exceed the 24-month benchmark price.

[36] If we assume the standard deviation of daily prices is constant over the entire 24-month window, an estimate of the sample size effect can be made. Specifically, the standard error of the sample mean (average) price is , where is the standard deviation of daily prices and T is the sample size. For the 24-month market benchmark, the sample size is about 500 business days, whereas the sample size for the 20-month market benchmark is about 420 business days. Hence, for a given standard deviation of daily prices, , the standard errors will differ by a factor equal to , which implies the variation in the 20-month benchmark should be about nine percent larger than the variation in the 24-month benchmark. This difference is what should be observed over a large number of repeated random samples of prices generated in an efficient market with a constant daily standard deviation. The actual differences in the variation of the two benchmarks over 1995-2003 are larger, 25% for corn, 13% for soybeans and 16% for 50/50 revenue. As noted in the text, one reason for the larger differences is that the assumption of a constant daily standard deviation is not appropriate for corn and soybean prices. Other possible reasons include random effects in the relatively small sample of available crop years and violation of the underlying assumption market efficiency.

[37] The "tracking" strategies terminology is adapted from the finance literature, where "tracking" errors arise as investment managers attempt to replicate the returns of a target benchmark portfolio (e.g., Roll, 1992; Frino and Gallagher, 2001).

[38] The website for the Illinois Agricultural Statistics Service is http://www.agstats.state.il.us/website/welcome.htm.

[39]"Pure" hedging assumes that futures and options markets are efficient and that the only motivation for hedging is to minimize risk (e.g., McNew and Musser, 2002).

[40] The question of bias in futures prices has a long and contentious history in the economics literature. If a bias exists in corn and soybean futures prices, the available evidence suggests the magnitude is small from an economic perspective. This evidence generally is based on long samples of futures prices. Over short sample periods, futures prices can have sharp upward or downward trends. Probably the most dramatic example is the upward trend in grain futures prices between 1972 and 1975. See Zulauf and Irwin (1998) for a thorough discussion and additional references.

[41] The argument here is that selective hedging by farmers, in aggregate, results in trading losses. This does not preclude the possibility that some individual farmers consistently earn trading profits through selective hedging.

[42] The exact wording of the question in the 2001 ARMS survey was, "Did you use farm management services for advice on input or commodity markets?" Neither the enumerator's manual nor training provided a specific definition of a farm management service, so the definition was subject to the respondent's interpretation (McBride, 2005).

[43] One of the study's authors (McBride, 2005) noted that the intention in asking this question was to find out whether or not farm operators sought and purchased professional advice from a service provider about input cost control or commodity marketing. Since market advisory services provide professional advice for fee, it is safe to conclude that the authors intended the definition of a farm management service to include advisory services.

[44] State average LDPs and MLG's for Illinois were collected from on-line Farm Service Agency reports at: http://www.fsa.usda.gov/dafp/psd/reports.htm.

[45] While dated, Paul, Heifner and Helmuth (1976) report survey estimates of forward contract usage that vary sharply across crop years.

[46] Please note that components of average net advisory prices or revenues presented in the text may not exactly equal components implied in Tables 10 through 21 due to rounding.

[47] Terms like "two-year average" are used to refer to averages of net advisory prices over multiple crop years.

[48] A measure of survivorship bias can be computed by subtracting multiple-year averages based only on the programs active in the first crop year of each sample from the overall averages presented in Tables 28 through 30. The differences vary between 0 and -2¢ per bushel for corn, +1 and -4¢ per bushel for soybeans and $0 and -$2 per acre for advisory revenue, with negative numbers indicating survivorship bias ("grand" average less than survivor average). The comparisons suggest survivorship bias is small or negligible in the overall averages in Tables 28 through 30.

[49] For example, one possibility is that advisory programs as a group fail to beat market benchmarks, yet at the same time some programs have "exceptional" performance. Testing whether performance is exceptional for a particular advisory program requires different statistical tests than the ones used here (Marcus, 1990).

[50] The different forms of averaging will produce equal estimates only if a time-series cross-section data set is "balanced." That is, the number of programs is the same for each crop year and there are no missing observations. This clearly is not the case here. It turns out that, after rounding, the two different methods of averaging produce the same estimates of the average proportion.

[51] Given that risk is not considered, this indicator is strictly applicable only to farm decision-makers with risk-neutral preferences. While this may seem unrealistic from a theoretical perspective, several observers suggest that farmers focus mainly on expected returns (e.g., Anderson and Mapp, 1996; Tomek and Peterson, 2001). More directly, Pennings et al. (2004) conduct a large-scale survey of advisory service subscribers in the U.S. and find that producers are more interested in the price-enhancing characteristics of market advisory service recommendations than risk-reducing features.

[52] Differences are calculated as advisory price minus benchmark price. So, a positive difference indicates an advisory price above the benchmark price and vice versa.

[53] See Appendix C for presentation of the statistical model underlying this discussion.

[54] Assume 25 advisory programs are included in each crop year over 1995-2003. Then, a total of 300 pair-wise correlation coefficients would have to be estimated. However, the sample would only contain 225 observations. There simply is not enough information (degrees of freedom) to estimate each correlation independently.

[55] The full set of regression results is available from the authors upon request.

[56] See Cabrini et al. (2004) for a detailed analysis of price and revenue correlations for a similar sub-set of advisory programs.

[57] This test was first proposed by Fama and MacBeth (1973) and it has been widely applied in studies of stock market returns.

[58] In more formal terms, "typical" means one can be 95% confident the true value of the difference will be contained in an interval about two standard errors above and below the average difference estimate.

[59] The standard error of the average difference is estimated as , where is the standard deviation of differences across crop years and T is the sample size (nine in this case).

[60] Differences reported in the text may not equal differences of the averages reported in Tables 35 and 36 due to rounding.

[61] The amount priced by farmers in the pre-harvest period is assumed to be about 18%, near the upper end of the 5% to 20% range suggested by the Coble et al. (1999), Katchova and Miranda (2004) and USDA ARMS (2003) surveys. Readers should note that the marketing profile for farmers is subjectively determined, and therefore, should be viewed cautiously. In the section on farmer benchmark prices, it was noted that almost no concrete evidence exists on the exact length of the typical marketing window of farmers or the precise pattern of forward pricing.

[62] For a given advisory program, the formula for estimating standard deviation is,

where T is the number of crop years in the sample, yt is the advisory program's net price for the tth crop year and is the average net advisory price over the T crop years.

[63] The restriction means that only advisory programs active all nine crop years are included in the average price and risk evaluation. As a result, there is the potential for survivorship bias in the average price and risk comparisons to the benchmarks. Survivorship bias in the average estimates appears to be negligible, with the average corn and soybean net advisory price for the 15 programs one cent more and one cent less, respectively, than the average price computed across all advisory programs active in the 1995-2003 sample period. This suggests that non-surviving advisory programs exited the sample for a variety of reasons, not just poor performance. It is difficult to assess the degree of survivorship bias in advisory program standard deviation estimates with the limited number of crop years available. However, the average comparisons suggest the magnitude of the bias in standard deviation estimates is likely to be small.

[64] Dominance comparisons can also be made between individual advisory programs. To do this, quadrants would be drawn based on the position of the "base" advisory program. Dominance comparisons then follow the same rules as used for benchmark dominance comparisons. It is possible for an individual program to be dominated by a benchmark, yet at the same time dominate other advisory programs.

[65] A joint statistical test of mean-variance equivalence developed by Collender is applied to the average prices and standard deviations of the randomly selected program and the benchmarks. The test results indicate that significance is not found for any case at the five percent level. This result is not surprising given the relatively small sample size available for testing. In addition, Collender's test does not take into account the paired nature of the comparisons, which reduces the power of the test in the present application. A joint test of mean-variance equivalence for paired samples has been developed (Bradley and Blackwood), but it cannot be applied here because a time-series of returns is not available for the randomly selected program.

[66] The tests presented in this section do not consider predictability of risk-adjusted performance measures. The nine-year sample period is not long enough to estimate risk-adjusted performance during sub-periods, which is required for predictability tests.

[67] Fisher's Exact Test is the appropriate statistical test because both row and column totals are pre-determined in the 2 x 2 contingency table formed on the basis of winner and loser counts.

[68] Fisher's Exact Test assumes sample observations are independent. As discussed in the section on average price performance, this clearly is not the case, and therefore, the p-values reported in Table 38 likely overstate the true significance of the results.

[69] A related question is the consistency of performance for a given advisory program across corn and soybeans in the same crop year. In other words, is strong performance in one commodity associated with strong performance in the other commodity and vice versa? Rank correlations of advisory service performance across corn and soybeans are computed for each crop year over 1995-2003. The lowest rank correlation, -0.15, occurs in 2001 and the highest, +0.51, occurs in 1995. The average rank correlation over 1995-2003 across corn and soybean performance is only +0.15, indicating that advisory program performance in one commodity has little relationship with the performance in the other commodity for the same year.

[70] Average differences of the top and bottom groups may not equal the difference of the averages for the groups due to rounding.

[71] It is not likely that the results can be attributed to survivorship bias even though the comparisons are restricted to the 15 programs active in all nine crop years. The average price for the 15 programs in corn over 2000-2003 is only two cents more the average price computed across all advisory programs active over 2000-2003. The average price for the 15 programs in soybeans over 2000-2003 actually is two cents less than the average price computed across all advisory programs active over 2000-2003.