Using A Peer Review Panel To Enhance Service Agreements

Are any of your current solid waste service provider contract(s) for collection, materials recovery, transfer or landfill services ending in the next several years? Will your city or county soon be contracting for nonsolid waste services such as wastewater or electricity? Will you shift to a new service provider or renegotiate with your existing contractor?

Before entering into a new agreement, you have an opportunity to step back and examine the effectiveness of your existing service contract. A peer review panel is a useful tool for broadening the range of industry skill and experience reflected in your next contract, enabling you to create a new or revised agreement that retains effective contract provisions, removes ineffective language, and includes new provisions to enhance future performance. This paper discusses how to use a peer review panel to strengthen your next service contract, drawing on recent experience developing a new solid waste facility operations contract. The paper concludes with broadly applicable observations about the process and how it can best be applied.

The City of Sunnyvale, California (City) recently used a peer review process as the existing operating agreement for the Sunnyvale Materials Recovery and Transfer Station (SMaRT Station) neared expiration. The City, and the cities of Mountain View and Palo Alto, which use the 1,500 ton-per-day materials recovery facility through agreement with Sunnyvale, wished to review the current operating agreement. The City determined that an outside perspective on the contract would be beneficial. On behalf of the City, Brown, Vence & Associates (BVA) assembled a peer review panel of nine solid waste experts, representing a variety of expertise, perspective, and experience to review and discuss the agreement. Participants included public agency representatives, private refuse haulers and facility operators, consultants, and attorneys.

The goal was to create an environment in which participants could speak candidly about the existing operating agreement. BVA developed a full-day agenda of activities for the panel, culminating with facilitation of a workshop in which over 40 issues were discussed. In order to accurately reflect the discussion during the workshop, and recognizing that in many areas there are no "right" answers, BVA's report to the City identified the degree to which there were differing opinions on each issue. The report material is being used by Sunnyvale in developing the new operating agreement.

The SMaRT Station
The City owns the SMaRT Station, a materials recovery facility and transfer station located in Sunnyvale. Municipal solid waste and source-separated green waste generated in the cities of Mountain View, Palo Alto, and Sunnyvale are processed and transferred at the SMaRT Station through a Memorandum of Understanding (MOU) among the cities. In addition, Mountain View and Sunnyvale deliver source-separated green waste collected through curbside collection programs. Waste Management, Inc. operates the facility under a seven-year operating agreement executed with Sunnyvale in February 1993 that expires on December 31, 2000. In anticipation of executing a new operating agreement, Sunnyvale decided to examine the effectiveness of the existing operating agreement and identify improvements.

Peer Review Goals and Objectives
With the end of the current contract term approaching, the three participating cities had the opportunity to modify the agreement to enhance its effectiveness. The updated agreement will be used if the City chooses to continue with its current operator or to select a new operator through a competitive procurement process. To provide Sunnyvale with a contract best suited to achieving the goals of the SMaRT Station, the three cities decided to conduct a peer review by convening a panel of solid waste professionals, considered to be experts in the field, to review and discuss the existing operating agreement. The charge of the peer review panel was to make recommendations regarding which areas of the current agreement should remain as is, be deleted, or be modified. BVA was selected by the cities to facilitate the peer review.

To initiate the process, staff from the three cities were asked to identify the goals for future operation of the facility; these goals are shown in Table 1. While the results may seem obvious, the goals provide the panel members context for their work. In addition, the process of identifying goals is particularly valuable when the contracted service is shared by two or more public agencies as in the case of the SMaRT Station. Developing a concise statement of goals requires discussion among public agency representatives that can serve as a positive means for returning to first principles. The discussion allows staff to step back and to revisit the original purpose of the contracted service and how it was to be provided.

Table 1 Facility Goals

The primary objective of the peer review process was to examine the existing operating agreement and outline areas of improvement so that the new agreement will better serve the needs of the three cities. Table 2 provides additional specific objectives identified by the City, which served as key guidance in our development and management of the peer review process. Note that panel members were provided with recent financial and operational data to assist in understanding the services the contractor must provide at the facility. However, the panel was not requested to comment on, or to critique, the manner in which the current contractor operates the facility. In reviewing Table 2, note also the emphasis on identifying aspects of the current contract that work well, rather than focusing on what provisions could be improved.

Table 2 City Peer Review Objectives

The Panel Members
In selecting the members, we sought to ensure a broad range of both expertise and perspective. Our goal was to have a panel with 8 to 10 members. We found that nine provided a good balance between the conflicting goals of creating a panel with as broad a base of expertise as possible, and one that was small enough to allow for relatively unstructured, informal, and open exchange of ideas. We began with several individuals identified by the staff of the three cities, and added names of our own, developing a list of about 15 individuals. Table 3 lists the key areas of expertise of the nine panel members selected for the peer review. Given the relatively small number of participants, we were also interested in identifying individuals with multiple relevant areas of expertise, such as an attorney with financial expertise or a private-sector facility operator with facility design experience.

Table 3 Panel Member Expertise

Facilitation Team
Our facilitation team consisted of three BVA staff, all experienced facilitators, to allow a range of expertise mirroring that of the panel members. While one team member presented a set of related issues, the other two team members were able to make notes regarding ties to upcoming issues and to "seed" the panel's discussion with related ideas and questions with the intent of encouraging dialogue and reaching a recommendation.

Peer Reviewer Responsibilities
The panel's charge was to review relevant materials provided in advance of the workshop and to attend a full-day of activities at the SMaRT Station.

Review of Relevant Materials
BVA developed a binder that included an overview of the peer review process; brief biographies of panel members and the facilitation team; the existing operating agreement with attachments and all subsequent amendments; background material such as the MOU among the three cities defining the roles of each in the development, financing and operation of the SMaRT Station, and recent facility financial and operational data; an annotated list of the provisions of the agreement; and key issues for consideration.

We recommend the inclusion of biographies in the package of relevant materials because it gave panel members the opportunity to "meet" each other before the workshop. The annotated list of contract provisions provided a means for tracking how such provisions had been added, modified, or deleted through the contract amendments entered into by Sunnyvale and Waste Management subsequent to the signing of the original contract. By including annotated comments, a reader could readily understand how the contract evolved over time. While it can be tedious to develop this type of annotated list, it is valuable in assisting both reviewers and facilitators to quickly find pertinent provisions in a complex contract. The facilitation team developed the key issues list as a means of guiding the panel to decisions and recommendations in each area of the contract. The list of key issues referenced related provisions that appear in various areas of the contract in order to ensure that issues were dealt with efficiently and in their entirety.

Peer Review Panel Meeting
The Panel members met for a full day at the SMaRT Station. The program included a tour of the facility, an orientation and lunch with the staff from the three cities, and a four-hour workshop discussion of the issues. City staff did not participate in the workshop so that panel members would feel no constraints in speaking. The four-hour workshop was recorded on audio tape. The tape proved valuable for ensuring that the opinions and positions of panel members were portrayed accurately in the summary report to the cities.

The Workshop
As noted above, we used three facilitators for the workshop. The key issues list included with the binder served as the primary agenda for the workshop. We began the workshop with a reiteration of the goals and objectives for the peer review process (Tables 1 and 2), and discussion of the workshop guidelines (Table 4). The panel was given the opportunity to comment on, or to add to, the guidelines developed by the facilitators, but there were no substantive changes.

Table 4 Workshop Guidelines

The facilitators took turns presenting a set of related issues to the panel. Issues discussed in the process included recycling and marketing of recovered materials, yard waste diversion, operator compensation, transfer operations, performance objectives and guarantees, and a range of more detailed legal questions related to default, force majeure, and assignment of the contract. As facilitators, our role was to engender targeted discussion. This task was relatively easy in that the panel was quick to engage in fruitful discussion on all of the issues. Each facilitator prepared a number of questions to pose on each issue in case discussion did not ensue. In practice, we did not have to rely on much of this material.

Among the most interesting aspects of the conversation were the areas in which the opinions of public-sector and private-sector representatives differed. Not surprisingly, public agency representatives were often concerned with providing for a greater degree of oversight through the contract than private-sector operators thought necessary or desirable.

The following observations are based on our experience in managing the peer review process, in facilitating the workshop, and participating in conversations during and after the process with city staff and workshop participants.
1. The peer review process resulted in useful guidance to Sunnyvale regarding the structure for the next agreement, and approaches to drafting language for key areas such as compensation, operator performance incentives, and the specifics of breach and default. It is important to clearly define workshop goals and objectives prior to embarking on the process.
2. The panel should consist of 8 to 10 members to provide for a balanced range of expertise and a group size that is conducive to relatively informal discussion.
3. The peer review process can be applied to any complex contracting process, whether in the solid waste, wastewater, and energy fields, or in cable televison franchising or medical services contracting. In each instance, the same general principles will shape a successful process.
4. Start early. In planning your procurement time line add two to four months for the peer review at the front end of the schedule.
5. The individuals managing the peer review process should have a broad technical grounding and significant experience relevant to the contract under review. In addition, the individuals managing the workshop should have prior facilitation experience.
6. Assembling the pool of participants proved relatively simple because all parties contacted as potential participants agreed to participate. They felt it was an honor to be asked to serve as a peer reviewer and looked forward to benefitting from participating in the discussion. The payment of an honorarium, in this case $500, provided an important symbolic indication of the value to the cities of their participation. In addition to actively participating on the day of the workshop, the panelists prepared well, having reviewed the binder of materials supplied them in advance. The total time commitment for the participants was roughly in the range of 16-24 hours, depending on degree of preparation and travel time.
7. Provide background materials well in advance of the workshop and make sure that peer reviewers recognize their obligation to throughly review them.
8. Do not include sponsoring public agency staff in the workshop. As discussed above, the facilitators discouraged this, and in our view the exclusion of staff from the workshop contributed to the willingness of the panel members to speak freely. Staff from the three cities had prior acquaintance with many of the panel members. The desire to participate may stem from wanting to be available to "defend" the current contracting process and/or to not miss out on an unusual professional opportunity to discuss these issues in this type of setting.
9. Do not attempt to reach consensus on all items. With the limited time available, facilitators should focus on getting the opinions of the members and understanding the reasons for those opinions. Facilitators should work to engage all participants in the dialogue. 10. Each peer reviewer should receive a copy of any summary report.
—Peter M. Deibler

   previous paper     back to top    back to media center

Top | Services | Case Studies | Media Center | Jobs | Client Area | Contact