Ask Tim Carter

December 12, 2012

NH Listens, Delphi Technique and Dr. Bruce Mallory

You just never know where a tap on the shoulder might take you, but the last person I ever thought would do that to me was Dr. Bruce Mallory, the head of the Carsey Institute and NH Listens.

I had the pleasure of meeting Dr. Bruce Mallory for the first time on July 31, 2012 at a meeting in Laconia, NH. It was a facilitated meeting that employed the Delphi Technique.

Dr. Mallory and several of his staff members met with me again on a sunny, warm day in August to talk about NH Listens and how conservatives felt about that organization.

At that second meeting I said that NH Listens uses the Delphi Technique in their facilitated meetings and Dr. Mallory denied it.

I picked up the conversation again on the morning of December 9, 2012 when I used the Contact Form at the NH Listens website. Here's what I sent:

I would like a list of all upcoming listening sessions that NH Listens plans to hold. Why isn't that list published on this website? The meetings are open to the public, right? If so, then they should be listed here weeks in advance. Do you get public money to facilitate these meetings? If so, it should be widely publicized here when and where the meetings will be held.

The following morning Dr. Mallory's #1 right-hand woman, Ms. Michele Holt-Shannon, responded very professionally via email. She Cc:d Dr. Mallory in the reply to me. Here's what Michele had to say:

Dear Tim,
Thanks for getting in touch with us. This is Michele Holt-Shannon responding back. I get these contact form messages and you can reach me there or at NH.Listens@unh.edu . All scheduled upcoming events are posted. Our most recent event was a facilitator training in Lancaster, NH just yesterday. Our most recent dialogue event was in Concord on November 7th and was widely publicized in the Concord area, including an ad in the Concord Monitor (150+ people attended). There are a few projects that will very soon have dates, times and places posted for spring months - one being the Granite State Future project. All events will be posted on the GSF calendar as well. We are still in the process of scheduling these. These are dialogue events rather than committee meetings related to the GSF effort and I look forward to seeing you there.

I realize that we have some basic differences in terms of how we would engage people in public life and decision making, but I had hoped you would correct your accusation that we use a Delphi method and psychological manipulation - these are both simply untrue. Still, know that if you attend one of our public events, we will welcome you and make sure your input is given the same attention as any other participant.

All the best,

Michele

I was grateful for the response, but I felt as if she was avoiding the issue as I could not readily locate a calendar at the NH Listens website that clearly showed upcoming events, where they would be held and at what time.

She said they were posted, but posted where?

I replied to her email with the following, making sure to hit Reply to All so Dr. Mallory was in the loop. Here's what I sent:

Michele,
Thanks for the timely reply.

I do have a followup question. Where on the NHListens.org website is the future calendar of events maintained?

I went to this primary navigation page at your site:

http://nhlistens.org/events

No calendar exists that's readily visible. I dug deeper and still couldn't locate one. I would expect your public organization to post upcoming events the moment they've been scheduled.

I'll be attending future sessions for sure to educate the attendees about the real truth of GSF. I just have to know when and where they'll be.

As for the allegations about your listening sessions and the Delphi Technique, I guess we'll have to disagree unless you convince me otherwise.

The format of your meetings is Delphi. That's a plain fact. The core of Delphi is:

Facilitated meetings

Three parts - Welcome / Divide and Conquer / Cliff's Notes

Sitting in circles

Pre-determined facilitator questions

I don't see how you can argue that. Perhaps the reason we're in disagreement is that you may have a different definition of the Delphi Technique. What is your definition? Point me to websites, books, etc that describe the Delphi technique the way you understand it.

Since I've only attended one of your sessions, I can't speak to the facts about the advanced techniques that involve bullying, plants in each of the Divide and Conquer Circle sessions, etc.

I'm more than happy to meet with you and Dr. Mallory again to discuss this. I suggest that we use Rosa Koire's Behind the Green Mask: UN Agenda 21 book as the basis for our discussion about the Delphi Technique.

She's got a fantastic chapter in the book about the methodology. One would think she wrote it after attending the Poverty Meeting in Laconia where I met Dr. Mallory!

I am disappointed that you and Dr. Mallory have not taken my advice with respect to full transparency about all these meetings. It's pretty obvious from the video I shot at the Laconia meeting this past summer that the presence of video cameras does not stifle the conversation in the Divide-and-Conquer part of the meetings.

I assume you've seen that video on YouTube.

This is where is started to get interesting. At this point, Michele stepped aside - or was told to stand down - and Dr. Mallory jumped into the fray.

He replied to me with this email:

Greetings Tim,
I just want to respond to your question about funding, in your earlier message. To be clear, we receive a mix of funds from public and private sources. These include state and federal agencies that contract with us. Currently, we receive funds from the Regional Planning Commissions (flow-through dollars from the US Dept. of Housing and Urban Development, as you know) to foster public engagement in the Granite State Future project; the US Department of Commerce (via NOAA and the National Estuary Reserve Research program), in the form of a grant to UNH and the town of Exeter to foster public engagement related to long-range climate adaptation planning; the Pittsfield Youth Workshop (flow through funds from the Nellie Mae Education Foundation) to support public engagement for school improvement in the town of Pittsfield; and the NH Charitable Foundation. A small portion of our core operating expenses are also supported by the private endowment that funds the Carsey Institute at UNH, where we are organizationally located.

Michele will continue to help you be aware of events that are posted on our web site. All events that we sponsor that are open to the public are posted on the web site as soon as we know the dates and locations. Sometimes that is weeks in advance, sometimes we operate with a shorter timeline, such as the recent work with did with the city of Concord. GSF-related events that we are involved in will be posted both on our website and the GSF website, as they are scheduled.

I agree that we will likely continue to disagree about the various processes we use for fostering deliberation. The surveys that participants complete at the end of each process or event indicate high degrees of satisfaction, a belief that a wide range of views has been expressed, and a belief that the process is useful for informing decision-makers. When participants make suggestions about ways to improve the process, we make modifications consistent with our principles of engagement.

All the best,

Bruce

Bruce L. Mallory
Professor of Education
Director, The Carsey Institute
Huddleston Hall
University of New Hampshire
Durham, NH 03824
603.862.2821
http://www.carseyinstitute.unh.edu/

Director, NH Listens
www.nhlistens.org

So I couldn't let Dr. Mallory off the hook, right? You wouldn't accept that reply without a rebuttal, correct? I thought so.

Here's what I send back to Dr. Mallory:

Bruce,
You pretty much dodged the questions. Perhaps Michele will answer me directly. I realize you're a busy man and may not have time for this discussion.

I asked for you / NH Listens for your definition of the Delphi Technique. It's important as we foster our relationship moving forward that I know how you and NH Listens define that process.

As for the participant surveys, that's got me grinning. Why? Just about any lamb that attends a facilitated meeting has no idea they've been Delphied.

A vast majority of the poor souls that attend your facilitated meetings have never heard of the Delphi Technique. You and I both know that. In fact, that should be a question on the evaluation form:

Have you ever heard of the Delphi Technique? Yes No

Are you aware of that old saying? A half truth is a whole lie.

I maintain that you're not being transparent with your listening-session participants. How about you provide them with a simple half-page handout as they walk-in and register. By the way, that getting the name thing at the door is a very powerful psychological trigger - it's called Commitment.

I challenge you to provide the attendees with a simple half-page description of the Delphi Technique and then see how the evaluations come back.

I found your last sentence the most interesting of all.

Can you please define for me your principles of engagement?

Yes, you can clearly see the shields being raised. Yes, I'm being provocative, but after all, NH Listens is a publicly funded group. If you decide to accept that money, then there are consequences. One of the consequences is the public deserves clear answers to questions.

Dr. Mallory, as you can see, is an expert at dodging questions. But that makes sense, at least to me. When you have something to hide, you dodge questions, you ignore them or you hope the person asking the questions just gets frustrated.

Can you anticipate Dr. Mallory's response to my last email? I know I did. I was positive he was going to not answer my questions. What a shame. It could have been done in a private forum, but now he forced it to go public.

Here's what Dr. Mallory shot back to me. You can tell he made the decision that he's not going to play nice.

Tim, Respectfully, I won't comment on the Delphi technique since we don't use it, never have, and have no plans to. I understand that you have a different perception, but that's what makes the world go 'round. The principles that NH Listens operates from are found at http://nhlistens.org/how-it-works. --Bruce

By now you pretty much can predict my response in this escalating Q and A session. Here's what I send back to Dr. Mallory:

Bruce,
I really appreciate your candor.

I'm not a betting man, but I actually bet in my mind before you replied that you would dodge the question again and send me to that page.

Bravo! So there at your NH Listens website you have the Delphi Technique buried in amongst lots of other verbiage.

Here it is in case you forgot or didn't approve the copy at your website.

By the way, I've already got a screen shot of the page and all the HTML coding in case you go back and change it. Just an FYI.

NH LISTENS.jpg

Once again, the key structure of Delphi, in case you're not aware of it, is:

Facilitated discussions

Break out into small groups

Meet again at the end where all information is compiled and sanitized

That's clearly what you say at your site, albeit in more clever language.

I think it's best that we continue this debate at a future time in a different arena.

Have a very Merry Christmas. I'm sure our paths will cross multiple times in 2013.

So there you have the facts.

You tell me. You're the Court of Public Opinion.

Does NH Listens use the Delphi Technique when it facilitates a meeting out in public?

Of course it does. The sad thing is Dr. Mallory doesn't have the courage to admit it.

Why do you think that's the case? Please feel free to comment below.

Posted by Tim Carter at 7:39 AM | Comments (3)


		Ask Tim Carter « October 2012 \| Main \| January 2013 » December 12, 2012 NH Listens, Delphi Technique and Dr. Bruce Mallory You just never know where a tap on the shoulder might take you, but the last person I ever thought would do that to me was Dr. Bruce Mallory, the head of the Carsey Institute and NH Listens. I had the pleasure of meeting Dr. Bruce Mallory for the first time on July 31, 2012 at a meeting in Laconia, NH. It was a facilitated meeting that employed the Delphi Technique. Dr. Mallory and several of his staff members met with me again on a sunny, warm day in August to talk about NH Listens and how conservatives felt about that organization. At that second meeting I said that NH Listens uses the Delphi Technique in their facilitated meetings and Dr. Mallory denied it. I picked up the conversation again on the morning of December 9, 2012 when I used the Contact Form at the NH Listens website. Here's what I sent: I would like a list of all upcoming listening sessions that NH Listens plans to hold. Why isn't that list published on this website? The meetings are open to the public, right? If so, then they should be listed here weeks in advance. Do you get public money to facilitate these meetings? If so, it should be widely publicized here when and where the meetings will be held. The following morning Dr. Mallory's #1 right-hand woman, Ms. Michele Holt-Shannon, responded very professionally via email. She Cc:d Dr. Mallory in the reply to me. Here's what Michele had to say: Dear Tim, Thanks for getting in touch with us. This is Michele Holt-Shannon responding back. I get these contact form messages and you can reach me there or at NH.Listens@unh.edu . All scheduled upcoming events are posted. Our most recent event was a facilitator training in Lancaster, NH just yesterday. Our most recent dialogue event was in Concord on November 7th and was widely publicized in the Concord area, including an ad in the Concord Monitor (150+ people attended). There are a few projects that will very soon have dates, times and places posted for spring months - one being the Granite State Future project. All events will be posted on the GSF calendar as well. We are still in the process of scheduling these. These are dialogue events rather than committee meetings related to the GSF effort and I look forward to seeing you there. I realize that we have some basic differences in terms of how we would engage people in public life and decision making, but I had hoped you would correct your accusation that we use a Delphi method and psychological manipulation - these are both simply untrue. Still, know that if you attend one of our public events, we will welcome you and make sure your input is given the same attention as any other participant. All the best, Michele I was grateful for the response, but I felt as if she was avoiding the issue as I could not readily locate a calendar at the NH Listens website that clearly showed upcoming events, where they would be held and at what time. She said they were posted, but posted where? I replied to her email with the following, making sure to hit Reply to All so Dr. Mallory was in the loop. Here's what I sent: Michele, Thanks for the timely reply. I do have a followup question. Where on the NHListens.org website is the future calendar of events maintained? I went to this primary navigation page at your site: http://nhlistens.org/events No calendar exists that's readily visible. I dug deeper and still couldn't locate one. I would expect your public organization to post upcoming events the moment they've been scheduled. I'll be attending future sessions for sure to educate the attendees about the real truth of GSF. I just have to know when and where they'll be. As for the allegations about your listening sessions and the Delphi Technique, I guess we'll have to disagree unless you convince me otherwise. The format of your meetings is Delphi. That's a plain fact. The core of Delphi is: Facilitated meetings Three parts - Welcome / Divide and Conquer / Cliff's Notes Sitting in circles Pre-determined facilitator questions I don't see how you can argue that. Perhaps the reason we're in disagreement is that you may have a different definition of the Delphi Technique. What is your definition? Point me to websites, books, etc that describe the Delphi technique the way you understand it. Since I've only attended one of your sessions, I can't speak to the facts about the advanced techniques that involve bullying, plants in each of the Divide and Conquer Circle sessions, etc. I'm more than happy to meet with you and Dr. Mallory again to discuss this. I suggest that we use Rosa Koire's Behind the Green Mask: UN Agenda 21 book as the basis for our discussion about the Delphi Technique. She's got a fantastic chapter in the book about the methodology. One would think she wrote it after attending the Poverty Meeting in Laconia where I met Dr. Mallory! I am disappointed that you and Dr. Mallory have not taken my advice with respect to full transparency about all these meetings. It's pretty obvious from the video I shot at the Laconia meeting this past summer that the presence of video cameras does not stifle the conversation in the Divide-and-Conquer part of the meetings. I assume you've seen that video on YouTube. This is where is started to get interesting. At this point, Michele stepped aside - or was told to stand down - and Dr. Mallory jumped into the fray. He replied to me with this email: Greetings Tim, I just want to respond to your question about funding, in your earlier message. To be clear, we receive a mix of funds from public and private sources. These include state and federal agencies that contract with us. Currently, we receive funds from the Regional Planning Commissions (flow-through dollars from the US Dept. of Housing and Urban Development, as you know) to foster public engagement in the Granite State Future project; the US Department of Commerce (via NOAA and the National Estuary Reserve Research program), in the form of a grant to UNH and the town of Exeter to foster public engagement related to long-range climate adaptation planning; the Pittsfield Youth Workshop (flow through funds from the Nellie Mae Education Foundation) to support public engagement for school improvement in the town of Pittsfield; and the NH Charitable Foundation. A small portion of our core operating expenses are also supported by the private endowment that funds the Carsey Institute at UNH, where we are organizationally located. Michele will continue to help you be aware of events that are posted on our web site. All events that we sponsor that are open to the public are posted on the web site as soon as we know the dates and locations. Sometimes that is weeks in advance, sometimes we operate with a shorter timeline, such as the recent work with did with the city of Concord. GSF-related events that we are involved in will be posted both on our website and the GSF website, as they are scheduled. I agree that we will likely continue to disagree about the various processes we use for fostering deliberation. The surveys that participants complete at the end of each process or event indicate high degrees of satisfaction, a belief that a wide range of views has been expressed, and a belief that the process is useful for informing decision-makers. When participants make suggestions about ways to improve the process, we make modifications consistent with our principles of engagement. All the best, Bruce Bruce L. Mallory Professor of Education Director, The Carsey Institute Huddleston Hall University of New Hampshire Durham, NH 03824 603.862.2821 http://www.carseyinstitute.unh.edu/ Director, NH Listens www.nhlistens.org So I couldn't let Dr. Mallory off the hook, right? You wouldn't accept that reply without a rebuttal, correct? I thought so. Here's what I send back to Dr. Mallory: Bruce, You pretty much dodged the questions. Perhaps Michele will answer me directly. I realize you're a busy man and may not have time for this discussion. I asked for you / NH Listens for your definition of the Delphi Technique. It's important as we foster our relationship moving forward that I know how you and NH Listens define that process. As for the participant surveys, that's got me grinning. Why? Just about any lamb that attends a facilitated meeting has no idea they've been Delphied. A vast majority of the poor souls that attend your facilitated meetings have never heard of the Delphi Technique. You and I both know that. In fact, that should be a question on the evaluation form: Have you ever heard of the Delphi Technique? Yes No Are you aware of that old saying? A half truth is a whole lie. I maintain that you're not being transparent with your listening-session participants. How about you provide them with a simple half-page handout as they walk-in and register. By the way, that getting the name thing at the door is a very powerful psychological trigger - it's called Commitment. I challenge you to provide the attendees with a simple half-page description of the Delphi Technique and then see how the evaluations come back. I found your last sentence the most interesting of all. Can you please define for me your principles of engagement? Yes, you can clearly see the shields being raised. Yes, I'm being provocative, but after all, NH Listens is a publicly funded group. If you decide to accept that money, then there are consequences. One of the consequences is the public deserves clear answers to questions. Dr. Mallory, as you can see, is an expert at dodging questions. But that makes sense, at least to me. When you have something to hide, you dodge questions, you ignore them or you hope the person asking the questions just gets frustrated. Can you anticipate Dr. Mallory's response to my last email? I know I did. I was positive he was going to not answer my questions. What a shame. It could have been done in a private forum, but now he forced it to go public. Here's what Dr. Mallory shot back to me. You can tell he made the decision that he's not going to play nice. Tim, Respectfully, I won't comment on the Delphi technique since we don't use it, never have, and have no plans to. I understand that you have a different perception, but that's what makes the world go 'round. The principles that NH Listens operates from are found at http://nhlistens.org/how-it-works. --Bruce By now you pretty much can predict my response in this escalating Q and A session. Here's what I send back to Dr. Mallory: Bruce, I really appreciate your candor. I'm not a betting man, but I actually bet in my mind before you replied that you would dodge the question again and send me to that page. Bravo! So there at your NH Listens website you have the Delphi Technique buried in amongst lots of other verbiage. Here it is in case you forgot or didn't approve the copy at your website. By the way, I've already got a screen shot of the page and all the HTML coding in case you go back and change it. Just an FYI. Once again, the key structure of Delphi, in case you're not aware of it, is: Facilitated discussions Break out into small groups Meet again at the end where all information is compiled and sanitized That's clearly what you say at your site, albeit in more clever language. I think it's best that we continue this debate at a future time in a different arena. Have a very Merry Christmas. I'm sure our paths will cross multiple times in 2013. So there you have the facts. You tell me. You're the Court of Public Opinion. Does NH Listens use the Delphi Technique when it facilitates a meeting out in public? Of course it does. The sad thing is Dr. Mallory doesn't have the courage to admit it. Why do you think that's the case? Please feel free to comment below. Posted by Tim Carter at 7:39 AM \| Comments (3)