Oh we are a bunch of high school kids at heart. Maybe high school never ends (and there’s a nightmare, god I hated high school). But, there’s been drama about the 2014 PASS Summit sessions and the Selection Committee’s work. I was on the committee. I worked as a part of the team responsible for rating sessions for the Azure track (said track is gone, more on that later). As self-serving a statement as this is, I think we did a good job. Further, I think the process worked. You can read the official explanation of the process here. Amy did great work and deserves your thanks. All the volunteers who reviewed over 900 submissions from more than 300 people, ON THEIR OWN TIME, FOR FREE, also deserve your thanks. The vitriol directed at the PASS organization over the outcome of this selection process is not directed only at the Board. It’s also directed at the volunteers. And, as a volunteer, that sucks.
The team I worked on rated, I forget, 50 sessions I think. We had to read through them and give them a score based on several criteria. We also had to write comments on each and every session. I was dinged by HQ for not writing a comment on a session that I gave 5’s to on the ratings (so I commented something like “Can’t wait to see this at Summit”). We were only given 10 slots to fill, so that means 40 sessions got kicked to the curb. That’s a lot of people who didn’t get selected. And not getting selected sucks (yes, I do know, I’ve been rejected by a number of events this year, big ones, even ones I’ve spoken at previously, not whining, just pointing out that I don’t have a secret method for getting accepted). Our track actually got eliminated and the sessions that we selected were distributed to other tracks. Also, a couple of sessions we rated highly didn’t do so well when the speaker scores were applied, so there was some shift there (one thing PASS could improve, give us some indication of the secret sauce there, we know there is one, but a little understanding of how it’s applied would help). But over all, the sessions we rated highly, got selected. Congratulations and well done to those speakers. Just look at the people presenting, many for the first time. That’s going to be an absolutely awesome event. And once more, thank the volunteers for doing all that work.
So, some of you are now thinking that, “Oh, Grant’s on the side of PASS” (well, actually, yes, I am, so should you be) “Grant has been told to be nice and play good and not be critical” (even though I’ve already made a criticism about the magic numbers and I was tweeting almost literally threatening messages this week) “Grant got selected so he’s being a <insert bad name here> about the whole thing” (I may or may not be a <insert bad name here> but I don’t agree that I’m being one about this) or, maybe you’re on the other side “OMG! He’s criticizing PASS in any regard, The HUMANITY! Have you at long last sir no decency” (no, not really).
Remember those comments, that I had to write for every abstract, including the great ones? I put a small critical review of the abstract in every one (OK, not the one that I gave 5s to). I said what was wrong with the abstract in my subjective opinion. And let’s be perfectly clear about that (channeling President Obama), they’re my opinions. If I thought you didn’t define the problem space your presentation was meant to address, I said that. You disagree? OK. If I thought your very clever and witty title seriously detracted from the clarity of what the session was about, and it wasn’t that witty, I said that too. You’re the wittiest person you know and everyone says so? OK. My opinion may not jive with yours. But, it’s the one thing I’ve seen everyone who has ever been rejected by the committee ask for, “Tell me what I can do to improve.” OK. I did. At least in my opinion. On every single abstract (except that one).
PASS didn’t release them.
The volunteers (unpaid, remember) did the work, and now it gets to see the light of day.
This brings up a number of points. First, when I got rejected by those other events, did I get a reason for my rejection? Nope. Other events just reject you, thanks for playing. I think PASS, which is all about community, should be different. We should tell people why, not just that there were higher rated sessions, but what they can do to improve. I’ve talked to people in the know, not all the comments provide that kind of information. I think we’ll get better next time. Second, peoples feelings are going to be hurt by these comments. Yes. Yes they are. Suck it up buttercup. You want to know what you can do to improve so you can get selected, but your abstract is absolute perfection (in your opinion), so how dare someone else suggest that it’s not worthy of inclusion, blah, blah, blah, We’re going to see lots of blog posts where people disagree with these comments and that could reflect back in some negative way on the organization. I suppose so, but if we’re going to be about community and we’re going to try to raise up new speakers, we’re going to have to be able to deal with some degree of friction. That may even come from experienced people irked that they didn’t get picked. Everyone has a bad day. Again, I think we can weather this. Finally, the different teams and individuals on the teams probably gave substantially different levels of comments with varying degrees of quality. Some of the comments are just going to be useless. Further, My opinion probably doesn’t jive with my teammates in every regard. Maybe a team didn’t put critical comments in at all (although they had to put in comments). Yes, these things are going to be uneven, maybe even contradictory. OK. Again, cope.
This blog post once started off as a rebuke of the selection process around those comments. It’s not now. I want to repeat, one more time, I think the committee did great work and selected an awesome set of presentations that will make for a wonderful Summit. Thank you for all your hard work. And thank you, Amy, for doing a great job organizing what is a daunting task. And thanks for releasing the comments.
Whenever there is a selection committee there are going to be those that are happy to be selected and those that are unhappy that they were not selected. It’s the way that each group/individual handles the selection or rejection that really defines the character of that group/individual. Being mature takes a lot of practice, but ultimately one feels better to be either a gracious winner or loser. It sounds like a tough job doing these evaluations, many are doing it voluntarily for the good of the SQL community. Thanks to everyone who makes this engine run.
Two things I wanna make sure we note, too:
First, the quality of the comments are REALLY GOOD given that the committee members weren’t told in advance that the comments would be shared with presenters. They’re open, honest, and constructive. I’m so freakin’ happy at how good they are under these circumstances.
Second, there’s been complaints lately that the community has been too negative or too outspoken. This loud, vibrant dialog produced real results – we’ve got change in the way PASS works. Dialog is a great thing!
Sandi, all the people working on the selection committee were volunteers. Whether they were working for the good of the community or not is a different discussion.
Brent, more dialog is always good. 100% agreement. And, for a community group, open, public dialogue. Which is going to be, at times, messy. Tough.
Grant,
Good post, but I wish you had left out the high school part – maturity level or not, it’s hard for me to blame anyone for getting frustated with the lack of feedback and lack of transparency. It’s not trivial.
On Brent’s comment, was there no plan for the feedback? If not, why capture it?
I’m also curious about your reply to Sandi, what does “Whether they were working for the good of the community or not is a different discussion.” mean?
Being part of PASS does require tough love sometimes, and I applaud you pushing for the comments. I hope you’ll push a little more towards that goal you mention of having PASS/Summit be different than other events.
Thanks for your efforts as a volunteer.
Andy
I like that you mention other conferences. Sometimes it’s good to see how it is done in other places to put things in perspective. I know of another conference that chose to do something different. Absolutely no rejection letters at all. There might be folks out there that still think they are in the running.
The grass is not always greener on the other side which is why I love this community and pour my heart into it wherever I can. (Yes I’m a bleeding heart kind of guy.)
Andy,
On the high school thing, I think some of us went a little Soc/Greaser during the initial discussion, but honest people can disagree.
There was no written plan to release the comments. But there was a lot of implication that was the plan. In theory the comments were just supposed to be used internally, but there were other communications that pretty much said they were going out to people. So it was poorly communicated internally. That lead to my assumption (and we all know what those do) that they were going out, which lead to me lobbying pretty intensely for that. I think it was 100% the correct thing to do. I applaud PASS & Amy for changing what was in the defined process w/o a great deal of muss or fuss.
My reply to Sandi was just a poor attempt at humor (what I think is funny and what is funny are frequently not part of an INNER JOIN). Nothing was meant by it but pointing out that everyone involved was a volunteer, not just some. Sorry for the stupidity on that one.
And I’m going to push on us being different. I’ve already volunteered. Whether I get picked or not, well, I’ll still make noise.
Thanks for the feedback Andy.
Ryan,
I’ve had to bug some conferences. Um, I submitted to you guys, what the heck? Still haven’t heard back from some of them. That’s even worse than just getting rejected for no reason at all.
Could you release the various criteria on which the sessions were scored? I’d like to know what PASS considers to be a good presentation.
All this angst brings to mind a quote from one of the greatest British prime ministers, Benjamin Disraeli. A friend asked him how he could stand all the public criticism directed his way. His reply? “Never complain, never explain.” 🙂
Hey David. I’m not sure if we can release all the scoring criteria. I’m not sure we should. It’s applied in a very subjective manner. But I suppose that’s something to explore.