Ok, chewing on the list from last week, I’m not sure I’ve got a final 5, but there are definitely a few standouts as good candidates, so I want to pick one of those and drill into it a bit. The first one on my mind is one that is implicit in a few of the items on the list, but was not called out as its own thing (but probably should have been) and that’s the issue of player engagement. Phrased as a question, it would probably be “How engaged was every player at the table?”
Now, practically, what you’re really asking is “How engaged was the least engaged player?” but that sounds kind of negative phrased that way, so I’d just keep that in mind.
Anyway, none of this is very useful if we can’t make it measurable, but this is thankfully made a little bit easier by the use of a compressed scale. So, we need to decide what we mean by engagement, and what a 0, 1 or 2 means. I’m starting with this one because I think it’s probably one of the simplest ones to measure, since I think it’s probably an 80/20 split.
The largest part is participation: did the players participate in the game? While what exactly participation might entail can vary from game to game, it’s pretty easy to suss out. Look at the activities engaged in by the players at large (talking in scenes, sharing ideas, taking action in combat) and use that as the checklist for each player. As a baseline, it will be pretty easy to judge the level of participation.
Now, there’s a catch to this: it is easy to equate participation with extraversion, and we all know quiet players who are less likely to step up and participate, but that’s what they want, and that’s ok, right? Well….no. That’s the easy out. It is far to easy to attribute someone’s lack of participation to their disinterest or introversion than it is to try to figure out what’s going on and try to draw them in. I don’t want to go off on a full-fledged tangent here, since the act of drawing out reticent players is a nuanced and involved one, but the short form is that there are so many possible ways that your game is discouraging engagement (speed of play, extroverted or “overly-helpful” players, high pressure) that you can’t take a pass on this metric just because someone “is like that”.
Yes, at some point, if you’ve tried everything and really wrestled with the issue, you can write it off, but it’s pretty much on your conscience to determine when that is.
Anyway, the other element of engagement is the ephemeral moment of cool. If everyone participated, that’s well and good, but did everyone get to do something cool? Does everyone have a moment that they can take away as their moment to shine? This is, admittedly, somewhat subjective, but I don’t think it’s too hard to measure.
So, with that in mind, I figure the engagement element looks something like this:
How engaged was every player at the table?
0 – Long stretches without participation.
1 – Everyone participated
2 – Everyone participated and had a moment of cool
This one is also pretty easily flipped from the GM to the individual player, if that’s the goal. I’m not sure it is, but I’m filing that possibility in my back pocket.
Anyway, I think this is a pretty good example of the idea. The question is slightly fuzzy, but the compressed scale makes it easy to answer with minimal wiggle room. So I ask, does this make the implementation of the model any clearer?
I’d have a hard time with this one. I regularly have a few new players that are just being introduced mixed in with more advanced players. Getting the new players to step up is hard because they’re often more than a little intimidated.
Also I have had players that disengage because the other players aren’t listening to their plans. That’s not the GM’s fault.
Also sometimes a player has told me that they really enjoyed a game when I thought they were disengaged especially new players.
I’d be more comfortable with it if each player gave a score for themselves.
I think this is a good start. I hate to make my comment about somebody else’ comment, especially since Emmett was brave and thoughtful enough to come out with his thoughts first, but I think what he says adds to the discussion, while his objections don’t actually invalidate what I think Rob is trying to do with the metric.
As I understand it, Rob is trying to develop a score for game masters to rate themselves and seek self-improvement. Obviously, you won’t be using this if you think you are already God’s gift to RPGs. For the rest of us, the idea is to have a score that lets you focus on improvments, maybe incremental improvements, in a thoughtful and measurable way.
Emmett definitely raises issues that must be kept in mind as one moves from development of the scoring tool to the implementation. In several situations, Emmett objects because he may get a 1 on the scale proposed for engagement by Rob, when A. it should have been a 2 or B. because it was beyond the control of the GM to get a 2 because of the actions/personalities of others. While the objections are thoughtful, I reject them as invalidating Rob’s approach. I think it is worth using the scale to make one stop and think. If you got a 1 or a zero because there was or seemed to be a lack of engagment, there should be some motivation to investigate into why? Now, the new player phenomenon may well always drag down the score. However, given the leadership role of the GM, shouldn’t we know that we tend to score lower with new players at the table and at least think about how things can be run differently to make new players engaged. I mean, this is not a grade that goes on a permanent transcript, this is a tool for reflection that shoudl, I think as Rob conceives it, help us hone our skills to make games more enjoyable for more people more often. As for the “hidden” engagement issue, that is something I think we should also be curious about. Table top gaming is a quirky human interaction, and we have to get to know the personalities invovled. If this is a one time encounter at a Con, we may not be able to pick up on the nuances, but if this is an ongoing game (and I think this scoring is going to be something to try to apply to all situations), it is worth engaging in some discussion motivated by the scoring.
I think if a score of 1 motivates a GM to talk to a seemingly un or low engaged player and you get feedback like Emmett talks about, then, you can readjust the score. Then, not only do you find out you were doing better than you thought, but you also learn more about the people sitting at your table.
That makes you a better GM and that, in the end, is what I think Rob is trying to build for us. A little engine to help us be better.
Anyway, that’s what I think, and I would not have been able to articulate all this without Emmett getting out in front and raising concerns, so thanks Emmett for really adding to the discussion.
The question we’re trying to answer with these metrics is, “Am I improving as a GM?”. Unfortunately, we can’t answer that question directly. Everything a GM does is in relation to the other players in the group. The closest we can get is to try and answer the question, “Am I improving as a GM, for this group?“. All of our metrics have to be geared towards that second question. It is only when you get an answer to that second question that you can try and extrapolate an answer to the first.
This is particularly rough if, as in Emmett’s case, you have new players rotating in and out of your group on a regular basis. It can also be rough if one or more of your players has a social disability (ADD, Asperger’s, etc). All you can do is try your best. Keep trying different tactics to engage new players. Learn how to spot subtle signs of engagement, for those players who appear to be quiet or distracted. They might actually be participating and having fun, just in a more introverted way.
In all these cases, your “Engagement” score is going to bounce around “0” and “1”, through no fault of your own. That’s just the way it’s going to be. That doesn’t mean that the metric is worthless, though; it just means that progress is going to be slow. As you improve your GM skills over time, you might start seeing more “1”s than “0”s, and may even catch a glimpse of the rare “2”.
(The reverse is true, as well. If you are blessed with a group of fantastic, extroverted players, it’s going to be easy to get “2”s week after week. It doesn’t mean that the metric is worthless, just that you have to watch for slow trends.)
It’s important to remember that the scores themselves don’t matter much. What we care about is how the scores change, over time. Did you try something new, and your metrics shot up? Keep doing it! Did you try something new, and your metrics plummeted? Stop doing it! Trying to compare scores of two different groups of players, let alone of two different GMs, is meaningless. There are hundreds of things that could bring scores down that are not the GM’s fault. All you can do is to try and identify the things you can improve, and work towards improving them.
I think the way you ask the question and the answers are good. Now who is answering the question? Is it the GM as a self-assessment, player(s) as an assessment of the GM, or all participants as a measure of the group dynamic? I think this engagement category in particular would be useful for the latter. I think whichever you choose it will and should shape the questions you ask and the answers available. I don’t think that in most traditional RPGs the GM can answer this one the way it is phrased because they have too many things going on to be an impartial observer. It’s too easy – especially for a beginning GM working to improve – to overlook the one quit player that isn’t engage while everyone else is having fun.
@Evan – Thanks for taking up those thoughts. Just to clarify though, I’m not looking to poh poh the idea or the metric. My comment was more pointing toward looking to the individual problems that a group has instead of as a whole. I know that looking at the whole make this a quicker and therefore more easily used metric. I’m saying that for me, it would be the most useful to know who in the group is trending low (not doing a good job with new players or a specific player) instead of a blanket measurement.