A Retrospective of SISTA's Game Design Workshop

As I mentioned last month, the University of Arizona’s new School of Information: Science, Technology, and Arts (SISTA) has been organizing a Game Design Workshop for kids. The workshop was a one-week summer program that aimed to teach kids about game design and development, and I was lucky enough to have been hired as a teaching assistant for it. There was one other TA, as well as three instructors, which meant we were a team of five teachers in total.

Our biggest challenge was the fact that this was a brand new workshop being run for the very first time, so we essentially had to design it from the ground up. The bulk of the work was done by the instructors, while we TAs were given specific tasks to learn Stencyl, take notes on our experience, and design some lessons for it. During the several weeks before the workshop started, we were mostly working remotely, using online tools like Google Docs to collaborate. During the final week of preparation, we all met up to bring everything together and to "test" the design of the workshop so far by running through a mock version of it. The workshop then ran for two sessions, where each session lasted for one school-week (five days), and then end with a big playtesting event, where all of the parents and relatives could come in to play the students' games.

So was the workshop a success? Many of the people involved—from the kids, to their parents, to the rest of the SISTA faculty—were blown away by how amazing they thought the workshop was. The positive feedback was pretty consistent across both sessions, and it’s been immensely satisfying to read through the anonymous surveys that we’ve been collecting.

In this post, I’ll point out the various aspects of the workshop that I thought made it work so well, while also pointing out the various problems that we failed to avoid. Even though I found the typical postmortem format to be pretty useful when writing this, this is more accurately described as a retrospective. I'm not really in a position to be able to speak for the entire team, and since part of the goal of this post was to critique my own performance as a TA, I've added a few self-evaluation sections as well.


What Went Right


#1: Smart Planning and Adaptability

Even though having two sessions meant that we had the opportunity to improve the quality of the second week based on the mistakes of the first, we never saw that as a fallback for a potentially low-quality workshop. We were personally motivated to create a great experience for these kids, regardless of which session they were in, and the week of preparation and testing before the first session was a huge help towards achieving this goal. We were able to find and fix dozens of small but significant problems that simply could not have been foreseen without actively testing the workshop’s design.

Once the workshop started, we continued to tweak its design by ending each day with a debriefing session, in which we would analyze how the day went, how to improve it for next time, and what we should change for the following day. These sessions also allowed us to exchange feedback on each other as well as share our concerns on how certain kids were doing. This system allowed us to be surprisingly flexible so that we could adapt quickly to problems as they came up.

For example, during the first week, we found that long lessons weren’t working so well, so we switched to a more lightning-fast, straight-to-the-point approach. This resulted in the creation of most of the slideshows on the Resources page of the workshop’s website (which hasn't been made public yet). The students could then access this information at any time, and we would often point them to these slides whenever they had a question that was related to them.

The new slideshows were designed to be short and simple.

The daily debriefs were especially useful during the second session, because they allowed us to adapt the workshop to a significantly different group of kids. The first session was a much tamer and more balanced group, and at least a third of the students were girls. But the second session was made up entirely of boys, who were generally older, more energetic, and more chaotic overall. And for some reason, this group was advancing through the content at an alarmingly faster rate than the previous week.

We knew we had to make a lot of changes to adapt the workshop to this new group of kids. But this was a good thing, because it meant that the second session ended up being much more tailored to the kids’ specific needs than the first session. During the first session, our main concern was about trying to fit the content that we thought was important for the kids to learn, while the second session prioritized the topics that the kids were getting the most out of. This allowed us to drop several segments of the workshop that simply weren’t working very well, and the result was a workshop that focused more on practical skills in the end.

#2: The Kids Learned a Ton!

The kids were able to learn an incredible amount of content within just five days. They learned how to make games using Stencyl, how to draw their own animations in an image editor called Pixen, how to create their own sound effects using Audacity and a microphone, and how to create their own music using GarageBand. We never got around to teaching them how to use cfxr (the Mac port of sfxr), but there were some kids who found the program on their computers and ended up teaching themselves how to use it. Most of the kids also learned how to use Stencyl’s built-in (but deeply flawed) image-editor called Pencyl for the sake of its text tool, which is a feature that Pixen lacks.

There was something magical about making your own sound effects.

The fact that they learned so much within a short amount of time probably helped the kids feel like the workshop was an immensely valuable experience. We were worried that they would fnd the pacing of the workshop to be overwhelming, but most of them actually found it too slow, which was what forced us to increase the pace. There are several possible reasons for why this worked so well, most of which will be hinted at throughout the rest of this article, but one of the most important factors was the fact that the workshop was very hands-on. Letting the kids experiment with new concepts was usually more effective and time-efficient than talking about them.

The workshop was pretty expensive ($250 per student), and while this was actually cheaper than most other summer programs, we were worried that parents might not feel like they were getting their money’s worth. This was partly why the kids received an entire bag full of supplies and SISTA merchandise, including a reusable water bottle, a 4GB flash drive, a game design notebook, a workshop t-shirt, and much more. But because the workshop was so intrinsically valuable on its own, I feel like the merchandise will instead do a pretty good job of reminding the kids of the experiences that they had here, and hopefully it means they’ll be less likely to forget the things that they learned.

What these kids accomplished during the workshop was nothing short of amazing. To be able to come up with a game idea and build a digital prototype with decent animations, sound effects, and music in just a few days is something that only the best game designers can do.

#3: Stencyl is a Great Tool

Of all the tools and resources that we used in this workshop, the game-making tool was perhaps the most important decision that we had to make. We needed a tool that would allow the kids to make a game in less than five days with minimal problems, while also being easy enough to use that the kids would feel encouraged to keep making games once the workshop had ended. Some of the biggest problems of SISTA’s new game design class last semester were directly caused by the decision to use GameMaker. At any one time, there were conflicts with up to four different versions of the program: PC Lite (v8.1), Mac Lite (v7.0), PC Pro (v8.1), and Mac Pro (v7.0). The gap in features between the Mac and PC versions, combined with the gap between the Lite and Pro versions, were infuriating.

Stencyl, on the other hand, avoids that problem beautifully. Unlike GameMaker, Stencyl seemed to have been designed to work first on Mac, and then it received one of the most flawless ports to PC that I’ve ever seen. The developers even transferred over subtle effects from the Mac interface, such as the ability to click-and-drag the entire window around by grabbing any piece of the window, not just the top bar. And rather than creating a disparity in features between the Pro and Free versions, Stencyl Pro simply offers the ability to publish your final game to certain platforms, such as iOS.

“Of course you can download Stencyl at home. And it’s exactly the same!”

The consistency of all of its versions is just one example of Stencyl’s amazing dedication to a great user experience. The developers have clearly put a lot of thought not just into the program’s interface, but also into the intuitiveness of its basic features and core systems. With other tools, especially with GameMaker, you often get a sense that the features and systems were too heavily influenced by engineering concerns rather than with the user’s perspective. Stencyl’s design feels like it prioritized the user’s perspective above all else, and while it must have been incredibly challenging to develop, the result feels much more natural and inviting.

I could write pages and pages about Stencyl’s design, in order to explain exactly why it’s so well done, but that’s outside the scope of this article. There are still quite a few problems with it, however. But because the developers seem to genuinely care about their users, I’m confident that Stencyl will only continue to improve as new versions of the program are released.

It was amazing to watch the kids use Stencyl throughout the week. While we tried to help the kids as they got stuck on problems, many of them found the joy of what it’s like to persevere through a tough glitch all on their own. We even had to force some of them to take breaks, or else they would have kept working all day, even skipping meals if they wanted to. They were becoming typical programmers without even realizing it.

#4: Frequent Breaks and Outdoor Play

Some of us were worried from the very beginning that the kids would be spending an unhealthy amount of time in front of a computer at this workshop, so we made sure to plan several non-computer activities into the schedule, including board-game playing, brainstorming exercises, snack time (not to be confused with lunch time), and even outdoor play. As we continued to adapt the workshop, we started to add spontaneous optional breaks, such as a random outdoor walk, or more outdoor play time. Some of the parents were actually concerned about outdoor play, because it meant we were taking their kids out into the deadly Arizona heat in the middle of July. Fortunately, the kids all had their own water bottle, and we always went to a space that provided plenty of shade.

Breaking up the workshop into several events also helped us maintain an engaging sense of pacing. Towards the beginning of a session, we would notice that there were some segments that just felt like a long grind because there weren’t enough changes within a couple hours. However, towards the end of a session, we found that the kids would actually start to prefer the long grinds, because they wanted to get as much work done on their games as possible before presenting them on Friday.

When this happened, we decided to give them their work time, but we also provided more optional breaks. Usually about half the class would decide to take these breaks, and we would even try to convince some of the more hard-working students to see why it was important to take breaks. Most people have a terrible sense of how to stay productive, so it was nice that we were able to provide a good example for how to get a lot of work done in a healthy way.

I personally thought that both the indoor and outdoor play times were the most fun parts of the workshop, not just because they allowed the kids to socialize with each other, but because they were often about getting them to create non-digital games. It’s a ton of fun to mash-up a few games together and to be able to see what happens almost immediately, without having to program it. In other words, they weren’t just playing games; they were playing with game designs.

Diagonal Chess was a popular favorite.

#5: A Five-Person Staff

It might have seemed excessive to have had three instructors and two TAs for a group of 20 students, but it was necessary considering the amount of things that we were doing during this workshop. Most of the time, it felt like we were really a team of four, because Kelland Thomas’ lessons on sound effects and music were basically guest lectures since he wasn’t very involved with designing and running the workshop (he still contributed enough to be worth counting, however). Meanwhile the other two instructors, Jane Strohm and Derek Green, contributed the valuable skills and experience that really made this workshop work in the first place.

It’s hard to assess the value of the TAs from within a TA position, but we were able to take a lot of the workload and stress off of the instructors, which probably made the scope of the workshop much more manageable. It was easier to organize outdoor activities when you had more people to regulate the kids, and it also meant that at any time we had at least two people going around the room answering questions, even if we were in the middle of a lesson.

It kept the lessons running smoothly because there were less class-wide interruptions. If some kids fell behind, we could help them catch up without impeding on everyone else’s progress. If some kids were ahead, usually stuck on a problem that required a more advanced understanding of Stencyl, we were able to teach them advanced concepts, many of which were never formally covered in a lesson. Because some kids ended up learning more than others, we encouraged the kids to ask each other for help, and we awarded the more helpful students with "badges", represented by stickers on their nametags that advertised the skills that they had mastered.

Derek (not a TA) explains a glitch to a student. (Photo by the Daily Wildcat)

The TAs were more than just a helping hand, however. The position was pretty demanding, because we were asked to do a lot to help plan the workshop. Both of us TAs maintained large Google Docs of notes about specific topics and concerns about the workshop, and we also developed many of the resources on the workshop website. I personally enjoyed the workload, because it made me feel like we were an actual part of the team, rather than servants or subordinates.


Self-Evaluation: What I Did Right


#1: Avoiding Some of my Weaknesses

Before the workshop started, one of my major concerns was that I wouldn’t be a very good public speaker. In the past, I’ve struggled with not speaking loud enough, and sometimes making the conscious effort to speak louder will make you self-conscious enough to make you feel awkward. Fortunately, I had already built up resistance against the typical fears associated with public speaking by giving plenty of presentations at my club during the past school year. Unfortunately, I never really fixed my habit of not speaking loud enough, since I would often be so focused on what I was trying to say that I would forget to keep my voice up.

So it was a huge help to have the instructors giving me the feedback that I needed whenever I spoke to the class. Our solution was to have someone stand on the other side of the room and make gestures whenever they had trouble hearing me. It also helped that running the workshop demanded a certain level of energy output from me, so I was more likely to feel natural putting in the extra energy needed to speak loudly. I still think my speaking could have been a lot better, but I’m putting this under the “what went right” category because I see it as an overall improvement.

Another major weakness that I was worried about was the fact that I’m generally really bad at explaining things. That’s partly why I put so much effort into this blog, because if I don’t, then everything will come out disorganized and confusing. One of the tasks that the TAs had was to give at least one lesson on any topic, and while I had many interesting game design topics that I wanted to talk about, my attempts to turn them into clear, coherent lessons that were less than 10 minutes long failed terribly. What went right in this case was the fact that I avoided this risk by sticking to much more practical topics, such as how to do specific things with Stencyl.

Many of our lessons ended up dissolving into slideshows for
the Resources page, especially during the second session.

#2: Honing my Debugging Skills

While the idea of going around the room and answering questions might not sound very demanding, it’s actually a very hectic job. These kids were programming a game using a tool that was completely new to them, and because they were allowed to go at their own pace, they were running into all sorts of weird glitches and problems. Sometimes they made a simple misstep and made a trivial mistake, but other times they were playing with multiple features that they had no introduction to and had managed to turn their game into a tangled mess of glitches.

After just the first day of this, I was surprised by how quickly and efficiently I had become at debugging these games. Once you see a few common mistakes, you tend to develop a subconscious glitch-hunting routine that helps you find the problem as fast as possible, and then the rest of the time is spent trying to make sure the student understands what was wrong and how to fix it. The amount of glitches that I solved in a typical session, combined with the speed with which they were solved, sometimes made me feel like I was at some kind of debugging olympics.

As the workshop went on, I made an effort to improve my teaching skills while debugging. I would try to ask the student more questions, such as “do you know how to do this?” or “do you know what this means?” Something that may have helped was the fact that I truly thought many of the things I was teaching them were interesting, and I would sometimes go on tangents about how I thought a certain feature in Stencyl was cool or lame. Some kids really did find the enthusiasm of my explanations entertaining, and there was even a short-lived running joke about one of my solutions that involved the use of trigonometry.

#3: Becoming the Stencyl Expert

While Stencyl has many built-in features to help you learn how to use the program and to help you ask for help, there aren’t that many resources out there at the time of this writing that will help you learn the intricacies of its deeper features. Because of this, a truly deep understanding of Stencyl is pretty hard to come by, since you mostly have to figure things out on your own. I was probably the one person on our team who got the farthest in understanding Stencyl (although Derek did pass me for a while).

It may have been my love for Stencyl that drove me to keep learning about the platform, but my respect for the developers also made me want to contribute more to the Stencyl community. Before the workshop began, I was submitting bug reports and posting feedback on some of the starter guides. During the workshop, I took some of my more complicated solutions to student problems and turned them into behaviors, which I then posted onto StencylForge as well the workshop’s Resources page. My behaviors became more polished and less glitchy as I made more of them, and some of the jokes embedded into their design bring back good memories about how much fun the workshop was.

Apparently chickens really can fly.

I started to notice that the more confusing problems were frequently forwarded to me, and even some of the students started requesting my help specifically. For better or worse, if a kid asked for help, I typically wouldn’t leave their side until we fixed their problem or until I got interrupted by something. There was only one time when I told a kid that I couldn’t figure out their problem, and that was partly motivated by the fact that he would rather talk to his friends rather than participate in the debugging process.


What Went Wrong


#1: Did we Handle Debugging Wrong?

This isn’t really something that went definitively wrong, but it’s a question that still worries me. As I mentioned previously, when the kids ran into a problem, we would point out the problem right away and explain it to them. There were even times when the problem was so advanced that we knew they couldn’t get things to work on their own, so we would often fix things ourselves, sometimes writing new behaviors for them to use.

There are probably teachers out there who would cringe at the idea that we did some of the work for them. The computer science department at the University of Arizona strictly trains its TAs to never solve any of the student’s problems for them. Instead, they are merely there to help student ask the right questions in order to guide them through the debugging process. I personally think that’s pretty cool, because not only does it prevent the TAs from accidentally doing the students’ homework for them, it’s also meant to teach people how to be a better debugger.

Unfortunately, there are a lot of problems with this approach. First, it’s very time-consuming, and we just didn’t have the time for such a roundabout way of solving problems. Second, this process can be pretty demotivating to someone with no programming experience. Most of the time, students just want to know what they’re doing wrong, and when they can’t get direct feedback on that, they get frustrated. If we had used the more indirect approach, it’s likely that we would have left these kids with a bad first taste of programming. Their experience would have been largely defined by the ugly problems that they got stuck at, rather than the triumphant pride of having built something as complicated as a game.

However, I’m still worried about this for some reason. Did some kids become so reliant on us fixing things for them that they won’t have the confidence to keep making games outside the classroom? Did some kids feel like we helped them so much that they don’t feel like it’s their game anymore? Maybe I’m just being too cynical, because it’s probably the exposure to lots of problems and their solutions that helps a programmer develop their debugging skills. We even took the time to have a few lessons about troubleshooting and debugging strategies.

From the slideshow on troubleshooting.

I also know that many kids were actually pretty engaged while I found their glitches for them. They were usually genuinely curious about what was wrong with their game, and it gave them a chance to see how messy the debugging process really was. Whenever I had to implement a more complex solution for them, they would watch me code in front of them, and some kids found my tendency to think out loud while working to be hilarious. I know that some of them actually did learn something from the complex solutions, because I would later see some of these kids trying to implement something similar into their friend’s game.

#2: Some Kids Didn’t Get Enough Help

Now this is a real problem. Because there was such a high demand for help, we TAs naturally tended to help only the students who asked for it. There were some students who were struggling but were just too shy, or too focused on something else, to ask for help. Some of us were careful enough to take the time to try to find and help these kids, but for the most part, the rest of us were too caught up in the debugging spree to do the same. We almost didn’t realize this problem existed if it wasn’t for some of the parent feedback that we got during the first session.

I still feel really bad about this mistake, because it wasn’t until the end of the second session that I started to fully understand the problem, despite the fact that we had discussed the topic earlier at one of our daily debriefing sessions. That’s why I wrote more about this under the “What I did Wrong” section.

The rest of the team, however, did a much better job reacting to this problem. Jane, especially, seemed to be the most alert when it came to finding kids who were secretly struggling. She would even email their parents early in the week in order to figure out how to deal with some of the kids. Some of them just weren’t as interested in the workshop as others, and there were also a few kids who needed a boost in confidence. Many of the parents were very impressed by how much effort we were putting into their kid.

#3: The Side-Effects of Adaptability

While the daily debriefing sessions were part of what made the workshop so successful, we actually made so many changes with the second session that we started to lose track of things. Even though we took care to write down all the modifications that we were making in advance, we never really wrote down the modifications that we made on the fly. We often needed to change things in the middle of the day, because the second session was progressing faster than we expected. We started to lose track of what exactly we had taught the kids, and we couldn’t rely much on memory because of how easy it is to confuse memories of one session with another.

We had a general idea of where they were with the content, but when it came to specifics (such as whether or not we taught them about collision groups), things started to become unclear. Although by the time we noticed these unintended gaps in the curriculum, it was unclear whether any action should be taken, because not only had many of the students learned the material from the TAs, others had also learned it from the slides on the workshop’s Resources page.

While this confusion over what exactly had been taught didn’t lead to much trouble in the end, it’s still a pretty embarrassing problem. I’m sure the instructors were more on top of this than I was, and maybe I just have bad memory. But if we had known we would be making so many changes to the workshop during a single day, we probably would have done a better job of documenting what we had actually accomplished, not just what we planned to accomplish.

#4: Not Enough on Design?

One of my major concerns during the weeks before the workshop was that there was going to be too much about how to use Stencyl and not enough about actual game design. We actually had quite a few design-oriented activities, but I sense that many of the kids didn’t really make the connection between that content and their own game project. This makes me think that they probably didn’t take much away from these segments.

There was one segment where we tried to make the kids fill out a small design doc template about their games before officially starting to make them, but there was quite a bit of resistance because most of them wanted to just play around with Stencyl and see what was possible before making design decisions. The irony is that this was actually a really smart decision on the kids’ part, because it’s a perfect example of the iterative development process followed well.

The kids wanted to experiment with Stencyl first, because they inherently knew that there were a lot of risks with not knowing a tool’s limits. They knew that fleshing out an idea too early was probably a waste of time if they were forced to change key aspects of it later. This is actually the exact same process that I followed when making my own sample Stencyl game. The only difference was that I simply had a few more iterations, and I had even saved copies of some of those iterations. We could have made a potentially great design lesson out of these events and resources, but we just didn’t have the time.

Do you recognize some of those sprites?

We were starting to think that the kids were so preoccupied with learning how to make a game that there wasn’t much room left for learning how to design a game. You can’t really understand how to design a game until you know how to make one, so perhaps the low emphasis on design was for the best. There’s been much talk about organizing a more advanced version of this workshop in the future, and perhaps that session could be completely design oriented. Design is a tricky subject to teach because it can be hard to keep everything grounded in practicality. Fortunately there are a lot of potentially great lessons we could hold that link back to the students’ experiences in this workshop in order to teach them more about core design skills.

#5: The Loose Ends

Despite its great success, the workshop’s design could still use quite a bit of polish. For instance, despite being really cool, the avatar cards were never really used again after the first day. Many kids also made the mistake of overestimating the scope of their games, so we should have made it clearer that they were only making a prototype of a game. And while we tried to improve this during the second session, we still feel like we didn’t emphasize the importance of playtesting early enough. The list of problems goes on, but it’ll probably take several more iterations to stamp out all of these smaller problems.

Some avatar cards

Also, while a lot of people are excited about the success of this workshop and what it means for the future of SISTA, I think it’s worth remembering that SISTA still doesn’t have an experienced game designer on staff. It would be unwise to get too excited about the future of SISTA’s game program until that basic problem has been fixed, but I also hope the great success of this workshop will be a huge help in recruiting such a person. Perhaps this person is already reading this very blog post?


Self Evaluation: What I Did Wrong


#1: There Were Some Kids that I Didn’t Help

As I mentioned earlier, I really regret how I failed to notice that some of the kids were silently struggling with their games. A lot of times if a kid gave up on a problem and needed help, they would often start working on a different part of their game. This was very deceptive, because they would appear to be very productive despite the fact that they were frustrated. And as the student became more absorbed into what they were doing, they were less likely to remember to ask for help when one of us passed by.

The only way to even notice the problem would have been to start a conversation with those kids by asking them how their game was going. Some of us adapted well to this solution, but I really failed to do that. Part of the reason may have been because I was usually on high demand to fix things, but I was also remarkably impulsive about which student I would go to help out next. I almost always went to the first student who I saw was asking for help, and while I learned to prioritize students based on how rarely they ask for help, I was still only giving help exclusively to those who asked for it.

On the positive side, I feel like I got to know some of the kids who frequently asked for help really well, and there would even be times when they would raise their hands just because they wanted to show us a cool thing that they made or a tough problem that they solved. But I really should’ve noticed that something was wrong when I realized that I barely knew anything about the quieter kids and the games that they were making.

#2: The Need for More Data

There were several times during the workshop when I wished I had maintained more data on how I was doing. The first metric I wish I had was how much time I was spending per problem, because I was worried about the impact of my “don’t leave until it’s fixed” style of answering questions. Next, I found myself wishing I had maintained a list of which glitches I was fixing the most, because I really wanted to write some kind of FAQ guide not just for the workshop’s website, but for the Stencyl community as well. And now, in hindsight, I wish I had kept a metric on which students I was helping the most, or more importantly, which ones I weren't helping enough.

I always assumed that maintaining such metrics would have been impractical (or just plain weird), but it really would have required nothing more than a simple notepad and a few extra seconds per question. I may have been more aware of the mistakes that I was making, and the simple act of slowing down to accurately measure my performance would have also helped me to stop being so impulsive about which kids I chose to help.


Conclusion

This workshop was technically my first real job, and even though it helped put every single one of my side projects on hold for a while (except for this blog, apparently), it was one of the most valuable experiences I've ever dedicated myself to.