QMED Computes

End of Session Feedback

15 June 2018

I asked participants to fill out a brief survey about the workshop series, and in this post I’m going to discuss this feedback. At the time writing this post, about 12 participants have responded to the survey and those results are included here.

NB: I asked that people who didn’t attend many (or all) of the workshops also fill out the feedback because I specifically wanted to hear from them.

What did participants learn?

I did not do any objective assessment of skills we worked on in the series, although earlier I discussed how I might do that. In the end, I decided that a subjective assessment was simpler to organize and to run at the end of the school year over the busy exam period. In addition, I suspect the participants’ attitudes towards their learning experience is an important factor in how they will approach and apply computing in future research situations – i.e. if they enjoyed this experience, and are more confident in using the tools we covered they will be more likely to pick them up in the future (but, of course, [citation needed]).

In my early musings about the intent of the workshop series, I pitched the overall series like so:

The gist of my workshop pitch is to equip future physicians with the computing literacy they’ll need to be researchers and research “consumers”. […] My goal here is to introduce students to the tools and ideas of the trade, so that when they need them later on in their careers they know what is possible and how to learn them: I’d be so pleased at the end of the year if students could load up a dataset into Rstudio, and make a plot with a subset of the data on two variables with a linear fit line, while knowing and being confident enough to read through documentation to extend their work.

So, how do respondents rate their own capability with the workshop skills after having taken the workshops:

A few observations:

• all respondents say either they believe they have learned various skills or are least unsure (‘neutral’) about it. No one disagrees about having learned something. So… go team!
• What I believe is the most important skill, that of knowing how to learn, is endorsed almost all respondents.
• The least endorsed statement of skill is about using R to solve problems. I’m not surprised about this because while these workshops gave people an introduction to the various skills, there wasn’t nearly enough practice time to become generally proficient with R.

Was the series worth it?

I wanted a quick “thumbs up, thumbs down” on each session in the series to know if they were perceived as worth taking, and thus, worth repeating. Here are the responses:

What you can clearly see from this is the attendance variance (and attrition) over the year. I’ll get into exploring why this happened later on.

The mini-session on statistics was clearly the least liked (and attended) session. It was also the shortest session (30 minutes), and one where participants got the least practice (i.e. none, unless they wanted to try during the Capstone project, which no one did) and also required a background in statistics because all I taught where the R mechanics (and, very few people that attended had that background). If I were to run this session again I would expand it to a full session and run it as workshop on statistics using R rather than require prerequisite stats know-how.

How well were the workshops run?

Happily for me, respondents found the learning environment positive and my instruction useful. Interestingly, while most people found the amount of time per workshop was sufficient, there is a strong interest in having more frequent workshops. This comes out in the comments below.

Strengths and Weaknesses

I invited free-form comments about the series and respondents really gave me some great feedback. I’ve organized the feedback into themes:

Instruction

positive:

• “Casual environment, great online resource, great educator”
• “Focus on problem solving.”
• “Major strength in having someone who is knowledgeable and a great teacher, but maybe underutilized for the first few sessions.”
• “The friendly atmosphere and the approachable and knowledgeable instructor!”
• “Very knowledgeable teacher, easy to follow”
• “Very knowledgable instructor.”
• “Very patient instructor; like the “go at your own pace” approach”
• “Jon is an awesome instructor. 10/10 would recommend.”
• “Also Jon is a great instructor, and he provided enough autonomy while jumping in when necessary”

Overall organization

postive:

• “broken down into easily digestible pieces; very applicable to science and the type of analyses people do”
• “Series was fun and engaging. Content was well-delivered and made fun with activities and tasks.”
• “Online resources were provided, which can be sought out at a later date as needed.”
• “The resources that Jon has compiled makes it easy for me to go back and refer to something that I remember learning, but not exactly how to do (e.g. a small command like VLOOKUP has been extremely helpful!).”
• “I found working through exercises in pairs extremely helpful in terms of me learning problem solving skills, and in particular, where/how to look for answers online, through various ‘help’ commands, etc.”

negative:

• “Nothing, maybe homework to be taken up? Because I have no self control and I didn’t practice :(“

Lesson structure

positive:

• “I think some of the best workshops were the ones where different online resources were combined to create the full session (e.g. Data wrangling).”
• “A brief intro with the major points/functions, etc. being done before starting the exercises were helpful, as were the summing up portions at the end, where you went through solutions. Ideally, I think most/all of the sessions should be structured that way, or with you spending more time “lecturing” and having us work along (e.g. ggplot session) (but also giving us exercises at the end of the session to make sure we can apply what we’ve learned).”
• “I found the “walk-throughs” at the beginning of the sessions, where Jon demonstrated what could be possible with x skills/program/medium we were learning, valuable. It was useful for me to know what I could potentially do with, e.g. ggplot or kniter, even if I didn’t necessarily learn how to do said more advanced thing during the workshop time.”
• “The way the sessions were set up (big ideas, questions to consider, several resources to learn the same thing, practice questions, etc.) made perfect sense!”
• “I really liked that workshops were task-oriented (which is how I like to learn a new skill). I also appreciated the flexible nature of the workshop series, so that I could learn what was relevant for me”
• “Moved forward at a very good pace according to student’s abilities and needs.

negative:

• “More structured approach in the first few session than freestyle approach.”
• “Didn’t think that the statistics session was worthwhile with the structure it had. I know it’s not your area of expertise, but unless there is something to practice, or some practical examples/exercises that we can go through, I don’t feel like there’s much point in having the session - you may as well just point us to the resources and let us figure it out outside of the session (and use that time to have extra sessions on the programming/data wrangling/plotting parts).”
• “Enjoyed the session where you “lectured” (ggplot) and had us working along but feel like I didn’t retain much since I didn’t have to work out much/anything on my own - a few individual exercises would help make that session better.”
• “Thought the sessions where we were just working through online resources (that we had no chance of finishing) could be made better by structuring them more like the data wrangling session.”
• “use of science data for the working examples (but not necessary)”
• “I tend to forget things after I learn them unless I do it again and again, so I think a quick refresher exercise going over the skills learnt at the previous session would be helpful.

Scheduling

positive:

• “A bit more regularity in offerings so I can better schedule around it; every session I attended was extremely well done”
• “I found the 2 hour workshops the ideal length in terms of covering the material vs. attention span. I really appreciated how you offered the same workshop on several time slots, which made attending them a lot easier.

negative:

• “It’s a bit tough and I don’t even know if it’s feasabile to change, but sometimes I feel the sessions were too spaced too far apart for the information from one session to be remembered and built on for the second?”
• “My attendance. :(“
• “Consistent schedule, homework?, more time in my life to do this stuff”
• “Perhaps a consistent date/time during the month, to ensure students can work around the timing in advance.”
• “Future Sessions

Relevance as a Medical Students

• “Great opportunity for beginners to learn about basic programming using medicine-related examples.”
• “Very relevant!”
• “Yes!”
• “Yes, this workshop series was very relevant to me as a medical student. I have applied a lot of what I’ve learned directly to my ongoing research projects, and foresee myself continuing to do so! “
• “Yes, very relevant; having done my Master’s project in basic science, as well as clinical research in the past, I only wish I would have learned these skills sooner.”
• “Not necessarily as a medical student, but as a researcher, absolutely! this could’ve shaved years of my life as a grad student”
• “Yes, definitely.”
• “100%. Using it for my project this summer.”
• “YES YES YES! But also, just as a human being. I use OpenRefine frequently just for random stuff. I feel like its a skill that makes life infinitely easier. Wish I had made it out to more programming sessions, but I intend to follow up with the QMED Computes “tutorials” online.”
• “Most likely not if I were just interested in practicing medicine in a clinical setting. I think it will be very useful if I end up doing other research projects during school or have research projects be a part of my job in the future. I think it was really helpful to know what resources are available, to have a general understanding of how programming in R works, and to have a basic understanding of what can be done computationally (so you actually know how and who to ask for help if you’re working on something in the future, even if you aren’t going to do it on your own).”
• “It was certainly relevant to me as a medical student. I may not use all the skills/programs in one single research project, but I think they will be helpful as I work on different tasks.”
• “Yes though I feel like because I didn’t practice enough, I shot myself in the foot and might end up taking the dumb Excel way still :( completely my fault though”
• “Many of these sessions sound highly relevant to my research work as a Med student.”
• “I feel much more self-sufficient at the end of these workshop series than I did before. I actually have been able to incorporate what I have learned into my research. For example, I recently used OpenRefine to help me clean up and analyze my data, and introduced the program to a mentee. And when I started a new project, I tried to keep the major principles I learned from the spreadsheets section in mind, and setting up my data this way has absolutely made the subsequent analysis easier than it would have been if I had not attended these workshops.”

Future sessions:

• intro to machine learning?