Uchicago Safe Spaces Sentiment Survey

Intro

I became interested in doing this survey partly because of the phenomenon of echo chambers. I found that the majority of people I'm connected to on facebook responded similarly to the University letter on safe spaces, and I wondered if we all simply agreed because we're similar e.g. studied Economics, worked in business/law etc.

So my goal was to gather data from a more diverse population and then objectively analyze it to see how trends differ across different groups of people, and understand directionally what are the strongest drivers of differing opinions.

Sampling

I distributed this survey through facebook, both on my own page asking my connections to share it, and through the Overheard on Uchicago facebook page. Some characteristics of the sample that I was able to collect:

  • ~500 responses
  • 58% of responses were from the class of 2016 or later, so I sampled younger classes more heavily than older alumni, which I'm ok with because I know less about younger classes, and their opinions are more interesting to me because they are not as far removed from the college experience myself and my contemporaries
  • In terms of racial diversity, the majority of respondents were White or Asian, however ~10% identified as Hispanic, ~4% as Black, and ~5% as Other. So Hispanic and Black students were a little underrepresented relative to the University statistics (~15% Latino/Hispanic, ~8% Black). Would have been great to collect a higher mix of non White/Asian, perhaps distributing the link to specific student groups rather than a general group like Overheard
  • In terms of sexual orientation, I didn't see university statistics but ~24% of respondents identified as non-straight
  • There was a good spread in different areas of study as well as industries

Limitations

This basically my disclaimer on what this is and what it isn't. This is not an in-depth study on the issues of safe spaces and trigger words, nor is it meant to provide recommendations on what the University should do. Rather it's more like a directional sentiment analysis cross-referenced with demographic cuts. The survey is purposely short because I wanted the barrier to participation to be low. Other things I could have improved on:

  • Better choices/definitions of ethnicity (as some people suggested to align with census or other official questions), as well as sexual orientation
  • It would have made my life easier to just ask for major of study rather than "check everything you study", there's still double majors of course (kudos to you), I ended up having to do some transformations to account for people having multiple areas of study
  • I could have broken down the Administration's letter into 3 parts, 1) Trigger words 2) Safe Spaces 3) Inviting controversial speakers. Presumably people could agree with one of these points but disagree with the others
  • I could have defined precisely what trigger words and safe spaces are, however I am not really qualified to do that, and the Administration didn't define it either. I think this is fine if we're just looking at sentiment, since that's more of a gut reaction that doesn't need as much nuanced definition
  • For simplicity I didn't study interaction effects between variables (for example certain industries may be skewed towards a certain gender or race)
  • Finally, I did this primarily to satisfy my own curiosity and put some data out there. So while I enjoyed doing this I couldn't spend weeks on it, so I hope you understand those constraints with regards to any shortcomings in the methodology

Results On a scale of 1 (strongly disagree) to 5 (strongly agree), how do you feel about the Administration's position on safe spaces and triggers? (Recognizing that safe spaces and triggers can be hard to define and parse, do you agree with the spirit of the position on the relative value of 'academic freedom/censorship' versus 'comfort/discomfort')

By Class Year

  • How to read this chart: I'll use pretty much the same framework throughout, the left side is just a count of responses by different groups of class year, and then the bar is split by 1-5 scale from strongly disagree to strongly agree. The right side, puts all three bars on the same 100% scale, so you can directly compare the % of people answering 1-5, even though there are a lot more people who were 2016 and later than 2010 and earlier for instance
  • This supports my earlier hypothesis that older classes tend to be more supportive of the administration's position compared to 2011 and onwards
  • Performing a Chi-squared/Fisher's Exact test yields p-values of ~0.02, which suggests that the response is statistically dependent on the class year group (or that class group is a statistically significant driver of variation)
  • A couple of possibilities, 1) perhaps times simply have changed and sensitivity is more valued by society 2) the university has become more diverse 3) perhaps there was stronger self-selection (implying less diversity) in Uchicago applicants prior the common application and the jump in rankings

By Gender

  • Slightly more men than women responded but it's roughly evenly split
  • Men tend to agree with the administration's position at a higher rate than women
  • Performing a Chi-squared/Fisher's Exact test yields p-values of ~0.0005, which suggests that the response is statistically dependent on gender, or that gender is a statistically significant driver of variation
  • People who had non male/female gender identification were a small sample size but were strongly against the administration's position

By Sexual Orientation

  • The majority of respondents were straight, people who identified as LBGQ or other disagreed with the administration's position at a higher rate
  • The Chi-squared/Fisher's Exact test p-value of ~0.001 was significant, implying statistically the response was dependent on sexual orientation

By Race

  • As I mentioned the majority of respondents were White/Asian
  • Interestingly when performing a Chi-squared/Fisher's Exact Test, the p-value was NOT significant implying that the response was statistically NOT DEPENDENT on race. We can also see in the right chart that while ~52% of African American students disagreed with the administration's position, this wasn't significantly different compared to other races. One caveat to the this is the potential undersampling of Hispanic/Black student populations

By Political Identity

  • The majority of respondents identified as more liberal
  • Political identity appears to very strongly correlate to the response. Interestingly even though the university skews more liberal, the overall average still "agrees" more than disagrees with the administration's position, partly because within group 2 (Liberal), over 50% of respondents support the administration's position
  • The Chi-squared/Fisher's Exact test p-value of ~0.0005 was significant, implying statistically the response was dependent on political leaning

By Area of Study

  • The Chi-squared/Fischer Exact test p-value of ~0.08-0.13 was NOT significant, implying statistically the response was NOT dependent on area of study
  • To generalize, more "quanty" based majors appear to more strongly support the administration's position, in particular Economics, Polysci, Computer Science
  • English, Public Policy, and International Studies majors were more likely to oppose the administration's position
  • Note: Displayed the most popular majors, so not every major is represented. Also if you study Econ and Political Science for example, you are represented in both populations

By Industry

  • The Chi-squared/Fischer Exact test p-value of ~0.01 was significant, implying statistically the response was dependent on industry. I would hypothesize that industry captures more "information" than area of study, because at least for me I felt more freedom to choose what to study, but what to do for living is more of an optimization problem of what you like to do and how much money you would like to do it
  • Who doesn't have a heart? Lawyers, bankers, business-people, and engineers. All joking aside, those are the groups that are most firmly in agreement with the administration's position
  • Conversely, who cares the most? Looks like people interested in the Arts and Media

By Donation Level

  • The majority of respondents donate $0-10, this is most likely driven by the sample skewing younger, I didn't really donate more than a nominal amount either when I first graduated
  • People who donate more are more likely to agree with the administration's position, this is fairly intuitive, they donate more because they're happy with their experience and the school, and tend to be older

Bonus statistical concept - Decision Trees

Now of the attributes we studied, what are the biggest drivers of differences between people? I'll apply the statistical concept of a decision tree. In a nutshell, this approach looks for variables that best partition the data set (maximize information gain), so in our case, we have the classification we're trying to predict (response to if you agree or disagree with the administration), and 9 other attributes that can explain it. Which one of the 9 attributes explains the most? After splitting on that attribute what is the next most significant of variation?

  • How to read this chart: each node is an explanatory variable and one or more "levels" of that variable. So Politics = ab corresponds to the politics question and the first two levels which are 1 (very liberal) and 2 (liberal). The left fork represents if the statement is TRUE and the right fork if the statement is false. So for the first node if Politics = Very liberal or Liberal TRUE, go left, FALSE implies you're likely to strongly agree. Each subsequent node is a further split of the subset that it belongs to. So for the left path, we further split the group that is Very Liberal or Liberal based on satisfaction
  • As we saw in the individual cuts, politics appears to be the strongest explainer. The labeling is condensed here, but it basically tells you if you're NOT 1 (Very Liberal) or 2 (Liberal) then you are likely to be in the Strongly Agree group
  • If you are Liberal (a) or Very Liberal (b), AND you are Highly Satisfied (e) with your college experience you are also likely to be in the Strongly Agree group. If you are Liberal/Very Liberal (ab) and NOT Highly satisfied (abcd) you move to the next split and so forth
  • We can summarize and say politics, satisfaction with your experience, and industry are likely to be the more significant variables that explain how you are classified with regards to the response variable
  • note: industry (abdeghij) corresponds to Arts/Business/Education & Academia/Finance/Healthcare/Law/Media, e.g. excluding Consulting/Government/Science&Engineering/Software&IT but recall that this is within the subset of liberal/very liberal students. The next node,Sexual Orientation (ab) corresponds to non-straight.

Summary

  • Several variables are appear to drive differences between how people feel towards safe spaces, including politics, industry, sexual orientation, gender, and class year/age with politics being the strongest factor
  • Interestingly area of study and race were not significant, however race should probably be tested with a larger minority sample size
  • I don't pass any judgment on the results and almost certainly this will change no one's mind about this subject, but I find it interesting to gain more insight on what groups of people form the two sides of the issue
  • I'm not sure what the next step to this will be, there may be a few more analyses I will try