One of my first responsibilities as a new hire at Enigma was to complete an onboarding project focusing on public data. The scope of the project included finding a new dataset to merge with existing Enigma Public data, transforming the data into the correct format, then building an app/visualization displaying insights from the data.
The full project is available here—you can explore the specific fields and see the full analysis and visualizations.
Choosing a dataset to explore was a challenge—the wealth of information on Enigma Public is vast and comprehensive, and provided me with lots of exciting options—but the one that caught my eye was the American Community Survey (ACS).
The ACS, sent to about 3.5 million addresses annually, is an ongoing survey that measures various demographics of each household including occupations, earnings, housing, educational attainment, race and veteran status. You can see the full list of subjects here. This information is then used by public-sector, private-sector and not-for-profit administrators to allocate resources to various communities.
The ACS is a great example of how gathering insights from large-scale datasets can be used to make vital, impactful decisions for individuals and communities. The ACS helps determine where to build new schools, plan for hospitals, allocate emergency services, establish housing affordability and inform businesses about new development opportunities, among many other things.
With a wide scope and so many demographics to explore, the ACS is a valuable launchpoint for examining significant social issues. While the raw numbers are essential in planning for the future, I was more interested in what the data was telling us about our present society—who are we as a people? What are our values and priorities? What are the issues affecting Americans in their day-to-day lives?
The dataset I used for the project focused on full-time, year-round workers sorted by occupation, gender and earnings. Sorting on these verticals exposed a well-known and still prevalent societal issue—the gender wage gap. Of the occupational fields with a large enough sample size (100+ cases), 96.5% (or 334 out of 346 fields) reported a pay gap. The chart below measures median earnings between men and women for some of the highest reported/most common occupations in the US. For the purposes of this study, the gender wage gap refers to the average difference in pay for services rendered or work performed between men and women.
While the gender wage gap is glaringly apparent in only a surface examination of the data, to fully understand the issue, it was important to dig deeper. The wage gap is a complicated problem with multiple factors contributing to both its existence and perpetuation in society.
The data showed what I knew to be an essential aspect of the gap: occupations that employ mostly women are lower paid than occupations that employ mostly men. Childcare workers, housekeeping staff, preschool and kindergarten teachers and home-health aids—all jobs that are overwhelmingly female dominated—command significantly lower salaries than occupations with similar experience and education levels that employ mostly men.
Another important finding showed that in addition to a wage gap, there is an opportunity gap. In high-level administrative and managerial occupations, there are simply less women at the top.
While these are valuable insights, there is plenty more to explore in this dataset. The wage gap goes beyond just earnings by field, or even equal pay for equal work. Further research has revealed that factors such as unemployment and career disruption (women leaving the workforce for circumstances like child-bearing or family care, for example) significantly impact the gender pay gap. The gap widens even more when adjusting for race.
This is not just a gender issue, but a societal one. Pay disparity has far-reaching economic consequences, contributing to poverty rates among working women and their families.
Using the ACS to explore the wage gap was a valuable exercise for me in understanding the power of public data and its importance in obtaining accurate information about important social and economic issues. The ACS’s purpose, as stated on their website, is that it “provides vital information on a yearly basis about our nation and its people.” The survey represents a snapshot in time that reveals what we, as a society, are doing well, and what we can do better.
As I was completing the project, more questions arose: What additional insights and ideas can be gleaned from this information? How do we objectively use this data to make informed decisions about our businesses, our communities, and our families? What other societal issues can be better examined and understood using public data? Enigma will be exploring this subject and other issues in future blog posts.
We’d love to hear about how you used Enigma’s public data in your projects! Visit the projects page and drop us a line.
Interested in helping others connect with public data? Enigma is hiring!