The ADVANCE Initiative

ADVANCE Faculty Equity Query Tool (AFEQT)

What is AFEQT?

Michigan Tech strives to cultivate and support an exceptional, diverse community of students, faculty, and staff. As a predominantly STEM institution, the university values data and uses it as evidence to motivate change. To help Tech slice the data and track trends over time, ADVANCE developed the AFEQT tool in collaboration with IT's Enterprise Application Services, Institutional Research, and university administrators. AFEQT enables department chairs and deans to get a clear picture of current and historical data. It also allows them to compare that data across parameters such as gender, race/ethnicity, and faculty level. Default reports can be generated on tenure-track and instructional-track faculty by gender and race/ethnicity; promotion; tenure applications and awards; years in rank for tenured faculty; faculty turnover; faculty hires, cohort analysis of tenure and promotion; academic leadership positions; salary; research space allocation; start-up packages; and teaching loads from student credit hours. Custom reports can also be generated by selecting and reordering parameters.  Output is table, graph or can be downloaded into excel or pdf.  Department chairs and deans may access data and compile it for their units. All of this, if used properly, will enable valuable and meaningful progress towards achieving greater diversity, equity and inclusion in a historically and predominantly white and masculine institution.

The ADVANCE Faculty Equity Query Tool was selected as an exemplar in the DEI for Data and Analytics category of the 2022 Data and Analytics Horizon Report sponsored by Educause. The link contains additional national perspectives on the importance of the AFEQT effort.

How was AFEQT developed and vetted?

The process followed was: AFEQT Development →Pilot Group 1 (Deans, Chairs)→Refinement → Pilot Group 2 (VPR, Chairs) → Refinement → Rollout (these tutorials, live sessions) → ongoing maintenance.

History

NSF requires all institutions applying for and holding ADVANCE grants to track data in eleven (11) main categories, which contain some subcategories.  Many of the items tracked were identified from the MIT Report as well as in the book Why So Slow?

In 2016-2017, while preparing proposals for submission to NSF, the ADVANCE PI  team leveraged focus groups to extract the data from the Compendium or Institutional Research (IR) and compile it for the 11 categories.  Each category utilized between 5 and 40 person hours to compile the data, was discussed openly, and reworked to address skepticisms (or other dependencies) raised.  

It was concluded that, to be successful, our ADVANCE program would need to be able to track university-verified data in near real time and strategically share it with the following purposes in mind: 

  1. Aggregate data for administrative decision-making, university audiences (university dashboard, training sessions, etc.), and/or the public.
  2. Separate data for colleges/units in order to inform decisions that influence the data trends in the 11 categories.  These could be achieved either within the query tool or alternatively crafted into excel files or formatted reports that are “pushed” to units at strategic times to inform decisions.  
  3. Easily generate the annual data required to be submitted to NSF as an ADVANCE institution.  

The ADVANCE budget was crafted with two personnel to make this successful: one in Institutional Research to oversee the data, and a second in Information Technology’s (IT) Enterprise Applications Services (EAS) department to build a query interface in WebFocus to generate reports and plots. 

An initial tool framework was developed and tested in 2020.  

In May 2021, we demonstrated the query interface for the Vice President of Diversity and Inclusion.  Suggestions for enhancements included expanding the tool to examine race/ethnicity, clarity in retention, and additional metrics.  We also had a valuable discussion related to potential audiences, data sensitivity, and public-facing data (within Michigan Tech and external to Michigan Tech).  It was decided we would clearly delineate two uses: public-facing static data and a dynamic query tool for unit leaders (AFEQT).  The VP's office is leading the effort for the public within Michigan Tech dashboard where protection criteria for personally-identifiable information are being met.  The ADVANCE team has thus led the refinement of the AFEQT query tool for a limited audience of unit leaders (administrators) who have been granted access to run reports.  The query tool may also be used to generate reports that could be distributed to a specified audience (all chairs) on a schedule.  

In June 2021, several faculty administrators were asked to serve as an initial pilot group to review and vet the query interface.  Their participation helped guide the development and refinement of the tool.  Once this initial pilot group felt they had fully tested the tool, a second stage pilot group was identified and was convened in August 2021 to further test AFEQT.  Rollout to all department chairs and deans occurred March-May 2021.  100% participation was achieved and all chairs have access to the tool.  Training documentation and training videos are complete and available to chairs.  Live training and Q&A sessions are ongoing, including a presentation to Academic Forum in September 2022.

Summary

  • A faculty equity query tool has been developed by ADVANCE that pulls from multiple University databases.  Processes are in place to ensure the data is accurate, consistent with other reporting rules, and compiled annually.  ADVANCE has successfully institutionalized this resource for university decision-makers and influencers. 
  • VPDI is working with Institutional Research  and IT-EAS to generate a Michigan Tech interfaceable dashboard from which select data will be published for Michigan Tech public audiences.

Tutorials

Orientation as well as Q&A session presentations and zoom recordings are archived for those with access to the tool.  In addition, there are four tutorials that have been developed that are in written documentation format as well as in video instruction format.  These tutorials are available in AFEQT’s help menu and include: 

Introductory Tutorial

Intro on why the tool was developed and how to use it, including default and custom reports as well as footnotes and help infrastructure.

Output Tutorial

How to slice and examine data in the ‘Salaries of Faculty’ Report and ‘Start-up Packages of Newly Hired Tenure-Track Faculty’ Report. The tutorial also covers plotting data via Bar Graphs and Line Graphs, exporting to PDF, Excel, or PowerPoint, and also directs to 'Saved Parameter Sets' tutorial.

Retention Tutorial

Retention analysis and the complimentary cohort analysis from the  ‘Faculty Turnover’ reports as well as ADVANCE’s Analysis of Retention {Link} and the prior mini retention study {Link} (conducted in 2016 with IRB approvals).

Student Credit Hours Tutorial

Explores a key dimension of workload with an equity lens including a guide to examining historical trends, SCH by term, or by fixed term/temp employees.

Help and Footnotes:

The help functionality includes all definitions for each parameter as well as instructions on operating, running, and customizing reports (also in tutorials).  

Each report includes detailed footnotes that illustrate the data the report was run, the data source for the data and any parameter rules for data inclusion or exclusion.  

ADVANCE Reports:

From AFEQT data, the ADVANCE team conducts analyses and periodically compiles these into reports intended to guide discussions by administration and others on campus.  These reports are always conducted campus-wide or with a focus on STEM faculty (per NSFs original program focus). The following two reports related to retention can be viewed at this time:

  1. ADVANCE’s analysis of retention (2009-2019) 
  2. Prior mini retention study (conducted in 2016 with IRB approvals)

2022-2023:

The ADVANCE team will continue to monitor usage, run additional intro/training sessions in collaboration with the Provost's Office, and work with administrators to develop up the ability to push reports to departments to coincide with key decision points.   

Ongoing Data Stewardship:

The AFEQT tool reflects the data available.  Much of the data is maintained by Institutional Research, but some data depends upon departmental reporting.  Thus, it is important that all individuals and units put effort into reporting data in a timely and consistent manner through each of the relevant database interfaces.  

One example of unit reported data are the startup budget forms submitted to the Vice President for Research when a new faculty is hired. The form is historically utilized by accounting to set up the startup account so it includes fund sources and category assignments for those funds.  However, this form overlooks other types of investments in new faculty startup including graduate teaching assistantships (GTAs), space remodeling costs, etc.  Please document these types of investments in the comment box on the VPR startup form. Developed with the ADVANCE Partnership, we also offer a Department Chair Resource Document.

AFEQT maintenance:

Data is updated annually by Institutional Research.  The IT-EAS team has AFEQT in their annual maintenance schedule to upgrade software and address any data changes from Institutional Research.  ADVANCE will continue to train and monitor report completeness and quality.