Introduction: The Dashboard Concept
Monitoring and assessment of progress towards our strategic goals requires a combination of qualitative and quantitative evaluations. Quantitative metrics may be found on the Strategic Plan website, following the link to the Dashboard.
The Dashboard Concept. Just as an automobile has thousands of moving parts and potential indicators of performance but uses just a few on the dashboard to help gauge the performance of the whole a University has many components that interact with each other as a complex system but can use a small number to monitor the collective operation.
Four University strategic metrics were developed by the Executive Team and the Deans, with several revisions and opportunities for comment from the campus community. Following selection of the four metrics, the Colleges and Schools set goals, which are targets to achieve by 2012. These unit goals were summed to obtain overall University goals and are internally consistent, meaning they sum from the unit level to the University total.
Metric Use and Interpretation. It is critical to remember that each metric is an indicator that represents a component of the overall University system. However, there should not be an unhealthy focus on individual metrics or over-interpretation of their value. Just as teachers can 'teach to the test', it is possible to overemphasize achievement of a given goal by losing sight of the overall system. Entering Freshman ACT Scores, for example, can be maximized by admitting only one student with a perfect score, and no one else. This would maximize the value obtained for the metric, but sacrifice overall system performance. Overemphasis on metrics may result in inappropriate and unintended pathological behavior. For that reason, there are several other levels of metrics that focus on more specific aspects of the University system:
- Initiative Metrics focus on a critical University initiative, such as on-line learning. These contribute to the overall achievement of the strategic plan, but provide a focused look at some University initiative that cuts across departmental boundaries and requires cooperation among many units and individuals to accomplish.
- Operational Metrics provide detail on some critical University activity, such as Research or Admissions. These relate to the four high-level metrics, but again provide greater detail regarding specific sub-systems within the University.
- Unit Metrics provide detail for a specific administrative or academic unit, such as the Department of Mathematics. The four University metrics drill down to the academic units, but units may need additional focused metrics to assess their performance in achieving their unit-level strategic goals. These obviously must be upwardly compatible with the University goals, but the tactical activities at the unit level may require additional metrics for evaluation.
At the highest level of detail, Institutional Analysis maintains and updates the University Compendium, which is available from the Institutional Analysis web site. The Compendium provides two pages of detail on both the unit and the aggregate level which spans several years. The four strategic metrics provide a high level assessment of condition, while the Compendium provides comprehensive information for the most detailed performance evaluation.
Metric selection is a subjective process. It is a balancing act that attempts to identify simple, easy to obtain, yet meaningful metrics that, in combination, provide an indication of the state of the system and progress toward achievement of the strategic plan. At different stages in institutional development, different suites of indicators may be appropriate. Likewise, maturation and adjustment of strategic goals requires periodic evaluation of the indicators being used.
The Executive Team and the Deans began with the strategic plan, and entered extensive discussions to determine a means of monitoring and assessing progress toward achieving the goals of the plan. The process included review of the metrics used by US News and World Report in their annual rankings of universities, as well as indicators used by groups such as the Association of University Technology Managers in their annual survey of university technology transfer activities. Reviews of metrics used by other universities to assess their strategic activities were also undertaken.
This process continued for several months, and in the end resulted in four: Entering Freshman ACT Score, Number of PhD Graduates, Sponsored Program Awards, and Endowment Value. Each of these is explained in detail in subsequent sections.
Some metrics were given extensive consideration and only eliminated late in the process. The number of tenured and tenure track faculty, for example, was seriously considered for inclusion as a fifth metric. Ultimately, this was considered more as a driving variable that should be considered in the budgeting process and in program development, and less as an indicator of progress toward institutional strategic goals. To achieve the goals and increase our national rankings we almost certainly have to expand the number of faculty, yet that was considered more of an operational decision instead of an indicator.
Similarly, the number of PhD Graduates could be replaced by PhD enrollment or other variables. Enrollment, in combination with time to completion and completion rate, drives the Number of PhD Graduates. While enrollment is inward looking, the number of graduates placed in faculties of other highly ranked institutions is a major driver of institutional reputation and the number of graduates is a more outwardly looking metric.
In the end, again, the metric selection is a subjective process that was undertaken jointly by the Executive Team and the Deans, and presented to the campus community for evaluation, comment, and suggestion. These final four university level metrics are the result of that process.
Incoming Student Average ACT Score
Description. The Incoming Student Average ACT Score is calculated in the fall term once the first year students are enrolled. It is simply the numeric average of the composite ACT scores of the new first year students; transfer students and international students are not included.
Interpretation. The maximum possible ACT score is 36 and the general minimum required for acceptance to Michigan Tech is 21, though there is variability across curricula. Higher values are commonly interpreted as indicating better prepared students, but there are other factors considered in the admission process, including leadership experience, etc. Admission is based on a suite of student characteristics, not simply ACT score.
This variable may be drilled down to the first year students accepted into the various curricula within the Colleges and Schools. It may also be tracked through out the year for new student applicants, those accepted to Michigan Tech, and those that have paid deposits. These values are updated weekly, but the final number is not known until the new students are actually enrolled in the fall.
Relationship to Strategic Goals.
People. Incoming Student Average ACT Score is a direct indicator of student academic ability, and thus is directly related to the first strategic goal. This is obviously only one facet of what a student brings to the University, and should not be over-interpreted.
Programs. Good students are attracted to distinctive and high quality programs, such as the Pavlis Technological Leadership Program, Honors Institute, and Enterprise. Bright, motivated, and adventuresome students are seeking a broad range of experiences, and not only distinctive academic programs, but high quality enrichment programs of all types will allow Michigan Tech to continue to attract high quality students.
Scholarly Activity. Many of the more academically prepared students actively seek a research opportunity as undergraduates. The opportunity to work directly with faculty and graduate students can immensely enrich the undergraduate experience. Opportunities like those provided by our Summer Undergraduate Research Fellowships [SURFs], as well as opportunities for student involvement in Michigan Tech's externally supported research programs, should allow us to continue to attract highly prepared students. Such highly prepared students can make valuable contributions to the research enterprise, further advancing this strategic goal.
Responsibility. Incoming Student Average ACT Score can be projected from information collected from students during the Admissions process by Admissions and the Vice President of Student Affairs. The final value is calculated and reported in the fall of the year by Institutional Analysis.
Number of PhD Graduate
Description. The Number of PhD graduates is reported annually, but changes after every semester. It is simply the numerical count of those completing the requirements for, and being awarded, the PhD.
Interpretation. PhD graduates play a large role in determining institutional reputation. Individuals going on to the faculty at other institutions or entering the corporate or public workplace all reflect on the institution they graduated from. With the PhD being a research intensive degree, the number of PhD recipients is also an indicator of the volume of research activities on a campus. The Carnegie Foundation, for example, has often used a threshold of 50 PhD recipients per year in at least 15 disciplines as a threshold in their university classification system. Michigan Tech has long had the requisite number of programs, but we have been graduating 40-50 PhD students per year since the 1980s.
This variable may be drilled down to the number of PhD recipients in each of the Colleges and Schools, as well as to the individual programs. There are also interdisciplinary degrees, such as Computational Science and Engineering, but students in those programs do have a home department and will be included in department level summaries. The number of PhD recipients is a function of the number of enrolled students, the average length of time to completion of the degree, and the proportion of entering PhD students that complete degree requirements. Normally, students must complete the requirements within 8 years of enrolling in a program. As a rough indicator, on a sustainable basis, the number of graduates is approximately one-eighth of the number of enrolled students. As enrollment changes though, there is a lag before the number of graduates will reflect new enrollment levels.
Relationship to Strategic Goals.
People. The Number of PhD Graduates is an indicator of the quality of our programs as well as our people. Top faculty advise the top students, and PhD programs are largely a one-on-one experience between an advisor and a student. High quality PhD students contribute to the intellectual environment of their department, School, or center, and may be involved in supervising undergraduate research experiences or undergraduate instruction.
Programs. PhD programs are often focused in a narrow field of sciences, whether in a traditional discipline or in an interdisciplinary area. The variety of PhD programs offered on a campus indicates the range of areas with vibrant intellectual activity levels. Not all units offer PhDs, and it is not appropriate that every unit do so. PhD programs are often ranked within various disciplines or through interdisciplinary ranking efforts, and erceptions and measures of program quality often include, if not center upon, the graduating students.
Scholarly Activity. The PhD degree is a research focused degree, and the Number of PhD recipients is a direct indicator of the volume of research activity on a campus. In many areas of science and engineering, it is difficult to imagine an institution ranking highly within a discipline without a significant number of PhD recipients. PhD students also author or coauthor with faculty many of the archival publications that serve to record knowledge and advance the various disciplines.
Responsibility. Number of PhD Graduates is reported annually by Institutional Analysis and tracked on a semester-by-semester basis by the Graduate School.
Sponsored Program Awards
Description. Sponsored Program Awards is the value of the new contracts and grants received by the University in a given year from external sponsors. This includes indirect costs to be recovered from sponsors. Since 2005, Sponsored Program Awards has also included research-related gifts and corporate gifts. This also includes funding received for instructional and other public-service projects. Research projects, though, have historically received about 90% of the Sponsored Program Awards.
Interpretation. The funds received in a new award are not expended until the work is completed, which may be several years after the award is received. Also, some federal projects, particularly larger ones, are not awarded all at once, but each annual increment or task order may be a separate award. Due to these factors, there is not a one-to-one relationship or a simple lag function that relates awards to expenditures. National rankings in research are almost always based on expenditures. They may use total expenditures, such as the National Science Foundation rankings, or may be based on only external expenditures. In any case, Sponsored Program Awards gives a forward looking indicator of future expenditures.
Sponsored Program Awards may be drilled down to the departmental, School, or College level. Sponsored program Awards that are associated with Centers or Institutes are still given unit coding if the investigators are administratively within academic units. Two research units, the Keweenaw Research Center and the Michigan Tech Research Institute, often have investigators that are not administratively affiliated with an academic unit. For this reason, the School and College Research Awards do not sum to equal the total University awards.
Relationship to Strategic Goals.
People. Sponsored Program Awards indicate the value of the ideas generated by the campus community in the competitive research arena. Sponsors award projects in response to those proposals that are most creative and effective in meeting the requirements of the solicitation. Increasing awards implies that University faculty, staff, and students are recognized by federal and other sponsors as deserving of support. In addition, funding provided by such projects supports graduate students directly and provides undergraduates with opportunities for research experiences.
Programs. Awards received by programs provide an indication of the quality of those programs as perceived by external sponsors. Research Awards also may provide initial seed money to start a new, distinctive program. The biomedical engineering program was started with funding provided by the Whittaker Foundation and the Enterprise Program was started with seed funding provided by the National Science Foundation. Strong, aggressive efforts in obtaining external funding support the development of distinctive educational and research programs.
Scholarly Activity. Sponsored Program Awards provides a direct indicator of the level of future research activity on campus. As noted above, most rankings of institutional research are based on one of several measures of expenditures, but Sponsored Program Awards gives a strong indication of future activity and is most helpful in the budgeting process.
Responsibility. Sponsored Program Awards are tracked on a continuing basis by the Office of Research and Sponsored Programs. These are summarized monthly and reported to the Board of Control on a quarterly basis. The metric is totaled annually on a fiscal year basis.
Description. The Endowment Value is the current financial value of the institutional endowment held and managed by the Michigan Tech Fund. It is not the total assets of the Fund, but only the portion of those held as endowment. The annual value is the value on the last day of the fiscal year.
Interpretation. The Endowment Value indicates the funds held for long-term investment, the purpose of which is to produce earnings for use in support of specific University activities. Endowment funds are often designated for a specific use, such as scholarships or in support of a chaired faculty position. The total value of the endowment indicates the amount of earnings that will be transferred from the Michigan Tech Fund to the University in support of endowed programs during a fiscal year.
Relationship to Strategic Goals.
People. Endowed funds may support a variety of programs related to attraction and retention of our most critical asset, which is our people. Endowed faculty chairs support the attraction and retention of highly productive faculty to support our distinctive programs and scholarly activities. Scholarship and fellowship endowments increase our ability to attract bright, motivated, and adventurous undergraduate and graduate students.
Programs. Endowments support the development and conduct of distinctive programs within the University. This can include academic enrichment programs, such as the Pavlis Technological Leadership Program, or cultural enrichment programs such as performing arts or a speaker series. These programs, supported by an endowment, enrich the educational experience of our students as well as providing cultural enrichment to the campus community and the surrounding communities.
Scholarly Activity. Endowed funds can support specific research activities or research centers or institutes, or they can support students and faculty critical to the advancement of our scholarly programs. Equipment endowments, such as the Century II Equipment Endowment, provide assistance to the University in maintaining and upgrading critical research and educational equipment.
Responsibility. The endowment is held by the Michigan Tech Fund and monitored continuously by the Chief Financial Officer of the Fund. The Endowment Value is reported regularly to the Board of Control and updated annually for monitoring progress toward achievement of the strategic plan.
The four University-level metrics provide an overall view of University progress in achieving our strategic goals related to People, Programs, and Scholarly Activity. Achievement of the targets set for each of the metrics is dependent on making progress toward the three strategic goals. The other lower level metrics provide more focused indicators of progress toward institutional initiatives or in monitoring the operational activities of the University. In addition, individual academic and administrative units can monitor the progress achieved by their day-to-day activities as related to the overall strategic goals of the institution.
If the institution is making progress toward the four strategic metric targets it is a strong indicator that we are making significant progress in achieving the institution's strategic goals. Put another way, we cannot make progress toward the targets for the metrics without making significant strides in developing our human capital and talents, increasing the distinctive nature of our programs, and increasing and improving our scholarly activity.