Monday, September 7, 2009

Regents Commission's First Speaker: Overview of the Delta Project

The "Gould Commission" is holding its first formal meeting tomorrow at UCSF-Mission Bay, and its speaker is Jane V. Wellman, Executive Director, Delta Project on Postsecondary Education Costs, Productivity, and Accountability.  Please listen to this meeting if you can: groundwork is being laid.
Most people may have no idea why the vaunted, controversial UC Commission on the Future would start with the Delta Project.  Here's quick background on three recent Delta reports. It's the last, I think, that got the Regents' attention.  I'm going to identify some strengths and then note a few serious limits to what these folks are up to.  Overall they are serious people, and they are honest about what their analyses really can't show. That's the good news.
The Delta report that has gotten the widest circulation is "Trends in College Spending." Their data are  shocking: they demonstrate that at public research universities, where state funding revenues corrected for inflation and enrollment increases have been declining, fee increases have not gone into instruction but have been diverted elsewhere.  For example, between 2002 and 2006, spending per student on instruction at public research universities actually declined, while student fees, adjusted for inflation, increased nearly 30%.
The obvious implication is that productivity problems in higher education do not stem from instruction, which is getting squeezed, along with its students, who are, as has become a refrain,  paying more for less.  So one possibility is that UCOF, as I think I must call it, would like to hear ideas for administrative reengineering. Everyone I know at UC has at least five ideas of internal systems and processes that need to be fixed. 
Second, the recent piece that was coauthored by the Regents' presenter, Jane Wellman, is called "Rethinking Conventional Wisdom about Higher Ed Finance."  One premise is that state budget disaster is interfering with "the urgent need to nearly double levels of degree attainment."  The appeal to a group like the UC Board of Regents, is another premise:
Part of the problem is that policy makers on all sides of the table keep looking to revenue solutions to the problem, when the evidence tells us that there isn’t going to be enough new money to return us to the funding levels of the past.
I would be satisfied if the state gave us back our old money.  But Delta's view here dovetails with the longstanding (and I always have to add, self-fulfilling) UC premise that state money will inevitably decline: no matter how much we've been cut we will be cut more. The logical conclusion is that therefore the only source of new money will come from efficiency experts, ones with a keen eye for the political need for "stronger accountability."
The running theme of the piece's ten points is that more money doesn't mean more quality, or more advanced knowledge, or more anything.  In fact I don't know a single person who actually believes that it does, especially not teachers, since we know quite well that quality is about rhetoric, relationships, interaction, explaining, affection, and learning from students as well.
I read this Delta piece as an appeal to the all-encompassing cost-cutting paradigm, close to the totemic essence of American business theory, and thus as the required ante for consultancy work, at which it has clearly succeeded.  There is also a buried remark about the value of "deregulation of benefit costs that now represent at least 30 percent of payroll in most states," which sounds like code for that eternal quest of California Republicans, figuring out how to  get rid of defined benefit pensions. And yet the piece mentions the important fact that while the US average funding for higher ed is the highest in the world, "Public investment in higher education in the U.S. actually falls below the OECD average."  And the piece asserts that the end that doubling current levels of attainment means that "we need to be reinvesting public resources, beginning with state appropriations." 
This brings us to the third Delta piece, by Patrick J. Kelly, who offers an analysis of higher ed productivity called "The Dreaded P Word: An examination of productivity in public postsecondary education."   He wants to contribute to the debate about how to define and measure the value-added of higher education (6), complains about the poverty of public expenditure data in a way with which I completely sympathize, and then weaves together two themes that clash by night, and which I have a hard time seeing the Regents Commission synthesizing.
The first is the claim that Delta has developed an alternative measurement of higher ed productivity that is better than the two main existing ones - graduation rates and credentials awarded as a percentage of students enrolled - a "measure of output" (7). Basically what Delta does is take this second measure, add state funding and student tuition together to get a figure for "total funding," and then divide that by the volume of different degrees to get the cost per degree.  The Delta twist - their alternative methodology - is that they weigh these degrees according to their "monetary value   . .. in each state's employment market."  So if you produce more degrees in, say, engineering, where a BA earns 20% more than a BA in political science, your weighted productivity goes up that much. The rationale is that this weighted productivity measure reflects "increased personal income" for individuals as well as increased tax revenue for the state.  Delta measures the added value of degrees by the added value of the income of the degree holder and of the income tax the degree holder pays.
The strength of this method is that it uses available data to make cross-state comparisons. The obvious weakness is that it is measuring eventual labor price and not actually the degree's added value. Lawyers cost so much that few individuals ever willingly hire them to do anything, which means that you would not want to live in a state inhabited mostly by people with law degrees. And yet a state university's productivity rating would mushroom if it closed all its college campuses and replaced them with law schools.
The same kind of problem appears on the input side.  A university can increase its productivity rating by producing the same or more degrees by getting rid of its faculty and using adjuncts only. Similar issues arise for many - though of coures not all - forms of teaching efficiencies.  When I was a graduate student I used to grade freshman compositions with a stopwatch because otherwise I would absorb myself with deep reconstructions of everything each student said.  I could hold myself to fifteen minutes for 5-7 pages, and do a good job. Or, I could set the alarm for five minutes and triple my productivity. If I did this, though, I would do a third as many corrections. Student learning would go down even as my productivity went up.
I like the baseball statistics approach to funding problems, as far as it goes.  I like spreadsheets and 535.com.  But the productivity methodology "does not address quality," as the report's author Kelly notes in a series of forthright caveats (28).   So it's hard to see how we are getting a handle on productivity in education when quality is bracketed and set aside.
This is bad enough when we're talking about individual benefits.  The complex process of developing human capacity involves a range of achievements that go far beyond income, and the report does not discuss the content of degrees or their individual impacts.  The problem is even worse when measuring social benefits. One of the reasons that public funding has been taking all these hits is the belief that education is or should be a private and not a public good, and an analysis that reduces the benefit to a state of having an educated population to tax returns is making the problem worse.  As an analysis of education's social productivity, the Delta report is at best irrelevant, and at worse seriously misleading.
UC does indeed have a range of productivity problems. Most of them involve cumbersome institutional structures that need careful redesign, rather than the panicky slashing of the UCOP restructuring project (see the UCPB critique), or the state budgetary slaughterhouse we're about to live through.  These productivity problems also rest on sheer distance and a unilateral, secretive,  not to say authoritarian management style that alienates talented, commited people and blocks the flow of information.  The Monitor Report of two years ago criticized UCOP's tendency to act "as a gatekeeper rather than as a partner."  The efficiencies inherent in fairness, interaction, and equitable, open, egalitarian, and trustworthy dealings are not addressed in any Delta materials. But they are a centerpiece of the great efficiency agenda that UC employees are at least as interested in as are the Regents.
The second of the report's two themes is never explicitly mentioned, but is in the figures. The theme is that California's funding is appalling.  Taking all three segements together, its "Total funding per FTE student by state and student share (2006-07) is 49th out of the 50 states (Figure 4, p 13).  In part as a result, its productivity is amazingly even worse - dead last (Figure 6, p 15).  California's production of STEM degrees (the science degrees prized by policymakers) is in the bottom half (Figure 8, p 18),   Here's a summary shot worth pondering:
 California is off by itself with low performance generated with few resources.  Though its productivity is higher at bachelor's and master's institutions (Figure 14, p 25),  even this productivity comes from higher salaries coupled with consistent state budget cuts in excess even of significant student fee increases.
This third report's bottom line doesn't exist: it cannot not actually say whether a given university system is under- or over-funded (28).  Unfortunately, the Commission may well avert its gaze from the now familiar evidence of poor spending and productivity rates in order to focus on the equation of productivity with the monetary value of degrees for higher ed dollars spent.  But we are going to need to do much better than that.

7 comments:

  1. Nice discussion, Chris. I worked through the "Trends in College Spending" report. It promised to tell where the money is spent, but I didn't see that it did much other than compare lots of different sorts of universities, public and private, research and not. The big finding for public research universities was that 92% of the time, tuition increases are related to losses of revenue from state appropriations. Well, I'm stunned there.

    What I did see was the assertion that higher education is an "industry" and its defect is that it hasn't developed a a "common metrics" so that different institutions can be compared for "productivity". The logic here appears to be, universities by being different in circumstances frustrate our ability to conceive of them as commodity producers. We call on them to make themselves comparable, so they can be compared. Is "higher education" an industry? Is that an assertion or statement of fact? Seems to me, once you go there, we're looking bottom line. As an alternative, one might look at Jim Collin's Good to Great and the Social Sectors with its tag "business thinking is not the answer".

    With regard to research universities, it appears the Trends report makes a clean separation--research and instruction are entirely separable in determining "productivity". For its purposes, UC is just like CSU. Just FTE students and revenue, regardless of context. I guess if instruction is a commodity, then this is fine.

    When one turns from considering averages and generalities gleaned from looking at many institutions, even public research universities, the question is, how does a view of the typical affect the directions for UC, which is anything but typical? One is always advised not to reason from the general to the individual unless one's goal is to conform the individual to the general.

    Wonder what the UCOF wants to see in this first presentation? A call to tighten the administrative belt? To put pressure on instructional activity to "boost" productivity, leaving research alone? Contract with Montana to open an inexpensive UC branch campus in Kalispell? Would making a show of productivity adjustments win back the legislature, to at least protect UC from the next round of cuts in the state budget?

    ReplyDelete
  2. I agree, Chris, that the information on total funding per FTE is rather stunning. Beyond that, it's hard to know what to make of the Delta Project's study on productivity. Research, service, and outreach are assigned zero value, and it seems unlikely that their actual values are constant across states.

    Concerning their output measure, even if it were true that the only product produced is measurable by the degree-recipient's income, the value of a PhD over a Master's degree is only the incremental salary, and it is earned over an entire career. Same for the other degrees. Those increments bear no direct relationship to the current, median income for PhDs in the same state. If any of those graduates start businesses that move others from the ranks of the unemployed to a salaried job, that is also not counted.

    Just like college football rankings, quantification thus seems to imply far more than it delivers for producing meaningful measures of university performance. The whole point of being UC is quality. Whether they count degrees produced or this index using statewide salary information is largely beside the point. Using the Delta Project method, UC would improve California's performance every time it replaced a regular faculty member with a grad student lecturer---same product, lower costs. Certainly, providing any infrastructure such as computers or libraries seems pointless; how many additional degrees are produced because there are fast computers or more books on campus? Given the Delta Project's measure of output, it's also unlikely that these investments show up in the performance measure.

    The Delta Project study includes the appopriate caveats concerning not measuring quality, and the pros and cons of such a study can be discussed because they have outlined their data sources and calculations. However, the caveats tend to get left behind when the quotable take-away messages get repeated by people who ought to know better. See the President's recent touting of the Washington Monthly rankings for another example of promoting an index number, just because the outcome looked good for UC.

    What I'd like to hear in today's UCOF session is that higher education's funding is severely inadequate, and that we produce a lot more than higher incomes for our graduates.

    What I expect to hear is that higher education has not done a good job making its case, and that better accountability is needed. How to square that with the other message that seems to accept lower funding as a permanent state of affairs isn't obvious to me. Moreover, California's state parks and numerous other public goods have also apparently fallen short on accountability, since we aren't funding those, either.

    To quote Mark Yudof (Change Magazine, March/April 2002), "students are not widgets".

    ReplyDelete
  3. To follow on the fine Yudof quote--students are not widgets, but the Delta project suggests faculty--insofar as they participate in an industry that delivers degrees(and nothing else)--are equipment....

    Is this the necessary upshot of a creative-class knowledge economy? That it gets expressed in terms of a metrics suited to industrial outputs? Is that where the thought leaders are headed?

    ReplyDelete
  4. Listening to the Commission's meeting right now, and it's hard to see how these folks *don't* think of students -- and faculty and staff -- as widgets, or even worse, as impediments to the re-direction of funds upwards. Bizarro world.

    ReplyDelete
  5. Would seem we have the following: consultant floats a number of themes so leadership can gauge responses from various quarters.

    Themes I captured are a) develop new metrics for management; b) contain retirement and health benefits; c) let tuition float to end hidden cross-subsidies across majors and between lower division/upper division and undergraduate/graduate/professional; d) consider academic restructuring and back office administrative sharing across programs; e) comprehensive state wide approach rather than internal re-arrangement; f) push lower division stuff back toward cheaper venues, like CCs; g) settle things out as a public policy debate not simply management changes.

    While Wellman did say at one point that folks shouldn't give up on state subsidies, the force of her talk was that the decline in state funding was inevitable and that it was long term, so folks had better start working toward a new model. She mentioned that CA actually ranked quite high in subsidies per FTE student. I wonder if that is also what Delta is telling the legislature...don't throw more bucks at UC until they achieve 3% administrative cost savings and commit to a program of degree attainment that has sharp edged metrics that anyone, including legislators, can follow.

    If one is into "unit cost to produce degrees" and "return on investment" analysis based on "finances, attainment, and demand", and a resolute focus on "degree attainment" as the core message about higher education in the economy then this must have felt like an inspired talk. The financial realities have to be addressed--it's just whether a metrics bound ROI approach is going to lead to a new inspired vision for education in California.

    ReplyDelete
  6. One theme at the Commission's meeting today was the lack of reliable data on most of anything.

    I was looking at the two Delta reports cited by Chris. I just don't understand the charts. If I look at Figure 11 in Trends in College Spending it shows that CA spends a lot on education compared to the other states. In fact it is the 4th among states. Good.
    If I look at Figure 4 in The Dreaded "P" Word it shows that CA is second to last in education funding. Abysmal.
    The numbers in the two charts are in the same general range but come from two different sources.
    Am I missing something?

    ReplyDelete
  7. Excellent question! The tables have different data sources, but I think the real difference is that Figure 4 in P-Word is funding or revenues, and Figure 11 is expenses or expenditures. Figure 11 would include Cal Grants expenditures and student contributions to the overall cost of attendance, while Figure 4 is a more accurate reading of the direct payments states and students make to fund campus operations. Figure 4 is a better gauge of what CA spends on direct educational funding, though it is lowered in part by our excessive reliance on gigantic community college enrollments. That's only a start of an answer though- educational data is messy

    ReplyDelete

Note: Firefox is occasionally incompatible with our comments section. We apologize for the inconvenience.