Gayblack Canadian Man

Foreign Policy Analysis
UFTL 2015 Faculty Panel: Do my grade distributions reflect my students’ learning?

UFTL 2015 Faculty Panel: Do my grade distributions reflect my students’ learning?


We’re going to get
started in a second here, and what I request from you
is to think up your hardest questions about grading, your
absolute hardest questions, because we’ve got a
panel that can handle it. I am going to try to be quiet. Impossible. Impossible, I know. Welcome to our session,
and let me tell you a little bit of background
on why we’re here right now. The little bit of background
on why we’re here right now is because the Senate had
some information about how we, as faculty,
should assign grades, and we, as faculty, think
we should assign grades. It is all about me. Well, it’s not really,
but there are constraints on what we can do, and we should
talk to our colleagues, and all that kind of fun stuff,
but rather than have an open breakout discussion
of the presentation and the fine panel
discussion that we had– that was the original
plan, but Patrick and Jane came up with the idea of, hey,
you guys have been talking about grades as a Senate. This is going to be
the set of people that have to assign grades
for the rest of their lives. Why don’t we have
a conversation? And so that’s
where this started. So I would like to start
this presentation out with giving an A to Patrick
and giving an A to Jane, and you can see there
that their appreciation A’s are there in front them. Thank them, please. [APPLAUSE] Thank you. Thank you. Did you get a present? Looks like he’s getting
one for everyone. Thank you. Thank you. This is a job well done, right? Ed, thank you. Isn’t this fun? I want a green one, too. And thank you. Now they have all A’s. We can go home for the semester. This is probably not the
way to think about grades. The way you should think
about student performance, evaluation, all of that
is inside your heads. It’s by your students, and these
panelists are here to help. Lew? I knew he’d do that. [LAUGHTER] First thing I said this morning. Good morning– is I told him,
Jim, I don’t want to go first. Well, you’ve all seen the
question, the position of what this session is about. I have an answer to
the session title, and the answer would
be yes and yes. What do I mean by that? Those of you that
don’t know me, I’m chair of the Department of
Management for the second time. That means I’m probably the
least smart person in the room. [LAUGHTER] And here’s what I mean by this. I’ve been teaching
since 2010 freshman in the College of Business. It’s the only freshman
class, and because I’m back in a chairs job I
have to give it up. There’s some good to
that, and there’s also some bad to that– giving it
up– but those of you that teach freshman in the B school,
we don’t know what that’s like. We don’t do it, except now we
have for the last five years, and in the fall, it’s a
large class, 300 to 400, and in the spring,
it’s a small one. And what I mean by
yes and yes is this. Over the years, especially
the last 23 years, those of us that have been
chairs and on the PAC forever if you’re
not a chair, we’ve read all the student
evaluations of all of our faculty and departments. And essentially what
we start reading is about we’re looking for
respect from the students. You weed out the I love
her and I hate him, and then you get
everything in between. And you look at the
level of respect, and then in our department we
also look at all the grades that are given that
year by that faculty member along with the
student evaluations, and we publish that for all
the faculty in the department to see. And what the funny
thing happens is that the person– the real
educators in the apartment that wind up with high
standards in grades also seemingly get high
marks from the student. In spite of what
we might believe, that generally is what happens
with our best educators. Then we start talking
to them, and if you ask them this question,
I’m sure most of them would say yes and yes. The students learn when
they put something into it, and they put something into it
when you hold them accountable, and are fair, and do all
those things that you should do as a good educator. So it’s hard for
me to answer yes on one side and no on the other. Student learning is
a core value for us and I’m sure for
everybody in the room. It’s about student learning. How do you generate
student learning? How do you get that done is what
we’re all doing in our careers as best we can. Some of us seem to be doing
it better than others, and those are the people
we go talk to and find out what it is that you’re doing. What are you doing that
apparently I’m not? So do you have something
to do with the grades? How could you not have
something to do with the grades? You designed the class. You set up the accountability
triggers on deadlines, and the assignments,
and the exams, and you have a great
deal to do with that, but then you also design
opportunities for the students to learn, and then
you cross your fingers just like you did with your
own kids and hope they do it. You hold them accountable,
but sometimes they don’t. I’m reminded of a story
in the very first class I did with the freshman. I was standing, by the way,
with some parents at an awards ceremony is how I got drafted
in the freshman class, because a couple of
university people came up and asked me right
in front of the parents would you stand up a
freshman discovery core course for freshman in
the College of Business? Well what am I going to say? Well, of course I will. In that first class, I
went home that first month and a half with my
tail between my legs. My wife is an elementary
school principal, and she said what is
the matter with you? You’ve been moping
for a month, and said, you know all those
teaching awards that I have over the years? I’m giving them back. I don’t think I’m
any good at this, and she says, well, you
have the digital natives. And she gave me a book to read
about the digital natives, and I did, and said, oh. I’m supposed to be the parent,
and hold them accountable, and do all that
kind of stuff, huh? She said, yeah, you are. Funny thing happened when you
do that with the freshman are– they’re busy staying up all
night because they can right now, but you wind up feeling
a little bit like the phys ed instructor at field
day in high school, but once you get past
all that, about midway through the semester they
realize that they really are accountable. They’re accountable
for attendance and all the various other things
that you’re doing, and they start to learn. So did I have something
to do with that by virtue of designing the course and
maintaining the accountability for the students? Yes, I had something
to do with that, and I’d be crazy to
say that I didn’t. Was it about me? That’s the hard part
of this session’s name? Is it about me? It’s about them learning
and whatever role I played to help them do that. And in some cases,
I’ve been rewarded well by watching students–
freshmam– succeed, and retaining them,
and not losing them. That’s been rewarding. That part I kind
of hate to give up. The field day director– I’m
glad to give that part up. So in a nutshell,
there is no way that you can be an
instructor and an educator. Those folks in this
room– there probably isn’t anybody in this room
my age, but when we were kids and when we went to
college, it would be not unusual for the
professor to say, well, if you want to learn
something, that’s up to you. I teach. I remember being told
that more than once. That’s not good enough in 2015. As a skilled educator, your
job is to design the course, set it up so the students
have the greatest opportunity to learn. True, you can’t read
the books for them. You can’t do the work for them,
but you can design the course with triggers of either
motivators or accountability triggers, whatever
you wish to call it, that do work if you
experiment with it. So yes, you have something to
do with it, but it’s about them. [APPLAUSE] They each have a very
short position statement. That was it. We’re all professors too. Yes and yes. I’m Brenda McCoy,
and I’m the chair of the Department of Community
and Professional Programs, which is really sort
of irrelevant to how I arrived on this panel. I got here because for
the last couple of years I have been serving
on the university committee on
retention, and we’ve been doing a lot of gathering
of data and analyzing that data, and I’ve been very privileged
to work with some really smart folks on that committee. And so I thought I might
share a little bit of what we’ve learned, because some of
the really interesting things when you’re looking at grade
distribution you’re really balancing these two R’s. One is rigor, and the
other is retention, and this is a tricky
balance, and all of you know what I’m saying. So we’ve actually got five
long semester’s worth of data that we’ve looked at,
and so just really quickly in your
head based on what you know from your
own experiences in the classroom with
students, tell me what you think in terms
of grade distribution where do we award
the most grades to freshmen and sophomores? A, B, C, D, or F? Answer that question. All students for the
last five semesters taking freshman/sophomore
level classes– where did we give
the most grades? The answer is? And you would be wrong. The answer is A. It’s
very, very stable. Over the last five semesters,
the largest category of grades have gone to A’s at 34%. At 34%. Between A’s and B’s, that
number is also stable at 63%. Why I tell you this is it’s not
just about I. It’s about we, and what I mean by that is,
if 62% of the students taking freshman and sophomore level
courses at this institution are making A’s and B’s, they
form ideas about their skill sets, and not only one or
two of them form ideas. A big old pack of them form
ideas about how good they are. Some I’m looking at
some of your faces, and I look at my faculty
members back there, and my Leslie that I was
commiserating with yesterday, and by the time they reach
junior/senior level courses, they have ideas about
what they deserve, how entitled they feel
to certain grades. I’m an A student, they tell me. You are doing something wrong
because clearly I’ve got this, and clearly you’re stupid. And honestly I’ve been feeling
pretty stupid recently, so I Lew in this, and my
hair color budget is not big enough anymore for
some of the problems that I have on this front. So just so you have that,
the category of C’s it’s also very, very, very
stable at about 17%. The smallest category grades we
give at the freshman/sophomore level are D’s. So now that you feel like maybe
you’ve used real life data analytics to take a look in
the mirror for UNT global at the freshman/sophomore–
this isn’t my college. This isn’t my department. This is we. This is us. This is what we are doing
together across disciplines. Then let me tell you a little
bit about our DFWI track record. So for those of you who
don’t throw these things around a lot, that’s
the percent of students that ear either a D, an F,
withdraw– either WF or W– or go incomplete, and so
that’s relatively stable at 20% also across the
last five semesters. And so ask yourself
what you think that means in light of the next
statistic that I’ll give you, and that is the top
quartile of the courses that we teach– we teach
about 400 courses a semester– produce 60% to 67% of all
the D’s, F’s, W’s, and I’s that we actually
have students earn. Half of the course
we teach generate 90% of the D’s, F’s, W’s and I’s. Half of the courses we
teach produce those. So we have a two headed monster. On the one hand, we’ve
got some rigor issues that we might want
to take a look at. Now, does that mean that you
do, or you do, or you do? Well, I don’t know. In my department, we look
at performance in courses semester by semester at
a granular level, section by section, course
by course, and we do that as a point of awareness. Lew and I have many
lunch discussions on this because management
does the same thing. We also look at DFWI rates, and
over the last several years, we’ve steadily driven
those down in my department just by keeping– it’s
like getting on a scale. You look at these numbers,
and you become aware of it, but I would say to you from
a data analytics standpoint we need to get smarter. We need to really start
relying on the data. We need to look at courses where
DFWI rates are very, very, very high, and while
it’s very difficult to agree on what that is–
what is a high DFWI percentage? I think that we can agree
on what it shouldn’t be, and it shouldn’t be over 40%. So think about that
with me for a second. If you had a 40% to
50% chance every time you got an airplane of
having it crash, tell me how many times you’d fly. So if our students, every
time they sign up for a course and they pay $1,200
in tuition and fees before they bought
the textbooks, have a 50% chance
of not finishing that course because they’ve made
a D, an F, or they withdrew, how satisfied should we be
if that persists semester after semester? So here’s my little
end of my dialogue to say that I really would
appeal to all of you. It’s about me,
but it’s about us. It’s about me looking at my
performance in the classroom. It’s about me looking
at my students individually, trying to
figure out what they need and if we’re getting it there
what are learning outcome should be, but it’s
got to be about us, and we’ve got to start a
dialogue about failure rates and success rates. We’ve got to start a
dialogue about rigor, and we are way behind
the eight ball on this. We needed to be
started yesterday and last year, and look at the
number of people in this room, and the thing speaks for itself. [APPLAUSE] Interesting. I think as a result I’m going
to change a little bit of what I planned to say. First I’ll just give you a
little background about myself. I teach in the College
of Music, and I’ve been an operatic performer
for about 42 years. This year I was a Grammy nominee
in the best opera recording category, so that’s exciting. [APPLAUSE] And I have over
40 commercial CDs. Most of them are still in
print– plus two films. I’ve taught at
three universities, and there is an
interesting trajectory, I’m sure you would
find, because I’ve gone from Stanford University,
which is an Ivy League, to University of Kentucky,
which is a land grant, to UNT, which you would
describe as regional. However, the music programs
have grown in size and prestige at each institution. At Stanford, I had a
rather entrepreneurial kind of experience. It was in the middle of all
the dot coming and dot bombing, and the environment
of creating the work, and creating the
learning, and all that was very prevalent
and very exciting. Here at UNT, I teach one on one
classes and small classes– one on one lessons
and small classes. Anywhere from 2 to
15, around there. In the past, I’ve taught
larger classes where I’ve produced and directed
opera, created and directed an early music ensemble, early
music being in classical music being music before 1750,
1800, something like that. Almost all of my students
throughout all of us have been advanced
hardworking students. Not all, but most. And my studio includes freshman
through doctoral students, both the performance
majors and concentrations. That’s music ed. I have very few concentrations
right now, only one. The rest are vocal
performance majors. In reference to what
I’ve heard so far, I just wanted to mention
that I, myself, got an F as an undergraduate. That, of course, was in the ’70s
when such things could happen and when school was cheap so you
could just move on, and do it again, or something like that. And it was in one of
the gatekeeper courses. So I managed to
survive that and go on to accomplish all of this. I just wanted to let you
know that some faculty in the College of Music– I
have to take a kind of microcosm approach because I’m not
looking at big data like deans and chairs are on
a regular basis. So I’m going from my
experience outward, but I do know that some
faculty in the College of Music teach courses up to 400,
600 students in size, and the sizes of the classes
range all over the map. In performance alone, we
have chamber ensembles to orchestras, symphonic
bands, et cetera. The music majors–
even the freshman– generally come to
us– a lot of them, anyway– come to us having
already dedicated their lives to music, this being UNT
and a well know program, and they have to
audition and compete to get in quite heavily. A lot of them– I would
say most of them– are more worried about getting
B’s than C’s in their major, at least. We you have some gatekeeper type
courses as exist in all fields, and I think my thoughts
on this during the EC is what ended up me
being invited to this, because we were looking at
these faculty versus DFWI rates and some faculty who have
the hard chore of teaching the course that
students must excel in will have more of those
kinds of grades, I think. I can’t be sure because I’m
not looking at statistics, but I do from
anecdotal conversations that they see more of the
drops, withdrawals, and so on. Music theory and musicianship
is this for music. A musician absolutely
must be able to hear on a complex and highly advanced
level rather quickly rhythm, pitch, harmony, and structure. Being able to analyze as
we do analyze a paper, analyze something
complex in writing– they have to do this through
the organ of the ear. And our students
usually progress. Most of them progress fairly
rapidly in such courses, and they must complete
them before they can take their upper divisional exams. So they’re all kind
of scared of that. They take it generally at the
end of the fourth semester of major study. Sometimes if they transfer
that’s a little bit later, something like that. It’s at that point that some
of them will not do well, and they will have to change
their major if they fail it twice. We try to sometimes encourage
them to change to the BA in music or help facilitate
them into a change of major. We’re all looking right
now at the 2.0 versus 2.75 and all of that in terms
of being able to graduate, and what do we do
with people who can’t make the grade in the major. What will we want
for them at UNT? But we do work in particular
in the College of Music to keep their love
of music undamaged, and I would say that we
are doing better on that just from the gut
feeling compared to 2007 when I first came. Short term incompletes in
music are quite common, and the reason for that
is because even relatively minor illnesses like colds
or injuries like a strain are quite serious for
musicians and must be accommodated
for their long term health and musical success. Our highly competitive
undergraduate students sometimes find this requires
an attitude adjustment, particularly freshman,
sophomores who in high school had been go, go,
go and get to UNT and find out that go, go, go
well get you hurt, hurt, hurt. So these are usually made up. These incompletes are
usually pretty short term. They’re usually made
up within months. They happen in December
quite frequently for singers in particular
because of the weather, and then they’ll be usually
made up in early January. Some of these happen
in the spring– fewer, thanks to the weather–
and occasionally– I even had a recent incident
where we get occasionally students who are
financially strapped, and this particular student
is elsewhere at the moment dealing with his family and
was sick during the juries, and so I’d write to him
from time to time to try to help him get that
finished within a year, because there’s no reason
he can’t at least whatever he’s doing have his
grade in his course. And students may drop or
withdraw from courses, by the way, in music for
any number of reasons, including some
specific to music. For example, vocal students. If they are cast
in the opera, they may quickly realize
that it would behoove them to
drop a core course and take it at a less busy time. Vocal students are also required
to be proficient in Italian, French, and German
pronunciation, so they quickly realize they
should prioritize these courses over the core courses that their
fellow freshmen and sophomores are taking early on. This can happen. Not always, but it does. These guys tend to
take them in the summer to try to make up for that. And finally our graduate
students are outstanding. Many come to us already
active as young professional performers, and although
today we’re focusing primarily on undergraduates,
it’s important to note that at least among our teachers
in the College of Music quite a few are graduate
teaching Fellows. In addition to their
other stellar credentials, a number of these come to us
with quite strong teaching backgrounds. They teach the music
secondary students. For example, if you’re
majoring in clarinet, and you secondary
in piano, so on, they teach them and also
many of the concentration students, which are
the music ed students. And these graduate students
are mentored by the faculty and mentored into
teaching, and also we are then drawn into helping them
mentor their students as well as our own as need arises. That’s it. [APPLAUSE] My name is Jim Meernik. I’m from the Department
of Political Science, and I, too, like Brenda
and Robbie are very much interested in the DFWI I rates. In political science, we
teach two core courses that every student
in Texas has to take. There’s political science 1040
and political science 1050. We have small sections that
may be 20 to 30 students. Most of them now
are above 300, 500. I’ll be teaching
a course this fall that’s capped at about
1,000, and so we’ve got a wide range of
student populations. Everybody’s got to
take these courses, so it’s a great
kind of environment in which to learn about why some
classes may be better or have better DFW rates than others. And so with a colleague of mine,
Victor Prybutok in the College of Business, we set out
to study these courses, and we had a
research assistant go through every syllabus for
our 1040 and 1050 classes and code every single thing
that was done in these courses, whether it was extra
credit, papers, attendance taken, clickers, how much
office hours there were, just every single piece of
information we could get about these classes to see
which classes were more successful than the others,
which ones had the lower DFWI rates. And I expected that
we’d see things like class size would
matter, that more engagement with the
students would matter, just in general more
opportunities for interaction with the student would matter. And so we gather
up all these data for about 24 different
classes over two semesters, and we crunched
the numbers, and we found that virtually nothing
predicted DFWI rates. Couldn’t find anything that
really seemed to matter. And so we did this study,
and at the end of the day, I just felt bad. I went back to Mike Simmons,
who I was working with this on, and said I
can’t seem to find anything that’s going on here. From the perspective of what the
faculty are doing, none of this really seems to be
making a difference, and I was particularly
struck by four faculty who were some of our best
teachers in the department. They’ve all won major
university awards. They were all teaching classes,
basically the same size, all well loved by the students. Two of them had what we
would probably considered to be kind of high DFW rates. The other two, pretty low–
lowest in the department DFW rates. What’s going on there? Maybe there’s something I can
learn by going and talking to them and saying, what
are you doing in this class? Why do you think
you’re rate is up here and somebody else’s
is down here? And I went and talked
to all four of them, and they didn’t have a clue. There wasn’t anything
that they could pinpoint, and so I was kind of left
with statistical explanations. Well, we don’t have
enough sort of variation. Perhaps it’s a whole
constellation of factors. Perhaps there are a variety of
things that these people do, and it’s not any one thing,
but it’s a whole lot of things together. And I went and looked at
the student evaluations thinking maybe a
lot of it’s about whether the students like you. That had nothing to do with
it either, and so at the end the day I had to conclude that
trying to understand these DFW rates is fairly complex. Certainly approaching
it from the perspective of the classes and the
faculty, just that one angle, isn’t going to work. We need to know a lot more about
who these students were not only academically, but also
socially, psychologically. We needed a whole lot more
data on these students to try to understand why it
is that some are dropping out of these classes than others. And so I have to end
my little bit here with a plea for a lot more data. I saw what Jason Simon
was talking about earlier. It looks like there will be
some great data in there. I think we need to do overtime
panel surveys of students to try to really sort this out,
because frankly if somebody comes to me and
says, what can I do? What are some things that
I can do maybe not too easy, but without
a whole lot of work to try to improve my DFW rate? And I’d have to say
I’m not really sure. You can do a whole
lot of things like I think Robbie was showing earlier
that can really have an impact, but short of doing a very
major course overhaul I’m not really sure. So I don’t usually like
to conclude my remarks by saying I don’t know,
but that’s the upshot. So I would love it if others
would collect some more data on this. Thanks. [APPLAUSE] Well, that concludes
our part of this. What we need in order
for this to be a success is for you to ask your questions
that you’ve thought this. Obviously you’re going to
think about in the future, and then as the extra
questions come along, there is a Twitter– is
that correct– feed that has been set up for this
to continue discussions if they occur to you afterward. If anything sparks an idea
for Jim, or for Jennifer, or for Brenda, or for
Lew– if it comes to me, I’ll send it to them– we
can go forward from here. So with that prelim,
do we have any folks that are willing to be brave and
ask their question right now? Mike? You have over 600 teaching
fellows at the university. How do they fit into this
conversation about grades? Well, I think that teaching
fellows and adjunct faculty are very important because we’ve
got a large number of courses taught across the enterprise
by contingent faculty members, and I think it grows ever
more important to pull them in tight, and hold
them close with us, and help them
understand that they’re a vital part of the team,
including giving them feedback on what DFWI rates look like
in their classes over a period of time, what their course
rigor levels look like, and talk to them about
establishing learning outcomes and deciding what skill sets
our students need to learn. I know that program coordinators
in our department do that. They work directly
with adjuncts and then in larger departments
where there are a large number
of teaching fellows I think that’s
important, as well. To go along with that, in
fact, one of our adjuncts is sitting right to my left
right here, Dr. Denise Philpot, teaches not only for us,
but she teaches for Brenda in [INAUDIBLE], and
she would tell you that that report that I told you
about that we’ve been doing now since 1993 all the teaching
fellows are listed in there because they’re teaching. So all of that
information and data comes, is collected,
put together, sent back to everyone
teaching in our department. We obviously don’t walk
around and put it up on the walls in the
College of Business, but anyone is free to
see it if they wish. So teaching fellows– we
feel like teaching fellows and adjuncts– they’re
teaching just like we are. So it should be with
us the same there. In the case of
teaching fellows, we refer to them as
colleagues in training, but they’re still colleagues. They’re teaching. Jim, I got some data for you. At a previous university, the
dean and I did the big study in the sky on what
we could come up with that was a significant
predictor of student evaluations, and we
found one– time of day. That was it. In a teaching
evaluation study that we did for the College of Business
to create a new instrument, we had to learn a
foreign field to us, and that’s the characteristics
of top teachers and things like that, And we
found this great study that said there are
21 characteristics of top teachers. It’s a fairly old study,
and I said, yes, finally. I’m going to learn the
answer, and it turns out any one teacher has
one, maybe two of those. The rest of it is
your style, and so we realized that we needed
to go back somewhere else and start looking for some of
the other things besides just teaching evaluations. About the College of Music
TFs– at least in my department, they are part of
the grading process. We view them and talk
about them as they are so talented
that they are going to be the star teachers and
star performers in the future so that, for example, as
I mentioned that they take the secondaries
in many of– even most of the
concentrations, and there’s an ongoing discussion with the
music ed department about how that sometimes raises
difficulties for them against other schools
in recruitment, but these students that
study with these TFs– we’re going to be long gone,
forgotten when those guys are really in the middle
of their career, and these students
are going to be able to say I studied at
UNT with that teacher. And then also they grade
their own students, and they tend to grade high. So you have to
kind of, you know, but they also comment
very, very encouragingly on each other’s students. They don’t grade our
students, but they’re able to submit grades
for their students, and then the faculty
grade– and they able to submit comments
for the faculty students, and so their grades
and their comments are factored into the
students that they hear. They hear all of
each other’s student and so on, so they get
practice with the jury grading. And then I wanted to tell a joke
that Jim’s comment reminded me of. Anybody remember
the Soviet Union? [LAUGHTER] You remember that? In 1985, I was in Vienna
taking a German course, and Vienna was a neutral
country, and all these students from all over the countries that
we couldn’t visit and couldn’t visit us were there. And one of them was a
16-year-old Russian prodigy, and he told a joke
that in the Soviet there were all these
people running over to one tobacco company. They were buying
all the cigarettes from this one tobacco
company, and that’s not supposed to happen in a
state regulated system where everything is
supposed to be the same. So they asked each of
these tobacco companies what are you putting
in your cigarettes? The answers were
invariably three parts of old newspapers, 20 parts goat
hair, sawdust, rug shavings, this kind of thing. And finally they got to the
one that was so popular, made the same list– goat hair,
shavings, et cetera, newspaper, and two parts tobacco. You’re putting tobacco? So it can take very
little of the real deal to get students really excited. Very, very little. [LAUGHTER] See, that’s good. Jim, I think it’s you. Sure. I don’t know how
to follow that one, but I would encourage
all teaching fellows, all students that want
to be teaching fellows, all faculty to encourage their
graduate students to sign up for the GSTEP program
over in Clear, which is a wonderful place to
get a lot of training, learning how to teach, and
grade, and things like that. I wish that had been
something we had when I was in graduate school. So I was putting in a
plug for that program. Good plug. So there can be a different
purpose for grading [INAUDIBLE] that they do. For instance, in
the Music School, if you’re a music major,
you better [INAUDIBLE]. You can’t. And if you’re a performance
major, [INAUDIBLE] otherwise you get moved into
another major category. And so if you look at the
solutions for grades there, that’s basically it. Your performance major had
better stellar [INAUDIBLE]. What happens is they have a
process of the qualification for disqualification,
where you have to earn your major status
again, and if you don’t get it and you can change your major. And I suspect that in other
performance based disciplines, it’s similar to that. So grades may not
be the total story. Part of the A’s, the have the
score necessity [INAUDIBLE]. Am I explaining that OK? Pretty close. I would say it’s a
little bit softer, that A’s and B’s happen. I’m talking about
undergraduates, and that’s fine in the major. Not very often. Mostly they do probably got A’s. Not always. Not every semester,
and not always. The barrier exam, the upper
divisional exam, is important. They have two
chances to take it. They can still get an A in
their jury and not pass– or an A for the
semester, shall I say, if they do excellent
work in their studio, but they don’t do the
best possible jury, or there are problems that
indicate that getting further along will get in the way. They can be prepared and
not sing well or play well, even though they’re prepared. They maybe have sorts of
physical ailments, or so, or habits, or injury
that’s holding them back, and unfortunately that’s
a real consideration in the real world of music. You can’t just be
prepared and succeed. You have to sound good. And so what we do in
a situation like that is give them a second try. We can’t diagnose,
but we can detect. So we get them to
someone who can and try and get that worked
out if that’s the case. If it’s something else, we
try to help them with that, but generally they can
make a good grade A or B and still not pass
their upper divisional because we don’t hear
them as having progressed. What’s happening
is potentially not going to move towards
the proficiency exam or towards their hearing
for their senior recital in a couple of years, but
usually on the second try they do. This is undergrads,
and their grades might range all over the map. They might get a C or
something in their– as I said, the gatekeeper courses,
but they have still passed, and they might do much better
in their performance courses. The graduate students,
doctoral students can’t get lower than a B.
Master’s students can’t get lower than a C.
We’re wrestling with this at the moment, too, as to
whether a doctoral student can have one C outside
of their major or so. We’ll see where that goes. Mine’s just a follow up on that. [INAUDIBLE] everyone is saying
yes to the first question. What about situations
where it’s [INAUDIBLE]? And we can hear that from just
this whole discussion that’s been going on today, where
students who are not only learning from the classroom,
they’re learning also outside of classrooms,
and you have situations where a
student gets and F, and the next thing
you know years later she’s getting a Grammy award. Or the situation
when [INAUDIBLE] freshman coming out with A’s and
not doing so well [INAUDIBLE] program. So in the art situations where
the first question the answer will be no, what then
can you recommend to be a better grade
distribution in the classroom for the students that can
counter their learning, because it’s not only
inside a classroom that learning happens. So I mean, do we continue with
the A, B, C, D’s, and F’s, or do we transcend beyond that? Maybe it’s accountability
or [INAUDIBLE]? If you don’t mind me
plunging in there since I’m the one who mentioned getting
one, I retook the course, and there’s so much
more baggage on that now that school
is more expensive. I retook it, and
whatever was in the way, I got it out of the way,
and I passed, and probably even did well, and then
it’s many, many years later, and that course– for
one thing, singers are well known to be generally
behind on all that musicianship because we come to music later
than the instrumentalists often. It’s better for us if we
have instrumental background, because then we
are faster, and can fit in with the
instrumentalists, and all of that. We’re in our 20s similar to
them when they’re 10 to 12 in terms of proficiency. So that’s kind of a known
quantity and in music, and even sometimes the
courses are divided up to mostly singers or
most instrumentalists, and singers figure out
which section to take. So that’s part of
it, but I think that the students, if possible,
get it together and retake it. I think Dr. Taylor wants to
weigh in on this because this is one of those really
important conversations that we hear a lot. Yes, we do, and it does
demonstrate such contrast between majors and colleges. Let me give you something
that most of my colleagues aren’t aware of that is
going on that I always talk to them– well, ever since
I learned about it last spring, I talk to my freshman about
it– and that is this. There are over 200 colleges
of business in the US alone that have
adopted exit exams, and those exit exams are
similar to SATs, ACTs, and so forth geared
toward the types of skills that our employers that
hire our graduates want. And the reason that
has happened is because many of our employers
don’t believe us anymore, and they don’t believe us
because of grade inflation, and not necessarily grade
inflation in North Texas, because what you see when you
read these reports and things, then they go to the big schools
like Stanford, and Harvard, and whatever, and they
everybody gets an A. And ultimately
because of that, they have said we wind up
interviewing, and then hiring, and then realizing they don’t
have the types of things we’d like them to have. So there’s a whole
movement that’s been created now that it’s
not something that is mandated by any state government. It is adopted by
colleges who want the employers to keep
coming to their schools and interview their students. Exit exams. Now, every time I mention
that to students it’s interesting what I get. From the freshman,
oh sure, that’s a good idea, but when you
get to seniors– what? I need to get out of here. So in answering
your question, it’s coming to us by the people
who that hire our graduates in the colleges of business. So when you hear that
statistic and think about it, we’re not going to
be immune to that. It’s going to reach
us at some point. I want to say something, too. Yolanda, it’s an
excellent question, and I think Lew is right. We can think of ourselves
as a place where we educate students,
which is one of the joys that I have doing this, or
we could consider ourself a credentialing institution. So we’re certifying that someone
has some sort of skill set, and I think Lew’s
point is bang on, because employers are
increasingly telling us we’re not getting it there. And as we frequently have this
discussion in my department, the grades that are
going out simply don’t match the skill sets that
we see when we are grading. As I so well know, when I’ll
sit down this afternoon to grade essay questions on my last
exam and I look at the reading and writing skills, and when
students at the junior level don’t know how to
calculate percentages, now I don’t consider
that hard math. Does that make sense? That’s not tough stuff, and so
I think that maybe at some point grades will become
irrelevant, because we’re making them irrelevant. That was my point, yeah. And Some other
stuff that’s there, and we may move
to a badge system where employers may have
their own way of determining whether someone has the
skill set that has nothing to do with a credential that we
hand them called a bachelor’s degree. And that would be a
very sad day for us, because that would signal
that we really haven’t been adaptable as an enterprise. To go along with
that, this movement is so new on the exit exams
in colleges of business, we don’t know yet if their
teaching to the test, but some employers probably
want us to teach to the test because it’s basic things
like communication, teamwork, and so forth. When you ask students–
I had a senior, who, in one of my harder classes that
I teach, came into my office and said, I just can’t believe
you’re making us write this. And I said, why? Well, I’ve never had
to write a paper. I said, what? He said I’ve never
had to write a paper, and he was able to hide
through team projects, and he had not written a paper
by himself, a big term paper, and so he was scared to death. So I had to sit down
and work with him, and, hey, here’s what it looks
like, what you need to produce. Well, that was a startling
story that day, for me, and it was recent. On the other hand,
when you go along that with exit exams,
and credentialing, and so forth, who knows where
we’re headed, but it is us. We’re the ones that create this. We’re the ones that manage it. We design it. We do it, and so
if we don’t fix it. Someone is going
to fix it for us. Jim just turned to me when the
word exit exam came up and said that’s what you have in the
College of Music, which we do, and I think I would call that
uncoupling grading and learning from capability, because
capability in performance is delivering on the day. Learning is a whole
different matter. You can do a fantastic
series of rehearsals and then be a little
under the weather when you get up on stage
and the reviewers are out in the audience, and you’ll
get a little bit of a trashing, and it’s not very fun. You still get paid. You still did your duty. You still did the teamwork. You still were a trooper,
but unfortunately your best performance was the
second one where the reviewer wasn’t there. So you have to
please the process, and you have to
please the outcome, and they’re two
different things. I also heard from a Stanford
colleague about a student who had written upon receiving
97.5 grade why is she wasn’t receiving– or he wasn’t. I can’t remember which–
wasn’t receiving an A plus, and so that sparked a
whole discussion of why the 97.5 wasn’t an A plus. And I remember from
my Stanford days that I would get occasional
messages of complaint about A minuses, because we
have the minus and plus system. And that’s a whole
other can of worms, but I do really believe that
we can focus our efforts both on teaching and learning in
the classroom and outcome step by step moments. Also uncouple teaching
from learning, as well, to a certain degree. Jim, you’re OK? Yeah, go ahead. Ask the next question. Sorry. Take the next one. It was mentioned
that, of course, we do want to do
teaching and learning, but we have to look
at the audience. I don’t know about the
statistics at UNT in general, but in my class, I find
that most of my students are working outside
the university more hours than I would
see possible to carry even a half load, and I don’t
see how much more they can do. It was mentioned, too,
that the students learn what they put into the course. If they don’t have hours to put
themselves into the homework, into the reading,
how can they learn? Well, I worked through college. Did anybody else in here work
while they were in school? It’s a never ending
problem, isn’t it? I do know this. My grades went up when
I worked because I managed my time better. I’m not saying that’s an answer. I don’t have an answer for
that one to be honest with you. [INAUDIBLE] I think, though, that speaks
to the need for data analytics, because I had a student
I visited with earlier this week who wanted to finish
up this summer, which involved taking 21 hours of online
courses in the summer session, and so I called her
and spoke to her, and I said you
spell that suicide. That’s a really
tough thing to do, and I’m looking at the
composition of courses and saying these are
tough courses that require a significant
amount of reading and effort outside the classroom. And I think that if I’m seeing
that like you more often and the question begs
itself how can they do that, and the answer is, if there
enough of them are successful, then there has to be
something else at play. And I think we have to
be tough on ourselves, and look for that kind of
thing, and ask should it really be possible for a student
to do that sort of thing. And if they’re coming out
of it, what are we really asking them to do? How much rigor really does
exist in those places? And I think that means
we have to be reflective, and we have to look at
ourselves as instructors. We get worn down. We get binged on every
semester by stuff, and so I’m an A student. I’m an A student,
maybe you can start singing that song with them. I don’t know. The freshman class, you
hear that all the time. Hey, listen. I was in the Plano accelerated
program in high school. I know, but you’re
failing my course. You have to come to class. You have to read the book. So that’s a tough one. This is a tough one. Jim? Yeah, I think our students
face a lot of challenges I think probably many of us
didn’t have to worry about. I think some of the issues with
paying for school– maybe we’re not all aware of how
much working is draining from their schedule, because a
lot of these kids and adults, too, some of the older students,
are working third shift jobs, and they’re coming into my 8
o’clock in the morning class straight from work. They’ve got that issue, and
I think a lot of them, too, even though we think that
they’re a lot more tech savvy than that a lot
of us are, but the fact is a lot of them are coming
from school districts where they don’t have access
to a lot of technology. When they get here,
there’s not only learning about the college lifestyle and
learning about the material. There’s also learning about
all the technology that goes with it, and the
learning systems, and all these different things,
and I think we sometimes overlook that they’re
facing a lot of challenges that we need to be very much
aware of when we’re teaching these courses,
because all it takes is one or two of those things
and these students become at risk pretty quickly when
they really don’t need to be. I think if we just work a
lot more on active engagement and these early alerts to try to
figure out how we can help them with these problems, we can keep
a lot more of them in school. So I was going to address
the question of when the first question
is answered no. Personally I have
had times where I have found myself
trapped by my syllabus into giving grades
that I wouldn’t give in an assessment of
the students learning, because in just designing
the syllabus I put points in in a certain way. It’s easier to award points
on concrete didactic kinds of things because
they’re easy to assess, and that’s really
not where you’re trying to go with learning. Just in terms of learning
how to structure a course so that the mechanisms you use
to report the grades actually report the learning is in and
of itself a pretty challenging thing. Just thought I’d
throw that there. Really that’s what
those employers of ours in the colleges of
business are telling us. We don’t believe your grades. We want them to know
how to do the following or have an awareness
of the following, so that’s why this
movement of exit tests. Dan? [INAUDIBLE] outside the room
is the for profit organization. Correct. Got it. And I think that’s a lot of
the reason why [INAUDIBLE]. The painter Thomas Cole
came up with the phrase an artist is a person and a
business, and for a singer, this it’s really
very, very vivid. We are the product. We maintain the product. We deliver the product,
and so anything you list that has to do with products
and the surrounding activities is the being and the action. And that’s kind of
a shocking thing to become aware of when
you’re a young artist. Thomas Cole’s
trajectory, for example, was headed down the same
path as Vincent van Gogh. In other words, living
out an entire lifetime without ever selling a painting,
and he then sold a painting and describes that as well
as this incredible moment when an artist
gets what he wants. Then after that became the guy
who sold the most paintings sort of ever. Thomas Cole is the Hudson River
School, and to this day we still– it became copied
and even an entity or a thing that took
on a life of its own and became really quite
profitable for him. So just thinking about that. Question? All right. This is just kind of
a summary statement, but what I hear as
I’m listening is that our focus needs
to maybe change to talking more about
learning than about teaching. First we need to figure
out what this beast is, and it has a whole different
body of research tied to it. It’s a paradigm shift
for us to look at it. Let’s get the
neuroscientists in here. I’d like to see more of the
DFW data upon exit interviews and that kind of thing–
just really probe what’s going on with
learning of our students. And when we talk about
coming in off a night shift to take a course, we know that
scientifically, physiologically that’s just not possible. So it’s a different
conversation when we change it to learning
rather than teaching. That it is. Question? [INAUDIBLE] making
sure that students are doing what they should
be doing [INAUDIBLE]. In case you didn’t hear,
the question about what was academic advising’s
role in these conversations, how can they help
us get data, or– OK, are they
playing [INAUDIBLE]? Are they playing a role? And counseling a person
not to take the 8:00 AM classes when getting off the
night shift, helping them [INAUDIBLE]? I think that’s a really, really
super important question. I’m trying to remember what the
advising ratio is here at UNT, but for every one adviser there
are about 450 or so students. And so that realistically
makes it tough on them. We’ve gone to touchpoint
advising for freshmen and sophomores so
that we get them through those first couple of
years with every semester you go in and see an adviser,
but in that period of time these days there’s
so much you have to cover with a
student from everything from they don’t feel like
they like their major. They don’t know what they’re
going to do with their career. They’re a single mother. They’ve got something else, and
so on top of that somehow we have to figure out how
to work in, gee whiz, you don’t seem to perform
well in online classes, and you keep taking them. And this doesn’t
work well for you because you don’t
have the discipline to do an online class
when you go two months and then you decide to sign
on or something on that front. So I think that’s
a real challenge, and it certainly warrants
greater discussion. All of us have all the set of
advisers in our departments. I have an adviser– management
has about 4,000 students in our classes. That’s all the majors
in the college. I have one adviser
who is excellent. She is fabulous if I can
get a student to her. I bring her to the class. I have her come and see the
class so they can meet her. I was at an institution where
all the faculty were assigned students at the university. I had 100 assigned to me. I saw maybe two the whole
time, and even worse I was lousy at it
because I didn’t know all the rules of the university. So advising with
people who are experts is absolutely super critical. We can’t get along
without it, yet I don’t think we have enough of them. When I was at Stanford
for six years, I was a resident fellow
there, and that’s equivalent to a house master. And I would say that this system
of residential education– first of all,
Stanford and Harvard are in areas where it’s
very, very expensive to live so most of the
students, including the graduate students,
live on campus. So you could get at
them, and the other thing is so then they’re within
the residential system. And inside that system is an
amazing now the nuanced peer mentoring plus a lot of
nuanced adult mentoring. Faculty would have
maybe six mentees that they would maybe have
dinner with once a quarter. As a resident fellow, we were
overseeing a student staff that had all that nuance that
would be directly dealing with students’ issues,
discuss them weekly. That’s the big, big difference–
is how much advising goes on, and how layered it
is, and nuanced. Well, one happy thing
I can do as a moderator is remind us what the
two questions were and ask you to reflect on it. So do you ask yourself do my
grade distributions reflect my students’ learning? Is that one of the
questions you ask yourself? Of course it is. And is it all about just me? Well, I think we have
learned that there are many other influences
besides just me, but it is about the way
we structure our course and that kind of fun stuff. So I would ask you to in your
couple of remaining minutes ask your colleagues
at your tables to see if there are any
questions that you can help with or if there’s
any advice that you can give to us as panelists. So let’s take a
couple of minutes, but before we do that,
can we give these people got an A at the beginning
a round of applause. [APPLAUSE]

Leave a Reply

Your email address will not be published. Required fields are marked *