The Abnormal Distribution

Well, folks, the promised “reform of the reform” has started. In my last post, I mentioned some of the troubles the transition from traditional german Diplom to Bachelor’s-Master’s degree courses has caused at my university. The Rectory has made money available for each faculty to discuss the reform of the reform, including overnight accomodation if they want to do it in a “retreat” setting. (I doubt there will be many takers for that. The last thing most German professors wish to do is to hole themselves up in some out-of-the-way place for a couple days with only their faculty colleagues for company.)

Announced last week, my faculty is holding its jaw-jaw next Monday in meeting rooms in some hotel in the center of town (I have thereby discovered a new way to love my dentist, with whom I have an appointment next Monday).

How successful is this going to be? I’ll mention two data points.

First, our degree course in Informatics for the Natural Sciences, and its derivatives, were for many years in the 1990′s and into the 2000′s easily the most successful non-professional course(s) in the university in terms of intake per (tenured) faculty member. And last year our course in Molecular Biotechnology was the most oversubscribed in the university in terms of applications for the limited number of places (by about 20 to 1). So we are doing comparatively pretty well, whatever the complaints.

Second, as far as I know, I am the only member of our faculty with any experience teaching or learning in the Bachelor’s-Master’s model outside of our own university (a good fourteen to fifteen years of teaching in it, in the U.S.). I have degrees from two of the top ten world-ranked universities, and have taught at two of the top ten, in my subjects (as far as such rankings mean anything – see below). I believe I have a feel for what works there, at the best places, and what not. The closest matching experience of anyone else in our faculty is with my two colleagues who were post-docs at ETH-Zürich, which comes in the top-twenty of some people’s lists. But nobody has thought to ask me for my opinion about what is wrong with our study programs and how it may be fixed. Indeed if I were to miss Monday’s discussion altogether I doubt whether anyone would notice.

People who know my writings and contributions to on-line discussion might be surprised that I am locally invisible (except of course when I need to be reprimanded for my poor teaching).

I was one of the two faculty members of my university (which has thousands of faculty) in the mid 1990′s to put most of my teaching materials on the WWW (the other was also a Brit, a computational linguist). Now, more people are doing it, but my material is the most extensive of that of any of my faculty colleagues. So I have been in “e-learning”, as it is called, for fifteen years. I have also been involved in, and made the most extensive content contributions to, two e-learning projects in our faculty led by my colleague who is now the Rector. Despite this, I have been recently invited by our faculty’s e-learning representative to a personal discussion, so that I can find out how e-learning can aid my teaching!

I think this shows at least that we have some communication issues within our faculty. Are we going to be able to solve our course-of-study issues? How likely is it that a group be able to solve a set of pressing organisational problems without soliciting the experience of someone – not necessarily me – who has extensive experience with that form of organisation?

I leave the question as posed.

Ranking lists – a digression

Ranking lists are very variable. For example, The Times/QS ranking has Oxford 5th, Stanford 16th, ETH Zürich 20th, and UC Berkeley 39th. But Berkeley leads the world in the number of Nobel and Nobel-Memorial prize winners, and its Math Department has or had 4 Fields medal winners, out of a total of 48 winners ever, and a couple more who could/should have but didn’t (only Princeton Uni with 6 and IAS Princeton with 5 have more. Cambridge Uni is equal with 4 – but one left for Berkeley ). Compare with University College London and Imperial College, London, at 4th and 5th respectively, which despite their undoubted eminence don’t come close in terms of such winners. Ulinks.com has Stanford at 4th and UC Berkeley at 6th. And UC San Francisco, which is purely a medical school, at 19th! And 4th International Colleges and Universities has the Autonomous National University of Mexico at Number 2, and the Institut Teknologi Bandung in Indonesia at 21st, well ahead of Stanford at 56th. Clearly, any user of these rankings needs to inquire very carefully about the criteria used and whether they are correctly applied.

The university where I teach, the University of Bielefeld, is forty years old this month. It was founded as a «reform university». Everything was in one huge building, as may be seen in this picture. The building has two main social features. First, it encourages interdisciplinary work, since students can hop between a lecture in math, coffee, then a lecture in art history, rather than travelling twenty minutes across town to get to specific lectures, as happens in places such as Hamburg (U.S. and U.K universities have this problem long solved, of course, through the campus structure). Second, since every corridor looks the same (apart from the colored stripe down the wall) in this half-kilometer-long construction, it is easy to get lost. Indeed, the majority of visitors to my office are not my students, but those who see the open door (rare in German workplaces) and a person inside, and say “Pardon me, I am trying to find ….”.

There were also supposed to be major reforms in study structure, I am told. Indeed, there is a whole book which explains the concept. I’m from outside and really can’t see the differences. I got here in 1995, with just an inkling of how Humboldt-style German universities work (gleaned largely from a 6-month Zuse Guest Professorship in Hamburg in 1991) and a thought that it was just the thing for me: I can offer my own classes, whatever I want, I have tenure, reasonable pay, fully-qualified pension. Sounds great, right? I thought so too.

The reality is somewhat different. One heads a “group” (mine consisted of me), and one must compete with colleagues, heading other “groups”, for resources. Others had big ones, indeed up to 35 people or so. Guess which way that goes. Hint: I don’t win. If one does not play the game, but prefers, say, to write one’s research papers (primarily what attracted me) and encourage interested students to join in, then one loses out, and can even hit bottom. For example, I sued the university in administrative court in 2005 because I didn’t have enough resources to get my agreed course offering accomplished (the same colleagues, indeed in practical terms the same colleague, who agreed that I should offer a compulsory module in computer networking then failed to provide me with enough resources to offer the required laboratory. I estimated then that it would have cost about €10,000 out of a total faculty budget, for twelve professors at the time, of about €750,000). What about the 25 students who could only take and therefore get credit for part of the module and thereby not finish their course of study? I understand it was offered to them that they could take a database course instead. Keep in mind that these were primarily creative artists who are using new media and really only care about the HW and SW in so far as it enables them to do magical things with computer screens, video cameras, and sound. They have no use for databases. Summary: there is a lot of show about paying attention to the needs of students, and student groups as a whole are “politically powerful”, but incidents like this demonstrate that some of it is pure show and 25 students do not necessarily get what they had been promised. As I said, the reality is somewhat different from the theory.

So where am I going here? Is it just another over-marinated professor grumping on his blog?

No (that is, not just ). This is going to get a bit long, but take heart! There is something here to offend everyone!

The university held an evening reception of talks and laudatios a week ago Tuesday, and planned a “Gala Reception” for this Saturday evening, to celebrate its 40 years. Glossy brochures were printed and distributed; the Great and the Good were invited, as well as people like me (I declined; Saturday night is the one day a week I have the chance really to cook, and I am loathe to forego it for somebody else’s cooking: nobody here uses enough chili). It is all cancelled, at short notice. A group of “students” (some of whom actually were, I am told) tried to invade the main auditorium during the talking-heads celebration a week ago Tuesday. Doors were locked, but they harrassed people who needed to leave early. There was apparently a large police contingent present (as there was in the centre of town for the demonstration earlier in the day), but “the security of the guests could not be guaranteed” and our Rector Gerhard Sagerer called the celebrations off. And then, a couple of days later, the Gala also. This is serious stuff. My partner heard Gerd’s voice cracking up with sadness when he talked on the radio about it, and I would have felt the same way.

There are real issues about study here. And there is a meta-issue, namely, how these issues are discussed, handled, resolved. The protest is at least as much about the meta-issue as about the underlying issues with course reform. But let me deal here with the real issues.

One real issue is that the previous Rector, Dieter Timmerman, presided over a major change in the way study is organised. The university has junked the traditional Humboldtian model and gone for Bachelor’s-Master’s instead. All within about three years (OK, four). The second real issue is that the university has started charging fees per semester of study. There is no doubt that, whether you approve of them or not, these reforms are a significant administrative achievement.

And then, Prof. Dr. Timmermann turned up in our local newspaper on his last day as Rector, indeed as a civil servant, saying there had been a mistake, and “the pressure during study is too high” (English here, original here, as I mentioned in my last post). Which is, as Heiko Holtkamp remarked when the interview appeared, exactly what the student organisation had been telling Prof. Dr. Timmermann, indeed fighting with him about, the years previously. Well, thanks, buddy! You’re gone, leaving the rest of us to deal with it.

Students are significantly involved in government of the university, and are represented as an equal administrative group with professors, teachers, and administrators, at all levels. This is much different from the situation in most U.S. or U.K. universities. The Rectory (that is indeed the translation ) is holding urgent talks with the student representatives about the reforms.

The upside to this student involvement is that students are stakeholders, forming half of the major purpose of the uni (the other half being research; those who do research are, in the German system, not students), and their needs should be appropriately represented. The downside is the usual litany: they are only here for a couple of years, so are oriented to the short term; they have very limited experience, with teaching, with higher education, indeed with anything much relevant over the age of thirty (except for the fine arts); neither do they have any knowledge about how things are achieved – they are here to learn it, after all. And, last but not least, the organisations and their activities are open to manipulation by political operators who are not really students at all and don’t necessarily care about student goals, such as the people who turn up at demonstrations and smash things.

The disruption was commented last week by Klaus Hurrelmann, laudator at the disturbed celebration and a recently-retired “star” of the university, who is a professional youth sociologist and now heads a research institute in Berlin. He praised the new Bachelor’s-Master’s structure, said there is no going back, commented effectively on the social background to the demonstrations (including how they were partly driven by non-student forces), and urged – this is the German solution to everything – talks between students and Rectory.

The student representatives welcomed the increased offer of urgent talks, but criticised Hurrelmann’s view, saying he had been responsible for the introduction of a three-year course which required 28 different examinations. These are called “Leistungsnachweise”, literally, demonstrations of performance, and usually consist of one of: a seminar talk, a term paper, a 1.5-hour written examination, or a 20-minute oral examination; but may be as little as “participation”, which means physical presence, or handing some object in. (Believe it or not, there is a legal requirement for “demonstrating performance” in a course designated as an “exercise”, equivalent to the small groups U.S. Teaching Assistants hold, say for calculus courses. It is that you have to show up once, or to hand something in, even if it is only a piece of paper with your name and some scribbles on it. The joke about passing if you can spell your name is no joke. It’s The Law.)

Compare with the situation in the typical U.S. university. A full-time student is required to take six courses per semester. Each course will typically have homework (weekly or term paper), a fifty-minute midterm examination, and a three-hour final examination. That is, three “demonstrations of performance” per course, one of which, the final exam, is twice the length of any written exam given by us. That is 18 “demonstrations of performance” per student per semester, about four times as much as the 28 over three years about which the student rep complained to our local newspaper.

Whatever one makes of this, I think one can conclude, from the public statements, that many stakeholders – students, faculty and administration – are, in those well-worn words, “not clear on the concept” of what the Anglo-Saxon study model is.

Some comparisons. U.K. universities attracted nearly 350,000 foreign students in 2007, those from the E.U. paying a few thousand pounds fees and those from outside paying more like ten thousand pounds. U.S. universities attracted a little over half a million, with people paying about 50% more (although fees at the most prestigious can be $30,000 per year or more). German fees are in comparison very low – our students pay €700 per year. Germany attracts about 230,000 foreign students. The student representatives are defending the position that study should be free, and that positions in some course of study or other should be available to all.

What is going to happen with all this? I have no idea what is going to happen tomorrow, or next week, but I can make some long-term predictions.

First, I agree with Hurrelmann that there is no going back. The traditional Humboldtian system was elitist, oriented towards the very clever and self-motivated (for many youngsters, that translates to being obsessed by something), but not very good for those youngsters, the vast majority whom I have seen in my 15 years here, who are intellectually somewhat capable, want to learn something more than they had learnt at school, but are not obsessive, and who then wish to get a job preferably using some of those skills. Unlike our former Rector Timmermann, who deprecated it, I applaud the wish of this majority. Indeed, the U.S. and U.K. systems serve it, offering it to half, respectively almost one-third, of their young people. Indeed, they closely fulfil the wish of our student representatives that a position in some course of study should be available to all.

In parallel, one can note that there is a traditional German system of “career education”, whereby if you decide at 19 that you want to be a bicycle mechanic, you can go study and qualify in bicycle mechanics – but woe betide you if you do not have that qualification, but you are good at fixing bicycles, put all your tools in a panel van, and drive around to where the broken bicycles are, rather than requiring customers to bring them to you, for this is a criminal offence (an actual event, from Bielefeld). This system may work for bicycle mechanics (or not, as some will say) but it certainly doesn’t work for those people with an academic high school qualification (“Abitur”) who decide they want to get stuck in to some history, or economics, or informatics, or all three, and then go work for a financial institution, or manage an info-tech company. Our former Rector Timmermann doesn’t think the universities, or the polytechnics (now called “universities of applied science”, in English, on their signs) should be serving this constituency. Follow that line of reasoning, and you will come to the conclusion that no one will be doing it except for business colleges, yet that is the wish of the vast majority of people who graduate from the academic high schools. How’s that for smart thinking from a professional educationalist?

First prediction: Hurrelmann is right, the students are right: the constituency that wants study and then job will be addressed by the universities and polytechnics, because no one else can address it.

Second, the kind of flexibility needed effectively to offer such courses comes at a price. I have solved my lab problem, above, through two grants of €10,000 in two years which comes directly from student fees. The committee that decides this is dominated by student representatives. Now I am negotiating with the university Facilities Management, which has vacillated on either giving me the room I have (which they already equipped with the necessary infrastructure five years ago) or building a new room (they said they would, but after technical assessment it appears it may cost money that no one has). The point: no fees, no lab. No lab, no twenty students per year who have connected computers up to routers and made a net work. Therefore no twenty graduates per year who can go into Bielefeld’s three hundred small high-tech firms and successfully configure their networks against the internet malware that plagues us all. Capisce?

This money, says the students, should come from central government. Let’s look at the plausibility of this. The health care infrastructure is putting unsustainable demands on its backbone personnel, the nurses (I know, my partner is one). There is increasingly less money available for the standard of care to which Germany is used. The state pension system is currently unable to fulfil its projected obligations, as in many other lands. It has been demonstrated in many lands, including Germany I believe, how much more income the average university graduate earns over a lifetime compared with colleagues who graduated similarly from high school but did not pursue a university/polytechnic degree course. The answer is, universally, lots more money. So, as a moral issue, where should our tax money go? Everyone gets sick, and everyone draws a pension except for those who got real sick, and an elite fragment of society goes to university, and pockets thereby over a lifetime more money than otherwise. It’s very hard for people not to get sick, and very hard not to get retired, but very easy to choose not to go to university because of the up-front cost. So where is the general constituency going to lie on that? If you were finance minister, and had to decide where to put your limited pot of money, would you decide to give it to those worried sick about getting sick and not being able to pay (I’ll guess 70% of the voters, and everyone over the age of 55 – this is Germany!), or those wondering how they will pay for the Bratwurst in their old age (I’ll guess over 60% of the voters – this is Germany!), or those studying in universities who complain about fees (about 2 million in 2008/9, 2.5% of the population).

Second Prediction: central government is not going to choose to put all its money into free university education, it is going to prioritise health care and pensions.

Third, consequent, prediction: fees are going to stay, and they are going to get a lot higher to pay for the flexibility which students need, will want and will come to expect. Just as in the U.S. and U.K.

Next, what about all this pressure on students? Let’s look whence it comes. And now I am going to get a bit technical. The courses are generally based on modules consisting of two to four related courses. Each course is worth 2 or 3 credit points, and each credit point is supposed to represent 30 hours of student time, including attending lectures. We package each course of study with three or four compulsory modules, and the rest as a choice from, oh, about 75 offered in our faculty and the same number offered by other faculties. That is a huge amount of choice. It is comparable to the kinds of things one can do in U.S. universities and because it just sort of happened, magically, over the last years, few people appreciate it as much as they could. Now, with so much choice, there is really only one practical way of assessing performance, which is one or more assessments per course. As we have seen, U.S. Universities generally require three per course. We require at most one. So we are at the minimum level already, there is no room for reduction at the level of a course.

The only way to reduce assessments is therefore to bundle modules together, call these super-modules, and to make one or more assessments per super-module. To make a practical difference, one would have to have about three modules per super-module, that is, twelve courses per supermodule (30 credit points) instead of the current two to four (5 or 10 credit points). And, of course, bundling modules together into super-modules comes with a decreased level of flexibility. Proof by example: a student could no longer do my System Safety I: Accident Analysis, learn our method for causally analysing failure behavior, and spend the rest of hisher time doing bioinformatics. Heshe would have to learn SysSafe I and then SysSafe II: hazard and risk analysis (involving all that probability calculation, and we all know how students just love math! ) and do Applied Logic I and II (all that delicious symbol manipulation which some people, incomprehensibly to me, find boring and hard). I get 20 people in SysSafe I, 10-15 in SysSafe II, and so far 3 in Applied Logic – which is a new course and the three are philosophy students who were intrigued by the books the bookstore ordered for me (doesn’t happen with our informatics students, many of whom can’t read books ). If I were to bundle these modules together, I would imagine I would get three or four for the entire super-module, and the philosophy students now learning logic from me would have nowhere to go (my modal-logician colleague in linguistics is overloaded with other things, and we are the only two research-qualified logicians in the university).

Fourth Prediction: Flexibility will be reduced, exactly the opposite of what everyone wants, but the only way to achieve reduced assessment, which will take priority because it is tangible whereas the benefits of flexibility are less so. The compulsory modules in a course of study will be bundled together as far as possible and one assessment will be given per module. The non-compulsory modules will remain as is, with one assessment per course, because you can, tangibly, mix and match courses within a module.

The students don’t realise how risky this is. If you walk in to a half-hour exam, face a grumpy examiner who asks you a bunch of esoteric questions to find out how “clever” you are, and then gives you a poor grade because you concentrated on the wrong thing, you are going to have failed 900 hours of your time (30 credit points, 30 hours per credit point). That is half a year’s full time work (“full-time work” in Germany is about 1800 hours, although it used to be less). That used to happen all the time in the traditional system and is hard to avoid also as an examiner with benign intent, but a student at that time could go away and come back a month later, or six months later, and make good. In the new system, there generally won’t be the time for that.

Fifth prediction: there will continue to be complaints about assessment. This will not be sorted out to everyone’s satisfaction. But students won’t disrupt university events because of it, because it is too esoteric and the political “smashers-up” won’t be motivated.

I conclude with a point that personally means a lot to me. As a faculty, we have worked together hard to put together this new system for our students. We did it intelligently (maybe intelligence is an emergent property ) by tasking an enthusiastic Private Docent with designing the whole new courses of study out of the existing offerings and implicitly promising him we would do whatever he asked (including decide promptly, when he asked for a decision). Miraculously, this worked, and we rewarded him appropriately. We have six new Bachelor’s-Master’s courses of study and to my mind they are all coherent. We have a published module handbook with 150 modules representing over 300 individual courses in it, regularly updated. Did we do it for ourselves? Hell, no. I’d rather be giving a half-dozen half-hour exams a semester and spend my time on my research, as I did before, rather than assessing each of 80-120 pieces of student work per semester, filling in the assessment forms, and fighting with my colleagues as to which form of assessment is appropriate. We did it and do it, we put in this time and effort, because we think, rightly or wrongly, that it is better for the young people who come to us to learn.

It would be nice if this goodwill could be recognised during negotiations. I would be deliriously happy if everyone who came to us for education loved every course and got straight A’s (or 1,0′s as it is here). It has never happened in the history of the world and won’t happen here. We have to do well by the failures as well as by the successes, and this is very much harder. As shown, for those who have not had enough of this by now, in this essay.

I was saddened yesterday to learn of the death of John Stallings exactly one year before. John was a mercurial Berkeley mathematician of occasional genius who, if you believed him, mostly enjoyed sleeping, doing nothing, and various scurrilous activities. Including, on the level of barely scurrilous, BSing with graduate students such as myself and my pals about how everything was so pointless you might as well occasionally do a bit of math. Except – and here is the difference – the bits he did occasionally set the world on fire.

The Berkeley Math Department still has his home page up. Please take a look and read especially (if you like math) his note on How not to prove the Poincare conjecture, and his famous reply to a research institute performing a survey on how professors spend their time. For a more sober non-self-assessment, the University obituary hints at just how really good he was.

Today I translated into English an interview given to our local newspaper, the Neue Westfalische (NW for short) by the departing Rector of the University of Bielefeld, Prof. Dr. Dieter Timmermann. The Rector is the academic head of the university. In his office as Rector, Professor Timmermann was a very snappy dresser with a perfect haircut and perfect suit and tie, as befits a man of importance in German society. He presided over a huge transformation of study at the University of Bielefeld, from the traditional german Diplom structure, described briefly in my old note, to a Bachelor’s-Master’s-based structure that is supposed to emulate the “Anglo-Saxon” model of university study. But he seems to think it went wrong. It is not surprising that not everything is perfect – very few German professors (for example, none besides me in my faculty) have any experience of teaching in, say, the U.S., the U.K., or Australia, whence this new model for Germany supposedly came. Imagine if, say, the University of Salford were to move to a traditional German model: take six years to do anything you like, find yourself, take a couple of short oral exams in the middle and a few more at the end, then write a 60pp research thesis in six months or more. Would everything go right? First, one has to know what “right” means! If you change your model so completely, it is no longer so clear. And we might wonder if this is the case now in Bielefeld for many of my colleagues.

I think the views expressed by Prof. Dr. Timmermann will astonish readers in U.K. or U.S. Universities, as they astonished me. Here I just want to point out how surprisingly consonant they are with those of John Stallings (as regards university work, I hasten to add!). More different social creatures in the professorial calling would be hard to imagine!

Where could such a similarity originate? It could originate in general ideas about what a university is for. Germany’s is often said to arise from Alexander von Humboldt’s ideas from 200 years ago, which, in a bowdlerised version that nevertheless reflects a popular conception in German university education, gives lonely original thinkers a place to expound their ideas, surrounded by those willing to learn from them. (This idea is consonant with the medieval idea of a university, except “lonely original thinkers” there were mostly called heretics!) If I had been an algebraic topologist, I could have learned a lot from John, as my pals did. And, people (such as my son’s former teacher) tell me that, if I were a school-level educationalist, I could learn a lot from Prof. Dr. Timmermann.

Another, less complimentary, idea might be termed “the arrogance of the clever”. If you are a mathematician of genius, it doesn’t matter what you do at university, someone will still give you a prestigious job to do what you’re good at, even if you only do it a couple hours a day (but you still have to teach calculus, at least at a U.S. university ). Similarly, if you are a good economist, you can play soccer your first year or two in university and still write your Ph.D. later. My reason for calling it “arrogance” comes from the observation attributed to Thomas Alva Edison that genius is 1% inspiration and 99% perspiration, meaning that even if you’re clever, you don’t get anything except by working hard at it. Indeed, thinking of John’s example one could almost say that genius is 1% inspiration, 60% perspiration and 39% dissimulation, namely hiding the fact you’re breaking a sweat over it at all.

Whatever. In today’s anxious world it is hard to imagine a character like John springing up unformed from under a Little Rock and stubbornly remaining so while pursuing a sometimes spectacularly brilliant career. I feel privileged to have known him and sad that he is gone.

The report on the RAF Nimrod accident in 2006 has recently come out and at least British safety engineers regard it as a major event. This is a milestone, and could be a watershed event, in system safety engineering in Britain.

Put briefly, the report found that there have been various technical questions about the Nimrod design dating back to 1969. In 1979 the aircraft were modifed with a supplementary cooling pack, some of whose hot lines were in proximity to parts of the fuel system. In 1989, the Nimrods were modifed for air-to-air refuelling, which allowed the possibility of overflow of fuel into the parts of the aircraft occupied by the hot line of the supplementary cooling pack. Indeed, this happened in 2006, the aircraft caught fire and the aircraft and crew were lost. This from a summary article in The Daily Telegraph recommended by John McDermid, who has also privately cautioned against drawing conclusions without having read the report.

Which I am about to do (both. But first, here, the drawing of conclusions )

There was a Safety Case prepared in this decade (that is, after the mods implicated in the accident) by BAe Systems. The Safety Case was criticised in the report as being incomplete and badly argued (as were the preparers and their MoD supervision – and these people are named!). I do not need persuading that many important Safety Cases are junk. Indeed, I have written publically on it. But this one landed on the first page of British newspapers.

I think it very likely that one of the consequences of the report will be increased attention paid to the quality of Safety Cases.

One obvious way to improve the quality of Safety Cases is to improve the quality of the argument in them. One could even say that these are the same: a Safety Case is nothing more nor less than one long argument. It is an argument that a system is sufficiently safe (whatever the criteria may be) to operate. And it is meant to reflect reality, which it can only do if the argument is valid (argues from true premisses to true conclusions).

So one obvious way to improve the quality of Safety Case is to check its quality – that is, to have it reviewed by experts in its subject matter. The common subject matter of all Safety Cases is argumentation. The study of argument is known as logic and the people who study argumentation are known as logicians.

Philosophy undergraduates are usually required to take at least two courses (one full year) of logic, no matter where they study. Our computer science students in Bielefeld study logic first in math (where they might be introduced to Boolean algebra), then in Theoretical Computer Science (where they are introduced to propositional logic and the syntax of predicate logic) and those who do a HW track will have to do some more in Technical Informatics (i.e. HW, which goes as far as expecting people to draw and evaluate Karnaugh diagrams and use the Quine-McCluskey minimisation algorithm).

I used to examine these people orally. Remember they had had three courses which handled formal logic. To see what they knew about propositional logic, for example, I used to introduce a sentence such as A IMPLIES B OR A and ask them what it meant according to the traditional semantics. About two thirds of them would get their knickers in a twist. Some might say “it’s a tautology” and then I’d say “no, it’s not!”. Others might say “it can’t be simplified any further, so it means what it says” and then I’d say “no, it can be simplified to TRUE”. And then I would ask what was going on. Most couldn’t say. Then I would point out that the meaning was not usually given by syntactic equivalence (that is, reading the sentence out loud and saying what other sentences it was the same as or different from). So how is it given?

So, Exercise 1 for readers of this note has two parts: first, what’s going on here with this formula? Second, how is its meaning given?

This is trivial stuff with which most philosophy students with a course in logic would have no problem whatsoever. But most computer science courses do not require any kind of understanding of basic argument forms. Neither do most engineering courses. Neither, indeed, does the University of York Certificate in System Safety Engineering. Nor its M.Sc. in Safety-Critical Systems Engineering

Here is Mr. Haddon-Cave QC saying that poor argumentation and its uncritical acceptance is largely responsible for the deaths of 14 people and loss of an aircraft. Thank you, Mr. Haddon-Cave! Some of us have been saying such things could happen for years, but you got it on the front page of the newspapers!

And no wonder we are in such a mess. Imagine if we allowed civil engineering firms to do business without anyone in them having any demonstrated training, experience or ability in Newtonian statics.

But traditional philosophy courses in logic will not suffice. The kind of facility with argumentation which is required for Safety Cases is far more than traditional mathematical logic (by which I mean classical propositional and predicate logic). It requires also the ability to understand the quality of inductive arguments (by which I mean arguments involving probabilities), causal arguments (which goes over and above classical logic into the domain of what is called modal logic), and the ability to take a piece of running text, represent the form of the argument which it presents, and analyse its quality. This latter skill is trained in many philosophy courses of study, where it is often called “informal logic”, which my former Berkeley-student-colleague John Burgess justifiably calls in his important new book an oxymoron. But I do not know of any computer science or engineering courses of study which require such a course or which test their students in any way on their skills in analysing text-based arguments. I will call the three areas of logic, which I have singled out, as follows: classical logic (classical propositional and first-order predicate logic), philosophical logic (the term Burgess uses for the non-classical-logical formal reasoning, such as probability-based inductive reasoning, temporal reasoning, and modal reasoning including – if one believes David Lewis et al. – causal reasoning), and text-based logic (“informal logic”).

Now we, as inter alia teachers of system safety, have the time, opportunity, and encouragement to do something about this situation.

I propose that a demonstrated ability in classical logic, some forms of philosophical logic, and text-based logic should be required for any formal qualification in system safety engineering. This likely won’t help us for the next twenty years, unless there is some magical way of making this requirement retroactive, but then it will have an impact.

Examples

This is so far an abstract argument without examples, so maybe I should provide some to back up
the manifesto.

I gave a paper analysing TCAS in 2004 to the annual Australian Safety-Critical Systems Club conference then in Brisbane. When I submitted the written version for review (a formality, since the paper was invited), one of the reviewers, an engineer with extensive TCAS background, who had indeed given a paper on TCAS to the same conference, professed confusion as to my first point that there had obviously been a TCAS requirements failure.

The BFU report into the Überlingen midair collision on 1 July 2002 glossed over possible requirements failures, even though there was a sentence acknowledging it in the report (try to find it!).

Yet Eurocontrol had known of this specific failure for at least two years before the accident and had filed a change request addressing it explicitly with RTCA in 2000. (This change request, known as CP112, in its by-then extended version CP112E, finally made it into the TCAS Version 7.1 standard in April 2008, some 7+ years, numerous incidents, and one fatal midair collision later).

The BFU Überlingen report acknowledged that, had the phenomenon I am calling a requirements failure not been manifested, the aircraft would have missed each other. This was the conclusion of simulationss apparently performed at their request by the Eurocontrol ACASA project (the exact results of those simulations have to my knowledge not been made public). That means, explicitly, that according to the Counterfactual Test the requirements failure is a necessary causal factor in the accident. Yet it does not appear in the list of causal factors at the end of the BFU report. This is clearly a causal-logical fallacy in one of the most widely-read causal analyses of our time. It is not the only one in the report. Yet most of the people I talk to accept fallacy-written accident analyses as routine. It is time we started applying higher standards.

Some years ago I took apart the Eurocontrol Safety Case for RVSM in European Airspace (see the WWW page on it. It didn’t take long. I was invited by the Eurocontrol RVSM Project Leader Joe Sultana, and the Safety Case author, Dr. Bernd Tiemeyer, to a discussion about my critique at Eurocontrol. Such appropriate and gracious gestures, and an interest in resolving a critique, are still all too rare amongst us engineers.

The gentlemen seemed to want to explain to me how I had misunderstood the report and my conclusions were mistaken. Since my argument was simple, based explicitly on the text, and correct, readers can imagine that they had an uphill battle One of their main points, which indeed required further analysis on my part, was that they had used a critical premiss which was the explicit result of research performed in Eurocontrol’s ACASA project. They gave me a huge folder of reports from ACASA – I guess 1,000 pp or so. I was able to review these quite quickly, since my argument was simple and the needed premiss quite clear in its use, and I determined that the work ACASA had performed did not justify the specific conclusion which the RVSM Safety Case had used as a premiss (in fact, I would argue that the RVSM people had reinterpreted the statement: as ACASA made it, it seemed to be a justifiable conclusion of their work, but the RVSM people wanted to take it to say something somewhat different. In other words, the statement as written was ambiguous).

I was at the IET System Safety Conference last week, and Bernd Sieker reported to me that in a talk about ATM issues, someone had identified themselves as the author of the Euro-RVSM Safety Case and that “despite what you may read on certain WWW sites” the Safety Case is correct (exemplary? I am not sure what degree of accolade was asserted). I didn’t see Dr. Tiemeyer at the conference. (Maybe it was the author of the Post-Implementation Safety Case? But then, I didn’t write anything about that because it wasn’t available at the time.) The following question arises: why hasn’t Eurocontrol succeeded in understanding what is wrong with its Safety Case argumentation in all these years? And fixed it? (It probably can’t be fixed in the way Eurocontrol might like, but at least it can be rewritten to draw correct conclusions from the evidence they adduce.) The answer could be because they don’t have anyone with the specific ability and formal training in argument to fix it.

Other examples, including some decisive and ultimately quite expensive ones, lie within the scope of non-disclosure agreements.