I remember first realising how true it is that hard facts can be socially constructed by reading John Searle’s brilliant book The Construction of Social Reality. There are lots of facts about money. Money is just as real as my house. But it is purely a social construction. That doesn’t mean that “all truths are relative to the truthsayer”, as people such as the late Richard Rorty notoriously claimed. Many psychologists one encounters like to speak of what is “true for you” or “true for him”, but many of them still intuitively know they can be flattened by the bus they hadn’t seen when they try to cross the road and don’t look first, whatever they might think their truth consists in. Still, there is real meat, not just relative meat, in the debate between “realists” and “relativists”. Robert Nozick wrote a fine book, Invariances, on it in 2002, in which he argues truth is relative and tries to show to what. A decade ago, even I made my own minor contribution to the debate around the Sokal hoax and the Bricmont-Sokal review of some sociological thinking about science. But all that is not really my point here.

Before I get to that point, let me make a relevant plug. I recently read Harry Collins’ lastest book, Gravity’s Ghost, which has a brilliant example of a social construction in science. Collins is notorious among some scientists (and some scientific philosophers, I think) for claiming that much of science is a social construction. No, say the scientists, it is about reality and fact (whatever those are, add the philosophers). Harry has studied the gravitational-wave community, the people trying to detect them, for many years, and far from playing one of those “truth-is-what-you-want-it-to-be” sociologists, he looks to have gone native. And he tells a great story. The book reads like a novel, and is quite short. If you want to know how “big science” is done, it is hard to think of a more entertaining introduction. The book also has one of the finest introductions I have read to the difficulties of statistical reasoning as a guide to reality, with insights I cannot recall having read elsewhere. I probably don’t agree with all of it, but I enjoyed reading it and wasn’t in critical mode as I did so.

One of the major problems, perhaps the major problem after that of money, in gravity-wave science, is knowing when you have seen something. Collins has a wonderful account of how a phenomenon was reified by vote. And the vote wasn’t unanimous! (Buy the book! Read it! End of Plug!)

I talk here about a social construction which worries me. Crimes and torts are being constructed out of Internet activity. I don’t mean here crimes and torts as traditionally conceived, which may be committed in ways enabled by internet technology. The Economist once argued against constructing specific internet crimes out of activities which are already proscribed, but which are enabled in different ways by new technology (in the 1990′s; I am sorry that I no longer have the reference). As a classical liberal in my political thinking, I am very wary of the invention of new crimes specific to the internet, or reinterpreting old categories of proscribed behavior in inventive new ways to cover internet activity.

When working in California in the 1980′s, I felt the consequences of the Morris Internet Worm. Robert Tappan Morris Jr. was widely vilified for having released the worm, and eventually convicted of a crime. I felt at the time that that was desperately all wrong. There is no doubt that it was a watershed moment. Companies and organisations were installing and selling actefacts that I felt were not completely fit for purpose. Specifically, large public companies were selling proprietary versions of the Unix operating system, with the Sendmail program compiled with the debug flag set, thus allowing a back door into root privilege on their customers’ computers. I felt that was simply negligent. That was just one example of people being unwise, arguably negligent, and enabling possible damage to innocent others. I was concerned that such things were becoming a major problem on the internet in those days (I long consider this to have been decisively proved!).

What is one going to do about it? Since such habits constitute force majeur, any action must be political – you must bring it inconveniently to people’s attention, because they will ignore you, as they had been doing for years, if you just say “I don’t think this is a good idea” and leave it at that. But inconvenience too many people, and some of them are likely to try to criminalise your activity; force majeur again. It’s a dilemma; there is a fine line.

We now know well the competing philosophies. One (mine) says that Morris was performing what was, at that time, for that act, a legitimate public service. The public needs to be shown how bad things are, once, to start serious public debate and hopefully rectify the situation. That did not turn out to be the majority view. The majority view added up the collective resources spent on combating the worm (stupidly or intelligently, all of it equal), puts it in dollar terms (or equivalent measure) and says to Morris “that is the damage you have caused“. And that is the view that prevails nowadays. That is the view proposed by the U.S. Department of Defence, which has added up all the resources it claimed were compromised by Gary McKinnon’s hack into and through their WWW sites. The fact that they were apparently using insecure software for a site that, by their own arguments, was essential for US national security is apparently no longer considered a significant and culpable act of negligence. I feel strongly that such unfortunate events as the McKinnon hacking episode and attempted extradition would not be happening had we collectively chosen the other reaction to Morris’s escapade some twenty-odd years ago; holding the owners of frangible systems more responsible for the effects of that frangibility.

Which is why I am exercised by what I take to be the invention of a tort. An article by Lauren Weinstein in today’s Risks 26.78 points to an interesting article by a Business-School computer scientist, Panos Ipeirotis, written a year ago, on a scheme for making money from internet advertising.

Someone set up a bunch of domains with benign names to host invisible ads which were then automatically clicked. The ads were of the pay-per-click variety, and the assessment was only made on the “host” site. However, as far as I understand it, the money went elsewhere.

What puzzles me, indeed slightly disturbs me, is that Ipeirotis and Weinstein describe this as “fraud”, and I presume the WSJ, which picked up on the story, does too. Yes, things that are happening and actions that are being taken are not what they seem, for example, an ad on a 0×0 frame cannot be “seen” by any human, the “click” on it is automatic, and a frame that is loaded is not necessarily the same frame that a cursory scan of the source code might lead you to think it was. But all of these things are commonplace in Internet commerce as it is currently practiced.

I wrote down what I think fraud is. Maybe readers would like to try it for themselves before reading below what I wrote, or what others wrote. I wrote

Fraud is a category of human behavior in which one party is led to believe something which is untrue or misleading by a second party, and is thereby led to engage in a valuable-goods or financial transaction whose nature is not in fact what the first party believes it to be; and where the second party benefits and the first party suffers deprivation.

I then looked it up in James’ Introduction to English Law, which is a standard short reference. It is part of the law of contract. There is something called a “representation”, which is a statement, in this case about goods or material involved in a potential transaction. An untrue representation is called a “misrepresentation”. The tort of “fraud” (or “deceit”) is committed when

a person makes a false representation of fact, knowing it to be false, or without believing it to be true, or recklessly, careless whether it be true or false. The false representation must , further, be made with the intention that it is to be acted upon by the party deceived, and if his claim is to succeed this person must prove that he actually did act upon it to his detriment. (Italics in original omitted)

So I got it more or less right, except that the second party does not need to benefit. A few moments thought shows why.

Now, I am sure that the notion of fraud has been extended to electronic actions, such as those made by automated trading devices, in various ways. But it seems to me that a notion of “misrepresentation” would be key to characterising something as fraud. Advertising is rife with misleading representation, but that misreprentation has to do with perceived content, and there is a lot of law based around that. According to the definition in James’, above, a null representation, such as that shown in a 0×0 frame, would not count as a misrepresentation, for it is not untrue; and it is not untrue for it simply is not!

If something is commonplace, such as pop-under windows and suchlike, or dummy frames such as those with size 0×0, then it is surely not clear in what way there is a misrepresentation (in this technical sense). Saying nothing (or putting an ad in a 0×0 frame) even when you know something, does not constitute fraud: caveat emptor. And whether you are paid for “clicks” surely depends on what the contract says it pays for, and I doubt it says that only clicks are paid for which are made by a bona fide human sitting at a keyboard or tapping on hisher iPhone with intent to view. And in a world in which people pay real money for virtual artefacts on games such as Second Life, it is hard to rely on one’s moral intuition to determine when paying money for something or nothing is “OK” and when it is “not OK”.

If we don’t like what this person is doing, and we collectively decide to proscribe it, then let’s approach our lawmakers and persuade them to do so. Let us not put new whines in old torts.

Dear Sirs and Mesdames,

You are performing a study of the effects of solar storms on UK infrastructure and asked for evidence to be submitted to this e-mail address by mid-April. I am hereby responding to that call.

Many nuclear power plants are dependent upon continuing supplies of electricity to support not only normal but emergency operations. Cooling systems, including emergency cooling systems, in most plants are dependent on continual (and continually reliable) supplies of electricity for control, and in some cases also operation.

The station blackout at the Fukushima Daiichi plant a year ago focused some attention (ours as well) on the vulnerability of such plants to design and operational assumptions which, in my view, a thorough hazard analysis of the modern variety would and should have made apparent.

Electricity supply to essential functions at nuclear power plants is provided “in depth”, that is, by redundant systems. The BWRs with Mark 1 containment at Fukushima Daiichi obtained power first self-generated, then from external grid supply, then from on-site diesel generators, finally from batteries. The self-generated power was lost upon shutdown, a response to the Tohoku earthquake, which also took out the external grid supply. Nearly an hour later, the diesel generators were flooded by the tsunami and the only electricity supply available became the batteries, which were scoped to supply for 8 hours. The physical requirement in that situation, though, was for far longer than that and the result (of that as well as other damage) was meltdown. I observe that, until recently, the requirement for battery power supply in “station blackout” conditions in some US power stations was only 4 hours. The US Nuclear Regulatory Commission has recently reconsidered that requirement.

One salient engineering phenomenon is that the satisfactory operation of many of these systems was predicated on independent failure of systems. For example severe ground movement and flooding seem to have been taken as independent events by the designers and builders, as far as we know. Whereas at Fukushima Daiichi those events had a common cause, namely the Tohoku earthquake. We see this phenomenon, an assumption of independence vitiated by common-cause events, time and again in engineering.

I understand that a recent solar storm in 1989 took out the grid power supply in Quebec for about nine hours http://en.wikipedia.org/wiki/March_1989_geomagnetic_storm . So it is possible for the consequences of a solar storm to exceed the “design basis” requirement for emergency-system operation at at many nuclear power plants of the BWR design (I understand well that the UK has no plants of similar design; I use this simply as an example of how design assumptions and physical reality may not always connect well).

However, I know of no current public study of the effect of a solar storm (coronal mass ejection, CME) on US nuclear power plants, although recent articles in the New York Times by journalist Matthew Wald have detailed some current thinking and practical exercises to install emergency power generators: http://green.blogs.nytimes.com/2012/03/19/a-speed-record-on-the-power-grid/

A Fellow of your institution, Martyn Thomas, has given talks recently on the possible consequences of solar storms on some engineered systems. I am in regular contact with Martyn. He gave a talk at the Workshop I organised last August on the Fukushima Daiichi accident, and has recently given a talk at the Safety-Critical Systems Symposium in Bristol in February 2012, which was filmed by the IET at http://scpro.streamuk.com/uk/player/Default.aspx?wid=12667&ptid=32&t=0 When Martyn asked during his talk who explicitly considered Carrington-type events in their hazard analyses, I was apparently the only person to respond positively. (I chair a standardisation committee in Germany which is performing a hazard analysis of charging electric road vehicles; we explicitly consider solar storms.) That suggests to me that awareness of the consequences of severe solar storms on UK infrastructure is not very high amongst even safety engineers. I would hope that your study could help remedy that.

The issues with hazard analysis and mitigation concerning complex safety-critical systems such as nuclear power plants are not trivial, and this is not the place to list them. But the most salient characteristic which came to light in our work is that the assumptions about what can happen which constitute what the US calls the “design basis” for these plants can be obscure, sometimes outmoded, and, as at Fukushima Daiichi, inappropriate. (It is particularly noteworthy that the vulnerability of the diesel emergency generation to flooding had been pointed out explicitly, most notably by the sociologist Charles Perrow in his 2007 book “The Next Catastrophe”. The possibility of station blackout of BWR designs due to flooding was not exactly an obscure phenomenon. How did the engineers miss that? We don’t know yet, although we have some hints. The answer will be given in my view by sociologists, not by the engineers themselves.)

I looked in the material on the WWW site of the Office of Nuclear Regulation for mention and consideration of solar storms, and found just one document from the Cabot Institute at Bristol University: http://www.hse.gov.uk/nuclear/fukushima/submissions/226920.pdf So the ONR is aware of the potential for such storms, but consequences of such storms are not considered at all in the vulnerability analysis which ONR performed at the request of the government in 2011: http://www.hse.gov.uk/nuclear/fukushima/final-report.pdf

I do think it essential that a careful analysis of the effect of severe solar storms on the safety and emergency infrastructure of nuclear power plants be performed. I think special attention should be paid to the general engineering problem of considering the issue of common-cause failures of kit whose design assumptions include independent failure.

I am moderately sure that you (and the ONR) will be aware of many of these issues already. I hope you understand, though, my desire to ensure that they are considered and hence this note.

Sincerely,

PBL

For those interested in analyses of the Fukushima Daiichi accident, I have a paper on it with clickable links (unlike the version in the Proceedings). I gratefully acknowledge the agreement of the Proceedings publisher, Springer-Verlag, for me to include the paper on our WWW site and note that the final publication from Springer is available at www.springerlink.com in the book Achieving Systems Safety, edited by Chris Dale and Tom Anderson. A video of the accompanying talk is also on IET.tv

Steven Tockey pointed to an article in the Huffington Post about delivering tacos by robotic helicopter. Apparently there is a Silicon Valley company of three people called Tacocopter (why not Tacopter?) which wants to use small robotic helicopters to deliver tacos ordered by your smartphone before they get cold. Huffington Post spoke with Star Simpson, one of the founders. She is reported to have had something to do with the MIT Media Lab Personal Robots Group, but she doesn’t turn up on the list of people, including alumni, on their WWW site.

The FAA says, of course, “you can’t fly drones in civil airspace”. Ms. Simpson’s characterisation of the “obstacles” to getting started is telling:

It’s really the legal obstacles in the U.S. that seem insurmountable at this time.

To which the journalist comments:

So, there you have it: The U.S. government is single-handedly preventing you from ordering a taco and having it delivered to you by a totally sweet pilot-less helicopter. So get out your pitchforks, sign those petitions, start calling your local congressmen, and let them know: We want our tacos hurled at us by giant buzzing robotic helicopters, and we want them now.

….. which does rather give me the impression that he doesn’t take it all quite as seriously as the founder.

Let’s see. Is it just “legal obstacles”? Well, as the journalist points out, there are plenty of real ones in the way of flying helicopters in urban areas, so one should surely also think of safety and liability insurance.

When it comes to buying liability insurance Ms. Simpson might well find that companies won’t do it and she has to ask Lloyds. The price of insurance is bound to include the almost-certainty that there will be at least one accident. What is one accident likely to cost? The cost of the aircraft (almost all that seriously crash are written off), plus the cost of repairs to or replacement of any infrastructure and personal goods that are damaged along with it, not to speak of the cost of people possibly being hurt. So take that, add insurer’s administrative costs, as well as a bit of profit on top. That’s just for one accident. How many might there be? The first accident might well shut the company down, because that’s what happens to very small airlines which have an accident. (Airline you say? She will in fact be running a cargo airline. I am not sure how good an idea it is to let the FAA know in advance that you think the Federal Aviation Regulations stand in the way of your business. The FAA is likely to point out that the FARs do a very good job of assuring safety in what used to be the very risky business of flying – and they are right!) So Tacocopter would either have to self-insure, if they are rich enough, or have to interest someone in the idea who is rich enough to self-insure, and most of those people are very interested in the business model, for that is how they got rich in the first place.

So let’s look at the business model. How many people are going to be willing to pay more than double the usual price for a taco so delivered? (I am thinking here that the cost of a new private airplane in the US is -still- over 50% liability insurance for the manufacturer.) It might indeed be a nice party trick. Then again, those wealthy enough to pay double the price for food might well care a lot about the quality of that food, so turning up in a van with a cooker in the back and preparing tacos on the spot might generate far more business, and is obviously an easy way to ensure the freshness of the delivered product. As well as not requiring the FAA to change the Federal Aviation Regulations.

Ms. Simpson has surely also taken the five minutes to think about such things. It would be nice to have read her answers.

But there is a more general phenomenon here worth remarking. There appears to me to be a blind spot amongst mobile-robot enthusiasts and researchers concerning safety, and this seems to occur in this article also.

This issue is “close to home” in the following sense. Until the end of October I run a research group in Interactive Safety at the “Cognitive Interaction Technology” research lab in Bielefeld, CITEC. As Germans like to express it, our interests in safety have not “resonated” in CITEC. People are building small mobile robots of various sorts, even having them run around in public areas and interacting with ordinary people. I have had many conversations about safety, what the issues are, why it is important, and what you can do about it (Hazard Analysis, Hazard Mitigation or Avoidance Measures, Residual Risk Analysis; it’s helpful to have a set of principles which help you avoid the most well-known major hazards). Indeed, I gave a Keynote Talk on exactly this topic at the IET System Safety conference in London in 2009, along with a paper. But I haven’t met anyone at CITEC who has read the paper or who knows what is in it (it’s only about 3000 words long, so length can’t be a factor). When I have talked to CITEC people about safety, the reaction is typically “that’s nice. How interesting, I hadn’t thought about that” and they turn back to what they were doing before.

I remember a long conversation I had in November 2010 at an evening reception with a CITEC researcher who was part of a team building a mobile robot which interacted with people. They were aiming to exhibit it in a local gallery as a robotic guide, along with the human guides. I suggested to her that the insurance company would likely want assurance against accidents, and the way to do that is Hazard Analysis, etc (as above), that we were expert at that and could help. I said that the classic mitigation for robots with moving parts (indeed, defined in a draft international standard which was in review) was defining a space of motion and installing interlocks to prevent people entering that space when the robot was in operation. I suggested that was likely to be what will happen if one doesn’t think about the issues more creatively, and it was probably not what she wanted. Indeed not, she said, it’s not really what we’d like.

I didn’t hear from her again, despite trying e-mail contact. (Remember, this is in the very institute in which I work! That “resonance” thing.) But a few months ago there was a picture in the local newspaper of the robot doing its guidance job – in an area roped off in the corner of the gallery, well away from exhibits and with gallery personnel on hand to ensure people stayed the other side of the rope. The classic measure, as I predicted.

A robotic helicopter has fast-moving parts which are open to the atmosphere – if it didn’t, it wouldn’t fly. Quite apart from damage caused by physical collision with the drone body, these fast-moving rotating parts are going to be moving in environments which have not been built with that in mind. Children get very curious and like to touch things, for example, so you have to keep it away from them. If operation was in a industrial plant, interlocks would be required to prevent any people from approaching the device when it was operating. And that is with presumed-trained, professional personnel. You don’t have three-year-old kids running here and there on the factory floor. That is the legal situation with human interaction with such devices currently, and it’s not just the FAA. In the US it’s OSHA and 150 years of experience with dangerous workplaces, such as 19th century and early 20th century railroads. Are people just going to throw all that out so that some company can deliver tacos? I doubt it. It took decades to get that level of protection, and I suspect that a lot of that was driven ultimately by consideration of the costs of accidents. So you probably can’t let the Tacocopter land (the journalist’s idea about getting tacos thrown at you is not that far-fetched!). So what about when it has to, for some reason – a fault, for example? Which aircraft doesn’t have those occasionally? How do you assure to keep it well away from kids during such a event? However you do it, that subsystem better not be the faulty one!

Here is an article from this week’s Economist which does say something about liability. Someone at MIT is trying to devise algorithms to interpret hand-signalled movement instructions, as used on aircraft carriers, reliably and accurately. I used to know one of the authors, Randall Davis, from conferences. He is a well-known and well-respected AI guru. The final question that occurs to the journalist is who would be prepared to trust “the fate of a multi-million-dollar drone to such a system” (it is only about 75% reliable at the moment).

But, says the Economist:

But it is a good start. If Mr Song can push the accuracy up to that displayed by a human pilot, then the task of controlling activity on deck should become a lot easier.

Another point of view is that experience shows that 75% is the easy part. When you’re at 90% and you want to get it up to 95%, that’s when the hard work starts. And from 95% to 99% may well take orders of magnitude more. For example, staying with AI, raw Circumscription is not the way to handle Blocks-World planning. This has been agreed for a couple of decades. But it does handle 90% or more of Blocks-World planning very effectively, as do many of the other methods from that era that “don’t work”.

But that’s SW engineering and “Symbolic AI”. Curiously, people who work in “neural informatics” (as it’s called in Germany) and approximation techniques seem often to have a different view, that when they can do 75%, or 80%, or 85%, they are “nearly there”. How can such different views of success prevail in one and the same subject, informatics?

What Martyn’s committee found was astonishing. For example, critical infrastructure functions whose builders and operators were convinced had no connection with any GPS functionality – and which stopped working when a GPS jammer was activated. The Committee’s report is well worth reading all the way through. Its remit includes all SatNav systems, not just GPS.

Martyn gave a Keynote talk at the 20th Safety-Critical Systems Symposium in Bristol a couple of weeks ago. A Google preview of Martyn’s paper is available, as well as an IET.tv film of his talk. (The Institution of Engineering and Technology, IET, filmed many of the presentations. You can check out my Keynote on the Fukushima Daiichi accident as well if you like )

It is amazing to me that anyone wouldn’t take Martyn’s observations very seriously indeed.

However, we do appear to have a few journalists that poo-poo it, for example Lewis Page again recently in The Register after his commentary a year ago upon the report’s release, just as we had an astonishing number of journalists who made public their opinion that Y2K was never a big deal. A very silly point of view. As Martyn points out in his talk, the reason Y2K was not a big deal is that people such as himself worked very hard to eliminate as many as possible of the Y2K vulnerabilities discovered in our critical infrastructure, and were obviously quite successful. He knows what they were, since he was the senior technical advisor for some of that work (for example, UK air traffic services provision), and knows what would have happened had they not been taken care of.

The main social point here is, I think, people who worry versus people who don’t. If we didn’t have people who worried, then we wouldn’t be able to operate because things would be continually going wrong, such as possibly UK air traffic services at the turn of the millenium had NATS not worked very hard to eliminate those vulnerabilities. And on the back of such successful effort there are journalists who say “everything’s OK, isn’t it? Why worry?”. Yes, things are OK. Why worry? Because if some of us didn’t, they wouldn’t be.

Here is an example of a daily vulnerability that bit. It’s also old hat. But it happened to me two days ago, and most of those involved are a professional computer scientists with a PhD (or about to obtain one) and decades of experience of such matters.

I have used my e-mail system as a memo system very effectively for the last few decades. I am based on IMAP, so it’s what people now call “in the cloud” but used to be called “stored on a server“. Over the years, when a subject or task occurs to me, I have got pretty good at remembering the context in which it occurred and indexing into e-mail (I send quite a few messages just to myself). It works for me very well. For decades.

Until Tuesday. I was writing an email, and the longish memo I was writing started losing characters backwards from where I had been typing, at the similar repetitive rate to that deriving from, say, a stuck delete key. It took a few seconds to realise what was happening. Then I went into the menu-strip at the top of the screen (I use the Apple OS+environment) and tried to quit my mail client (Thunderbird – Apple Mail apparently does not work well with IMAP. I lost all my mail for about a year at one point a few years ago and it took a couple of days to generate a solution from backup. The second time it happened, I switched to Thunderbird). The menu would come down, but disappeared again as I moved the mouse onto it. This happened repeatedly. I tried the same on the Apple main menu (so I could “Force Quit” the mail client) but the same happened there. I tried a hardware shutdown – the OS refused because Thunderbird would not quit and it advised me to quit Thunderbird and then try again. I have never actually tried to log in as root and am not sure I remember the root password, so trying that, and if successful getting the process number and performing “kill -9” didn’t seem like a good option given the urgency.

So, hardware kill: press the “off” switch and hold until the machine powers down. Good news for me: this worked.

When it came back up and I fired up the mail client, it showed me that all the messages from Wednesday 15 February at 16:35 (15:35 UTC) until that Tuesday morning, 21 February, were no longer there. There are a bunch of important interventions that had disappeared.

So I asked the faculty computer services to restore the mails from backup. One of the two officers is Jan Sanders, with whom I have worked closely for over a decade; he also works with Causalis (people from SSS2012 may remember him from the booth) and will shortly finish his Ph.D. with me. And he installed and maintains this blogging system. These two people, along with 50-75% more help from assistants, manage the Technology Faculty’s (TechFak) computer systems, which account for over half the data volume per day of the entire university. A couple of years ago, we purchased backup hardware for some €30,000 because the university computer center proved to be unable to provide backup services as needed by some high-data-volume colleagues. The university is trying to centralise as many “routine” computing services as possible, and this situation was and is a major negotiating point over the future organisation of research computing services in the university.

Well, our backup HW+SW didn’t work. Jan + colleagues were unable to extract my e-mail Inbox directory alone. They ended up rebuilding the entire TechFak mail-server IMAP file system on a restore disk, some seven hundred gigabytes or so to be restored from main+incremental backup tapes. Estimate on Tuesday lunchtime was Wednesday morning. But on Wednesday morning, when they came in to work, the job had terminated with an error, and then only had up to 6 February cleanly restored.

Moral: the cloud is vulnerable in the ways that people concerned with the provision of computing services have known about for a long time. This is not the first time this has happened to me (indeed, the third time I have lost amounts of mail in five years). There are obvious ways to avoid specific problems, but there is mostly neither time nor resources to implement and manage all those solutions perfectly all the time. In this case, there were (at least) two failures, and it is clearly impractical for the faculty computing services to check continuously whether they can effectively restore data through such two failures, as well as all the other possible failures that could occur. This is a resource-intensive on-demand function and it is combinatorially impossible to check regularly the execution of all such functions in even a moderately complex system such as e-mail backup.

When someone comes up with easy ways to solve any digital-computational vulnerabilities, say to GPS interference, that is less than half the tale. The rest of the tale concerns whether those solutions are implemented, and also continuously and effectively maintained.

There is a lot of superb computer science behind this nowadays. Versions of Leslie Lamport’s Paxos algorithms are enabling Google’s servers to provide us with our daily informational bread (Paxos logically serialises distributed database transactions).

Most journalists and digital-services marketing people have not heard of, let alone understand, the combinatorial impossibility of checking and maintaining all your on-demand functions, or even routinely how the various Paxos variants work and three-phase commit doesn’t. To find out what is possible and what is not, in other words, you still have to talk to computer scientists with authoritative knowledge. Such as Martyn and his GPS-vulnerability team from the Royal Academy of Engineering. And be wary of what is said in thoughtful articles about “cloud computing” in news media unless it comes from such people.

What actually happened to me? I don’t know. The “stuck delete key” hypothesis seems to me to be implausible (it has worked fine since). And a software glitch in my mail client alone would not explain why the windowing system pull-down menus failed to operate as expected. I am not unfamiliar with forensic analysis of this sort (indeed we do it for major accidents) but this is not the first time an explanation has eluded me and I doubt it will be the last.

I should note, first, in reference to the Washington Post article that the US term “college” refers to all higher-education which leads to a qualification called a degree. This includes “community colleges”, tax-supported institutions which provide the equivalent of the first two years of a four-year university education and which grant degrees called “associate degrees” to successful students, as well as universities, which may be four-year or six-year institutions, as for example the California State University system is, granting Bachelor’s and Master’s degrees, or “research universities” such as the University of California which also grant Ph.D. degrees.

I recall British Prime Minister Blair saying in 1997 (do I?) that the Labor government intended to push degree achievement rates up to 35% of the population, up from the 15-18% or so which it was when I graduated in 1973. I didn’t realise until I looked at the WP graphic, based on the 2009 data, that this had been achieved. I herald it as a major national accomplishment.

(I get the figure of 15-18% as follows. This 2000 report by David Greenaway and Michelle Haynes says that about 400,000 young people were in tertiary education then. If one takes the average lifetime, a little under 80 years, considers that 3 years is a twenty-fifth of that, and that the population of Britain is about 60 million, one would expect 2.4 million people of university-visiting age. 400,000 is thus one in six, about 17%. I should perhaps mention that Laura Spence, who was rejected by “Oxford” but given a scholarship at Harvard, had in fact applied to my college. Not the greatest marketing moment in history).

Similarly, I had, until today, oft quoted the rate of young people in the US entering higher education as a sign of what I thought was desirable, and used the figure of 55% of school leavers. I doubt if this has changed significantly. But I am disturbed to find out that that apparently only 41%, about three-quarters, complete to some sort of degree. Considering that includes associate degrees, which are only two-year courses of study, that does not bode well for the US, if you believe as I do that the more people learning skills in a short time which they otherwise would not have, then the greater the productivity of their society, in the richness of hobbies and other pursuits in life and not just in stuff measured in standard economic measures.

I am intrigued by the Box on p18 of the OECD report entitled “Germany rethinks its assumptions about education and social equity”. Yes, indeed! People here were quite convinced about the “quality” of the education system, despite the obvious inequities and inadequacies apparent to those of us with wider experience, until the PISA reports on comparative achievement in secondary education started appearing from 2000 on, which showed German school achievement in a poor light compared with Germany’s economic peers. Then it couldn’t be ignored any more, and it wasn’t.

PISA was to do with secondary education. I am still somewhat disturbed by the relatively poor showing of Germany in tertiary education, at 26%. Some comments on that, some of which I have made before.

We currently have huge building projects going on around our Bielefeld University campus, which is itself huge (put “Bielefeld University, Universitätsstrasse 25, Bielefeld, Germany” into Google Maps). The main university building, in which almost everything goes on, is some third of a kilometer long, as you can see. Two new campuses are being constructed, one adjacent to the old building on a parking lot just to the north of the main building, between the two branches of what is labelled “Universitätsstrasse, some two hundred meters long and the better part of a hundred meters wide, and one “over the road”, almost a kilometer away, in (Google Maps again) “Lange Lage, Bielefeld, Germany”, which is also large, and will house the University of Applied Sciences (what the Brits used to call a “Polytechnic” and Germans a “Fachhochschule”), a teaching university which does not grant research degrees, and which is now largely scattered in old and often unsuitable buildings around town. This all amounts to a huge public works (which Google Street View does not yet show). And, if the above figure is to be believed, this will only be usable by a quarter of the young adults in the city and surrouding areas.

Do we have a town-and-gown problem? Less so than we did, I think, but more so than we might. The university does some outreach, including a science fair each year called Geniale (some pictures of GENIALE 2011 – the German for “pictures” is “Bilder”), spread over selected spots in the Old Town. But why aren’t most of the young people in this area passing through some part of this enormous spreading campus to take part in something? After all, they and their parents pay the taxes that create all these large buildings and pay their occupants. Future auto mechanics and hairdressers could surely benefit personally from participating in a course on 1960′s popular music, couldn’t they? Germany has no equivalents to Brian Patten, Roger McGough, Adrian Henri or Carol Ann Duffy, but we have plenty of slam poetry (link only in German, unfortunately), indeed a local slam poet who has turned into a valued writer and raconteur, Mischa-Sarim Verollet (also only German). Here is the announcement for the next one in April 2012.

Such educational offerings are available through the Volkshochschule Bielefeld, the Community Further Education Center, but this is largely less formal – courses are not assessed, the qualifications of course-offerers fulfil no standards (either experiential or formal), one doesn’t obtain a transcript of courses completed, and, importantly, it does not constitute the kind of accomplishment which a prospective employer expects to see on an applicant’s résumé. I am thinking that all these things should happen. I am also thinking about the impoverished financing of the Volkshochschule compared with the heroic building works around the university campus.

I cannot see that expensive tertiary education can thrive unless it includes way more than the elite. We are well past the days when people said “well, that’s for them rich and clever kids” and turned their backs. Nowadays, people say “I pay taxes too; why can’t I come in here?” and I think that question is very well founded. Especially when the expenditure is so massively visible, as it is in Bielefeld.

German university education has changed, though, massively in the last decade. The previous system has been more or less junked, and every university now offers Bachelor’s and Master’s degrees, instead of the old Vordiplom/Diplom, which were not recognised outside Germany for what they were (a Vordiplom was like a US associate degree, and a Diplom like a Master’s, but with nothing in between). It is astonishing how everyone just threw the old tradition away in the early 2000′s and went with what, for most here, was a completely foreign system with which they had little or no experience. I did find out why from a colleague in Sociology, though. They had over a 90% drop-out rate in their Diplom course. And this in one of the most well-reputed Sociology faculties in the country that invented it.

I think student contact with the rest of Europe was also slowly bringing a new perspective. German university students were finding themselves relatively immobile compared with their peers in other European countries, because the organisation of their degrees did not easily translate. For example, in the late 1990′s, students studying for degrees in my faculty returning from studying abroad for a year in the ERASMUS program still had to take an oral degree examination in the studies they had completed abroad to have it count for our degree, even though they had already been assessed by the foreign institution for that work and the EU ERASMUS agreement requires that we honor that assessment. To those who came to me, I asked for the transcript, or equivalent document showing successful completion, asked them to tell me about what interested them in the work, and passed them. In other words, the exam was purely formal, and the result identical to what they had already achieved. That is the best way I could see to fulfil the EU requirement, which our internal faculty procedures at that time still contradicted.

Besides that, successful graduates (the Sociologists’ 10%; our proportion in Informatics was much, much higher!) were leaving tertiary education with a degree equivalent to a Master’s at the age of 26-28 (and some even older), whereas their British and US peers were obtained such qualifications at the ages of 22-24. People on the ERASMUS exchange were noticing they were somewhat older than their local peers, and those starting Ph.D. programs in other countries noticed it even more.

Now, we have Bachelor’s and Master’s degrees, credit points for each course, and credit points are transferable between all European tertiary-educational establishments.

I cannot necessarily say that the quality of education has improved, however. With the more extensive evaluation requirements (per course, now), much of this is being farmed out to tutors and other helpers, and the quality of that education and assessment does not seem to be monitored as I feel it should be. I monitor the courses in my group, which are all based on lab work, or seminars which consist largely of student contributions with commentary from the lecturer, and my group has considerable continuity in our student tutors, who were picked for (or, better said, who picked themselves by) their enthusiasm and capabilities. But some of our larger courses appear to have problems (one of my bright people, who has coauthored an important chapter in our system safety text, is on his third attempt at one of the required practical courses, for what appear to me to be spurious reasons).

The throughput has, however, improved. One reason in the past was the introduction of modest fees, some few hundred euros per semester. Suddenly, all our 6-year and 7-year students (of which we had plenty) wanted to finish – and most did. And the fee money was directly given to the Faculty, in which a largely student committee, which did include the Dean, decided what to do with the money to finance improved teaching. More tutors for some courses. Lab equipment – my lab was built with this money. The faculty also hired a highly motivated and very successful lecturer whose courses are loved by students and who does lots of lab work, indeed he uses the lab which we built.

The other reason is that students in our Bachelor’s and Master’s programs are spending much of the day in courses, and most of the rest of their time doing the homework. Their time is filled with study-related work. This is very different from ten years ago. But I think it is a benefit, more on a par with what their peers do in other countries with a higher percentage of college graduates.

I was a freshman at Oxford in mathematics, interested in logic. I had been reading Chomsky in my first quarter because I had been told Chomsky had mathematised language. My tutor in algebra, Ian Macdonald (same jacket as in the picture!), an algebraic geometer, suggested I could look at a logic textbook he recommended (which I read with some difficulty over the Christmas break). Derek Goldrei, a graduate student tutoring in logic at my college Magdalen, suggested I listen to Michael’s lectures in set theory.

Michael didn’t lecture. Michael thought out loud. He distributed notes telling his listeners what he was going to be thinking about during that appointment. I learnt, by watching and listening, how to think. About set theory. About inference rules. About non-classical logic (Michael was drawn to intuitionist thinking about mathematics, because he thought it was right to base your assertions on the concrete evidence you had).

I had been attending freshman mathematics lectures, which went “Theorem” “Proof” “Let x be…” and had despaired of ever being the kind of person who thought like that. Then I attended Michael’s thinking-out-loud sessions and understood what really went on in people’s minds; how the symbols were shorthand for notating thoughts. And, in my second year, I could do it! Just like Michael! Actually, not just like Michael. Not anywhere near “just like Michael”. For, as John Mackie is reported to have said in The Times’s obituary, Michael was a genius. Michael was ineffable.

Michael was different. A mass of wavy white hair, he would array himself longitudinally on a bench in the lecture hall and clean his cigarette holder while leaning on an elbow, with his head just above the seat backs, and crack jokes about his friends and colleagues while waiting for the lecture to begin. At which point the jokes would reduce in number as he concentrated on what was being said. If there is anything any undergraduate wished to be in the course of study he had in large part created, Maths and Philosophy, it was to be “just like Michael”.

Simply put, Michael taught me how to think, in logic; by extrapolation, in mathematics. About the deep philosophical questions concerning truth, mathematics, the use of language. Differently put, I learned how to think by watching and listening to him.

When I graduated in 1973, I attended a ceremony in the Sheldonian Theatre, in Latin, much foreshortened from the original, during which my degree was conferred. A ceremony designed over centuries to give its recipients the indelible impression: you have done it! I had done it! I felt it and they’d said it in Latin! After the ceremony, I went straight across the road in my academic dress to purchase a copy of Michael’s new book, on Frege’s philosophy of language. Michael had shown how to think about these matters in pellucid English prose.

I went right afterwards to the other side of the Northern Hemisphere, to Berkeley in California. Michael had helped me get there, for he had written me a recommendation for graduate school. I have no idea what he said, but I it can’t have been all disastrous. (I can imagine: “Ladkin is mortal and does OK for one. But I’m afraid I don’t really know much about mortals.”)

I was required at the end of my first year by Bill Craig, my advisor in Berkeley, he of Craig’s Interpolation Theorem, to take the qualifying exam in philosophy. I protested and threw tantrums and all that, but you know you can’t really rebel. Bill said “you will do it” so I did it. I read Michael’s book, and its seemingly impenetrable prose. And I read it again. And understood more. And again. And more. And again. It wasn’t that Michael’s prose was impenetrable. Michael wrote exactly what he was thinking and his thinking was non-trivial and exact. It took me a while to absorb his train of thought. His prose was, indeed, pellucid. When I had done so, I went into the exam room (actually the philosophy library) for six hours and wrote exactly what I thought about the matters about which I had learned from reading Michael’s book so carefully. Non-trivially and exactly. I think Ernie Adams graded the exam. I passed. Turns out I was the first student in the history of Tarski’s program to pass the philosophy exam in my first year. Thank you, Michael!

(You have to understand – I was rotten at written exams. I got so nervous I couldn’t even read the questions straight. It’s a miracle I ever got into and out of Oxford, at which assessment is based on a student’s brilliance at written exams.)

I saw Michael in Berkeley once. He gave an evening lecture which I attended. I did get to exchange a brief word, amongst all the others earnest to talk with him.

I saw him again in 2009, at the 40th anniversary reunion of Maths and Philosophy graduates in Oxford, of the course which he had done so much to establish, and to which I owe my subsequent career. Derek Goldrei was the First Graduate (he switched in his final year; graduating in 1969 when the course was established). I in 1973. I was one of only two or three from that era at the reunion and felt quite The Establishment. Michael was there, and Dana Scott. Michael was old and frail. Gave an endearing and well-constructed speech. When I approached him after the dinner, he didn’t remember who I was, but then so many had passed through the gate since I had. I simply thanked him. He accepted graciously.

Michael is gone, on 27 December 2011. For me, he was Philosophy. When he was with us, Philosophy was alive. Now he is gone, Philosophy is gone. Maybe not, but it sure feels like it. It turns out I seem to have assumed he was immortal. Apparently not. It is -let me say- hard for me to adjust.

Here is The Guardian’s take. The Times has a fine obituary, forwarded to me by Chris Miller, but it lies behind a paywall, just as now Michael does, though with a currency which I only wish I had. As an atheist without this currency, I can only say: God be with you, as you wished.

Some Coincidences.

Racism. Two of the killers of Stephen Lawrence were convicted in early January 2012. Here is a poem about it by Poet Laureate Carol Ann Duffy. Michael and his wife Ann devoted themselves to race relations in 1960′s and early 1970′s Britain, efforts well documented in the obituaries. He only returned to philosophical work after he felt the efforts to turn Britain away from racist habits had failed. But they haven’t failed, Michael, and neither had you.

Brains. Apparently some people claim now that our brains start to go downhill at age 45 It is not clear this is news: The Guardian had something about it 12 years ago. Michael published his first book at 48, and there followed many more, all of them worth reading very carefully indeed.

Note Added 11.01.2012

It’s not just philosophy. Thinking it over, there are three fundamental developments in technical elementary logic which I have kept coming back to throughout my career. Things which are simple, clear, brilliant, which increase one’s understanding almost instantly, and continually prove to be useful. One is Dana Scott’s Consequence Relations, a formulation of logics which, to me, turns out to be the most efficient way to perform formal deductions, the raw material of logic. I keep meaning to translate into LaTeX the mimeographed notes which Dana handed out almost 40 years ago now. Another is Saul Kripke’s possible-worlds semantics for normal modal logics, and his similar epistemic-worlds semantics for logics of belief and evidence, such as inference in intuitionistic mathematics, and the inferences of Pen Maddy’s “Second Philosopher”. I learnt these partly from Michael. The third is Michael’s and John Lemmon’s formal correspondence between the modal logics from S4 to S5 and the propositional logics between intuitionist and classical.

Second Note Added 11.01.2012

Timothy Williamson, Michael’s successor in the Wykeham chair of Logic (David Wiggins came between Michael and Tim), pointed me to a series of tributes in the New York Times Opinionator blog last week.

The fix is apparently to modify the BITE test of the ADIRU specifically to look for such anomalies, and to modify the data-filtering algorithms of the Flight Control Primary Computers (FCPC, also known as PRIM) of the A330.

The Final Report is now available on the ATSB WWW site.

There was a note from Andrew Heasley in Risks 26-67 with a title saying the accident was “Blamed on Software“, pointing to a newspaper article. I find this claim misleading. The problem which arose had nothing to do with anything for which any software engineer would have been responsible.

The fixes were implemented in both SW and HW, but fixes to non-SW problems are very often implemented in SW.

The PRIMs ran a data-assurance algorithm for data received from three different ADIRUs, which are electronic boxes built by a different manufacturer. This data assurance algorithm had a specific vulnerability to spiky angle-of-attack (AoA) data presented in a particular time-sequential manner, which was exploited during the occurrence. The algorithm, which uses AoA data from three ADIRUs, filters out multiple data spikes from a unit which occur within a specific time frame. Spikes on the culprit ADIRU occurred with similar values just over the boundary of this time frame, and were thus taken as veridical by the PRIMs. The resolution algorithms for the AoA data (with that from the other ADIRU units) in the PRIMs let these values through, and the PRIMs reacted accordingly by commanding sudden nose-down pitch.

Responsibility for the design of such algorithms lies clearly with those who are experts on the engineering of electronic data generation and transmission equipment, not on any software engineers.

To give a similar example with which I been recently involved, it turns out that signals of certain frequencies in AC electric circuits can bypass the Type A and Type B circuit protection equipment (circuit breakers) that are required in most electric circuits (household and industrial) in Germany. A committee on which I sit has recently considered attaching equipment which is, as far as we know, theoretically capable of generating such frequencies to such circuits. A similar situation, how to handle anomalous signals, but no SW in sight. Pure electrical engineering.

Concerning my earlier note here on Certification Requirements for Commercial Airplanes, I find it interesting and commendable that the Bureau considered likelihoods of events in their summary (quoted below). However, I don’t believe they formulated it in quite the words I would have liked to have read.

They give reason to classify the event as “hazardous”, and with a fleet operating experience of 28 million flight hours this occurrence fits within the expected value (a technical term) of the operating time within which the effects of a hazardous event may occur (defined to be less than or equal to one occurrence within ten million operating hours), according to the acceptable means to determine compliance with certification criteria (now known as AMC 25). Notice it is not the event itself of which they assess the occurrence – that has occurred three times – but the deleterious effects upon safety of the event, which have only occurred once.

They speak of “certification requirements“. Strictly speaking, this is incorrect. The certification requirements are expressed in CS 25 and do not involve probabilities. The severity classification terms “catastrophic”, “”hazardous” etc and their associated acceptable/unacceptable frequencies occur in risk-matrix-type form in the Acceptable Means of Compliance document which accompanies the certification requirements (AMC 25), not the requirements themselves. (I note that these documents were called something slightly different at A330 certification time, 1993).

The certification requirements themselves are quite clear: the airplane shall behave in such-and-such a manner. If a wing falls off, or a flight control computer sends it into a loop, it is obviously not behaving in that manner; thus violating certification requirements. However, it is accepted that one cannot provide proof that such untoward things will never ever happen (will the sun rise tomorrow? Will your steering wheel come off in your hands? WIll your control sidestick come out of its holder in your hand?), so a less strenuous regime based on arguing likelihoods is defined as an “Acceptable Means of Compliance” with the regulations for purpose of certification.

This is not hair-splitting. It has consequences, in particular in this case, for how anomalies are dealt with, as follows.

If the requirement were that, say, “hazardous effects shall only occur on average once in between 10^7 and 10^9 operating hours“, which is what the AMC says you have to show to demonstrate compliance acceptably, then it would have been open to the manufacturer to do nothing in reaction to the QF72 event: the hazardous effects occurred only within the expected time value of their occurrence. If you think about it, it would also be open to a manufacturer to do nothing until the second occurrence of any hazardous or indeed catastrophic effects, even if the problem occurred first within the early experience of flying the aircraft! This is simply a consequence of the meaning of the probabilistic concepts used.

Whereas, as things now stand, separating requirements, which are absolute, from acceptable compliance (which may be based on occurrence frequency) any in-flight anomalous behavior must be fixed or the airworthiness certificate will be withdrawn. This is because such behavior violates the written requirements, that the aircraft shall not behave that way. To repeat, the conditions on behavior are absolute, not likelihood-based.

And that is how one wants things: The requirements are absolute, but it is accepted that in science and engineering you are often only convinced to some degree, so it is regarded as acceptable to argue your conviction up to a certain degree, and not to have to prove it, which would likely be impossible. But if something does go wrong, you want it fixed right away.

One can argue that any given set of occurrences is compatible with any probability requirement whatever, and thus that probabilistic requirements are inappropriate to determine airworthiness in any case. However, I don’t think such an argument works. Say these three events had occurred within 3 million operating hours, each with damage. One could estimate the likelihood that an piece of equipment fulfilling the condition of an expected value of at most once in 10 million operating hours to exhibit three events within 3 million operating hours. One would conclude that it is unlikely, say with small probability P. It follows that the situation that the aircraft fulfills the acceptable-compliance criterion has the same probability P. The small probability P that the aircraft acceptably complied with certification requirements would provide good reason for withdrawing the airworthiness certificate.

Concerning the data anomaly itself stemming from the ADIRU, its cause remains a mystery. The report says:

Some of the potential triggering events examined by the investigation included a software ‘bug’, software corruption, a hardware fault, physical environment factors (such as temperature or vibration), and electromagnetic interference (EMI) from other aircraft systems, other on-board sources, or external sources (such as a naval communication station located near Learmonth). Each of these possibilities was found to be unlikely based on multiple sources of evidence. The other potential triggering event was a single event effect (SEE) resulting from a high-energy atmospheric particle striking one of the integrated circuits within the CPU module. There was insufficient evidence available to determine if an SEE was involved, but the investigation identified SEE as an ongoing risk for airborne equipment.

The report says that the manufacturer is developing a modification to the BITE to detect such failure modes:

Without knowing the exact failure mechanism, there was limited potential for the ADIRU manufacturer to redesign units to prevent the failure mode. However, it will develop a modification to the BITE to improve the probability of detecting the failure mode if it occurs on another unit.

Here is the executive summary. It is well and concisely written. I include the three paragraphs about seat belts and the investigative process for completeness.

Executive Summary

At 0132 Universal Time Coordinated (0932 local time) on 7 October 2008, an Airbus A330-303 aircraft, registered VH-QPA and operated as Qantas flight 72, departed Singapore on a scheduled passenger transport service to Perth, Western Australia. At 0440:26, while the aircraft was in cruise at 37,000 ft, ADIRU 1 started providing intermittent, incorrect values (spikes) on all flight parameters to other aircraft systems. Soon after, the autopilot disconnected and the crew started receiving numerous warning and caution messages (most of them spurious). The other two ADIRUs performed normally during the flight.

At 0442:27, the aircraft suddenly pitched nose down. The FCPCs commanded the pitch-down in response to AOA data spikes from ADIRU 1. Although the pitch-down command lasted less than 2 seconds, the resulting forces were sufficient for almost all the unrestrained occupants to be thrown to the aircraft’s ceiling. At least 110 of the 303 passengers and nine of the 12 crew members were injured; 12 of the occupants were seriously injured and another 39 received hospital medical treatment. The FCPCs commanded a second, less severe pitch-down at 0445:08.
The flight crew’s responses to the emergency were timely and appropriate. Due to the serious injuries and their assessment that there was potential for further pitch-downs, the crew diverted the flight to Learmonth, Western Australia and declared a MAYDAY to air traffic control. The aircraft landed as soon as operationally practicable at 0532, and medical assistance was provided to the injured occupants soon after.

FCPC design limitation

AOA is a critically important flight parameter, and full-authority flight control systems such as those equipping A330/A340 aircraft require accurate AOA data to function properly. The aircraft was fitted with three ADIRUs to provide redundancy and enable fault tolerance, and the FCPCs used the three independent AOA values to check their consistency. In the usual case, when all three AOA values were valid and consistent, the average value of AOA 1 and AOA 2 was used by the FCPCs for their computations. If either AOA 1 or AOA 2 significantly deviated from the other two values, the FCPCs used a memorised value for 1.2 seconds. The FCPC algorithm was very effective, but it could not correctly manage a scenario where there were multiple spikes in either AOA 1 or AOA 2 that were 1.2 seconds apart.

Although there were many injuries on the 7 October 2008 flight, it is very unlikely that the FCPC design limitation could have been associated with a more adverse outcome. Accordingly, the occurrence fitted the classification of a ‘hazardous’ effect rather than a ‘catastrophic’ effect as described by the relevant certification requirements. As the occurrence was the only known case of the design limitation affecting an aircraft’s flightpath in over 28 million flight hours on A330/A340 aircraft, the limitation was within the acceptable probability range defined in the certification requirements for a hazardous effect.

As with other safety-critical systems, the development of the A330/A340 flight control system during 1991 and 1992 had many elements to minimise the risk of a design error. These included peer reviews, a system safety assessment (SSA), and testing and simulations to verify and validate the system requirements. None of these activities identified the design limitation in the FCPC’s AOA algorithm.

The ADIRU failure mode had not been previously encountered, or identified by the ADIRU manufacturer in its safety analysis activities. Overall, the design, verification and validation processes used by the aircraft manufacturer did not fully consider the potential effects of frequent spikes in data from an ADIRU.

ADIRU data-spike failure mode

The data-spike failure mode on the LTN-101 model ADIRU involved intermittent spikes (incorrect values) on air data parameters such as airspeed and AOA being sent to other systems as valid data without a relevant fault message being displayed to the crew. The inertial reference parameters (such as pitch attitude) contained more systematic errors as well as data spikes, and the ADIRU generated a fault message and flagged the output data as invalid. Once the failure mode started, the ADIRU’s abnormal behaviour continued until the unit was shut down. After its power was cycled (turned OFF and ON), the unit performed normally.

There were three known occurrences of the data-spike failure mode. In addition to the 7 October 2008 occurrence, there was an occurrence on 12 September 2006 involving the same ADIRU (serial number 4167) and the same aircraft. The other occurrence on 27 December 2008 involved another of the same operator’s A330 aircraft (VH-QPG) but a different ADIRU (serial number 4122). However, no factors related to the operator’s aircraft configuration, operating practices or maintenance practices were found to be associated with the failure mode.

Many of the data spikes were generated when the ADIRU’s central processor unit (CPU) module intermittently combined the data value from one parameter with the label for another parameter. The exact mechanism that produced this problem could not be determined. However, the failure mode was probably initiated by a single, rare type of trigger event combined with a marginal susceptibility to that type of event within the CPU module’s hardware. The key components of the two affected units were very similar, and overall it was considered likely that only a small number of units exhibited a similar susceptibility.

Some of the potential triggering events examined by the investigation included a software ‘bug’, software corruption, a hardware fault, physical environment factors (such as temperature or vibration), and electromagnetic interference (EMI) from other aircraft systems, other on-board sources, or external sources (such as a naval communication station located near Learmonth). Each of these possibilities was found to be unlikely based on multiple sources of evidence. The other potential triggering event was a single event effect (SEE) resulting from a high-energy atmospheric particle striking one of the integrated circuits within the CPU module. There was insufficient evidence available to determine if an SEE was involved, but the investigation identified SEE as an ongoing risk for airborne equipment.

The LTN-101 had built-in test equipment (BITE) to detect almost all potential problems that could occur with the ADIRU, including potential failure modes identified by the aircraft manufacturer. However, none of the BITE tests were designed to detect the type of problem that occurred with the air data parameters.

The failure mode has only been observed three times in over 128 million hours of unit operation, and the unit met the aircraft manufacturer’s specifications for reliability and undetected failure rates. Without knowing the exact failure mechanism, there was limited potential for the ADIRU manufacturer to redesign units to prevent the failure mode. However, it will develop a modification to the BITE to improve the probability of detecting the failure mode if it occurs on another unit.

Use of seat belts

At least 60 of the aircraft’s passengers were seated without their seat belts fastened at the time of the first pitch-down. Consistent with previous in-flight upset accidents, the injury rate, and injury severity, was substantially greater for those who were not seated or seated without their seat belts fastened.

Passengers are routinely reminded every flight to keep their seat belts fastened during flight whenever they are seated, but it appears some passengers routinely do not follow this advice. This investigation provided some insights into the types of passengers who may be more likely not to wear seat belts, but it also identified that there has been very little research conducted into this topic by the aviation industry.

Investigation process

The Australian Transport Safety Bureau investigation covered a range of complex issues, including some that had rarely been considered in depth by previous aviation investigations. To do this, the investigation required the expertise and cooperation of several external organisations, including the French Bureau d’Enquêtes et d’Analyses pour la sécurité de l’aviation civile, US National Transportation Safety Board, the aircraft and FCPC manufacturer (Airbus), the ADIRU manufacturer (Northrop Grumman Corporation), and the operator.

The British government has decided for a project to convert plutonium waste into MOX fuel, maybe for “a new generation of nuclear power plants“.

The decision, which ends decades of uncertainty on how to deal with a growing stockpile of more than 112 tonnes of plutonium waste, was presented as a written Parliamentary statement by the energy minister, Charles Hendry.

Indeed for half a century Britain, like many other countries with nuclear power plants, has not known what to do with nuclear power’s most toxic waste product.

Nuclear power relies on highly radioactive “fuel”, formed usually in the shape of rods, which engage in a chain reaction in the core of a nuclear reactor and produce heat. The chain reaction converts substances eventually into other substances which are no longer suitable for purpose; the fuel is “spent” and must be replaced. But the “spent fuel” remains highly radioactive. It is very toxic, must be carefully shielded from the environment and people, and this must go on with current spent fuel for (the most optimistic minimum estimate) 10,000 years (the level at which radioactivity has reduced to that of the originally-mined uranium and the original basis for US standards).

What do you do with it? Where do you put it?

It is not clear that anyone has come close to solving this problem. Nuclear power has been around for half a century, this waste has been accumulating, and the nation with the most plants, the US, has no solution. There are and have been many proposals, but so far none has turned out to be workable. Most of the spent fuel is still stored on-site in pools filled with water (water is pretty good at stopping the neutrons which are the main product of radioactivity in nuclear fuel rods. You only need a few meters of it to trap all but a few which get lost in the background). No one thinks that is a solution for more than a few decades, let alone a minimum of 10,000 years. There is a movement to store as much as possible in so-called “dry casks”: sealed physical containment vessels which are self-cooling after the spent fuel has been sitting around for some number of years. But you still have to put the casks somewhere where they will be safe for a minumum of 10,000 years. Yucca Mountain in Nevada was for many years the preferred prospective location. One wonders, however, about the stability of any structure in a seismically active area of recent volcanism. Eight volcanoes have erupted within 50km of the site in the last million years (op. cit.), but maybe it’s OK for 10,000 years? That is the main point: nobody really knows. No one with a decent set of choices could reasonably choose a place in a seismically and volcanically active area. That says, correctly in my view, that there is no decent set of choices. That is the way it has been for half a century.

It is a problem in Germany also. Germany processes spent fuel in France (and soon in GB) and transports the processed product in dry casks (called “Castor”) by rail back into Germany. The transport has been regularly plagued by protests which block the rail lines, and a transport typically takes days to weeks. Protesters used to aim for Germany’s withdrawal from nuclear power. Now that the German Government has committed to that, what is the latest protest (ongoing at time of writing) about? The protesters are apparently not content with the “temporary” storage site at Gorleben in Lower Saxony (it is in an underground salt deposit, which they claim with some reason is geologically unstable over the long term) and apparently want it to be stored at a reactor site at Philippsburg, near Karlsruhe. That is unlikely to be long term (in the sense of 10,000 years) either, since most authorities judge that any long-term site must be underground, in geologically stable ground. The storage issue has not been solved in Germany, either.

What about Britain? The Independent speaks of

……..decades of uncertainty on how to deal with a growing stockpile of more than 112 tonnes of plutonium waste, was presented as a written Parliamentary statement by the energy minister, Charles Hendry.
Plutonium waste has been a headache for successive governments because it is a highly dangerous radioactive material that can be converted into weapons-grade material, making it a security risk. It’s also expensive to store.

So Britain doesn’t have a long-term solution either. Who does? (Maybe France or Japan?) What to do with the waste is a major unsolved issue with nuclear power.

According to the Independent, the “uncertainty” has gone. It’s going to be converted into “mixed oxide” (MOX) fuel. Fuel? Yes, for reactors which have not yet been built. So you solve the waste problem by building new reactors – which, um, then don’t create waste? Of course they do. You are thus using the present waste in a process which will ultimately generate even more waste, as well of course as some electricity. So, problem solved? Obviously not.

Suppose one just wants to store MOX fuel, not use it. Is it, say, less toxic than spent fuel? No. Can be stored more easily? Not as far as I know. Can be used somehow? Yes, in those new nuclear power plants; we’ve just been that route.

Does this solve the nuclear-waste-product problem in any reasonable way? No. Since the UK government is full of clever people who can think at least this far, it could be that there is another explanation for this decision.

One thought. Somebody will be paid £3bn pounds for doing it, if it happens. Money goes somewhere, and I imagine the prospective recipients might be rather keen on their share. The new waste generated by the new reactors that use the MOX fuel that came from the old waste is, well, a problem for someone who comes along later. Science will solve everything, won’t it?

But it’s not going to be clear sailing. The Independent continues:

But although Mr Hendry made it clear that the Government sees the “Mox option” as a priority, it is not certain that a new £3bn plant to convert the plutonium into Mox fuel will ever be built.

Mindful of the financial and technological disaster of the current Mox fuel plant at Sellafield in Cumbria, which has cost £1.34bn and produced a tiny fraction of the fuel it was scheduled to make, Mr Hendry said that a clear case has still to be made for a second Mox plant at Sellafield.

Oh. So the first, smaller attempt to do this kind of thing failed?

Well, let me qualify that. £1.34bn went somewhere, somebody got it for doing something, so that all went OK. But it apparently didn’t go into the ostensible goal of processing X amount of plutonium into MOX.

And on the basis of that experience apparently the best option is to try again, more and bigger?

I am sure the mistakes made in building the first reprocessing plant will all have been cataloged. I am also sure that attempts will be made assiduously to avoid them when building the second, bigger plant. I have also studied troubled large projects, indeed giving evidence before a UK Parliamentary committee on one. Many big projects fail to deliver on the goals at the time of commencement. Indeed, it’s a first for me to see someone suggest a larger second project on the back of a failed, smaller first one. Surely it should be received wisdom by now that any serious, careful estimate of the cost of such a second, bigger plant be accompanied with an equally serious, careful estimate of the likelihood of success or failure?

Given that this plan for apparently “dealing with” nuclear waste leaves all the questions open about how one ultimately deals with the waste, could something else be going on? What could it be?

First, contractors earn money for building the plant, whether it works or not, so they would be happy. Second, a current government can be seen to be “doing something” about the problem, no matter how superficial. Third, by processing and reusing fuel, the issue of what finally to do about the nuclear waste is put off into the future. (That strategy has clearly worked for governments in the past!)

Let us, though, be clear what the situation is. There is a real scientific and social problem of what on earth one can do with the highly toxic waste products of fission reactors. One cannot expect the current UK government, indeed any government at all, to implement a true solution when none is known yet to exist.

So maybe the Independent is being inappropriately forthright when it claims that uncertainty is at an end. Here is what Energy Minister Hendry actually wrote, as reported by the Independent:

“Only when the Government is confident that its preferred option could be implemented safely and securely, that is affordable, deliverable, and offers value for money, will it be in a position to proceed with a new Mox plant,” Mr Hendry said. In its response to a public consultation on Britain’s plutonium problem, the Government has not rejected other options. One is to convert the 112 tonnes of plutonium dioxide powder stored at Sellafield into glass or concrete blocks that could be buried permanently in a deep waste repository. Another is to use the plutonium directly as fuel for fast reactors, if these can be developed commercially in the coming decade.

“While converting the plutonium into Mox is the most credible and technologically mature option, the Government remains open to any alternative proposals for plutonium management that offer better value to the taxpayer, and will seek to gather more details on all options,” Mr Hendry said.

That seems less than certain to me. According to this, the UK government has set priorities on the “viable” options. It has not actually decided to do anything.

So am I (and the Independent) making a lot of fuss about not very much? Here’s a thought. We all agree that something does indeed need to be done about nuclear waste. Suppose somebody “does something”, what is it going to be? Well, it’s going to be starting to implement this “plan”, since, as the priority option, it is obviously the thing to pick if anything is to be done.

But options remain open. In case a detractor says “why on earth are you doing this? It makes no sense“, the Energy Minister can reply “only when we are confident, etc, etc, the Government remains open to any alternative proposals, etc.”

And when a sufficient amount of money has been spent, someone can say “oh look, we’ve got half a MOX plant! Well, better get on and finish it, then! Don’t like to waste money…..”

Maybe it’s just the time of year. I haven’t hung my Christmas lights either. Or maybe the UK government has been reading its seasonal literature and the nuclear contractors hired a lobbyist name of Bob Cratchit.

I attached myself to the subgroup concerned with the assurance and certification of such systems.

We all seemed to have a whale of a time figuring out what a cyber-physical system (CPS) is. Tom Maibaum and others wondered how they might differ from embedded systems. People said, well, it is important that there are lots of subsystems interacting more loosely than with a hierarchically-developed complex embedded system. So John Fitzgerald wondered whether they were mostly systems of systems. (Actually, the “so” is causally misplaced. John, being an “F”, had his one-minute say before Tom, being an “M”). Social systems of mostly artificial agents, of which many examples were given, seemed to fit the “cyber-physical” conception, so CPS includes at least those. Platooning road and rail vehicles, swarms of robotic aircraft or ground robots, coordinated flying or other motion, coordinated searching tasks, and so on. There are enough examples to point and say “that’s what we mean!”.

I also learnt, once again (strange how short one’s memory can be!) to avoid uttering the phrase “emergent behavior”, at the risk of inciting a riot, or at least the closest one can come to a riot at a Dagstuhl seminar.

So what about assurance of such systems? Sadly, as I was on my way back, having had a beautiful bike ride back over the Hunsrück to Trier and caught the train, there occurred a horrendous road accident in Britain on the M5. You can read commentary about it on the York safety-critical systems mailing list. Go to The 2011 collection, sort by date, read the contributions on Sunday 6 November through Tuesday 8 November including “M5 Road Accident” in the title, or go to Paul Cleary’s initiating query and follow the thread(s) through (there are two slightly different titles, but the thread-following links persist through). I also had some private correspondence with Gérard Le Lann, who now works on road-vehicle platooning algorithms and associated questions.

As a result of the Dagstuhl discussions, and the e-mail discussions of the accident, I was able more concretely to formulate what I think is a new assurance problem which arises with (this conception of) cyber-physical systems. It is a little too long for a blog post, so I wrote it in a note called The Assurance of Cyber-Physical Systems: Auffahr Accidents and Rational Cognitive Model Checking and put it on the RVS WWW site Publications page.

The term ‘risk’ in 31000 is described as the ‘effect of uncertainty on objectives’ where one of the ‘effects’ can be ‘a deviation from the expected’ (4360 describes it more succinctly as: ‘a chance of something happening’). These ‘risk’ definitions differ markedly from…..

…the standard definition which has been around for 300 years and 10 months: Abraham de Moivre, De Mensura Sortis, or On the Measurement of Chance, Phil. Trans. Roy. Soc No. 329, January, February, March 1711, reprinted with a commentary by O. Hald in International Statistical Review 52(3):229-262, 1984, which may be retrieved from JSTOR. The definition given there is, in modern terms, that risk is the expected value of loss. “Expected value” is a technical term from probability. I give the word-for-word de Moivre definition below.

This definition is also that used for “risk” in finance. See Peter L. Bernstein, Against the Gods: The Remarkable Story of Risk, John Wiley & Sons, 1996/1998. Which book, as the publisher proudly proclaims on the cover, was a “Business Week, New York Times Business, and USA Today Bestseller” and includes praise from reviews by Galbraith, Heilbronner, the NYT, the WSJ and The Economist on its cover. (Indeed, Bernstein is where I got my original lead to Le Moivre).

The meaning of the term in system safety is always close to that of de Moivre, but usually avoids the explicit arithmetic of finance, expected value of loss, by saying “combination of” likelihood and severity. There are good reasons for being somewhat vague, namely that in many cases in system safety the numbers are not there to enable a calculation of expected value. Especially, for example, in a completely new type of system. (An example I am currently working on is the recharging systems for electric road vehicles. There aren’t many around, so in particular there are no reliable numbers on frequencies of untoward things happening.) In response to this common situation, engineers have developed “qualitative” and “semi-quantitative” methods for assessing risk.

One of the issues then becomes what you take the word to mean in technical contexts. Any definition which is not equivalent to the expected value of loss defines a different concept from that, but the same word, “risk”, is used. For good reason: most definitions are conceptually related and the main issue is to get “close” while not having all the numbers.

So what do you do when some branch of human activity, indeed apparently some standard, takes the same word, “risk”, and uses it to mean something different? (I don’t actually know what “effect of uncertainty on objectives” is supposed to mean. I don’t see how “objectives” can be affected by uncertainty. I can see how your chances of attaining them are.)

Well, maybe you cite de Moivre, the finance industry, and system safety use, and say to your correspondant “you mean something different. I think that is unhelpful; and indeed our notion has historical precedence, so for the purposes of this conversation let’s use a different word for your new notion.” Or heshe could say the same to you. In any case, you agree to use two different words.

And for good measure, you write a blog post about it, as here.

This is not a new issue. Here’s a story from six and a half years ago. In the May/June 2005 issue of IEEE Software, Richard Fairley proposed a definition of risk for the Software Engineering Glossary of the IEEE (which is supposed to be canonical, although it turns out that Prof. Fairley doesn’t think so):

(Richard Fairley, proposed IEEE Software Engineering Glossary): The probability of incurring a loss or enduring a negative impact.

So a risk is a to be a probability, which means all risks have values between 0 and 1. Tell that to Lehmann Brothers. Well, I guess you can’t any more. Try Bear Stearns and Morgan Stanley. But we’re talking software, not money.

In common use, someone talking to his teenager speaking of “the risk of your not catching the bus in time” is likely talking about the chances of that event. Someone talking of “the risk that Lehman Brothers will go under” is likely also meaning the chances. But someone talking of “the risk of Lehmann Brothers going under” is likely also thinking of the repercussions as well as just the chances. So much meaning can a relative pronoun versus a copula+gerund carry! As with any other term you wish to be a technical term, you need to decide which meaning (of, here, two) you are going to use. And stick with it. What should be clear is that software engineers working in safety-critical systems need to speak both of likelihoods or chances, and about expected levels of loss. It seems obvious to use “chance” or “likelihood” or “probability” for the former, and some other word for the latter. Since it has been called “risk” for 300 years, why not carry on doing so? And so it is. But some people choose differently. If one is then going to use “risk” to mean “likelihood”, what word does one choose to mean the combination of likelihood and severity? There is not an obvious candidate. But you do need a word for it.

I wrote to the author, Prof. Fairley, Richard Thayer, the person overall responsible for the SW Glossary, and Merlin Dorfman, I believe the IEEE editor responsible for the section, pointing out de Moivre’s definition, the definition from Nancy Leveson’s book Safeware (Addison-Wesley, 1995), and that from the standard for functional safety of E/E/PE systems, IEC 61508, which all cohere modulo the caveats above.

Here is de Moivre:

The Risk of losing any sum is the reverse of Expectation, and the true measure of it is, the product of the Sum adventured multiplied by the Probability of the Loss

Here is Nancy Leveson:

the hazard level combined with (1) the likelihood of the hazard leading to an accident… and (2) hazard exposure or duration…

[The notion of hazard level is] the combination of severity and likelihood of occurrence.

Here is IEC 61508:

combination of the probability of the occurrence of harm and the severity of that harm

I also copied my note to Fairley in this note to the York Safety-Critical Mailing List.

Dorfman agreed that the definition could be misunderstood, but that “I believe the reader is given a fair, complete, and accurate picture of the use of terminology in this area.”. “Accurate”?

What do you do if you are a sofware engineer working in safety-critical systems? Use the IEEE SE Glossary definition, or use the IEC 61508 definition? Use different definitions for different meetings, depending on who is there? And what happens if you misjudge your audience?

Thayer was dismissive. The entire content of his reply:

The overall title of the glossary is Software Engineering Glossary.  This covers it I believe. 

In other words, he doesn’t care much for the dilemma of the software engineer working in safety-critical systems. One could well wonder why he is editing this vocabulary if he doesn’t care about such issues.

I responded to Thayer and Dorfman:

The use in finance and in PRA of the notion of risk equates it to the expected value of loss. A partial list of standards that use some version of this notion is

* IEC 61508, the international standard on functional safety of E/E/PE
safety-related systems
* IEC 300, the international standard on dependability management, in
Part 3, Section 9, “Risk analysis of technological systems”
* IEEE 1228, the standard for software safety plans
* the American Institute of Chemical Engineers guidelines for safe
automation of chemical processes
* US DoD MIL STD 882C, System Safety Program Requirements
* USAF Systems Command, Software Risk Abatement
* CENELEC 50129, Railway applications: Safety related electronic systems
for signalling (the European norm for railways; derivative from IEC
61508)
* European Space Agency Glossary of Terms
* UK Ministry of Defence Standards 00-56, safety
management requirements for defence systems; and Def Stan 00-58,
HAZOP studies on systems containing programmable electronics
* German Standards Institute (DIN), DIN-V-VDE 0801, Principles for
computers in safety-related systems

In particular, I expressed my concern that the IEEE as an organisation had publically given two meanings for risk pertaining to software engineering: one in IEEE 1228 on software safety plans, and another in the Glossary proposed by Prof. Fairley. I got no response.

Prof. Fairley responded, inter alia:

Concerning my definition of risk:  In most, if not all, situations encountered in software engineering, “risk” is the composite result of numerous factors.  In the glossary, I characterize these as “risk factors,” each of which is assigned a probability and an impact (or a range of each).  Risk factors are usually interrelated (e.g., an inaccurate size estimate affects schedule, budget, memory usage; an inaccurate schedule estimate affects product quality) so overall risk (i.e., probability of suffering loss) must be calculated using conditional probabilities or Bayesian analysis.  It is not possible to characterize a situation by a simplistic pair of numbers, unless one is dealing with a narrow, well-defined situation such as a game of chance.  It is dangerous and misleading to attempt to characterize a complex situation in this way.

Given the constraints of a glossary, it was not possible to explain the rationale for my definition or why it differs from the traditional definition; nor was it possible to explain the basis of definition for the other terms in the glossary.

Which to my mind is confused. If risk is “the composite result of a number of factors” each of which is “assigned a probability and an impact”, why ignore the impact and define it as a probability? Either it is a probability simpliciter, or it is the composition of a number of items, each of which exhibits a probability and an “impact”. It can’t be both.

That was it. End of story. The section editor thinks the definition is “accurate”; the Glossary editor is unconcerned; the author is confused. No one seems to worry about the IEEE proposing two incompatible definitions of risk in software contexts.

I wrote to some colleagues I thought might be interested: Dave Parnas, John Knight and Bev Littlewood (as well as a couple of German colleagues), explaining my dissatisfaction with this state of affairs.

Dave sympathised with my frustration, which was similar to his. He said he had seen lots of examples, and that he considered trying to write a glossary for SW terms a fool’s errand, and explained why. John thought this situation to be serious, the Fairley definition of risk wrong, and deserving of public correction. He also said that many people are concerned about a lack of precision and took Dave’s comments to reflect that. Bev strongly agreed with both John and Dave. He was particularly concerned about the dismissive response.

Continuing along the same lines, here is the definition of risk from the US National Research Council study Understanding Risk: Informing Decisions in a Democratic Society (National Academies Press, 1996), p215 (you can read this study on-line):

A concept used to give meaning to things, forces or circumstances that pose danger to people or to what they value. Descriptions of risk are typically stated in terms of the likelihood of harm or loss from a hazard and usually include: an identification of what is “at risk” and may be harmed or lost (e.g., health of human beings or of an ecosystem, personal property, quality of life, ability to carry on an economic activity); the hazard that may occasion this loss; and a judgement about the likelihood that harm will occur.

So descriptions include a likelihood of harm and an identification of what may be harmed or lost. Unless you are a software engineer using the IEEE Glossary (but not IEEE 1228), in which case it’s just a number between 0 and 1.

Here is the definition from a standard text, Probabilistic Risk Assessment and Management for Engineers and Scientists, Hiromitsu Kumamoto and Ernest J. Henley, IEEE Press (them again!) 1996, a book “sponsored by the IEEE Reliability Society”, p2:

Primary Definition of Risk: A weather forecast such as “30% chance of rain tomorrow” gives two outcomes together with their likelihoods: (30%, rain) and (70%, no rain). Risk is defined as a collection of such pairs of likelihoods and outcomes:

{(30%,rain), (70%, no rain)}

So they don’t even go for the combination of likelihood and outcome, nor do they designate certain outcomes as harmful. But if you do designate certain outcomes as harmful, then you can combine these values to calculate de Moivre risk and system-safety risk from this set.

The standard textbook Probabilistic Risk Analysis: Foundations and Methods, Tim Bedford and Roger Cooke, Cambridge University Press, 2001 (not the IEEE for a change ), discusses the definition of risk over some three pages in Section 1.2. They base their notion on that of S. Kaplan and B.J. Garrick, On the Quantitative Definition of Risk, Risk Analysis 1:11-27, 1981.

A risk analysis tries to answer the questions
(i)What can happen?
(ii)How likely is it to happen?
(iii)Given that it occurs, what are the consequences?

Kaplan and Garrick … define risk to be a series of scenarios s_i, each of which has a probability p_i and a consequence x_i.If the scenarios are ordered in terms of increasing severity of the consequences, then a risk curve can be plotted [of severity against probability of at least that level of severity]. The risk curve illustrates what is the probability of at least a certain number of casualities in a given year. Kaplan and Gattrick…. further refine the notion of risk in the following way [to talk about frequency of an event instead of probability, and then uncertainty associated with a frequency]

Again, this concept is somewhat different from that of a number between 0 and 1.

John suggested I contact the then-editor of IEEE Software, Warren Harrison, which I did. Warren suggested that the appropiate action would be a letter to the editor, allowing the author and the section and glossary editors to respond if they wished.

I never did so. I regret it.

So six and a half years later, here I am writing a blog post on it. I doubt the issue will go away. Neither will this note. I do think the IEEE should work to get its definitional house in order.