SC34 is the JTC1 committee dealing with the fast-track proposal that the MS OpenXML standard for office documents should become an ISO/IEC standard. At the moment most attention is focussed on the letter ballot that closes on 2 September. I have explained in many previous articles, most recently here that about a hundred “National Bodies” (NBs) who are ISO members are eligible to vote by submitting a voting form “G18″. Submitting a form gives you a ticket to the Ballot Resolution Meeting (BRM) which is currently pencilled in for February 2008. Comments submitted with the voting forms are pooled. The BRM goes through the comments and sees whether a subset can be adopted for inclusion in the draft so that the amended draft will attract the requisite majority (3/4 of those voting and 2/3 of P-members voting).

So (except in rare circumstances, unlikely to apply here) the proportion of yes, no and abstains in the letter ballot is of no consequence. What matters is whether a subsequent vote at the BRM is able to muster he requisite majority for an amended draft. The conduct of the BRM will be substantially in the hands of the “Convenor” Alex Brown. If an amended draft is approved, then the “Project Editor” (who appears to be Rex Jaeschke of MS) has the job of producing the actual text of the amended draft in accordance with the BRM decisions. Presumably in practice he is overseen by the Convenor and SC34 secretariat. According to the SC34 website completion of the process is currently forecast for December 2008.

But what would happen then (if it was ultimately approved)? 13.13 of the JTC1 Directives provides that

If the proposed standard is accepted and published, its maintenance will be handled by JTC 1 and/or a JTC 1 designated maintenance group in accordance with the JTC 1 rules.

Now what do you think has happened? Well just go to the SC34 website and find document 885:

The full proposal is about 45 lines long, but the important part is those words with Ecma TC45 … carrying out the key duties. Anyone who thinks this might be an unwise procedure needs to make sure that their NB is appropriately briefed for the SC~34 Plenary meeting in December 2007 when this proposal is to be discusssed.

Two points arise. The first is that it turns out that a vote of “abstain”, as opposed to a failure to return a G18 form, does give a National Body (NB) a ticket to the Ballot Resolution Meeting (BRM). This has recently been clarified by ISO, according to Alex Brown (in Comment 2), who will be the convenor of the BRM. So my earlier guess on what the JTC1 Directives meant on that point (which I repeated here) was wrong.

The second point is whether this document supports the idea that a NB can vote “Yes with comments” rather than simply “Yes”. Unfortunately, it is as badly drafted as the rest of the JTC1 Directives and supports both interpretations. In the middle we have if a national body votes affirmatively, it shall not submit comments. That seems clear-cut. But then lower down we have a space to tick the line comments (editorial or other) appended, which apparently implies that a NB can append non-editorial comments.

So this really does not advance matters. In earlier posts, I discussed how the wording of the main part of the JTC1 Directives bore on this issue. Irrespective of whether there is a majority for or against approval in the letter ballot, there is a BRM. NBs who return a DIS Letter Ballot form (shown above) before the deadline get invited to the BRM. All “comments” are pooled and essentially form the agenda for the BRM. It is fairly clear that the procedure envisages that NBs voting Yes will continue to support the draft in the BRM. Those voting No, must give “technical reasons”, informally known as “comments” for their vote. The concept is that those who voted No will change their votes to Yes if enough of their “comments” are met by changes to the draft. So the BRM goes through the comments trying to see which are worth adopting. However, any NB at the BRM is free to vote any way it wants. If the BRM is able to get the requisite majority for an amended draft, then that is adopted. If not the draft fails completely.

The precise BRM procedure is likely to be largely determined by the “convenor”, who will be Alex Brown. So his views are clearly important. He gave them in comment 6 to Bob Sutor’s blog on 2 August. Like me, he thinks that a NB probably can submit “comments” with a Yes vote, but “The fact remains, though, that comments accompanying an approval vote are less likely to get addressed than comments accompanying a disapproval vote”.

Since comments are pooled, it only matters in the case of a comment which is not submitted by any NB voting No. But then, it is hard to be sure until it is too late whether a comment will fall into that category or not. Because one thing is completely clear - comments submitted after the 2 September 2007 deadline will be ignored.

Of course, MS wants people to vote “Yes with comments” rather than “No with comments” in the letter ballot, because they think that will tie the hands of the NB’s delegate to the BRM. A delegate from a NB which has voted Yes is likely to feel an obligation to vote for approval at the BRM irrespective of what changes are made or not made, whereas a delegate from a NB which has voted “No with comments” is likely to feel an obligation to refuse to vote Yes until the comments are adequately dealt with.

At least I assume they think that. It may be that MS just misunderstands the JTC1 procedure and thinks the draft will go through without a BRM.

realdodgeman was correct on the deadlock in the International Committee for Information Technology Standards (INCITS) executive commitee which voted 8-7 against approving OpenXML according to the official INCITS site. But the “International” in INCITS is misleading. It is not an international body, it is simply the “National Body” which represents the US at JTC1 which is currently conducting a letter ballot on OpenXML. The INCITS executive committee vote is simply the latest step in the attempts to decide which way the US will vote.

Although the US vote is obviously interesting, it is just one among many. Approval of OpenXML as an ISO/IEC standard requires a 75% majority of those members of ISO that cast a vote (that is a simplification - the full detail is here and in several other articles on this site around the same time.). Moreover, the current letter ballot is most unlikely to be the decisive vote. It is far more likely that there will be a “Ballot Resolution Meeting” which will probably meet several times next year before getting to a final ballot.

It seems quite possible that the US will end up abstaining, in which case it will have no voice in the way this issue is decided. But even if it does vote, its vote has just the same weight as that of the smallest voting country! In some ways that it clearly unreasonable, but that is the way many international bodies work.

Of course, in this case a US company, MS, is clearly having a substantial influence on the votes of many National Bodies, much more than it appears to have had on the US body.

The BSI (British Standards Institute) is the “National Body” which decides how the UK should vote on ISO/IEC standards and represents the UK at ISO/IEC and JTC1 meetings. The BSI process is open in that anyone who looks technical qualified can participate in the bottom committee (which does most of the detailed work), but it is otherwise opaque. The membership of the committees is confidential, and their proceedings are confidential. But Alex saw “absolutely no evidence” that the process had been abused, and implied that the BSI process was of high quality.

From what I saw of it (as a member of the bottom committee and an observer at some discussions of the advisory committee above), I agree with that judgment. The MS and IBM members both behaved courteously and reasonably and neither made any obvious attempt to pack the committee. The BSI rules require that it proceed by “consensus”. That is a somewhat ambiguous term and is not the same as “unanimity”. It requires those who are obviously in a small minority to back down and not press their views beyond a certain point. I certainly watched that happening on a number of issues.

But I still disagree fundamentally with Alex. It may be true that confidential proceedings in committees whose membership is not known can, on occasion, give excellent results. It is certainly true that such proceedings are easier for the participants, who are not exposed to lobbying or abuse. But standard-setting needs to be conducted in the open, just as courts need to conduct their business in the open. Exactly the same principles apply. I would favour opening all meetings to press and public and keeping reasonably full minutes which were published on the internet. Justice needs to be seen to be done.

Oddly enough, I suspect that Alex is not completely unsympathetic to this view. He was the convenor (a kind of chairman) of the bottom BSI committee, so was presumably responsible for the decision to display its technical work in a Wiki on the internet. Although the public could not see who was responsible for which entries, the Wiki did have the effect of placing the key technical work into the public domain. In practice, I think that will make it exceedingly difficult for the BSI to vote anything but “No with comments”. It would look ridiculous if it abstained or voted “yes”, or even voted No with only trivial comments, when everyone can see that its technical committee has come up with large numbers of significant comments.

However, we will have to wait and see!

The purpose of a standard is to standardize - to establish a single way of doing something. This is generally of considerable benefit to users, because it allows them to use the products of any vendor. That is usually a prerequisite for competition. It can also benefit vendors, but not always. Indeed a dominant vendor often prefers a proprietary specification.

OpenXML is the latest MS file format for office documents. It has already been approved as a standard (Ecma 476) by the standards body Ecma International. “Office documents” is a well-defined area. It means documents that are intended to be printed, usually on regular office paper (A4 in Europe, Letter in the USA), and which are subject to editing. Once they have been finalized they can be “archived” in a different file format (usually pdf). Different standards are appropriate for archived documents, web documents and various more specialist categories.

Last year, ISO/IEC 26300 (popularly known as ODF or OpenDocument Format) was adopted as the ISO/IEC standard for office documents. ISO/IEC standards carry a particular prestige and uniqueness because ISO and IEC are intra-governmental bodies, so the standards are effectively government-endorsed standards. Not all important standards get this imprimatur. For example, in the internet software area most of the important standards are “W3C Recommendations”, and some ISO/IEC standards have flopped (failed to gain acceptance in the marketplace).

Attempts have been made over the last few months to argue that we need “competition in standards”. If you pause for thought, that is silly. Competition in standards is precisely the evil that standardization is designed to eliminate! Attempts have also been made to argue that OpenXML covers a different type of document from ODF. This argument is also totally bogus, see for example this article and many others on this site.

The first product using OpenXML, Office 2007, was only released this year, so it currently has only a modest market share. However, MS has over 95% of the market for office document software, so it is likely that OpenXML will become the overwhelmingly dominant format over the next few years.

95% market dominance is almost invariably an indication that competition is not working. Such extreme dominance is rarely sorted out by the market; it requires government action. In MS’s case this was recognized by the US government, which brought an anti-trust case. This established a consistent pattern of abuse by MS over a long period and also showed that MS had been able to “lock-in” customers by file formats and other devices. For reasons that have not yet been adequately explained, the incoming Bush administration decided to take no effective action against MS. A parallel case by the European Union has not yet been resolved.

In the meantime, the MS monopoly in office document software persists. A glance at the latest 10K shows that about a quarter of MS’s revenues come from such software and that the profit margins are extraordinary. An article on this site explains the peculiar characteristics of software that make this possible, and shows the serious scale of the problem.

Historically, MS has used proprietary “binary formats” for saving office documents. Apart from the anti-competitive effect, this has also meant that users have been forced to use MS software in order to access their own documents. This is clearly a highly unsatisfactory situation for governments and major businesses. This issue has been substantially ameliorated by the publication of Ecma 476 (the existing OpenXML standard). There is no doubt that the publication of this standard was a major benefit to users. Not because it is a standard, but because the specification has been put into the public domain and because MS’s freedom to change it arbitrarily has been reduced (although not by much, given the way Ecma seems to work).

It is certainly an unsatisfactory situation when the dominant vendor does not support the ISO/IEC standard. But adopting a second standard is not the answer. The answer is obvious: MS should adopt the existing ISO/IEC standard. It would also be highly desirable for governments to mandate the existing ISO/IEC standard for their own documents, in order to put pressure on MS to adopt it.

Of course, this prospect is highly unappealing to MS. It will be forced to incur some significant cost and will be opening the door to its competitors. I find it hard to be sympathetic. It chose to boycott the development of the ODF standard. It should suffer the consequences of its misjudgment.

… this Promise also applies to the required elements of optional portions of such specifications.

So, just to be clear, let us go through this again. The OSP takes the form of a promise not to sue where an implementation of OpenXML requires the use of a MS patent and that patent is used without permission. This approach is currently the state of the art. It is used by MS, IBM and maybe others. It has several advantages for third parties:

(1) the OSP applies to a long list of specifications, not just OpenXML, so people only have to get used to one wording;

(2) no consultation with MS is required, the promise applies automatically (and under most systems of law is legally enforceable when a third party acts in reliance on it);

(3) it is royalty-free (whereas ISO/IEC, for example, only require reasonable royalties negotiated in a non-discriminatory way).

I have discussed in an earlier article the issue, which many have raised, that it does not apply to future versions of the standard. My conclusion is that whilst it would be good if MS was prepared to make this extension, I can understand its reluctance, and as far as I know no one else has done it (certainly the IBM and Sun promises on ODF do not appear to do so).

So the remaining question is whether the OSP actually covers all relevant patents. The doubt, which I raised yesterday and the day before, is whether it covers any patents needed for the many “optional” parts of the standard.

Starting at the beginning, the OSP is:

… not to assert any Microsoft Necessary Claim against you … [in relation to] any implementation to the extent it conforms to [a specification in the list, which includes Ecma 376, the current version of OpenXML] … ‘Microsoft Necessary Claims’ are those [patent claims] … necessary to implement only the required portions of [Ecma 376] that are described in detail and not merely referenced in [Ecma 376] … this Promise also applies to the required elements of optional portions of … [Ecma 376].

Despite its inordinate length, Ecma 376 does have several elements which are not “described in detail” but “merely referenced”, typically legacy features. So that is certainly a problem.

But leaving that aside for a moment, the critical wording is that coverage extends to:

those [patent claims] necessary to implement only the required portions of [Ecma 376] … this Promise also applies to the required elements of optional portions of … [Ecma 376]

Certainly I deserve some humble pie (perhaps someone could supply it, I am not sure I have ever seen it, I believe it is made from deer). But whereas I was earlier fairly clear what the wording meant and that it did not cover enough, now I am just mystified. What on earth does it mean?

“only the required portions” plus the “required elements of the optional portions”

Evidently, it is envisaged that the “portions” of a specification can be classified as “required” or “optional”. But the optional portions can be further subdivided, so that they contain some “required elements”. Unfortunately, it is certainly a possible view, as discussed here and here that essentially everything in Ecma 376 is optional, so if this right, the OSP still covers no patents relating to it.

Do not get confused between “necessary” and “required” (which is easy to do)! The OSP only covers patent claims which are “necessary” for implementing the specification. That is perfectly reasonable. So that is not what “required” is on about. It is a property of the specification not of any related patents.

But the clear implication of Stephen McGibbon’s comment is that MS was not simply intending to exclude all optional parts of OpenXML from the OSP. So what on earth was it intending to do? And there is still the problem left aside above about legacy features not fully described.

So, unless I am still missing something or being stupid (more pie), I come back to my earlier conclusion that it is completely unclear whether or not the OSP is sufficient or not, and the National Bodies need to insist as a matter of urgency that MS clarify the position.

I mention all that because of the word “only” in this critical extract from the MS OSP:

those claims … that are necessary to implement only the required portions of [OpenXML]

Why is “only” there? It certainly reads oddly. A clumsy writer might have used it thinking that it just reinforced “necessary”. But the OSP was clearly drafted by, or in conjunction with, lawyers, probably expensive lawyers, and they are careful with words.

Suppose OpenXML has a “required” feature X and another feature Y which is not “required”, and that a MS patent P is necessary to implement each of these two features. If you deleted the word “only”, then the OSP would cover P, because it was necessary for X, a required feature. But, as drafted, the OSP does not cover P, because it is not only necessary for X, but also for Y, which is not a required feature!

Either MS is devious, or its lawyers had an off-day. I am inclined to think the latter, but in any case we urgently need clarification from MS.

Here I want to deal with a rather different point from most of those that Rob is making, namely the bearing that this has on MS’s OSP.

In OpenXML and patents - Part 1 and Part 2, I discussed whether MS’s Open Specification Promise (OSP) was (1) compliant with ISO/IEC requirements, and (2) compliant with best practice in this area, which I took to be what IBM has done.

My conclusion was that although there are differences between IBM, Sun and MS, they all seem to have adopted a broadly similar approach which goes some way beyond the ISO/IEC requirements. In other words, MS is compliant with best practice.

However, there was one important caveat which resulted from two exclusions in the wording of the MS OSP. One of these (inclusion by reference) need not concern us here. The other was that the OSP does not apply to those parts of the standard which are not “necessary”. I had not researched as carefully as I would like which those parts were, so I simply said that this was an important issue which needed to be clarified. Clearly, if significant parts of the standard are excluded from the OSP, then far from being compliant with best practice, it is not even compliant with the ISO/IEC requirements.

Rob’s basic assertion is: “Everything in OOXML is optional. This should be repeated until it sinks in. Everything in OOXML is optional.” If that is correct, then it would seem to follow that nothing is necessary and hence that the OSP is completely vacuous in relation to OpenXML and totally fails even to meet the ISO/IEC requirements.

Is this reasoning correct? Well, I don’t know.

We need to look carefully at some wording. Like most standards, OpenXML distinguishes between normative and informative parts. Roughly speaking the operative parts of the standard are the normative parts, whilst the informative parts are there to help you understand the normative parts. So that is not the distinction we are looking for, but it tells us that we can confine our attention to normative parts. Part 1 of this massive standard is relatively short (a mere 163 pages) and sets out “fundamentals”. Clause 2 (normative) explains “conformance”. A “conforming document” must meet various criteria (clause 2.4), but can (unsurprisingly) be almost arbitrarily short and empty of content. A “conforming application” “shall not reject any conforming documents of the type expected by that application … [and] shall be able to produce conforming documents”.

Developers are obviously interested in “conforming applications”, and the things that are necessary for a “conforming application” would seem to be the things that are “necessary” in the OSP language.

At first sight, essentially nothing positive is required of a conforming application. It must be able to produce conforming documents and it must not “reject” anything. A sufficiently empty document could certainly be made free of patent infringements, and if displaying nothing counts as “not rejecting” a document when it is opened, then an application could certainly do that without infringing any patents.

I wonder, however, whether a court would take that line. Suppose the application was intended to be a full-fledged document suite, so that it “expected” to be able to open any document, surely failing to do so in a useful way would count as “rejecting” the document. So if implementing the specification in the standard sufficiently to be able to open and properly display a complex document required use of MS patents, then surely these patents would be (in the words of the OSP) “necessary to implement only the required portions of [OpenXML]”? The fact is that I don’t know!

So I think I end up more or less where I got to a few days ago: it is completely unclear whether or not the OSP is sufficient or not, and the National Bodies need to insist as a matter of urgency that MS clarify the position.

(A) it gives you a ticket to the Ballot Resolution Meeting (BRM);

(B) any comments (formally, “technical reasons”) you append to a No vote get pooled with other NB comments, and the comment pool essentially forms the agenda for the BRM.

2. It is irrelevant whether there is a majority for or against OpenXML (or a split vote) in the letter ballot. There will still be a BRM. [Yes, there are some precedents for not holding a BRM, but it is almost inconceivable that there will not be one in this case.]

3. There are only two kinds of vote: Yes (formally, approve), and No with comments (formally, disapprove with technical reasons). Many people who should know better get this wrong. Look at the JTC1 Directives! [Yes, it is arguable that if you append unique comments (not made by any NB voting no) to a yes vote, they may be accepted as part of the BRM comment pool, but it is not certain. In any case, by voting yes you are signalling clearly that you do not really care about the comments.]

4. Prior to the BRM the comments from all National Bodies will be sorted, “deduped” (duplicates removed), and non-technical comments junked. It is hard to tell at this stage, but it is likely that this process will give somewhere in the range 400 - 1,000 comments. It could take the BRM a substantial amount of time to work through these comments. Much will depend on how the meeting is organized and chaired. It seems unlikely that the BRM will start before February or March 2008. It could have dozens of meetings, spread over many months (formally, these may count as the same meeting with adjournments). In any case, attending the BRM could be a substantial commitment of time. The real votes will take place at the BRM, so to participate you have to find representatives able to commit the time (and remember to vote Yes or No in the letter ballot, voting “abstain” doesn’t count);

5. You can vote however you please at the BRM, without regard to your letter ballot vote - although you may attract some adverse publicity if your votes are flagrantly inconsistent;

6. The BRM is fundamentally a negotiation. That means you have to approach it as a professional negotiator. Technical types are often purists. They say objections X and Y are inconsistent, so I can only put one forward in my “No with comments”. This is completely wrong. A comment is effectively a suggested amendment. A good negotiator often puts forward inconsistent amendments, or amendments he cares little about. One thing is certain, you are not going to get everything you want. Putting forward at the start only the things you regard as absolutely essential is bad negotiating! You need chips you do not value that you can trade for chips you do value.

7. A “no with comments” vote in the letter ballot is effectively a “conditional yes” vote. You are signalling that if you can be adequately accommodated on the comments, then your vote will switch to yes. But suppose you want to vote an “unconditional no”? Then you should put forward “technical reasons” which are in practice impossible to accommodate - for example, that the standard is incompatible with the existing ISO/IEC standard in the area (ODF). Of course, you might decide, after the BRM had agreed to substantial changes to the standard, that although this comment was still valid, you wanted to vote Yes. You are entitled to do so, and no one is likely to complain too much. After all, compatibility with ODF is a matter for your (technical) judgment. On the other hand, whereas no one can stop you voting yes in the letter ballot and then voting a solid no at the BRM irrespective of what changes are proposed, that course is likely to attract criticism. Indeed it might garner ill-will from other NBs that you wish to work with in the future on other matters.

There is obviously a difficulty about a covenant not to sue covering future changes, because no one knows what they might be. The general concept of the covenant not to sue is that the patent rights are not completely given up, but may be maintained and yield royalties or block rivals in other areas apart from the standard in question. It is possible that future revisions of the standard might radically extend it, perhaps against the wishes of the vendor, into an area where the vendor was enjoying a significant royalty stream which it did not wish to forego.

As far as I can see, IBM and MS both restrict their covenants to the existing standard, although IBM makes clear in the associated FAQs that it would normally extend it to future revisions. MS has been repeatedly attacked by the Open Source community for its failure to cover all future revisions of OpenXML upfront.

On the other hand, Sun has sought to cover future revisions of ODF by extending the covenant to: “any subsequent version thereof (”OpenDocument Implementation”) in which development Sun participates to the point of incurring an obligation, as defined by the rules of OASIS, to grant (or commit to grant) patent licenses or make equivalent non-assertion covenants”. This has also been attacked in a pro-MS blog on the basis that “if Sun does not like the development of ODF it can hold up the development of the standard until there is certainty that it does not violate any of Sun’s patents. This is quite a big deal …”. I am not sure I quite follow the reasons why it is a big deal, but I can see that it suffers from similar problems to the IBM/MS approach.

I cannot get too excited about this debate. ISO/IEC require that licences to any necessary patents must as a minimum be made available on non-discriminatory terms, and MS has clearly opted for the covenant not to sue approach, so in practice either MS would extend its OSP to cover the revision, or the revision would have to be redrafted to avoid the relevant patent claims. It would be nice if MS took the view that an intergovernmental body like JTC1 was not going to absurdly over-extend a future revision of OpenXML simply to deprive MS of the benefit of its patent claims, and hence agreed upfront that the OSP covered all future revisions. But IBM and Sun have not yet gone much further than MS, so I cannot see why MS should be singled out for criticism on this point.

The second point of criticism (see, for example, Grokdoc) is that in the OSP the patent claims covered are those “that are necessary to implement only the required portions of [OpenXML] that are described in detail and not merely referenced”. The wording is infelicitous, but there appear to be two exclusions:

(A) patent claims necessary for parts of OpenXML that are not “required” parts; and
(B) patent claims necessary for parts of OpenXML that are merely referenced, not described in detail.

It is not clear to me what MS was trying to achieve by these exclusions, but they present a serious problem in the context of OpenXML because large parts of the standard are apparently optional and there are numerous references to material outside the standard itself. Accordingly, it is completely unclear at first sight, what effect these exclusions have.

The Common Guidelines (.pdf) which contain ISO/IEC’s minimal requirements on patent issues require “any party participating in the work” to “draw … attention to any known patent or to any known pending patent application, either their own or of other organizations”. There is the usual problem of poor ISO/IEC/JTC1 drafting, but this requirement clearly places an obligation on MS to disclose all relevant patent claims. As far as I know it has not done so, presumably on the basis that the OSP makes it unnecessary for anyone to negotiate a licence as envisaged by the Common Guidelines.

However, this basis falls apart if it is unclear whether the OSP does have this effect. So it seems fairly clear that MS must:

(1) remove these exclusions from the OSP; or
(2) clarify which patent claims are covered and which are not, so that it is clear how it has complied with the ISO/IEC requirements.

(2) may well lead national bodies to look carefully at which parts of the standard are “required” and which are not. In many cases there are alternative ways of representing the format of a document, one of which is meant to accommodate “legacy documents”. Suppose that implementation of these alternatives is not “required” parts by the standard (which appears to be the case), then they are apparently not covered by the OSP. So if they are covered by MS patent claims (which seems likely in at least some cases), then the only vendor who can implement them is MS or anyone whom MS chooses to favour with a licence.

However, the whole rationale for having this standard in addition to ODF is, according to MS, the way in which it allows legacy documents to be represented. So these “non-required” parts of the standard are actually absolutely fundamental, and it would be absurd, and quite contrary to ISO/IEC policy, to have a standard which only a single vendor (and its favoured associates) could implement (because of patent constraints).