Background on "OMG finds true love"
Here are the program notes for the play that is this very moment being lived out before our eyes. Its participants are right now performing the second of its three Acts, which are entitled, respectively, Past, Present, and Future.
You may recognize many of to-night’s protagonists – and even some costume, props and scenes – from the same author’s recent works (of which a summary for present purposes is in the background document).
You may gauge the relevance of the allegory by its consistence with the other accounts and by its correspondence with your own experience of recent history. (The playwright knows the story’s plausibility by its correspondence with his other - and more tangible - kind of programming.)
* * *
Once upon a time there was a beautiful princess. Her father, King Customer, had no other children (He knew he should have but one set of standards if his kingdom is to grow harmoniously, supportive of his citizens’ ventures). He dearly loved his daughter, he had invested all his hopes in her, and he would endow her family with all his riches.
So he had enjoined the wisest sages of his realm to help establish the finest standards-setter amongst the sons of the influential Archbishop Informatico-Technologus. As you might imagine, each most eagerly sought to be the elected one, and had soon – many would say afterwards, too soon – plotted all manner of stratagem.
Now the youngest son had well taken heed of the sense of urgency in the kingdom, and was evidently more enterprising and energetic than his elder and rather slow-moving brothers.
But he knew it, and knew that others knew it. So he soon grew to put on airs, and had taken the habit of admiring himself attired in one of his father’s most venerable mitres. (That mitre’s twin crosses at its peak were most naturally mistaken for the letters "IT", and often even "II" by the more short-sighted faithful, too preoccupied as they then were by the pervasive but superficial Interface Inheritance tradition and – latterly – under the sway of the brilliant but slyly quick-fingered Sorcerer of Redmond. This latter had adopted and adapted – "embraced and extended" – many recipes from others, including II, and had fed most of the citizens of the kingdom with addictive cookies, keeping their attention with little games. But bigger tricks were coming from such wide experience and from the many adepts that fame attracts, so many saw him as the imminent usurper of the archdiocese.)
Thus it was easy for our hero to convince himself that if only he could, thus bedecked, just meet the princess alone, she would be so taken by his decisiveness and so enamoured of such high majesty that the beautiful prize – and the dowry – would certainly be his ...
So it came to be, in Act I of our play, that this youngest son succeeded in contriving such a meeting but was enormously frustrated in his grand ambition. The princess – like her father – was too exacting and too subtle, and would have none of the still rather crude lad. Thus, impulsively, not only did he behave most improperly, but even in his wounded pride spread false rumours that he had won the princess’ heart and that they had together decided on their firstborn’s name: OMA.
Fortunately, however, the experienced and wise King Customer could see the boy for the muddled and confused young rogue he was, certainly not yet capable of expanding or even managing a kingdom as diverse as this, with so many fractious barons and productive hence deservedly-demanding good citizens in it. The Archbishop and the sages had at the outset advised the king to expect something more generalized as a foundation for the future, so he was not fooled by the boastful but simple-minded and even classic "bird in hand" OMA story. Thus it became evident to the more active and perspicacious citizens that no sound marriage would flow from that episode.
But there was indeed an OMA developing, and the self-reproachful princess, in her eagerness to please her worried and doting father, was covering up her growing belly and the foolish misdeed that had caused it.
The sages nodded wisely at the turn of events, but took no effective action to secure a worthy succession, or address the princess’s apparent lack of serenity.
Happily, however, she in due course came to notice that the ill-begotten thing would eventually be still-born.
* * *
Act II opens with the kingdom in even greater turmoil and anxious about the succession. The various bishops’ sermons are creating confusion, the sages merely adding to it. Rather than allow the Sorcerer to amass yet more influence over the more impressionable citizens, the ever more desperate King Customer is willing to give the ambitious archiepiscopal offspring yet another chance. So, to great optimism in the realm, he has even anointed him the Bishop ArchiBoard.
But the still-vain knave is not only ignorant of the impending miscarriage – else preferring not to think of it – he even remains in love with his father’s old mitre, and is often seen strutting the cloistered halls of his ivory-coloured castle thus bedecked.
The sages in the castle’s towers do nothing about it, whether or not they like II, though some do manage applause or even a little show of "II"-symbol-manipulation when they feel obliged to earn their suppers.
* * *
So all is prepared for the centrepiece of our play.
The drama having played rather too long already, our maturing Bishop ArchiBoard, after much consultation with other bishops and the barons of the realm, has decided on a plan of action.
He has organized that a Grand BOF Banquet be held in his own castle. All the chefs of the archdiocese have been invited to submit their finest menus for the feast. The winning menu – ArchiBoard is sure (and he was right) – will so impress the demanding tastes and melt the hearts of both princess and king, that it will be the delicious harbinger of his triumphal wedding.
However, ArchiBoard’s misguided old fixation still pervades the event. Five of the six chefs have submitted complete menus. Though each one has its own peculiar style, all the main BOF courses – as in a bizarre ritual of loyalty – have been cast into forms of insipid and indigestible IDL and are bedecked with the most ridiculous "II"-shaped decorations! (The one exception only has proposals – admittedly excellent – for some of the CBO side-dishes: its creator’s conscience could not allow him to make-believe, as at least one of the more perspicacious of the BOF chefs had done, out of optimistic but misplaced respect of their client and host, our ArchiBoard.)
The scene is now set for gathering disaster, into a tragedy of epic proportions. What is our courageously persistent ArchiBoard possibly to do? The expectations in the kingdom are such that a menu has to be chosen, but even ArchiBoard ... even though he can’t help himself from feeling that they do have a certain impressive aura about them ... even ArchiBoard can see that the only submitted options have no synergetic and exciting semantic substance inside their hollow shells, and that any chef cooking to such recipes would have to engineer his own ingredients into them.
In addition there is both injury and insult: our embattled ArchiBoard is apprehensively eyeing his brilliant and friendly neighbour, Maestro Rational. Frictions are looming over the Maestro’s venerable UML tree. Its deep state-roots have long been opening cracks in the castle walls, spoiling its ideal beauty, and now its spreading branches are dropping leaves into the BOF dishes.
Meanwhile the sages – knowing that the decision is not theirs, and wisely reflecting that they are better off well away from messy commercial battles – have retreated into their towers, whence, like ominous vultures, they await the pieces.
But here, gentle reader, is where you yourself might come to our hapless ArchiBoard’s rescue from the desperate predicament into which cruel circumstance has thrust him despite his best and slowly-reforming intentions. Read on to find your own allotted role in Act III, in which the princess and King Customer are finally gratified and fulfilled.
* * *
These are the broad lines of how Bishop ArchiBoard of OMG was brought back on track with MACK, so simply, almost simplistically. Your own opportunity is fully clear by now.
The third and final Act opens just as it has begun to transpire to our select audience that our ever-searching ArchiBoard would find the secret to the succession of King Customer’s vast and ancient lineage in the most normal place: the Archbishop’s own kitchen!
Our Archbishop Informatico-Technologus – as we have all long known – has an amazingly well-provided larder. There is a great array of ingredients for the choosing: common or exotic, some very mature, others quite fresh. In fact, there is more than enough from which to conceive the most surprising yet nourishing, pleasantly-rich and appetizing meals to appeal to every palate.
Now it is here that Little Jack Horner the kitchen boy, like Cinderella, has long been working away. In the remotest corner of the kitchen he is quite safe from the influence of ArchiBoard and that absurd mitre (During the build-up to the period of the young ArchiBoard’s "insidious shift [...] – nay, drift!" under the banner of that old mitre, Jack had been working with a strong non-procedural 4GL and some DBMSs). Never having intended a castle-bound life, he has also long been slipping out into the streets, homes, working-places and meeting-places, mixing with the citizens of the kingdom, so he has been well observing their ways, their needs, their tastes, their dreams...
So, after enough experience of the work of famous chefs, and by dint of long perseverance over many years, after varied experiment and painful learning, and much help from humbler cooks, Jack is now busy preparing a dish with the qualities that King Customer, all the citizens and even ArchiBoard have been imagining, and the Archbishop has been promising.
Not surprisingly, it contains mainly the commonest ingredients, so overlooked by the archdiocese’s fancy chefs but much appreciated by the citizens of the kingdom. It also makes judicious use of some of the most mature ingredients which even the sages had ignored or disregarded.
Now Jack, not having Cinderella’s charms, and having other kinds of ambitions, is not tempted by the Great BOF Banquet. So he has invited some of the chefs to join him in the kitchen for the final preparation of a dish for the princess. Some do so, and even add many significant finishing touches to the creation. Impressed and thankful, Jack shows them where a magnificent plum is hidden in it, and how, when their turn comes to be served, to bring it back to share with him.
That is what he has long and consistently had in mind. Indeed, he was quite specific in the opening lines of his first public invitation, nine months ago: "... surely together we can pull a plum out!"
In the event, of course, the Metaset dish truly wins our hero ArchiBoard his princess, and Jack’s MACK recipe has enough variability and other potential complexity to feed and delight King Customer, his sages, his barons and all the citizens of the realm happily ever after.
The sages delightedly refine and reshape the various ingredients, and compose many elegant and nourishing variations on the theme.
The Maestro, respectful of the new BOF dish, prunes his tree accordingly, while ArchiBoard adjusts his plan to the tree’s powerful roots and repairs his castle.
ArchiBoard, assisted by the Maestro and his many followers and rivals, ensures that all derived recipes remain in graceful harmony with the earlier offerings, and are easily digestible together. The task is surprisingly easy, such is the good foundation and potential of the basic pattern.
The Sorcerer is greatly confused by the discrediting of the old mitre, his wand loses its magic, and his erstwhile influence wanes rapidly. So the king, not wishing to see confusion amongst the many citizens who had come to depend on his old tricks, suggests he more closely considers the corrected culinary track. Quick and inventive as ever, the wizard sees the new recipes, converts his lair into a workshop and his wand into a chisel, and nobly saves himself, his household and all his followers from damnation and ruin by fashioning fine new serving-bowls for both new recipe and old.
Jack chooses to remain in the corner he knows so well, in the companionship of his neighbours, ever learning with them, and enjoying the more delicious and healthy meals that they could by then all prepare so easily together, thanks to the new example and leadership of the well-paired hence fully-reformed, wiser, and more humbly-hatted ArchiBoard.
* * *
Some key details
A wishful-thinking fantasy? Well, the present being the present, it is from there that the recommended outcome diverges from its current inertial and predictably tragic course, so we return to that point and examine the underlying causes of its premature or date-rape build-up (We may ignore its exact mechanisms as having – at most – merely historical importance). We also look at some of the reasons for the happy future outcome depicted.
So we are back at Act II. That promises to be quite enjoyable, as it is the most intrigue-filled and initially most unfathomable part of the play. More importantly, though, if its issues are not understood, it could delay our fairy-tale’s eventual realization (sure enough though its final end-point already is). Delay would also lead to an unnecessarily messy process for all concerned.
The exercise is doubly useful, for if the new culinary team is to be effective quickly, a fittingly diplomatic resolution to the predicament needs to be devised to prepare the way forward. Excuses need to be found so that the relevant chefs may take proper leave of the assembled company, and with enough provisions join Jack in the kitchen. (Jack insists that they can assure ArchiBoard that the king and princess would be kept waiting less than any other BOF-recipe would do.)
The author of these program notes once rather heavily summarized the situation in this way:
"Thus all five BOF submissions – thanks to MACK-based hindsight – must be dismissed as (in a Shakespearean way) "signifying nothing": much mouthing of the spirit but no resuscitation of the substance of OO ideals and opportunities, no awareness of the classical object model miscarriage (of an ill-formed date-rape product of blinkered fixation on a misconception of encapsulation, with no respect of complexity, no deep mutual benefit from interconnectedness), no reform of the OMG services and facilities architecture (see below), nothing more friendly than harsh ACID for workflow, no high-granularity BO architecture with synergy, nothing substantial to chew on after all."
Let’s expand, phrase by phrase, that paragraph from my document of Workers’ Day (as in South Africa we have entitled May Day, May 1):
"Thus all five BOF submissions must be dismissed"
Eight submissions have been published on the OMG Web site:
So I have bet on JBOF, rather than on my own "long shot". The latter does not depend on me alone. Considering my secrecy, it depends – for long-shot success – on finding an extraordinarily well-informed and perspicacious gambler, and in general such is not a creature to bet on finding!
That said, as time passes I am expecting the perceived risk to fade in any minds that can persevere and piece together all my argument, image and confirmed prediction. That picture’s very high degree of coherence and growing cogency is, I guess, not so obvious at this stage. (After MACK’s launch – if necessary without the OMG-linked teammates hereby sought – the whole ballgame changes anyway, as the completed picture’s clarity will indeed shine out as the targeted "complexity simplified".)
Unfortunately, however, such recognition will probably be too late for my "long shot" towards a "sooner and better" MACK launch not to fall short: the fairy-tale will not materialize and the OMG will have a tragically-unnecessary and messy short-term way ahead. Or can you prove me wrong?
"– thanks to MACK-based hindsight –"
But why not share that hindsight fully now?
Indeed! So let’s examine my apparently obsessive "trade secret" position (which – I must admit – goes against all my instincts, conditioning and experience involving all those contrary notions such as "truth for its own sake", "humanity’s heritage of knowledge", "the universe of learning", "shared discovery", "two heads are better than one", "brainstorming", and so on).
Jack is sure that only a very few additional chefs and assistants are required at this stage. He further believes that too many cooks would spoil the broth, that is, unless a new chef is a far better manager of others than he (admittedly not difficult, considering Jack’s relevant experience), and can ensure that an enlarged team escapes the effects of Brooks’ "mythical man-month" Law.
Uncontrolled cooks might also spill the beans. But what would be wrong with that? Why not any number of independent teams from anywhere else? Especially if all he wants (other than a share of the plum that he has contributed) is to be but another user of a universally-used full-cycle marketing facility (helping us all simplify complexity together) such as Metaset has always aimed to lead to?
True, any possibly ill-cooked dishes to the same basic recipe would certainly be seen in due course to be inferior. But that would most likely dull the taste-buds of the guests and ruin their appetites. That would spoil the occasion, and it is the success of an occasion which most quickly spreads the use of a fine recipe. And wide use is essential to reaping the intended benefits of a shared architecture: reusability and interoperability.
So it is my position – unless persuaded otherwise – that Metaset will be the only contender in the race for the first realization of MACK. Setting up and managing the institutional and operational system to ensure that that remains so would be one of the immediate tasks of a new team leader.
Furthermore, there are still numerous fine details of MACK to be concluded, and divergence on such details would hamper the realizations’ interoperability, as CORBA 1 suffered. My present position is that MACK 1.0 must be sufficiently complete to ensure immediate inter-realization interoperability, with Metaset’s and other realizations’ far superior repository, A&D and groupware facilities supporting the market-wide process, and with MACK’s innate version capabilities smoothly enabling the further evolution that will most certainly take place.
I might add that any chef who might be tempted now to try to cook with Jack’s recipe, however obtained or guessed, would do best to join the others overtly in the kitchen. Jack rather doubts that there would be too many there, and skilled and eager cooks are usually welcome. There is a lot to MACK that I have hardly written about yet, most particularly on "relativity" (See "relativistic realtime" in my faq q 3 reply) and the allied "time component of data" issues (See the faq q 6 reply). I must warn any bold reverse-engineers that I would bet no money at all on anyone’s ability to carry that one off at this stage without the lengthy and peculiar background of Metaset and MACK!
Yet another issue supporting a "trade secret"-based approach is that any new team members would have their own or any investing partners’ investments to recoup and risk-taking to reward. The incentive of that plum (See my faq q 8 & 11 replies) is better not thrown away. And our times favour healthy long-term diets.
On trying to share that hindsight anyway
The development of MACK was not a bottom-up process, nor was it top-down. As usual, it was a strange mix of both.
There are too many detailed facts at the bottom for anyone to select – or be pointed out – the appropriate ones and see the way ahead from there. It is also unreasonable to expect anyone to spot at the top – a priori – the single and small set of elements and axioms from which MACK is now built. Or to see how the entire applied construction will evolve in such terms (or "be deduced from them", as the myth has it) without seeing it in action.
But, bit by bit, a picture develops which is very different from the accustomed one. That is indeed the way of the market: a progressive matching of supply and demand. In the end, the product bought and used is not what was initially demanded, nor what was initially supplied. It evolves: supply suggests and learns, demand understands better and adjusts. New paradigms do result.
The strange combinations of opposites continue. I repeat my favourite trumpet call, "Ride The Mainstream!" But how can an advance be so different from a present direction that the call should be so insistent, and yet be so implicit in that history? Both "a new paradigm" and "The Mainstream"?
There is surely no easy or immediately convincing answer to that question. (I do like to philosophize – as I did in the "Fifthly" point in my faq q 3 reply – that it is indeed reasonable to expect that the very nature of our infinitely complex reality will forever support such major progress. But it is not reasonable to try to convince on that basis in this instance at this stage.)
Paradox again: Whenever there is a new paradigm, it is those most steeped in the old who have the greatest difficulty in seeing things differently. Yet it is also they who by the same token most need a resolution to the known or merely suspected (or at least alleged) problems in the status quo.
So – mindful of whom I’m addressing (and of Medawar's Dictum - see my paper and faq q 2 reply under the "new robes" point) – I think it’s worth while to point out, but not try to argue, some of the many seemingly little but problematic issues that are happily resolved by MACK’s different view on things. If you recognize the issue, good! If you don’t, don’t worry, just read on.
We may start with a bald though slightly ornamented summary of part of the fundamental contrast.
MACK is so very very different from what you are used to if the latter is conventional OO with its:
MACK at once captures and encapsulates the distilled essence of OO, which is the plain logic that everybody uses all the time:
‘... as (in a Shakespearean way) "signifying nothing"’
Macbeth in Macbeth, on "life" (for which, in our story, read "anything OMA-based"):
It is a tale, told by an idiot, full of sound and fury, signifying nothing.
Though – I hasten to add! – the idiot here is not any of our characters, it is the situation with its constraints. Such, of course, is the best dramatic tragedy: marvellous people, ineffective, caught in a naturally-misconceived situation. (However, evolution's victims do tend to leave instructive historical debris ... if I may give another unfortunate Job's answer to that form of The Problem of Evil.)
"... much mouthing of the spirit but no resuscitation of the substance of OO ideals and opportunities"
The essence of OO is plain logic. Unfortunately it has been diluted and even drowned by the floods of words and forms with which the procedural programming tradition has deluged it, and no amount of artificial respiration from mantra-mumbling BOFs will revive it.
The logical substance of OO takes new and clearer form in MACK (as noted above), and is even the basis of its concrete behaviour which so often pleasantly surprises (as we shall now see).
"... no awareness of the classical object model miscarriage (of an ill-formed date-rape product of blinkered fixation on a misconception of encapsulation, with no respect of complexity, no deep mutual benefit from interconnectedness)"
In this section, probably more so than in the others, you will find it very tempting to condemn outright with natural but wrong interpretations. Try to see my sense rather than its perhaps conceivable nonsense if its terms are seen as in your familiar contexts. I am speaking from my own knowledge of how, in evaluating large proposals, and when my mind is already leaning in the direction of an alternative, I – like everyone else, I suspect – so easily try to grasp the too-glib way out, especially when the situation does not demand and enforce the discipline of the systematically-balanced assessment. Big IS/IT proposals in our complex world increasingly expose us to such temptations. The many-aspected-elephant-in-the-land-of-the-blind allegory tells the story well.
Talking of Jekyll/Hyde characters (cf. also the introduction in my Workers’ Day document), current OO has a serious problem of that nature, and its misconception of encapsulation is at the root of it. There is an irresoluble tension between two aspects of such conventional encapsulation: effecting complexity-hiding within objects (the dark and dangerous loner, Mr Hyde) makes it more difficult to effect and maintain any required collaboration between objects (the sociable Dr Jekyll). MACK, on the other hand, promotes a natural and safe blackbox/whitebox interplay.
Let’s summarize the conventional position (adding parenthetic comments):
Complexity-hiding is part of the public-behaviour-rather-than-unfathomable-state predisposition of classical modular programming (in the "loose coupling" principle, of such merely pragmatic sense), classical methods (of those "island" classes – see my Booch quote below), and the classical object model (with its IDL’s so meagre support of interrelationship, bearing none of the synergy of high granularity, and its classical method basis hence often effectively obliging the target-object to break out of its own encapsulation, else indicating workarounds such as the Facade and Mediator patterns of the Design Patterns book by Gamma, Helm, Johnson, Vlissides (popularly known as the Gang Of Four, or GOF)).
It is held that behaviour and state are merely dual views of the same thing, but that state is too inessential and even variable a matter to be a suitable basis for knowing an object, and that therefore a blackbox approach should be taken, whereby our representation and hence use of objects is founded on behaviour descriptions as seen from outside (though one has to ignore how poor such descriptions are, or attempt to enrich them as indeed the JBOF CDL does, or otherwise "make a silk purse from a sow’s ear" as in my Workers’ Day document I accused the TRC Inc team of envisaging, despite their own evidently-better judgment).
That conventional position (without my parenthetic comments) seems so plausible – even without the better restatement of the case that you could doubtless make – that it appears quite vain to attempt to question it! And indeed, it is rather foolish, as in the absence of the "replacement" (in Medawar’s sense), all one can do is adduce some "facts" that will eventually be seen to be part of what should ideally (in that mythical fully-rational world we tend to believe we are in) have been able to "displace" the current view, but which cannot at this stage be expected to do more than just unsettle you a little bit. My argument will however gradually develop in a more positive direction...
As a minor comment to start with, one might ask (from a MACK-like state-based viewpoint) that if behaviour is such a sufficient basis, then why does one find popular constructs such as the GOF State and Memento patterns? The supposed state/behaviour "duality" sits very uncomfortably! (that being how the allegory also portrayed the II and UML neighbours, with their jostle-for-territory that is bound to brew unless...)
More generally, come to think of it, I might also comment that Metaset has virtually no use for any of the GOF book’s patterns, which might be a sign that there is a whole set of problems that a mainly declarative and state-based approach does not land one in. Having said that, I must also add that the GOF will feel very much at home with many of Metaset’s structures, and that the pattern concept is fully subsumed – mutatis mutandis – in MACK’s typology-based approach to analysis, design and implementation (See also my faq q 7 reply).
As far as the Inheritance aspect of II itself is concerned, just two quick comments for now: (1) its use tends to lose all sight of true substitutability as distinct from the OMA’s so-called "interface substitutability", which is something quite different; and (2) even Microsoft insists that their interface-based approach with DCOM/ActiveX is "object-based" rather than OO.
The background to my contesting of conventional encapsulation may also be seen in this quote, from my faq q 7 reply, of a quote from Booch:
Meanwhile, here is Grady Booch on the OMG’s own "World Guide to Object Technology" CD:
"Perhaps the most strikingly consistent feature I find among the successful projects I encounter – and noticeably absent from those projects which seem to fail – is the presence of a strong architectural vision. Architecture is, unfortunately, a much abused word, but in my experience a well-engineered object-oriented architecture has at least two important dimensions to it: a sea of classes that capture the vocabulary of the problem space and a set of mechanisms that specify how certain classes collaborate.
"The first dimension is fairly obvious. However, there are two important lessons that many beginning projects miss. First, for systems beyond a certain complexity, the class is a necessary but insufficient vehicle for decomposition. Second, no class is an island.
"Indeed, this is the point of the second dimension: classes collaborate, and the common way those collaborations play out is fundamental to the conceptual integrity of an architecture. Furthermore, actively applying a small set of common mechanisms helps to bring simplicity to an architecture for even the most complex domains."
(where the italics were mine.) What a fine statement of what should be such an obvious architectural criterion! But how the OMG itself seems to have overlooked it in their premature adoption of the OMA as it is at present (or "miscarriage of an OMA", with its historical context adequately portrayed in the little fable).
It is such collaboration that is hidden or obfuscated by the very basis of the classical object model. As a result every procedural-language programmer has to work around the problem, exposing it to yet further possible complication (i.e. further to my parenthetic comments above) in the form of the realtime / algorithm-time confusion (See my paper and faq q 1 reply).
One can apply workarounds such as the already-mentioned GOF Facade and Mediator pattern implementations (whose very names, incidentally, betray some basic problem). But no amount of fixes can make order from the classical object model chaos. An analogous foul-up led to Macbeth’s despair already quoted, and also to Lady Macbeth’s own lament: "All the perfumes of Arabia will not sweeten this little hand!"
Ok, but then how does MACK handle the situation? (With what shall the poor theory be replaced?) Rather than further belabour my negative criticisms, let’s be more positive (even though still skating around the black hole I am keeping in the middle of my published MACK descriptions).
Some more MACK at last!
Instead of trying to ride the collaboration horse with one single-target classical rein, let’s rather ride it confidently with a complete saddle and harness. Let’s channel that sea of classes into The Mainstream that history has shown us how to ride.
MACK pushes out the frontier between the "application operating system" and the procedural program unit, leaving ever less of the latter. The latter (see my paper and faq q 5 reply) is always in the form of the RE-method, and is encapsulated even more strictly than the conventional method is. The former has quite deep roots: that was how I described my 1976 "IDIOM" (in my faq q 3 reply). It is effectively another "Mainstream" notion anyway. Maybe quot;operating system as workflow manager" is a better description nowadays for such a concept. So let’s just call it the o/s.
Thus the o/s (i.e. Metaset or any other MACK realization, possibly on top of a conventional o/s) is one vast big Mediator in the GOF sense (though its form is more recognizable in the TRC Inc "generic agent" I quoted in my previous document, with its event-driven nature that is another good "Mainstream" start for a move away from the algorithmic-procedure-based way of thinking).
But the GOF comments (ibid, p 277):
It centralizes control. The Mediator pattern trades complexity of interaction for complexity in the mediator. Because a mediator encapsulates protocols, it can become more complex than any individual colleague. This can make the mediator itself a monolith that’s hard to maintain.
So I immediately point out that Metaset is not unduly complex or hard to maintain, nor does it encapsulate application protocols. Its own requirements have themselves been decomposed and its structure already built largely in a MACK-canonical way. (What a fine "proof of the pudding" that is too!)
Yes, MACK – like conventional OO – has types, relatively simple in themselves, so how does MACK manage the real inter-type complexity, which is where the problem is thus shifted? The secret (and still largely secret secret...) is of course MACK’s "typology" (as in "the typology of a problem-area", characterizing its "semantic topology" or relativistic semantic net locality).
The typology has enough semantics – and really quite plain semantics after all – to fully define virtually all required inter-type relationships. It is strictly defined so that it is a handily and automatically manipulable basis for building big integrated meanings easily and reliably. In that respect it is much more than conventional notions such as "framework", "object schema", or the JBOF "GenericModel", and was foreseen as the delight for the "sages" of the play, especially the mathematically-inclined ones.
Thus – thanks also to the upcoming MACK-compliant market infrastructure – simple plug-and-play will often build 100% of normal client/server applications (where I express the process in reasonably familiar conventional terms). Establishing new typologies might require new RE-methods, but to an ever-reducing degree. (Here I am merely slightly rephrasing and expanding part of the material under the "Synthesis..." heading in my paper and in my faq q 5 reply.)
An atomic RE or "Realworld Equivalent" is typically an attribute-value (a measurement, or "sense-datum" analogue). Though itself abstract, in the sense of being part of the abstract model of the application domain, it is the closest apparent equivalent to a realworld entity. An atomic-RE type is called an "RE-domain". RE-domains thus form the more obvious "exterior surface" of the abstract model. RE-domains always have RE-methods encapsulating such crucial meaning. Other types may however also have associated RE-methods, though that will occur relatively rarely.
One consequence is that every type (and hence every instance) is defined ever less in terms of meaning questionably encapsulated in code form (greatly purified though that is), and far more completely by its objectively-visible relationships with others, that is, its interconnectedness.
(As you might deduce from some words in the above four paragraphs, some fine-tuning of MACK is still taking place: it is presently my target, on which I am still working, to have no RE-methods for non-RE-domains. However, in practice as well as in any questionable theory, the target will probably only be approached asymptotically. That might seem frustrating, but in fact there is a rather fundamental beauty as well as reassurance in that falling-short. Complexity is infinite, and our products, our concepts, will never fully capture it. However, the promise of more reality for the discovering – or conceptualizing or inventing – will always be there.)
A further effect of this shifting of the procedural language boundary (other than a dramatic reduction in conventional coding) is to expose far more application semantics than is conventionally the case. That ties in closely with the whole "Common Knowledge" aspect.
The residual essence of "complexity-hiding" is met by the "relativity" concept: every user has his or her own limited and hence simplified view of the realworld model, and is exposed only to what is required by the situation or – at most – what may be relevant to it ("View" here is being used both in the more generalized "frame of reference" sense (somewhat like the old DBMS subschema), as well as in the more specific displayed window or printed page "presentation view" or "projection" sense).
There are transformations (effected by an "MVC-like" agent, this being The Mainstream) between model and view, and hence between views. Such transformations (very analogous to those in physical relativity) are expressed in a combination of pure logic and certain extremely generally-applicable RE-methods, many having a strong time orientation. Transformations – like everything else in MACK – are cast in the typology form.
One respondent found on Jeff’s website what he called my "thought-provoking papers" (Thank you Chris Olds of Wall Data!) and was also both persevering and smart enough to single out one of the last sentences in my very long faq and want to know more:
Our model [in Wall Data's SALSA® product] is an outgrowth of semantic data modeling techniques (called Semantic Object Modeling - an overused name, as it turns out), and so your statement that "A semantic net is a static picture. [...] it is the dynamic aspects that are crucial" was a familiar (and resonant) one.
What more are you willing/able to share about your work? I am (obviously) most interested in "MACK operations", but would be interested in any thoughts you have on how to add operations to semantic networks.
I regretfully had to decline as "unwilling", but the declarativeness of the typology does invite that question, and the answer is as crucial as I had then said it was. In my previous document I intimated that the semantic dynamics were driven by one of the TRC Inc’s "generic agents". But what is so driven is still not obvious. I will however permit myself at this stage to add that the operation is defined in the simple axiomatic terms of the basic model, and that precision in relevance of semantics, giving precision in its application, enables computational efficiency.
Some benefits of interconnectedness
Thus MACK embodies an epistemological correction of the notion of encapsulation, based on a more appropriate conception of real-life structuring and application of abstract models.
More knowledge is cast in and is available to the system, where the interconnections are, while less is in the would-be and even pretentious "intrinsic" reality of methods, with their so-called complexity that needs to be hidden. We should never regard the programs we write as worthy of being regarded as blackboxes. And as for the programs that others write, we regard them as blackboxes only at our own peril. Let us respect the right complexity! The real complexity that we must grapple with is in the real realworld, and not in our own artifices or representations of it.
Thus, for example, the old "complexity-hiding" process easily oversimplifies in its ignoring of the relationships between the "internals" of methods and other objects that are not relevant to the prime purposes of the method but often, sooner or later, become otherwise important. Exception-handling provides some obvious examples. Workflow and transaction management provides many more.
There are many further examples, especially during the analysis phase of any activity, where we must preserve the substance of complexity-respecting doubt and constantly address that ever-present danger of "cognitive dissonance" (much emphasized in the TRC Inc BOF submission, and most appropriately so).
In that statement I am not confusing analysis and operation, but reintegrating them! Market processes – the occurrence of problems followed by their attempted resolution – may manifest themselves throughout anything we do and anything that happens. Even that thin coupling in the form of a conventional parameter-list or method-signature often allows a wide semantic shift to stream in and create cacophonous cognitive dissonance.
People who appreciate the idea (if not necessarily the reality) of pretended "holisms" will also appreciate the MACK "indefinitely interconnectable system" perspective.
Such apparent (and partly real) complexification might seem to be unnecessary, even inviting undue mystification as well as nitty-gritty practical problems. But it has the benefit that it enables a realization such as Metaset to assist far more than the application designer is accustomed to. It "knows" when it might perform more services without necessarily being asked to do so.
One major kind of service is to assist the user in being more reflective. Such behaviour – viewed from the perspective of the past – might be called a reintegration into the live application of its help subsystem. (We need a less questionable word than "self-consciousness" and a kinder word than "auto-explicativity" to describe that desirable quality for an application!) Applications need something far more systematic, integrated and intelligent than the so-called "intellisense" of Microsoft’s Office and its emerging Office Assistant. Such functionality must further be so pervasive and its mechanisms so simple that the user can effectively teach the PDA not to be a pain-in-the-neck. That must be doable in an easy and natural way. Plug-and-play typologies to support the process will be one of the fun growth directions for MACK developers.
At the application level, Metaset's built-in typologies will enable much of an application to be evolved automatically, under the coordinated control of its various users and stakeholders, as a function of the various expressed needs, the existing structure and other real constraints of the application, and the growing next-generation catalog of available products. Other typologies will then elicit, support and manage any further enterprise-specific refinement, whether external or internal market related.
Practically-deeper benefits of MACK’s far greater semantic self-exposure are found very close to a realization’s kernel functionality, from view-construction through consistency-maintenance to transaction management and recoverability of all kinds.
Thus the shift of application semantics from procedural program into o/s-driven system is not a mere epistemological nicety or philosophical dream. It is an absolute essential to managing IS system complexity. Hence it is a key enabler of the crazy project that Metaset/MACK must seem to the jaded observer, worn out as one so easily is by the many serious difficulties of to-day’s paradigm-disadvantaged scene.
"... no reform of the OMG services and facilities architecture (see below)"
The "(see below)" referred to the TRC Inc’s BOF submission, which calls for such reform, as illustrated by some of my Worker’s Day document’s yellow-highlighted quotations from it.
MACK’s own reform of the whole scene is mediated by its RE-method concept. RE-methods still effectively encapsulate external services, e.g. real time, i/o-related, or mathematical algorithm-based, which are regarded as being outside the scope of the abstract model as expressed in terms of MACK’s typology-specified interrelationships.
However, "reform" is maybe not the word to describe what MACK will do to the Common Object Services as they are now! Most of the functionality of services such as Event, Life Cycle, Transaction, Concurrency Control, Relationship, Query, Licencing, Property and Security is taken over quite simply by more integrated and further extensible abstract MACK typologies. So is a large part of Naming, Persistent Object, Externalization and Time, though more RE-methods are found here. All 12 of the "Future Object Services"of the COS Specification may be grouped in the first category (Interestingly, such services have probably been left till last precisely because of their high internal service interconnectedness. The latter, of course, is MACK’s delight).
As an example, take the Security Service. Security architecture has evolved from its null "security through obscurity" state, to encryption, digital signatures, and constructs such as complicated access-control-list facilities and artificial devices such as impersonation that are difficult to design, control and maintain. What could MACK possibly contribute? I will just comment at this stage that MACK allows the full integration of security into an inherent quality of an application and its infrastructure, the concept of "relativity" again being central. The very complexity of IS and its managed reality is even turned into a strength, as can be sensed by considering that there is a natural partnership between the complexity of any single user and the complexity of a total application as individually shaped and used by that user.
On the facilities side even less of conventional practices will remain, as all are typically integrated into a MACK realization, again implemented in terms of appropriate typologies.
Thus in MACK’s reform of the whole services and facilities area we see another benefit of appropriate semantic interconnectedness: an implicit rationalization of functionality, and the elimination of overlaps, incoherences and conflicts.
In short, complexity is properly respected, overcomplication is cut away, and – surprise! – humanly-workable simplicity is the welcoming result.
"... nothing more friendly than harsh ACID for workflow"
It is Workflow that does it. Workflow, extending over such long periods, having such manifold realworld dependencies on top of the DBMS complications we already know so well, and being so visible and important to the players in the game, clashes with all four of the ACID criteria.
It is easy to point out potential oversimplifications in each one from the user’s point of view:
Durability or persistence only has meaning if it has an opposite, namely volatility. In a workflow environment it is desirable that no work by the user should be volatile. Failures and user-interruptions at any point and for any length of time should be catered for systematically. Fully flexible work-activity suspension with smooth resumption should be mandatory. But conventional process- or thread-switching cannot do it alone without imposing volatility. So all but the most trivial of the usual procedure-stack volatility should rather become durability anyway. Metaset’s MACK-based event-, task- and memory-management architecture allows that.
Having demanded persistence, we must of course also provide maximally for comfortable reversal thereof, namely undoability, even in a cooperative environment. Not easy in a conventional transaction environment, but very doable thanks to the intrinsic reflectivity of a MACK-compliant abstract model.
Atomicity in long workflow transactions can be very bothersome to the user. Even the most elementary manual system easily supports partial commits / partial rollbacks, obviously with appropriate user-negotiation where required. But no conventional OLTP-designer likes even to consider such complication, and quite understandably so with conventional technology. Not having a function-based notion of operation, MACK easily supports such splitting of the atom.
Atomicity is further complicated by nested transactions. The conventional concept of a nested transaction (namely its eventual commit is conditional on the containing transaction’s commit) is worthless and required only in an "encapsulated"-function-based architecture. It is worthless because it either forces the outer transaction to be merely a long Pessimistic one, in the complex extreme destroying responsiveness, or its long Optimistic nature in the same extreme destroys either consistency or undoability. A nested transaction, to be of any enduser interest, must be immediately and durably committed, but must be traceable and undoable, with fully-managed user-negotiation where required, if its containing transaction is to be rolled back or otherwise undone.
Consistency is implicit in everything in MACK, but a workflow system should also support inconsistency for long periods of time and manage its resolution, whether in CASE A&D or in normal application operation. (See also my previous document, of Worker’s Day, where I mentioned MACK’s tolerance of situations where significant cognitive dissonance might exist, though of course needing to be pursued and clarified in all good time.)
Isolation in the sense of serializability is really only a means to the other three’s ends, so can be ignored if the latter are provided for. The other aspect of it, namely predictability for the user, is more interesting. Now predictability is not something that should be defined and imposed by the enforcement of Procrustean transaction design of the above harsh ACID kinds. It is something that should flow from good BPR and the good applications that should naturally result, and will result, I have everywhere asserted in some detail, in a MACK-compliant and truly demand-driven world.
"... no high-granularity BO architecture with synergy"
The typology, strict hence reliably manipulable in so many ways, adds synergy to its components. They end with more than you put into them. Their newfound greater logically-coherent form is what does it. And even that form, with its various consistency-criteria, is pursued and prompted by Metaset’s kernel-level typologies. It is robust, resilient, flexible, extensible, dynamic, just right.
Again, there is no magic. There is only good epistemology. But synergy is always surprising. It’s like when a reassembled engine kicks into life when you turn the ignition-key.
As my previous document (of Worker’s Day, building on the TRC Inc BOF submission) put it: "But will it have that special magic? Well ... the MACK typology works in a very logical, AI-like, and often surprising way, and in AI – as the Turing Test insists – it is the appearance that counts."
In line with my earlier contrasting with the JBOF submission: the lack of such synergy is the single most significant shortcoming of any OMA-compliant BOF’s BO-definition capabilities. And with the OMA’s rotten epistemological foundation there is no fix.
"... nothing substantial to chew on after all."
The situation calls for a really different, wholesome, appealing and quickly-finalizable dish.
By now it should be clear that there is at least a good chance that "Jack’s MACK recipe has enough variability and other complexity to feed and delight King Customer, his sages, his barons and all the citizens of the realm happily ever after." See also my faq q 3 reply.
My previous document had the subject-line "Inviting and preempting scatological eschatology", where the implied expletives (more plainly: "Shit! Go to hell!") were imagined directed at myself, for having set myself up so presumptuously vis-à-vis the OMG. That document’s dense paragraph that was expanded here may be recompacted into an equally dense instance of the same multi-syllabically pretentious abstraction, but in respect of the OMA: "It’s bullshit! Dump it!" (where the china-shop bull – the product-maker's confident hence iconoclastic persona of that message – is seriously provoked by the OMA-based menus that were on offer...)
But (thanks to Medawar once again) we know only too well: "Theories are not displaced by facts, they are replaced by better theories."
So, in that constructive vein again (and ultimately preferring the mediating Little Jack Horner persona resumed in this play), and seeing that Andrew has provided us with a historical "Coda: OMG Rationale for Choosing the Classical Object Model", Jack is looking forward to the next spectacle, hoping that he has entertained and tempted you with an "Overture to a Dramatic Replacement" in which the eventual play includes you.