Be Newsletters - Volume 2: 1997

Issue 2-25, June 25, 1997

Be Engineering Insights: What's the Fragile Base Class (FBC) Problem?

By Peter Potrebic

Try as we might to separate and hide our implementation of the BeOS, certain dependencies are created the moment a developer compiles and links against our dynamically loaded libraries. These dependencies include:

The size of objects (i.e. structs or classes)
The offsets to "visible" (public or protected) data
The existence and size of the vtable
The offsets to the virtual functions in the vtable

When your app is compiled and linked, it records all these statistics. If any of these things changes in the library, the compiled app will no longer run. This is the "Fragile Base Class" problem.

With the almost-in-your-hands Preview Release, we're putting a stake in the ground: The Preview Release will be forward- compatible with subsequent releases of the BeOS. How far into the future will this compatibility last? Frankly, we don't know—we'll run it out as far as we can, but if we hit a brick wall we'll reassess our position.

To archive the goal of forward-compatibility, we had to take certain steps. If you're designing your own library, and if you want to be able re-release your library without breaking your client's code, you may want to follow similar steps.

The Bright Side

Before we look at our FBC solution, let's look at what CAN change without breaking compatibility:

Non-virtual functions. A class can adopt as many new non-virtuals as it wants. Old code won't be able to take advantage of the new functions, of course, but you won't break anything.
New classes. New classes are permitted as long as they don't change the inheritance hierarchy of the existing classes.
Implementation. The way that existing functions are implemented is allowed to change. Re-implementing old functions is obviously not something to be done lightly, but it's not going to tickle the FBC problem.

The Dark Side

Here are the matters that affect the FBC problem, and our solution for each:

The Size of Objects Cannot Change
The "size of an object" means the cumulative size of its data members. If more data members are added, the class will break compatibility. In the Preview Release, we've reserved an "appropriate" amount of data in each class:
```
uint32 _reserved[X];
```
Where "X" is determined on a class-by-class basis. If we anticipate that the class won't ever change (BRect, for example), then we didn't add any padding. If the object is small but it might grow, then we added a little —maybe 25-50% of the current size. For example, the BMessenger object has 13 bytes of real data; we've padded it with 7 extra bytes. Large classes get even more.
So what happens three months from now when the "right amount" ends up being too little? The final "_reserved" value can be used to point to another structure that accommodates the new data (taking into account that a pointer and a int32 might not be the same size).
Offsets of Publicly Visible Data Members Cannot Change
When thinking about the FBC problem realize that the "protected" C++ keyword really means "public." Anything that is "protected" is publicly visible. Any "public" or "protected" data members are fixed in stone: Their offsets and sizes can never change.
There isn't a "solution" to this because it really isn't a problem; it's just something you must be aware of if you're making your own library.
Be Wary of inline Functions
If an inline function exposes the size/offset of a private data member then that private member is actually public: Its size and offset can never change. We've removed all such inlines from the kits. Unless there's some overriding performance issue I'd recommend you do the same in your libraries. Remember: The only safe inlines are those that only reference public members (data or functions) or non-virtual private member functions.
VTable Now for the Future
If a class/struct is *ever* going to have a vtable it better have one now. Adding that first virtual function changes the size of the class, and possibly the offsets of every single data member.
Add a dummy virtual (or a few) now or forever hold your peace. If a class doesn't need virtual functions, then you don't need to do anything.
Even if a class already has virtual functions, you may want to add more —do it now or never. In the Be kits, most classes have additional reserved virtual functions; look at the beginning of any private section in a Be header file and you'll see them:
```
class BWhatAPain {
  public:
      ...
  private:
      virtual void  _ReservedWhatAPain1();
      virtual void  _ReservedWhatAPain2();
      virtual void  _ReservedWhatAPain3();
      ...
};
```
The Ugly Safety Net
For some classes, it's difficult to estimate the correct number of extra virtuals. Too many is okay, but too few can be bad.
To solve this problem, an additional "ioctl"-like virtual function can be added to the class hierarchy for unlimited (but ugly) extensibility. "Perform" is the name of choice for this type of function. As an example, look in the BArchivable class:
```
class BArchivable {
  public:
      ...
      virtual status_t    Perform(uint32 d, void *arg);
};
```
If the function is needed, we can define "selector codes" and use the Perform function like so:
```
ptr->Perform(B_SOME_ACTION, data);
```
It's not pretty, but it gives us room if a class runs out of dummy virtuals.
Public Virtual Function Order Cannot Change
The order that public virtual functions appear in the header file is set in concrete. The Metrowerks compiler orders vtable entries based on the order in which they appear. Virtual function order can not be shuffled later on. (We're lucky that entries aren't alphabetized! Think about it.)
private virtuals, on the other hand, *can* be reordered. But that's only because in the kits we don't define any private virtuals that can be (or should be) overridden.

A Dilemma

Looking at the last two items leads to an unfortunate problem. Let's say that in a subsequent BeOS release, we want to use one of the dummy virtuals. We can't simply move it to another part of the header file -- vtable order is set in stone. But a function CAN be moved from private to public. As we (at Be) need a dummy virtual, we'll simply "peel" the topmost private function up into the public section.

For example:

class BWhatAPain {
  public:
      ...
      virtual int32  NewDR10Function(... arglist ...);
  private:
      virtual void  _ReservedWhatAPain2();
      virtual void  _ReservedWhatAPain3();
      ...
};

This is why we chose to stick the dummy virtuals at the top of the private section.

But what if the class has a protected section that contains virtual functions? Remember, you can't reorder your virtuals, but you can "interleave" sections. It's not pretty...

class BAreWeHavingFunYet {
  public:
      ...
  protected:
      ...
      virtual int32  SomeOldProtectedVirtual();

  public:
      virtual int32  NewDR10Function(... arglist ...);

  private:
      virtual void  _ReservedAreWeHavingFunYet2();
      virtual void  _ReservedAreWeHavingFunYet3();
      ...
};

...but it works.

Another Dilemma

There's another subtle issue dealing with overriding a virtual function. I'll explain the problem in a moment, but first the solution: If a class might ever need to override an inherited virtual function it's much better and simpler to override that function *now*.

Here's the problem. Let's say a kit declares a couple of classes thus:

class A {
  public:
      virtual X();
};

class B : public A
  { ... };  // i.e. B *doesn't* override A::X()

Now a developer creates their own class (C) that inherits from B, and overrides the X() function as follows:

C::X() {
  ...
     inherited::X(); // OUCH! statically resolved
                        call to A::X()
  ...
}

The call to inherited isn't virtual. It's a statically resolved call to the "closest" override; in this case, it resolves to A::X(). That's okay as far as it goes—but what if, in a subsequent release, class B *does* override X()? The developer's code will *still* resolve inherited::X() as A::X()—in other words, the developer will skip right over B::X().

The solution that covers all cases is to fully override all inherited virtuals (where the implementation simply calls inherited). But that's overkill; it could impact performance and would complicate our API. So we applied some discretion in the kits; some classes override some functions, others don't.

But let's say it's getting close to DR10, and we now realize that a couple of our do-we-need-to-override guesses were wrong. There's a solution, but it's complex, *very* ugly, and too much trouble to explain here.

Other Miscellaneous Items

Never put an object that contains a vtable into shared memory. The vtable contains addresses that are only valid for a particular address space.
You can't override the new/delete operators after the fact. It's now or never.

That's all there is to it!

News From The Front

By William Adams

Requiem for a BeBox

A moment of silence please for my recently departed BeBox...

I take my BeBox "portable" home almost every night because I simply can't get enough. If you are one of the programmers such as myself who has had the privilege of programming on one of these machines, you have come to appreciate a fun and enjoyable experience. Just the thought of dual processors makes you feel like you have something no one else does and are therefore on the cutting edge.

I have another BeBox at home, but my wife is now a BeBox programmer as well, so I can't simply use that machine. So I cart the one from my desk back and forth. The weight isn't too much because I know the reward on the other end will be a few more hours of programming this beautiful OS.

Last night I carted my machine home as usual. In my machine I have two hard disks, each with a couple of partitions, a FireWire card, a Hauppauge board, a Matrox Millennium board, and a Jaz drive, and the usual chunk of memory.

After Yasmin was well asleep, I thought I'd sit down for a little evening news. I hooked up the box in the office, and went to sit on the couch. A few minutes later I was thinking to myself, "my that grass fire smells strange." There was a fire in the local hills earlier in the day, so I thought nothing of it. Then a couple minutes later I thought, wait a minute, I live in Silicon Valley and that smells strangely like silicon frying!! I dashed into the room. There were no visuals of smoke, but something was definitely frying. The LEDs were pegged so I reached for the switch. Upon inspection, I found that the disk drive that I had unsecured was pressing against a part of the motherboard that it shouldn't have been touching. Result, fried disk or motherboard. I'm afraid to turn it on to check.

Your mind races in such situations. I wasn't too bent out of shape though. I have that Jaz drive for a reason, and I do use it. The only thing lost was the latest modifications to the /dev/wacom driver. I can reproduce that and be back in business. The one thing I really lament is that this was an excellent opportunity to use:

is_computer_on_fire()

and I missed it.

DARN, DARN, DARN, DARN!!!

So, it's been asked enough times, what's the difference between Datatypes and Rraster codecs. And I've answered enough times, "Use Datatypes." Rraster exists by necessity. Datatypes didn't go out with the Advanced Access release, and we wanted to have an image viewer. The codecs were written quite a few months ago. Rraster codecs only do reading, Datatypes are for reading and writing. Datatypes will also translate between any format, Rraster will only do images. Datatypes is a nice solid framework for the inclusion of data in any format including movies, sound, still images, text, and whatever else can be thought up in the future.

The Datatypes library will be included in the upcoming Preview Release. It is based on the 1.52 release from Jon Watte. It is not quite as Be-ified as it could be, that will be a future exercise, but it's there and you can create those codecs as freely as you like. This library has found great usefulness in many apps to date. We are not quite giving it our full support, but merely including it as a matter of convenience. You should expect that in the future it will move more into the Be fold and be a more integrated part of the system. The primary benefit for the future might be that we start to utilize it, or similar functionality within our own applications such as Rraster.

One benefit I did see about the recent histrionics on this topic is that a couple more Datatypes aware applications have been written, once again proving the worth and ease of this framework. That can't be bad.

A side discussion of this whole Datatypes thing has been this question: How do I get my application's "launch" directory. Well, the BApplication object doesn't provide a method directly, but you can do this:

status_t HomeDirectory(char *dir, constint buffLen)
{
  app_info info;
  be_app->GetAppInfo(&info);

  BEntry appentry;
  appentry.SetTo(&info.ref);

  BPath path;
  appentry.GetPath(&path);
  strncpy(dir, path.Path(),buffLen);
  dir[buffLen] = '\0';

  return B_NO_ERROR;
};

Or variations on the theme to return a BDirectory object instead. From there you can locate resources that might be located within your launch directory. Of course if you're writing a POSIX application, you'll probably just use the getcwd() function, or argv[0].

Have you ever in your career participated in the final days of a major software release, prepared for developer conferences on two continents, written interesting sample code and commentary for one of those conferences, taken care of your fire ball two year old because her nanny is on vacation, all within the space of a couple of weeks? Thank goodness it's all for the BeOS. Just one more compile ought to do it!

Persistent Ideas and Culture

By Jean-Louis Gassée

CompuServe occupies a special place in my affections.

Once upon a time they were the model on-line service, reliable, available everywhere, full of "good stuff." Sixteen years ago, when we started Apple France, we suffered from CompuServe envy. It hadn't absorbed The Source yet, a wonderful name, and wasn't easily accessible overseas. To make the Apple II more attractive, we decided it needed its own, local, on-line service. So, with the help of The American College in Paris, we paid homage to CompuServe and started Calvados, a word play on apple jack. Calvados went through several transformations and is now essentially a French ISP.

The idea endured. In the early versions of our business plans, as recounted in a previous newsletter, we intended to build a BBS in order to wire together the Be community, developers, customers and ourselves. The Web mercifully freed us—and our shareholders—from having to build and debug such an infrastructure.

There are more ties to CompuServe. Before starting Be, as I was leaving Apple, I looked at several business opportunities; one of them was buying CompuServe and modernizing it. At the time, I was of the opinion there was much "unexpressed" value in CompuServe and I thought of ways to make it more visible to customers and, as a result, to shareholders. Negotiations didn't even start when we heard H&R Block, the owners, thought the business was worth about $900 Million on the basis of $125 million revenue, while NYC investment bankers felt they could justify $400 million.

Later, around 1992-1993, we tried to build a CompuServe client into the BeOS. We still wanted to wire the Be-ers together and we felt we could do a nice job of it by neatly integrating CompuServe's HMI (Host Micro Interface) protocol into our product, thus creating a better user experience.

CompuServe was reluctant to work with us; the stated reason revolved around the support headaches our HMI implementation would create. In one guise or another, the old ideas are alive and well as we debug the Preview Release version of our NetPositive browser.

I re-signed with CompuServe as I prepared for a trip to several European locations. I had canceled my subscription a while ago, preferring an Internet account with a local ISP. This time, I wanted the convenience and reliability of access to e-mail on the road without spending a fortune in international calls. CompuServe came to mind with local access in many European locations.

The sign-up was painless and I thought I had secured an easy solution to my problem. In a way, I had, but there were glitches reminding me of the differences between old style on-line services and Internet ways. First, I received several urgent messages advising me of unspecified problems with my sign-up data and requesting resubmission "using the enclosed form."

The form refused to be filled out. Fearing cut-off, I e-mailed fully restated data, including the billing address I thought was the source of the problem. When I logged in from Paris, I found my reply to customer service had been rejected because their mailbox was full. No complaints yet, I'm still connected as I write this.

The worse news is the custom browser used as the standard CompuServe 3.0 client. The mail part can't reliably open attached files and is much less flexible than either Eudora or the mail function of Microsoft's or Netscape's browsers. The better news is you can forget the client altogether, dial up CompuServe and use a "standard" browser and e-mail package.

From that perspective the site is just another site, nice but unremarkable. And, with a local call, I can connect to my Silicon Valley ISP, retrieve mail and browse my favorite Web pages—which is what I wanted most. I should be happy.

Yet, I feel a little sad as I see CompuServe struggle with its conversion to the Internet. Its contents and user interface remind me of the good old days, but no longer feel competitive. And making money as an ISP, even an international one, looks like a hard road back to financial health. Company cultures are mysterious, how they're born, how they evolve, how they can or can't be changed once they've become a liability. Something to fear, something to admire.