Be Newsletters - Volume 4: 1999
I get a lot of mail—work related, private mail, automated bug reports, correspondence with old friends, and those unsolicited advertisements that we all love so much. I separate work and private mail by using two different accounts. At work, I use BeMail and break it down with queries to break it down (usually the breakdown is between "Unread" and everything else). I confess that at home I store my mail on my headless Linux machine, filter it with procmail, and read it with mutt (via telnet from my BeOS machine, of course). Why? Because I can connect to the machine from anywhere on the planet (at least anywhere that I can use ssh to get through Brian's gestapo firewall) and read my mail remotely.
All this is a result of the way that mail is usually handled. One of the older mailbox protocols on the net is POP, or Post Office Protocol. It's simple and reliable, but it's really only designed to let you store mail on a server temporarily, until you can download and read it. There is no provision for storing messages on the server, much less sorting, categorizing, or shuffling them.
Enter IMAP—the Internet Message Access Protocol. Designed and proposed in late December of 1996, it provides an interface for remotely storing and retrieving e-mail messages. However, since it's complicated compared to POP, it doesn't seem to be widely implemented. At least not on BeOS, the only operating system that really matters in this context.
An Internet protocol is usually defined in a "Request for Comment," or RFC. This document is a proposal by someone that explains the goals of their protocol and methods for implementation. IMAP version 4 is documented in RFC 2060, available at http://sunsite.auc.dk/RFC/rfc/rfc2060.html, and finer RFC mirrors everywhere. It may help if you read along.
I started this project with the goal of creating a set of objects that I could use to represent the bits of data and processes that need to be shuffled around in order to deal with IMAP. I looked through the specification to find the best way to break down the process into units of information and the processes that can be done to them. After scanning the RFC, I came up with the following group of objects. I'll list them in simplest to most complex order, because many of them are defined in terms of each other and I wish to avoid forward declarations.
IMAP messages and mailboxes can have a set of flags associated with them. These can be used both by the user and by the system. In addition, flags can either be "permanent," in which case they'll be preserved from session to session, or "temporary," in which case they'll only be valid during the current session. Other than that, there isn't much to do with a flag object, except own it.
IMAP messages consist of an RFC-822 (you know where to look it up) or MIME message, with an associated 32-bit unique identifier. The server must be capable of providing several different representations of the message. For now, this system just deals with the raw message. Somewhere around here is a set of classes for parsing messages, but they don't belong to me. I'll pressure the author or something.
IMAP mailboxes contain lists of messages and a unique "validity ID" used
to avoid collisions. The server also provides a "Next UID" which can be
used by clients to create new messages. I simply use a BList to keep
track of the messages (or at least the Message IDs) in the mailbox. In
addition, mailboxes can be heirarchial—but I've chosen not to
implement this feature of IMAP for the time being.
This is where the magic happens. A Connection is responsible for
connecting to the server, logging in, handling the listing of mailboxes
and their messages, and eventually the downloading message bodies. It
needs to know information like the hostname, username, and password. It
also maintains lists of mailboxes, as well as an API for dealing with
them. This is the most complicated object, containing functions for
opening a network stream, composing requests, and parsing the results. It
also uses a BList of Mailbox objects to track the boxes for the current
session.
An IMAP transaction consists of a request tagged with an ID, followed by one or more reponses from the server, terminated with a final response which is tagged with the original tag from the request. The tag can be any set of alphabetic or numeric characters, and should be different for every transaction. The IMAPConnection class has a private function to generate tags based on a counter, by appending the counter's value to the word "KRIMEZ" (for historical reasons) and then incrementing the counter. A typical transaction works like this:
Client sends a request tagged with a string: KRIMEZ3 LIST "" * Server responds with an untagged response: * LIST (\NoInferiors) NIL INBOX Server ends the transaction with the tagged response KRIMEZ3 OK LIST completed
IMAP servers are allowed to respond with any set of valid responses they
wish. This means that the parser must be able to deal with just about
anything, and figure out quite a bit based on its idea of the current
state of the connection. I've chosen to have a single function,
HandleResponse(), called after every command is sent. The function reads
lines from the network stream and deals with them until it detects a
tagged response. Each line is identified and dealt with accordingly. Many
commands are simply ignored, as they require no acknowledgement and we
are not interested in them. Some commands contain information, which we
grab, parse, and store.
After that, this project is pretty much just a parser and a network wrapper. I'm running out of space, so I'll hope that the code speaks for itself. You'll find it at <ftp://ftp.be.com/pub/samples/network_kit/IMAP.zip
This is a minimal video producer node. It reminds me of Whack.
My contribution to society:
<ftp://ftp.be.com/pub/samples/media_kit/VideoProducer.zip>
The life cycle of the node, in terms of the hooks that are called, looks something like this:
VideoProducer()
NodeRegistered()
Preroll()
HandleStart()
PrepareToConnect()
Connect()
Disconnect()
HandleStop()
~VideoProducer()
You should not rely on steps 3 through 8 being called in any particular order (or at all).
The binary belongs in
/boot/home/config/add-ons/media
Writing an add-on for the Translation Kit, like writing most software, involves laying some groundwork before you get to the good stuff. For that reason, I've written a framework into which you only need drop code which is specific to the format you want to support. The rest is done for you. Pick up the archive here:
<ftp://ftp.be.com/pub/samples/translation_kit /TranslatorTemplate.zip>
If you're eager to get started, you can pretty much skip the rest of this article. Just get the code, and search for !!!, which will indicate where you need to make changes. Please note that the code as is will compile and run, but not do anything useful, since the encoding and decoding routines have been left blank (they're up to you).
For the purpose of illustration, I've picked an up and coming image format, MNG. This should not be construed as a hint that we're going to support it.
The main steps you need to take are:
Define some strings and constants, as follows:
#defineNATIVE_TRANSLATOR_ACRONYM"MNG" #defineNATIVE_TRANSLATOR_FORMAT'MNG ' #defineNATIVE_TRANSLATOR_MIME_STRING"image/mng" #defineNATIVE_TRANSLATOR_DESCRIPTION"MNG image" #definecopyright_string"© 1999 Be Incorporated"
Add controls to the TranslatorView if you have any user selectable
options.
Modify the Identify() function to recognize images in your format.
Write the encode and decode routines.
Rename the executable as appropriate (MNGTranslator, for instance).
That's all there is to it. A couple of things to note:
As a good translator, this framework provides an app and a window should the user run the add-on directly. This is not much extra work, but is useful if there are options to set.
Translating from and to the same format is done using a dumb copy between the input and output streams. This was the source of a bug in two of the Be-supplied translators, and has been fixed.
Invoking this translator with a type constant of 0 will output a
B_TRANSLATOR_BITMAP, as explained in my previous article.
Last week's column on peaceful coexistence brought more e-mail than I expected. I thought the amusing situation I reported was just that, an anecdote illustrating how well we managed to exist on the hard disks of PC clones next to the Windows partition. Your Windows system refuses to work, so boot BeOS and we might help you retrieve your data and/or edit system files. The usual caveats are hereby stipulated, as counsel recommends, and we have no plans at this time to start a Windows utilities business.
After several readers and BeOS users reported similar experiences, I went to the in-house gurus and asked why we succeeded where our worthy elders failed. What seemed initially like a happy accident now looked like a systematic feature our FAT file system extension; I wanted to know where and why our implementation was superior.
Victor Tsou, the engineer who implemented this file system extension, put it more modestly. He thought it might be imprudent to call our FAT module "better." We certainly don't have (yet) the benefit of the same large scale testing as the Windows FAT implementation does. Ours is a different implementation. We use a different approach to achieve the same goal of safely reading and writing FAT files. The tests we perform might exhibit subtle differences here and there. As a result, in a small number of cases, we might be able to read FAT files that Windows can't read. Conversely, in a similarly small number of instances, Windows might be able to read FAT files that BeOS can't read.
In other words, either system might fail to read .02 percent of FAT files, but not the same .02 percent. (I use .02 percent as a place holder; the actual number is much smaller.) Statistically, Windows is called upon to read and write FAT files much more often than BeOS. As a result, Windows' "bad" .02 percent is more frequently seen than our own "bad" .02 percent.
None of this diminishes the merit of our ability to coexist nicely with a FAT file system. It's a very pleasant feature, one that I personally enjoy greatly, but we don't need to harm a good piece of engineering by exaggerating its reach. And while we're on the subject of reach and file systems, our readers know that we support more than FAT;
See http://www.beatjapan.org/mirror/www.be.com/support/qandas/faqs/faq-0259.html for more.
We might even claim to have both a very robust 64-bit journaling native file system with BeFS and a very (the most?) extensive set of external file system plug-ins. This is consistent with a philosophy we share with many: facilitate the movement of data, avoid roach motels.
Lastly, a note I received from Russia, from Max Bazarov, a.k.a. Baza, as he calls himself:
If you like Norton Commander...I think...final release of BeFar will be interesting for you :)
http://www.beos.ru/projects/befar/
http://www.beos.ru/projects/befar/shot_0_0_6.gif
gl, Max Bazarov ( Baza )
BeNews.Ru Editor,
http://www.benews.ru/
baza@benews.ru
I used the URL and, yes indeed, a BeOS version of Norton Commander appeared on my screen. Now I'm fond of Norton Commander. Not the least reason for this affection concerns the time in 1995, when I upgraded my home PC from Windows 3.11, the one that really worked, to Windows 95. I'd heard all the stories about the end of the "Windows on top of DOS" era. With 3.11, you booted DOS or, in my case, launched Norton Commander at the end of the AUTOEXEC.BAT file. Then, you could forage around, or type WIN to launch Windows. With Windows 95, Microsoft declared this ancient history, presenting Windows 95 as a totally integrated product.
So—I upgrade my system and, after a number of restarts, it reboots for
good and.... I see the familiar Norton Commander. I type WIN and Windows
95 starts. I restart, same story. This time, I type VER, for the DOS
version, and get a Version 7.0 reply. The upgrade process preserved my
AUTOEXEC.BAT file, complete with NC as the last two (visible) characters.
This, in turn, exposed the Win 95 marketecture.
