Displaying Newsletter
Issue 7, 15 Dec 2001


  In This Issue:
 
Exploring the infinite domain of the ignosecond by Daniel Reinhold 
Just this morning, while trying to connect to a website in NetPositive, the netserver seems to be halting, the browser window is looking frozen, and finally, the mouse stops responding. I am totally locked up.

The screen is now a snapshot of a once working and vibrant center of activity. Before my eyes is the image of a dozen windows of different sizes splattered across the screen in unyielding stiffness. They taunt me with the memory of their former vigor. All of my work has been reduced to an inglorious bitmap that hogs the entire surface of the monitor. Of what use is the computer in this catatonic state? It has been relegated to an awkward and ungainly piece of furniture. Damn it!

Knowing the inevitable chain of events, I reach for the power button on the computer. By a curious coincidence understood only by multiple lightning strike victims, at the exact precise moment that my thumb pushes in on the power button, the computer suddenly awakens. The browser window repaints itself, bytes start downloading again, the hard disk starts buzzing away. What a recovery! A near death experience averted! Victory snatched from the jaws of defeat!

Unfortunately, I was then stuck with two possibilities: keep my thumb depressed on the power button and attempt to continue working with my one free hand, or let go and watch with disgust as my miraculously recovered system is shut off. Since the first possibility was about as realistic as my sitting down at a piano and flawlessly executing a Bach sonata, I opted for the second. Zap. Good-bye cruel system.

Yes, my friend. I was once again caught in the domain of the ignosecond.

What? You're not aware of the ignosecond? But of course you are. You are intimately acquainted. You just don't know the term. Forget the millisecond, the microsecond, the nanosecond. That's just children playing in the park. I'm talking about a unit of time so devastating, that it defies measurement. But oh, it exists alright.

The ignosecond is that instantaneous point in time at which your hands leaves the car door as it shuts while simultaneously you realize that the car keys are inside. It is the moment that you click the Send button that will transmit your email masterpiece describing in uncanny detail what an a**hole the boss is when you realize he is on the CC list. It is the moment that your lips touch the fetid white liquid just as you remember that you've forgotten to throw out that bad carton of milk.

The ignosecond is that mysterious connection between quantum physics and cognitive science. It is the interplay between cosmology and psychology. It is where photon meets neuron.

The domain of the ignosecond expands infinitely in all directions, built into the fabric of space-time itself. It's mathematical complexity is exceded only by its tenacity. It attaches itself to the tendrils of your brain more securely than peanut butter to the roof of your mouth. It's a freaky fractal of the third kind.

But enough theorizing, you protest. What can be done about it? Are there no steps that can be taken that will eliminate this dreaded metaphysical faux pas? Should we eat more fiber? Be kinder to others? Recite 'The Love Song of J. Alfred Prufrock' repeatedly in a darkened room filled with portly virgins?

Alas, I must tell you -- there's nothing that can be done to rid ourselves of this unwanted guest. You may as well request that pigs fly, that politicians tell the truth, that TV sitcoms become thought-provoking and intelligent, that porn sites are developed with tasteful and restrained html.

But you can follow some general advice that will help with so many things in so many ways:

  • Support the OpenBeOS Project
  • Drink plenty of water
  • Don't believe everything you read
 
Is there any hope for Clippy? by Michael Phipps 
Pesky paperclips pretend palhood, purloin professionalism, perjure participants; prescription purported! News at 11.

The dreaded paperclip. Undoubtedly the second fastest "feature" that people disable, right after HyperActiveDesktop (too much Java, maybe). Touted as a friendly face for MS Office, it is instead a cloying disaster.

Why is the paperclip such a weak feature and what (if any) hope is there for Clippy?

The "Office Assistant" (OA) is the new faceplate bolted onto help for Office. You can click on OA and type quasi-natural language questions. A heuristic is used to mangle your clear question into one you didn't ask and you are rewarded 10 choices of help topics completely unrelated to the topic that you asked about. In the mean time, the OA plays all sorts of cute animations to distract/amuse you while you work. OA can also "notice" what you are doing and offer inane suggestions about things that are completely unrelated.

Oh? That wasn't the intention? That is certainly my perception of it. I think that many others find it to be the case, as well.

Is an OA a bad idea? Ever since "Bob", an attempt at Windows with a personality (as compared to an attitude), Microsoft has continued to attempt to anthromorphize its products. This runs the gamut from "Windows is trying to locate your hard disk. Would you like me to continue?" to OA.

Microsoft doesn't generally waste piles of money for no good reason. It seems that their marketing department believes that this helps to sell to, or make complacent, some group of users. There is a group of users out there that believes that computers need a personality. A face to "talk" to, instead of a tool. Interestingly enough, all of the people who I have ever spoken to who like "Clippy" are female. I wonder if women's stronger social skills make them more comfortable with dealing with a computer as an equal instead of a tool.

What redeeming features, then, are there in an OA? Cute. Don't laugh - cute sells a lot. And it is endearing to some people. There is also a glimmer of usefulness, too. It ... feels helpful... to ask for help instead of going and looking for help. Not to go off on a psychological rant, but when you are frustrated, it FEELS better to "ask" for help after you have attempted all of the things that you know how to do.

Ask for help... reminds me of another fad - agents. Agents were promised to be a panacea for many things. They would scour the web for you. They would do distributed searches. They would watch the newscasts for interesting articles, then deliver them to you.

What about a combination of the two? A user interface (let's say something like Clippy, for the moment) that appears on command (a particular keypress, mouse event, whatever). You can ask it to do anything that it "knows" how to do.

For example, search the web. It would know to go to (say) Yahoo, Google, etc, trying the keywords you specify. It would combine and coalesce the results, fetch the pages and store them locally on your machine.

It could watch cspan.com for keywords of interest to you. It could interface with your PIM to give you reminders and you could tell it to add reminders. It could run jobs, a humane cron.

When the system is not busy, it could perform cleanup (write out dirty cache, for example). Maybe when the network is not busy (or in the middle of the night, for the modem dwellers among us), it could search for updates to web sites and cache them locally.

You could say "Music". It would choose an mp3 playlist and start playing. You tell it "Jazz" and you start to jam to the tunes of Miles Davis, thanks to BeFS attributes. It might me your preferred app launcher.

Maybe it could tell jokes. Maybe alert you of emergency storm weather. The possibilities seem endless. And not quite so annoying as Clippy, the help engine that couldn't.

 
Ufetch: a basic web client by Daniel Reinhold 
The web has certainly become one of the most ubiquitous and important publishing spaces around. What defines the web, technically, is html and http. Html is the publishing format and http is the transport protocol.

Let's consider http. This protocol is so simple. Some would say simplistic -- it certainly has it share of detractors. While it may not be the most sophisticated protocol around, it gets the job done.

It's a client/server protocol and, like many internet standards, is text-based. The client sends an http request and the server responds with a reply. There are only a few commands in the repertory: 'GET', 'PUT', 'POST', and a few others. See RFC2616 for all the details.

The bytes sent out across the network are composed of a header and the data (content). The header is nothing more than a few lines of simple text. The first line contains the command, the remaining lines contain 'key: value' pairs. If either server or client don't understand a particular key, it's ignored -- this leaves quite a bit of leeway for fun.

Normally, you never see these headers. As a counter-example, consider email. Most email clients allow you to see the message headers if you want. This makes it more accessible to understand the mail protocol. But most web clients, such as web browsers, never let you see this stuff. Too bad -- it can be interesting and informative.

I wrote a simple web client for just that purpose. Called ufetch, it's a command line utility that fetches data from web servers. For example, in a Terminal window type:

ufetch www.bebits.com
This will download the home page of BeBits and put it in a file called f.data. As it runs, it spits out various status info to the screen.

You may be familiar with an similar Unix utility called wget. wget is actually more powerful, as it will download from ftp servers as well. But ufetch is simpler, both in its design and its source code. I think this makes it a spiffy tool for learning about various details of the http protocol and web client/server communication.

ufetch was inspired by an old BeOS Newsletter article by Benoit Schillings called Mining the Net. Benoit created a sample C++ program called site_getter for fetching URL resources. I took the code, converted it to C, removed stuff I didn't need, added other stuff, tweaked, coddled, and massaged the code to my heart's content. It is so completely modified that I don't think there's one line of code remaining in ufetch from Benoit's original code. But it certainly was inspired by his work and his comments.

It's really not very hard to implement a web client. The simple text format of the headers makes them trivial to deal with. Most of the work in ufetch involves establishing connections to the web servers and sending/receiving data. Even this, however, is pretty simple because the sockets interface handles all the low-level grunge. If you are a member of the OpenBeOS networking team, then you have the task of implementing the sockets interface. But as a network programmer, you needn't be concerned with the details and only need to know how to use the sockets themselves.

The sockets interface was originally designed by Unix programmers at Berkeley. Which is why they are often referred to as "berkeley sockets". This interface has been ported to other platforms such as Windows, often with many changes and alterations. The BeOS sockets interface is very close to the BSD module, but varies slightly (most notably in that sockets are not true file descriptors).

The semantics of socket operations is similar to file operations. You create a socket and then bind or connect it to a network address (similar to 'open' for a file). While connected, you cand send and receive data (like the 'read' and 'write' for files). When finished, you close the socket. You are required to know the IP address of a remote socket in order to connect, but there are database functions for determining the IP address when given a URL.


Walking thru an example

Ok, let's see how this works in practice. Consider the sample command line:
ufetch www.bebits.com

First, the URL is split into (protocol, host, port, resource). There is no "http://" in the URL, so 'http' will be assumed for the protocol. The host is 'www.bebits.com'. No port is specified, so it defaults to the standard web port 80. No resource is specified either, so it defaults to '/'.

Next, the IP address of host 'www.bebits.com' is looked up using the standard network function gethostbyname(). In this case, it returns 28.245.212.78 (in dotted format).

A socket is created using the socket() function. Then the socket is connected to the web server with the connect() function using port=80 and IP=28.245.212.78. If it's unable to connect, an error message is sent to the screen and the program exits.

The following request header is generated:

GET / HTTP/1.1
Host: www.bebits.com
User-Agent: ufetch
Accept: */*
Connection: close
The first line is the status line. 'GET' is the command and '/' is the requested resource. This is followed by the http version. The remaining lines are standard header tags. The 'Accept: */*' line says to accept anything (ufetch is not picky). The 'Connection: close' was added after some real world testing: HTTP 1.1 supports persistent connections (unlike 1.0), so you need this tag to avoid a delay in terminating the connection.

This request header is sent using the send() function. Then recv() is called to receive the reply. A block of memory is allocated to hold the incoming data. The reply header will be the first part of this data, followed by the data bytes for the resource. It's difficult (impossible?) to know exactly how big the header will be, but the end of header is always easy to find -- the first blank line in the incoming stream marks the end.

Here's the header received from the BeBits server for this example:

HTTP/1.1 200 OK
Date: Fri, 14 Dec 2001 19:29:07 GMT
Server: Apache/1.3.9 (Unix) TARP/0.42-alpha PHP/4.0.4pl1 secured_by_Raven/1.4.2
X-Powered-By: PHP/4.0.4pl1
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html 
A web client might want to parse this header into all the different tags and make use of the info. For the most part, ufetch doesn't bother. The one exception is for redirects. Often a particular URL simply redirects to another URL. In this case, the response code is 301 or 302 and there's a tag called 'Location:' that identifies the URL to redirect to. This is the only header tag that ufetch cares about.

The header is sent to the screen so that it can be viewed along with the status info. The data bytes, however, are written out to a file called f.data. This makes it easy to find, but it also means that each run of the program will clobber the output of the previous run.

ufetch is certainly limited in what in can do. But it's loads of fun to use. You can see all kinds of interesting info being returned by web servers. Try it and see how many web sites are sending cookies you didn't know about. Or snoop on just what software the server is running. It might even be useful as a way to debug connections to certain troublesome servers. Expanding ufetch in any number of ways would not take too much effort. Have fun.


Source Code:
ufetch.zip