How I Worked in the Shetland Islands without Leaving California by Dennis Allard allard@oceanpark.com Appeared in On The Internet, Vol. 1, No. 1, March/April 1995 When MES, a firm based in the Shetland Islands in Scotland, sought help over the internet to develop a distributed PC database for Sullom Voe, the largest oil port in Europe, I jumped at the opportunity. It began with a posting by MES's Mike Robertson to the comp.databases.ms-access newsgroup looking for progammers and ended with two small businesses being transformed into a transoceanic office environment. All communication, from initial contact through final delivery and invoicing was done on the net via electronic mail. I worked entirely from my home in Santa Monica, California using PCs and the simplest Internet tools. Although Mike and I developped a close working relationship during the project, we have yet to speak by telephone or hear each other's accent! Bidding, Payment and Trust Trust played a critical role in this project in everything from ensuring the smooth interplay of responsibilities to rate negotiation and payment. Initially, I was sent a 16 page partial system specification. I put in a couple of hours analysing the specification, enough to convince Mike that I knew what I was doing, and a rate negotiation followed. Although I bid at a rate somewhat below my local commute-by-automobile rate, I did so out of my desire to work on this project as well as my intrigue over working on the net. I stipulated that my work would be invoiced weekly and would continue into the following week based on mutual satisfaction. Payments were made by wire transfers between our respective banks at the prevailing exchange rate at the time of payment. A trusting relationship developed between us within two or three weeks and soon I was invoicing on a monthly basis. No formal contract was ever signed. The importance of our initial e-mail parlaying can not be understated as it replaced the normal in-person interview process that typically precedes a client-vendor relationship. In retrospect, our avoidance of telephone communication seems odd, especially during the early phases of the project. What makes it less surprising was my client's busy schedule of e-mail contacts with other prospective participants. A few initial messages established our interest in working together and before we knew it, the project was underway. Getting Down to Work Initially, MES recruited four prospects from a pool of about 20 replies to the USENET posting. Two of the original participants left the project, apparently for lack of time and deadline constraints (which, as deadlines often go, went by the wayside as the project matured). Hence, this became a two-person effort. Had more than the two of us been involved, it would have made sense to devise an automated distribution list for communication. The purpose of the project was to implement a distributed PC database using Microsoft Access and Visual Basic in Windows. The database is a small one in computer terms, containing records for approximately 1,200 vessels and a history of vessel visits to the port containing more than 7,000 records dating back to 1983. Information about current traffic and pollution incidents is maintained. The entire database contains 6 Megabytes of data. What made this project unusual for a progammer was that the database is replicated on several PCs throughout the port and has no central database server. My initial assignment was to help specify and implement a poor-man's data replication method for automatically updating PC databases via a Microsoft Mail connection over local telephone lines. The choice of a proprietary mail transport was a preordained requirement and would not have been my preferred choice; I being a TCP/IP kind of guy. I had access to neither Microsoft Mail nor a local area network (my LAN is the Internet), which meant that I had no way of directly simulating the transport layer. However, my code was to interface to the transport process via shared files, which made it possible for me to implement and test our replication method by simulating communication between two Access databases running on a single machine. My other assignment was to develop a standalone reports facility. The relative independence of my tasks with respect to the rest of the system as well as our careful definition of intermodule interfaces, enabled me to work independent of whatever Mike was doing. I was able to concentrate on my tasks and benefit from his frequent feedback. Time Zones Nearly 250 e-mail messages were exchanged during the course of this project. Our shared document base consisted of two evolving specification documents (each with embedded pictures), the relational database (represented by two or three Access .MDB files), and a sequence of evolving partial databases and programs. The e-mail stream itself provided an important record of decisions and notes. My involvement comprised 170 hours of labour over the course of four months, which amounted to roughly 15 percent of the total project effort. Often I would spend a day or an evening on the MES project and then use the time-zone difference to my advantage. Scotland is eight or nine hours ahead of California. E-mail messages I sent in the evening (California time) would be waiting for Mike the next morning (Scotland time). His messages, sent at the end of his working day, would arrive on my system at what was for me the following morning. In a sense, we kept the work going 24-hours-per-day during stages of intense work. The Internet Tools that Made it Happen Both MES and my consulting business, Ocean Park Software, operate using PCs. MES has an e-mail link provided by win-uk.net. This is part of the PC User Group (PCUG) running WinNET software. It is basically an e-mail-only system with Usenet support, although the company plans to include online services for WWW, FTP, etc. in 1995. Oceanpark.com is a 486DX-33 PC host linked to the net at 14.4 kbps via a dedicated SLIP connection provided by Netcom, operating under Windows running a complement of TCP/IP tools provided by NetManage's Chameleon NFS/X and various shareware and beta products. [*** 'a full Internet connection' --> 'a full Internet connection at that time'] No use was made of direct TCP/IP connections between our two sites, such as ftp or UNIX talk, since the Scotland end of the project did not have a full Internet connection at that time. Nevertheless, communication flowed naturally and efficiently using e-mail alone. Although most of our communication via e-mail was straight ASCII text, there were times when we needed to exchange large binary files, including Word for Windows documents and Microsoft Access databases. In order to send and receive binary files via e-mail, we used uuencode and uudecode. Uuencode stands for UNIX to UNIX encoding, named after the UNIX system on which it was originally developed. It works by converting any binary file into a form consisting of printable characters suitable for insertion into an e-mail message. You may have seen such messages on the net. They are the ones that contain hundreds of lines looking like M+RU. G(J4&S_M^..CX^/LZB+P/B\_%_)9R!?M&$.3D"Y@;5X@XES0;E (BD3<. Uudecode converts such information back to its binary form. Use of uuencoding to send binary via e-mail is necessary for reasons that seem to me obscure, beyond the scope of this article, and beyond the scope of common sense. We also made use of the venerable pkzip, which compresses data in order to reduce the amount of time, disk space, and network bandwidth involved in sending, receiving, and archiving large files. The largest binary file we sent was the main Access database. It was 6 megabytes in size but pkzipped down to 2 meg. Pkzip and pkunzip actually work on entire directories of files. Hence, our primary communication path for transferring large amounts of data was: < file or files here> --> pkzip --> uuencode --> email --> uudecode --> pkunzip -->. This worked quite well. And Yes, There Were Technical Problems It should not be assumed that Internetting with PCs is a cake walk yet. I have invested countless hours bringing up, experimenting with, and settling on TCP/IP tools and methods in order to make my PC behave like an Internet host, demonstrating an acceptable degree of reliability and performance. However, the situation continues to evolve and improve. Reliable Internet connectivity is just beginning to become a friendly environment for those who are not technically inclined. Early in the project, we discovered that the e-mail-based ftp client used by MES could not successfully talk to the ftp server at oceanpark.com. It was for this reason that we ultimately resorted to uuencoding. If not for that, it would have been possible for us to maintain a document database at oceanpark.com, queried and updated on a regular basis by MES via ftp. Due to a file-size limitations in Chameleon's e-mail uuencoding tool, I was forced to bring UNIX hosts into play via ftp and telnet. In order to move megabyte-size files to MES, I would telnet to one of my UNIX accounts in the United States or France and use UNIX ftp from there to pull a pkzipped file from my PC to the UNIX host, where size and resource limitation problems are significantly rarer than in Windows. From there, I would uuencode it and email it to MES. To verify accurate transfer prior to shipping the ftp'd data to MES, I would run the UNIX sum program on both UNIX and here in DOS via a DOS version of the sum program. This verified that checksums of the files were the same, giving me higher confidence that what I was shipping to Mike was, so far, intact. I probably was being overly cautious. The most mysterious problem that arose, also having to do with large files, occurred when a large file sent by Mike from MES failed to arrive, eventually arrived successfully, and then continued to arrive every hour for the next four days; sometimes successfully, sometimes not. A mail gateway located somewhere in England continued to resend the large file (containing about 2 megabytes) and refused to believe it had been delivered. This was a problem since it tied up my modem completely, bogged down my PC's TCP/IP software, and ate up disk space. I contacted various gurus as well as the postmaster at the suspect sight but no one had any idea how to turn off the repeating sends. I dealt with this by turning off mail for a few days. This was inconvenient for my Internet life in general. I turned my mail back on for short periods of time in order to receive whatever could get through and telnetted to a UNIX host when I needed to send something out. Conclusion At no time during the project were we tempted to interact by telephone, which testifies to the richness of even the simplest Internet tools. Using e-mail and some auxiallary tools, small groups of people can already form virtual offices spanning the globe. As bandwidth and direct Internet connectivity become increasingly available at low cost, more sophisticated tools such as ftp, nfs, www and conferencing servers will make such cooperative efforts commonplace. These days I am starting to experiment with the Cornell CUSeeMe audio/video conferencing tool at 14.4 kbps and believe such technology will become widespread within the next year. Maybe I will be able to convince MES to acquire that technology as well as a direct Internet connection so I might hear the bagpipes and see the green hills of Scotland!