Burning Man has steadily grown throughout its history, and the growth of the organization has required expansion of the information technology infrastructure. During 2004, the systems administration team kept busy maintaining existing services for ever-increasing numbers of users. The web site continues to grow, along with the number of people involved in maintaining it. The San Francisco office has been busier than ever, with recent expansion to another floor in the building. In addition, needs have continued to grow in our Gerlach, Nevada office and on the playa.
Support services received a lot of attention in 2004, as well. Most notably, the server backup system underwent considerable improvements. The year also brought strides forward in the area of network and server monitoring, although this issue continues to require attention.
In addition to the expansion of existing services, of course the team encountered new projects that needed support:
- In February the Engineering Team launched a new questionnaire system to handle the Theme Camp, Art Installation, Mutant Vehicle, and Scholarship questionnaires submitted by participants.
- In March, a T1 connection was installed to link the office in Gerlach to the rest of the world. (Broadband before cell phones!) A great deal of work was done around this time improving the network setup throughout all of the Nevada facilities and configuring the network for connection to the T1. In addition to supporting its own needs, Burning Man expanded this connectivity by making wireless internet access available as a gift to the town of Gerlach. The systems administration team has provided a great deal of support to town residents who wish to use this service.
- In August, shortly before the event, the primary mail server was threatening to collapse under the heavy load that spam filtering software was placing on it. Members of the sysadmin team performed emergency surgery, removing the prior spam filter and installing ASSP, the Anti-Spam SMTP Proxy server in its stead. ASSP has proven very effective for our needs; it works well while being easy to administer and train, and it puts considerably less load on our server than our previous software.
On-playa the sysadmin team undertook much more activity than in years past. The webcast ran smoothly from Thursday through Sunday night, with the notable exception of the actual burning of the Man on Saturday night. A number of servers had been allocated ahead of time to handle an increased number of connections, but in spite of all efforts, the software used to serve the stream (Darwin Streaming Server running on Solaris) was unable to withstand the onslaught of simultaneous requests from those eager to watch; the stream would work for about 10 seconds, then the server would crash and restart, breaking all the connections.
One of the reasons for the T1 installation in Gerlach was to provide internet access to the Box Office during the event, making real-time ticketing information available to the ticket sellers. A long distance 802.11b connection between Gerlach and Black Rock City provided a nearly constant connection. This connection was also eventually extended to the airport and our technical building to support a few important functions. Some delays resulted form the late arrival of several of our buildings and the necessary power.
This year the Department of Mutant Vehicles (DMV) instituted a new, much more involved process of vehicle application processing. The questionnaire system mentioned above not only accepted questionnaires from participants, but also provided administrative access for the DMV’s pre-event and on-playa processing. The sysadmin team deployed this system on-playa, also making it available to the Black Rock Rangers for license verification and tracking purposes.
Looking forward, in 2005 the sysadmin team hopes to increase its numbers. Current members are improving the documentation processes to increase the ease with which new folks can learn about our systems and start contributing. If you have Unix/Linux systems administration skills and are interested in volunteering, please let us know by filling out the volunteer questionnaire.
Submitted by,
Rob Miller Tech Department Lead