One goal of the Technology Department following the 2004 event was to focus some time and resources on infrastructure systems. Faced with growing technology needs in both the San Francisco and Gerlach offices, as well as continuing expansion of on-playa technology, a few much needed boosts reinforced support in those areas.
The intervention came not soon enough, though, as the fall of 2004 was rough for Burning Man servers and required some heroics from the administration team. A server problem caused an interruption of web and mail services in early November. The team worked diligently to restore services as quickly as possible, including rebuilding the spam filter database. Later in the month, the website was hacked through a new phpBB security hole the day after Thanksgiving. Fortunately it wasn’t a malevolent attack, but since the machine had been compromised, the entire system was rebuilt on a different server by the dedicated team over the Thanksgiving holiday.
In the spring of 2005, the team moved beyond several years of administration exclusively by core team members from the Bay Area, who were stretching thinner each year supporting many technical endeavors at once. A dedicated systems administrator joined the team for the first time. One of the first fun tasks for the new sys admin role was to help source, spec out, and purchase two new servers to replace two existing machines and to help distribute some of the processing load.
The administrator migrated the Burning Man collaboration tool, the Extranet, onto one of the new machines, which significantly improved response time and eliminated frustrating outages. Some improvements were made in backup practices and some email services were migrated to a new machine before the team had to switch gears and ramp up for the on-playa network and technology support. A fair amount of technical gear had to be prepped and shipped to Nevada to support the event, including a server for on-site registration by the Department of Mutant Vehicles.
While on the playa, about a week prior to the event, the main system administrator had to immediately relocate to Seattle due to a family situation and wasn’t even able to attend the event! Other members of the volunteer administration team kept an eye on things and stood by in case a crisis arose. Needless to say, finding a new San Francisco-based systems administrator was a first priority after the 2005 event.
Despite the staffing setback, a great deal of systems work continued in Nevada. In the spring of 2005, various new Gerlach properties were wired for internet and phones. A file server and WiFi were installed. Future plans call for setup of another office space to expand work areas for the technical staff for the event and year-round geek needs.
The WiFi coverage set up by Burning Man in 2004 for the town of Gerlach was extended a great deal in 2005. By 2006 coverage will be complete, and many of the town residents will have access. Many in the town own computers, and they were used to dial-up access speeds. A growing desire for high-speed Internet is a good thing, except that WiFi was not designed to carry signals over several square miles, through buildings and trees. Very few people know which equipment they will need or how to install it, so systems administration staff helps them out during the off-season. The network itself will most likely be re-designed in 2006 for improved performance without inconveniencing users.
In addition to an internet connection for Gerlach, Burning Man’s Nevada systems administrator supports a satellite-based internet connection to the work ranch, Black Rock Station. Weather causes the signal to degrade very badly, so a challenging project for 2006 is to extend the T1 line from Gerlach to the ranch to replace the satellite service.
Many members of the Burning Man community have expressed curiosity about what is going on in Gerlach during the months around the event as well as all year long. A live Gerlach web camera has been installed at the office with a great view down Main Street.
In 2005 as in 2004, the tech team ran a high-speed 802.11b internet connection to the event site to support ticket sales, a newly created paging system run by the Emergency Services Department (ESD), and pre-event internet connectivity for staff. An antenna mounted on one of the Gerlach buildings is barely tall enough to bridge the 7.77 miles to the event site. Believe it or not, the curvature of the earth affects the signal noticeably at this distance. In 2006, a permanent 60-foot tower is planned for Gerlach to help with this limitation. The events-site link will move from a 40-foot tower at the Box Office to the Network Operations Center (NOC) in Center Camp. A minor tower placement issue arose with the Center Camp layout in 2005, and the configuration will be planned more carefully in 2006, along with all technology and broadcast- related tower usage at the event. As infrastructure matures, and participants bring more and more technology to the desert, managing the airwaves is becoming a greater challenge.
The main link in 2006 will be via microwave from Gerlach, and the Internet connection will be bumped up from a frame relay T1 to a full 10 megabyte connection. The Box Office will get an Internet connection via the NOC in Center Camp, with a backup link pointing to Gerlach in case the main link ever drops. This connection will be shared with our generous partner, a private participant-driven project called IBM (Internet at Burning Man) to benefit all.
The team recovered from many hardware failures due to various causes in 2005. In 2006, several backup computers will stand ready to handle any failures that occur, and most of the root causes of past failures will have been removed via improved air conditioning, better clean rooms, etc. No the failures were due to the power spikes that plagued the system in 2004. In the spring of 2005, a gracious donation of many uninterruptible power suppies helped all teams using technology on the playa.
Respectfully submitted by:
Heather (CameraGirl) Gallagher, Technology Dominatrix
Chris (Taz) Petrell, Nevada Technology Support