San Francisco
Systems Administration
2012 was another year of great uptime, both on and off playa. Our systems are solid, kept up to date and secure. We continually explore new technologies and solutions, looking for innovative and low-cost options, and as a result, we moved a few systems to cloud-hosting platforms.
After repeated performance issues, the ePlaya was moved to Amazon EC2, which reduced server load issues and increased system flexibility. We have begun a transition of our core staff email into Google Apps, which improves web accessibility of email, increases mailbox storage sizes and reduces our server load, administration needs and overall maintenance costs. We also began managing email, websites and other services for the Burning Man Project.
Tech Support
After the big move to Market Street the previous year, in 2012, we focused on incremental changes and refinements to Burning Man Headquarters and Black Rock City. We have been upgrading and adding redundancy to our office network, creating failovers and making necessary adjustments to our firewall. Since the move downtown, we have been challenged by a series of thefts from our offices. As a result, we have improved office and equipment security, and fortunately were able to quickly recover from the losses to get our staff working again. A planned upgrade to our office backup system more than tripled the capacity and improved reliability. This good backup strategy played a major role in recovering from the thefts; however, these unexpected losses impacted our normal hardware upgrade cycle, which will be addressed in the coming year.
We continue to advise on and support transitions to new platforms within the organization. This year Human Resources transitioned their storage of highly confidential personnel data away from the Accounting network to a more robust, secure, and available hosted storage platform. The Accounting department successfully transitioned to another new hosted software solution called Paylocity. We are upgrading more and more Accounting machines to Windows 7. Security, Backups, and Protection all worked well, saving files and enabling smooth transition to new hardware.
In 2013, we will continue our hardware upgrade cycle for staff laptops, and begin the transition to the latest version of Mac OS X.
Nevada
After many years of frustratingly slow and bandwidth-limited satellite Internet connectivity to our production facility, we were able to setup a high-speed terrestrial link to Black Rock Station and it works great. The setup was originally implemented as a test, however people love it so much that we’re keeping it and are planning necessary upgrades. In addition to a significantly faster connection, we rebuilt and improved the wireless network at Black Rock Station. We still have the satellite connection as a backup in case the new link fails.
In Gerlach, we completed many more network and Wi-Fi improvements throughout the town. After years of making due with donated or lesser equipment, we’re making it a priority to keep relatively current with evolving technologies and more enhancements are on the way. In 2012 we retired a small mountain of antiquated desktops and servers used in Gerlach and Black Rock City, and replaced them with new, up to date equipment. We also improved and completed asset management tracking of all equipment.
Black Rock City
On-playa operations have benefited greatly from bringing on additional seasonal staff and from replacing outdated systems. We formalized an on-call pager schedule for our network engineers, which provided 24/7 coverage of our network while also allowing our staff more flexibility to enjoy the event.
One major improvement was the implementation of automated programming of radio and switch monitoring. This allowed for faster deployment of gear, better flexibility for changes on the fly, and it will greatly decrease the time needed to setup the network in future years. With this and many other changes, we had notably better network reliability than ever. We added a new backbone link for early setup and restoration crews through the DPW Depot, which saved us from having to do extra work re-pointing and re-programming dishes. We are planning to provide even more robust networking for next year.
The DMV on-playa registration system, implemented for the first time in 2011, underwent continued minor improvements, particularly to the Wi-Fi range and reliability of their local network.
During the event in 2011, the power went out in Gerlach, shutting down the main hub for our primary Internet link. To prevent this from happening again we installed a failover power switch and generator at our tower facility in town. This year, even though power outages continue to be normal occurrences in Gerlach, the power stayed on throughout the event. If the power does fail, universal power supplies can sustain the system while a backup generator fires up, and within 30 seconds power will be restored to the building. The main link will be preserved and downtime will be completely avoided.
Submitted by,
Cat Fougere
Eric Haugen
Brendan McKenna
Chris Petrell