http://msdn.microsoft.com/en-us/library/ee384252(v=sql.100).aspx

Basically my issue occured right after installing and configuring SSRS on one of the SharePoint farm servers. I got that “The request failed with HTTP status 401: Unauthorized” when I clicked on Set Server Defaults in the Reporting Services section of SharePoint Central Administration. From Microsoft’s link, all I did was instead of using Windows Authentication, I used Trusted Authentication when configuring SSRS SharePoint integration for the first time. Hope this helps out those guys with this same issue.

What I am going to talk about is the story of disaster recovery from an IT perspective in the days after 9/11.

No one would have guessed that an IT disaster would also include two planes crashing into two neighboring towers. In the chaos and panic, no one would have thought to grab the nightly backup tapes before leaving the building. That is precisely what happened. Those tapes were left behind in the panic of the moment. Maybe it is the thought that the building would be ok and we will be able to get back to work after all this. No one would have guessed what would happen eventually. I was glad that everyone at the office got out safely. We told each other to just go home to our families and never look back at the burning buildings. That is exactly what I did. Walked all the way up to West 4^th St. telling people to go home and stay away from that area.

Like I said, I am not going to mention my personal experience from that day.

The funny part is that I was able to remote into our servers from home during all that time. All the systems were pretty much normal considering what is happening outside. Even with the collapse of the two towers, I can still access our systems. As a result, I was pretty confident that we would be able to recover with little issues. However it is one surprise after another. A lot of people knew that the two main towers collapsed that day. I would have never thought that a third building would also collapse and this building was merely across the street from our office building. That building was 7 World Trade Center. I saw it collapsed on TV while connected to one of our server. As soon as I saw that happened, I lost all connectivity. My first thought was that I should have transferred some important files while I still had the chance. Now I was left wondering on the state of our building after
that moment.

Two days passed, and my boss called me to help in the disaster recovery. Good news was that we did have off-site backup tapes but they were a few days old. The bad news was that we did not have a disaster recovery site to mount our backup tapes and run our systems again. Luckily we had a sister company loan us a UNIX server at another location to run our payroll and financial systems. We were a small IT shop so there was not that much to recover from. However during that time, there was no email, no Active Directory, none or our web-based applications were available. However, the immediate priority was running our payroll so people can get paid on time.

We needed PCs to start the recovery process. Unfortunately most of our workstations were desktops and of course they were left behind after the attack. We were contemplating on the states of those machines because some folks tend to save files and documents on the local hard drive instead of their network drive. Naturally we had backups of our file servers but how do recover data from a local hard drive. I must tell you an interesting story of how we did that. More to come but for now we needed PCs. First thing I did was picked up my boss and drove to the nearest now defunct CompUSA in Queens, NY. Two PCs would do for now. Not sure why we did not purchased laptops for portability but I guess they were pretty expensive back in those times. Obviously, people were talking about what had transpired and we were telling the sales people our story and situation. Let’s just say they were very accommodating in helping us purchase computer equipment. Most New Yorkers were abnormally nice during that time. It was quite strange. We also got a few Ethernet cables and a switch to start rebuilding our network. I must say that this was a first for us because our previous network was a Token Ring based network and we heard all nice things about Ethernet. It takes a disaster to push us to implement other technologies.

In the meantime, a colleague of mine was busy with the temporary UNIX server trying to run our payroll. Amazingly my boss and a colleague were given special permission to go back to ground zero and recover the most recent backup tapes. They told me that our building took some damage. The south side sustained the most damaged. Basically the collapse of 7 World Trade Center scraped the south side of our building bringing down the walls in the process.

Another day passed, and we were coming up with a game plan on how to recover all the workstation hard drives and other hardware. Again with special permission from the mayor’s office, we were given clearance to enter our condemned building and recover anything we can find. We were given hard hats and gas masks for protection. The smell of the area still lingers in my memories. It was truly like a war zone there. There were so many military personnel guarding the perimeter of ground zero. We rode on an all-terrain vehicle to our building. We were given only a few minutes to recover anything we can carry with our bare hands. As we approached our building, the site of the rubble amazed me. Never in my life have I seen such damage in a wide area. Our building had this gaping hole on the south side. I was afraid that we could not recover those PCs on that side of the office. Everything practically was destroyed on that side. Even with gas masks on, the smell of the area was very overwhelming. We occupied two floors so we split up to cover more ground. My tool of choice was a plain old screwdriver. We had over 100 plus PCs to extract the hard drives but a good number of them were lost on the south side. Luckily the IT area and the server room were on the north side of the building. It was a relief that our server room was not completely destroyed. I went to each PC and extracted the hard drive. Time was of importance, so we just left the PCs the way they were. We considered them destroyed anyway and the insurance company would recognize that. The moment I would never forget was when I was on the edge of the gaping hold on the south side. Basically I had a front row seat view of the entire ground zero. I wish I had a camera phone to capture the event. In any case, I took a moment and gazed at the entire disaster site. Anyway, it was off to work on getting those hard drives out. Unfortunately we could not recover personal items from people’s offices and we considered them lost and destroyed. We got what was needed and it was the last time we will ever go back to the building again.

The next few days, was all about getting our other systems online again. We now had office space for all our employees and IT equipment. We had to purchase new servers right away because we had web based systems that many remote users needed to access. We worked long hours trying to set up our various systems like networking, firewall, Active Directory, Exchange, SQL Servers, file servers, and custom applications. I would say that it took a few weeks for everything to be back the way it was before 9/11 but we all know that I will never be the same.

This was indeed a good learning experience from an IT perspective. Because of 9/11, we now formally established a disaster recovery site in case something like this happens again. Employees are well aware of saving their files to network drives. We made sure we have a good backup plan. Considering what had transpired, we were very fortunately that the server room was intact after the disaster. Most of the workstations were still there and we were able to extract the hard drives. There was of course some data lost due to bad hard drives and destroyed PCs on the south side but overall, we were lucky.

IT is the lifeblood of many information based companies. We all stress proper disaster recovery strategy but a disaster can throw a curve ball sometime and you find yourself scrambling to do the unexpected. Personally, the key for me was to stay focus on the priorities. Do whatever it takes to get all the systems back online immediately, even if requires long hours. You can take a vacation later which I did eventually. All that hard work will eventually pay off and people will have a new appreciation of IT. This was my IT disaster recovery during 9/11.

Maynard