An Experience With A Complex WAN Project

 

This organization (Org) is rolling out SAP to be accessed from 30+ cities. It was already working with a Service Provider (SP) for hosting its enterprise servers and had outsourced its WAN from the same SP. The SAP roll-out meant upgrading the WAN and leasing more space for its SAP servers. At some remote locations, connectivity was being established for the first time by any SP. An additional complexity was introduced by the SP by offering to host the SAP servers in his new data center while the current servers (mail, etc.) were retained in the old locale at least for some time.

 

This was seen as a Project to be completed from start to finish in about 12 weeks including the transition of the enterprise servers to the new data center, under the management of a Project Office. The third parties involved in the Project: the main SP, his subcontractors (visible to a very limited extent), computing and network hardware suppliers and SAP implementation partner. There are some state agencies to deal with figuring as Basic Service Operators. Also some statutory clearances from state bodies have to be obtained for erecting RF masts. The in-house entities involved were the various offices (30+), the SAP implementation team (core group of users in the Org) and importantly the Procurement function for ordering equipment expeditiously. There are business-partners who have to arrange for their connectivity with the SAP servers.

 

This Project brought out in sharp focus a few crucial aspects of Project Management:

 

1. Firstly the Project posed a challenging problem of coordination and communication with multiple vendors on one side to whom the SP had subcontracted the field work of installing WAN infrastructure and the Org’s various remote offices on the other side. The incidents were not uncommon of contractor’s men being refused permission to erect a mast or lay cables by the office, the reason being the intimation had not reached from the Project Office to a specific person in administration function of the office. Or, the contractor’s men were landing at the office on a Sunday when the office was closed. The initial strategy to address this problem was to beat the SP during Project Reviews with the need for closer coordination between his team and his subcontractors. It simply did not work on the ground owing to the multiple levels of subcontracting and at the lowest levels the subcontractor’s men were not capable of resolving problems of communication on the spot. They were easily thwarted. Things improved only when the Project Office handled the problem of coordination and communication with its own offices through a heavy dose of communication about the Project and how vendors would come into office premises for purposes of installing infrastructure and they should be allowed to work without any impediments.

 

2. The importance of communication across silos in the Project Office itself. The Team overseeing the readiness of the data center had accepted a staggered schedule from the SP for augmenting the power supply at the new data center. Another Team was coordinating with the hardware supplier (also the SAP implementation partner) for the installation of SAP servers. At the time of installation, the hardware supplier insisted on switching on all servers at the same time in a cluster thereby demanding full power. This meant a delay of a few days until the power supply was fully augmented to final specified levels. The staggered schedule that appeared to be a smart thing to do to gain time at one time, did not work out eventually. Perhaps this could have been uncovered earlier if the two teams had interacted on the plan that involved building up the power supply in stages. Of course the Team interacting with the hardware supplier missed capturing the fact that all of the power specified would be needed right from the outset. Not to mention an iteration and loss of a few days due to incorrect understanding (On whose part? Not resolved yet!) of the required rack space for the servers.

 

3. The SP himself was probably not used to reviewing detailed plans with his customer in the manner it was done in this case! His plan for laying a fiber in city A was different from that in city B though the activities in both cases were almost identical. The number and description of activities did not appear to be standardized. The Excel based plans did not sequence the activities rigorously either by time or by location. So getting a picture of where the project was during the review, was not easy. These were partially corrected subsequently.

 

The item-wise planning and a weekly time-consuming review had to be accepted by the SP after he badly missed some deadlines owing to some subcontractors’ non-performance. This level of planning details was not insisted upon by the Project Office initially.  

 

4. The SP’s plans showed activities some of which were several weeks long. The SP had to be persuaded to break up the plans with greater granularity consistent with the over-all 12-week time-frame for the Project.

 

5. For risk mitigation, there had to be independent ways of checking on the progress of the Project. Intermediate points in the Project Plan were carefully selected which would allow a physical check by an Org representative on the progress of the Project. Example: receipt of a mast at site, power connection at a remote place, router-to-router check of a segment, etc. This provides reassurance that things are happening on the ground as claimed during the review.   

 

6. The Project reviews takes place with the SP in the Org’s Project Office. The Procurement function joined the reviews with SP to be sensitized about the urgency in procurement of networking equipment and issuance of individual PO’s to the SP under an umbrella PO. This worked quite well as the Procurement function was now for expediting the Project. The intervening Christmas holidays have delayed the order release and execution for the supply of networking equipment.

 

The SAP implementation team however did not participate in the reviews.

 

Similar reviews do not take place with the other third parties involved in the Project.

 

7. Back to communication. The Project Office had originally given a date of 31^st Dec for the entire infrastructure to be ready for use against the final go-live date of 1^st April specified by the SAP Implementation Team. When it was delayed for various reasons to 31^st January (largely due to deadlines missed by the subcontractors of SP), the Project Office was not greatly worried because it was still sufficiently ahead of the go-live date. What was not known to the Project Office was that the Implementation Team had made some revisions to its training plans and now it had scheduled batches of country-wide training from Jan 15^th assuming that the infrastructure would in any case be available from 31^st December. Only a belated meeting between the Project Office and the Implementation Team in mid-December revealed the conflict of time-lines. The training had to be put off to last week of January.

 

This gap in understanding the dates was in spite of the weekly progress report issued to by the Project Office to all stake-holders! This and a few other aspects are the subject matter for the next post.

 

Despite the snafus the Project Office is doing a great job of managing the SP and other parties towards successful completion of the Project.

 

(To be Contd.)

Tags: Best Practices, IT Infrastructure, Project Management