NACS - Network Backbone Plans for 1998

Summary: This document summarizes a plan developed by UCI's Network & Academic Computing Services (NACS) to upgrade the UCInet backbone in 1998.

August 1998

This paper summarizes a plan developed by UCI's Network & Academic Computing Services (NACS) to upgrade the UCInet backbone. This plan was submitted to the UCI administration in the April 1998 budget cycle and subsequently approved.

The backbone is the primary data network infrastructure connecting every building at UCI. Upgrading the UCInet backbone will benefit every department by providing a foundation for the ongoing network improvements required each year. It will benefit all, immediately and in the future. It is needed to ensure continued, transparent operation of the critical campus data communications infrastructure, and to provide for deployment of CalREN-2, the UC-wide, high-speed regional network planned for deployment in late 1998.

The proposed backbone employs a Gigabit (1000 Megabits/second) Ethernet fault-tolerant core to connect buildings all over campus at Fast Ethernet (100 Megabits/second) and Gigabit Ethernet speeds. Existing departmental networks are then connected to this core through newer, higher-speed routing and switching devices over fiber optic cable. The campus benefits additionally by proactively replacing older building routers before an end to their anticipated life-cycle.

Cost
The cost of implementing this plan is estimated at $1.5 million. The UCI campus benefits from removing a number of roadblocks to higher-performance network-based computing and communication applications, arising from collaborative research, education and other campus activities. It does so by providing a modern, standards-based network of exceptional quality and performance characteristics. It provides a reliable and scaleable foundation for campus departmental networks, in support of growing use and new campus computing applications.

The core of UCI's campus network (UCInet) is "the backbone" -- the "network of networks", that provides connectivity between departmental Local Area Networks (LANs), as well as connectivity from them to the Internet, UCInet, remote sites such as the UCI Medical Center, and soon, CalREN-2. The following is a plan to upgrade this infrastructure.

There are several reasons for this upgrade, not the least of which is continuous growth in the number of network users and in the traffic they generate through increased use of collaborative applications -- some employing voice and video. Additionally, electronic communications have become an increasingly important aspect of life at UCI, in the UC system, and in education and research, in general. Inter-discipline and inter-campus collaboration supported by data networks is becoming increasingly critical. As Internet2 has shown us, collaborative research requires better network support and speed.

Currently, portions of the backbone run at only 10 Megabits/second LAN speeds, and equipment approaching the end of the life-cycle will need replacement in another year. The campus needs a modernized, faster backbone in order to support existing and anticipated research and educational activities and to realize the benefits of CalREN-2.

With the growing sophistication of UCInet and its components, new management tools and intelligent devices are required to coordinate and operate the network as a whole. Additionally, network devices must do more than simply handle data traffic; they must keep statistics about it. Such modern backbones also help reduce external "hacker" attacks from the Internet by supporting new, robust network security solutions.

UCInet has worked well in the past largely due to the use of routers (devices which connect together disparate networks) at the core of the network, in the backbone. The plan continues to leverage that aspect of UCInet's backbone, while augmenting it with high-speed Ethernet switches. Maintaining compatibility with existing devices and protecting the campus network investment are important aspects of the plan.

The backbone now employs a mixture of 10 Megabits/second switched Ethernet, 100 Megabits/second FDDI (Fiber Distributed Data Interface) and routing, provided by routers. The older portion of the backbone is the switched-10 infrastructure and an accompanying population of older-technology Cisco AGS routers. There are also some AGS routers on the FDDI ring. Please see Figure 1 for detail on the existing backbone.

In Figure 1, the FDDI ring connects to four (most recently, five) 10 Mb/s Ethernet switches via two core routers. The switches are shown as square boxes below the FDDI ring which have two arrows pointing to the left and two to the right. A sample detail shows what a router connected to a switch port looks like. There are many routers on these switches, not just the two shown.

The plan is to replace the switched 10 Megabits/second section of the backbone with a blend of 100 Megabits/second Fast Ethernet and 1,000 Megabits/second Gigabit Ethernet interfaces. These interfaces will connect each UCI building to the backbone, radiating from from a core of four, powerful switches interconnected at Gigabit speeds. Switches and switching routers which will replace the older, routing-only technology in campus buildings. Figure 2 shows a picture of the planned core backbone.

Figure 2: Planned New Core Section of the Backbone, Operating at Gigabit Speeds.

The equipment in the four main campus fiber nexus points house the most critical components of the new core backbone. The nexus points are located in the current telephone system "LIM" sites: Central Plant, Social Sciences Plaza, Computer Sciences, and Medical Sciences. This allows the greatest flexibility and speed at the core of the backbone and provides good distribution to the campus buildings. The balance of the backbone exists in the buildings themselves, in the form of building switches which connect to the core via fiber. Each of the core backbone switches has a built-in router, known as a Route Switch Module.

In cases where a no-fault backbone connection is required, it is possible to connect a building to two switches in the backbone mesh, to ensure no single point of failure. The backbone mesh itself can lose an entire switch and continue running. The detail in Figure 3 shows a building switch called BST which connects to two of the switches in the core backbone mesh. A concept known as HSRP (Hot Swap Reservation Protocol) allows the BST switch to ignore failure of either of the core backbone mesh switches.

Figure 3: Detail View of Sample Building Switch BST with "No-Fault" Backbone Connection

Delivery of switched 10 or switched 100 (Fast Ethernet) to a department currently on shared Ethernet requires appropriate wiring and the purchase of networking equipment for the connections. As many campus buildings already have campus-standard wiring, a hundred dollars per person in a departmental switch would provide the former, and around two hundred dollars a person would provide Fast Ethernet to the desktop. In fact, with installation of this backbone, Gigabit Ethernet could be delivered to special research areas requiring it where fiber optic cable is already installed in the building.

The following two figures show the planned backbone in greater detail; figure 4 shows a more detailed view of the entire network, figure 5 shows a detailed view of a typical building connection.

This plan benefits the UCI campus by removing a number of roadblocks to advanced network-based computing and communication applications, arising from collaborative research, education and other campus activities. It provides support for new applications such as multimedia and distance learning by providing a modern, standards-based network of exceptional quality and performance. It also provides for the replacement of aging equipment while protecting existing campus network equipment and training investments. It provides a robust, scaleable network as a foundation for campus departmental networks.

It important to partner with a vendor in creating the new backbone, and to use a single vendor's equipment. Vendor and equipment selection go beyond the technology chosen. Other important factors include:

The existing campus backbone is based upon Cisco Systems routers, and this is one of the reasons the backbone has been so successful and trouble-free to date. Routing both binds the campus LANs together and isolates them, preventing problems on one LAN from affecting others in the network. Routing needs to be a central part of the new backbone. At the same time, routers are expensive, and not as fast as switches. The campus will be best served by the proposed combination of routing and switching. It is thus important that the selected equipment be able to support layer 3 switching, and be standards compliant for routing. The ability to put in place a switch which has the ability to internally support routing by adding a single card containing a router is compelling.

Equipment performance is another consideration. It must be fast and robust, and expandable to meet new demands as the network continues to grow and CalREN-2 use evolves. The backbone should support Ethernet, Fast Ethernet, Fast Ether channel (200 megabits per second to 800 megabits per second), Gigabit Ethernet and Gigabit Ether channel (2 to 8 Gigabits per second) and ATM. The reference, Cisco-systems based, configuration used to create cost estimates for the planned backbone upgrade answers all these points. Any vendor who proposes to participate in backbone planning should as well.

CalREN-2 connectivity will play an important part in the campus network. Providing one OC-12 ATM circuit and one OC-12 POS (Packets over SONET) circuit, both operating at 622 Megabits per second, the campus backbone needs to interoperate with this network in order to supply the benefits to campus LANS desiring "high-speed pipes" for research purposes. All equipment in CalREN-2 is Cisco supplied, and Cisco Systems has partnered with the UC Office of the President to provide a special 40% discount on Cisco equipment ordered by all UC campuses.

The backbone provides a critical link between campus LANs and CalREN-2, and must support services in all areas compatibly. Further, it must compatibly link these areas to UCI's WAN connections, such as the UCI Medical Center, also based upon Cisco equipment.

The one-time cost of data networking equipment, software and anticipated cable or fiber infrastructure improvements, is estimated at $1.2 million. Installation services provided by the vendor are estimated at $175,000, and first year vendor support services at $160,000. This brings the estimated total expense to $1.5 million. Details on the expected costs are shown below:

Category:	Cost:
Data Networking Hardware and Software	$1,082,000
Required Cable/Fiber Infrastructure Changes	$80,000
Vendor Network Implementation Services	$175,000
Vendor Support and Maintenance Services	$160,000
Total Estimated Cost	$1,497,000

The scope of the project is such that assistance will be required in deploying all of the equipment, testing it, programming it, getting it to communicate and debugging connections to existing networks, particularly in programming new VLANs (virtual LAN subnets), subnets and routes. Cisco Systems, for example, offers professional services for this type of thing, estimated here at $175,000.

NACS typically places UCI critical network systems on a premium maintenance service, which will increase the ongoing costs in post-installation years. The support and maintenance costs shown above includes premium maintenance for the entire backbone. Additionally, it provides for remote diagnosis of problems, for help in fine-tuning the new network, and provides four hour resolution on any network problem, 24 hours a day, 7 days a week.

UCInet Backbone Upgrade Plan

August 1998