Cardiff University – Microduct Dark Fibre to Link Cathys Park (main campus) to Heath Hospital Solution
![Download as PDF Download as PDF](https://community.jisc.ac.uk/sites/all/modules/contrib/print/print_pdf/icons/pdf_icon.png)
Edward Wincott, Janet
Anthony Morgan, Cardiff University
Tom Franklin, Franklin Consulting
Overview
This case study describes the implementation of an innovative dark fibre solution to provide inter-connectivity for three remote sites across the centre of Cardiff. The technology employed is a rapid deployment, slot-cut conduit system through which fibre optic cables are blown. The solution satisfied the need for an unlimited bandwidth inter-connection with high security, moderate capital cost and low recurrent costs. The motivation for the project, the planning, the equipment and its installation are considered. Post installation issues such as the performance and benefits are also assessed.
Executive Summary
Main Points:
- Needed to connect the main campus of the Cardiff University to the campus of the recently merged University of Wales College of Medicine, 3.5 km away.
- Two separate slot cut fibre routes were cut to provide resilience.
- Total cost including circuit and equipment and five years maintenance was £300,000.
- One cable has been severed by utility companies on ten occasions, but there has as yet been no network outage due to the fault tolerant duplicate routing.
- The main benefit is a comparatively inexpensive inter-connection solution providing virtually unlimited bandwidth (with up to 48 dark fibres per cable).
Introduction
Cardiff University is a medium sized university with 22,000 students, 5,000 staff and an annual turnover of £300 million. It has 28 schools and almost 5,000 student residences, most of which are within walking distance of the main campus.
The University merged with University of Wales College of Medicine (UWCM) in December 2004 and connectivity between the previously separate organisations was required, in particular between the University and a hospital, a teaching block and a hall of residence some distance from the main campus. The perceived benefits of connecting the remote sites included access to resources on the University network, electronic communications and access to JANET.
Service Requirements and Constraints
Cardiff University’s network is configured as three separate VLANs (Virtual Local Area Networks) – one for students, one for staff and one for management. Where possible these logical networks are also kept physically separate to maximise security, so a connectivity solution was sought which would support multiple physically separate networks. The solution also had to be affordable in terms of both capital and recurrent cost when viewed over a five year period.
An infrastructure solution that supported an extension of the network to more sites in the future was preferable. The initial sites connected in this project were the main Cathys Park Campus and the Heath Hospital site some 3.5 km away. Since the completion of the initial project the University has also connected a hall of residence and will be connecting a new building which is currently under construction.
The original budget for the project was provided by a £200,000 grant from HEFCW (the Higher Education Funding Council for Wales). This proved to be insufficient and the University added a further £100,000 to the budget. There was no budget for recurrent costs so a solution which capitalised the costs was considered advantageous.
Identification of Possible Solutions
The University considered laser and wireless, deep trench and ‘microduct’ fibre optic cable and leased line solutions.
Laser/Wireless
Wireless systems were considered but at least two separate laser or wireless links between sites would have been required to enable the physical separation of the student and administration networks. However, where wireless or laser installations are mounted too close together there is a danger of interference between systems.
In the case of laser links, there is a small but significant divergence of the laser beam. When two systems are mounted in parallel the spreading of the laser beams can result in transmissions from one unit being picked up at the receiver of the second system, resulting in interference and data loss. To avoid this, transceivers must be mounted sufficiently far apart or the beams angled to prevent beams overlapping at the receivers. For parallel beam alignment, the minimum recommended separation is 1.2°, ie. 20 metres for each kilometre.
Beam divergence is much greater with microwave wireless systems than with laser systems. To prevent interference a very large separation of antennae must be achieved. Where antennae cannot be widely separated, a different technique must be employed in which completely different channels or frequency bands have to be used to avoid interference and data loss.
For the inter-site links required at the University, the distances involved and the limited mounting points for the equipment meant that the necessary transceiver separation for laser links could not be achieved. This ruled out a laser solution. A potential wireless solution was also ruled out due to the cost and complexity of achieving a high bandwidth, multiple channel system, given the geography of the sites.
Fibre Optic Cable
The next option was a fibre optic cable-based connection. This would provide high bandwidth as well as multiple physically separate data paths. The traditional method of implementing a fibre optic connection is for deep trenches to be dug, wide bore ducting installed and multi-core fibre optic cable pulled through. This is ideal when connecting buildings on a campus where distances are short or where ducting is already in place to support other services. Deep trench ducting becomes more expensive and problematic when distances become great, the land it must cross is not owned or the public highway has to be crossed.
The traditional method of installing fibre optic cabling was preferred by the University because the depth of the trenching provides security from damage by contractors during any subsequent work. Although capital costs can be very high, there is no on-going annual cost and the cabling system can support network needs for many years into the future. However, during the research into a possible solution, a novel system for installing fibre optic links was identified which promised greatly reduced installation costs.
FibreSpan offers a low cost, dark fibre solution for inter-site connectivity through their patented method of laying cables in a shallow slot cut in the road or pavement, to a depth of 280‑300 mm. FibreSpan's microduct is laid in a 12 mm wide slot which is cut by a large cutting wheel. Access chambers are installed and fibres (up to 72 for multimode and 96 for single mode) are blown along the microduct. The fibres are terminated at each end and can then be connected to switches or routers as for standard fibre optic links.
FibreSpan holds a DTI/OFTEL Telecommunications Licence and has code powers which permit it to dig trenches along the public highway for the installation of communications cabling.
The cost model for FibreSpan microduct based systems comprises an initial installation charge, in common with deep trench/ducted solutions, plus an annual maintenance cost. The maintenance service is essential as this covers damage to the fibre due to third party contractor excavation. The key point to note is that there is no bandwidth related cost and the FibreSpan system is therefore one of unrestricted bandwidth. This contrasts sharply with the solutions generally offered by most mainstream fibre companies. The usual fibre optic connection solution is on a leased line basis using client end links to the company’s core network infrastructure. Telcos typically sell leased line bandwidth as a commodity in Mbit/s or Gbit/s service packages at varying costs depending upon bandwidth. It is uncommon for any telco to lease their private dark fibre circuits at a fixed cost and where this is offered, the cost is very high.
With traditional multi-core fibre cable connection, one pair of fibres would be used for the student network and a second pair for the staff network. Normal practice would be for 8, 16 or more multi-core cable to be installed. This would leave spare fibres for use if one fibre became damaged or if the organisation wished to expand the network. This provides a degree of resilience and upgradeability.
With the relatively low cost of the FibreSpan microduct solution, it became possible within the University’s budget to install two completely separate microduct links between the two sites. With 48 core fibre cable blown through between sites, the solution developed consisted of two fibre channels per microduct route. Both student and staff networks could be supported in each of the independent conduits, so providing essential fault tolerance.
Leased Line
The University considered the possibility of using leased lines (such as LAN Extension Service as these were considered to be the most reliable), but the flexibility afforded by dark fibre which allows management of the network to meet evolving needs was preferred.
Cost/Technical Information
Fibre optic connection via microduct was only available from FibreSpan so a direct cost / technology comparison with other potential suppliers of similar systems was not possible. The comparison that could be made was with fibre optic cable run in traditional deep trench ducting.
The five year cost for a fibre optic cable laid in a traditional deep trench would have been around £600,000 as opposed to the estimate of £300,000 for two slot cut fibre cable ducts each with 48 fibres over five years. The specification for two independent slot cut fibres was considered highly desirable in order to incorporate fault tolerance into the link. These prices covered both the capital and recurrent costs.
Considering both capital and recurrent costs over five years, the FibreSpan microduct solution was seen as highly attractive as it was half the cost of digging a deep trench.
Parameter |
Technology |
|
Deep trench traditional fibre optic cable |
FibreSpan slot cut microduct |
|
Installation cost |
For private circuits – expensive to install |
For private circuits – a quarter the cost of traditional method to install |
For leased circuits – cost rapidly increases with bandwidth |
Compared to low bandwidth leased circuits – more expensive Compared to high bandwidth leased circuits – fixed cost results in increasing cost attractiveness |
|
Annual cost |
For leased circuits, cost escalates with bandwidth |
Fixed maintenance cost, independent of bandwidth |
Installation speed |
Time consuming to install |
Very rapid to cut slot, insert microduct and blow fibre cable |
Vulnerability to contractor excavation |
Limited vulnerability |
Vulnerable due to shallowness of installation |
Resilience |
Only provision is via spare cores in fibre cable |
As result of low cost, multiple microduct routes can be installed to provide resilience |
Maintenance contract availability |
Not usually covered |
Maintenance contract available |
Table 1: Comparison of traditional fibre optic cabling and FibreSpan microduct
Although deep trench installed fibre which is less vulnerable to being severed would have been preferred, the University chose FibreSpan’s microduct system based on the excellent value for money and total project cost.
The FibreSpan Solution
There are three elements to the service offered by FibreSpan:
- Planning. Network design, including a full site survey and liaison with local authorities and landlords.
- Deployment. Rapid, unobtrusive and discreet installation of the ‘all fibre’ network.
- Support. All networks are fully maintained and supported.
There is an installation fee and an all-inclusive fixed annual fee for dark fibre connectivity which includes comprehensive maintenance.
Fibre Cable to be Installed
The cable is G652 single mode fibres to specification MHT 1400, set in a buffer layer to provide dimensional and thermal stability. The outer thermoplastic layer is designed to provide protection and facilitate installation. The units are designed for blowing into microducts and tube bundles. The fibres are dry, not coated with gel, thus permitting fast and contamination-free connections, and have the following specification:
Fibre type: |
Single mode to G652 (ITU-T ) and MHT 1400 |
Attenuation typical: |
0.4dB/km max at 1310nm (at room temp) 0.3dB/km max at 1550nm (at room temp) |
NB. dB is the difference (or ratio) between two signal levels and is used to describe the effect of system devices on signal strength.
|
2 fibre unit |
4 fibre unit |
8 fibre unit |
12 fibre unit |
Outer diameter (nom) |
1.1mm |
1.1mm |
1.5mm |
1.6mm |
Mass ‘w’ (nominal) g/m = kg/km |
1.0 g/m |
1.0 g/m |
1.8 g/m |
2.2 g/m |
Min bend radius (MBR) when deployed |
50mm |
50mm |
80mm |
80mm |
Fibre Colours |
Blue, orange |
Blue, orange, green, red |
Blue, orange, green, red, grey, yellow, brown, violet |
Blue, orange, green, red, grey, yellow, brown, violet, black, aqua, pink, white |
Table 2: FibreSpan fibre bundle specifications
Feasibility and Planning
To determine the feasibility of a microduct solution it is necessary to work out a route which will not cut other service cables or pipes. However, as the microduct is installed in slots there is little real risk of cutting through other services. The corollary is the risk that utility companies will cut the microduct fibre when they dig up their own cables or pipes. FibreSpan takes responsibility for checking with the local planning authority where existing cables are, and for negotiating with them the exact route for the cable.
Cardiff University carried out the initial design of the fibre routes and then discussed these with a site engineer from FibreSpan. A key element of the plan for the link was that there should be two independent cable routes in order to build resilience into the fibre network.
There were some minor modifications proposed to the routes. It was then FibreSpan’s responsibility to negotiate the precise route with the local council. The University owns some service ducts in the campus area and so it was planned to use these where possible. This would reduce the cost of the installation (as there would be less route to cut) and was technically sensible since the University ducts were considerably deeper than the FibreSpan slots would have been. FibreSpan would then run microduct from the University ducts to the destinations.
Procurement
FibreSpan has a patent on the technology used for their method of laying cable, and as such are the sole suppliers of the equipment. Cardiff University IT Services therefore had to certify to the University's purchasing department that there was only a single supplier for the solution and consequently it was not possible to go out to tender. However, a sufficient market research had been carried out to ensure that the selected solution was the most economically advantageous.
Implementation
Installation of the first fibre was straightforward with no problems encountered and was completed in Spring 2003. However, the second route to the hospital crossed some land owned by the NHS. It took many months of negotiation before this short stretch of grass could be crossed so the second line was not completed until June 2004.
Inspection chambers are required every 450 metres along the microduct route and a total of 15 were required for this installation along the two routes.
The fibre optic cables installed by FibreSpan were terminated on fibre patch panels in the normal way, presented as SC connectors. The fibre cables were then simply patched to the network switches and operate as another piece of the network.
Figure 2: Picture of the slot cutting machine in action
Figure 3: Picture of the microduct laying operation
Project Timescales
It took six months from the provision of funding by HEFCW through the identification of a solution and the securing of additional funding to the signing of contracts. From then it took FibreSpan five weeks to complete the first route, including getting the necessary permissions from the Council to lay the cables and laying the microduct and fibre cables. The second cable took considerably longer to complete due to the negotiations with the NHS.
Operational Performance and Reliability
As with any other part of the network the fibre is managed and, apart from when it has been cut by contractors digging up the roads, it has given no problems.
The main problem has been repeated road works on one stretch of the cable route which have resulted in it being cut ten times in the last two and a half years. This section is at a busy roundabout and the local Council will only allow that stretch of road to be dug up on a Sunday, and then only if there is not a major event (such as the cup final) going on. This has on several occasions led to significant delays in the cables being repaired. Since the link comprised two cables which followed different routes, the University has not, as yet, lost service by having both cables cut or other faults developing at once. Both cables are in service all the time, with automatic switching when there is an outage on one of the cables to ensure fault tolerance.
Benefits of Project
FibreSpan’s microduct based slot-cut fibre system provided a cost effective solution to high-bandwidth inter-site connectivity between the University’s sites.
Since it was possible to install two separate microduct routes between the two sites the University achieved a resilient connection between the sites. With 48 core fibre cable in each microduct, it was possible to maintain physical separation of the VLANs between sites along each fibre optic cable.
Lessons Learned
The need to select a cable route where the roads are unlikely to be dug up is the most important lesson learned. Whether utilities companies check a route for other services before digging it up is unclear, but where a number of ducts, pipes etc. are concentrated, the chances of a microduct cable being cut are very high. FibreSpan cable is so near to the surface that other services will invariably be below the fibre and when these are excavated the microduct is highly likely to be damaged.
Thought should also be given to the time when access to the cable will be granted by the local authority should faults occur, as this can cause significant delays in effecting repairs. It is therefore worth checking what restrictions exist on undertaking repairs that require roads to be dug up.
Unfortunately the Council has considerable control over the routes that the cables may take, and in this case insisted that the cable was routed through the interchange which led to the problems with the cable being cut and delays in its repair.
To build resilience into a slot cut fibre link it is worth considering laying two microducts along two different routes where this is affordable.
To simplify the procurement process the University has also moved to having preferred suppliers for both their CAT-5 cable and fibre optic cables, which means that they do not have to go to full open tender or OJEC (Official Journal of the European Communities) procurement for each project.
Summary
The implementation of a FibreSpan microduct connection between the University’s main campus at Cathys Park and Heath Hospital using two separate duct routes has been highly successful and cost effective. Overall the solution has been put in place at a lower cost than a traditional deep trench method with the added benefit that fault tolerance has been introduced. Minimal annual costs are being incurred for a fibre circuit which is supporting current bandwidth needs and which will support unrestricted future bandwidth when needed at no additional circuit cost.