Hall Network Specifications
Below we propose a design for fiber connections between the Rack Room and the Hall, and network switch and terminal server placement. The plan below now includes multi-mode fibers for interoperation with the accelerator.
- Elliott Wolin, Mark Dalton
- Based on 10**8 photons/sec, 200 kHz trigger rate, existing L1 rejection algorithm, block=1 mode, no compression.
- Ethernet port counts from Fernando's rack layouts and information from Dave Butler concerning other controls requirements.
- We own about 15 24-port managed switches with 10 GBit/s fiber uplinks, some being used in EEL building and other places.
- Switches can uplink 1.25 GBytes/sec.
- 8-port managed switches with 1 GBit/s fiber uplinks exist.
- Only managed switches (no hubs!) allowed in hall.
- Fibers come in bundles of 12 or 24, 2 fibers needed for one uplink.
- Only single-mode fibers are used in JLab networking systems, multi-mode fibers are used for triggering and fast DAQ applications.
- 24 multi-mode fibers are run from the Rack Room to the Telecom Room, we will extend them downstairs to the Accelerator rack area.
- Standard JLab practice is to terminate all fibers in patch panel when bundles installed.
- Bundle/patch panel installation is expensive, can run much cheaper patch fibers from switch/ROC in one rack area to patch panel in nearby rack area.
- Recommendation from Networknig group is to run two fiber patch cables between rack areas for redundancy, also consider running them inside innerduct for protection.
- DAQ crates require two Ethernet connections each, one for the ROC and one for the fan tray.
- Some ROC's are only used for control, not DAQ (e.g. BCAL LED Pulser).
- ROC's each need an RS232 connection to a terminal server, terminal servers require an Ethernet connection.
- Each terminal server has 32 ports.
- Can run copper RS232 patch cable from one rack area to terminal server in nearby rack area.
- Will eventually need direct fiber link to ROC if max bandwidth > 80 MB/s, assuming 10**8 photons/sec.
- We have no information about US1!
- A minimal-cost option exists whereby we eliminate some 12-strand fiber bundles and patch additional switches into nearby rack areas using fiber patch cables. We will explore this option if the plan below costs too much. With the minimal option we would have to install more fiber bundles to run at 10**8 photons/sec.
|Rack||DAQ Connections||Controls Connections||TS Connections||Max Single ROC Bandwidth||Total Uplink Bandwidth||Fiber Bundle||Switches||Terminal Servers||Possible ROC Fiber Uplinks||Comment|
|T1||6||12||6||<80 MB/s||151 MB/s|| 12 strand single-mode plus 24-strand multi-mode
(Tagger area, already installed)
|U1A||0||7||1||0||low||none||1||0||0||Fiber patch switch to U1B, TS port patch to U1B|
|U1B||5||8||5||<80 MB/s||93 MB/s||12 strand single-mode||1||1||0||Fiber uplinks from U1A, U1C, UN1|
|U1C||2||2||2||low||low||none||1||0||0||Fiber patch switch to U1B, TS port patch to U1B|
|U1D||0||1||0||0||low||none||0||0||0||Copper Ethernet patch to switch in U1C|
|N1||8||13||8||< 80 MB/s||450 MB/s||12 strand single-mode||1||1||0||2 ROC fiber patches from U2A if needed|
|S1||8||13||8||< 80 MB/s||450 MB/s||12 strand single-mode||1||1||0||2 ROC fiber patches from U2B if needed|
|US1||0||0||0||0||0||none||0||0||0||Not sure what's going on here|
|U2A||9||12||9||100 MB/s||770 MB/s||12 strand single-mode||1||1||5||2 ROC fiber patches to N1 if needed at 10**8|
|U2B||9||12||9||136 MB/s||916 MB/s||12 strand single-mode||1||1||5||2 ROC fiber patches to S1 if needed at 10**8|
|D2A||9||19||10||< 80 MB/s||370 MB/s||12 strand single-mode||2||1||0||includes 2 gas shed controls connections|
|D2B||6||8||6||< 80 Mb/s||300 MB/s||12 strand single-mode||1||1||0|
|UN1||0||20||0||0||low||none||1||0||0||Fiber patch switch to U1B|
|ACC||0||0||0||0||0||24 strand multi-mode||0||0||0||For interoperation with accelerator|
- Need 7 x 12 = 84 single-mode fibers plus 24 multi-mode fibers in Hall D.
- 12 single-mode and 24 multi-mode fibers already installed in Tagger building under previous contract.
- Need 12 switches total.
- Need 8 terminal servers.
- 10 ROC's may need fiber uplinks at 10**8 photons/sec, none needed at 5 10**7 photons/sec.
- Inexpensive fiber patch cables used to minimize number of fiber bundles needed.
- Inexpensive RS232 patch cables used to minimize number of terminal servers needed.
- Important: these recommendations do not follow Fernando's note exactly:
- Removed fiber patch panel and TS from U1A
- Removed TS from U1C
- Removed fiber patch panel and switch from US1
- Removed switch from U2A
- Added TS to U2B
- Added TS to D2B