Hall Network Specifications

From GlueXWiki
Jump to: navigation, search

Below we propose a design for fiber connections between the Rack Room and the Hall, and network switch and terminal server placement. The plan below now includes multi-mode fibers for interoperation with the accelerator.

Elliott Wolin, Mark Dalton
22-Oct-2013


Notes

  • Based on 10**8 photons/sec, 200 kHz trigger rate, existing L1 rejection algorithm, block=1 mode, no compression.
  • Ethernet port counts from Fernando's rack layouts and information from Dave Butler concerning other controls requirements.
  • We own about 15 24-port managed switches with 10 GBit/s fiber uplinks, some being used in EEL building and other places.
  • Switches can uplink 1.25 GBytes/sec.
  • 8-port managed switches with 1 GBit/s fiber uplinks exist.
  • Only managed switches (no hubs!) allowed in hall.
  • Fibers come in bundles of 12 or 24, 2 fibers needed for one uplink.
  • Only single-mode fibers are used in JLab networking systems, multi-mode fibers are used for triggering and fast DAQ applications.
  • 24 multi-mode fibers are run from the Rack Room to the Telecom Room, we will extend them downstairs to the Accelerator rack area.
  • Standard JLab practice is to terminate all fibers in patch panel when bundles installed.
  • Bundle/patch panel installation is expensive, can run much cheaper patch fibers from switch/ROC in one rack area to patch panel in nearby rack area.
  • Recommendation from Networknig group is to run two fiber patch cables between rack areas for redundancy, also consider running them inside innerduct for protection.
  • DAQ crates require two Ethernet connections each, one for the ROC and one for the fan tray.
  • Some ROC's are only used for control, not DAQ (e.g. BCAL LED Pulser).
  • ROC's each need an RS232 connection to a terminal server, terminal servers require an Ethernet connection.
  • Each terminal server has 32 ports.
  • Can run copper RS232 patch cable from one rack area to terminal server in nearby rack area.
  • Will eventually need direct fiber link to ROC if max bandwidth > 80 MB/s, assuming 10**8 photons/sec.


Important Notes

  • We have no information about US1!
  • A minimal-cost option exists whereby we eliminate some 12-strand fiber bundles and patch additional switches into nearby rack areas using fiber patch cables. We will explore this option if the plan below costs too much. With the minimal option we would have to install more fiber bundles to run at 10**8 photons/sec.



Hall D Fiber Requirements and Specifications for 10**8 photons/sec
Rack DAQ Connections Controls Connections TS Connections Max Single ROC Bandwidth Total Uplink Bandwidth Fiber Bundle Switches Terminal Servers Possible ROC Fiber Uplinks Comment
T1 6 12 6 <80 MB/s 151 MB/s 12 strand single-mode plus 24-strand multi-mode

(Tagger area, already installed)

1 1 0
U1A 0 7 1 0 low none 1 0 0 Fiber patch switch to U1B, TS port patch to U1B
U1B 5 8 5 <80 MB/s 93 MB/s 12 strand single-mode 1 1 0 Fiber uplinks from U1A, U1C, UN1
U1C 2 2 2 low low none 1 0 0 Fiber patch switch to U1B, TS port patch to U1B
U1D 0 1 0 0 low none 0 0 0 Copper Ethernet patch to switch in U1C
N1 8 13 8 < 80 MB/s 450 MB/s 12 strand single-mode 1 1 0 2 ROC fiber patches from U2A if needed
S1 8 13 8 < 80 MB/s 450 MB/s 12 strand single-mode 1 1 0 2 ROC fiber patches from U2B if needed
US1 0 0 0 0 0 none 0 0 0 Not sure what's going on here
U2A 9 12 9 100 MB/s 770 MB/s 12 strand single-mode 1 1 5 2 ROC fiber patches to N1 if needed at 10**8
U2B 9 12 9 136 MB/s 916 MB/s 12 strand single-mode 1 1 5 2 ROC fiber patches to S1 if needed at 10**8
D2A 9 19 10 < 80 MB/s 370 MB/s 12 strand single-mode 2 1 0 includes 2 gas shed controls connections
D2B 6 8 6 < 80 Mb/s 300 MB/s 12 strand single-mode 1 1 0
UN1 0 20 0 0 low none 1 0 0 Fiber patch switch to U1B
ACC 0 0 0 0 0 24 strand multi-mode 0 0 0 For interoperation with accelerator


Summary

  • Need 7 x 12 = 84 single-mode fibers plus 24 multi-mode fibers in Hall D.
    • 12 single-mode and 24 multi-mode fibers already installed in Tagger building under previous contract.
  • Need 12 switches total.
  • Need 8 terminal servers.
  • 10 ROC's may need fiber uplinks at 10**8 photons/sec, none needed at 5 10**7 photons/sec.
  • Inexpensive fiber patch cables used to minimize number of fiber bundles needed.
  • Inexpensive RS232 patch cables used to minimize number of terminal servers needed.
  • Important: these recommendations do not follow Fernando's note exactly:
    • Removed fiber patch panel and TS from U1A
    • Removed TS from U1C
    • Removed fiber patch panel and switch from US1
    • Removed switch from U2A
    • Added TS to U2B
    • Added TS to D2B