Utilizing 100G with one server at SC2014


The Canadian High Energy Physics network organization, HEPnet/Canada at the University of Victoria and CANARIE,  together with their colleagues at Caltech and University of Michigan, are gearing up for 100 GE demonstration using a single high performance server in Victoria and another in University of Michigan booth at SC2014.

High energy physics experiments, such as ATLAS and CMS at the LHC at the CERN Laboratory in Geneva, are collecting hundreds of petabytes of data. The data is distributed around the world using high-speed networks.   As the volume of data increases, we need to develop faster networks and storage systems.

At the University of Victoria, this will require the use of the Brocade MLXe-4 and CFP2 module to establish a dedicated 100G circuit to the Super Computing show floor. The path goes over BCNet and CANARIE to PacificWave in Seattle where it will be carried by a dedicated CenturyLink 100G path to the Caltech Booth and the University of Michigan Booth via the SCinet (SC 2014) network.

The high performance data transfers nodes by the Victoria and Michigan groups are Dell R920 servers populated with FusionIO (SanDisk) solid state storage cards and will connect to the MLXe-4 using 3x40GE NICs. An OpenDaylight controller employing custom multipath extensions will be established on the show floor and used to control the MLXe-4 at Victoria.

100G WAN Transfer nodes

Figure shows the network from the University of Victoria to the Caltech and University of Michigan booths at the SC 2014 Show Flow .


November 18: Work is ongoing to maximize the disk-to-disk transfer rate from the SanDisk Fusion ioMemory system.

November 17:  Challenges due to unexpected changes in switches lead to issues in the link between UVIC and SC during Monday.  These were identified and corrected in the evening.   The plot shows the transfer from UIVC (red) to the Michigan(yellow) booth.


November 15: The attached plot showing a perfectly stable 96 Gbps mem-to-mem transfer between one machine on the show floor and a single machine at UVic using only 9 tcp streams.

2014-11-15-both directions-mem-to-mem


This entry was posted by igable. Bookmark the permalink.