2025-08-23
Step 1: divide the file into N segments evenlyend up with F = f0 + f1 + ... f999time = 0Step 2:Set up connections from origin machine to targetmachines. Each connection uses R/N IOStep 3:Set up connections between target machinesEach connection uses R/N IOTaking m0 as example:connect m0 to its next 499 neighbors, ie m1-m499Step 4:origin machine: streaming f0 to m0m0: streaming f0 to m1-m499behavior is same for all m0-m999 machine: m(n) writes to m(n+1) through m(n+499)!!! All streaming tasks start and finish at the same time.At completion, m0 receives f0 from origin machine,receives f501-f999 from m501-m999Total data on m0: S / 1000 * 500 = S / 2 =250GBytesTime is just the time used by origin machine to streamf0: (S / N) / (R/N) = 400sStep 5:repeat step 4, but stream f500 from origin to m0;similarly, streaming f(n + 500) to m(n)At completion, m0 received another 250GBytesTime is same as Step 4 = 400sSolution Total time: 800sOriginm0Problem:Replicate a 500GB file to 1000 Machines1. Network IO each machine: R = 10GBit/s2. File size: S = 500GByte3. Destination machine number: N = 1000Solutionf0m1f1m999f999m0Originf0m1m499f0f0m501m999f501f999Origin has1000 outconnectionm0 has 499 outconnection;500 in connection