Show HN: Visualizing pipeline parallel algorithms: GPipe, 1F1B (& -2bw, & eager) https://ift.tt/8PtBXfo
Show HN: Visualizing pipeline parallel algorithms: GPipe, 1F1B (& -2bw, & eager) Visualizing how pipeline mode parallel works and how batches are scheduled under various scheduling scheme: GPipe (F-then-B), Pipedream (1F1b), Pipedrea-2BW (no flushes between iterations) and Eager-1F1B (better computation-communication overlapping). https://ift.tt/aNilQhF September 18, 2023 at 12:40AM
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment