Distributed Systems
CSCI-B 534/ENGR-E 510 (Spring 2023)

Course Description

Distributed computing systems are complex, difficult to understand, and everywhere.

This course will cover the necessary principles, techniques, and tools for understanding, analyzing, and building distributed applications and systems. We will be looking at both distributed computing fundamentals, as well as study the design of popular distributed systems.

We will look at how systems can communicate and coordinate through message passing, and study classical distributed algorithms involving logical and vector clocks, leader election, fault-tolerance, data-consistency, and consensus. Students will also learn about the design of large-scale distributed systems, and be expected to implement many of the ideas studied in class as part of homework assignments and projects.


"To be able to use a second computer, you must know how to use the first one".

Distributed systems build upon and extend many classical areas in Computer Science. Strong fundamentals in Operating Systems, Computer Networks, and Algorithms are a must.


We will use a combination of books and research papers.

  • Required: Distributed Systems: Principles and Paradigms, 3rd Edition (Maarten Van Steen and Andrew Tanenbaum) Online version
  • Recommended: Elements of Distributed Computing (Vijay Garg)

Learning Objectives

  • A fundamental shift in how you think about computing: from serial programs to loosely coupled asynchrnous distributed systems.
  • Design and implement moderately complex distributed systems of your own
  • Understand classic distributed algorithms for synchronization, consistency, fault-tolerance, etc.
  • Reason about correctness of distributed algorithms, and derive your your own algorithms for special cases
  • Understand how modern distributed systems are designed and engineered.


This course is designed and optimized for in-person socratic teaching. A typical in-class lecture comprises of starting with a simplistic solution, and collaboratively iterating on it to develop the final, correct solution.


Lecture Topic Reading Notes
Module A Overview and Prerequisites    
1 Introduction to Distributed Computing Chapter 1 1-Intro
2 Operating Systems: Processes   2-OS
3,4 Computer Networks Chapter 4 3-net
5 OS Concurrency   5-concurrency
Module B Logical Clocks and MapReduce    
6 Event ordering and logical clocks Lamport Clocks, Chapter 6 6-Lamport
7 Total Order Multicast   [See previous]
8,9 MapReduce MapReduce paper 7-MapReduce
Module C Classic Distributed Algorithms    
10 Vector Clocks   8-VC
10 Vector clock applications and Causal Orders Garg Chapter 4, 6 Vector clock proof
11 Mutual exclusion and leader election Chapter 6 10-Mutex
12 Shared Memory mutual execution   11-Bakery
12 Distributed Snapshots Chapter 10 from Garg 12-Snapshots
13 Midterm prep    
March 2 Midterm Exam    
Module D Advanced Networking    
  Remote Procedure Calls Birrel and Nelson Lec4-slides
  High-level communication and publish-subscribe ZeroMQ, Kafka Lec6-slides
Module D Distributed Data Storage    
13 Load balancing   Lec12-notes
14 Consistency Models: Sequential Consistency Chapter 7 Lec13-slides
15 Causal Consistency models Chapter 7 Lec14-slides
16 CAP Theorem, Eventual Consistency   Lec15-slides
17 CRDT   Lec16-slides
18 Failures Chapter 8 Lec17-slides
19–20 Consensus: Paxos Chapter 8 Lec18-slides
20 Raft and Zookeeper   raft Zookeeper
21 Byzantine fault tolerance Chapter 8 Lec21-slides
22 Spark Fault Tolerance   Spark
26 Distributed Filesystems NFS, Ceph Lec22-slides
27 Distributed Machine Learning TensorFlow Lec23-slides
28 Distributed Resource Management Mesos, DRF, Sparrow Lec24-slides

Important Dates

Date Event
Around Lecture #12 Mid-term 1

Evaluation Criteria

The rough breakdown is as follows:

Mid-term 20%
Final 30%
Assignments and Homework 40%
Class participation and Quizzes 10%


The exams will test how well students have understood various distributed algorithms, correctness proofs, edge-cases, tradeoffs, and real-life implementation considerations.

Programming Assignments

The assignments will be a mix of theory and distributed system design. Students will implement various classic distributed algorithms (such as Map-Reduce, totally ordered multicast, logical clocks, various consistency models in a distributed key-value store, etc.).

The design oriented assignments will involve a large degree of programming and debugging. In most cases, the programming assignments are language agnostic (you can pick any reasonable programming language).

A key learning objective of this course is to design, architect, and implement a distributed system from scratch, and to design useful test-cases for evaluating the implementation. Therefore, no starter-code or templates will be provided, to give students the maximum flexibility and freedom to explore the unconstrained design space. Points will be awarded for correct and faithful designs, complete implementation, adequate testing, and reports and documentation.

Most programming assignments will take significantly longer than you anticipate. Start early. Please see the assignment descriptions below (from last year), to get a sense of how they will look like. In general, all programming assignments in this course only specify the "end goal", and you must figure out how to get there: what and how to implement, what libraries to use, etc. There will be no starter-code, no templates, no training wheels. You are on your own.

Simple Data Store  
Distributed Map-Reduce  
Total Order Multicast  
Project: Distributed KV Store  


Classic distributed systems papers will be assigned for reading and review.

Active learning/In-person class participation

Students will learn about distributed algorithms using group activities in class. Typically, small groups of students will "emulate" a message-passing-based distributed algorithm, by passing messages (on post-it notes).

Late submission policy

Students can avail a total of four late submission days as they wish.

Administrative Information

Class Information

  When Where
Main Class Tuesdays and Thursdays 4:45 to 6 PM Woodburn Hall 120
Lab 1 Friday  
Lab 2 Friday  

Labs serve as office hours and assignment help for all students. Grading will also be performed during these times, where students will be asked to explain and justify their work.

Office Hours

Who Email Office Location Office Hours
Prateek Sharma prateeks @iu Luddy 4126 Wed 9–10 am, or by appointment
Adit Sadiwal asadiwal @ iu Wylie Hall 125 Friday 9:45–11
Harsh Atha hatha @ iu Wylie Hall 125 Friday 11:30–12:45
Prashasti Kelkar prkarl @ iu Wylie Hall 125  

Author: Prateek Sharma

Created: 2023-04-25 Tue 11:16