Skip to content

Latest commit

 

History

History
337 lines (234 loc) · 12.9 KB

lesson-plan.md

File metadata and controls

337 lines (234 loc) · 12.9 KB
Raw

PNA Rust Lesson plan

A training course about practical systems software construction in Rust.

Over a series of projects, you will build a networked key-value database, with multithreading and asynchronous I/O. In between projects you will study and practice individual subjects necessary to complete the next project. Along the way you will explore multiple designs and their tradeoffs.

See README.md for the overview, goals, audience, and prerequisites.

Prerequisites

As described in the README.md, this is not a course for novice programmers, and there are significant prerequisites. Ensure that you meet them all before proceeding.

Getting the materials

All material for this course is in the

https://github.com/pingcap/talent-plan

git repository on GitHub, in the rust subdirectory. You will want a copy of it on your local computer, particularly for easy access to the conformance tests for each project.

Course structure

The overall arc of the course is defined by a series of coding projects that incrementally introduce new subjects important to systems programming in Rust. Each project extends the previous project, so it is reasonable to simply start your work on each project in the same git repository where you left off on the previous (though you may want to add a git tag indicating where the previous project ended).

Because building an entire database from scratch while also learning all the concepts involved is a daunting task, each project is proceeded by a "building blocks" section, where individual concepts are explored individually. These building blocks will consist of external readings, external programming problems, and other single-subject content.

The building blocks sections are an opportunity to clear your mind, step away from the larger project, and focus your attention on learning and practicing a single subject in isolation. Do not pass them up.

Each project builds on the last — the APIs, command-line interfaces, and parts of the implementation will often remain the same between projects, while you only integrate new features and improvements related to a project's subject matter. As such, it is ok to begin each project by copying your source code from the last.

The projects live in the projects subdirectory, each in their own directories, and consist of a project description in project.md, and a Cargo project containing a complete example solution, and the project test suite. Each project comes with a test suite, and the project should not be considered finished until it passes with cargo test.

You should not read the example code for any project until completing the project yourself — good learning requires trying on your own and failing, over and over until you succeed. You are encouraged though to learn from and apply techniques they contain to your own projects retroactively. Keep in mind though that the example projects are not the only way — or even the best way — to solve the problems. Trust yourself and your own creativity.

You will recieve further instruction about setting up the source code and test suite, as well as project specifications, as you progress through the individual projects.

The expected time to complete each section is not currently well-estimated, but both "building blocks" and "project" will probably take hours, not days, with the projects taking more time. If you are spending much less time than that, or are spending more time, don't worry: these are just bad estimates, and everybody's experience is different.

Getting help

You will run into problems that you don't know how to solve. Don't suffer in silence. Everybody needs help sometimes.

And fortunately the Rust community is amazing and welcoming, and wants to help you. As you are progressing through this course, you are strongly encouraged to join the Rust community experience.

Here are the resources you should consider turning to first:

  • The #rust-training channel on the TiKV Slack. This channel exists just for this course. Please consider joining to support your fellow students and other course-takers. There will always be someone there to answer questions, but there may be a delay due to time zones and other factors. Both English and Chinese languages are welcome here.

  • The #beginners channel on the official Rust Discord. You are almost guaranteed to get some answer here, and if not, don't hesitate to ask again. The people who hang out here are there specifically to help. Because of time zone differences it may take time for somebody to respond. Only English will be consistently understood here.

  • The QQ Rust group #1 (QR code). For Chinese students, this is one of the major Rust communities in China. This is a great place to hang out generally, and for those unconfident in their English skills, a great place to ask for help. There are also WeChat groups, but with their low population caps and invite requirements, they are more difficult to deal with here.

  • The QQ Rust group #2 (QR code). Like the above. If group 1 is at capacity you can get into this one.

These resources may also be helpful:

  • The official users forum. Apply the "help" tag to your post. Questions usually recieve an answer here, but the responses can be limited.

  • StackOverflow. Apply the "rust" tag. You may or may not recieve a satisfying answer.

You are also welcome to email the primary author of this course, Brian Anderson, at [email protected]. I will happily answer questions, and am eager to hear your experience with the course.

Finally, if there is a Rust meetup near you, go check it out. As a Rust programmer these groups are where you will build some of your strongest connections. (Note that that link goes to the old Rust website and may not be up to date).

Making PNA Rust better

As you work through the course content, please be on the lookout for things you would improve about the course, and either submit issues explaining, or submit pull requests with improvements. (If you are being graded, accepted pull requests to this or any other repo used in the course may count toward extra credit during final evaluation — let your instructor or evaluator know!). This is an opportunity to gain experience contributing to an open-source Rust project. Make this a better course for the next person to take it than it was for you.

See CONTRIBUTING.md for more.

Practical Networked Applications in Rust

This is an outline of the course. Clicking each header will take you to the instructions for that section.

Building Blocks 1

Topics: CLI programming, the cargo manifest and environment variables, documenting Rust projects.

Project 1: The Rust toolbox

Task: Create an in-memory key/value store that passes simple tests and responds to command-line arguments.

Goals:

  • Install the Rust compiler and tools
  • Learn the project structure used throughout this course
  • Use cargo init / run / test / clippy / fmt
  • Learn how to find and import crates from crates.io
  • Define an appropriate data type for a key-value store

Topics: testing, the clap crate, CARGO_VERSION etc., the clippy and rustfmt tools.

Extensions: the structopt crate.

Building Blocks 2

Topics: log-structured file I/O, the bitcask algorithm, Rust error handling, comparing collection types.

Project 2: Log-structured file I/O

Task: Create a persistent key/value store that can be accessed from the command line.

Goals:

  • Handle and report errors robustly
  • Use serde for serialization
  • Write data to disk as a log using standard file APIs
  • Read the state of the key/value store from disk
  • Map in-memory key-indexes to on-disk values
  • Periodically compact the log to remove stale data

Topics: log-structured file I/O, bitcask, the failure crate, Read / Write traits, the serde crate.

Building Blocks 3

Topics: unstructured vs. structured logging, the Redis protocol, benchmarking.

Project 3: Synchronous client-server networking

Task: Create a single-threaded, persistent key/value store server and client with synchronous networking over a custom protocol.

Goals:

  • Create a client-server application
  • Write a custom protocol with std networking APIs
  • Introduce logging to the server
  • Implement pluggable backends with traits
  • Benchmark the hand-written backend against sled

Topics: std::net, logging, traits, benchmarking.

Building Blocks 4

Topics: multithreading, thread pools, aliasing and mutability, concurrent data types.

Project 4: Concurrency and parallelism

Task: Create a multithreaded, persistent key/value store server and client with synchronous networking over a custom protocol.

Goals:

  • Write a simple thread pool
  • Use channels for cross-thread communication
  • Share data structures with locks
  • Perform compaction in a background thread
  • Share data structures without locks
  • Benchmark single-threaded vs multithreaded

Topics: thread pools, channels, locks, lock-free data structures, atomics, parameterized benchmarking.

Building Blocks 5

Coming soon! (preview)

Project 5: Asynchronous programming in Rust

Coming soon! (preview)

What next?

So you have completed Practical Networked Applications in Rust. That's a Rusty accomplishment! Now you are on the path to being a great Rust programmer. Want to know where to go next on that path? We've got some ideas.