Update 2024-1-16-seminar.md

_posts/2024-1-16-seminar.md

@@ -30,44 +30,43 @@ Students should have experience with machine learning, data analytics, and deep
 ## Course Format
 The course is a [role-playing paper reading seminar](https://colinraffel.com/blog/role-playing-seminar.html) that is structured around reading, presenting, and discussing weekly papers. Each class will involve the presentation and discussion of one paper. Each student will have a unique, rotating role per week. This role defines the lens through which each student reads the paper and determines what they prepare for the group in-class discussion. All students, irrespective of their role, are expected to have read the paper before each class and come to class ready to discuss. There will be no exams or traditional assignments. Instead, students will work on a course project focusing on open research questions in trustworthy LLMs.
 
-**Presentation Roles**: This seminar is organized around the different “roles” students play each week, that define the lens through which students read the paper. Students will be divided into two groups, one group presenting on Tuesdays and the other on Thursdays. In a given class session, students in the presenting groups will each be given a rotating role (described below): Presenter (two students), Reviewer, Archaeologist, Researcher, Industry Expert, and Blogger OR Hacker (pick one). Presenting groups should create a formal presentation, i.e., have slides prepared for the group in-class discussion. For each student in a presenting group, their assigned role determines what they should include in the slides. The Hacker and Blogger roles are the only exceptions to the rule. Hackers should provide a Jupyter Notebook instead of slides and Bloggers go over their written articles.
+**Presentation Roles**: This seminar is organized around the different “roles” students play each week, that define the lens through which students read the paper. Students will be divided into two groups, one group presenting on Tuesdays and the other on Thursdays. In a given class session, students in the presenting groups will each be given a rotating role (described below): Presenter, Reviewer, Archaeologist, Academic Researcher, Industry Expert, and Host. Presenting groups should create a formal presentation, i.e., have slides prepared for the group in-class discussion. For each student in a presenting group, their assigned role determines what they should include in the slides.
 
-Depending on changes in course enrollment, the roles might change, for example, remove roles or make roles optional in case enrollment decreases or allow groups of two students for all roles in the event of enrollment increase. Improving based on student feedback, as we go along with the readings, is crucial.
+Depending on changes in course enrollment, the roles might change, for example, remove roles or make roles optional in case enrollment decreases or allow groups of two students for all roles in the event of enrollment increase.
 
-- **Presenter**: Create the main presentation, describing the motivation, problem definition, method, and experimental findings of this paper.
+- **Presenter** (1-2 students): Create the main presentation for around 30 min, describing the motivation, problem definition, method, and experimental findings of this paper.
 
-- **Reviewer**: Complete a full—critical but not necessarily negative—review of the paper. Follow the guidelines for NeurIPS reviewers (under “Review Content”). Please answer questions 1-6 under “Review Content”, and assign an Overall score (question 9) and a Confidence score (question 10). Skip the rest of the review, including writing a summary. Note that you can bypass questions by filling N/A. For example, you really liked the paper and can’t think of any disadvantages. Therefore you can skip the respective question (but use this skip option sparingly). Also, please note that this role does not require going over related work, and is not an exhaustive list of all arguments you can think of. The goal is to enhance your overall critical thinking. The instructor reserves the right to contact students who overuse the N/A option.
+- **Reviewer** (1-5 students): Complete a full—critical but not necessarily negative—review of the paper. Follow the guidelines for [NeurIPS reviewers](https://nips.cc/Conferences/2020/PaperInformation/ReviewerGuidelines) (under “Review Content”). Please answer questions 1-6 under “Review Content”, and assign an Overall score (question 9) and a Confidence score (question 10). Skip the rest of the review, including writing a summary. Note that you can bypass questions by filling N/A. For example, you really liked the paper and can’t think of any disadvantages. Therefore you can skip the respective question (but use this skip option sparingly). Also, please note that this role does not require going over related work, and is not an exhaustive list of all arguments you can think of. The goal is to enhance your overall critical thinking. The instructor reserves the right to contact students who overuse the N/A option.
 
-- **Archaeologist**: Determine where this paper sits in the context of previous and subsequent work. Find and report on one older paper that has substantially influenced the current paper and one newer paper citing this current paper.
+- **Archaeologist** (1-5 students): Determine where this paper sits in the context of previous and subsequent work. Find and report on one older paper that has substantially influenced the current paper and one newer paper citing this current paper. Create 3-5 slides for each paper, summarizing the key points such as the research problem, methodology innovation, and main results.
 
-- **Researcher**: Propose an imaginary follow-up project that has now become possible due to the existence and success of the current paper.
+- **Researcher** (1-3 students): Propose an imaginary follow-up project that has now become possible due to the existence and success of the current paper.
 
-- **Industry Expert**: Propose a new application or company product for the method in the paper (not already discussed in class), and discuss at least one positive and negative impact of this application. Convince your industry boss that it’s worth investing time and money to implement this paper. Your arguments should be particularly applicable to the chosen industry market.
+- **Industry Expert** (1-3 students): Propose a new application or company product for the method in the paper (not already discussed in class), and discuss at least one positive and negative impact of this application. Convince your industry boss that it’s worth investing time and money to implement this paper. Your arguments should be particularly applicable to the chosen industry market.
 
-- **Hacker** (optional between two choices): This role is optional, i.e., students can declare if they would like to be a Hacker or a Blogger. Implement a small part of the paper on a small dataset, a toy problem, or any other simplified version of the paper. Another valid and useful option is to try to reproduce results from the paper, either by downloading and running an existing implementation (with proper credit given to the code sources) or by implementing a core method from the paper. Share a Jupyter Notebook with the code of the algorithm with the class. Your code does not have to be bug-free or run perfectly in all scenarios. Also, you are welcome to use (and give credit to) an existing implementation for “backbone” code (e.g. model building, data loading, training loop, etc.).
-
-- **Blogger** (optional between two choices): This role is optional, i.e., students can declare if they would like to be a Hacker or a Blogger. Write a paragraph each about the two papers and an additional paragraph comparing and contrasting them. The summary of each paper should cover the motivation behind the paper, a description of any of the proposed methods, and an overview of the key findings. Include visual aids such as figures, charts, or graphs to illustrate key points. Explain how these papers relate to one another within the broader context of their shared theme. Explore how these papers may complement, challenge, or build upon one another. Provide links or references to additional resources that complement your blog. This could include related research articles, videos, or online discussions. Your insights should reflect critical thinking, encouraging discussion within the class. Think about how your blog can be useful and interesting to an actual online reader.
+- **Host** (1-2 students): Host the whole discussion in class, including introducing presenters and answering all the questions proposed by the non-presentating group on Piazza.
 
 **Non-presenter Assignment**: 
-- If you are not in the presenting group during a class session, please submit the day before class (due 11:59pm EST) at least one question about either paper - could be something you’re confused about or something you’d like to hear discussed more. Questions that open debates and make in-class discussions explore different viewpoints are a plus. 
+- If you are not in the presenting group during a class session, please submit the day before class (due 11:59pm EST) at least one question on Piazza non-anonymously. The questions can be about either paper (e.g., something you’re confused about) or something you’d like to hear discussed more. Questions that open debates and make in-class discussions explore different viewpoints are a plus. 
 
-- After class and before the end of the day 11:59pm EST, provide constructive feedback to the presenting group. You may focus on one or more reading roles, or on the presentation holistically. Evaluate the clarity of the presentation, the strength of the arguments, and the quality of visuals, if any. Highlighting strengths and areas for improvement. This feedback will be shared post each presentation.
+- After class and before the end of the day 11:59pm EST, provide constructive feedback to the presenting group on Piazza non-anonymously. You may focus on one or more reading roles, or the presentation holistically. Evaluate the clarity of the presentation, the strength of the arguments, and the quality of visuals, if any. Highlighting strengths and areas for improvement. This feedback will be shared post each presentation.
 
-**Everyone, every week** (Optional): After each class session, you may post your thoughts on Piazza, for example, which parts did you enjoy reading, what results and insights did you find interesting, a missing result the paper could have included, any useful additional links and resources, etc. Whenever you agree with the comments of a student’s post, make sure to endorse their answer. You can also post a reply with your additional thoughts.
+**Everyone, every week** (Optional): After each class session, you may post your thoughts on Piazza. For example, which parts did you enjoy reading, what results and insights you found interesting, a missing result the paper could have included, any useful additional links and resources, etc. Whenever you agree with the comments of a student’s post, make sure to endorse their answer. You can also post a reply with your additional thoughts.
 
-**Final Project**: The main project goal is to engage students in research on trustworthy LLMs. In particular, students should try to extend papers from topics covered in class and present the research outcomes as a research paper, in a standard conference paper format. Students are encouraged to work in groups of no more than four members, taking into consideration that the work produced should be proportional to the number of members in a team. Groups are required to include a “contributions” section in the final project report, listing each member’s contributions in detail. Projects will be hosted on GitHub and should include a written report accompanied by a descriptive Jupyter Notebook, with a format similar to this notebook. In addition, groups will present their final projects during the last two class sessions. A PowerPoint or LaTex final presentation is required.
+**Final Project**: The main project goal is to engage students in research on trustworthy LLMs. In particular, students should try to extend papers from topics covered in class and present the research outcomes as a research paper, in a standard conference paper format (NeurIPS or ACL). Students are encouraged to work in groups of no more than four members, taking into consideration that the work produced should be proportional to the number of members in a team. Groups are required to include a “contributions” section in the final project report, listing each member’s contributions in detail. Projects should include a written report accompanied by the code (in a zip file or on GitHub). In addition, groups will present their final projects during the last two class sessions. A PowerPoint or LaTex final presentation is required.
 
-**Technology**: Piazza will be used for announcements, general questions, and discussions. If you are unable to register to Piazza, please email me. Please familiarize yourself with GitHub, Zoom, LaTeX and paper writing practices. To enhance class participation, and unless restricted by low internet bandwidth, please try to keep your video turned on during class. Please keep your audio muted unless you would like to respond to an ongoing discussion or have a question. You can also use the “raise hand” option, type in the chatbox, or use the Zoom reactions for nonverbal feedback. Please remember that all in-class discussions should adhere to Virginia Tech’s Principles of Community. To keep track of student order during office hours, please type your name in the chat as soon as you enter the Zoom room. For one-on-one interactions with the instructor, please post a private note on Piazza or use Slack.
+**Technology**: Piazza will be used for announcements, general questions, and discussions.
 
 ## Grading Policy
-**Readings**: 60 points: Each student will be in the presenting role for 12 sessions and the non-presenting role for the remaining 12. You can earn up to 4 points each time you present (all presenting roles are considered equal). You will receive full credit if you do a thorough job of undertaking your role and present it in a clear and compelling way. When you aren’t presenting, you can earn up to 1 point by completing the non-presenting assignment and by participating in the class. At the end of the semester, extra credit of up to 3 points will be assigned to the most well-made presentation, blog, and notebook.
+**Readings**: 60 points: Each student will be in the presenting role for 10 classes and the non-presenting role for the other 10 classes. You can earn up to 4 points each time you present (all presenting roles are considered equal). You will receive full credit if you do a thorough job of undertaking your role and present it clearly and compellingly. When you aren’t presenting, you can earn up to 2 points by completing the non-presenting assignment and by participating in the class. 
 
 **Final Project**: 40 points divided into the following categories:
 - Proposal: 5 points.
 - In-class presentation: 15 points; your final presentation should be clear to the audience and provide a solid review of your work as if you were presenting at a conference. You can find examples in the NeurIPS’20 schedule (Oral Spotlight sessions such as this one).
-- Novelty: 5 points; your paper should propose something new (a new method, application, or perspective).
-- Clarity: 10 points; your paper should be readable, contain well-defined and clear motivation and contribution statements and appropriately make connections with related work. In general, your project report should follow standard machine learning conference paper formatting and style.
-- Code: 5 points; the code accompanying your project should be well-documented and your experimental results should be reproducible. Your repository should include a README file with full instructions on how to run the code. Moreover, your code should be easy to run with one simple command; if there are multiple steps involved, please make a bash script.
+- Final report: 20 points:
+  - Novelty: 5 points; your paper should propose something new (a new method, application, or perspective).
+  - Clarity: 10 points; your paper should be readable, contain well-defined and clear motivation and contribution statements and appropriately make connections with related work. In general, your project report should follow standard machine learning conference paper formatting and style.
+  - Code: 5 points; the code accompanying your project should be well-documented and your experimental results should be reproducible. Your repository should include a README file with full instructions on how to run the code. Moreover, your code should be easy to run with one simple command; if there are multiple steps involved, please make a bash script.
 
 ## Late and Missed Work
 All assignments are due on the date assigned at the listed time. **No late assignments will be accepted.**