CSC6052/5051/4100/DDA6307/MDS5110 Natural Language Processing

This grading policy is for students in CSC4100, for students in CSC6052/DDA6307/MDS6002, please refer to the section below.

Assignments (50%)

• Assignment 1 (15%): Training word vector.
• Assignment 2 (15%): Using API for testing prompt engineering and LLM agents.
• Assignment 3 (20%): Training NLP model with SFT and RLHF.
All assignments need a report and code attachment if it has coding. See the relevant evaluation criterion as the final project.

Final project (40%)

The project could be done by a group but each individual is separately evaluated. You need to write a project report (max 6 pages) for the final project. Here is the report template. You are also expected to make a project poster presentation. After the final project deadline, feel free to make your project open source; we appreciate if you acknowledge this course

• Project poster (10%): Your poster presentation will be rated by other groups and TAs. The average rating will be the final credit.
• Poster quality (1%): We all like well-presented posters.
• Oral presentation (4%): Presenters are encouraged to speak clearly and with enthusiasm.
• Overall subjective assesment (5%): Although subjective assesment might be biased, it happens everywhere!
• Project report (30%): The project report will be publicly available after the final poster session. Please let us know if you don't wish so.
• Technical excitement (7%): It is encouraged to do something that is either interesting or useful!
• Technical soundness (10%):
• Clarity in writing (10%):
• Individual contribution (3%):

• TA favorites (2%): If one of TAs nominates the project as his/her favorite, the involved students would get 1% bonus credit. Each TA could nominate one and he or she could reserve his/her nomination. This credict could only be obtained once.
• Instructor favorites (1%): If the instructor nominates the project as his/her favorite, the involved students would get 1% bonus credit. Instructor could nominate at most three projects. One could get both TA favorites and Instructor favorites.
• Project early-bird bonus (2%): If you submit the project report by the early submission due date, 2% bonus credit will be entitled.
• Code reproducibility bonus (1%): One could obtain this If TAs think they could easily reproduce your results based on the provide material.
• Ethics concerns (-1%): If there are any serious ethics concerns by the ethics committee (The instructor and all TAs), the project would get 1% penalty.

Participation (10%)

Here are some ways to earn the participation credit, which is capped at 10%.

• Attending guest lectures: In the second half of the course, we have invited speakers. We encourage students to attend the guest lectures and participate in Q&A. All students get 2% per guest lecture (in total 4%) for either attending in person, or by writing a guest lecture report if they attend remotely or watch the recording.
• Attending Tutorials: Students are supporsed to attend tutorials, each tutorial account for 1% credit, you are free to be absent for 1 tutorial out of 6, this participation is capped at 5%.
• Completing feedback surveys: We will send out two feedback surveys during the semester to evaluate the course and teaching.
• User Study: Students are welcone to conduct user study upon their interest; this is not mandatory (thus it does not affect final marks).
• Course and Teaching Evaluation (CTE): The school will send requests for CTE to all students. The CTE is worth 1% credit.
• Volunteer credit (1%): TAs/instuctor can nominate students for a volunteer credit for those who help the poster session organization, or help answer questions from other students (not writing assignments).

This grading policy is for students in CSC6052/DDA6307/MDS6002, for students in CSC4100, please refer to the section above.

Assignments (40%)

• Assignment 1 (10%): Training word vector.
• Assignment 2 (15%): Using API for testing prompt engineering and LLM agents.
• Assignment 3 (15%): Training NLP model with SFT and RLHF.
Both assignments need a report and code attachment if it has coding. See the relevant evaluation criterion as the final project.

Final project (55%)

The project could be done by a group but each indivisual is separately evaluated. You need to write a project report (max 6 pages) for the final project. Here is the report template. You are also expected to make a project poster presentation. After the final project deadline, feel free to make your project open source; we appreciate if you acknowledge this course

• Project poster (10%): Your poster presentation will be rated by other groups and TAs. The average rating will be the final credit.
• Poster quality (1%): We all like well-presented posters.
• Oral presentation (4%): Presenters are encouraged to speak clearly and with enthusiasm.
• Overall subjective assesment (5%): Although subjective assesment might be biased, it happens everywhere!
• Project report (45%): The project report will be publicly available after the final poster session. Please let us know if you don't wish so.
• Technical excitement (10%): It is encouraged to do something that is either interesting or useful! Part of them is about the problem to be solved and others are about the solution itself.
• Technical soundness (15%): A) discuss the motivation on why you work this project and your algorithm or approach. Even you are reproducing a published paper, you should have your own motivation. B) Cite existing related work. C) Present your algorithms or systems for your project. Provide key information for reviewers to judge whether it is technically correct. D) Provide reasonable evaluation protocol, it should be detailed to contexualize your results; E)Report quantitative results and include qualitative evaluation. Analyze and understand your system by inspecting key outputs and intermediate results. Discuss how it works, when it succeeds and when it fails, and try to interpret why it works and why not.
• Clarity in writing (15%): The report is written in a precise and concise manner so the report can be easily understood.
• Individual contribution (5%): This is based on individual contribution, probably on a subjective basis.
• Bonus and penalty Note that the project credit is capped at 40%
• TA favorites (2%): If one of TAs nominates the project as his/her favorite, the involved students would get 1% bonus credit. Each TA could nominate one and he or she could reserve his/her nomination. This credict could only be obtained once.
• Instructor favorites (1%): If the instructor nominates the project as his/her favorite, the involved students would get 1% bonus credit. Instructor could nominate at most three projects. One could get both TA favorites and Instructor favorites.
• Project early-bird bonus (2%): If you submit the project report by the early submission due date, 2% bonus credit will be entitled.
• Code reproducibility bonus (1%): One could obtain this If TAs think they could easily reproduce your results based on the provide material.
• Ethics concerns (-1%): If there are any serious ethics concerns by the ethics committee (The instructor and all TAs), the project would get 1% penalty.

Participation (5%)

Here are some ways to earn the participation credit, which is capped at 5%.

• Attending guest lectures: In the second half of the course, we have invited speakers. We encourage students to attend the guest lectures and participate in Q&A. All students get 1.5% per guest lecture (in total 3%) for either attending in person, or by writing a guest lecture report if they attend remotely or watch the recording.
• Completing feedback surveys: We will send out two feedback surveys during the semester to evaluate the course and teaching.
• User Study: Students are welcone to conduct user study upon their interest; this is not mandatory (thus it does not affect final marks).
• Course and Teaching Evaluation (CTE): The school will send requests for CTE to all students. The CTE is worth 1% credit.
• Volunteer credit (1%): TAs/instuctor can nominate students for a volunteer credit for those who help the poster session organization, or help answer questions from other students (not writing assignments).