Version
1.4
(April 27th, 2020): Added a table with the description of the final decision values from reviewers.Version
1.3
(April 21st, 2020): Added some details about the submission and reviewing process.Version
1.2
(April 16th, 2020): Added the instructions for report submission and changed some deadlines.Version
1.1
(March 15th, 2020): Changed the deadline for the focus selection.Version
1.0
(February 18th, 2020): First version.
As a project for the seminar, you are asked to write a report about the literature review you will conduct during the seminar period.
Your report will be the result of a literature review you will conduct for an assigned topic.
You are asked to select a main topic, and then to come up with a subtopic on which focus.
You will receive all the indications to review the literature and produce a report, including how to write a document in LaTeX
, during the lectures.
You can use the main sources for scientific research papers, such as Google Scholar, IEEE Xplore, and ACM Digital Library. Other options are looking through the proceeding of the main conferences in software engineering, such as ICSE, FSE, ASE, MSR, ICPC, etc.
Starting from a target paper, you can look for related works backward in the past, i.e., snowballing, following the references cited in the text. On the contrary, you can use search engines functionalities to look to the papers that have cited the target paper after its publication.
You must always report the process you adopted to find the related work.
The report has to be written by using LaTeX
and IEEE double-column template.
You have 6 pages for the review and 2 more for the references.
The paper has to be written in correct and understandable English.
Your work must be original. As instructors, we are required to run an anti-plagiarism tool against the documents you provide. The University of Zurich provides PlagScan, which is a tool comparing the text of documents with plenty of resources in the web and scientific work.
Please, consider this to not incur into serious issues.
Machine learning is more and more integrated into the activities of companies. Even if it is classifiable as a software, the traditional software engineering processes are not enough to deal with it. The companies need to adopt a specific workflow to let new roles, such as data scientists, and data processes to work effectively together with the legacy operations and software.
Reference:
Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLine, Harald Gall, Ece Kamar, Nachiappan Nagappan, Besmira Nushi, and Thomas Zimmermann. “Software Engineering for Machine Learning: A Case Study.” In: IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 2019, pp. 291-300.
Agile methodologies are used to lighten the development process of software. However, they proved to be effective also in other production environments. Recently, teachers of any grade started to bring these methodologies into classrooms.
Reference:
Pasquale Salza, Paolo Musmarra, and Filomena Ferrucci. “Agile Methodologies in Education: A Review.” In: Agile and Lean Concepts for Teaching and Learning: Bringing Methodologies from Industry to the Classroom. Springer, 2019, pp. 25-45.
Flaky tests are a worrying phenomenon in continuous integration. They consists in non-deterministic results of the test running on the same version of the software under test. Their reasons are not perfectly clear, and their presence can make the testing phase totally unreliable.
Reference:
Qingzhou Luo, Farah Hariri, Lamyaa Eloussi, and Darko Marinov. “An Empirical Analysis of Flaky Tests.” In: ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE). 2014, pp. 643-653.
Let us consider a function that asks the user to have a choice between 0, 1, or 2, then stored into an integer variable. What if the user transmits 3? It would be possible if the switch case were not been implemented securely. 3 is a possible input for the function that impact on security.
Fuzzing is an automated testing technique that “randomly” generates test inputs for programs to expose faults in a highly effective manner.
Reference:
Valentin J.M. Manes, HyungSeok Han, Choongwoo Han, Sang Kil Cha, Manuel Egele, Edward J. Schwartz, and Maverick Woo. “The Art, Science, and Engineering of Fuzzing: A Survey.” In: IEEE Transactions on Software Engineering. 2020.
Events:
26.02.2020 @ 23:59
- 📌 RegistrationWorking time: from
20.02.2020
to26.02.2020
(1 week)
After the kick-off lecture, where we present the organization of the seminar together with the description of the topic options, you have to register officially. The deadline to complete this is the day before the second lecture.
You are required to register by sending an e-mail, indicating a ranking of preference of the topics you would like to discover. Please, also indication some motivations for your choices.
We will try to balance the assignments of topics, preferably accustoming students’ needs. You will be notified during the lecture of the day after the deadline what your final assignment is.
Events:
11.03.2020 @ 23:59
- ✉️ Focus selection18.03.2020 @ 23:59
- ✉️ Focus selectionWorking time: from
27.02.2020
to18.03.2020
(2 weeks)
After having chosen the main topic, you will have to choose a subtopic on which to focus, especially if the main topic is too broad to deal with. You can notify your choice and motivation by sending an e-mail. We will briefly discuss choices during the lecture that will take place during the day after the deadline.
Events:
15.04.2020 @ 23:59
- 📄 Report submission18.04.2020 @ 23:59
- 📄 Report submissionWorking time: from
12.03.2020
to18.04.2020
(5 weeks)
You will have almost five weeks to produce your literature review report.
You are asked to conduct your research and report the results in the form of a report.
At the end of this sprint, you will submit the PDF
file to EasyChair to start the reviewing process.
Please, you are requested to send an anonymous version of your document, i.e., your name must not appear on it.
Events:
16.04.2020 @ 14:00
- 👉 Start paper bidding period19.04.2020 @ 14:00
- 👉 Start paper bidding period19.04.2020 @ 23:59
- 👉 End paper bidding period20.04.2020 @ 23:59
- 👉 End paper bidding period20.04.2020 @ 14:00
- ☑️ Start reviewing period21.04.2020 @ 14:00
- ☑️ Start reviewing period26.04.2020 @ 23:59
- ☑️ End reviewing period27.04.2020 @ 14:00
- 🗣️ Start discussion period03.05.2020 @ 23:59
- 🗣️ End discussion period04.05.2020 @ 23:59
- 🗣️ End discussion period04.05.2020 @ 14:00
- ✉️ Notification to the authors05.05.2020 @ 14:00
- ✉️ Notification to the authorsWorking time: from
19.04.2020
to03.05.2020
(3 weeks)
Your report will follow a simulated blind peer reviewing process. You will not know the name of your reviewers and the reviewers for the same report will knot know each other. After the submission, you have three days for paper bidding. You are asked to select your preferences for reviewing, according to the list of titles and abstracts you find on EasyChair.
Each of you has to review a total of three papers in a week. After that, one week is reserved to have a discussion between reviewers to reach an agreement. The day after that, the notifications for revision, if any, will reach the authors of the papers.
Events:
11.05.2020 @ 23:59
- 📄 Final report submission12.05.2020 @ 23:59
- 📄 Final report submission14.05.2020 @ 14:00
- 🎤 Project presentationsWorking time: from
04.05.2020
to11.05.2020
(1 week)
You have to take into account the revision indications you received from the reviewers of your paper. You have 1 week to submit the final version of the report by e-mail.
You have to show the work you concluded with a presentation. It will be part of the evaluation.
Here you find the instructions for the submission and reviewing process. We are going to simulate the process that happens for a real conference. You will act both as an author and program committee member (reviewer). Each of you will be asked to review a total of three reports. We will also simulate a bidding process to indicate your preferences for the reviews. Once received the three reviews, we will start a discussion period in which a senior program committee member from the s.e.a.l. group will act as the moderator to reach a final decision. The senior member will also provide a metareview that will be delivered, together with the three reviews, to the author of the report. Finally, the author will use the feedback to produce the final version of the report.
Please, notice that we will use a double-blind reviewing process: as a reviewer, you cannot know the identity of the reports’ authors and neither the names of the other reviewers.
I will provide the link for the conference on EasyChair. This link will be used for the whole process. The first step will be to submit the PDF of your paper.
To do this, you will have to create an account and provide your information as an author.
Please, note that you have to use your full name and UZH account.
Then, you will be asked to insert the title, abstract, and at least three keywords to help identifying the topic of your report. Finally, the last field is to upload the PDF file.
Please, be careful to hide your name on the PDF to satisfy the blindness requirement.
I will manually manage the submission time to guarantee that everyone has correctly uploaded the document, without any blindness violation.
Once I will have received all the expected submission, the bidding period will start. You will first receive an invitation to act as a program committee member for the conference so that you can join the reviewing process. Basically, you will be enabled to operate on EasyChair also as a reviewer, with the same account you used as an author. You have to indicate your preferences, which will help me deciding for the final assignment. Each of you will receive a total of three reviews to complete. I will do my best to satisfy your preferences, but also to balance the level of seniority of the members, since there are both BSc and MSc students.
Once you will have received your three assigned papers, you will have about one week to produce a review for each of them
You will find a form on EasyChair to write your reviews. My suggestion is to read the paper and take down your comments (digitally, or with a pen if you print it). Then you write your review in a document. Once ready, you copy and paste your reviews on EasyChair.
Do not forget that you’ll be evaluated also (20%) on the level of contribution you will apply during the reviewing time.
As indications for the review, provide an overall evaluation and explain your rating:
Please, be as factual as possible in your review, free of judgment, and do not use opinion as an argument. A statement similar to “I like X”, “I do not think”, “I assume” has no room in a professional review. Instead, base your response on (counter-) arguments.
Be positive and fair to each other!
Once received all the reviews, we will start with the discussion period. A senior member from the s.e.a.l. group will join the discussion for each of the papers. Your goal is to reach a consensus at the end, and produce the final version of your review. Indeed, you are allowed to revise your original review, and successive versions, including the score.
An effective way to join the conversation would be that of reading the other two reviews and comment the points on which you agree, and those where disagree.
The following table describes the possible decision values.
Decision | Description |
---|---|
😖 Major revision | The reviewers have major concerns about your report. You should definitively revise your report before the final submission. |
😔 Revision | You should carefully consider the reviews and thoroughly revise your report. |
🙂 Good | Good report, try to consider the reviews to make improvements for the final submission. |
🥳 Great | Great report, only minor changes are required. |
Consider that the author of the paper will receive all of your anonymous reviews that s/he will use to revise the final version of the report. Moreover, the senior program committee member will write a metareview that will accompany the ones you will write.
Finally, every author will receive a notification about the decision about the reports.
Every one will be asked for some modifications to the report, if any.
Once ready, you can send me an e-mail with the final version that has to include the source code (LaTeX
) as well.