Computer Science - DeMeTeR: Interpretation of diffusion models through representation analysis

In power from: 27 May 2024

Registration criteria

Principal Investigator: prof. dr hab. inż. Przemysław Biecek

E-mail: przemyslaw.biecek@gmail.com

Project description

Diffusion models are the latest revolution of generative modelling in computer vision. However, we still lack an in-depth understanding of their inner workings from both an empirical and theoretical standpoint. Improving this understanding is not only crucial to their continued development. It also allows for enhancing their safety and provides opportunities to develop new methods for explainability of predictive models in computer vision.

 Considering that, the main goals of the DeMeTeR project are:

- to broaden the practical and theoretical understanding of diffusion-specific latent representations and architecture specific internal representations of diffusion models,

- to develop novel methods of manipulating these representations that allow for enhancing safety and explainability of deep learning models.

Questions and hypotheses:

We aim to address the following research questions and hypotheses. How to improve the understanding of state-of-the art diffusion models trained on images through their latent and internal representations? What are the critical limitations of their generative process and how to overcome them? Latent and internal representations of diffusion models trained on images allow precise control of the generative process. How to transfer this understanding to improve the safety of diffusion-based foundation models? How to exploit these representations for explaining visual predictive models?

Understanding the representations of diffusion models can improve the safety and explainability of other deep learning models.

 

Requirement

- research experience of working on XAI methods

- research experience of working with diffusion methods

Admission limit: 1

Recruitment schedule

Recruitment fee

The admission fee is PLN 200 and is paid to the candidate's individual account generated in the IRK system.

Form of the selection procedure

Evaluation of the following elements shall be taken into account in the selection process:

1) scientific activity of the candidate based on a CV or resumé, documented with scans of materials attached to the application for admission to the School;

2) interview with the candidate / qualification examination;

3) other achievements.

Language of the selection process, including the interview

The interview shall be carried out in Polish or English – in accordance with the candidate’s preferences presented in IRK. If the Polish language is selected, the interview may include parts in English.

Required documents

The candidate shall submit a School admission application only through the IRK. The application shall include the following:

The candidate shall submit a School admission application only through the IRK. The application shall include the following:

1) indication of the selected discipline in which the candidate plans to pursue education or in the case of applying for the Interdisciplinary Doctoral School – fields of science with the specification of the leading field (and where there is no leading field – at least two equivalent disciplines), PESEL number or passport number, nationality, contact information (residence address, e-mail address, telephone number), information whether the candidate agrees to receive administrative decisions by means of electronic communication, consent for processing of personal data for the purposes of the admissions procedure;

2) a scan of the graduation diploma of uniform master’s degree or postgraduate studies or an equivalent diploma obtained under separate regulations. In the case of a diploma equivalent to a uniform master’s degree or postgraduate studies graduation diploma, a candidate shall justify such equivalence. In case the diploma was issued in a language other than Polish or English, the candidate shall attach its certified translation;

3) a description of the initial research project proposal with the title, the description should not exceed 4 pages, type style: New Times Roman size min. 11, single leading, top and bottom margins min. 1,5 cm, outside margins min. 2cm;

4)  a resume or CV outlining the candidate’s scientific activity, including scholarly interests and achievements during the five calendar years preceding the application (if a candidate became a parent during this time, as evidenced by a scan of the child’s birth certificate attached to the application, this period shall be extended by two years for each child), including, but not limited to:

-        publications,

-        research and organizational work at student research groups,

-        participation in scientific conferences,

-        participation in research projects,

-        awards and honorable mentions,

-        research internships,

-        research skills training programs completed,

-        activities promoting science,

-        activity in science movement representative bodies,

-        average of their university grades,

-        professional career,

-        level of proficiency in foreign languages;

5)     scans of materials evidencing scientific activity mentioned in their CV and/or resume;

6)     a document confirming at least B2 proficiency level in English or a declaration of the level of proficiency in English allowing education at the School;

7)     the scan of a declaration by the planned supervisor, confirming their agreement to undertake the duties of a supervisor and of the number of doctoral students, for whom they perform the duties a designated supervisor, in accordance with the template constituting Appendix no.4 to the Resolution no. 17 of the Senate of the University of Warsaw of 20th January 2021 on rules of admission to doctoral schools at the University of Warsaw (the University of Warsaw Monitor of 2024, item 120), the candidate may also attach a scan of their planned supervisor’s opinion and opinions of other academics about the candidate and their scientific activity and/or proposed research project;

8)     the photograph of a candidate’s face that allows for their identification;

9)     a declaration confirming whether the candidate was or is a doctoral student or a participant of doctoral studies or whether they have initiated a doctoral dissertation process or whether proceedings to award them a doctoral degree have been initiated – and if yes, the title of their doctoral dissertation or the research project prepared by a candidate, including the name and last name of the candidate’s tutor or supervisor;

10)  a declaration confirming that they have reviewed the Resolution no. 17 of the Senate of the University of Warsaw of 20th January 2021 on rules of admission to doctoral schools at the University of Warsaw (the University of Warsaw Monitor of 2024, item 120) and Articles 40 and 41 of the Code of Administrative Procedure;

11)  scanned transcripts of records of the graduate and postgraduate studies or the uniform Master’s degree studies, or equivalent documents (e.g. diploma supplement);

 

12)  abstract of the master’s thesis or master’s project in English (up to 3,000 characters with spaces);

Evaluation criteria

The main criterion is experience in research work in the area of machine learning (with particular emphasis on explainable machine learning and diffusion models).

 

The candidate should describe his/her experience by indicating research work conducted, software developed, scientific articles developed and published, and participation in conferences related to the application topic.

Scholarships

PRELUDIUM BIS doctoral scholarships shall amount to:

shall be awarded pursuant to the Act on Higher Education and Science of 20 July 2018.

Recruitment Legislation and Acts to Know