Chengyu Dong
UCSD CSE
About me
I am a PhD student in Computer Science at UCSD. I am interested in the principles (and theories, if possible) of learning efficiency, including data efficiency and model efficiency, as well as principle-guided efficient learning algorithms in realistic applications. I believe learning efficiency defines intelligence.
Fun facts about me:
- My previous research focus was Physics and Astronomy, in particular Solar System Dynamics, that means the motion of any objects outside Earth and within our solar system (Yes, you are right! The three-body problem!)
- Football (or soccer?) is my life. I play and watch games every week, rain or shine. DM me if you want to watch or play together! Football forever ⚽!
- I love drama. I read (and hopefully also write in the near future) drama scripts and act in stage plays. My favourite playwright is Harold Pinter.
Publications
(Selected)
Generalized Uncertainty of Deep Neural Networks - Taxonomy and Applications
We review existing and potential methods for improving the efficiency and robustness of learning systems by exploiting the uncertainty of deep neural networks.
SoTeacher: Toward Student-Oriented Teacher Network Training For Knowledge Distillation
How to train an ideal teacher for knowledge distillation? We call attention to the discrepancy between the current teacher training practice and an ideal teacher training objective dedicated to student learning, and study the theoretical and practical feasibility of student-oriented teacher training.
Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting
We found that label noise implicitly exists in adversarial training and can explain the intriguing and problematic robust overfitting phenomenon. Robust overfitting is in fact an early part of an epoch-wise double descent.
LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classification
We leverage the persistent and consistent order of deep neural networks in learning data examples to identify high-quality pseudo-labels for text classification. Those pseudo-labels are obtained cheaply based on keyword match.
Towards Adaptive Residual Network Training: A Neural-ODE Perspective
We motivate from the Neural-ODE perspective and design an adaptive training algorithm for ResNet, which can save ~50% training time.
Data Quality Matters For Adversarial Training: An Empirical Study
We show that those well-known problems in adversarial training, including robust overfitting, robustness overestimation, and robustness-accuracy trade-off, are all related to low-quality samples in the dataset. Removing those low-quality samples can greatly alleviate these problems and often boost the robustness as well.
