Junwei Pan

Junwei Pan

I am a Machine Learning Researcher at Tencent, focusing on recommendation systems, computational advertising, and multi-task learning. Before that, I was a Principal Research Engineer at Yahoo! Labs/Research, working on streaming news recommendations, relevance search, and online advertising. I received my M.S. degree from Renmin University of China, advised by Prof. Jiaheng Lu.

jonaspan [AT] tencent.com, pandevirus [AT] gmail.com
[Google Scholar] [Semantic Scholar] [ResearchGate]
Binhai Building, Tencent, Shenzhen, China

Preprint

(* Equal Contribution)

  1. Towards Unifying Feature Interaction Models for Click-Through Rate Prediction
    Yu Kang, Junwei Pan, Jipeng Jin, Shudong Huang, Xiaofeng Gao, Lei Xiao [arxiv]

  2. Long-Sequence Recommendation Models Need Decoupled Embeddings
    Ningya Feng*, Junwei Pan*, Jialong Wu*, Baixu Chen, Ximei Wang, QianLi, Xian Hu, Jie Jiang, Mingsheng Long [arxiv]

  3. Towards Scalable Semantic Representation for Recommendation
    Taolin Zhang, Junwei Pan, Jinpeng Wang, Yaohua Zha, Tao Dai, Bin Chen, Ruisheng Luo, XiaoxiangDeng, Yuan Wang, Ming Yue, Jie Jiang, Shu-Tao Xia [arxiv]

  4. Crocodile: Cross Experts Covariance for Disentangled Learning in Multi-Domain Recommendation
    Zhutian Lin, Junwei Pan, Haibin Yu, Xi Xiao, Ximei Wang, Zhixiang Feng, Shifeng Wen, Shudong Huang, Lei Xiao, Jie Jiang [arxiv] \[\mathcal{L}_{Cov}= \frac{1}{d^2}\sum\limits_{p, q \in M\times M, p>q}||[\mathbf{O}^{(p)}-\overline{\mathbf{O}}^{(p)}]^T[\mathbf{O}^{(q)}-\overline{\mathbf{O}}^{(q)}]||_1\]

Publications

(* Equal Contribution)

  1. Ads Recommendation in a Collapsed and Entangled World
    Junwei Pan, Wei Xue, Ximei Wang, Haibin Yu, Xun Liu, Shijie Quan, Xueming Qiu, Dapeng Liu, Lei Xiao, Jie Jiang
    ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD) , 2024 [arxiv] [slides] [code] [poster]

  2. Understanding the Ranking Loss for Recommendation with Sparse User Feedback
    Zhutian Lin*, Junwei Pan*, Shangyu Zhang, Ximei Wang, Xi Xiao, Shudong Huang, Lei Xiao, Jie Jiang
    ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD) , 2024 [arxiv] [slides] [code] [poster] \[\nabla_{z_j^{(-)}} \mathcal{L}_\text{Rank} = \frac{1}{N_+} \sum_{i=1}^{N_+} \sigma(z_j^{(-)}-z_i^{(+)}) > \frac{1}{N_+}\cdot N_+\cdot \sigma(z_j^{(-)}) = \sigma(z_j^{(-)}) = \nabla_{z_j^{(-)}} \mathcal{L}_\text{BCE}\]

  3. On the Embedding Collapse when Scaling up Recommendation Models
    Xingzhuo Guo, Junwei Pan, Ximei Wang, Baixu Chen, Jie Jiang, Mingsheng Long
    International Conference on Machine Learning (ICML) , 2024 [arXiv] [slides] [code] [poster] \[\textbf{e}_i =\textbf{E}_i^{\top}\textbf{1}_{x_i},\ \forall i\in\{1,2,...,N\}; h=I(\textbf{e}_1,\textbf{e}_2,...,\textbf{e}_n); \hat{y}=F(h)\]

  4. Deep Pattern Network for Click-Through Rate Prediction
    Hengyu Zhang, Junwei Pan, Dapeng Liu, Jie Jiang and Xiu Li
    ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) , 2024 [arXiv]

  5. Temporal Interest Network for User Response Prediction
    Haolin Zhou*, Junwei Pan*, Xinyi Zhou, Xihua Chen, Jie Jiang, Xiaofeng Gao, Guihai Chen
    World Wide Web Conference (WWW), 2024 [arXiv] (Oral) [slides] [code] [poster] \[\textbf{u} = \sum_{i \in \mathcal{H}} \alpha(\tilde{\textbf{e}}_{i}, \tilde{\textbf{v}}_{t}) \cdot (\tilde{\textbf{e}}_{i} \odot \tilde{\textbf{v}}_t)\]

  6. Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning
    Ximei Wang, Junwei Pan, Xingzhuo Guo, Dapeng Liu, Jie Jiang
    AAAI Conference on Artificial Intelligence (AAAI), 2024 [arXiv] [poster]

  7. STEM: Unleashing the Power of Embeddings for Multi-task Recommendation
    Liangcai Su*, Junwei Pan*, Ximei Wang, Xi Xiao, Shijie Quan, Xihua Chen, Jie Jiang
    AAAI Conference on Artificial Intelligence (AAAI), 2024 [arXiv] [code] [poster] \[\textbf{o}^{t} = \sum_{i}^{K_2} \textbf{g}^{S\rightarrow{t}}_{i} \textbf{h}^{S}_{i}(\textbf{E}^S) + \sum_{i}^{K_1} \textbf{g}^{t\rightarrow{t}}_{i} \textbf{h}^{t}_{i}(\textbf{E}^t) + \sum_{t' \in {\mathcal{T}}, t'\neq{t}}\sum_{i}^{K_1} \textbf{g}^{t'\rightarrow{t}}_{i}\texttt{SG}(\textbf{h}^{t'}_i(\textbf{E}^{t'}))\]

  8. Multi-Sequence Attentive User Representation Learning for Side-information Integrated Sequential Recommendation
    Xiaolin Lin, Jinwei Luo, Junwei Pan, Weike Pan, Zhong Ming, Xun Liu, Shudong Huang, Jie Jiang
    ACM International Conference on Web Search and Data Mining (WSDM), 2024 [PDF]

  9. AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning
    Enneng Yang*, Junwei Pan*, Ximei Wang, Haibin Yu, Li Shen, Xihua Chen, Lei Xiao, Jie Jiang, Guibing Guo
    AAAI Conference on Artificial Intelligence (AAAI), 2023 [PDF] (Oral) \[\theta_{t+1,i} = \theta_{t,i} - \sum_k \frac{\eta}{\sqrt{ {G}_{t,i}^k} + \epsilon} {g}_{t,i}^k, G_{t,i}^k = G_{t-1,i}^k \!+\! {(g_{t,i}^k)}^2 \]

  10. ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning
    Junguang Jiang, Baixu Chen, Junwei Pan, Ximei Wang, Liu Dapeng, Jie Jiang, Mingsheng Long
    Neural Information Processing Systems (NeurIPS), 2023 [arXiv]

  11. AutoAttention: Automatic Field Pair Selection for Attention in User Behavior Modeling
    Zuowu Zheng, Xiaofeng Gao, Junwei Pan, Qi Luo, Guihai Chen, Dapeng Liu, Jie Jiang
    IEEE International Conference on Data Mining (ICDM), 2022 [arXiv] \[a(\textbf{v}_i, \textbf{e}_{F_{1:M}}) = \sigma\left(b + \sum_{p=1}^P \sum_{j=1}^M \langle \textbf{v}_{B_p}, \textbf{e}_{F_j} \rangle r_{B_p,F_j}\right)\]

  12. Cross-task Knowledge Distillation in Multi-task Recommendation
    Chenxiao Yang, Junwei Pan, Xiaofeng Gao, Tingyu Jiang, Dapeng Liu, Guihai Chen
    AAAI Conference on Artificial Intelligence (AAAI), 2022 [arXiv]

  13. Trading Hard Negatives and True Negatives: A Debiased Contrastive Collaborative Filtering Approach
    Chenxiao Yang, Qitian Wu, Jipeng Jin, Xiaofeng Gao, Junwei Pan, Guihai Chen
    International Joint Conference on Artificial Intelligence (IJCAI), 2022 [arXiv]

  14. Follow the Prophet: Accurate Online Conversion Rate Prediction in the Face of Delayed Feedback
    Haoming Li, Feiyang Pan, Xiang Ao, Zhao Yang, Min Lu, Junwei Pan, Dapeng Liu, Lei Xiao, Qing He
    ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR, short) , 2021 [PDF]

  15. Impression Allocation and Policy Search in Display Advertising
    Di Wu, Cheng Chen, Xiujun Chen, Junwei Pan, Xun Yang, Qing Tan, Jian Xu, Kuang-Chih Lee
    IEEE International Conference on Data Mining (ICDM) , 2021 [arXiv]

  16. A Unified Solution to Constrained Bidding in Online Display Advertising
    Yue He, Xiujun Chen, Di Wu, Junwei Pan, Qing Tan, Chuan Yu, Jian Xu, Xiaoqiang Zhu
    ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD) , 2021 [PDF]

  17. DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving
    Wei Deng*, Junwei Pan*, Tian Zhou, Deguang Kong, Aaron Flores, Guang Lin
    ACM International Conference on Web Search and Data Mining (WSDM) , 2021 [arXiv]

  18. FM^2: Field-matrixed Factorization Machines for CTR Prediction
    Yang Sun, Junwei Pan, Alex Zhang, Aaron Flores
    World Wide Web Conference (WWW) , 2021 [arXiv] \[ \sum_i \sum_j x_i x_j \langle \textbf{v}_i M_{F(i), F(j)}, \textbf{v}_j \rangle \]

  19. Optimizing Multiple Performance Metrics with Deep GSP Auctions for E-commerce Advertising
    Zhilin Zhang, Xiangyu Liu, Zhenzhe Zheng, Chenrui Zhang, Miao Xu, Junwei Pan, Chuan Yu, Fan Wu, Jian Xu, Kun Gai
    ACM International Conference on Web Search and Data Mining (WSDM), 2021 [PDF]

  20. An Efficient Deep Distribution Network for Bid Shading in First-price Auctions
    Tian Zhou, Hao He, Shengjun Pan, Niklas Karlsson, Bharatbhushan Shetty, Brendan Kitts, Djordje Gligorijevic, San Gultekin, Tingyu Mao, Junwei Pan, Jianlong Zhang, Aaron Flores
    ACM SIGKDD International Conference on Knowledge Discovery & Data mining (KDD), 2021 [PDF]

  21. Bid Shading by Win-Rate Estimation and Surplus Maximization
    Shengjun Pan, Brendan Kitts, Tian Zhou, Hao He, Bharatbhushan Shetty, Aaron Flores, Djordje Gligorijevic, Junwei Pan, Tingyu Mao, San Gultekin, Jianlong Zhang
    ACM SIGKDD International Conference on Knowledge Discovery & Data mining (AdKDD workshop in KDD), 2020 [arXiv]

  22. Bid Shading in The Brave New World of First-Price Auctions
    Djordje Gligorijevic, Tian Zhou, Bharatbhushan Shetty, Brendan Kitts, Shengjun Pan, Junwei Pan, Aaron Flores
    IEEE International Conference on Data Mining (ICDM), 2020 [PDF]

  23. Predicting Different Types of Conversions with Multi-task Learning in Online Advertising
    Junwei Pan, Yizhi Mao, Alfonso Lobos Ruiz, Yu Sun, Aaron Flores
    ACM SIGKDD International Conference on Knowledge Discovery & Data mining (KDD), 2019 [PDF] \[ \sum_i \sum_j x_i x_j \langle \textbf{v}_i, \textbf{v}_j \rangle r_{F(i), F(j)}^t \]

  24. A Batched Multi-armed Bandit Approach to News Headline Testing
    Yizhi Mao, Miao Chen, Abhinav Wagle, Junwei Pan, Michael Natkovich, Don Matheson
    IEEE International Conference on Big Data (Big Data ), 2018 [arXiv]

  25. Field-weighted Factorization Machines for Click-through Rate Prediction in Display Advertising
    Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, Quan Lu
    World Wide Web Conference (WWW), 2018 [PDF] (Oral) [slides] \[ \sum_i \sum_j x_i x_j \langle \textbf{v}_i, \textbf{v}_j \rangle r_{F(i), F(j)} \]

  26. A Practical Framework of Conversion Rate Prediction for Online Display Advertising
    Quan Lu, Shengjun Pan, Liang Wang, Junwei Pan, Fengdan Wan, Hongxia Yang
    ACM SIGKDD International Conference on Knowledge Discovery & Data mining (AdKDD workshop in KDD) , 2017 [PDF]

Awards