Photo of Zhang, Xinhua

Xinhua Zhang

Assistant Professor

Department of Computer Science

Contact

Building & Room:

Daley Library 3-190 G

Office Phone Voice:

(312) 413-2416

Related Sites:

About

Research Interests

My current research in machine learning focuses on convex models for learning predictive representation. Most existing machine learning methods infer representation from data in a way that is independent of its subsequent use, e.g. learning a predictive model. This is suboptimal. My research goal is to jointly infer latent representation and learn predictors for massive datasets by combining them into a single convex optimization problem. Convexity allows jointly optimal solutions to be found for these two tasks, and scale up efficiently to large application problems. To achieve this goal, my key strategies are: 1) find appropriate convex relaxations that retain the structure of the data, e.g. semi-definite relaxations; and 2) design efficient algorithms for optimization such as low-rank approximation.

I work on applications in pattern recognition, document analysis, image processing, and any prediction problem that is useful in life.

Biography

Prior to joining UIC in Nov 2015, I was a Senior Researcher at the Machine Learning Research Group of National ICT Australia (NICTA, now Data61). From April 2010 to September 2012, I was a post-doc working with Prof Dale Schuurmans at the Department of Computing ScienceUniversity of Alberta.  From March 2006 to October 2009, I was a NICTA-endorsed PhD student of the Research School of Computer ScienceAustralian National University (ANU), working with Prof SVN Vishwanathan and Prof Alex Smola.  I visited Prof SVN Vishwanathan at the Department of Statistics at Purdue University from February 2009 to March 2010.  From January 2004 to March 2006, I pursued my Master's degree (by research) under the supervision of Prof Wee Sun Lee at theDepartment of Computer ScienceNational University of Singapore (NUS). I received my B.E. from the Department of Computer Science and Engineering atShanghai Jiao Tong University in July 2003. My hometown is Shanghai.

Find Me

Code

1 Generalized Conditional Gradient (GCG)
[link]
GCG is an open source Matlab solver for gauge (norm) regularized problems, that are commonly used in sparse coding and compressive sensing.  Examples include matrix completion, dictionary learning, and structured sparse estimation.
2 Smoothing for Multivariate Scores (SMS)
[link]
 SMS is an open source, extensible and scalable convex solver for a number of machine learning problems cast in the form of regularized risk minimization problem. It is particularly advantageous for optimizing multivariate performance measure. The implementation is "extensible" because the (problem-specific) loss function modules are encapsulated with a common interface for the main optimizer. Thus it is very simple to incorporate solutions to new problems.
3 Convex Subspace Learning
[link]
 This Matlab package implements the convex subspace learning model proposed at NIPS'12 and AAAI'12. The optimization is based on alternating direction of multiplier method. Example applications include semi-supervised learning, image denoising, and multi-label learning.
4  Bayesian Online Multilabel Classification (BOMC)
[link]
BOMC is an open source toolkit for online multilabel classification using Bayesian models. It is implemented in F# 1.9.3.4 on Microsoft Visual Studio 2008, and can be compiled and run on Linux systems via Mono. The graphical model is extend from TrueSkillTM [2] to deal with multilabel, and the inference engine is expectation propagation.
5 Conditional Random Fields for Policy Gradient Multi-agent Reinforcement Learning
[tar.bz2  700 KB] [paper]
This package implements the tree sampling for inference in conditional random fields.  With the sampled states and approximate expectations, the package implements the natural actor-critic which performs collaborative multi-agent reinforcement learning.  Three simulators are provided namely grid gate control, sensor network, and traffic light control.
6 Faster Rates for Training SVMs using Optimal Gradient based Methods
[tar.bz2  800 KB] [paper]
This package implements the three versions of Nesterov's first-order methods proposed in 19832005and 2007.  Its rate of convergence is O(1/k^2), which is proved to be optimal in this class of optimizers.  The 1983 version optimizes a smooth function with Lipschitz continuous gradient.  The 2005 version extends to the primal-dual setting, and the 2007 version can automatically estimate the unknown Lipschitz constant of the gradient.This code is built upon the package BMRM.
7 Hyperparameter Learning for Graph based Semi-supervised Learning Algorithms
[tar  100 KB] [paper]
This package implements the leave-one-out method for learning the hyperparameters in graph based semi-supervised learning.  Practical efficiency is achieved via the Sherman–Morrison formula and by facting out the common terms in feature weight updates.This code relies on the math library of Matlab.  See this link for details.

Teaching


University of Illinois at Chicago
Fall 2018 CS412 Introduction to Machine Learning  (Syllabus)
Spring 2018 CS594 Advanced Machine Learning  (Syllabus)
Fall 2017 CS412 Introduction to Machine Learning  (Syllabus)
Spring 2017 CS411 Artificial Intelligence I  (Syllabus)
Fall 2016 CS594 Advanced Topics in Machine Learning  (Syllabus)
Project of training CRFs using TAO and Torch  [.PDF.zip]
Spring 2016 CS411 Artificial Intelligence I  (Syllabus)
 

Australian National University

2015 COMP2610/6261 Information Theory
co-taught with Aditya Menon and Mark Reid
2014 &
2013
COMP4680/8650 Advanced Topics in Statistical Machine Learning
co-taught with Stephen Gould and Justin Domke
My lecture notes on Bregman divergence and mirror descent are here.


Purdue University
 
2009 Guest lecture at STAT 598Y (Statistical Learning Theorey)
My lecture notes on accelerated gradient methods are here.

Selected Publications

Refereed Journal Papers

1 Yaoliang YuXinhua ZhangDale Schuurmans

Generalized Conditional Gradient for Sparse Estimation

Journal of Machine Learning Research (JMLR)

Vol 18(144):1−46, 2017. [PDF] [Longer version with non-convexity] [Code]

2 Xinhua ZhangAnkan SahaS. V. N. Vishwanathan

Accelerated Training of Max-Margin Markov Networks with Kernels

Journal of Theoretical Computer Science (TCS)

Vol 519, pages 88–102, January 2014. [PDF]

3 Xinhua ZhangAnkan SahaS. V. N. Vishwanathan

Smoothing Multivariate Performance Measures

Journal of Machine Learning Research (JMLR)

Vol 13, pages 3589–3646, December, 2012. [PDF] [Code]

4

Xiang Yan, Xinhua Zhang, and Liang Huang

Computational Analysis and Optimization of the Integrity Distribution

Journal of Engineering Mathematics, 20(5), 2003. (in Chinese)   [link]

Refereed Conference Papers

.

Yingyi Ma, Vignesh Ganapathiraman, Xinhua Zhang

Learning Invariant Representations with Kernel Warping

International Conference on Artificial Intelligence and Statistics (AISTATS), 2019. [PDF]

.

Rizal Fathony, Ashkan Rezaei, Mohammad Ali Bashiri, Xinhua Zhang, and Brian Ziebart

Distributionally Robust Graphical Models

Advances in Neural Information Processing Systems (NIPS), 2018. [PDF]

.

Vignesh Ganapathiraman, Zhan Shi, Xinhua Zhang, and Yaoliang Yu

Inductive Two-Layer Modeling with Parametric Bregman Transfer

International Conference on Machine Learning (ICML), 2018. [PDF]

.

Rizal Fathony*, Sima Behpour*, Xinhua Zhang, and Brian Ziebart (*equal contribution)

Efficient and Consistent Adversarial Bipartite Matching

International Conference on Machine Learning (ICML), 2018. [PDF]

.

Mohammad Ali Bashiri and Xinhua Zhang

Decomposition-Invariant Conditional Gradient for General Polytopes with Line Search

Advances in Neural Information Processing Systems (NIPS), 2017. [PDF]

.

Zhan Shi, Xinhua ZhangYaoliang Yu

Bregman Divergence for Stochastic Variance Reduction: Saddle-Point and Adversarial Prediction

Advances in Neural Information Processing Systems (NIPS), 2017. [Spotlight, PDF]

.

Shin Matsushima, Hyokun Yun, Xinhua ZhangS.V.N. Vishwanathan

Distributed Stochastic Optimization of the Regularized Risk via Saddle-point Problem

European Conference on Machine Learning (ECML), 2017. [PDF]

.

Vignesh Ganapathiraman, Xinhua ZhangYaoliang YuJunfeng Wen

Convex Two-Layer Modeling with Latent Structure

Advances in Neural Information Processing Systems (NIPS), 2016. [PDF]

.

Hao ChengYaoliang YuXinhua ZhangEric XingDale Schuurmans

Scalable and Sound Low-Rank Tensor Learning

International Conference on Artificial Intelligence and Statistics (AISTATS), 2016. [PDF]

.

Parameswaran Kamalaruban, Robert C WilliamsonXinhua Zhang

Exp-Concavity of Proper Composite Losses

Conference on Learning Theory (COLT), 2015. [PDF]

.

Ozlem AslanXinhua ZhangDale Schuurmans

Convex Deep Learning via Normalized Kernels

Advances in Neural Information Processing Systems (NIPS), 2014. [PDF]

.

Changyou ChenJun ZhuXinhua Zhang

Robust Bayesian Max-Margin Clustering

Advances in Neural Information Processing Systems (NIPS), 2014. [PDF] [Appendix]

.

Hengshuai YaoCsaba SzepesvariBernardo Avila Pires,Xinhua Zhang

Pseudo-MDPs and Factored Linear Action Models

Symposium on Adaptive Dynamic Programming and Reinforcement Learning (IEEE ADPRL), 2014. [PDF]

.

Xianghang Liu, Xinhua ZhangTiberio Caetano

Bayesian Models for Structured Sparse Estimation via Set Cover Prior

European Conference on Machine Learning (ECML), 2014. [PDF] [Long]

.

Xinhua ZhangWee Sun Lee, Yee Whye Teh

Learning with Invariance via Linear Functionals on Reproducing Kernel Hilbert Space

Advances in Neural Information Processing Systems (NIPS), 2013. [PDF]

.

Xinhua ZhangYaoliang Yu, Dale Schuurmans

Polar Operators for Structured Sparse Estimation

Advances in Neural Information Processing Systems (NIPS), 2013. [PDF]

.

Ozlem AslanHao Cheng, Dale SchuurmansXinhua Zhang

Convex Two-Layer Modeling

Advances in Neural Information Processing Systems (NIPS), 2013. [PDF]

.

Hao ChengXinhua ZhangDale Schuurmans

Convex Relaxations of Bregman Divergence Clustering

Uncertainty in Artificial Intelligence (UAI), 2013. [PDF]

.

Yi ShiXinhua Zhang, Xiaoping Liao, Guohui Lin, andDale Schuurmans

Protein-chemical Interaction Prediction via Kernelized Sparse Learning SVM

Pacific Symposium on Biocomputing (PSB), 2013. [PDF]

.

Xinhua ZhangYaoliang Yu, and Dale Schuurmans

Accelerated Training for Matrix-norm Regularization: A Boosting Approach

Advances in Neural Information Processing Systems (NIPS), 2012. [PDF] [Code]

.

Martha WhiteYaoliang YuXinhua Zhang, and Dale Schuurmans

Convex Multi-view Subspace Learning

Advances in Neural Information Processing Systems (NIPS), 2012. [PDF]

.

Yi Shi, Xiaoping Liao, Xinhua ZhangGuohui Lin, andDale Schuurmans

Sparse Learning based Linear Coherent Bi-clustering

Workshop on Algorithms in Bioinformatics (WABI), 2012.

Lecture Notes in Bioinformatics 7534, 346-364. [PDF]

.

Yaoliang YuJames Neufeld, Ryan Kiros, Xinhua Zhang, and Dale Schuurmans

Regularizers versus Losses for Nonlinear Dimensionality Reduction

International Conference on Machine Learning (ICML), 2012.  [PDF] [Supplementary]

.

Xinhua ZhangAnkan SahaS. V. N. Vishwanathan

Accelerated Training of Max-Margin Markov Networks with Kernels

Algorithmic Learning Theory (ALT), 2011.  [PDF] [Talk]

.

Xinhua ZhangAnkan SahaS. V. N. Vishwanathan

Smoothing Multivariate Performance Measures

Uncertainty in Artificial Intelligence (UAI), 2011.  [PDF] [Long] [code]

.

Xinhua ZhangYaoliang YuMartha White, Ruitong Huang, and Dale Schuurmans

Convex Sparse Coding, Subspace Learning, and Semi-supervised Extensions

AAAI Conference on Artificial Intelligence (AAAI), 2011. [PDF]

.

Ankan SahaS. V. N. VishwanathanXinhua Zhang

New Approximation Algorithms for Minimum Enclosing Convex Shapes

ACM-SIAM Syposium on Discrete Algorithms (SODA), 2011. [PDF]

.

Xinhua ZhangAnkan SahaS. V. N. Vishwanathan

Lower Bounds on Rate of Convergence of Cutting Plane Methods

Advances in Neural Information Processing Systems (NIPS), 2010.

[PDF] [Long] [Detail on Nesterov (arXiv)] [Formalization of weak/strong lower bounds]

.

Xinhua ZhangThore GraepelRalf Herbrich

Bayesian Online Learning for Multi-label and Multi-variate Performance Measures

International Conference on Artificial Intelligence and Statistics, (AISTATS) 2010. [PDF]

.

Xinhua ZhangLe SongArthur GrettonAlex Smola

Kernel Measures of Independence for non-iid Data

Advances in Neural Information Processing Systems (NIPS), 2008. [PDF]  [Appendix] [Spotlight]

.

Le SongXinhua ZhangAlex SmolaArthur Gretton, and Bernhard Schoelkopf

Tailoring Density Estimation via Reproducing Kernel Moment Matching

International Conference on Machine Learning (ICML), 2008.  [PDF]

.

Li ChengS. V. N. Vishwanathan, and Xinhua Zhang

Consistent Image Analogies using Semi-supervised Learning

IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2008.  [PDF]

.

Xinhua ZhangDouglas Aberdeen, and S. V. N. Vishwanathan

Conditional Random Fields for Multi-agent Reinforcement Learning

International Conference on Machine Learning (ICML), 2007.  [PDF]

(Best student paper award)

.

Xinhua Zhang and Wee Sun Lee

Hyperparameter Learning for Graph based Semi-supervised Learning Algorithms

Advances in Neural Information Processing Systems (NIPS), 2006. [PDF]

.

Xinhua Zhang and Peter K K Loh

A Fault-tolerant Routing Strategy for Fibonacci-class Cubes

Asia-Pacific Computer Systems Architecture Conference (ACSAC), 2005.  [PDF]

Refereed Workshop Oral Presentations

1 Xinhua ZhangDouglas Aberdeen, and S. V. N. Vishwanathan

Conditional Random Fields for Multi-agent Reinforcement Learning

Learning Workshop (Snowbird), 2007.  [PDF]

2 Peter K K Loh and Xinhua Zhang

A Fault-tolerant Routing Strategy for Gaussian Cube using Gaussian Tree

International Conference on Parallel Processing (ICPP) Workshops, 2003.  [PDF]

Book Chapters

1 Xinhua Zhang

Seven articles: Support vector machines, kernel, regularization, empirical risk minimization, structural risk minimization, covariance matrix, Gaussian distribution.

In Claude Sammut and Geoffrey Webb, editors

Encyclopedia on Machine Learning. Springer, 2010.

Technical Reports

1 Xinhua ZhangAnkan SahaS. V. N. Vishwanathan

Regularized risk minimization by Nesterov’s accelerated gradient methods: Algorithmic extensions and empirical studies

http://arxiv.org/abs/1011.0472, 2011.  [PDF]

Theses
PhD Thesis (Australian National University)
Graphical Models: Modeling, Optimization, and Hilbert Space Embedding [PDF, 3.5 MB]
MSc Thesis (National University of Singapore)
Hyperparameter Learning for Graph Based Semi-supervised Learning Algorithms  [PDF]
Undergraduate Final Year Project (Nanyang Technological University)
Analysis of Fuzzy-Neuro Network Communications  [PDF]

Notable Honors

2015, Outstanding Program Committee member, AAAI

2007, Best student paper award, International Conference on Machine Learning

Education

Ph.D., Australian National University, 2010