返回
类型 应用研究 预答辩日期 2017-12-28
开始(开题)日期 2015-04-03 论文结束日期 2017-10-09
地点 东南大学中心楼2楼教育部重点实验室 论文选题来源 国家自然科学基金项目     论文字数 6.21 (万字)
题目 表情变化下三维人脸识别中特征提取与分类研究
主题词 三维人脸识别,表情变化,特征融合,局部协方差描述子,多尺度融合
摘要 三维人脸识别技术能够充分利用空间几何信息,可以克服二维人脸识别在光照、化妆和姿态等方面的局限,因此,受到越来越多研究人员的青睐。三维人脸本质上是一类非刚性曲面,其表情变化会导致局部区域产生非刚性形变,特别是包含嘴部区域的下半张人脸,这将影响基于形状匹配的三维人脸识别算法的性能。因此,本文对非刚性自由曲面的形变进行分析,研究了表情变化下三维人脸识别中的特征提取与分类问题。主要研究成果和创新性工作如下: (1)提出特征级融合和区域特征融合的方法,以提取能表征三维人脸的高区分度特征。为了减小受表情影响较大的非刚性区域对三维人脸识别性能带来的影响,并保持三维人脸曲面的拓扑完整性,本文将三维人脸分割成半刚性区域和非刚性区域,采用连续点漂移算法对非刚性区域进行非刚性点集配准;然后对低频和垂直高频特征进行特征级融合,并对半刚性区域与经过非刚性点集配准之后的非刚性区域进行区域特征融合。该方法基于低频和垂直高频的Haar小波特征的特征级融合技术,能将具有判别性的特征融合在一起,充分发挥各自的优势。所提出的区域特征融合技术可有效地利用三维人脸的全局信息。 (2)提出自适应特征选择下表情鲁棒的三维人脸识别方法,以更准确地利用中性和带表情人脸之间重建残差的差异性。为了避免对所有测试人脸的非刚性区域都进行统一处理,并充分利用人脸的几何结构,以得到更多准确的区域细节信息,本文提出基于特征点的不规则区域人脸表示;然后提取多尺度融合的塔形局部二值模式;最后基于精确定位的特征点和重建残差,提出自适应选择特征用于最后的分类识别。所提方法可以自适应地去除表情变化下对人脸形状扭曲非常大的区域,以消除人脸表情对识别率的影响。 (3)提出直接在三维人脸网格上提取局部协方差描述子并采用黎曼核稀疏分类进行识别,以避免复杂的配准和阈值估计。首先,利用最远点采样方法检测人脸关键点;其次,提取关键点邻域不同类型的有效特征,建立具有内在属性的局部协方差描述子;最后,利用平均稀疏重建残差的相似性度量来降低类内差异的同时增加类间差异,适当的黎曼核稀疏表示分类用于最后的识别。该方法能快速有效地融合不同的人脸曲面特征,较好地表征曲面的内在属性。 (4)提出基于多尺度局部协方差描述子与局部敏感黎曼核稀疏分类的三维人脸识别,以充分利用人脸的多尺度信息及局部协方差描述子的局部性。首先,采用无限特征选择方法来选择特征空间中“贡献率”较大的特征,用以构建局部协方差描述子,同时根据特征值的均方根误差进行尺度选择;其次,提取关键点邻域的局部协方差描述子,并对这些描述子进行多尺度融合;最后,提出局部敏感的黎曼核稀疏分类方法进行三维人脸识别。该方法利用连续变化的尺度参数获得不同尺度下的局部协方差描述子,能有效提高单一尺度局部协方差描述子的表述能力;局部敏感黎曼核稀疏分类可有效地利用多尺度描述子的局部性。 在公开的三维人脸数据库FRGC v2.0和Bosphorus上分别设计实验对本文所提方法进行评估,并与一些先进方法的结果进行比较分析。实验结果证明了本文算法的有效性,为三维人脸识别更好地走向实际应用奠定了基础。
英文题目 RESEARCH ON 3D FACE RECOGNITION UNDER EXPRESSION VARIATIONS BASED ON FEATURE EXTRACTION AND CLASSIFICATION
英文主题词 3D face recognition, Expression variations, Feature fusion, Local covariance descriptor, Multi-scale fusion
英文摘要 Three-dimensional (3D) face recognition technology can make full use of spatial geometric information, and can get around the limitations of two-dimensional (2D) face recognition in light, makeup and gestures, so it has attracted extensive attention from more and more researchers. 3D face is essentially a kind of non-rigid surfaces, whose expression variations can cause non-rigid deformations in the local region, especially for the lower half of the face, which contains the mouth area. This will affect the performance of 3D face recognition algorithms based on shape matching. Therefore, this paper analyzes the deformation of non-rigid free-form surface, and deeply studies the feature extraction and classification in 3D face recognition under expression variations. The main research results and innovative work are given as follows: (1) A method of feature-level fusion and feature-region fusion is proposed to extract the high discriminative features of 3D face. In order to reduce the adverse effect from the large facial expressions in the non-rigid regions as well as to maintain the topological integrity of 3D face surfaces, 3D face is divided into semi-rigid region and non-rigid region, and the non-rigid point set registration is performed for non-rigid region using Coherent Point Drift (CPD) algorithm; then the feature-level fusion of low frequency and vertical high frequency features is carried out, and the semi-rigid region feature is combined with the non-rigid region feature after non-rigid point set registration for feature-region fusion. Feature-level fusion based on low-frequency and vertical high-frequency Haar wavelet feature is combined with the distinctive features to give full play to their strengths. The proposed feature-region fusion technology can effectively use the global information. (2) An adaptive feature selection method for expression-robust 3D face recognition is proposed. The proposed approach can more accurately use the difference of reconstruction residuals between the neutral and expression face. In order to avoid that all the non-rigid regions of the probe face are uniform treated, and to make full use of the geometry of the face, an irregular landmark-based patch facial representation based on the located facial landmarks is proposed; then, the multi-scale fusion of the pyramid local binary patterns (FPLBP) is proposed; finally, an adaptive feature selection method is proposed for the final classification and recognition based on reconstruction residual and accurately located landmarks. The proposed method can adaptively eliminate the facial area where the facial shape will distorts largely under expression variations to reduce the effect of the facial expression. (3) In order to avoid the complicated registration and threshold estimation, a 3D face recognition method is proposed based on local covariance descriptor directly extracted on the 3D face mesh and Riemannian kernel sparse representation-based classifier. Firstly, the keypoints are detected by the farthest point sampling method; secondly, different types of the efficient features are extracted to construct the local covariance descriptor with intrinsic property; finally, the similarity measure of the average sparse reconstruction residual is used to reduce intraclass differences while increasing interclass differences, and the appropriate Riemannian kernel sparse representation-based classifier is used for the final recognition. This method can quickly and effectively fuse different facial surface features to characterize the intrinsic property of the surface. (4) In order to make full use of the multi-scale information of 3D face as well as the locality of local covariance descriptor, a 3D face recognition method based on multi-scale local covariance descriptor and local-sensitive Riemannian kernel sparse representation-based classifier is proposed. Firstly, the infinite feature selection method is used to select the feature with larger ‘scores’ in the feature space to construct the local covariance descriptor, and the scale selection is based on the Root Mean Square Error (RMSE) of the eigenvalue; secondly, the local covariance descriptors of the keypoints’ neighborhood is extracted, and multi-scale fusion of these descriptors is performed; finally, 3D face recognition is performed by local-sensitive Riemannian kernel sparse representation-based classifier. In this method, the local covariance descriptor at different scales can be obtained by continuously changing the scale parameters, which can effectively improve the representation ability of local covariance descriptor under the single scale. The local-sensitive Riemannian kernel sparse representation-based classifier can utilize the localization of the multi-scale descriptor. The methods proposed in this paper are evaluated and compared with the results of some state-of-the-art approaches based on the public FRGC v2.0 and Bosphorus 3D face database. The experimental results prove the effectiveness of our algorithms, which lays the foundation for the better application of 3D face recognition.
学术讨论
主办单位时间地点报告人报告主题
东南大学自动化学院 2014-04-25 科研楼320 邓星 基于分块的Mean Shift跟踪的研究报告
东南大学自动化学院 2017-07-04 科研楼320 邓星 Locality-constrained Riemannian Kernel Sparse Representation Classification for 3D Face Recognition
东南大学自动化学院 2016-10-19 中心楼二楼自动化学院教育部重点实验室会议室 王雁刚 Video-based Motion Capture
东南大学自动化学院 2014-11-28 科研楼320 邓星 表情鲁棒的三维人脸识别方法
东南大学自动化学院 2015-04-02 科研楼320 邓星 表情变化下的三维人脸识别若干关键问题研究
东南大学自动化学院 2016-10-30 科研楼320 邓星 基于局部协方差描述子和黎曼核稀疏编码的三维人脸识别方法
东南大学自动化学院 2016-12-01 中心楼二楼自动化学院教育部重点实验室会议室 Xu Richard教授 Deep Learning Fundamentals
东南大学自动化学院 2016-12-08 礼东101 欧阳万里教授 Modeling Deep Structures with Application to Object Detection and Pose Estimation
东南大学自动化学院 2017-04-26 中心楼二楼自动化学院教育部重点实验室会议室 Cao Dingcai教授 ipRGCs视觉信号处理和视觉功能研究
东南大学 自动化学院 2017-07-23 中心楼二楼自动化学院教育部重点实验室会议室 凌海滨教授 增强现实和计算机视觉——发展、协作、和探索
     
学术会议
会议名称时间地点本人报告本人报告题目
IEEE International Conference on Image Processing (ICIP) 2015-09-29 加拿大魁北克 A 3D face recognition method using region-based extended local binary pattern
International Conference on Image and Graphics 2015-08-13 中国天津 An Automatic Landmark Localization Method for 2D and 3D Face
     
代表作
论文名称
Expression-robust 3D Face Recognition using Region-based Multiscale Wavelet Feature Fusion
Expression-robust 3D face recognition based on feature-level fusion and feature-region fusion
基于自适应人脸切割的三维人脸识别算法
Adaptive feature selection based on reconstruction residual and accurately located landmarks for exp
 
答辩委员会组成信息
姓名职称导师类别工作单位是否主席备注
杨静宇 正高 教授 博导 南京理工大学计算机科学与工程学院
曹云峰 正高 教授 博导 南京航空航天大学自动化学院
费树岷 正高 教授 博导 东南大学自动化学院
袁晓辉 正高 教授 博导 东南大学自动化学院
盖绍彦 副高 副教授 博导 东南大学自动化学院
      
答辩秘书信息
姓名职称工作单位备注
王辰星 副高 副教授 东南大学自动化学院