A new expression recognition method is presented by combining image difference and improved convolutional deep belief network is proposed.Firstly, some pre-processing steps of images are used,such as cropping and intensity no rmalization.Then facial expressional details are obtained by calculating the dif ference between the images of basic expressions and neutral expression,which re flect the information irrelevant to identities.To extract the high-level expression features of the key regions in the expression image,the input visible layer of the convolutional restricted boltzman machine (CRBM) is divided into multiple region s,and the features are learned by partitioning.The improved CRBMs are stacked to form a partitioned co nvolutional deep belief network (PCDBN).Then the obtained difference expression images are used as the input of the partitioned convolutional deep belief network,the network is trained by improve d contrastive divergence algorithm layer by layer,and the softmax network is added to the top layer as th e output layer to form the classification surface.Finally,the partitioned convolutional deep belief netwo rk which is well-trained is used to identify the expression images from test samples.The experimental resul ts on JAFFE and CK+ expression database can achieve the recognition rate above 95% all.After expand ing the training samples, the recognition rate on CK+ expression database can reach more than 99%.