In order to solve nonnegligible problems in brain tumor MRI segmentation, such as few samples, class imbalance and low accuracy of small districts, this essay proposes a new multi-scale and multitask deep-learning algorithm called TDDU-Net based on 3D No-New U-Net. Firstly, this paper applied the structure with an encoder and three different decoders to the network. Next, the ConvXt module is a custom-designed pre-processor set in front of the original decoders in order to overcome the underutilization of high-level semantics in some core region. Then, InConvXt is a generalized feature processing module set at the bottom layer between the encoder and decoder to ensure the accuracy of the generalized features and enhance the stability of the network. Finally, the deepwise convolution is used to reduce the amount of the network parameter at the appropriate location while ensuring accuracy. The experiments show that the dice similarity coefficients of the predicted segmentation in the BraTS18 dataset reaching 0.907, 0.847, 0.807 in the whole tumor region, the core tumor region and the enhanced tumor region. The method performs better and makes fewer mistakes than others, which is helpful in segmenting the smaller tumor area in MRI.