- Efficient Machine Learning and Model Compression
Deep learning is great for many tasks but very expensive in terms of raw storage and computation cost. Especially when considering edge platforms. In our group, we have been working on efficient deep learning compression from both algorithmic and hardware perspectives. Including but not limited to network architecture search (NAS), novel quantization/binarization scheme, pruning, knowledge distillation, low-rank factorization, model re-parameterization, novel floating-point number format, etc. Meanwhile, automated design space exploration for resource-constrained targeting platforms by algorithm/compiler/hardware level co-design is the main theme of our lab.
Related Publication:
- Alnemari, Mohammed, and Nader Bagherzadeh. “Efficient deep neural networks for edge computing.” 2019 IEEE International Conference on Edge Computing (EDGE). IEEE, 2019.
- Kim, HyunJin, Mohammed Alnemari, and Nader Bagherzadeh. “A storage-efficient ensemble classification using filter sharing on binarized convolutional neural networks.” PeerJ Computer Science 8 (2022): e924.
- Ye Qiao, Alnemari, Mohammed, and Nader Bagherzadeh. “A Two-Stage Efficient 3-D CNN Framework for EEG Based Emotion Recognition.” 23rd IEEE International Conference on Industrial Technology (ICIT). IEEE, 2022.
2. Process-in-Memory Architecture and Acceleration
Recently, memristive crossbar arrays have gained considerable attention from researchers to perform analog in-memory vector-matrix multiplications in machine learning accelerators, with low power and constant computational time. The low power consumption and in-memory computation abilities of crossbars arrays make it an attractive method of analog AI acceleration. However, crossbar arrays have many non-ideal characteristics such as memristor device imperfections, weight noise, device drift, input/output noises, and DAC/ADC overhead. Thus our current research in this field explores novel and state-of-the-art machine learning models and their performance on crossbar array-based accelerators. We also research novel architectures for crossbar array-based AI accelerators. To measure the performance of ML models on analog AI accelerators, we use simulation tools such as the IBM Analog Hardware Development Kit, DNN Neurosim, and/or Spice simulations.
Related Publication:
Ding, Andrew, Ye Qiao, and Nader Bagherzadeh. “BNN an Ideal Architecture for Acceleration With Resistive in Memory Computation.” IEEE Transactions on Emerging Topics in Computing (2023).
3. Alzheimer’s disease and medical image segmentation
The third major field of research we are working on is the medical image segmentation and Alzheimer’s disease Predicting with PET and MRI scanning using the deep learning approach. Early diagnosis of AD is important for patient care and the development of future treatment. We are working on a new classification framework based on the combination of 3-D CNN and transformer encoder to extract spatial and time-varying information from 4D PET images. Our model is an end-to-end data-driven model, which was more convenient for processing 4D PET data and including various efficiency optimization.
4. More Projects
Furthermore, some Undergraduate students in our lab are working on various other downstream machine learning applications that potentially can benefit from the aforementioned compression algorithm and hardware accelerator project, such as an efficient low-level vision model for image reconstruction, transformer encoder for stock price prediction, Multiple sclerosis early prediction, an edge drone navigation system, etc.