Brief Review — Andrew Ng, AI Minimalist: The Machine-Learning Pioneer Says Small is the New Big

Not Just About Data-Centric AI

I’ve taken many his AI courses in deeplearning.ai and was a mentor in one of his courses in Coursera. His courses really strengthen me a lot about the deep learning knowledge.
This time, I would like to share an article that I’ve read recently, from IEEE Spectrum Magazine in April 2022, namely “Andrew Ng, AI Minimalist: The Machine-Learning Pioneer Says Small is the New Big”
IEEE Spectrum Magazine is a monthly magazine, which talks about technology of all kinds. It has impact factor of 3.578.
In this article, Andrew Ng has shared a lot of his valuable visions and broad views about AI, e.g.: NLP, CV, semiconductor manufacturers, and his company, even from his first NeurIPS workshop paper, to recent NeurIPS data centric AI workshop!

He mentioned that, there are many research works developing the foundation model in NLP, like GPT-3.

It is time to have the foundation model for video despite there are practical difficulties of it.

Over the past years, Andrew has been promoting Data-Centric AI.
“For example, if you have 10,000 images where 30 images are of one class, and those 30 images are labeled inconsistently, … So you can very quickly relabel those images to be more consistent, and this leads to improvement in performance.” by Andrew Ng.

“Data-Centric AI is the discipline of systematically engineering the data needed to successfully build an AI system.”

He also mentioned an example, let say we got a model performance which is okay for most of the data set, but its performance is biased for just a subset of the data.
If we try to change the whole neural-network architecture to improve the performance on just that subset, it’s quite difficult.

We can engineer a subset of the data to address the problem in a much more targeted way.

Sometimes, it is crucial to IMPROVE THE DATA instead of improving the model architecture to boost the system performance.

(I’ve just shared a little bit of it. Please feel free to read the article directly if interested.)

2021 [CheXternal] [CheXtransfer] 2022 [Small is the New Big]