Start with a series of baseline models.
And then you use forward-search NAS techniques that can automatically generate neural networks.
Interesting? ๐งตโฌ๏ธ
And then you use forward-search NAS techniques that can automatically generate neural networks.
Interesting? ๐งตโฌ๏ธ
Project Petridish is a NAS algorithm that can produce neural networks for a given problem.
It was inspired by feature selection and gradient boosting techniques.
2/5
It was inspired by feature selection and gradient boosting techniques.
2/5
2 types of techniques for exploring the NAS space:
1. Backward-search methods are the most common approach. But they require human domain knowledge.
2. Forward methods do not need to specify a finite search space.
1. Backward-search methods are the most common approach. But they require human domain knowledge.
2. Forward methods do not need to specify a finite search space.
The Petridish algorithm works in three phases:
1) Petridish starts with some parent model
2) Petridish connects the candidate layers to the parent model using stop-gradient and stop-forward layers
1) Petridish starts with some parent model
2) Petridish connects the candidate layers to the parent model using stop-gradient and stop-forward layers
3) If a particular set of candidates is found to be beneficial to the model, we remove the stop-gradient and stop-forward layers. Then train the model to convergence.
Petridish is a remarkable milestone for a forward-search NAS technique.
Petridish is a remarkable milestone for a forward-search NAS technique.
Loading suggestions...