Alden Butzke

Hyperparameters can be set using the keras deep learning library. The learning rate controls how the model is updated based on error. "The learning rate is perhaps the most important hyperparameter. If you have time to tune only one hyperparameter, tune the learning rate." — Page 429, Deep Learning, 2016 The learning rate can be specified in the optimizer class of your model object, model.compile(... optimzier.Adam(...lr=.001) Model callbacks can also be defined with the EarlyStopping function from the keras.callbacks module. It requires a metric to monitor, such as validation loss, and a patience parameter, the number of epochs to wait before halting the training due to no improvement in the metric.

The optimal set of hyper-parameters, including learning rate, must be discovered through experimentation. A small learning rate means the model may never converge on the optimal set of weights, while a large learning rate may mean the optimal solution is missed.

The learning rate is used to effectively train a model by scaling the updates to the weights. It is known as a hyper-parameter, which are the parameters the model does not optimize during training. This means that an optimal set of hyper-parameters, including learning rate, must be discovered through experimentation. The error between the target variable and the prediction create a set of updates to each weight. The learning rate affects how much the model learns each iteration, or epoch, by scaling the update to each weight. A small learning rate means the model may never converge on the optimal set of weights, while a large learning rate may mean the optimal solution is missed.

Working a project like NativelyAI, teaching machines to speak Inupiaq, has forced me to go through so much growth. Every day spent is encountering unknown unknowns and figuring out how to learn the answers by reading through papers, open source projects, and Youtube. I've sought out mentors, connected with others pursuing similar projects, and realized how much I have to learn. Most importantly, no work gets done unless I do it, and I have become more resourceful, knowledgeable, and dedicated as a result.

Mentors in general can be career defining, and I'm lucky to have several incredible ones. As a mentee, I've learned that it is our responsibility to learn the necessary background knowledge, to follow up on ideas discussed, to ask informed questions, to work on projects, and many times to initiate the mentorship. The mentorship will only be as valuable as the work and time spent.

The best way to build relationships is to start a dedicated machine learning community in your organization. Three of the ways we do this are: 1. A slack channel that randomly pairs you with another person in AI/ML/Data for a coffee chat 2. A biweekly group discussion on a published paper in the industry 3. Centralized location of resources and points of contact