Stock prices

George had an exciting meeting with his new client.

A fund manager wants George’s team to design a machine learning model that evaluates the price of stocks. The model should use publicly available information to predict the true share price of each company.

That’s all the fund manager needs. He will take it from there and look deeper into under-priced stocks (where the true price is higher than the current market price.)

The model will act as a pre-filter to sort out potential companies, and the fund manager will do a more detailed analysis before deciding whether to invest in the stock. The fund manager doesn’t care if there are occasional wrong predictions, even by a large margin, but he doesn’t want to have too many false positives because detailed research will cost a lot of time.

George is trying to decide which loss function the team should use for the model.

Which of the following loss functions do you think are suitable for this problem?

1. Mean Squared Error (MSE)

2. Mean Absolute Error (MAE)

3. Binary Cross-entropy Loss

4. Huber Loss

Expand to see the answer

2, 4

We don’t have too much information about the problem, but the client has clear priorities regarding the errors they are willing to tolerate. Significant outliers are not a big problem because the fund manager will spot them quickly and reject them. However, they expect excellent overall performance to avoid wasting time researching unpromising stocks.

With this in mind, the Mean Absolute Error (MAE) and the Huber loss seem like better options than the Mean Squared Error (MSE.) The MSE penalizes high errors of the model much more than the others. This means that the model will try to learn to avoid big mistakes in the prediction, which will likely come at the cost of worse overall performance. We don’t want this.

On the other hand, the MAE and the Huber loss are more robust to outliers, and they will not dominate the training process. Therefore, the second and fourth choices are the correct answers.

The cross-entropy loss doesn’t apply to regression problems, so the third choice is also incorrect.

Recommended reading

• Check “Regression Metrics for Machine Learning” for an overview of some of the most popular metrics used for regression problems.
• “RMSE vs MAE, which should I use?” is a great summary by Stephen Allwright about the properties of these two functions and how you should think about them. For more information about the Huber loss, take a look at “Huber Loss: Why Is It, Like How It Is?”.