As someone who develops machine-learning models as job, I can say that there are snippets from Kaggle kernels that run in minutes, and outperform months of manual effort by a business analyst.
Winners have often developed really interesting feature engineering strategies for a domain, as well as very well organised tuning and stacking systems.
Maybe a lot of difference between 1st and 5th percentile is luck, but top end of Kaggle is a really valuable insight into building effective models, even if you might want to simplify implimentation a bit in the commercial world.
That's exactly what the author is trying to say, I think. We shouldn't put so much emphasis on who got into first place (which is mostly determined by luck) but rather investigate all techniques used by the top 5th or 10th or whatnot percentile, which is meaningfully separated from the rest.
Winners have often developed really interesting feature engineering strategies for a domain, as well as very well organised tuning and stacking systems.
Maybe a lot of difference between 1st and 5th percentile is luck, but top end of Kaggle is a really valuable insight into building effective models, even if you might want to simplify implimentation a bit in the commercial world.