You're facing setbacks in data collection. How can you ensure your mining results remain top-notch?

Welcome to reality. Nothing ever goes as planned, and this is as true in data mining as it is everywhere else. Data quality is especially challenging in data mining, as it's common to use data from several different sources, none of which were developed with your project in mind. It's especially important to understand the sources of your data, and the variations in units and coding. (NASA crashed its Mars Climate Orbiter in 1999 for this very reason.) Date and time encoding are particularly challenging, given regional variations, so pay special attention to those. And double-check the levels of observation: is the data individual level, group level, country level, and so on. All of these will affect your results.

I often develop dashboard to monitor the data capturing, data health and quality. I provide regularly feedbacks to take action to control the quality of data capturing. This will immensely help to find meaningful results from the data analytics.

Despite setbacks, keep refining data collection methods. Analyze gaps and adapt solutions, enlisting expertise. Maintain quality throughout by spot-checking work, setting standards rigorously followed by all. Don't compromise thoroughness - rework imperfect derivations. Persistence and team unity overcome challenges; top results stem from refusing to quit until task succeeds.

One of the great perks of data mining is that you will typically have very large, diverse datasets. The size of those datasets makes it easy to split the data into train-validation-test sets (or, depending on your source, train-test-validation), along with k-fold cross-validation in the training set. While more common in machine learning classification models, this multiple-subset approach is also useful in data mining, as it allows you to check the generalizability of any patterns you may find when data mining. With a large enough dataset, even patterns that exist for only relatively small subgroups in the data should show up reliably with this multiple split approach.

The data mining world is not static, and new methods will make different kinds of data potentially useful in your products. Even more importantly, data mining is typically performed in commercial settings where competitive advantage is critical. That means that once a method becomes common, it no longer provides the same competitive advantage that it did previously, and newer methods will likely need to be implemented. It's difficult, because there is an element of diminishing returns, but that's why it's important to keep learning. If possible, it helps to keep a journal of things that you've learned (with dates!) so that you can more easily identify code that you wrote before those eureka moments and revisit your work.