First make sure that simpler tools won’t work. (regression, Bayesian analysis, etc., etc.)
Deep Learning (DL) is not the answer to all problems.
Before you attempt to solve your problem using DL make sure that you are able to ask the right questions. If you can’t, then you won’t be able to tell if the “answers” are useful or even correct. Domain expertise matters.
Make sure that you preprocess your data. Clean it and normalize the values. Statistical analysis can help here. Remember GIGO.
DL can require very careful adjustments of learning parameters.
Remember that these systems offer no explanation for outputs.
Again, make sure that someone has sufficient domain knowledge to separate wheat from chaff in output. A black box system will not easily convince domain end users of its “expertise.”
How many layers should be used? What is more important is optimization of learning rate across layers.
Sometimes such systems can make very “dumb” errors that aren’t easily detectable or explainable, as the system seems to work fine across all tests. A good example was one of the early DOD attempts to create a system to analyze satellite images and recognize presence or absence of Russian tank formations in Eastern Europe. This required a simple binary output of TANKS or NO TANKS. During supervised training the system worked perfectly. When implemented with new real time images it failed. There was no explanatory capability built in, so humans had to examine the new input images to see where an error could have occurred. What jumped out to them, was the fact that all training images with tanks were taken on sunny days. All training images without tanks were taken on shady days. The system had learned to differentiate between images of sunny and cloudy days. This problem was easy for humans to diagnose based on the inputs and outputs being easy to read visual images being assigned to 2 categories. The diagnosis is not so simple with massive data sets.
How a system learns is important also. Early input examples used can influence learning. Supervised and unsupervised systems differ in learning speed and capability. Creation of local minima and maxima across layers can affect output and sometimes create systems that appear to be working correctly until they break.
For all the above reasons, it is hard to test DL systems. You can never be sure that a system will act properly as new data is being introduced. New values, ranges, limits, etc. could affect operation. Can you adequately test? What happens when data is introduced 100,000 times a second?

Computational performance, perturbations and sensitivity analysis yield deficiencies in deep learning methods.