Apply transformation on entire dataset
Current status: Each transformation (e.g. standardisation) is applied for each station separately. So finally, the data for each station has mean of 0 and std=1. All information in station mean level and its spread is stored in the transformation information. This information is hidden for the NN model, because it works only on the transformatted data. Therefore, the NN can not figure different level information out (e.g. chemistry changes with "real" concentration of NO2 and not with the deviation of a station-wise mean)
Future status: (different solutions shown)
- Possibility to pre-scan all data and calculate a train data mean and std and apply this to all data
- Set hard coded mean and std (initial data inspection required to find "good" mean and std values)
- similar for first: also pre-scan data but the precision of the mean and the std is not so important. Therefore, calculate mean of all station-wise means (this still deviates from the ground-truth mean because it is not calculated on all values but on an aggregated station means) and the mean of all station-wise std (actually, this potentially deviates much more from the real std, but it is still a reasonable guess for a std estimation and in the end it is "just" scaling).