faster DataGenerator
Current status: Data is always loaded from disk for each call of get_data_generator(id)
. During this call data is loaded and preprocessed. This steps are required to be executed on each call, because the processed data is not stored. Therefore, the same data is processed in the same way multiple times, instead of just using previous processed data.
Solution: Save the processed data as temporary data. During training and all others operations, this data just needs to be loaded without any further calculations.