Tutorial 6: Neural Networks¶

This tutorial covers the TensorFlow Keras Neural Network surrogate modeling method via the plugin “keras_nn”. The Surrogates module also supports PyTorch and Scikit-learn Neural Network surrogate modeling methods, which follow the workflow below using the “pytorch_nn” and “scikit_nn” plugins, respectively.

More information on TensorFlow Keras model building is described by (Wu et al. 2020). Users may follow the recommended workflow to install and use TensorFlow in a Python environment, as described in the TensorFlow documentation: https://www.tensorflow.org/install.

Users may obtain a great deal of usage standards and best practices information as described in the PyTorch documentation: https://pytorch.org/docs/stable/index.html.

Users may find further information on the Scikit-learn package in the documentation: https://scikit-learn.org/stable/index.html and further information on deep learning capabilities as well: https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html#sklearn.neural_network.MLPRegressor.

The FOQUS file for this tutorial is Simple_flow.foqus, and this file is located in: examples/tutorial_files/Flowsheets/Tutorial_4

Note

The examples/ directory refers to the location where the FOQUS examples were installed, as described in Install FOQUS Examples.

The Python package tensorflow must be installed to use this surrogate plugin. Before starting this tutorial, you will need to install the package by referring to the installation instructions in the TensorFlow documentation.

Load the “Simple_flow.foqus” file from the examples/tutorial_files/Flowsheets/Tutorial_4 folder.
Click the Surrogates button from the Home window (Figure Keras Neural Network Session Set Up).
Select “keras_nn” in the Tool drop-down list. For PyTorch, select “pytorch_nn”. For Scikit-learn, select “scikit_nn”.
Select the Variables tab and select the desired input and output variables. Note that “graph.error” should not be selected.
Select the Method Settings tab.
Set “Initial Data Filter” to “All”.
Set “n_hidden” to “1”, “n_neurons” to “12”, “layer_act” to “relu”, and “out_act” to “sigmoid”.
Ensure there are values present for “numpy_seed”, “random_seed”, and “tensorflow_seed” so results are reproducible.
Set “epoch” to “500”, “verbose” to “0” (i.e. epoch results will not print during training) and ensure “output_file” has a file name of the form “user_ml_ai_models/[NAME].keras”. NAME may be changed, if desired. The model will be saved to the working directory folder user_ml_ai_models. For PyTorch, the file extension should be “.pt”. For Scikit-learn, the file extension should be “.pkl”.
Click the Run icon (Figure Keras Neural Network Session Set Up).

Keras Neural Network Session Set Up¶

The execution window will automatically display. While the regression training is finished running, TensorFlow Keras will display some information on the model size and shape.
When the run completes, the script produces a saved model file which is compatible with the Machine Learning & Artificial Intelligence Plugin. (See Section Machine Learning & Artificial Intelligence Flowsheet Model Plugins.)

In the future, these plugins will support additional items yielding enhanced surrogate modeling capabilities.

Expanded data normalization options
Offline data loading from a file, e.g. CSV, Excel, PSUADE
Support for regression of multi-output neural networks
Automatic regression parameter selection during model training from user-supplied parameter options