How to use this on my deep neural network?

[Nithanaroy]

I'm looking for ways to do ENAS search on my deep neural network. Could you share some ideas on how to achieve it using your repo?

[countif]

It depends on which kind of neural network you use.
Show me some details.

[Nithanaroy]

It is a standard simple dense neural network with tf.keras API for a classification problem. The structure looks like,

import nni

def train(params):
    # fetch data
    train_dataset = tf.data.TFRecordDataset(...)
    test_dataset = tf.data.TFRecordDataset(...)

    # build model
    x = create_input_layer(...)

    y = Dense(params['layer1'], activation='relu', name="dense0")(x)
    y = BatchNormalization()(y)
    y = Dropout(params['dropout'])(y)
    ...

    y = Dense(4, activation='softmax', name="softmax")(y)

    model = tf.keras.models.Model(x, y)
    optimizer = Adam(lr=params['lr'])
    model.compile(loss='categorical_crossentropy',
                        optimizer=optimizer,
                        metrics=['accuracy']
                     )
    
    # train
    h = model.fit(
                    train_dataset,
                    steps_per_epoch=...,
                    epochs=params['epochs'],
                    validation_data=test_dataset
                    validation_steps=...
                )

    # report reward
    metrics = dict([(metric, h.history[metric][-1]) for metric in h.history])
    nni.report_final_result( metrics['val_acc'] )

def main():
    params = nni.get_next_parameter()

with config.yaml as,

authorName: default
experimentName: nni_enas_training
trialConcurrency: 1
maxExecDuration: 1h
maxTrialNum: 10
trainingServicePlatform: local
# The path to Search Space
searchSpacePath: search_space.json
useAnnotation: false
tuner:
  builtinTunerName: ???
# The path and the running command of trial
trial:  
  command: python3 mydnn.py
  codeDir: .
  gpuNum: 1

and search_space.json as,

{
    "dropout": {
        "_type": "uniform",
        "_value": [0.5, 0.9]
    },
    "layer1": {
        "_type": "choice",
        "_value": [124, 512, 1024]
    },
    "layer2": {
        "_type": "choice",
        "_value": [124, 512, 1024]
    },
    "epochs": {
        "_type": "choice",
        "_value": [100, 200, 300]
    },
    "lr": {
        "_type": "choice",
        "_value": [0.0001, 0.0003, 0.001, 0.01]
    }
}

• What would be builtinTunerName and how integrate ENAS into the code?
• Is it possible to make ENAS dynamically generate the number of layers also instead of hardcoding it to, say 2, like in the above example?

[Nithanaroy]

Any help @countif ? Thank you :)

[countif]

Which dataset you use?
It only support cifar10 actually.

[Nithanaroy]

Oh, regular tables of rows & columns. Not images. So ENAS in general works for images and temporal data only?

On Tue, Apr 9, 2019 at 4:53 AM Bin Yuan ***@***.***> wrote: Which dataset you use? It only support cifar10 actually. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAo7XLCQS1kZcFp07yrwrgIGZb2emdpQks5vfH86gaJpZM4cgoHw> .

-- Regards Nitin Pasumarthy Sent from a mobile