Skip to content

Error with multiple FI in layer_output mode #8

@uditagarwal97

Description

@uditagarwal97

Here's the error log:

(tfi2) uditg@2403c486b851:~/UBC/TFI2_Original/TensorFI2/experiments/layer-outputs$ python3 -c 'import keras; print(keras.__version__)'
2.4.3
(tfi2) uditg@2403c486b851:~/UBC/TFI2_Original/TensorFI2/experiments/layer-outputs$ python3 cnn-mnist.py ./confFiles/sample.yaml ./result/ 100 1 > /dev/null
2021-10-22 18:27:29.528464: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303)
/home/uditg/UBC/TensorFI2/src/config.py:13: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  fiConf = yaml.load(fiConfs)
Traceback (most recent call last):
  File "cnn-mnist.py", line 79, in <module>
    res = tfi.inject(model=model, x_test=test_images[i:i+1], confFile=conf)
  File "/home/uditg/UBC/TensorFI2/src/tensorfi2.py", line 46, in __init__
    self.val = fiFunc(model, fiConf, **kwargs)
  File "/home/uditg/UBC/TensorFI2/src/tensorfi2.py", line 316, in layer_outputs
    pred = get_pred([fiLayerOutputs])
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/keras/backend.py", line 3792, in __call__
    outputs = self._graph_fn(*converted_inputs)
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1605, in __call__
    return self._call_impl(args, kwargs)
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1645, in _call_impl
    return self._call_flat(args, self.captured_inputs, cancellation_manager)
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1746, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 598, in call
    ctx=ctx)
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.InvalidArgumentError:  In[0] is not a matrix. Instead it has shape [1,12,12,32]
         [[node dense/BiasAdd (defined at /home/uditg/UBC/TensorFI2/src/tensorfi2.py:315) ]] [Op:__inference_keras_scratch_graph_1104]

Function call stack:
keras_scratch_graph

(tfi2) uditg@2403c486b851:~/UBC/TFI2_Original/TensorFI2/experiments/layer-outputs$ cat ./confFiles/sample.yaml 
Target: layer_outputs
Mode: multiple
Type: bitflips
Amount: 10
Bit: N
(tfi2) uditg@2403c486b851:~/UBC/TFI2_Original/TensorFI2/experiments/layer-outputs$ 

System config:

(tfi2) $ python3 -c 'import tensorflow as tf; print(tf.__version__)'                                                                    
2.2.0                                                                                                                                                                                                             
(tfi2) $ python3 -c 'import keras; print(keras.__version__)'                                                                            
2.4.3                                                                                                                                                                                                             
$ python --version                                                                                                               
Python 3.7.10                                                                                                                                                                                                     
$ python3 --version                                                                                                          
Python 3.7.10                                                                                                                                                                                                     
$ uname -a                                                                                                                    
Linux 2403c486b851 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux                                                                                               
$ cat /etc/os-release                                                                                                    
NAME="Ubuntu"                                                                                                                                                                                                     
VERSION="18.04.5 LTS (Bionic Beaver)"                                                                                                                                                                                                                                                                                                                                                                               
PRETTY_NAME="Ubuntu 18.04.5 LTS"                                                                                                                                                                                  
VERSION_ID="18.04"                                                                                                                                                                                                

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions