Problems regarding implementing a model in SMAUG

[Jiacheng Liu]

Hi everyone!

Firstly, thank you for building the awesome SMAUG. I want to use it with some self-defined models (start from Mobilenet now).  

Now, I can build the protobuf files for Reference and SMV backend. But when build the trace, the SMV backend runs into the following error.

```

.....

Scheduling data_8 (Data).

Scheduling data_9 (Data).

Scheduling input (Data).

Scheduling conv0 (Convolution3d).

smaug-instrumented: build/smaug/core/tensor.h:531: T *smaug::Tensor::data() [T = unsigned short]: Assertion `ToDataType<T>::dataType == dataType' failed.

trace.sh: line 8: 32900 Aborted                 (core dumped) ${bmk_dir}/smaug-instrumented ${topo_file} ${params_file} --sample-level=high --debug-level=0 --num-accels=1

```

For the Reference backend, I can get the trace, but when running the real simulation, I got the following error,

```

info: Entering event queue @ 50191742400.  Starting simulation...

warn: Replacement policy updates recently became the responsibility of SLICC state machines. Make sure to setMRU() near callbacks in .sm files!

info: Received mapping for array input at vaddr 3e4280 of length 12288.

fatal: Unable to insertArrayLabelMapping: No accelerator with id 0x1.

Memory Usage: 9918832 KBytes

```

Any ideas about how to fix these?

The main modifications are,

1. I found the depth-wise convolution is implemented in C++, but does not provide a python API.  So I add a python API in "nn_ops.py" for this operator.

2. I build the mobilenet model with the same structure as the Keras code.

During this process,  I also have several questions.

1. The depth-wise convolution only implements the Reference back-end. How will this affect the SMV backend, regardless of the performance?

2. It seems SMAUG does not support the padding operator. Is there any quick fix for it? I have tried the numpy.pad, but it seems not right. Any suggestions about this?

```

input_tensor = np.pad(input_tensor, ((0, 0), (0, 1), (0, 1), (0, 0)),

'constant', constant_values=0)

```

[Sam Xi]

Hi Jiacheng,

Thanks for your interest. Lots of questions here, I hope I've covered all of them:

--------------

Failing to run the reference model with the error:

fatal: Unable to insertArrayLabelMapping: No accelerator with id 0x1

The reason is that the gem5.cfg file is missing an accelerator definition with an ID of 1. Here is what your gem5.cfg probably looks like if you followed the tutorial:

[acc0]
	# Most of these parameters are pretty arbitrary. The important ones are the
	# memory type, accelerator id, and the input file paths. Otherwise, I make the
	# TLB and other structures larger so the simulation goes faster.
	memory_type = spad
	accelerator_id = 3
	input_dir = .
	bench_name = %(input_dir)s/outputs/nnet_fwd
	trace_file_name = %(input_dir)s/dynamic_trace_acc0.gz
	config_file_name = %(input_dir)s/smv-accel.cfg

In gem5-aladdin, one trace file = one "accelerator". This mapping from trace file to accelerator (via its ID) is specified by this section - in this case, it says "accelerator_id 3 maps to %(input_dir)s/dynamic_trace_acc0.gz". So where does accelerator id 3 come from? This is the ID for all the SMV blocks (code), since that's what the tutorial discusses. For the reference backend, you will need to either:

1. Make a copy of this section in gem5.cfg for all the IDs in the reference backend. I believe you should have produced 5 different trace files for this backend, so you'll need five of these sections.

2. Change all the reference block IDs to share a single ID, then re-trace. You'll now get one dynamic trace file and only need one section here. The tradeoffs between these two approaches is that with multiple trace files, you can simulate them in "parallel", whereas you cannot do so with a single trace file.

--------------

smaug-instrumented: build/smaug/core/tensor.h:531: T *smaug::Tensor::data() [T = unsigned short]: Assertion `ToDataType<T>::dataType == dataType' failed.

It looks like there is a mismatch between what C++ thinks is the datatype of the Tensor (T = unsigned short) and what was actually stored in the Tensor (which we don't know from this error message). The only thing I can think of that would cause this is if your Python API did not specify to use "float16" as the input data type for the Convolution3D input tensors. The SMV backend only supports float16. I think this was not documented very clearly. Filed https://github.com/harvard-acc/smaug/issues/93.

-------------

1. The depth-wise convolution only implements the Reference back-end. How will this affect the SMV backend, regardless of the performance?

If you build the SMV backend, it will use the Reference backend's depthwise convolution operator as a fallback, since it does not have its own hardware-accelerated implementation. See here for what operator delegates to what implementation. 

2. It seems SMAUG does not support the padding operator. Is there any quick fix for it? I have tried the numpy.pad, but it seems not right. Any suggestions about this?

numpy.pad will not work. The way SMAUG works is that every SMAUG python operator gets translated into an equivalent C++ operator. All execution is done in C++. numpy.pad is not a SMAUG operator, so the C++ half doesn't know about it. Unfortunately this doesn't look like something we can workaround with the existing operators, so someone will have to add it. I've filed https://github.com/harvard-acc/smaug/issues/94, but I don't know when I can get around to doing this. However, we enthusiastically welcome PRs :)

Sam Xi Google Inc., Software Engineer http://www.samxi.org

--
You received this message because you are subscribed to the Google Groups "gem5-Aladdin users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gem5-aladdin-us...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gem5-aladdin-users/86a1bee5-0367-41bc-a447-4ef5582b4bb9n%40googlegroups.com.

[Sam Xi]

The results may be different because SMV typically uses float16 instead of float32, but otherwise should be the same.

For the padding operator, I'd recommend looking at the array ops, which are implemented purely in software (they are not hardware accelerated).

On Sun, Jun 27, 2021, 2:48 AM Bean <liujiac...@gmail.com> wrote:

Hi Sam, 

    Thanks for your kind reply. I have tried your suggestions and the simulation can work now. I still have some confusing points.

As for "operator delegates"

I understand the process now, and would like to double-check here that they are producing the same simulation results except for the difference in running speed?

As for “padding operator”

I'd like to implement it, is there any suggestions or can you give me a reference operator for implementation?

[Bean]

Hi Sam, 

Thanks for your suggestions. I guess you mean the "unary_op.h" and the operator inherent from this one? And I spent some time writing the padding operator. I think I stuck into some problems. Can you kindly offer some help?

The main modification is 

1. I create a new file in "smaug/core/operators/padding_op.h", The content is  

```

template <typename Backend>

class PaddingOperator : public Operator {

   public:

    PaddingOperator(const std::string& name,

                    Workspace* workspace,

                    const std::vector<int> _padders)

            : Operator(name, OpType::Repeat, workspace), padders(_padders),

              padders(_padders) {

        inputs.resize(1, nullptr);

        outputs.resize(1, nullptr);

    }

    void setPadders(const std::vector<std::vector<int>>& _padders) {

        padders = _padders;

    }

    auto getPadders() {

      return padders;

    }

    void run() override {}

    enum { Inputs, kNumInputs };

    enum { Outputs, kNumOutputs };

protected:

std::vector<std::vector<int>> padders;

};

```

2. In "types.proto" I add     `Padding = 29;`  in OpType.

3. In "backend.h" I add `template <typename Backend> class PaddingOp;` Meanwhile, I add `DECL_CREATE_OP(PaddingOp);` for Reference and SMV

4. In "bachend.cpp" I add `#include "smaug/operators/padding_op.h"` and `DEF_CREATE_OP(PaddingOp, ReferenceBackend)`,  `DEF_CREATE_OP(PaddingOp, SmvBackend)`

5. In "network_builder.cpp", I add a new brach for padding operator.

```

else if (type == OpType::Padding) {  // how to set this

        auto op = Backend::createPaddingOp(name, workspace);

        network->addOperator(op);

```

However, I can't get this work, the error message as,

build/smaug/core/backend.cpp: In static member function 'static smaug::PaddingOp<smaug::ReferenceBackend>* smaug::ReferenceBackend::createPaddingOp(const string&, smaug::Workspace*)':

build/smaug/core/backend.cpp:43:51: error: invalid use of incomplete type 'class smaug::PaddingOp<smaug::ReferenceBackend>'

         return new OpType<Backend>(name, workspace);                           \

                                                   ^

build/smaug/core/backend.cpp:83:1: note: in expansion of macro 'DEF_CREATE_OP'

 DEF_CREATE_OP(PaddingOp, ReferenceBackend)

 ^

In file included from build/smaug/core/backend.cpp:1:0:

build/smaug/core/backend.h:90:7: note: declaration of 'class smaug::PaddingOp<smaug::ReferenceBackend>'

 class PaddingOp;

       ^~~~~~~~~

build/smaug/core/backend.cpp: In static member function 'static smaug::PaddingOp<smaug::SmvBackend>* smaug::SmvBackend::createPaddingOp(const string&, smaug::Workspace*)':

build/smaug/core/backend.cpp:43:51: error: invalid use of incomplete type 'class smaug::PaddingOp<smaug::SmvBackend>'

         return new OpType<Backend>(name, workspace);                           \

                                                   ^

build/smaug/core/backend.cpp:113:1: note: in expansion of macro 'DEF_CREATE_OP'

 DEF_CREATE_OP(PaddingOp, SmvBackend)

 ^

In file included from build/smaug/core/backend.cpp:1:0:

build/smaug/core/backend.h:90:7: note: declaration of 'class smaug::PaddingOp<smaug::SmvBackend>'

 class PaddingOp;

       ^~~~~~~~~

make/Makefile.native:41: recipe for target 'build/smaug/core/backend.o' failed

make[2]: *** [build/smaug/core/backend.o] Error 1

make/Makefile.native:66: recipe for target 'tests' failed

make[1]: *** [tests] Error 2

Makefile:16: recipe for target 'test' failed

make: *** [test] Error 2

root@a5b0bbeb0475:/workspace/smaug# make test

make[2]: Nothing to be done for 'protos'.

g++ -c -std=c++17 -O3 -g -DDMA_MODE -DDMA_INTERFACE_V3 -msse3 -msse2 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -Wno-deprecated-declarations -Ibuild -Ibuild/gem5 -I/workspace/gem5-aladdin/src/aladdin/gem5 -I/workspace/gem5-aladdin/src/aladdin/../..//include -I/usr/include/include -Ithird_party/Catch2/single_include -Ithird_party/FP16/include build/smaug/operators/ref/ref_padding_op.cpp -o build/smaug/operators/ref/ref_padding_op.o

build/smaug/operators/ref/ref_padding_op.cpp:8:39: error: invalid use of incomplete type 'class smaug::PaddingOp<smaug::ReferenceBackend>'

 void PaddingOp<ReferenceBackend>::run() {

                                       ^

In file included from build/smaug/operators/ref/ref_padding_op.cpp:1:0:

build/smaug/core/backend.h:90:7: note: declaration of 'class smaug::PaddingOp<smaug::ReferenceBackend>'

 class PaddingOp;

       ^~~~~~~~~

make/Makefile.native:41: recipe for target 'build/smaug/operators/ref/ref_padding_op.o' failed

make[2]: *** [build/smaug/operators/ref/ref_padding_op.o] Error 1

make/Makefile.native:66: recipe for target 'tests' failed

make[1]: *** [tests] Error 2

Makefile:16: recipe for target 'test' failed

make: *** [test] Error 2

[Sam Xi]

Just a wild guess: did you put your new PaddingOp in the smaug namespace?

Sam Xi Google Inc., Software Engineer http://www.samxi.org

To view this discussion on the web visit https://groups.google.com/d/msgid/gem5-aladdin-users/3c6a3d7c-a9b9-444a-bb50-7e2c988c8b0bn%40googlegroups.com.

[Bean]

Yes,  I use the smaug namespace. The full code is 

```

#ifndef _OPERATORS_PADDING_OP_H_

#define _OPERATORS_PADDING_OP_H_

#include "smaug/core/backend.h"

#include "smaug/core/operator.h"

#include "smaug/core/tensor.h"

// #include "smaug/core/tensor_utils.h"

#include "smaug/core/workspace.h"

namespace smaug {

/** \ingroup Operators

 * \brief Pad a given tensor in a different dimensions.

 *

 * This has a software-based implementation.

 *

 * @tparam Backend The Backend that sets Alignment.

 */

template <typename Backend>

class PaddingOperator : public Operator {

   public:

    PaddingOperator(const std::string& name,

                    Workspace* workspace,

                    const std::vector<int> _padders)

            : Operator(name, OpType::Repeat, workspace), padders(_padders),

              padders(_padders) {

        inputs.resize(1, nullptr);

        outputs.resize(1, nullptr);

    }

    /** Set the number of padders of the Tensor along each dimension. */

    void setPadders(const std::vector<std::vector<int>>& _padders) {

        padders = _padders;

    }

    auto getPadders() { return padders; }

    enum { Inputs, kNumInputs };

    enum { Outputs, kNumOutputs };

   protected:

    std::vector<std::vector<int>> padders;

};

}  // namespace smaug

#endif

```