Research Blunders Log 2As the project grows larger, adding new features directly to the existing code really introduces more and more bugs. Today, I found two similar bugs in one day—so frustrating!kernel_size and stride are not one-to-oneIn the previously defined model, the nn.Conv2d's kernelsize and stride were one-to-one. However, the expanded model has a layer that does not follow the previous correspondence rule, causing the code that assigns stride based on kernelsize to break directly.If I hadn't suddenly written an assert to check the shape today, I probably would never have discovered it in my lifetime.PYTHONif kernel_size == 3: padding = 1 stride = 1 elif kernel_size == 5: padding = 2 stride = 2 else: raise ValueError() Not all convolutions are followed by ReLUIn the previous model, every convolutional layer was followed by an nn.ReLU. But to adapt to new resolution requirements, I designed a transition layer that does not have an nn.ReLU after it, and its naming convention is the same as the previous convolutions. Meanwhile, during simulation computation, the previously written judgment rule directly went wrong. o( ̄┰ ̄*)ゞDIFF-if quant.startswith('classifier_conv'): +if quant.startswith('classifier_conv') and quant != 'classifier_conv_transfer': x[x < 0] = 0 In summary, the two bugs found today both originated from migrating the original code and adding something, resulting in incorrect conditional checks. These problems seem unavoidable unless you design an extensible architecture from the start, but how could you foresee the problems that might arise later? Research is not like product development; new ideas come up from time to time, making it even less likely to consider so much.
Research Blunders Log 2
Research Blunders Log 2
As the project grows larger, adding new features directly to the existing code really introduces more and more bugs. Today, I found two similar bugs in one day—so frustrating!
kernel_size and stride are not one-to-one
In the previously defined model, the
nn.Conv2d's kernelsize and stride were one-to-one. However, the expanded model has a layer that does not follow the previous correspondence rule, causing the code that assigns stride based on kernelsize to break directly.If I hadn't suddenly written an
assertto check the shape today, I probably would never have discovered it in my lifetime.Not all convolutions are followed by ReLU
In the previous model, every convolutional layer was followed by an
nn.ReLU. But to adapt to new resolution requirements, I designed a transition layer that does not have annn.ReLUafter it, and its naming convention is the same as the previous convolutions. Meanwhile, during simulation computation, the previously written judgment rule directly went wrong. o( ̄┰ ̄*)ゞIn summary, the two bugs found today both originated from migrating the original code and adding something, resulting in incorrect conditional checks. These problems seem unavoidable unless you design an extensible architecture from the start, but how could you foresee the problems that might arise later? Research is not like product development; new ideas come up from time to time, making it even less likely to consider so much.