[Tracker] Fix tests added to xfail_set due to Roll PyTorch and LLVM bump update

[vivekkhandelwal1]

This issue is for tracking the failing tests due to the roll PyTorch update done here: #3727. Some of the tests are common for the FX_IMPORTER_XFAIL_SET and FX_IMPORTER_STABLEHLO_XFAIL_SET.

FX_IMPORTER_XFAIL_SET

• "CrossEntropyLossModule_basic",
• "CrossEntropyLossNoReductionModule_basic",
• "ElementwiseExpm1IntModule_basic",
• "ElementwiseExpm1Module_basic",
• "IndexPutImpl1DFloatAccumulateModule_basic",
• "IndexPutImpl1DFloatNonAccumulateModule_basic",
• "IndexPutImpl1DIntAccumulateModule_basic",
• "IndexPutImpl1DIntNonAccumulateModule_basic",
• "IndexPutImpl2DFloatNonAccumulateModule_basic",
• "IndexPutImpl2DImplicitModule_basic",
• "IndexPutImpl2DIndexModule_basic",
• "IndexPutImpl2DNoneIndexStaticModule_basic",
• "IndexPutImpl3DFloatNonAccumulateModule_basic",
• "IndexPutImplIndexWithNoneModule_basic",
• "InterpolateDynamicModule_sizes_nearest",
• "IouOfModule_basic",
• "MeshgridIndexingIJ_basic",
• "MeshgridIndexingXY_basic",
• "Meshgrid_basic",
• "OneHotModule_basic",
• "BernoulliFloatModule_basic",
• "BernoulliTensorModule_basic",
• "UniformModule_basic",
• "UniformStaticShapeModule_basic",
• "RollModule_basic"

FX_IMPORTER_CRASHING_SET

• "AdaptiveAvgPool1dGeneralDynamicNoBatches_basic",

FX_IMPORTER_STABLEHLO_XFAIL_SET

• "AdaptiveAvgPool1dGeneralDynamic_basic",
• "AdaptiveAvgPool1dGeneralDynamicNoBatches_basic",
• "AdaptiveAvgPool1dStaticEvenMultiple_basic",
• "AdaptiveAvgPool1dStaticLargerOutput_basic",
• "AdaptiveAvgPool2dDynamicNoBatch_basic",
• "AdaptiveAvgPool2dDynamic_basic",
• "AddIntModule_basic",
• "AtenIntTensorByteDtypeModule_basic",
• "AtenIntTensorCharDtypeModule_basic",
• "AtenItemIntOpModule_basic",
• "CrossEntropyLossModule_basic",
• "CrossEntropyLossNoReductionModule_basic",
• "EinsumStaticContractRhsModule_basic",
• "EinsumStaticFourDimensionModule_basic",
• "EinsumStaticModule_basic",
• "EinsumStaticWithEllipsisSlicingAndBroadcastModule_basic",
• "EinsumStaticWithEllipsisSlicingModule_basic",
• "ElementwiseExpm1IntModule_basic",
• "ElementwiseExpm1Module_basic",
• "InterpolateDynamicModule_sizes_nearest",
• "IouOfModule_basic",
• "IscloseStaticModuleTrue_basic",
• "IscloseStaticModule_basic",
• "MeshgridIndexingIJ_basic",
• "MeshgridIndexingXY_basic",
• "Meshgrid_basic",
• "MulIntModule_basic",
• "OneHotModule_basic",
• "ReduceFrobeniusNormComplexModule_basic",
• "ScalarImplicitIntModule_basic",
• "ScaledDotProductAttentionBoolMaskModule_basic",
• "ScaledDotProductAttentionDifferentCausalModule_basic",
• "ScaledDotProductAttentionDifferentDynamicCausalModule_basic",
• "ScaledDotProductAttentionDifferentModule_basic",
• "ScaledDotProductAttentionMaskModule_basic",
• "ScaledDotProductAttentionSameCausalModule_basic",
• "ScaledDotProductAttentionSameDynamicModule_basic",
• "ScaledDotProductAttentionSameModule_basic",
• "SubIntModule_basic",
• "TensorToIntZeroRank_basic",
• "UpSampleNearest2dDynamicFactor_basic",
• "UpSampleNearest2dDynamicSize_basic",
• "UpSampleNearest2dStaticFactor_basic",
• "UpSampleNearest2dStaticSize_basic",
• "UpSampleNearest2d_basic",
• "BernoulliFloatModule_basic",
• "UniformModule_basic",
• "UniformStaticShapeModule_basic",

FX_IMPORTER_STABLEHLO_CRASHING_SET

• "CrossEntropyLossModule_basic",
• "CrossEntropyLossNoReductionModule_basic",

FX_IMPORTER_TOSA_XFAIL_SET

• "AdaptiveAvgPool1dNonUnitOutputSizeDynamicModule_basic"
• "AdaptiveAvgPool1dNonUnitOutputSizeStaticModule_basic"
• "AdaptiveAvgPool1dUnitOutputSizeDynamicModule_basic"
• "AdaptiveAvgPool1dUnitOutputSizeStaticModule_basic"
• "ElementwiseRreluTrainModule_basic"
• "ElementwiseRreluTrainStaticModule_basic"
• "MaxPool1dEmptyStrideStaticModule_basic"
• "MaxPool1dStaticCeilModeTrueModule_basic"
• "MaxPool1dStaticModule_basic"
• "ReduceFrobeniusNormKeepDimModule_basic"
• "RepeatInterleaveSelfIntModule_basic"
• "ElementwiseSpecialExpm1IntModule_basic",
• "ElementwiseSpecialExpm1Module_basic",

[justin-ngo-arm]

Hi @vivekkhandelwal1, @sjarus and I are also tracking fx_importer_tosa regressions from this update. Here are some additional tests that failed on fx_importer_tosa that haven't been included in your list. I also have the full list if you want:

• "AdaptiveAvgPool1dNonUnitOutputSizeDynamicModule_basic"
• "AdaptiveAvgPool1dNonUnitOutputSizeStaticModule_basic"
• "AdaptiveAvgPool1dUnitOutputSizeDynamicModule_basic"
• "AdaptiveAvgPool1dUnitOutputSizeStaticModule_basic"
• "ElementwiseRreluTrainModule_basic"
• "ElementwiseRreluTrainStaticModule_basic"
• "MaxPool1dEmptyStrideStaticModule_basic"
• "MaxPool1dStaticCeilModeTrueModule_basic"
• "MaxPool1dStaticModule_basic"
• "ReduceFrobeniusNormKeepDimModule_basic"
• "RepeatInterleaveSelfIntModule_basic"

[vivekkhandelwal1]

Hi @vivekkhandelwal1, @sjarus and I are also tracking fx_importer_tosa regressions from this update. Here are some additional tests that failed on fx_importer_tosa that haven't been included in your list. I also have the full list if you want:

Hi @justin-ngo-arm, thanks for the update. Actually, the CI does not run the fx_importer_tosa tests hence I missed them. I have updated the above list.

[sjarus]

@vivekkhandelwal1 which targets does CI run now ? We'd previously discussed potentially swapping one or more existing TOSA targets for fx_importer_tosa.

[vivekkhandelwal1]

@vivekkhandelwal1 which targets does CI run now ? We'd previously discussed potentially swapping one or more existing TOSA targets for fx_importer_tosa.

Hi @sjarus, we run the following script https://github.com/llvm/torch-mlir/blob/main/build_tools/ci/test_posix.sh to run the tests in CI. The configs for which tests are run are: onnx, fx_importer, fx_importer_stablehlo.

[sjarus]

@vivekkhandelwal1 which targets does CI run now ? We'd previously discussed potentially swapping one or more existing TOSA targets for fx_importer_tosa.

Hi @sjarus, we run the following script https://github.com/llvm/torch-mlir/blob/main/build_tools/ci/test_posix.sh to run the tests in CI. The configs for which tests are run are: onnx, fx_importer, fx_importer_stablehlo.

Thanks! What would be the logistical impact of adding fx_importer_tosa to that list ? There are multiple contributors to the TorchToTosa path who would benefit from this.

[vivekkhandelwal1]

@vivekkhandelwal1 which targets does CI run now ? We'd previously discussed potentially swapping one or more existing TOSA targets for fx_importer_tosa.

Hi @sjarus, we run the following script https://github.com/llvm/torch-mlir/blob/main/build_tools/ci/test_posix.sh to run the tests in CI. The configs for which tests are run are: onnx, fx_importer, fx_importer_stablehlo.

Thanks! What would be the logistical impact of adding fx_importer_tosa to that list ? There are multiple contributors to the TorchToTosa path who would benefit from this.

I don't exactly remember why did we remove this but AFAICR the reason might have been irregular maintenance of this code path. If you think we should have this in CI again then I think @stellaraccident is the best person to take a call on this.

[stellaraccident]

If it is maintained and of reasonable duration, I think we can handle that. Certainly open to give it a try and see how it goes.