[Bug]: Example __asan_on_error hangs when triggered

[martingalvan-volue]

Describe the issue

Hi, I'm trying to run a simple GTest example with Address Sanitizer (asan) enabled on Windows 11 using Visual Studio 17.14.13 and GTest 1.17.0. I'm installing GTest through vcpkg, and using a custom vcpkg triplet to make sure it's built with asan enabled as well.

I'm trying to see how the __asan_on_error function sample from https://google.github.io/googletest/advanced.html#sanitizer-integration works in practice. However, when I reach this function the test program hangs entirely.

Steps to reproduce the problem

My code is as follows:

vcpkg.json:

{
    "$schema": "https://raw.githubusercontent.com/microsoft/vcpkg/master/scripts/vcpkg.schema.json",
    "name": "gtest-asan",
    "version-string": "1.0.0.0",
    "builtin-baseline": "dd3097e305afa53f7b4312371f62058d2e665320",
    "dependencies": [
        "gtest"
    ]
}

CMakeLists.txt:

cmake_minimum_required(VERSION 3.29.0 FATAL_ERROR)

set(CMAKE_TOOLCHAIN_FILE
    "C:/Users/martin/vcpkg/scripts/buildsystems/vcpkg.cmake" CACHE PATH ""
)

set(VCPKG_BOOTSTRAP_OPTIONS "-disableMetrics")
set(VCPKG_OVERLAY_TRIPLETS "${CMAKE_CURRENT_SOURCE_DIR}")
set(VCPKG_TARGET_TRIPLET "x64-windows-asan")

project(gtest_asan CXX)

find_package(GTest CONFIG REQUIRED)

add_executable(gtest_asan
    gtest_asan.cpp
)

target_compile_options(gtest_asan PRIVATE
    /fsanitize=address
    /Zi
)

target_link_options(gtest_asan PRIVATE
    /INCREMENTAL:NO
)

target_link_libraries(gtest_asan PRIVATE
    GTest::gtest_main
)

x64-windows-asan.cmake:

set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE dynamic)

set(VCPKG_C_FLAGS "/fsanitize=address")
set(VCPKG_CXX_FLAGS "/fsanitize=address")
set(VCPKG_LINKER_FLAGS "/INCREMENTAL:NO")

gtest_asan.cpp:

#include <gtest/gtest.h>

extern "C" {

void __asan_on_error()
{
    FAIL() << "Encountered an address sanitizer error";
}

}

char get_char()
{
    char* p = new char[42];

    delete p;

    return p[5];
}

TEST(MyTest, MyTest)
{
    EXPECT_EQ('a', get_char());
}

int main(int argc, char** argv)
{
    ::testing::InitGoogleTest(&argc, argv);

    return RUN_ALL_TESTS();
}

To build:

cmake -B build -GNinja -DCMAKE_BUILD_TYPE=Release
cmake --build build

When running this test without __asan_on_error I can see the correct asan output and the test program finishes immediately after. However, when using this __asan_on_error I see that the test program hangs forever. Leaving the function empty seems to "fix" this, so my thought is that FAIL() isn't working as it should in this context.

Am I doing something wrong here? Thanks.

What version of GoogleTest are you using?

1.17.0

What operating system and version are you using?

Windows 11

What compiler and version are you using?

Visual Studio 17.14.13
Microsoft (R) C/C++ Optimizing Compiler Version 19.44.35215 for x64

What build system are you using?

cmake version 3.31.5